From c9dd35d3d78b8adcf9e67bd3f0d40bf4bbae3c38 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 7 Oct 2026 01:09:30 -0400 Subject: [PATCH 01/44] docs: retroactive DCO signoff for cliffhall's unsigned commits (#2617) Signed-off-by: cliffhall --- README.md | 1 + docs/dco-retroactive-signoffs.md | 30 ++++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 docs/dco-retroactive-signoffs.md diff --git a/README.md b/README.md index 53e031ab6..a2f947ced 100644 --- a/README.md +++ b/README.md @@ -92,6 +92,7 @@ Each client has its own README with client-specific detail: | [MCP server configuration](./docs/mcp-server-configuration.md) | Which server(s) the Inspector connects to, and the config file format | | [Reviewing an MCP App](./docs/mcp-app-review.md) | The CLI-first → one-shot-web recipe for automated App-tool review | | [Smoke-testing an MCP server](./docs/cli-smoke-testing.md) | The connect → list → call → assert workflow for a shell or CI job: `--format json` + `jq`, the exit-code map, and keeping OAuth non-interactive | +| [Retroactive DCO signoffs](./docs/dco-retroactive-signoffs.md) | The ten commits that merged without a `Signed-off-by:` while no DCO check ran, and each author's signed-off certification of them | | [Launcher and config consolidation](./docs/launcher-config-consolidation-plan.md) | Why the launcher runs a client in-process rather than spawning it | | [Roadmap, Aug 2026 → Feb 2027](./docs/inspector-roadmap-2026-h2.md) | The six-month plan: spec-following work aligned to the published MCP roadmap, official extension support, and the experience work we choose | | [MCP Inspector: Our AI Software Factory](./docs/ai-software-factory.md) | How contributions actually happen since v2.0.0 — issue-driven work end to end, the rules/skills split, the sweeps that replaced Dependabot, the quality gate, and where it's headed | diff --git a/docs/dco-retroactive-signoffs.md b/docs/dco-retroactive-signoffs.md new file mode 100644 index 000000000..d6c975893 --- /dev/null +++ b/docs/dco-retroactive-signoffs.md @@ -0,0 +1,30 @@ +# Retroactive DCO signoffs + +Every commit in this repository carries a Developer Certificate of Origin +signoff (`Signed-off-by:`), checked by `.github/workflows/dco.yml`. The commits +listed below merged on 2026-08-11/12 without one, while the probot DCO app that +used to enforce this was suspended and nothing replaced it (#2566, #2616). They +are already on `main` and in released versions, so their history is not +rewritten. Instead each author certifies them here. + +Each section below was added by its author in a commit that the author both +wrote and signed off. That commit is the author's retroactive `Signed-off-by:` +for every commit the section lists. To verify, run: + + git log --format='%h %an <%ae>%n%(trailers:key=Signed-off-by)' -- docs/dco-retroactive-signoffs.md + +## cliffhall + +I, Cliff Hall, am the author of the commits below. I certify that each of +them was contributed in accordance with the Developer Certificate of Origin, +version 1.1 (https://developercertificate.org/), and this signed-off commit +adds my `Signed-off-by: cliffhall ` to each of them. + +| Commit | PR | Subject | +| ------------------------------------------ | ----- | ---------------------------------------------------------------------------------------- | +| `eb5bba62ede2c27d9082a833bcf23e34d4cda2ee` | #1983 | fix(tui): bundle React-rendering deps so a consumer install can't split React | +| `3c1bbbcb5fda7e4eb393f7323f1b347726ccc440` | #1983 | fix(tui): correct ink-form's misspelled incomplete-form hint | +| `39575ff060bcc7f4a7478a19d1d58ec568c39e2c` | #1983 | test(tui): drive the ink-form patch through the plugin, and drop the esbuild type import | +| `9fd5bc177cb9e40e651756c533fed14526043bc6` | #1983 | chore: drop the orphaned root override, and correct the TUI coverage note | +| `94a227b9c12451ee7810f1177e4992ac4713742c` | #1983 | fix(tui): widen the root react range so an external ink can't split React | +| `b70aed691418a7e2e98c897cba6ab44ad422f648` | #1987 | test(fix): probe a single-source route when waiting for the dead transport | From 094847aaa23ceacbdf0f3a03f78e247ac6b9d2af Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 18:06:13 -0400 Subject: [PATCH 02/44] fix: bump open to v11 and restrict openUrl to http(s) Bump `open` from ^10.2.0 to ^11.0.4 at the root, and pin web's Storybook-transitive copy to the same version with an `overrides` entry so every install holds one version. v11 resolves only after the child has spawned and rejects on a spawn failure itself, then stops listening for `'error'`. Replace the `'spawn'`-waiting helper with a no-op `'error'` absorber after launch, and update the test doubles to match. openUrl now refuses any URL that does not parse as http: or https: before reaching `open`; callers already print the URL for manual opening on rejection. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/package-lock.json | 63 +++++++++---- clients/web/package.json | 1 + clients/web/server/web-server-config.ts | 8 +- .../web/src/test/core/node/openUrl.test.ts | 88 ++++++++++--------- .../server/server-auto-open.test.ts | 20 ++--- core/node/openUrl.ts | 71 +++++++++------ package-lock.json | 51 ++++++++--- package.json | 2 +- 8 files changed, 191 insertions(+), 113 deletions(-) diff --git a/clients/web/package-lock.json b/clients/web/package-lock.json index 2aa6033f9..68b416571 100644 --- a/clients/web/package-lock.json +++ b/clients/web/package-lock.json @@ -4109,9 +4109,9 @@ } }, "node_modules/bundle-name": { - "version": "4.1.0", - "resolved": "https://registry.npmjs.org/bundle-name/-/bundle-name-4.1.0.tgz", - "integrity": "sha512-tjwM5exMg6BGRI+kNmTntNsvdZS1X8BFYS6tnJ2hdH0kVxM6/eVZ2xy+FqStSWvYmtfFMDLIxurorHwDKfDz5Q==", + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/bundle-name/-/bundle-name-4.1.1.tgz", + "integrity": "sha512-DdH81/zPLVS11EUgWq3tEu/xn+EzljlMYooDNdzWEnFha3R3NBMpMV1UqYIpjYHV/SgpFKMIX1Oh7o06SjM/oA==", "dev": true, "license": "MIT", "dependencies": { @@ -4467,9 +4467,9 @@ "peer": true }, "node_modules/default-browser": { - "version": "5.5.0", - "resolved": "https://registry.npmjs.org/default-browser/-/default-browser-5.5.0.tgz", - "integrity": "sha512-H9LMLr5zwIbSxrmvikGuI/5KGhZ8E2zH3stkMgM5LpOWDutGM2JZaj460Udnf1a+946zc7YBgrqEWwbk7zHvGw==", + "version": "5.5.1", + "resolved": "https://registry.npmjs.org/default-browser/-/default-browser-5.5.1.tgz", + "integrity": "sha512-m1pAzaJgZ/gssEqlOhJkPJp8Xly7QyW6xcrkUa2KKcDeDSEMP7X8xipU3snUcfisTQx0w1AGae+9UtJSfVnXGw==", "dev": true, "license": "MIT", "dependencies": { @@ -5537,6 +5537,19 @@ "url": "https://github.com/sponsors/wooorm" } }, + "node_modules/is-in-ssh": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-in-ssh/-/is-in-ssh-1.0.0.tgz", + "integrity": "sha512-jYa6Q9rH90kR1vKB6NM7qqd1mge3Fx4Dhw5TVlK1MUBqhEOuCagrEHMevNuCcbECmXZ0ThXkRm+Ymr51HwEPAw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/is-inside-container": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/is-inside-container/-/is-inside-container-1.0.0.tgz", @@ -7086,19 +7099,21 @@ } }, "node_modules/open": { - "version": "10.2.0", - "resolved": "https://registry.npmjs.org/open/-/open-10.2.0.tgz", - "integrity": "sha512-YgBpdJHPyQ2UE5x+hlSXcnejzAvD0b22U2OuAP+8OnlJT+PjWPxtgmGqKKc+RgTM63U9gN0YzrYc71R2WT/hTA==", + "version": "11.0.4", + "resolved": "https://registry.npmjs.org/open/-/open-11.0.4.tgz", + "integrity": "sha512-++Zlftm0kVLPmzC06t6epuWmcRMDbI4z5P3NNX979WA/k23+NtSOynEGzsVfZwguKw2mi5umVgnBlJQMwRz4Pg==", "dev": true, "license": "MIT", "dependencies": { - "default-browser": "^5.2.1", + "default-browser": "^5.5.1", "define-lazy-prop": "^3.0.0", + "is-in-ssh": "^1.0.0", "is-inside-container": "^1.0.0", - "wsl-utils": "^0.1.0" + "powershell-utils": "^0.2.1", + "wsl-utils": "^1.0.0" }, "engines": { - "node": ">=18" + "node": ">=20" }, "funding": { "url": "https://github.com/sponsors/sindresorhus" @@ -7520,6 +7535,19 @@ } } }, + "node_modules/powershell-utils": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/powershell-utils/-/powershell-utils-0.2.1.tgz", + "integrity": "sha512-C+y9x90UElAddDZmV4qOx9W53B61PO7cIqWz2dQsWlwswuq4mr8NEwytdGKboYbQlGZ3awrkTeNvcZiZNHnQ8A==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/prelude-ls": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/prelude-ls/-/prelude-ls-1.2.1.tgz", @@ -9363,16 +9391,17 @@ } }, "node_modules/wsl-utils": { - "version": "0.1.0", - "resolved": "https://registry.npmjs.org/wsl-utils/-/wsl-utils-0.1.0.tgz", - "integrity": "sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw==", + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/wsl-utils/-/wsl-utils-1.0.1.tgz", + "integrity": "sha512-c16PKLIRxBYpR3jSWsdS7SFp0COm5SOQdiFjY818dYFHJbMGuLBszpF07LBWmOS1uzrHa9VGdynsEgoeKy0oOQ==", "dev": true, "license": "MIT", "dependencies": { - "is-wsl": "^3.1.0" + "is-wsl": "^3.1.0", + "powershell-utils": "^0.2.0" }, "engines": { - "node": ">=18" + "node": ">=20" }, "funding": { "url": "https://github.com/sponsors/sindresorhus" diff --git a/clients/web/package.json b/clients/web/package.json index 08e07ad01..361899ec1 100644 --- a/clients/web/package.json +++ b/clients/web/package.json @@ -85,6 +85,7 @@ "overrides": { "@types/node": "^24.12.4", "esbuild": "^0.28.2", + "open": "^11.0.4", "vitest": "4.1.11" } } diff --git a/clients/web/server/web-server-config.ts b/clients/web/server/web-server-config.ts index 29ca4c151..81fbc6c46 100644 --- a/clients/web/server/web-server-config.ts +++ b/clients/web/server/web-server-config.ts @@ -262,10 +262,10 @@ export function printServerBanner( * `.catch` is two chances for one of them to drift back to a bare `open(url)`. */ export function openBrowser(url: string): void { - // Through core's `openUrl`, not a bare `open(url)`: `open` resolves before a - // spawn failure (the opener missing from `PATH`) arrives as an `'error'` - // event on the child, which no `.catch` here could see and which crashed - // the server at startup (#2533). `openUrl` turns it into a rejection. + // Through core's `openUrl`, not a bare `open(url)`: it bounds a stalled + // opener with a timeout, refuses anything but http(s), and absorbs an + // `'error'` on the child after launch, which unlistened would crash the + // server (#2533). Every one of those failures reaches the `.catch` below. openUrl(url).catch((err: unknown) => { // The message only, on one line, matching the CLI's wording // (`core/cli/cli-oauth-navigation.ts`). Logging the error object printed diff --git a/clients/web/src/test/core/node/openUrl.test.ts b/clients/web/src/test/core/node/openUrl.test.ts index 1fbe59da9..5b53ad992 100644 --- a/clients/web/src/test/core/node/openUrl.test.ts +++ b/clients/web/src/test/core/node/openUrl.test.ts @@ -2,20 +2,12 @@ import { EventEmitter } from "node:events"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; /** - * A stand-in for the `ChildProcess` `open` resolves with. Like a real spawn, - * it reports the outcome on a later `process.nextTick`: `'spawn'` when the - * opener launched, `'error'` when it could not be spawned. + * A stand-in for `open` v11, which resolves with the `ChildProcess` only once + * the opener has emitted `'spawn'` (and rejects when it cannot be spawned), + * then stops listening for `'error'` on it. */ -function fakeChild(outcome: "spawn" | Error = "spawn"): EventEmitter { - const child = new EventEmitter(); - process.nextTick(() => - outcome === "spawn" ? child.emit("spawn") : child.emit("error", outcome), - ); - return child; -} - const openMock = vi.fn<(...args: unknown[]) => Promise>( - async () => fakeChild(), + async () => new EventEmitter(), ); vi.mock("open", () => ({ @@ -25,7 +17,7 @@ vi.mock("open", () => ({ describe("openUrl", () => { beforeEach(() => { openMock.mockClear(); - openMock.mockImplementation(async () => fakeChild()); + openMock.mockImplementation(async () => new EventEmitter()); }); afterEach(() => { @@ -46,51 +38,67 @@ describe("openUrl", () => { ); }); - it("rejects when the opener rejects", async () => { - openMock.mockRejectedValue(new Error("xdg-open not found")); + it("forwards a plain http URL (the web backend's own local address)", async () => { const { openUrl } = await import("@inspector/core/node/openUrl.js"); - await expect(openUrl("https://example.com/auth")).rejects.toThrow( - "xdg-open not found", + await openUrl("http://localhost:6274/?MCP_INSPECTOR_API_TOKEN=abc"); + expect(openMock).toHaveBeenCalledWith( + "http://localhost:6274/?MCP_INSPECTOR_API_TOKEN=abc", ); }); - it("rejects when the opener cannot be spawned, instead of crashing", async () => { - const enoent = Object.assign(new Error("spawn open ENOENT"), { - code: "ENOENT", - }); - openMock.mockImplementation(async () => fakeChild(enoent)); + it.each([ + ["file:", "file:///C:/Windows/System32/calc.exe"], + ["javascript:", "javascript:alert(1)"], + ["ms-settings:", "ms-settings:privacy"], + ["smb:", "smb://attacker.example/share"], + ["data:", "data:text/html,"], + ])("refuses a %s URL without calling open", async (protocol, href) => { + const { openUrl } = await import("@inspector/core/node/openUrl.js"); + await expect(openUrl(href)).rejects.toThrow( + `refusing to open a non-http(s) URL (${protocol})`, + ); + expect(openMock).not.toHaveBeenCalled(); + }); + + it("refuses a non-http(s) URL instance without calling open", async () => { + const { openUrl } = await import("@inspector/core/node/openUrl.js"); + await expect(openUrl(new URL("file:///etc/passwd"))).rejects.toThrow( + "refusing to open a non-http(s) URL (file:)", + ); + expect(openMock).not.toHaveBeenCalled(); + }); + + it("refuses an unparseable URL without calling open", async () => { + const { openUrl } = await import("@inspector/core/node/openUrl.js"); + await expect(openUrl("not a url")).rejects.toThrow( + "refusing to open an invalid URL", + ); + expect(openMock).not.toHaveBeenCalled(); + }); + + it("rejects when the opener rejects", async () => { + openMock.mockRejectedValue(new Error("xdg-open not found")); const { openUrl } = await import("@inspector/core/node/openUrl.js"); await expect(openUrl("https://example.com/auth")).rejects.toThrow( - "spawn open ENOENT", + "xdg-open not found", ); }); - it("catches a spawn failure when entered from a timer or I/O callback", async () => { - // From a macrotask, Node drains process.nextTick BEFORE promise reactions, - // and on macOS `open` reaches spawn() without an earlier await, so a - // listener chained on its promise alone would attach too late. + it("rejects when the opener cannot be spawned, instead of crashing", async () => { const enoent = Object.assign(new Error("spawn open ENOENT"), { code: "ENOENT", }); - openMock.mockImplementation(async () => fakeChild(enoent)); + openMock.mockRejectedValue(enoent); const { openUrl } = await import("@inspector/core/node/openUrl.js"); - const outcome = await new Promise((resolve) => { - setImmediate(() => { - openUrl("https://example.com/auth").then( - () => resolve("resolved"), - (err: unknown) => resolve(err), - ); - }); - }); - expect(outcome).toBe(enoent); + await expect(openUrl("https://example.com/auth")).rejects.toBe(enoent); }); it("absorbs an opener error that arrives after launch", async () => { - let child: EventEmitter | undefined; - openMock.mockImplementation(async () => (child = fakeChild())); + const child = new EventEmitter(); + openMock.mockResolvedValue(child); const { openUrl } = await import("@inspector/core/node/openUrl.js"); await openUrl("https://example.com/auth"); - expect(() => child!.emit("error", new Error("late"))).not.toThrow(); + expect(() => child.emit("error", new Error("late"))).not.toThrow(); }); it("rejects when the opener does not settle within the timeout", async () => { diff --git a/clients/web/src/test/integration/server/server-auto-open.test.ts b/clients/web/src/test/integration/server/server-auto-open.test.ts index 4f061abf3..cf33383be 100644 --- a/clients/web/src/test/integration/server/server-auto-open.test.ts +++ b/clients/web/src/test/integration/server/server-auto-open.test.ts @@ -34,16 +34,12 @@ const warning = (reason: string) => `Could not open a browser automatically (${reason}). Open the URL above manually.`; /** - * A stand-in for the `ChildProcess` `open` resolves with: like a real spawn it - * reports on a later `process.nextTick` — `'spawn'` when the opener launched, - * `'error'` when it could not be spawned (#2533). + * A stand-in for the `ChildProcess` `open` resolves with. Since v11 `open` + * resolves only after the child's `'spawn'` and rejects on a spawn failure + * itself, so the child it hands back has already launched. */ -function fakeChild(outcome: "spawn" | Error = "spawn"): EventEmitter { - const child = new EventEmitter(); - process.nextTick(() => - outcome === "spawn" ? child.emit("spawn") : child.emit("error", outcome), - ); - return child; +function fakeChild(): EventEmitter { + return new EventEmitter(); } /** An unhandledRejection fires on a later macrotask than the rejection. */ @@ -94,8 +90,8 @@ describe("openBrowser", () => { }); it("warns instead of crashing when the opener cannot be spawned", async () => { - // `open` resolves first and the ENOENT arrives afterwards as an 'error' - // event on the child; unlistened, that event throws and kills the server. + // `open` (v11+) rejects with the child's ENOENT 'error'; before #2533 an + // unlistened 'error' event threw and killed the server. const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); // Shaped like the real error: `spawnargs` carries the tokenized URL, // which must not reach the terminal a second time (#2645). @@ -104,7 +100,7 @@ describe("openBrowser", () => { code: "ENOENT", spawnargs: [url], }); - openMock.mockImplementation(async () => fakeChild(enoent)); + openMock.mockRejectedValue(enoent); expect(openBrowser(url)).toBeUndefined(); await settle(); expect(warn).toHaveBeenCalledTimes(1); diff --git a/core/node/openUrl.ts b/core/node/openUrl.ts index 4d3eaee6e..faa362f6a 100644 --- a/core/node/openUrl.ts +++ b/core/node/openUrl.ts @@ -24,40 +24,60 @@ import open from "open"; export const OPEN_URL_TIMEOUT_MS = 5_000; /** - * Settle once the opener process has actually launched. `open` resolves with - * the `ChildProcess` as soon as it has called `spawn()`, and a spawn failure - * (the opener binary missing from `PATH`: `ENOENT`) arrives afterwards as an - * `'error'` event on that child. `open` only listens for it when called with - * `{ wait: true }`, which is unusable here: on macOS it adds `open -W`, which - * waits for the browser to quit. Unlistened, the event crashes the process - * instead of reaching the caller's fallback (#2531). + * The only schemes {@link openUrl} hands to the platform opener. * - * Chained directly onto `open`'s promise, this runs as a microtask. That is - * only ahead of the `process.nextTick` emitting `'error'` or `'spawn'` when - * `open` itself ran inside a microtask: from a timer or I/O callback, Node - * drains `nextTick` before promise reactions, and on macOS `open` reaches - * `spawn()` with no earlier `await`. So {@link openUrl} calls `open` from a - * queued microtask, which puts the spawn and this listener in the same drain - * whatever the caller's context. The listener stays attached (`on`, not - * `once`) so an error after settling is still absorbed. + * The URL is not always ours: the CLI and TUI open an OAuth authorization URL + * built from the `authorization_endpoint` a remote MCP server advertises, so + * whatever that server returns reaches this function. `open` passes its target + * to `open` / `xdg-open` / PowerShell's `Start`, all of which also launch + * files, `file:` paths and custom protocol handlers, and its README is explicit + * that it does not sanitize untrusted input. Nothing this function is for + * needs anything but a web page, so anything else is refused before `open` is + * reached and the caller falls back to printing the URL. */ -function launched(child: ChildProcess): Promise { - return new Promise((resolve, reject) => { - child.on("error", reject); - child.once("spawn", () => resolve()); - }); +const OPENABLE_PROTOCOLS: ReadonlySet = new Set(["http:", "https:"]); + +/** Throw unless `href` parses as an `http:` or `https:` URL. */ +function assertOpenable(href: string): void { + let protocol: string; + try { + protocol = new URL(href).protocol; + } catch { + throw new Error("refusing to open an invalid URL"); + } + if (!OPENABLE_PROTOCOLS.has(protocol)) { + throw new Error(`refusing to open a non-http(s) URL (${protocol})`); + } +} + +/** + * Keep absorbing the opener's `'error'` events once it has launched. + * + * Since v11, `open` itself waits for the child's `'spawn'` event (on Windows, + * for PowerShell to exit) before resolving, and rejects when the opener cannot + * be spawned (the binary missing from `PATH`: `ENOENT`) — so the spawn failure + * that used to crash the process (#2531) now arrives as a rejection the caller + * sees. What it no longer does is listen afterwards: it removes its own + * `'error'` listener on launch, so a later `'error'` on the child would be + * unlistened and crash the process. A no-op listener keeps that failure where + * it belongs, in a best-effort open nobody is waiting on. + */ +function absorbLateErrors(child: ChildProcess): void { + child.on("error", () => {}); } /** - * Open a URL in the user's default browser (best-effort). Rejects when the - * opener fails, cannot be spawned, or does not launch within `timeoutMs`; - * callers print the URL themselves and decide what to tell the user. + * Open a URL in the user's default browser (best-effort). Rejects when the URL + * is not `http:`/`https:`, when the opener fails or cannot be spawned, or when + * it does not launch within `timeoutMs`; callers print the URL themselves and + * decide what to tell the user. */ export async function openUrl( url: string | URL, timeoutMs: number = OPEN_URL_TIMEOUT_MS, ): Promise { const href = typeof url === "string" ? url : url.href; + assertOpenable(href); let timer: ReturnType | undefined; const timeout = new Promise((_, reject) => { timer = setTimeout( @@ -67,10 +87,7 @@ export async function openUrl( ); }); try { - const opened = Promise.resolve() - .then(() => open(href)) - .then(launched); - await Promise.race([opened, timeout]); + await Promise.race([open(href).then(absorbLateErrors), timeout]); } finally { clearTimeout(timer); } diff --git a/package-lock.json b/package-lock.json index 5e503876d..2e1908191 100644 --- a/package-lock.json +++ b/package-lock.json @@ -24,7 +24,7 @@ "commander": "^15.0.0", "hono": "^4.13.7", "ink": "^6.0.0", - "open": "^10.2.0", + "open": "^11.0.4", "pino": "^9.14.0", "proper-lockfile": "^4.1.2", "react": "^19.0.0", @@ -3432,6 +3432,18 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/is-in-ssh": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/is-in-ssh/-/is-in-ssh-1.0.0.tgz", + "integrity": "sha512-jYa6Q9rH90kR1vKB6NM7qqd1mge3Fx4Dhw5TVlK1MUBqhEOuCagrEHMevNuCcbECmXZ0ThXkRm+Ymr51HwEPAw==", + "license": "MIT", + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/is-inside-container": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/is-inside-container/-/is-inside-container-1.0.0.tgz", @@ -4131,18 +4143,20 @@ } }, "node_modules/open": { - "version": "10.2.0", - "resolved": "https://registry.npmjs.org/open/-/open-10.2.0.tgz", - "integrity": "sha512-YgBpdJHPyQ2UE5x+hlSXcnejzAvD0b22U2OuAP+8OnlJT+PjWPxtgmGqKKc+RgTM63U9gN0YzrYc71R2WT/hTA==", + "version": "11.0.4", + "resolved": "https://registry.npmjs.org/open/-/open-11.0.4.tgz", + "integrity": "sha512-++Zlftm0kVLPmzC06t6epuWmcRMDbI4z5P3NNX979WA/k23+NtSOynEGzsVfZwguKw2mi5umVgnBlJQMwRz4Pg==", "license": "MIT", "dependencies": { - "default-browser": "^5.2.1", + "default-browser": "^5.5.1", "define-lazy-prop": "^3.0.0", + "is-in-ssh": "^1.0.0", "is-inside-container": "^1.0.0", - "wsl-utils": "^0.1.0" + "powershell-utils": "^0.2.1", + "wsl-utils": "^1.0.0" }, "engines": { - "node": ">=18" + "node": ">=20" }, "funding": { "url": "https://github.com/sponsors/sindresorhus" @@ -4346,6 +4360,18 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/powershell-utils": { + "version": "0.2.1", + "resolved": "https://registry.npmjs.org/powershell-utils/-/powershell-utils-0.2.1.tgz", + "integrity": "sha512-C+y9x90UElAddDZmV4qOx9W53B61PO7cIqWz2dQsWlwswuq4mr8NEwytdGKboYbQlGZ3awrkTeNvcZiZNHnQ8A==", + "license": "MIT", + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sponsors/sindresorhus" + } + }, "node_modules/prelude-ls": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/prelude-ls/-/prelude-ls-1.2.1.tgz", @@ -5543,15 +5569,16 @@ } }, "node_modules/wsl-utils": { - "version": "0.1.0", - "resolved": "https://registry.npmjs.org/wsl-utils/-/wsl-utils-0.1.0.tgz", - "integrity": "sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw==", + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/wsl-utils/-/wsl-utils-1.0.1.tgz", + "integrity": "sha512-c16PKLIRxBYpR3jSWsdS7SFp0COm5SOQdiFjY818dYFHJbMGuLBszpF07LBWmOS1uzrHa9VGdynsEgoeKy0oOQ==", "license": "MIT", "dependencies": { - "is-wsl": "^3.1.0" + "is-wsl": "^3.1.0", + "powershell-utils": "^0.2.0" }, "engines": { - "node": ">=18" + "node": ">=20" }, "funding": { "url": "https://github.com/sponsors/sindresorhus" diff --git a/package.json b/package.json index c87ca88e3..7b25f77d5 100644 --- a/package.json +++ b/package.json @@ -138,7 +138,7 @@ "commander": "^15.0.0", "hono": "^4.13.7", "ink": "^6.0.0", - "open": "^10.2.0", + "open": "^11.0.4", "pino": "^9.14.0", "proper-lockfile": "^4.1.2", "react": "^19.0.0", From 8bee8cd26a49830e115c4ef281640d6d5bf5adf5 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 20:07:11 -0400 Subject: [PATCH 03/44] test(cli): isolate the suite's OAuth state from ~/.mcp-inspector (#2719) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit oauth-runner.test.ts and programmatic-ergonomics.test.ts reached the default NodeOAuthStorage with no override, so every CLI run rewrote the developer's real ~/.mcp-inspector/storage/oauth.json — the CLI twin of #2670. A new setupFiles entry, helpers/oauth-state-isolation.ts, points MCP_INSPECTOR_OAUTH_STATE_PATH at a per-file mkdtemp dir and fails any test whose resolved state path lies outside os.tmpdir(). The guard caught two suites whose afterEach deleted the variable when the test had not set it; they now capture the previous value in beforeEach. The MCP_STORAGE_DIR fallback case blanks the override explicitly. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/cli/__tests__/README.md | 18 ++++--- .../clear-stored-auth-for-relogin.test.ts | 9 +++- .../helpers/oauth-state-isolation.ts | 53 +++++++++++++++++++ .../cli/__tests__/relogin-revocation.test.ts | 9 +++- clients/cli/__tests__/stored-auth.test.ts | 4 +- clients/cli/vitest.config.ts | 6 ++- 6 files changed, 89 insertions(+), 10 deletions(-) create mode 100644 clients/cli/__tests__/helpers/oauth-state-isolation.ts diff --git a/clients/cli/__tests__/README.md b/clients/cli/__tests__/README.md index 122aea452..28e830e4a 100644 --- a/clients/cli/__tests__/README.md +++ b/clients/cli/__tests__/README.md @@ -7,6 +7,11 @@ Tests live under `__tests__/` and run via Vitest. coverage gate. Suite-wide `helpers/mock-open-url.ts` (vitest `setupFiles`) mocks core's `openUrl` so an armed interactive OAuth path cannot launch a real browser. +- Suite-wide `helpers/oauth-state-isolation.ts` (vitest `setupFiles`) points + `MCP_INSPECTOR_OAUTH_STATE_PATH` at a per-file temp dir so no test touches + the real `~/.mcp-inspector` OAuth store, and fails any test whose state path + resolves outside `os.tmpdir()`. A test that overrides the variable must + restore the previous value, not delete it (#2719). - `e2e.test.ts` (and root `scripts/smoke-cli.mjs`) spawn the built binary for shebang / `process.exit` paths — `pretest` builds `test-servers` + the CLI bundle first. @@ -56,12 +61,13 @@ npm run validate # format:check && lint && typecheck && test ## Helpers -| Helper | Role | -| -------------------------- | --------------------------------------------------------- | -| `helpers/cli-runner.ts` | In-process `runCli` with stdout/stderr capture | -| `helpers/mock-open-url.ts` | Suite-wide `openUrl` mock (vitest `setupFiles`) | -| `helpers/assertions.ts` | `expectCliSuccess` / `expectCliFailure` / output matchers | -| `helpers/fixtures.ts` | Temp config / client.json factories | +| Helper | Role | +| ---------------------------------- | -------------------------------------------------------------- | +| `helpers/cli-runner.ts` | In-process `runCli` with stdout/stderr capture | +| `helpers/mock-open-url.ts` | Suite-wide `openUrl` mock (vitest `setupFiles`) | +| `helpers/oauth-state-isolation.ts` | Suite-wide temp OAuth state path + guard (vitest `setupFiles`) | +| `helpers/assertions.ts` | `expectCliSuccess` / `expectCliFailure` / output matchers | +| `helpers/fixtures.ts` | Temp config / client.json factories | OAuth interactive tests stub or drive the loopback callback in-process rather than requiring a real browser. diff --git a/clients/cli/__tests__/clear-stored-auth-for-relogin.test.ts b/clients/cli/__tests__/clear-stored-auth-for-relogin.test.ts index 09b02f18f..47d171c45 100644 --- a/clients/cli/__tests__/clear-stored-auth-for-relogin.test.ts +++ b/clients/cli/__tests__/clear-stored-auth-for-relogin.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect, afterEach, vi } from "vitest"; +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import * as fs from "node:fs"; import * as os from "node:os"; import * as path from "node:path"; @@ -15,6 +15,13 @@ describe("clearStoredAuthForRelogin", () => { let dir: string | undefined; let prevPath: string | undefined; + // Capture before every test, not only the ones that point the store at a + // fixture: restoring an uncaptured `undefined` deletes the suite's isolated + // path and sends later tests to the real ~/.mcp-inspector store (#2719). + beforeEach(() => { + prevPath = process.env.MCP_INSPECTOR_OAUTH_STATE_PATH; + }); + afterEach(() => { if (prevPath === undefined) delete process.env.MCP_INSPECTOR_OAUTH_STATE_PATH; diff --git a/clients/cli/__tests__/helpers/oauth-state-isolation.ts b/clients/cli/__tests__/helpers/oauth-state-isolation.ts new file mode 100644 index 000000000..236e57c11 --- /dev/null +++ b/clients/cli/__tests__/helpers/oauth-state-isolation.ts @@ -0,0 +1,53 @@ +/** + * Keeps the CLI suite off the developer's real OAuth state (#2719). + * + * Any CLI path that reaches the default `NodeOAuthStorage` with no override — + * `oauth-runner.test.ts` and `programmatic-ergonomics.test.ts` both do — + * resolves to `~/.mcp-inspector/storage/oauth.json`, so a `local:gate` run + * rewrote the stored OAuth state of whoever ran it, and every concurrent + * worktree's gate raced on that one file. This is the CLI twin of the web + * integration project's `integrationStorageSetup.ts` (#2670). + * + * `MCP_INSPECTOR_OAUTH_STATE_PATH` is the override `getStateFilePath()` reads + * at call time, ahead of `MCP_STORAGE_DIR` and the default. It is set + * unconditionally: a value inherited from the developer's shell is exactly the + * real file this exists to keep tests away from. A setup file runs once per + * test file, so each file gets its own directory. Child processes the suite + * spawns inherit it through `process.env`. + * + * The `beforeEach` guard fails a test whose state path resolves outside + * `os.tmpdir()` — the shape a test leaves behind when its cleanup deletes the + * variable instead of restoring it. A test that points the variable elsewhere + * on purpose does so inside its own body or hooks, which run after this one. + */ + +import { afterAll, beforeEach } from "vitest"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { getStateFilePath } from "@inspector/core/auth/node/storage-node.js"; + +const tmpRoot = fs.realpathSync(os.tmpdir()); +const isolatedStorageDir = fs.mkdtempSync( + path.join(tmpRoot, "mcp-inspector-cli-oauth-"), +); + +process.env.MCP_INSPECTOR_OAUTH_STATE_PATH = path.join( + isolatedStorageDir, + "oauth.json", +); + +beforeEach(() => { + const resolved = path.resolve(getStateFilePath()); + if (!resolved.startsWith(tmpRoot + path.sep)) { + throw new Error( + `OAuth state path resolves outside ${tmpRoot}: ${resolved}. ` + + "A previous test deleted MCP_INSPECTOR_OAUTH_STATE_PATH instead of " + + "restoring it (#2719).", + ); + } +}); + +afterAll(() => { + fs.rmSync(isolatedStorageDir, { recursive: true, force: true }); +}); diff --git a/clients/cli/__tests__/relogin-revocation.test.ts b/clients/cli/__tests__/relogin-revocation.test.ts index 46025c2a8..629607f48 100644 --- a/clients/cli/__tests__/relogin-revocation.test.ts +++ b/clients/cli/__tests__/relogin-revocation.test.ts @@ -13,7 +13,7 @@ * the connect, so the assertions are about the requests made and the warnings * printed, not about the exit code. */ -import { describe, it, expect, afterEach, vi } from "vitest"; +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import * as fs from "node:fs"; import * as os from "node:os"; import * as path from "node:path"; @@ -27,6 +27,13 @@ const REVOKE_URL = `${AS_ISSUER}/revoke`; let dir: string | undefined; let prevPath: string | undefined; +// Capture before every test, not only the ones that point the store at a +// fixture: restoring an uncaptured `undefined` deletes the suite's isolated +// path and sends later tests to the real ~/.mcp-inspector store (#2719). +beforeEach(() => { + prevPath = process.env.MCP_INSPECTOR_OAUTH_STATE_PATH; +}); + afterEach(() => { if (prevPath === undefined) delete process.env.MCP_INSPECTOR_OAUTH_STATE_PATH; else process.env.MCP_INSPECTOR_OAUTH_STATE_PATH = prevPath; diff --git a/clients/cli/__tests__/stored-auth.test.ts b/clients/cli/__tests__/stored-auth.test.ts index 3147baf21..27c05655c 100644 --- a/clients/cli/__tests__/stored-auth.test.ts +++ b/clients/cli/__tests__/stored-auth.test.ts @@ -547,7 +547,9 @@ describe("--use-stored-auth", () => { "--method", "tools/list", ], - { env: { MCP_STORAGE_DIR: dir } }, + // Blank out the suite-wide isolated path (oauth-state-isolation.ts) so + // resolution falls through to MCP_STORAGE_DIR, as the title says. + { env: { MCP_STORAGE_DIR: dir, MCP_INSPECTOR_OAUTH_STATE_PATH: "" } }, ); expectCliSuccess(result); const last = server.getRecordedRequests().at(-1)!; diff --git a/clients/cli/vitest.config.ts b/clients/cli/vitest.config.ts index a71af7352..da185516c 100644 --- a/clients/cli/vitest.config.ts +++ b/clients/cli/vitest.config.ts @@ -17,7 +17,11 @@ export default defineConfig({ globals: false, environment: "node", include: ["__tests__/**/*.test.ts"], - setupFiles: ["__tests__/helpers/mock-open-url.ts", NO_RETRY_SETUP], + setupFiles: [ + "__tests__/helpers/mock-open-url.ts", + "__tests__/helpers/oauth-state-isolation.ts", + NO_RETRY_SETUP, + ], // OAuth tokens/client secrets are split into the selected secret store by // the file persistence backend the CLI shares with the web server. Pin the // in-memory store so stored-auth tests never probe or write the real OS From 756318c1d4536e6a5d8ce377244518e3cc30d122 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 20:25:29 -0400 Subject: [PATCH 04/44] fix: resolve a request's Protocol entry when its send fails MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A server that refuses a JSON-RPC request with a bare HTTP 403 sends no JSON-RPC response, so the request's Protocol-tab entry waited for one that could never arrive and read "Pending" for good — while the Tools panel already showed the error. The client's trackSendFailure now dispatches a `requestFailed` event (skipping request timeouts, which are requests the server received and has not answered). MessageLogState resolves the matching pending entry as a client-side error carrying the reason, led by the HTTP status when the error has one. ProtocolEntry titles such an entry "Request failed without a response" rather than "Rejected by the Inspector". Closes #2721 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../ProtocolEntry/ProtocolEntry.test.tsx | 40 ++++++++++++++ .../groups/ProtocolEntry/ProtocolEntry.tsx | 33 ++++++++++-- .../core/mcp/connectionDiagnostics.test.ts | 52 +++++++++++++++++- ...pectorClient-diagnostics-callbacks.test.ts | 36 +++++++++++++ .../core/mcp/state/messageLogState.test.ts | 54 +++++++++++++++++++ core/mcp/connectionDiagnostics.ts | 34 ++++++++++++ core/mcp/inspectorClient.ts | 8 +++ core/mcp/inspectorClientEventTarget.ts | 11 ++++ core/mcp/state/messageLogState.ts | 20 +++++++ core/mcp/types.ts | 3 ++ 10 files changed, 285 insertions(+), 6 deletions(-) diff --git a/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.test.tsx b/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.test.tsx index c7f19756e..20eba4af8 100644 --- a/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.test.tsx +++ b/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.test.tsx @@ -910,6 +910,46 @@ describe("ProtocolEntry — client-rejected response", () => { expect(screen.queryByText("Pending")).not.toBeInTheDocument(); }); + // A request whose send failed — a bare HTTP 403, a closed connection — has + // a clientError and no response; it was never "Pending" for anything (#2721). + const sendFailedEntry: MessageEntry = { + id: "send-failed", + timestamp: new Date("2026-07-28T10:00:00Z"), + direction: "request", + origin: "client", + message: { + jsonrpc: "2.0", + id: 12, + method: "tools/call", + params: { name: "echo", arguments: { message: "hi" } }, + }, + duration: 40, + clientError: "HTTP 403 Forbidden: Error POSTing to endpoint: Forbidden", + }; + + it("renders a failed send as Error, not Pending", () => { + renderWithMantine(); + expect(screen.getByText("Error")).toBeInTheDocument(); + expect(screen.queryByText("Pending")).not.toBeInTheDocument(); + }); + + it("titles a failed send as a request failure, not an Inspector rejection", () => { + renderWithMantine( + , + ); + expect( + screen.getByText("Request failed without a response"), + ).toBeInTheDocument(); + expect( + screen.getByText( + "HTTP 403 Forbidden: Error POSTing to endpoint: Forbidden", + ), + ).toBeInTheDocument(); + expect( + screen.queryByText("Rejected by the Inspector"), + ).not.toBeInTheDocument(); + }); + it("shows no rejection alert on an ordinary success", () => { renderWithMantine( , diff --git a/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.tsx b/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.tsx index 0ffa242d6..59deec628 100644 --- a/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.tsx +++ b/clients/web/src/components/groups/ProtocolEntry/ProtocolEntry.tsx @@ -207,6 +207,17 @@ const ClientErrorAlert = Alert.withProps({ title: "Rejected by the Inspector", }); +// A request whose send failed carries a `clientError` too, but nothing was +// rejected by the Inspector: the server refused it at the HTTP layer (a bare +// 403) or it never reached the wire, so there was no response to decode +// (#2721). The title says that instead. +const SendFailedAlert = Alert.withProps({ + variant: "light", + color: "red", + icon: , + title: "Request failed without a response", +}); + // Link (button-styled) that jumps to the correlated HTTP entry in the Network tab. const RevealLink = Anchor.withProps({ component: "button", @@ -306,6 +317,13 @@ function extractStatus( return "success"; } +// A client error on a request that never got a response is a failed send +// (messageLogState resolves it from the client's `requestFailed`); one on an +// entry that has a response is a response the Inspector refused (#1953). +function isSendFailure(entry: MessageEntry): boolean { + return entry.direction === "request" && !entry.response; +} + function statusColor(status: "success" | "error" | "pending"): string { if (status === "success") return "green"; if (status === "error") return "red"; @@ -566,11 +584,16 @@ export function ProtocolEntry({ - {entry.clientError && ( - - {entry.clientError} - - )} + {entry.clientError && + (isSendFailure(entry) ? ( + + {entry.clientError} + + ) : ( + + {entry.clientError} + + ))} {specError && ( { }); }); +describe("describeSendFailure", () => { + it("leads with the HTTP status of a bare refusal (#2721)", () => { + const err = new SdkHttpError( + SdkErrorCode.ClientHttpNotImplemented, + "Error POSTing to endpoint: Forbidden", + { status: 403, statusText: "Forbidden", text: "Forbidden" }, + ); + expect(describeSendFailure(err)).toBe( + "HTTP 403 Forbidden: Error POSTing to endpoint: Forbidden", + ); + }); + + it("states a status with no status text on its own", () => { + const err = Object.assign(new Error("Remote send failed (502): bad"), { + status: 502, + }); + expect(describeSendFailure(err)).toBe( + "HTTP 502: Remote send failed (502): bad", + ); + }); + + it("does not repeat a status the message already states", () => { + const err = Object.assign( + new Error("Error POSTing to endpoint (HTTP 403): Forbidden"), + { status: 403, statusText: "Forbidden" }, + ); + expect(describeSendFailure(err)).toBe( + "Error POSTing to endpoint (HTTP 403): Forbidden", + ); + }); + + it("falls back to the message, or the value itself, with no status", () => { + expect(describeSendFailure(new Error("Connection closed"))).toBe( + "Connection closed", + ); + expect( + describeSendFailure( + Object.assign(new Error("odd"), { status: "403", statusText: 1 }), + ), + ).toBe("odd"); + expect(describeSendFailure("boom")).toBe("boom"); + expect(describeSendFailure(null)).toBe("null"); + }); +}); + describe("isRequestTimeoutError", () => { it("recognises the SDK's per-request timeout", () => { expect(isRequestTimeoutError(sdkTimeout())).toBe(true); diff --git a/clients/web/src/test/core/mcp/inspectorClient-diagnostics-callbacks.test.ts b/clients/web/src/test/core/mcp/inspectorClient-diagnostics-callbacks.test.ts index 35adb3c67..9c710f28e 100644 --- a/clients/web/src/test/core/mcp/inspectorClient-diagnostics-callbacks.test.ts +++ b/clients/web/src/test/core/mcp/inspectorClient-diagnostics-callbacks.test.ts @@ -67,6 +67,42 @@ describe("InspectorClient diagnostics tracking callbacks", () => { expect(outstandingIds(client)).toEqual([1]); }); + it("reports a failed send so the request's log entry resolves (#2721)", () => { + const { client, callbacks } = makeClient(); + const failed: { id: string | number; reason: string }[] = []; + client.addEventListener("requestFailed", (e) => failed.push(e.detail)); + callbacks.trackRequest!(request(3), "client"); + callbacks.trackSendFailure!( + request(3), + Object.assign(new Error("Error POSTing to endpoint: Forbidden"), { + status: 403, + statusText: "Forbidden", + }), + ); + expect(failed).toEqual([ + { + id: 3, + reason: "HTTP 403 Forbidden: Error POSTing to endpoint: Forbidden", + }, + ]); + }); + + it("does not report a request timeout as a failed send", () => { + const { client, callbacks } = makeClient(); + let failed = 0; + client.addEventListener("requestFailed", () => { + failed += 1; + }); + callbacks.trackRequest!(request(4), "client"); + callbacks.trackSendFailure!( + request(4), + new SdkError(SdkErrorCode.RequestTimeout, "Request timed out", { + timeout: 60_000, + }), + ); + expect(failed).toBe(0); + }); + it("only tracks client-originated requests, and ignores a rollback for an unknown id", () => { const { client, callbacks } = makeClient(); callbacks.trackRequest!(request(7), "server"); diff --git a/clients/web/src/test/core/mcp/state/messageLogState.test.ts b/clients/web/src/test/core/mcp/state/messageLogState.test.ts index f6c11db36..ea0071fa7 100644 --- a/clients/web/src/test/core/mcp/state/messageLogState.test.ts +++ b/clients/web/src/test/core/mcp/state/messageLogState.test.ts @@ -299,4 +299,58 @@ describe("MessageLogState", () => { expect(state.getMessages()[0]?.clientError).toBeUndefined(); }); }); + + // A request whose send failed — refused with a bare HTTP 403, or never + // delivered — gets no response frame, so without this it stays "Pending" + // in the Protocol tab for good (#2721). + describe("requestFailed", () => { + const REASON = "HTTP 403 Forbidden: Error POSTing to endpoint: Forbidden"; + + it("resolves the pending request entry as a client-side error", () => { + client.dispatchTypedEvent("message", requestEntry(1, "tools/call")); + + const seen: MessageEntry[] = []; + state.addEventListener("message", (e) => seen.push(e.detail)); + client.dispatchTypedEvent("requestFailed", { id: 1, reason: REASON }); + + const [entry] = state.getMessages(); + expect(entry?.clientError).toBe(REASON); + expect(entry?.response).toBeUndefined(); + expect(entry?.duration).toBeGreaterThanOrEqual(0); + expect(seen).toEqual([entry]); + }); + + it("stops matching a late response against the failed request", () => { + client.dispatchTypedEvent("message", requestEntry(1)); + client.dispatchTypedEvent("requestFailed", { id: 1, reason: REASON }); + client.dispatchTypedEvent("message", responseEntry(1)); + + const [failed, stray] = state.getMessages(); + expect(failed?.response).toBeUndefined(); + expect(stray?.direction).toBe("response"); + }); + + it("leaves an already-answered request alone", () => { + client.dispatchTypedEvent("message", requestEntry(1)); + client.dispatchTypedEvent("message", responseEntry(1)); + + let fired = 0; + state.addEventListener("messagesChange", () => { + fired++; + }); + client.dispatchTypedEvent("requestFailed", { id: 1, reason: REASON }); + + expect(fired).toBe(0); + expect(state.getMessages()[0]?.clientError).toBeUndefined(); + }); + + it("stops listening once destroyed", () => { + client.dispatchTypedEvent("message", requestEntry(1)); + const [entry] = state.getMessages(); + state.destroy(); + client.dispatchTypedEvent("requestFailed", { id: 1, reason: REASON }); + + expect(entry?.clientError).toBeUndefined(); + }); + }); }); diff --git a/core/mcp/connectionDiagnostics.ts b/core/mcp/connectionDiagnostics.ts index 9163f557a..064decb46 100644 --- a/core/mcp/connectionDiagnostics.ts +++ b/core/mcp/connectionDiagnostics.ts @@ -112,6 +112,40 @@ export function isRequestTimeoutError( ); } +/** + * The reason a request's send failed, as the Protocol log shows it (#2721). + * + * A server that refuses a request at the HTTP layer — a bare `403` with no + * `WWW-Authenticate` — never sends a JSON-RPC response, so the log has no + * frame to show; this text is the entry's only record of how it ended. The + * HTTP status is the fact a reader needs first, so it leads whenever the + * error carries one (the SDK's `SdkHttpError`, or the remote transport's own + * `Remote send failed` error) and the message does not already state it. + */ +export function describeSendFailure(err: unknown): string { + const message = err instanceof Error ? err.message : String(err); + const status = httpStatusOf(err); + if (status === undefined || message.includes(`HTTP ${status}`)) { + return message; + } + const statusText = httpStatusTextOf(err); + return `HTTP ${status}${statusText ? ` ${statusText}` : ""}: ${message}`; +} + +function httpStatusOf(err: unknown): number | undefined { + if (typeof err !== "object" || err === null || !("status" in err)) { + return undefined; + } + return typeof err.status === "number" ? err.status : undefined; +} + +function httpStatusTextOf(err: unknown): string | undefined { + if (typeof err !== "object" || err === null || !("statusText" in err)) { + return undefined; + } + return typeof err.statusText === "string" ? err.statusText : undefined; +} + /** * A compact human duration: `850ms`, `12s`, `4m12s`, `1h02m`. Sub-second * values are shown in ms so a very short timeout still reads as a number diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index ea9db7a2e..573a8d83a 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -285,6 +285,7 @@ import { import { annotateRequestTimeout, isRequestTimeoutError, + describeSendFailure, type ConnectionDiagnostics, type LastResponse, type NotificationStreamState, @@ -1171,6 +1172,13 @@ export class InspectorClient extends InspectorClientEventTarget { if (this.outstandingRequests.delete(message.id)) { this.dispatchConnectionDiagnosticsChange(); } + // The request's log entry is resolved too: no response frame is + // coming for it, so without this it reads "Pending" forever — even + // when the server answered with a definitive HTTP refusal (#2721). + this.dispatchTypedEvent("requestFailed", { + id: message.id, + reason: describeSendFailure(error), + }); }, trackResponse: ( message: JSONRPCResultResponse | JSONRPCErrorResponse, diff --git a/core/mcp/inspectorClientEventTarget.ts b/core/mcp/inspectorClientEventTarget.ts index e57d30a5a..d85d494cc 100644 --- a/core/mcp/inspectorClientEventTarget.ts +++ b/core/mcp/inspectorClientEventTarget.ts @@ -90,6 +90,17 @@ export interface InspectorClientEventMap { * JSON-RPC id of the rejected response. */ responseRejected: { id: string | number; reason: string }; + /** + * A request this client sent that ended without any response: its send + * failed, so the server's answer — if it gave one — was not a JSON-RPC + * frame (a bare HTTP `403`, a failed fetch, a closed connection). Without + * this the request's Protocol entry would wait for a response that can + * never arrive and stay "Pending" for good (#2721). `id` is the JSON-RPC id + * of the failed request; `reason` is the failure as the log shows it. Not + * fired for a request timeout, which is a request the server did receive + * and has not answered. + */ + requestFailed: { id: string | number; reason: string }; stderrLog: StderrLogEntry; fetchRequest: FetchRequestEntry; /** Fired when an in-flight fetch's response body is read asynchronously. */ diff --git a/core/mcp/state/messageLogState.ts b/core/mcp/state/messageLogState.ts index 02a27224e..d4758ab8a 100644 --- a/core/mcp/state/messageLogState.ts +++ b/core/mcp/state/messageLogState.ts @@ -123,6 +123,24 @@ export class MessageLogState extends TypedEventTarget { } }; + // A request whose send failed gets no response frame, ever — the server + // either never saw it or refused it at the HTTP layer (a bare 403). Close + // its entry as a client-side error so it stops rendering as "Pending" + // (#2721). Only a still-pending entry is resolved: one already answered, + // or evicted from the log, is left as it is. + const onRequestFailed = ( + event: TypedEventGeneric, + ): void => { + const { id, reason } = event.detail; + const requestEntry = this.pendingRequestEntries.get(id); + if (!requestEntry) return; + this.pendingRequestEntries.delete(id); + requestEntry.clientError = reason; + requestEntry.duration = Date.now() - requestEntry.timestamp.getTime(); + this.dispatchTypedEvent("message", requestEntry); + this.dispatchTypedEvent("messagesChange", this.getMessages()); + }; + const onStatusChange = (): void => { if (isTerminalStatus(this.client?.getStatus())) { this.messages = []; @@ -132,11 +150,13 @@ export class MessageLogState extends TypedEventTarget { }; this.client.addEventListener("message", onMessage); this.client.addEventListener("responseRejected", onResponseRejected); + this.client.addEventListener("requestFailed", onRequestFailed); this.client.addEventListener("statusChange", onStatusChange); this.unsubscribe = () => { if (this.client) { this.client.removeEventListener("message", onMessage); this.client.removeEventListener("responseRejected", onResponseRejected); + this.client.removeEventListener("requestFailed", onRequestFailed); this.client.removeEventListener("statusChange", onStatusChange); } this.client = null; diff --git a/core/mcp/types.ts b/core/mcp/types.ts index bdd3aa086..ad19617b9 100644 --- a/core/mcp/types.ts +++ b/core/mcp/types.ts @@ -401,6 +401,9 @@ export interface MessageEntry { * `ttlMs`/`cacheScope`. Distinct from a JSON-RPC `error` response: the server * answered successfully and the wire frame is valid, so without this the * entry renders as a clean success even though the call failed (#1953). + * Also set on a request whose send failed — refused at the HTTP layer (a + * bare `403`) or never delivered — since no response will ever be folded + * into it and it would otherwise render as "Pending" for good (#2721). */ clientError?: string; } From 6fcbefe8b6d3509287d2d48f081c64f6631dc5cc Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 20:28:00 -0400 Subject: [PATCH 05/44] ci: install pinned npm 11 before every dependency install Node 22's bundled npm 10.x ignores each install's `allowScripts`, so the esbuild/fsevents denials from #2712 never applied in CI. Every job that installs dependencies (build, coverage, package, and the monthly dependency-refresh sweep) now installs an exact-pinned npm first, with --ignore-scripts, as `publish` already did for OIDC. main.yml states the version once as CI_NPM_VERSION, which `publish` now reads too. Also corrects comments and docs that said CI installs with npm 10. Closes #2715 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .github/workflows/dependency-refresh.yml | 7 +++++ .github/workflows/main.yml | 34 ++++++++++++++++++++++-- docs/quality-gate.md | 2 +- scripts/install-clients.test.mjs | 6 ++--- scripts/verify-install-scripts.mjs | 12 +++++---- 5 files changed, 50 insertions(+), 11 deletions(-) diff --git a/.github/workflows/dependency-refresh.yml b/.github/workflows/dependency-refresh.yml index 4e5f87a6f..320e6a8e5 100644 --- a/.github/workflows/dependency-refresh.yml +++ b/.github/workflows/dependency-refresh.yml @@ -50,6 +50,13 @@ jobs: node-version: '22.x' cache: 'npm' + - name: Install the pinned npm CLI (allowScripts needs >= 11.19) + # Node 22's bundled npm 10.x ignores each install's `allowScripts`, so + # the install below would still run the denied esbuild/fsevents + # scripts (#2715). Keep this version equal to CI_NPM_VERSION in + # main.yml, which states the full reasoning. + run: npm install -g --ignore-scripts npm@11.20.0 + - name: Install dependencies (root + all clients) run: npm install diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index a05fbcf93..aba4a9f2a 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -36,6 +36,23 @@ concurrency: group: ${{ github.event_name == 'release' && 'release-npm' || format('run-{0}', github.run_id) }} cancel-in-progress: false +# The npm the jobs below install before running npm, pinned EXACTLY and +# stated once (#2715). Node 22's bundled npm is 10.x, and two things here +# need 11.x: `publish`'s OIDC +# trusted publishing (>= 11.5.1, #2483) and the `allowScripts` denials in each +# install's package.json (>= 11.19, #2712) — npm 10 ignores that field, so on +# the bundled npm CI would still run the `esbuild` and `fsevents` install +# scripts the review decided against. (`verify:install-scripts` reads the +# lockfiles, so it enforces the DECISION on any npm; only npm >= 11.19 does +# the BLOCKING.) Exact rather than a range for the reason `publish` gives: a +# range would let whatever 11.x is current into `package`, which builds what +# the credentialed job ships (#2484). Every job that installs dependencies +# installs this npm first, with `--ignore-scripts` so even npm's own lifecycle +# scripts stay out. Bump deliberately, here and in dependency-refresh.yml +# together; stay on 11.x until 12's Node floor is checked against Node 22. +env: + CI_NPM_VERSION: '11.20.0' + # The `package`, `publish` and `publish-github-container-registry` jobs pin # every action to a commit SHA with a `# vX.Y.Z` comment; `build` and # `coverage` stay on moving major tags (#2235). A job holding a credential — or @@ -70,6 +87,10 @@ jobs: node-version: '22.x' cache: 'npm' + - name: Install the pinned npm CLI (allowScripts needs >= 11.19) + # See CI_NPM_VERSION at the top of this file (#2715). + run: npm install -g --ignore-scripts "npm@$CI_NPM_VERSION" + - name: Install dependencies (root + all clients) # The root postinstall (scripts/install-clients.mjs) cascades # `npm install` into clients/web, clients/cli, clients/tui, and @@ -203,6 +224,10 @@ jobs: node-version: '22.x' cache: 'npm' + - name: Install the pinned npm CLI (allowScripts needs >= 11.19) + # See CI_NPM_VERSION at the top of this file (#2715). + run: npm install -g --ignore-scripts "npm@$CI_NPM_VERSION" + - name: Install dependencies (root + all clients) run: npm install @@ -260,6 +285,10 @@ jobs: node-version: '22.x' cache: 'npm' + - name: Install the pinned npm CLI (allowScripts needs >= 11.19) + # See CI_NPM_VERSION at the top of this file (#2715). + run: npm install -g --ignore-scripts "npm@$CI_NPM_VERSION" + - name: Assert release tag matches package version # `npm publish` ships whatever `version` is in the root package.json, # regardless of the release's tag. Fail fast (before the heavy install / @@ -355,9 +384,10 @@ jobs: # holds `id-token: write`, so it is the one package this job trusts, and # a range would let whatever 11.x is current at release time into it. # `--ignore-scripts` keeps even that package's lifecycle scripts out. - # Bump deliberately; stay on 11.x until 12's Node floor is checked. + # The version is CI_NPM_VERSION at the top of this file, shared with + # every job that installs dependencies (#2715) — bump it there. - name: Install the pinned npm CLI (OIDC trusted publishing) - run: npm install -g --ignore-scripts npm@11.20.0 + run: npm install -g --ignore-scripts "npm@$CI_NPM_VERSION" - name: Download the tarball built by the package job uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 diff --git a/docs/quality-gate.md b/docs/quality-gate.md index 6d761ff38..69e4e19be 100644 --- a/docs/quality-gate.md +++ b/docs/quality-gate.md @@ -40,7 +40,7 @@ That is the readable half, and prose rots. The enforced half is `scripts/lib/wor | `npm run verify:build-gate` | Runs a real `vite build` with a Node built-in forced into the browser graph and asserts the build **fails** via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of `npm run local:gate` and of GitHub CI. | | `npm run verify:bundle-externals` | Guards the must-not-bundle invariant (#2067): for each tsup-bundled client it reads the **built** `build/` output and fails if any package that must stay external was inlined anyway. Candidates are the union of the client's own `external` array **and the root manifest's `dependencies`** — the latter because #2067 was a *missing* `external` entry, which a self-referential check would have passed. Detection is via esbuild's `// ` module banners, so it covers both shapes — a separate `-HASH.js` chunk (what a dynamically `import()`ed CommonJS package produces) and a statically-imported package folded straight into `index.js`, which emits no chunk at all. A build with no banners fails as such rather than passing clean, so enabling `minify` cannot silently retire the check. `undici` was declared only in the root and `clients/cli` manifests, and tsup auto-externalizes only what the _nearest_ manifest declares, so the web and TUI bundles inlined 1.05MB of it — and CommonJS inlined into an ESM bundle throws `Dynamic require of "assert" is not supported` on first use, from a specifier no user-side install can satisfy. Reads the output rather than the config because those two disagreed for four releases. Part of `npm run local:gate` and of GitHub CI. | | `npm run verify:install-fresh` | Guards against a **stale install** ([#2494](https://github.com/modelcontextprotocol/inspector/issues/2494)). Every other dependency guard reads lockfiles, so none can see `node_modules` lagging the lockfile beside it — the ordinary state of a long-lived checkout after a `git pull` brings in a bump. When the MCP SDK moved 2.0.0 → 2.1.0, such a checkout passed every guard, lint and typecheck and failed only in `transportAuthorizationPrecedence.test.ts`, reporting the old SDK's behavior as a product defect. For the root and every `clients/*` install, compares each `package-lock.json` package entry against the installed copy's own `package.json` version (ground truth, not npm's hidden lockfile), exempting absent `optional`/`devOptional` entries (platform binaries), and fails naming each stale or missing package and `npm install`. Runs first in `validate:guards`; CI installs fresh, so it never fires there. | -| `npm run verify:install-scripts` | Guards against an **unreviewed dependency install script** ([#2712](https://github.com/modelcontextprotocol/inspector/issues/2712)). npm 11.x still runs a script that `allowScripts` neither allows nor denies, so a new dependency's `postinstall` would otherwise run on every install unnoticed. For the root and every `clients/*` install, reads each `package-lock.json` entry flagged `hasInstallScript` and fails unless that install's **own** `package.json` `allowScripts` decides it (`true` or `false`), matched as npm 11.19 matches it: identity from the lockfile's `resolved` registry URL rather than the folder name, and only a bare name or exact versions joined by `||` as keys — npm ignores `^`/`~` ranges. It also reports a **non-registry** dependency (git, file, remote tarball) even when the lockfile does not flag a script, because npm runs `prepare` for those and the lockfile records only preinstall/install/postinstall; such an entry is cleared by a key equal to its exact `resolved` spec. A bundled dependency with a script is always reported, since no key can decide it. Reads the lockfile rather than `npm install-scripts ls` because CI's Node 22 npm 10.x has no such subcommand. `esbuild` and `fsevents` are denied: neither script is needed (platform binaries arrive via `optionalDependencies` / prebuilt). | +| `npm run verify:install-scripts` | Guards against an **unreviewed dependency install script** ([#2712](https://github.com/modelcontextprotocol/inspector/issues/2712)). npm 11.x still runs a script that `allowScripts` neither allows nor denies, so a new dependency's `postinstall` would otherwise run on every install unnoticed. For the root and every `clients/*` install, reads each `package-lock.json` entry flagged `hasInstallScript` and fails unless that install's **own** `package.json` `allowScripts` decides it (`true` or `false`), matched as npm 11.19 matches it: identity from the lockfile's `resolved` registry URL rather than the folder name, and only a bare name or exact versions joined by `||` as keys — npm ignores `^`/`~` ranges. It also reports a **non-registry** dependency (git, file, remote tarball) even when the lockfile does not flag a script, because npm runs `prepare` for those and the lockfile records only preinstall/install/postinstall; such an entry is cleared by a key equal to its exact `resolved` spec. A bundled dependency with a script is always reported, since no key can decide it. Reads the lockfile rather than `npm install-scripts ls` because Node 22's bundled npm 10.x has no such subcommand. It enforces the **decision**; the **blocking** is npm's, from 11.19 — which is why every CI job that installs dependencies first installs the exact npm pinned as `CI_NPM_VERSION` in `main.yml` ([#2715](https://github.com/modelcontextprotocol/inspector/issues/2715)). `esbuild` and `fsevents` are denied: neither script is needed (platform binaries arrive via `optionalDependencies` / prebuilt). | | `npm run verify:format-coverage` | Parses the `format:check` globs out of every `package.json` (only those reachable from `validate`), enumerates all tracked source files, and **fails** listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in `validate`. | | `npm run verify:skills` | Guards `.claude/skills` ([#2163](https://github.com/modelcontextprotocol/inspector/issues/2163)). A skill's body loads only when it is invoked, so one that stops being reachable loses behavior **silently** — and the way that happens is frontmatter Claude Code cannot parse, which loads the body with an *empty* description: `/skill-name` still works, so a manual spot check passes, while nothing is left for the model to match against. This parses each `SKILL.md` the same way, requires an explicit invocation mode on every skill and committed eval cases (positives **and** negatives) on every model-invoked one, and fails when the skill listing exceeds its recorded character budget. Also runs `claude plugin validate` — the authoritative schema — when the installed CLI is **exactly** the pinned version, and otherwise says so and moves on. It runs inside `validate:guards`, which both `validate` (CI) and `local:validate` (the gate's first stage) execute *before* the pinned step, so accepting any other local CLI would let it reject skills that CI accepts and exit the gate before the reproducible check ever ran. Runs in `validate`; because that hand-off is best-effort, the authoritative validator also gets a guaranteed step of its own — see `verify:skills:cli` below. | | `npm run verify:skills:cli` | Runs `claude plugin validate` — the **authoritative** skill schema — as a guaranteed step, in `local:gate` and in CI. `verify:skills` reimplements the parse and *skips* when the CLI is absent, which is right for `validate` (fast, offline, no Claude Code required) but adds up to "never runs". This one resolves the CLI instead: an installed one **only when it matches the pin exactly**, otherwise the pinned package via `npx -y @anthropic-ai/claude-code@…`. Exact, not a floor: accepting a newer local CLI would validate against a different schema than CI's, so the same `local:gate` could disagree across machines — which is the failure a pin exists to prevent. Needs no authentication. Not in `validate` — it can reach the network. | diff --git a/scripts/install-clients.test.mjs b/scripts/install-clients.test.mjs index 9c2ac28ac..c83763779 100644 --- a/scripts/install-clients.test.mjs +++ b/scripts/install-clients.test.mjs @@ -181,9 +181,9 @@ for (const nodeArgs of [[], ["--preserve-symlinks-main"]]) { /** * A stand-in `npm` on PATH that fails, as npm 11.19+ does, when * allow-scripts reaches it through the environment. It pins the spawn wiring - * whatever npm the runner has: CI's Node 22 bundles npm 10, which predates - * the gate, so against the real npm there the cascade passes with or without - * the strip. + * whatever npm the runner has: Node 22 bundles npm 10, which predates the + * gate, so against the real npm there the cascade passes with or without the + * strip. */ function stubNpm() { const dir = mkdtempSync(join(tmpdir(), "install-clients-npm-")); diff --git a/scripts/verify-install-scripts.mjs b/scripts/verify-install-scripts.mjs index a48e1b7c9..306a7f831 100644 --- a/scripts/verify-install-scripts.mjs +++ b/scripts/verify-install-scripts.mjs @@ -5,8 +5,9 @@ // in the installing project's `package.json`, but in 11.x a package that is // neither allowed nor denied still RUNS its script, with only a warning — only // an explicit `false` blocks it. npm 12 blocks unreviewed scripts by default, -// but this repo installs with npm 10 (CI's Node 22 bundle) and 11.x, so here a -// new dependency that brings a `postinstall` still runs it on every install, +// but this repo installs with 11.x (CI pins it, #2715) or with Node 22's +// bundled npm 10, which ignores `allowScripts` entirely, so here a new +// dependency that brings a `postinstall` still runs it on every install, // silently. // This guard turns each one into a decision made in review: every package with // an install script must carry an explicit `true` or `false` in the @@ -19,9 +20,10 @@ // // It reads the LOCKFILE's `hasInstallScript` rather than shelling out to // `npm install-scripts ls`, which is what the issue first proposed: that -// subcommand only exists from npm 11.19, and CI's `build` job runs Node 22's -// bundled npm 10.x, where it is absent — so the guard would either fail in CI -// for an unrelated reason or skip there and enforce nothing. The lockfile flag +// subcommand only exists from npm 11.19, and Node 22's bundled npm 10.x — +// which a contributor on the supported Node floor may well be running, and +// which CI ran until #2715 pinned 11.x — has no such subcommand, so the guard +// would either fail there for an unrelated reason or skip and enforce nothing. The lockfile flag // is what npm itself records for exactly this question (preinstall, install, // postinstall, or an implicit `node-gyp rebuild` from a `binding.gyp`), and // reading it keeps the guard offline and independent of the npm on PATH. The From b54eb82433a13ff8aa470542f0a8c47dd55c8180 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 20:21:43 -0400 Subject: [PATCH 06/44] feat(web): optional TLS for the web UI listener INSPECTOR_TLS_KEY_PATH + INSPECTOR_TLS_CERT_PATH serve the UI over https from both the prod Hono listener and Vite development. Both or neither: a half-set pair, an unreadable file, or an unusable key/cert pair fails startup with a message naming the variable, never a silent http fallback. With TLS on, the banner, the port-in-use error and the default ALLOWED_ORIGINS all use https. Under TLS Vite serves HTTP/2, whose pseudo-headers made the dev /api bridge throw on every request, so that conversion moves into a tested node-request-bridge module. The MCP Apps sandbox and app-origin listeners stay plain http, per the issue. docs/tls.md covers mkcert, custom hostnames, NODE_EXTRA_CA_CERTS and why not NODE_TLS_REJECT_UNAUTHORIZED=0. Closes #2446 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- README.md | 1 + clients/web/README.md | 1 + clients/web/server/node-request-bridge.ts | 63 ++++++++ clients/web/server/server.ts | 12 +- clients/web/server/start-vite-dev-server.ts | 4 + clients/web/server/tls.ts | 93 ++++++++++++ clients/web/server/vite-hono-plugin.ts | 33 ++-- clients/web/server/web-server-config.ts | 71 ++++++--- .../web/src/test/fixtures/selfSignedTls.ts | 55 +++++++ .../server/node-request-bridge.test.ts | 70 +++++++++ .../src/test/integration/server/tls.test.ts | 142 ++++++++++++++++++ .../server/web-server-config.test.ts | 140 ++++++++++++++++- clients/web/vite.config.ts | 4 + docs/environment-variables.md | 11 +- docs/tls.md | 71 +++++++++ 15 files changed, 733 insertions(+), 38 deletions(-) create mode 100644 clients/web/server/node-request-bridge.ts create mode 100644 clients/web/server/tls.ts create mode 100644 clients/web/src/test/fixtures/selfSignedTls.ts create mode 100644 clients/web/src/test/integration/server/node-request-bridge.test.ts create mode 100644 clients/web/src/test/integration/server/tls.test.ts create mode 100644 docs/tls.md diff --git a/README.md b/README.md index 7a7e7f388..eeeb6def8 100644 --- a/README.md +++ b/README.md @@ -87,6 +87,7 @@ Each client has its own README with client-specific detail: | [Writing a skill](./docs/skill-authoring.md) | How to write a skill description that actually fires, and eval cases that measure it — the case shapes that work, and the tuning loop | | [Test servers](./docs/test-servers.md) | The composable test servers and the showcase config for every feature — what to run, what to click, and what the broken build did | | [Publishing](./docs/publishing.md) | What ships in the tarball, the packaging invariants, and `pack:verify` | +| [Serving the web UI over HTTPS](./docs/tls.md) | `INSPECTOR_TLS_KEY_PATH` / `INSPECTOR_TLS_CERT_PATH`, a trusted local mkcert setup, custom hostnames and `ALLOWED_ORIGINS`, and `NODE_EXTRA_CA_CERTS` for outbound private-CA servers | | [Docker](./docs/docker.md) | Running the container image — ports, volumes, and making secrets durable in a container | | [Where secrets are stored](./docs/secret-storage.md) | How the secret store is chosen on every runtime — OS keychain, `secrets.json` or memory — plus file encryption, locking, and moving back to a keychain | | [Migrating from v1 to v2](./docs/v1-to-v2-migration.md) | CLI flag mapping, `--config` vs. `--catalog`, the Node engine bump, env-var renames | diff --git a/clients/web/README.md b/clients/web/README.md index fd64ea5df..5a16451d7 100644 --- a/clients/web/README.md +++ b/clients/web/README.md @@ -18,6 +18,7 @@ The `server/` directory holds the Node-only backend: - **`run-web.ts`** / **`start-vite-dev-server.ts`** — entry points the launcher calls for prod `--web` and `--web --dev`. - **`web-server-config.ts`** — env parsing, the `GET /api/config` payload, the startup banner, the default origin allow-list. - **`resolve-bind-host.ts`** — the shared bind-host guard (refuses an all-interfaces `HOST` unless `DANGEROUSLY_BIND_ALL_INTERFACES`), used by both bind points (`web-server-config.ts` + `vite.config.ts`); see [Host binding & the origin allow-list](#host-binding--the-origin-allow-list). +- **`tls.ts`** — resolves the optional `INSPECTOR_TLS_KEY_PATH` / `INSPECTOR_TLS_CERT_PATH` pair both listeners serve `https` with, failing startup on a half-set or unusable pair (see [Serving the web UI over HTTPS](../../docs/tls.md)); **`node-request-bridge.ts`** — converts a dev-server Node request into the Fetch `Request` Hono takes, including the HTTP/2 form Vite serves under TLS. - **`inject-auth-token.ts`** — embeds the API token into the served `index.html` (see [Auth token](#auth-token)). - **`health.ts`** — the unauthenticated `GET /healthz` liveness/readiness probe both backends answer (see [Health check](#health-check)). - **`sandbox-controller.ts`** — the MCP Apps sandbox HTTP server; **`app-origin-controller.ts`** — the dedicated app-origin server for `_meta.ui.domain` (see [MCP App dedicated origins](#mcp-app-dedicated-origins-metauidomain)); **`public-address.ts`** — validates `MCP_SANDBOX_FULL_ADDRESS` / `MCP_APP_ORIGIN_FULL_ADDRESS`, the public addresses those two advertise behind a reverse proxy; **`ensure-web-build.ts`** — builds `dist/` on demand for prod `--web`; **`vite-base-config.ts`** — shared `optimizeDeps` exclusions. diff --git a/clients/web/server/node-request-bridge.ts b/clients/web/server/node-request-bridge.ts new file mode 100644 index 000000000..3790c10bf --- /dev/null +++ b/clients/web/server/node-request-bridge.ts @@ -0,0 +1,63 @@ +/** + * The Node-request → Fetch-`Request` half of the Vite dev plugin's `/api` + * bridge, factored out of `vite-hono-plugin.ts` (which is coverage-excluded + * runtime glue) so it can be tested directly. + * + * It exists as its own module because of TLS (#2446): with `server.https` set, + * Vite serves HTTP/2 (falling back to HTTP/1.1 for clients that don't + * negotiate it), and an HTTP/2 request differs from an HTTP/1.1 one in two + * ways the original inline code did not survive: + * + * - its headers include the pseudo-headers `:method`, `:scheme`, `:authority` + * and `:path`, which `Headers.set` rejects as invalid names — so copying + * them verbatim threw and every `/api/*` call failed; + * - it carries no `host` header; the authority is in `:authority`. + * + * And on the way back, HTTP/2 forbids connection-specific response headers + * (`Connection`, `Keep-Alive`, `Transfer-Encoding`, …); Node drops them with + * an `UnsupportedWarning` per request, so the bridge skips them instead. + */ + +import type { IncomingHttpHeaders } from "node:http"; + +/** The request URL Hono sees, with the listener's real scheme and authority. */ +export function fetchUrlForNodeRequest( + headers: IncomingHttpHeaders, + pathname: string, + scheme: "http" | "https", +): string { + const authority = headers.host ?? headers[":authority"]; + return `${scheme}://${authority}${pathname}`; +} + +/** Copy a Node request's headers into Fetch `Headers`, minus HTTP/2 pseudo-headers. */ +export function fetchHeadersFromNode(headers: IncomingHttpHeaders): Headers { + const out = new Headers(); + for (const [key, value] of Object.entries(headers)) { + if (!value || key.startsWith(":")) continue; + out.set(key, Array.isArray(value) ? value.join(", ") : value); + } + return out; +} + +const CONNECTION_SPECIFIC_HEADERS = new Set([ + "connection", + "keep-alive", + "proxy-connection", + "transfer-encoding", + "upgrade", +]); + +/** + * Whether a response header may be set on a response of this HTTP major + * version. Everything is allowed on HTTP/1.x; HTTP/2 refuses the + * connection-specific headers (RFC 9113 §8.2.2). + */ +export function isResponseHeaderAllowed( + name: string, + httpVersionMajor: number, +): boolean { + return ( + httpVersionMajor < 2 || !CONNECTION_SPECIFIC_HEADERS.has(name.toLowerCase()) + ); +} diff --git a/clients/web/server/server.ts b/clients/web/server/server.ts index efe82b7bd..cc0080d29 100644 --- a/clients/web/server/server.ts +++ b/clients/web/server/server.ts @@ -9,12 +9,12 @@ import { readFileSync } from "node:fs"; import { join, dirname } from "node:path"; import { fileURLToPath } from "node:url"; import { randomBytes } from "node:crypto"; +import { createServer as createHttpsServer } from "node:https"; import { serve } from "@hono/node-server"; import { serveStatic } from "@hono/node-server/serve-static"; import { Hono } from "hono"; import type { Context } from "hono"; import { createRemoteApp } from "../../../core/mcp/remote/node/server.ts"; -import { formatHostForUrl } from "../../../core/node/hostUrl.ts"; import { createSandboxController } from "./sandbox-controller.js"; import { createAppOriginController, @@ -28,6 +28,7 @@ import { webServerConfigToInitialPayload, printServerBanner, openBrowser, + portInUseMessage, } from "./web-server-config.js"; import type { WebServerHandle } from "./types.js"; @@ -152,6 +153,11 @@ export async function startHonoServer( fetch: app.fetch, port: config.port, hostname: config.hostname, + // https only when both TLS paths were set (#2446); otherwise + // @hono/node-server's default plain `http.createServer`. + ...(config.tls + ? { createServer: createHttpsServer, serverOptions: config.tls } + : {}), }, (info) => { const sandboxUrl = sandboxController.getUrl(); @@ -172,9 +178,7 @@ export async function startHonoServer( httpServer.on("error", (err: Error) => { if (err.message.includes("EADDRINUSE")) { - console.error( - `MCP Inspector PORT IS IN USE at http://${formatHostForUrl(config.hostname)}:${config.port}`, - ); + console.error(portInUseMessage(config)); process.exit(1); } else { throw err; diff --git a/clients/web/server/start-vite-dev-server.ts b/clients/web/server/start-vite-dev-server.ts index 18299b197..34e1338d4 100644 --- a/clients/web/server/start-vite-dev-server.ts +++ b/clients/web/server/start-vite-dev-server.ts @@ -64,6 +64,10 @@ export async function startViteDevServer( // `config.port`, so a drifted bind would 403 every connect and CSP-block // the MCP Apps iframe while the banner advertises the unusable port. strictPort: true, + // The same TLS pair the prod listener uses (#2446), matching + // `vite.config.ts`. Vite serves it over HTTP/2 with an HTTP/1.1 + // fallback; the Hono plugin's /api bridge handles both. + https: config.tls, // Allow Vite to serve source files from the repo root (core/ lives // outside clients/web), matching `vite.config.ts`'s `server.fs.allow`. fs: { diff --git a/clients/web/server/tls.ts b/clients/web/server/tls.ts new file mode 100644 index 000000000..34c964790 --- /dev/null +++ b/clients/web/server/tls.ts @@ -0,0 +1,93 @@ +/** + * Optional TLS for the Inspector web listener (#2446). + * + * TLS is on only when BOTH `INSPECTOR_TLS_KEY_PATH` and + * `INSPECTOR_TLS_CERT_PATH` are set. Both callers — the prod Hono server and + * the Vite dev server — read the same resolved pair off `WebServerConfig.tls`, + * so dev and prod cannot disagree about whether the UI is on https. + * + * Every failure is a startup error rather than a fallback to plain http: a + * user who configured a certificate and silently gets http instead would see + * their https origin 403 (or the page refuse to load) with nothing pointing + * at the cause. So a half-configured pair, an unreadable file, and a key and + * certificate Node cannot use together all throw, each with a message naming + * the variable(s) involved instead of a raw `ENOENT` or OpenSSL stack. + * + * Only the UI listener is covered. The MCP Apps sandbox and app-origin + * listeners stay plain http — browsers treat loopback http as a secure + * context, and a public deployment fronts them with a proxy via + * `MCP_SANDBOX_FULL_ADDRESS` / `MCP_APP_ORIGIN_FULL_ADDRESS` (see docs/tls.md). + */ + +import { readFileSync } from "node:fs"; +import { createSecureContext } from "node:tls"; + +export const TLS_KEY_PATH_ENV = "INSPECTOR_TLS_KEY_PATH"; +export const TLS_CERT_PATH_ENV = "INSPECTOR_TLS_CERT_PATH"; + +/** + * The PEM key and certificate the listener serves. Shaped to drop straight + * into `https.createServer` options and Vite's `server.https`. + */ +export interface TlsConfig { + key: string; + cert: string; +} + +/** The URL scheme the web listener serves, given its resolved TLS config. */ +export function webScheme(tls: TlsConfig | undefined): "http" | "https" { + return tls ? "https" : "http"; +} + +function readPem(env: string, path: string): string { + try { + return readFileSync(path, "utf-8"); + } catch (err) { + // A filesystem error always carries `code` (ENOENT, EACCES, EISDIR, …); + // the code alone names the cause without the raw `open ''` message. + const { code } = err as NodeJS.ErrnoException; + throw new Error( + `Cannot read ${env}="${path}" (${code}): check that the file exists and is readable.`, + { cause: err }, + ); + } +} + +/** + * Resolve the TLS pair from the environment, or `undefined` when neither + * variable is set. A blank value counts as unset, matching how the other web + * variables (`CLIENT_PORT`, `HOST`) treat an empty string. + * + * @throws when only one of the two is set, when either file cannot be read, + * or when the two do not form a usable key/certificate pair. + */ +export function resolveTlsConfig( + env: NodeJS.ProcessEnv = process.env, +): TlsConfig | undefined { + const keyPath = env[TLS_KEY_PATH_ENV]?.trim(); + const certPath = env[TLS_CERT_PATH_ENV]?.trim(); + if (!keyPath && !certPath) return undefined; + if (!keyPath || !certPath) { + const missing = keyPath ? TLS_CERT_PATH_ENV : TLS_KEY_PATH_ENV; + throw new Error( + `TLS needs both ${TLS_KEY_PATH_ENV} and ${TLS_CERT_PATH_ENV}; ${missing} is not set. Set both to enable https, or neither for plain http.`, + ); + } + const key = readPem(TLS_KEY_PATH_ENV, keyPath); + const cert = readPem(TLS_CERT_PATH_ENV, certPath); + // Validate the pair now rather than at bind time, where the same OpenSSL + // error surfaces from inside the server constructor with no hint of which + // variable to fix. Catches a non-PEM file and a key that does not match the + // certificate alike. + try { + createSecureContext({ key, cert }); + } catch (err) { + // createSecureContext only throws Error instances (OpenSSL failures). + const reason = (err as Error).message; + throw new Error( + `${TLS_KEY_PATH_ENV}="${keyPath}" and ${TLS_CERT_PATH_ENV}="${certPath}" are not a usable PEM key/certificate pair (${reason}).`, + { cause: err }, + ); + } + return { key, cert }; +} diff --git a/clients/web/server/vite-hono-plugin.ts b/clients/web/server/vite-hono-plugin.ts index 0852fcb06..079f3fb59 100644 --- a/clients/web/server/vite-hono-plugin.ts +++ b/clients/web/server/vite-hono-plugin.ts @@ -22,6 +22,12 @@ import { } from "./app-origin-controller.js"; import { injectAuthToken } from "./inject-auth-token.js"; import { handleNodeHealthRequest } from "./health.js"; +import { + fetchHeadersFromNode, + fetchUrlForNodeRequest, + isResponseHeaderAllowed, +} from "./node-request-bridge.js"; +import { webScheme } from "./tls.js"; import type { WebServerConfig } from "./web-server-config.js"; import { getSecretStorageInfo } from "../../../core/auth/node/secret-store-selection.ts"; import { @@ -179,18 +185,14 @@ export function honoMiddlewarePlugin(config: WebServerConfig): Plugin { if (!pathname.startsWith("/api")) { return next(); } - const url = `http://${req.headers.host}${pathname}`; - const headers = new Headers(); - Object.entries(req.headers).forEach( - ([key, value]: [string, string | string[] | undefined]) => { - if (value) { - headers.set( - key, - Array.isArray(value) ? value.join(", ") : value, - ); - } - }, + // Over TLS Vite serves HTTP/2, whose requests carry pseudo-headers + // and no `host` — see node-request-bridge.ts. + const url = fetchUrlForNodeRequest( + req.headers, + pathname, + webScheme(config.tls), ); + const headers = fetchHeadersFromNode(req.headers); const init: RequestInit = { method: req.method, headers }; if (req.method !== "GET" && req.method !== "HEAD") { const chunks: Buffer[] = []; @@ -213,7 +215,9 @@ export function honoMiddlewarePlugin(config: WebServerConfig): Plugin { const response = await honoApp.fetch(new Request(url, init)); res.statusCode = response.status; response.headers.forEach((value, key) => { - res.setHeader(key, value); + if (isResponseHeaderAllowed(key, req.httpVersionMajor)) { + res.setHeader(key, value); + } }); const isSSE = response.headers .get("content-type") @@ -221,7 +225,10 @@ export function honoMiddlewarePlugin(config: WebServerConfig): Plugin { if (isSSE) { res.setHeader("X-Accel-Buffering", "no"); res.setHeader("Cache-Control", "no-cache"); - res.setHeader("Connection", "keep-alive"); + // HTTP/2 streams have no per-connection keep-alive to request. + if (isResponseHeaderAllowed("connection", req.httpVersionMajor)) { + res.setHeader("Connection", "keep-alive"); + } } if (response.body) { res.flushHeaders?.(); diff --git a/clients/web/server/web-server-config.ts b/clients/web/server/web-server-config.ts index 29ca4c151..695fb0584 100644 --- a/clients/web/server/web-server-config.ts +++ b/clients/web/server/web-server-config.ts @@ -29,8 +29,10 @@ import { } from "./public-address.js"; import { canonicalUrlHost, + formatHostForUrl, isAllInterfacesHost, } from "../../../core/node/hostUrl.ts"; +import { resolveTlsConfig, webScheme, type TlsConfig } from "./tls.js"; // The single-source Inspector version (root package.json), read once at load. // The browser can't read the filesystem the way the CLI/TUI do, so the backend @@ -96,6 +98,11 @@ export interface WebServerConfig { autoOpen: boolean; /** Root directory for static files (index.html, assets). When runner starts server in-process, pass path to dist/. */ staticRoot?: string; + /** + * PEM key/certificate the UI listener serves https with, or undefined for + * plain http (#2446). Covers the UI listener only — see `tls.ts`. + */ + tls?: TlsConfig; } function defaultEnvironmentFromProcess( @@ -211,14 +218,14 @@ export function printServerBanner( // address); otherwise use the SAME canonical host the allow-list emits, so the // advertised URL's origin is always a member of `allowedOrigins` // (`banner ⊆ allowedOrigins`) — including the IPv4-mapped case where - // `canonicalUrlHost` unmaps but `formatHostForUrl` wouldn't. `httpOrigin` - // keeps the banner and the list agreeing on the default-port form (both drop - // :80). Computing `h` once keeps the predicate and the value reading the same + // `canonicalUrlHost` unmaps but `formatHostForUrl` wouldn't. `webOrigin` + // keeps the banner and the list agreeing on the scheme and the default-port + // form (both drop :80 for http, :443 for https). Computing `h` once keeps the predicate and the value reading the same // form; `isAllInterfacesHost` also canonicalizes internally, so this is // belt-and-braces, not what makes them agree. const h = canonicalUrlHost(config.hostname); const bannerHost = isAllInterfacesHost(h) ? "localhost" : h; - const baseUrl = httpOrigin(bannerHost, actualPort); + const baseUrl = webOrigin(webScheme(config.tls), bannerHost, actualPort); const url = config.dangerouslyOmitAuth || !resolvedToken ? baseUrl @@ -249,6 +256,14 @@ export function printServerBanner( return url; } +/** + * The fatal message when the UI listener's port is taken, in the same scheme + * the banner would have printed so the two never name different URLs. + */ +export function portInUseMessage(config: WebServerConfig): string { + return `MCP Inspector PORT IS IN USE at ${webScheme(config.tls)}://${formatHostForUrl(config.hostname)}:${config.port}`; +} + /** * Launch the user's browser at `url`, reporting a failure rather than throwing. * @@ -299,20 +314,28 @@ export interface BuildWebServerConfigOptions { const LOOPBACK_HOSTNAMES = new Set(["localhost", "127.0.0.1", "[::1]"]); /** - * Build an http origin string. The port is omitted when it's the http scheme - * default (80): browsers drop the default port from `Origin`, and the guard is - * an exact string match, so `http://host:80` could never match a real request. + * Build an origin string. The port is omitted when it's the scheme's default + * (80 for http, 443 for https): browsers drop the default port from `Origin`, + * and the guard is an exact string match, so `http://host:80` could never + * match a real request. */ -function httpOrigin(host: string, port: number): string { - return port === 80 ? `http://${host}` : `http://${host}:${port}`; +function webOrigin( + scheme: "http" | "https", + host: string, + port: number, +): string { + const defaultPort = scheme === "https" ? 443 : 80; + return port === defaultPort + ? `${scheme}://${host}` + : `${scheme}://${host}:${port}`; } /** The three interchangeable loopback origin forms for a port. */ -function loopbackOrigins(port: number): string[] { +function loopbackOrigins(scheme: "http" | "https", port: number): string[] { return [ - httpOrigin("localhost", port), - httpOrigin("127.0.0.1", port), - httpOrigin("[::1]", port), + webOrigin(scheme, "localhost", port), + webOrigin(scheme, "127.0.0.1", port), + webOrigin(scheme, "[::1]", port), ]; } @@ -346,27 +369,32 @@ function loopbackOrigins(port: number): string[] { * For a specific non-loopback host (a real IP/hostname) we return the single * exact origin — canonicalized and, for an IPv6 literal, bracketed, so it * matches the `Origin` header the browser actually sends. + * + * Every origin carries `scheme` — `https` when the UI listener serves TLS + * (#2446) — since the browser's `Origin` includes it and an `http://` entry + * can never match a page loaded over https. */ export function defaultAllowedOrigins( hostname: string, port: number, + scheme: "http" | "https" = "http", ): string[] { // The loopback lookup and the emitted origin both read the canonicalized `h` // (the wildcard check via `isAllInterfacesHost` canonicalizes internally), so // every branch reasons about the same address. const h = canonicalUrlHost(hostname); if (LOOPBACK_HOSTNAMES.has(h)) { - return loopbackOrigins(port); + return loopbackOrigins(scheme, port); } if (isAllInterfacesHost(h)) { return [ - ...loopbackOrigins(port), - httpOrigin("0.0.0.0", port), - httpOrigin("[::]", port), + ...loopbackOrigins(scheme, port), + webOrigin(scheme, "0.0.0.0", port), + webOrigin(scheme, "[::]", port), ]; } // `h` is already the canonical, URL-ready host (bracketed for IPv6). - return [httpOrigin(h, port)]; + return [webOrigin(scheme, h, port)]; } /** @@ -397,6 +425,10 @@ export function buildWebServerConfig( ); } const hostname = resolveBindHostname(); + // Read before the allow-list below, whose default origins take its scheme. + // Throws on a half-set pair or an unreadable/unusable file; run-web and the + // Vite config both surface that message as a startup error. + const tls = resolveTlsConfig(); // Only an explicit `true`/`1` omits auth — `!!value` read `=false` as "on" // and silently disabled the /api/* bearer gate (#2331). const dangerouslyOmitAuth = isEnvFlagEnabled( @@ -494,7 +526,7 @@ export function buildWebServerConfig( .filter((o): o is string => o !== null); const allowedOrigins = configuredOrigins?.length ? configuredOrigins - : defaultAllowedOrigins(hostname, port); + : defaultAllowedOrigins(hostname, port, webScheme(tls)); // Resolved after the allow-list because both are checked against it: neither // listener may share an origin with the Inspector page (#1862). The sandbox @@ -527,6 +559,7 @@ export function buildWebServerConfig( appOriginPublicOrigin, logger, autoOpen: resolveAutoOpen(), + ...(tls ? { tls } : {}), }; } diff --git a/clients/web/src/test/fixtures/selfSignedTls.ts b/clients/web/src/test/fixtures/selfSignedTls.ts new file mode 100644 index 000000000..484efe08c --- /dev/null +++ b/clients/web/src/test/fixtures/selfSignedTls.ts @@ -0,0 +1,55 @@ +/** + * A throwaway self-signed key/certificate pair for the TLS listener tests + * (#2446), generated into a fresh temp directory per suite. + * + * Generated rather than committed: a checked-in private key trips secret + * scanners and a checked-in certificate eventually expires, turning a green + * suite red on a date nobody chose. `openssl` ships on every runner this repo + * targets (CI is ubuntu-latest; macOS has LibreSSL), and a missing binary + * fails the suite loudly in `beforeAll` rather than skipping it. + */ + +import { execFileSync } from "node:child_process"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +export interface SelfSignedTlsFixture { + dir: string; + keyPath: string; + certPath: string; + cleanup: () => void; +} + +export function createSelfSignedTls(): SelfSignedTlsFixture { + const dir = mkdtempSync(join(tmpdir(), "inspector-tls-")); + const keyPath = join(dir, "key.pem"); + const certPath = join(dir, "cert.pem"); + execFileSync( + "openssl", + [ + "req", + "-x509", + "-newkey", + "rsa:2048", + "-nodes", + "-keyout", + keyPath, + "-out", + certPath, + "-days", + "1", + "-subj", + "/CN=localhost", + "-addext", + "subjectAltName=DNS:localhost,IP:127.0.0.1", + ], + { stdio: "ignore" }, + ); + return { + dir, + keyPath, + certPath, + cleanup: () => rmSync(dir, { recursive: true, force: true }), + }; +} diff --git a/clients/web/src/test/integration/server/node-request-bridge.test.ts b/clients/web/src/test/integration/server/node-request-bridge.test.ts new file mode 100644 index 000000000..f4cd0c5d8 --- /dev/null +++ b/clients/web/src/test/integration/server/node-request-bridge.test.ts @@ -0,0 +1,70 @@ +import { describe, it, expect } from "vitest"; +import { + fetchHeadersFromNode, + fetchUrlForNodeRequest, + isResponseHeaderAllowed, +} from "../../../../server/node-request-bridge.js"; + +describe("fetchUrlForNodeRequest", () => { + it("uses the HTTP/1.1 host header", () => { + expect( + fetchUrlForNodeRequest({ host: "localhost:6274" }, "/api/config", "http"), + ).toBe("http://localhost:6274/api/config"); + }); + + it("falls back to the HTTP/2 :authority pseudo-header", () => { + expect( + fetchUrlForNodeRequest( + { ":authority": "localhost:6274" }, + "/api/config", + "https", + ), + ).toBe("https://localhost:6274/api/config"); + }); +}); + +describe("fetchHeadersFromNode", () => { + it("drops HTTP/2 pseudo-headers, which Headers rejects", () => { + const headers = fetchHeadersFromNode({ + ":method": "GET", + ":scheme": "https", + ":authority": "localhost:6274", + ":path": "/api/config", + "x-mcp-remote-auth": "Bearer t", + }); + expect([...headers.keys()]).toEqual(["x-mcp-remote-auth"]); + }); + + it("joins multi-value headers and skips empty ones", () => { + const headers = fetchHeadersFromNode({ + "x-multi": ["a", "b"], + origin: "https://localhost:6274", + cookie: undefined, + "x-empty": "", + }); + expect(headers.get("x-multi")).toBe("a, b"); + expect(headers.get("origin")).toBe("https://localhost:6274"); + expect(headers.has("cookie")).toBe(false); + expect(headers.has("x-empty")).toBe(false); + }); +}); + +describe("isResponseHeaderAllowed", () => { + it("allows every header on HTTP/1.x", () => { + expect(isResponseHeaderAllowed("Connection", 1)).toBe(true); + expect(isResponseHeaderAllowed("content-type", 1)).toBe(true); + }); + + it("refuses connection-specific headers on HTTP/2, case-insensitively", () => { + for (const name of [ + "Connection", + "keep-alive", + "Proxy-Connection", + "transfer-encoding", + "upgrade", + ]) { + expect(isResponseHeaderAllowed(name, 2)).toBe(false); + } + expect(isResponseHeaderAllowed("content-type", 2)).toBe(true); + }); +}); diff --git a/clients/web/src/test/integration/server/tls.test.ts b/clients/web/src/test/integration/server/tls.test.ts new file mode 100644 index 000000000..d5a638628 --- /dev/null +++ b/clients/web/src/test/integration/server/tls.test.ts @@ -0,0 +1,142 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { generateKeyPairSync } from "node:crypto"; +import { readFileSync, writeFileSync } from "node:fs"; +import { createServer, get } from "node:https"; +import type { AddressInfo } from "node:net"; +import { join } from "node:path"; +import { + resolveTlsConfig, + TLS_CERT_PATH_ENV, + TLS_KEY_PATH_ENV, + webScheme, +} from "../../../../server/tls.js"; +import { + createSelfSignedTls, + type SelfSignedTlsFixture, +} from "../../fixtures/selfSignedTls.js"; + +let fixture: SelfSignedTlsFixture; + +beforeAll(() => { + fixture = createSelfSignedTls(); +}); + +afterAll(() => { + fixture.cleanup(); +}); + +const env = (key?: string, cert?: string): NodeJS.ProcessEnv => ({ + ...(key !== undefined ? { [TLS_KEY_PATH_ENV]: key } : {}), + ...(cert !== undefined ? { [TLS_CERT_PATH_ENV]: cert } : {}), +}); + +describe("resolveTlsConfig", () => { + it("returns undefined when neither variable is set", () => { + expect(resolveTlsConfig(env())).toBeUndefined(); + }); + + it("treats blank values as unset", () => { + expect(resolveTlsConfig(env("", " "))).toBeUndefined(); + }); + + it("reads process.env by default", () => { + // The suite's own environment carries neither variable. + expect(process.env[TLS_KEY_PATH_ENV]).toBeUndefined(); + expect(process.env[TLS_CERT_PATH_ENV]).toBeUndefined(); + expect(resolveTlsConfig()).toBeUndefined(); + }); + + it("refuses a key path without a cert path, naming both variables", () => { + expect(() => resolveTlsConfig(env(fixture.keyPath))).toThrow( + `TLS needs both ${TLS_KEY_PATH_ENV} and ${TLS_CERT_PATH_ENV}; ${TLS_CERT_PATH_ENV} is not set.`, + ); + }); + + it("refuses a cert path without a key path, naming both variables", () => { + expect(() => resolveTlsConfig(env(undefined, fixture.certPath))).toThrow( + `TLS needs both ${TLS_KEY_PATH_ENV} and ${TLS_CERT_PATH_ENV}; ${TLS_KEY_PATH_ENV} is not set.`, + ); + }); + + it("counts a blank partner as unset rather than enabling TLS", () => { + expect(() => resolveTlsConfig(env(fixture.keyPath, " "))).toThrow( + `${TLS_CERT_PATH_ENV} is not set`, + ); + }); + + it("reads both PEM files when both are set", () => { + expect( + resolveTlsConfig(env(` ${fixture.keyPath} `, fixture.certPath)), + ).toEqual({ + key: readFileSync(fixture.keyPath, "utf-8"), + cert: readFileSync(fixture.certPath, "utf-8"), + }); + }); + + it("produces options an https listener serves with", async () => { + const tls = resolveTlsConfig(env(fixture.keyPath, fixture.certPath)); + const server = createServer(tls!, (_req, res) => res.end("ok")); + await new Promise((r) => server.listen(0, "127.0.0.1", r)); + try { + const { port } = server.address() as AddressInfo; + const body = await new Promise((resolve, reject) => { + // Trust exactly the fixture's certificate — verification stays on. + get({ host: "127.0.0.1", port, path: "/", ca: tls!.cert }, (res) => { + let data = ""; + res.on("data", (c: Buffer) => (data += c.toString())); + res.on("end", () => resolve(data)); + }).on("error", reject); + }); + expect(body).toBe("ok"); + } finally { + await new Promise((r) => server.close(() => r())); + } + }); + + it("names the unreadable key path and its error code, not a raw ENOENT", () => { + const missing = join(fixture.dir, "nope-key.pem"); + let message = ""; + try { + resolveTlsConfig(env(missing, fixture.certPath)); + } catch (err) { + message = (err as Error).message; + } + expect(message).toBe( + `Cannot read ${TLS_KEY_PATH_ENV}="${missing}" (ENOENT): check that the file exists and is readable.`, + ); + }); + + it("names the unreadable cert path", () => { + // A directory is not readable as a file (EISDIR). + expect(() => resolveTlsConfig(env(fixture.keyPath, fixture.dir))).toThrow( + `Cannot read ${TLS_CERT_PATH_ENV}="${fixture.dir}" (EISDIR)`, + ); + }); + + it("refuses a file that is not PEM", () => { + const junk = join(fixture.dir, "junk.pem"); + writeFileSync(junk, "not a certificate\n"); + expect(() => resolveTlsConfig(env(fixture.keyPath, junk))).toThrow( + `${TLS_KEY_PATH_ENV}="${fixture.keyPath}" and ${TLS_CERT_PATH_ENV}="${junk}" are not a usable PEM key/certificate pair (`, + ); + }); + + it("refuses a key that does not match the certificate", () => { + const otherKey = join(fixture.dir, "other-key.pem"); + const { privateKey } = generateKeyPairSync("rsa", { modulusLength: 2048 }); + writeFileSync( + otherKey, + privateKey.export({ type: "pkcs8", format: "pem" }) as string, + ); + expect(() => resolveTlsConfig(env(otherKey, fixture.certPath))).toThrow( + /are not a usable PEM key\/certificate pair/, + ); + }); +}); + +describe("webScheme", () => { + it("is https with a TLS config and http without", () => { + expect(webScheme({ key: "k", cert: "c" })).toBe("https"); + expect(webScheme(undefined)).toBe("http"); + }); +}); diff --git a/clients/web/src/test/integration/server/web-server-config.test.ts b/clients/web/src/test/integration/server/web-server-config.test.ts index 8682dd42b..9262ab0e1 100644 --- a/clients/web/src/test/integration/server/web-server-config.test.ts +++ b/clients/web/src/test/integration/server/web-server-config.test.ts @@ -1,13 +1,29 @@ -import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; +import { + describe, + it, + expect, + beforeAll, + afterAll, + beforeEach, + afterEach, + vi, +} from "vitest"; +import { readFileSync } from "node:fs"; import { buildWebServerConfig, buildWebServerConfigFromEnv, defaultAllowedOrigins, + portInUseMessage, printServerBanner, webServerConfigToInitialPayload, type WebServerConfig, } from "../../../../server/web-server-config.js"; import { DEFAULT_SANDBOX_PORT } from "../../../../server/sandbox-controller.js"; +import { TLS_CERT_PATH_ENV, TLS_KEY_PATH_ENV } from "../../../../server/tls.js"; +import { + createSelfSignedTls, + type SelfSignedTlsFixture, +} from "../../fixtures/selfSignedTls.js"; import { API_SERVER_ENV_VARS, LEGACY_AUTH_TOKEN_ENV, @@ -32,8 +48,25 @@ const MUTATED_ENV_KEYS = [ "SERVER_PORT", "MCP_LOG_FILE", "MCP_AUTO_OPEN_ENABLED", + TLS_KEY_PATH_ENV, + TLS_CERT_PATH_ENV, ] as const; +let tlsFixture: SelfSignedTlsFixture; + +beforeAll(() => { + tlsFixture = createSelfSignedTls(); +}); + +afterAll(() => { + tlsFixture.cleanup(); +}); + +const tlsPair = () => ({ + key: readFileSync(tlsFixture.keyPath, "utf-8"), + cert: readFileSync(tlsFixture.certPath, "utf-8"), +}); + const baseConfig = (): WebServerConfig => ({ port: 6274, hostname: "127.0.0.1", @@ -554,6 +587,44 @@ describe("defaultAllowedOrigins", () => { "http://[fe80::1]:6274", ]); }); + + describe("with the https scheme (TLS, #2446)", () => { + it("emits https loopback origins for a loopback bind", () => { + expect(defaultAllowedOrigins("localhost", 6274, "https")).toEqual([ + "https://localhost:6274", + "https://127.0.0.1:6274", + "https://[::1]:6274", + ]); + }); + + it("emits https loopback and wildcard origins for an all-interfaces bind", () => { + expect(defaultAllowedOrigins("0.0.0.0", 6274, "https")).toEqual([ + "https://localhost:6274", + "https://127.0.0.1:6274", + "https://[::1]:6274", + "https://0.0.0.0:6274", + "https://[::]:6274", + ]); + }); + + it("emits the single https origin for an explicit host", () => { + expect(defaultAllowedOrigins("inspector.test", 6274, "https")).toEqual([ + "https://inspector.test:6274", + ]); + }); + + it("drops https's default port 443, but not 80", () => { + expect(defaultAllowedOrigins("inspector.test", 443, "https")).toEqual([ + "https://inspector.test", + ]); + expect(defaultAllowedOrigins("inspector.test", 80, "https")).toEqual([ + "https://inspector.test:80", + ]); + expect(defaultAllowedOrigins("inspector.test", 443)).toEqual([ + "http://inspector.test:443", + ]); + }); + }); }); describe("buildWebServerConfig", () => { @@ -606,6 +677,41 @@ describe("buildWebServerConfig", () => { ]); }); + it("leaves TLS off and the origins http when neither TLS variable is set", () => { + const cfg = buildWebServerConfig(); + expect(cfg.tls).toBeUndefined(); + expect("tls" in cfg).toBe(false); + expect(cfg.allowedOrigins.every((o) => o.startsWith("http://"))).toBe(true); + }); + + it("reads the TLS pair and derives https origins when both are set", () => { + process.env[TLS_KEY_PATH_ENV] = tlsFixture.keyPath; + process.env[TLS_CERT_PATH_ENV] = tlsFixture.certPath; + const cfg = buildWebServerConfig(); + expect(cfg.tls).toEqual(tlsPair()); + expect(cfg.allowedOrigins).toEqual([ + "https://localhost:6274", + "https://127.0.0.1:6274", + "https://[::1]:6274", + ]); + }); + + it("keeps an explicit ALLOWED_ORIGINS as-is under TLS", () => { + process.env[TLS_KEY_PATH_ENV] = tlsFixture.keyPath; + process.env[TLS_CERT_PATH_ENV] = tlsFixture.certPath; + process.env.ALLOWED_ORIGINS = "https://inspector.test:6274"; + expect(buildWebServerConfig().allowedOrigins).toEqual([ + "https://inspector.test:6274", + ]); + }); + + it("fails startup on a half-configured TLS pair rather than serving http", () => { + process.env[TLS_KEY_PATH_ENV] = tlsFixture.keyPath; + expect(() => buildWebServerConfig()).toThrow( + `TLS needs both ${TLS_KEY_PATH_ENV} and ${TLS_CERT_PATH_ENV}`, + ); + }); + it("preserves remote transport initialMcpConfig", () => { const initialMcpConfig = { type: "streamable-http" as const, @@ -779,6 +885,19 @@ describe("printServerBanner", () => { expect(defaultAllowedOrigins(cfg.hostname, 6274)).toContain(url); }); + it("prints an https URL when TLS is configured", () => { + const cfg = { ...baseConfig(), tls: { key: "k", cert: "c" } }; + const url = printServerBanner(cfg, 6274, "secret", undefined); + expect(url).toBe( + `https://127.0.0.1:6274?${API_SERVER_ENV_VARS.AUTH_TOKEN}=secret`, + ); + expect( + defaultAllowedOrigins(cfg.hostname, 6274, "https").some((o) => + url.startsWith(`${o}?`), + ), + ).toBe(true); + }); + it("prints the sandbox URL when provided", () => { printServerBanner(baseConfig(), 6274, "tok", "http://sandbox:9999/sandbox"); expect( @@ -833,6 +952,25 @@ describe("printServerBanner", () => { }); }); +describe("portInUseMessage", () => { + it("names the http URL without TLS", () => { + expect(portInUseMessage(baseConfig())).toBe( + "MCP Inspector PORT IS IN USE at http://127.0.0.1:6274", + ); + }); + + it("names the https URL with TLS, bracketing an IPv6 host", () => { + const cfg = { + ...baseConfig(), + hostname: "::1", + tls: { key: "k", cert: "c" }, + }; + expect(portInUseMessage(cfg)).toBe( + "MCP Inspector PORT IS IN USE at https://[::1]:6274", + ); + }); +}); + // Minimal console.log capture so banner-format assertions don't pollute stdout. function vitestSpyOnConsoleLog(): { lines: string[]; diff --git a/clients/web/vite.config.ts b/clients/web/vite.config.ts index 3d22adb3d..b6e0b53da 100644 --- a/clients/web/vite.config.ts +++ b/clients/web/vite.config.ts @@ -189,6 +189,10 @@ export default defineConfig(({ command }) => { // HOST="" is Node's all-interfaces address, the one value this guards. host: devConfig?.hostname ?? (process.env.HOST?.trim() || "localhost"), strictPort: true, + // https when both INSPECTOR_TLS_* paths are set (#2446) — the pair + // `devConfig` already read and validated, so dev and the prod listener + // agree. Undefined (plain http) for build/vitest, which never bind. + https: devConfig?.tls, fs: { allow: [path.resolve(dirname, "../..")], }, diff --git a/docs/environment-variables.md b/docs/environment-variables.md index a4d7feb09..9387dff4f 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -19,7 +19,16 @@ These guard the web backend, which spawns processes on request. Read [Host bindi | `DANGEROUSLY_OMIT_AUTH` | web | unset | Disables the API token entirely when set to `true` or `1` (trimmed, case-insensitive). Any other value — including `false`, `0` and empty — keeps auth on. | | `HOST` | web, CLI | `127.0.0.1` | Address the web server binds. An all-interfaces host (`0.0.0.0`, `::`, an empty string, and equivalent spellings) is **refused** unless `DANGEROUSLY_BIND_ALL_INTERFACES` is enabled. The CLI reads it only to build its deep link. | | `DANGEROUSLY_BIND_ALL_INTERFACES` | web | off | Opts in to an all-interfaces `HOST`. Only `true` or `1` (case-insensitive) enable it, so `false` reads as off. The Docker image sets it. | -| `ALLOWED_ORIGINS` | web | derived from `HOST` | Comma-separated origins allowed to call the API. Unset, the list follows `HOST` at `CLIENT_PORT`: the loopback origins for a loopback host, the loopback origins plus `http://0.0.0.0` and `http://[::]` for an all-interfaces bind, and otherwise only the configured host's own origin (so binding a LAN address does **not** also allow `localhost`). **Replaces** the default list rather than adding to it, so list every form you browse from. Each entry must include the scheme (`http://localhost:6274`). The same list is the MCP Apps sandbox proxy's embedder allow-list (its `frame-ancestors` header and its referrer check), so a public Inspector origin must be listed here for the Apps tab to render. | +| `ALLOWED_ORIGINS` | web | derived from `HOST` | Comma-separated origins allowed to call the API. Unset, the list follows `HOST` at `CLIENT_PORT`: the loopback origins for a loopback host, the loopback origins plus `http://0.0.0.0` and `http://[::]` for an all-interfaces bind, and otherwise only the configured host's own origin (so binding a LAN address does **not** also allow `localhost`). The default origins are `https://` when [TLS](#tls-for-the-web-ui) is on. **Replaces** the default list rather than adding to it, so list every form you browse from. Each entry must include the scheme (`http://localhost:6274`). The same list is the MCP Apps sandbox proxy's embedder allow-list (its `frame-ancestors` header and its referrer check), so a public Inspector origin must be listed here for the Apps tab to render. | + +## TLS for the web UI + +Serve the web UI over `https`. Both must be set, or neither: one without the other fails at startup, as does a file that cannot be read or a key and certificate that are not a usable pair. With TLS on, the launch banner prints `https://` and the default `ALLOWED_ORIGINS` uses `https://` origins. Covers the UI listener only, not the MCP Apps sandbox or app-origin listeners, and not outbound connections (see [Node.js variables](#nodejs-variables)). See [Serving the web UI over HTTPS](./tls.md) for an mkcert walkthrough. + +| Variable | Read by | Default | Effect | +| ------------------------- | ------- | ------- | ----------------------------------------------------------------------------------------- | +| `INSPECTOR_TLS_KEY_PATH` | web | unset | PEM private key for the UI listener, in prod and in Vite development. Empty counts as unset. | +| `INSPECTOR_TLS_CERT_PATH` | web | unset | PEM certificate (or chain) matching `INSPECTOR_TLS_KEY_PATH`. Empty counts as unset. | ## Ports diff --git a/docs/tls.md b/docs/tls.md new file mode 100644 index 000000000..98baccf82 --- /dev/null +++ b/docs/tls.md @@ -0,0 +1,71 @@ +# Serving the web UI over HTTPS + +The web UI is served over plain `http` by default. Set **both** of these and it is served over `https` instead: + +| Variable | Effect | +| ------------------------- | ------------------------------------------- | +| `INSPECTOR_TLS_KEY_PATH` | Path to the PEM private key. | +| `INSPECTOR_TLS_CERT_PATH` | Path to the PEM certificate (or its chain). | + +They work the same way for every web entry point: the production listener (`npm run web`, `mcp-inspector --web`) and Vite development (`npm run web:dev`, `cd clients/web && npm run dev`). With TLS on: + +- the launch banner prints an `https://` URL; +- the default `ALLOWED_ORIGINS` list uses `https://` origins (loopback, all-interfaces, or the configured host, exactly as for `http`); +- in Vite development the UI is served over HTTP/2, falling back to HTTP/1.1. + +**Startup fails instead of falling back to `http`** when only one of the two variables is set, when a file cannot be read, or when the key and certificate are not a usable pair. Each error names the variable and path to fix. A blank value counts as unset. + +## A trusted local certificate with mkcert + +[mkcert](https://github.com/FiloSottile/mkcert) creates a local certificate authority, installs it in your system and browser trust stores, and issues certificates from it, so the browser trusts the UI without an exception: + +```sh +mkcert -install +mkcert localhost 127.0.0.1 ::1 + +export INSPECTOR_TLS_KEY_PATH="$PWD/localhost+2-key.pem" +export INSPECTOR_TLS_CERT_PATH="$PWD/localhost+2.pem" + +npm install +npm run build +npm run web +``` + +The certificate covers `127.0.0.1` because that is the default `HOST`, and therefore the host the banner prints. + +Check the listener independently of the browser, with verification left on: + +```sh +curl --cacert "$(mkcert -CAROOT)/rootCA.pem" https://localhost:6274/ +``` + +## A custom local hostname + +To browse the UI at a name such as `inspector.test`, point the name at loopback, issue a certificate that covers it, and add its `https` origin to the allow-list: + +```sh +echo "127.0.0.1 inspector.test" | sudo tee -a /etc/hosts +mkcert inspector.test localhost 127.0.0.1 ::1 + +export INSPECTOR_TLS_KEY_PATH="$PWD/inspector.test+3-key.pem" +export INSPECTOR_TLS_CERT_PATH="$PWD/inspector.test+3.pem" +export ALLOWED_ORIGINS="https://inspector.test:6274,https://localhost:6274,https://127.0.0.1:6274" +``` + +The default allow-list cannot include the name: it is built from the bind address (`127.0.0.1`), and a browser at `https://inspector.test:6274` sends that as its `Origin`, so every `/api` call is refused with `403` until it is listed. `ALLOWED_ORIGINS` **replaces** the default list, so name every origin you browse from, each with its `https://` scheme. An `http://` entry never matches a page loaded over `https`. See [Environment variables](./environment-variables.md#authentication-and-network-exposure). + +## Outbound connections are separate + +These variables cover the UI listener only. The Inspector connects to MCP servers and OAuth endpoints **from its Node backend**, not from the browser, so trusting a certificate in the browser does nothing for those connections. When a server you are testing has a certificate from a private CA (mkcert's included), give Node that CA: + +```sh +export NODE_EXTRA_CA_CERTS="$(mkcert -CAROOT)/rootCA.pem" +``` + +Node reads it once at startup and keeps verifying every other connection against its bundled CAs. See [Connecting to a server with a self-signed certificate](./environment-variables.md#connecting-to-a-server-with-a-self-signed-certificate). + +⚠️ **Do not use `NODE_TLS_REJECT_UNAUTHORIZED=0` to get past a certificate error.** It turns off verification for every TLS connection the process makes, including every OAuth authorization server and token endpoint, not only the one you are testing. A connection that should be refused as an impersonation is then accepted, and the tokens it is handed go with it. `NODE_EXTRA_CA_CERTS` is the fix. Disabling verification only hides the problem. + +## The MCP Apps listeners stay on `http` + +The MCP Apps sandbox (`MCP_SANDBOX_PORT`) and app-origin (`MCP_APP_ORIGIN_PORT`) listeners are not covered by these variables and keep serving plain `http`. On loopback that is still loadable from an `https` page, because the Mixed Content spec treats `http://127.0.0.1` and `http://localhost` as potentially trustworthy. On any other address the browser blocks it as mixed content. There, put an `https` reverse proxy in front of each and advertise it with `MCP_SANDBOX_FULL_ADDRESS` / `MCP_APP_ORIGIN_FULL_ADDRESS` (see [Public addresses](./environment-variables.md#public-addresses-reverse-proxy)). The Inspector warns at startup when one of those is plain `http` while `ALLOWED_ORIGINS` lists an `https` origin. From 936505977a350b603fb7d6a72d82c9eae229ee3d Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 20:42:16 -0400 Subject: [PATCH 07/44] fix(subscriptions): stop retrying a subscriptions/listen refused over HTTP A modern-era subscriptions/listen refused with a bare 403 (or a 400, 404 or 405) was handed to the reconnect machinery as though it were a drop, re-sent at the backoff until the eight-attempt cap and never reported. The refused URI's rollback also cleared the badge's `active` gate, so the refusal showed nowhere. Classify the rejection by the HTTP status on the SDK's generic ClientHttpNotImplemented error (auth-coded 401/403s never carry that code), settle the stream on a new "refused" status carrying the status, and surface it in the web Subscriptions section and the TUI pane whatever the subscription count. Transient statuses (5xx, 408, 429) keep retrying. Adds a refuseSubscriptionsListen test-server option and the subscriptions-listen-refused-http showcase. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../tui/__tests__/SubscriptionsTab.test.tsx | 24 ++ .../tui/src/components/SubscriptionsTab.tsx | 16 +- .../SubscriptionStreamBadge.stories.tsx | 8 + .../SubscriptionStreamBadge.test.tsx | 19 ++ .../subscriptionStreamUtils.ts | 5 + .../ResourceControls.test.tsx | 45 ++++ .../ResourceControls/ResourceControls.tsx | 40 ++- .../test/core/mcp/subscriptionRefusal.test.ts | 86 +++++++ .../inspectorClient-subscriptions-era.test.ts | 235 +++++++++++++++++- core/mcp/inspectorClient.ts | 55 +++- core/mcp/subscriptionRefusal.ts | 67 +++++ core/mcp/types.ts | 21 +- docs/test-servers.md | 11 + .../subscriptions-listen-refused-http.json | 16 ++ test-servers/src/composable-test-server.ts | 8 + test-servers/src/load-config.ts | 1 + test-servers/src/test-server-http.ts | 34 +++ 17 files changed, 676 insertions(+), 15 deletions(-) create mode 100644 clients/web/src/test/core/mcp/subscriptionRefusal.test.ts create mode 100644 core/mcp/subscriptionRefusal.ts create mode 100644 test-servers/configs/subscriptions-listen-refused-http.json diff --git a/clients/tui/__tests__/SubscriptionsTab.test.tsx b/clients/tui/__tests__/SubscriptionsTab.test.tsx index 236551a87..df4b9c181 100644 --- a/clients/tui/__tests__/SubscriptionsTab.test.tsx +++ b/clients/tui/__tests__/SubscriptionsTab.test.tsx @@ -274,6 +274,30 @@ describe("SubscriptionsTab", () => { expect(frame).not.toContain("last updated"); }); + // The refused subscribe was rolled back, so nothing is subscribed and the + // stream is inactive — the refusal must show anyway (#2722). + it("shows an HTTP refusal with nothing subscribed", () => { + const { lastFrame } = renderTab({ + width: 260, + streamState: { + active: false, + status: "refused", + honoredUris: [], + refusal: { status: 403, statusText: "Forbidden" }, + }, + }); + const frame = lastFrame() ?? ""; + expect(frame).toContain("Listen stream: refused by the server"); + expect(frame).toContain( + "The server refused subscriptions/listen (HTTP 403 Forbidden). Not retrying", + ); + }); + + it("shows no stream line for an inactive stream that was not refused", () => { + const { lastFrame } = renderTab(); + expect(lastFrame()).not.toContain("Listen stream:"); + }); + it("ignores keys while a modal is open", async () => { const { stdin, client } = renderTab({ modalOpen: true }); stdin.write("\r"); diff --git a/clients/tui/src/components/SubscriptionsTab.tsx b/clients/tui/src/components/SubscriptionsTab.tsx index 6742df559..d703c3477 100644 --- a/clients/tui/src/components/SubscriptionsTab.tsx +++ b/clients/tui/src/components/SubscriptionsTab.tsx @@ -27,6 +27,7 @@ import { AuthRecoveryRequiredError, findNestedAuthError, } from "@inspector/core/auth/challenge.js"; +import { subscriptionListenRefusalMessage } from "@inspector/core/mcp/subscriptionRefusal.js"; import { useSelectableList } from "../hooks/useSelectableList.js"; import { errorMessage } from "../utils/errorText.js"; import { @@ -47,6 +48,7 @@ const STREAM_STATUS_TEXT: Record< text: "closed without acknowledging (server answered listen with a result)", color: "red", }, + refused: { text: "refused by the server", color: "red" }, }; interface SubscriptionsTabProps { @@ -166,6 +168,8 @@ export function SubscriptionsTab({ }, [selectedIndex]); const streamText = STREAM_STATUS_TEXT[streamState.status]; + const streamRefusal = + streamState.status === "refused" ? streamState.refusal : undefined; // On the modern era a server may honor only part of the filter; say so for // the selected URI once the stream has been acknowledged. const notHonored = @@ -284,7 +288,10 @@ export function SubscriptionsTab({ )} )} - {streamState.active && ( + {/* A refusal is shown whatever `active` says: the refused subscribe + was rolled back, so there is no live subscription to gate on + (#2722). */} + {(streamState.active || streamRefusal) && ( Listen stream:{" "} @@ -292,6 +299,13 @@ export function SubscriptionsTab({ )} + {streamRefusal && ( + + + {subscriptionListenRefusalMessage(streamRefusal)} + + + )} {error && ( {error} diff --git a/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.stories.tsx b/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.stories.tsx index 42f8bf352..258043e63 100644 --- a/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.stories.tsx +++ b/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.stories.tsx @@ -33,3 +33,11 @@ export const Ended: Story = { export const NeverAcknowledged: Story = { args: { status: "never-acknowledged" }, }; + +/** + * The server refused `subscriptions/listen` at the HTTP layer — a bare 403, or a + * 400/404/405 — so the stream never opened (#2722). Not retried. + */ +export const Refused: Story = { + args: { status: "refused" }, +}; diff --git a/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.test.tsx b/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.test.tsx index 663d4eecb..6562d4edf 100644 --- a/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.test.tsx +++ b/clients/web/src/components/elements/SubscriptionStreamBadge/SubscriptionStreamBadge.test.tsx @@ -26,6 +26,19 @@ describe("subscriptionStreamPresentation", () => { color: "orange", label: "Not acknowledged", }); + expect(subscriptionStreamPresentation("refused")).toMatchObject({ + color: "red", + label: "Refused", + }); + }); + + // The refused tooltip has to say nothing is retried — the silent retry run is + // what #2722 is about. The HTTP status itself is per-event, so it lives in the + // panel notice rather than this static copy. + it("says a refused listen is not retried", () => { + expect(subscriptionStreamPresentation("refused").tooltip).toMatch( + /refused the subscriptions\/listen request over HTTP.*Not retrying/, + ); }); // The never-acknowledged tooltip must carry the *reason*, not just the generic @@ -43,6 +56,7 @@ describe("subscriptionStreamPresentation", () => { "reconnecting", "ended", "never-acknowledged", + "refused", ] as const) { expect(subscriptionStreamPresentation(status).tooltip).toContain( "subscriptions/listen stream", @@ -71,4 +85,9 @@ describe("SubscriptionStreamBadge", () => { renderWithMantine(); expect(screen.getByText("Not acknowledged")).toBeInTheDocument(); }); + + it("renders a labelled refused badge", () => { + renderWithMantine(); + expect(screen.getByText("Refused")).toBeInTheDocument(); + }); }); diff --git a/clients/web/src/components/elements/SubscriptionStreamBadge/subscriptionStreamUtils.ts b/clients/web/src/components/elements/SubscriptionStreamBadge/subscriptionStreamUtils.ts index 233d8295a..4ea629aac 100644 --- a/clients/web/src/components/elements/SubscriptionStreamBadge/subscriptionStreamUtils.ts +++ b/clients/web/src/components/elements/SubscriptionStreamBadge/subscriptionStreamUtils.ts @@ -45,6 +45,11 @@ const PRESENTATION: Record< label: "Not acknowledged", tooltip: `${STREAM_INTRO} ${NEVER_ACKNOWLEDGED_SUBSCRIPTION_MESSAGE} Re-subscribe to try again.`, }, + refused: { + color: "red", + label: "Refused", + tooltip: `${STREAM_INTRO} The server refused the subscriptions/listen request over HTTP, so the stream never opened. Not retrying — the server would answer the same way again. The HTTP status is in the Subscriptions panel. Re-subscribe to try again.`, + }, }; /** diff --git a/clients/web/src/components/groups/ResourceControls/ResourceControls.test.tsx b/clients/web/src/components/groups/ResourceControls/ResourceControls.test.tsx index af885ba64..d36887b32 100644 --- a/clients/web/src/components/groups/ResourceControls/ResourceControls.test.tsx +++ b/clients/web/src/components/groups/ResourceControls/ResourceControls.test.tsx @@ -7,6 +7,7 @@ import type { } from "@modelcontextprotocol/client"; import type { InspectorResourceSubscription } from "@inspector/core/mcp/types.js"; import { NEVER_ACKNOWLEDGED_SUBSCRIPTION_MESSAGE } from "@inspector/core/mcp/subscriptionAck.js"; +import { subscriptionListenRefusalMessage } from "@inspector/core/mcp/subscriptionRefusal.js"; import { renderWithMantine, screen } from "../../../test/renderWithMantine"; import { ResourceControls, @@ -415,6 +416,50 @@ describe("ResourceControls", () => { ).toBeInTheDocument(); }); + // The refused subscribe was rolled back, so there is no subscription left + // to gate on — the refusal has to show regardless, and the section has to + // open to show it (#2722). + it("spells out an HTTP refusal with no subscription left", async () => { + const user = userEvent.setup(); + const refusal = { status: 403, statusText: "Forbidden" }; + renderWithMantine( + , + ); + expect(screen.getByText("Subscriptions (0)")).toBeInTheDocument(); + expect(screen.getByText("Refused")).toBeInTheDocument(); + + await user.click(screen.getByText("Subscriptions (0)")); + expect( + await screen.findByText(subscriptionListenRefusalMessage(refusal)), + ).toBeVisible(); + }); + + it("shows no refusal on the legacy era", () => { + renderWithMantine( + , + ); + expect(screen.queryByText("Refused")).not.toBeInTheDocument(); + }); + it("shows no such notice while the stream is healthy", () => { renderWithMantine( 0 - ? subscriptionStreamState.status + streamRefusal || + (isModernEra(protocolEra) && + subscriptionStreamState?.active === true && + filteredSubscriptions.length > 0) + ? subscriptionStreamState?.status : undefined; // Subscriptions are only meaningful when the server advertises the @@ -244,7 +263,9 @@ export function ResourceControls({ const sectionItemCounts: Record = { resources: filteredResources.length, templates: filteredTemplates.length, - subscriptions: filteredSubscriptions.length, + // The refusal notice counts as content, so the section can be opened to + // read it with no subscription listed (#2722). + subscriptions: filteredSubscriptions.length + (streamRefusal ? 1 : 0), }; const visibleOpenSections = openSections.filter( (section) => @@ -386,10 +407,10 @@ export function ResourceControls({ value="subscriptions" flex={sectionFlex( visibleOpenSections.includes("subscriptions"), - filteredSubscriptions.length, + sectionItemCounts.subscriptions, )} > - + {formatSectionCount( @@ -409,6 +430,11 @@ export function ResourceControls({ {NEVER_ACKNOWLEDGED_SUBSCRIPTION_MESSAGE} )} + {streamRefusal && ( + + {subscriptionListenRefusalMessage(streamRefusal)} + + )} {filteredSubscriptions.map(({ subscription: sub, key }) => ( { + const postError = (status: number, statusText?: string) => + new SdkHttpError( + SdkErrorCode.ClientHttpNotImplemented, + "Error POSTing to endpoint: whatever", + { status, ...(statusText ? { statusText } : {}) }, + ); + + it.each([400, 403, 404, 405])("classifies HTTP %i as a refusal", (status) => { + expect(subscriptionListenRefusal(postError(status, "Nope"))).toEqual({ + status, + statusText: "Nope", + }); + }); + + it("omits an absent status text", () => { + expect(subscriptionListenRefusal(postError(403))).toEqual({ status: 403 }); + }); + + // The statuses that mean "try again later" stay with the retry machinery. + it.each([408, 429, 500, 502, 503, 504])( + "leaves HTTP %i retryable", + (status) => { + expect(subscriptionListenRefusal(postError(status))).toBeUndefined(); + }, + ); + + // The status alone is not enough: a 403 that went through the auth flow and + // still failed carries an auth code, and belongs to that flow, not to this. + it("ignores an auth-coded 403", () => { + expect( + subscriptionListenRefusal( + new SdkHttpError(SdkErrorCode.ClientHttpForbidden, "denied", { + status: 403, + }), + ), + ).toBeUndefined(); + }); + + it("ignores anything that is not an SdkHttpError", () => { + expect( + subscriptionListenRefusal( + new SdkError(SdkErrorCode.ConnectionClosed, "closed"), + ), + ).toBeUndefined(); + expect(subscriptionListenRefusal(new Error("socket hang up"))).toBe( + undefined, + ); + expect(subscriptionListenRefusal(undefined)).toBeUndefined(); + }); +}); + +describe("subscriptionListenRefusalMessage (#2722)", () => { + it("names the method, the HTTP status, and that nothing is retried", () => { + const message = subscriptionListenRefusalMessage({ + status: 403, + statusText: "Forbidden", + }); + expect(message).toContain("subscriptions/listen"); + expect(message).toContain("HTTP 403 Forbidden"); + expect(message).toContain("Not retrying"); + }); + + it("reads without a status text", () => { + expect(subscriptionListenRefusalMessage({ status: 405 })).toContain( + "(HTTP 405)", + ); + }); +}); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts index e724d8d5c..2d95734e0 100644 --- a/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts +++ b/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts @@ -5,7 +5,11 @@ import { eraToVersionNegotiation, MODERN_PROTOCOL_VERSION, } from "@inspector/core/mcp/types.js"; -import type { McpSubscription } from "@modelcontextprotocol/client"; +import { + SdkErrorCode, + SdkHttpError, + type McpSubscription, +} from "@modelcontextprotocol/client"; import { createTestServerHttp, type TestServerHttp, @@ -1231,6 +1235,235 @@ describe("resource subscriptions era fork (#1630)", () => { }); }); + /** + * A server (or a proxy in front of it) that refuses `subscriptions/listen` + * with a bare HTTP status (#2722). A 403 with no `WWW-Authenticate`, or a + * 400/404/405, is a deterministic answer: the Inspector must settle on the + * first one and say what it was, not re-send the listen until the + * eight-attempt cap. Transient statuses keep the retry. + * + * Against the real test server for the same reason as the #2097 block: the + * classification reads the SDK's error code and HTTP status, so an SDK that + * surfaces the refusal differently fails here rather than silently + * reverting to the retry loop. + */ + describe("server refuses the listen over HTTP (#2722)", () => { + /** Give a reconnect timer room to fire had one been armed (base 500ms). */ + const settle = () => new Promise((resolve) => setTimeout(resolve, 1_200)); + const listensSent = (messages: MessageEntry[]) => + methodsSent(messages).filter((m) => m === "subscriptions/listen"); + + it("settles a bare 403 on the first answer, with the status, after the rollback", async () => { + // List-change opt-ins left on (the default), so the filter stays + // non-empty after the refused URI is rolled back — the shape the smoke + // observed, where that is what kept the retry loop alive. + const started = await startServer({ refuseSubscriptionsListen: 403 }); + const { connected, messages } = await connect(started.url, "modern"); + messages.length = 0; + + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(/Forbidden/); + await settle(); + + expect(listensSent(messages)).toHaveLength(1); + expect(connected.getSubscribedResources()).toEqual([]); + expect(connected.getResourceSubscriptionStreamState()).toEqual({ + active: false, + status: "refused", + honoredUris: [], + refusal: { status: 403, statusText: "Forbidden" }, + }); + expect(internals(connected).modernReconnectAttempts).toBe(0); + }); + + it("reports the refusal even when the rollback empties the filter", async () => { + // No list-change opt-in, so nothing would be retried anyway — but the + // refusal must still be reported rather than collapsing to inactive. + const started = await startServer({ refuseSubscriptionsListen: 403 }); + const { connected } = await connect( + started.url, + "modern", + NO_LIST_CHANGED, + ); + + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(); + expect(connected.getResourceSubscriptionStreamState()).toMatchObject({ + active: false, + status: "refused", + refusal: { status: 403 }, + }); + }); + + it("settles the connect-time listen without retrying", async () => { + const started = await startServer({ refuseSubscriptionsListen: 403 }); + const { connected, messages } = await connect(started.url, "modern"); + await settle(); + + expect(listensSent(messages)).toHaveLength(1); + expect(connected.getResourceSubscriptionStreamState()).toMatchObject({ + status: "refused", + refusal: { status: 403 }, + }); + }); + + it.each([400, 404, 405])( + "treats HTTP %i as a refusal too", + async (status) => { + const started = await startServer({ + refuseSubscriptionsListen: status, + }); + const { connected, messages } = await connect(started.url, "modern"); + messages.length = 0; + + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(); + await settle(); + + expect(listensSent(messages)).toHaveLength(1); + expect(connected.getResourceSubscriptionStreamState()).toMatchObject({ + status: "refused", + refusal: { status }, + }); + }, + ); + + /* + * The tests above drive the Node transport, which reports a failed POST + * through `onerror` as well as rejecting it — so the client is already + * `error` by the time the reconnect timer would fire, and the timer bails + * on the terminal status either way. The web client's remote transport + * rejects without `onerror`, which is the path the smoke observed the + * eight-attempt run on. These stub `listen()` with the error that + * transport rebuilds (#2660) to model it: the session stays connected, + * and only the classification stands between the refusal and the retry. + */ + function httpError(status: number, statusText: string): SdkHttpError { + return new SdkHttpError( + SdkErrorCode.ClientHttpNotImplemented, + `Error POSTing to endpoint: ${statusText}`, + { status, statusText }, + ); + } + + function stubListen(int: StreamInternals, error: SdkHttpError) { + let calls = 0; + int.client.listen = () => { + calls += 1; + return Promise.reject(error); + }; + return () => calls; + } + + it("does not retry a refusal while the session stays connected", async () => { + const started = await startServer({}); + const { connected } = await connect(started.url, "modern"); + const calls = stubListen( + internals(connected), + httpError(403, "Forbidden"), + ); + + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(/Forbidden/); + await settle(); + + expect(calls()).toBe(1); + expect(connected.getStatus()).toBe("connected"); + expect(connected.getResourceSubscriptionStreamState()).toEqual({ + active: false, + status: "refused", + honoredUris: [], + refusal: { status: 403, statusText: "Forbidden" }, + }); + }); + + it("keeps retrying a transient status", async () => { + // 503 is "try again later": the reconnect run proceeds, so a second + // listen goes out at the base delay. + const started = await startServer({}); + const { connected } = await connect(started.url, "modern"); + const calls = stubListen( + internals(connected), + httpError(503, "Service Unavailable"), + ); + + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(); + await vi.waitFor( + () => { + expect(calls()).toBeGreaterThanOrEqual(2); + }, + { timeout: 5_000 }, + ); + const state = connected.getResourceSubscriptionStreamState(); + expect(state.status).toBe("reconnecting"); + expect(state.refusal).toBeUndefined(); + }); + + it("ends a reconnect run at the first refusal", async () => { + // An acknowledged stream that dropped and now re-lists against a server + // refusing it: the run stops there rather than counting to the cap. + const started = await startServer({}); + const { connected } = await connect( + started.url, + "modern", + NO_LIST_CHANGED, + ); + await connected.subscribeToResource(RESOURCE_URI); + const int = internals(connected); + const fake = await installFakeSubscription(int); + const calls = stubListen(int, httpError(403, "Forbidden")); + + int.onModernSubscriptionClosed( + fake.sub, + "remote", + int.modernListenGeneration, + ); + await vi.waitFor( + () => { + expect( + connected.getResourceSubscriptionStreamState(), + ).toMatchObject({ + active: true, + status: "refused", + refusal: { status: 403 }, + }); + }, + { timeout: 5_000 }, + ); + await settle(); + expect(calls()).toBe(1); + expect(int.modernReconnectAttempts).toBe(0); + }); + + it("tries again when the user re-subscribes, and clears on disconnect", async () => { + const started = await startServer({ refuseSubscriptionsListen: 403 }); + const { connected, messages } = await connect(started.url, "modern"); + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(); + + messages.length = 0; + await expect( + connected.subscribeToResource(RESOURCE_URI), + ).rejects.toThrow(); + // The refusal is a fact about the last attempt, not a latch. + expect(listensSent(messages)).toHaveLength(1); + + await connected.disconnect(); + expect(connected.getResourceSubscriptionStreamState()).toEqual({ + active: false, + status: "ended", + honoredUris: [], + }); + }); + }); + it("ignores a close callback from a superseded generation", async () => { const started = await startServer({}); const { connected } = await connect(started.url, "modern"); diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index ea9db7a2e..2254df85c 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -53,6 +53,7 @@ import type { InspectorClientOptions, PendingRequestOrigin, ResourceSubscriptionStreamState, + SubscriptionListenRefusal, ExcludedTool, RequestMetadata, InspectorTask, @@ -83,6 +84,10 @@ import { subscriptionFailureMessage, NEVER_ACKNOWLEDGED_SUBSCRIPTION_MESSAGE, } from "./subscriptionAck.js"; +import { + subscriptionListenRefusal, + subscriptionListenRefusalMessage, +} from "./subscriptionRefusal.js"; // Fallback client identity, used ONLY when a caller doesn't pass // `clientIdentity`. Real clients supply their own: the Node clients (CLI, TUI) // read the single-source version from the root package.json via @@ -698,6 +703,10 @@ export class InspectorClient extends InspectorClientEventTarget { // site to the two failure handlers. Cleared by any user-initiated refresh (a // subscribe/unsubscribe is a fresh attempt, and the server may have changed). private modernNeverAcknowledged = false; + // Set when the last `listen()` was refused at the HTTP layer with a status a + // retry cannot change (#2722) — the sibling of `modernNeverAcknowledged`, + // carried and cleared the same way, holding the HTTP answer the UI reports. + private modernListenRefusal: SubscriptionListenRefusal | null = null; /** Which in-flight task calls own each progress token (#2316). */ private readonly taskProgress = new TaskProgressRouter(); // Abort controller for the in-flight ordinary (non-task) tool call. Aborting @@ -1598,6 +1607,7 @@ export class InspectorClient extends InspectorClientEventTarget { this.clearModernReconnectTimer(); this.modernReconnectAttempts = 0; this.modernNeverAcknowledged = false; + this.modernListenRefusal = null; this.modernSubscription = null; // Announced only once both have moved: a listener that ran between them // would see an empty set with an `active` stream — the combination this @@ -5610,6 +5620,7 @@ export class InspectorClient extends InspectorClientEventTarget { this.clearModernReconnectTimer(); this.modernReconnectAttempts = 0; this.modernNeverAcknowledged = false; + this.modernListenRefusal = null; } const generation = ++this.modernListenGeneration; @@ -5643,11 +5654,12 @@ export class InspectorClient extends InspectorClientEventTarget { // already cleared the flag for its own attempt, and a stale caller writing // to it afterwards would make that attempt's *unrelated* failure look // deterministic and end a stream that deserved a retry. - if ( - generation === this.modernListenGeneration && - isNeverAcknowledgedSubscriptionClose(error) - ) { - this.modernNeverAcknowledged = true; + if (generation === this.modernListenGeneration) { + if (isNeverAcknowledgedSubscriptionClose(error)) { + this.modernNeverAcknowledged = true; + } + // The HTTP-refusal sibling (#2722), recorded under the same gate. + this.modernListenRefusal = subscriptionListenRefusal(error) ?? null; } throw error; } @@ -5760,6 +5772,14 @@ export class InspectorClient extends InspectorClientEventTarget { * its error either way — the retry is about the subscriptions, not the call. */ private reconcileModernStreamStateAfterFailedRefresh(): void { + // Ahead of the empty-filter test, unlike the never-acknowledged case below: + // a refused subscribe is rolled back, and on a server with no list-change + // opt-in that rollback empties the filter — reporting it inactive would + // swallow the refusal the same way the retry run did (#2722). + if (this.modernListenRefusal) { + this.endModernStreamRefused(this.modernListenRefusal); + return; + } if (!this.wantsModernStream()) { this.setModernStreamState(INACTIVE_SUBSCRIPTION_STREAM_STATE); return; @@ -5797,6 +5817,24 @@ export class InspectorClient extends InspectorClientEventTarget { }); } + /** + * Settle the stream on an HTTP refusal of the listen (#2722): the + * `"refused"` status carrying the HTTP answer, no reconnect, and a log line + * with the sentence the UI shows. Reported whatever `active` says — see + * `ResourceSubscriptionStreamState`. + */ + private endModernStreamRefused(refusal: SubscriptionListenRefusal): void { + this.clearModernReconnectTimer(); + const message = subscriptionListenRefusalMessage(refusal); + this.logger.warn({ message }, "subscriptions/listen refused over HTTP"); + this.setModernStreamState({ + active: this.modernStreamActive(), + status: "refused", + honoredUris: [], + refusal, + }); + } + /** * Schedule a reconnect re-listen after the current backoff delay (#1630). * `modernReconnectAttempts` reflects the number of *consecutive failed* @@ -5845,6 +5883,13 @@ export class InspectorClient extends InspectorClientEventTarget { this.endModernStreamNeverAcknowledged(); return; } + // Likewise a re-listen the server refused over HTTP (#2722) — the observed + // shape: a stream reconnecting after its URI was rolled back, kept alive by + // the list-change opt-ins, re-sent until the cap. + if (this.modernListenRefusal && !isTerminalStatus(this.status)) { + this.endModernStreamRefused(this.modernListenRefusal); + return; + } this.modernReconnectAttempts += 1; if ( this.modernReconnectAttempts > MODERN_RECONNECT_MAX_ATTEMPTS || diff --git a/core/mcp/subscriptionRefusal.ts b/core/mcp/subscriptionRefusal.ts new file mode 100644 index 000000000..47991ff89 --- /dev/null +++ b/core/mcp/subscriptionRefusal.ts @@ -0,0 +1,67 @@ +import { SdkErrorCode, SdkHttpError } from "@modelcontextprotocol/client"; +import type { SubscriptionListenRefusal } from "./types.js"; + +/** + * Recognizing a modern-era `subscriptions/listen` the server refused at the + * HTTP layer with a status a retry cannot change (#2722). + * + * The sibling of the never-acknowledged close in `subscriptionAck.ts` (#2097): + * both are deterministic answers, and both used to be fed to the reconnect + * machinery as though they were drops — eight re-lists over roughly a minute, + * then a bare "Stream ended" that said nothing about why. That module covers + * a listen the server closed before acknowledging; this one covers a listen the + * server would not accept at all. + * + * The SDK surfaces every non-auth HTTP failure of a POST as an `SdkHttpError` + * with the same code, `ClientHttpNotImplemented`, so the HTTP status in its data + * is the only thing that tells a refusal from a transient failure. The auth + * statuses never arrive with that code: a 401, and a 403 carrying + * `insufficient_scope`, go through the auth flow and surface (if at all) under + * the SDK's auth codes. Requiring the code as well as the status is what keeps + * them out of this predicate. + */ + +/** + * The HTTP statuses that refuse a listen deterministically. A 403 reaching this + * point carries no `insufficient_scope` challenge (see above), so it is a plain + * "forbidden" with no auth flow to run. 400/404/405 say the request or the + * endpoint is wrong, which re-sending the same filter cannot fix. + * + * Everything else stays retryable — notably 5xx, 408 and 429, which are the + * statuses that mean "try again later". + */ +const REFUSAL_STATUSES: ReadonlySet = new Set([400, 403, 404, 405]); + +/** + * The refusal a rejected `listen()` represents, or `undefined` when the failure + * is one a retry may fix (a transport failure, a timeout, a transient HTTP + * status, or anything that is not an `SdkHttpError` at all). + */ +export function subscriptionListenRefusal( + error: unknown, +): SubscriptionListenRefusal | undefined { + if ( + !SdkHttpError.isInstance(error) || + error.code !== SdkErrorCode.ClientHttpNotImplemented || + !REFUSAL_STATUSES.has(error.status) + ) { + return undefined; + } + return { + status: error.status, + ...(error.statusText ? { statusText: error.statusText } : {}), + }; +} + +/** + * The explanation shown to the user, in the UI and in the client log. Lives in + * `core/` so every client says the same thing about the same wire event. + */ +export function subscriptionListenRefusalMessage( + refusal: SubscriptionListenRefusal, +): string { + const status = refusal.statusText + ? `${refusal.status} ${refusal.statusText}` + : String(refusal.status); + return `The server refused subscriptions/listen (HTTP ${status}). Not retrying — the server would answer the same way again.`; +} diff --git a/core/mcp/types.ts b/core/mcp/types.ts index bdd3aa086..3ef720db3 100644 --- a/core/mcp/types.ts +++ b/core/mcp/types.ts @@ -578,13 +578,25 @@ export interface InspectorResourceSubscription { * in the same breath it was opened (#2097). Distinct from `"ended"` because * the condition is deterministic rather than a shutdown of an established * stream: the Inspector does not retry it, and the UI says why. + * - `"refused"` — the server refused the `listen()` request at the HTTP layer + * with a status a retry cannot change (a bare 403, or a 400/404/405), so the + * stream never opened (#2722). Like `"never-acknowledged"` it is deterministic, + * is not retried, and has a status of its own so the UI can say why; the HTTP + * answer is on `ResourceSubscriptionStreamState.refusal`. */ export type ResourceSubscriptionStreamStatus = | "connecting" | "acknowledged" | "reconnecting" | "ended" - | "never-acknowledged"; + | "never-acknowledged" + | "refused"; + +/** The HTTP answer a refused `subscriptions/listen` carried (#2722). */ +export interface SubscriptionListenRefusal { + status: number; + statusText?: string; +} /** * State of the modern-era resource-subscription listen stream (#1630). @@ -601,11 +613,18 @@ export type ResourceSubscriptionStreamStatus = * list-change opt-ins, so it can be open with no subscribed URI at all (#1920). * This state describes the Subscriptions section, which has nothing to show for * such a stream. + * + * The one exception is `"refused"` (#2722), which is reported whatever `active` + * says: a subscribe the server refused is rolled back, so the subscribed set is + * empty and `active` false by the time the state is announced — and gating the + * refusal on `active` is what hid it before. */ export interface ResourceSubscriptionStreamState { active: boolean; status: ResourceSubscriptionStreamStatus; honoredUris: string[]; + /** The HTTP refusal, present exactly when `status` is `"refused"`. */ + refusal?: SubscriptionListenRefusal; } /** The stream state reported on the legacy era (or before any subscription). */ diff --git a/docs/test-servers.md b/docs/test-servers.md index aea4f1202..1b073f760 100644 --- a/docs/test-servers.md +++ b/docs/test-servers.md @@ -61,6 +61,7 @@ as a missing capability rather than an error. | `logging-{legacy,modern}-http.json` **(era per file)** | Logging, both eras | [#1629](https://github.com/modelcontextprotocol/inspector/issues/1629) | | `subscriptions-{legacy,modern}-http.json` **(era per file)** | Resource subscriptions, both eras | [#1630](https://github.com/modelcontextprotocol/inspector/issues/1630) | | `subscriptions-never-acknowledged-http.json` **(modern era)** | A `subscriptions/listen` answered with a bare result | [#2097](https://github.com/modelcontextprotocol/inspector/issues/2097) | +| `subscriptions-listen-refused-http.json` **(modern era)** | A `subscriptions/listen` refused with a bare HTTP 403 | [#2722](https://github.com/modelcontextprotocol/inspector/issues/2722) | | `tasks-{legacy,modern}-http.json` **(era per file)** | Tasks, both eras | [#1631](https://github.com/modelcontextprotocol/inspector/issues/1631) | | `cancellation-modern-http.json` **(modern era)** | Cancelling a call by closing its response stream | [#2140](https://github.com/modelcontextprotocol/inspector/issues/2140) | | `skills-http.json` **(either era)** | Skills tab: `skills/list`, `resources/directory/read`, digest verification, the frontmatter cross-check, and the non-conforming cases | [#2234](https://github.com/modelcontextprotocol/inspector/issues/2234), [#2248](https://github.com/modelcontextprotocol/inspector/issues/2248) | @@ -732,6 +733,16 @@ On the broken build that second click produced eight `subscriptions/listen` requ The first resource-subscription listen is acknowledged **so the badge is reachable at all**: it is gated on a live subscription, which a server refusing from the outset never lets you hold. That variant — refuse every listen, the literal shape in the report — is what the integration tests drive; it is the same code path, minus the badge. And `never-acknowledged` is a status of its own rather than **Stream ended** on purpose: `ended` covers the two *expected* closes (a server tearing an established stream down, and reconnection abandoned after repeated failures), and reading a deterministic conformance failure as either of them is the silence the issue is about. +## A listen refused over HTTP + +`subscriptions-listen-refused-http.json` serves the same three `numbered_resources` on the modern leg, and answers **every** `subscriptions/listen` with a bare HTTP `403 Forbidden` — no JSON-RPC body and no `WWW-Authenticate`, the shape a proxy or gateway in front of a server produces. Every other request is served normally. Connect with **Protocol Era = Modern**. + +The connect-time listen the Inspector opens for its list-change opt-ins ([#1920](https://github.com/modelcontextprotocol/inspector/issues/1920)) is refused first, so the Resources tab already shows a red **Refused** badge on `Subscriptions (0)`. Open the section to read why — *"The server refused subscriptions/listen (HTTP 403 Forbidden). Not retrying…"*. **Subscribe** to `resource_1` and it fails, the URI is rolled back, and the badge and notice stay; the Protocol tab shows exactly **one** further `subscriptions/listen`. + +On the broken build each refusal was re-sent at the reconnect backoff until the eight-attempt cap, roughly a minute, and settled on a bare **Stream ended** — or, after a rolled-back Subscribe, on nothing at all, since the badge was gated on a live subscription ([#2722](https://github.com/modelcontextprotocol/inspector/issues/2722), found in the v2.11.0 milestone smoke). Unlike the never-acknowledged case this one has no way to hold a subscription first, which is why the refusal is shown whatever the subscription count. + +The refusal statuses are 403 (with no `insufficient_scope` challenge, which would run step-up auth instead), 400, 404 and 405; set `transport.modern.refuseSubscriptionsListen` to any of them. A transient status — 5xx, 408 or 429 — is still retried, so the same option set to `503` exercises the other half. + ## Tasks, both eras **Legacy** (`tasks-legacy-http.json`) advertises `capabilities.tasks` (`tasks: { list, cancel }`) with the `simple_task` / `progress_task` / `elicitation_task` presets. Run one of those tools with **Run as task** on, and the **Tasks** tab lists it (populated via `tasks/list`), polls `tasks/get`, fetches the payload with the blocking `tasks/result`, and cancels with `tasks/cancel`. diff --git a/test-servers/configs/subscriptions-listen-refused-http.json b/test-servers/configs/subscriptions-listen-refused-http.json new file mode 100644 index 000000000..14424518b --- /dev/null +++ b/test-servers/configs/subscriptions-listen-refused-http.json @@ -0,0 +1,16 @@ +{ + "serverInfo": { + "name": "subscriptions-listen-refused", + "version": "1.0.0" + }, + "resources": [{ "preset": "numbered_resources", "params": { "count": 3 } }], + "subscriptions": true, + "listChanged": { "resources": true }, + "transport": { + "type": "streamable-http", + "port": 3224, + "modern": { + "refuseSubscriptionsListen": 403 + } + } +} diff --git a/test-servers/src/composable-test-server.ts b/test-servers/src/composable-test-server.ts index 28cc7a2a5..cc75c6c83 100644 --- a/test-servers/src/composable-test-server.ts +++ b/test-servers/src/composable-test-server.ts @@ -877,6 +877,14 @@ export interface ServerConfig { * included — the literal shape in the report. */ neverAcknowledgeSubscriptions?: boolean | "after-first"; + /** + * When set, the modern HTTP leg answers every `subscriptions/listen` with + * this bare HTTP status — no JSON-RPC body, no `WWW-Authenticate` — the + * shape a proxy or gateway produces (#2722). Used by the + * `subscriptions-listen-refused-http` showcase (403). A transient status + * (503, 429) exercises the other half: the Inspector keeps retrying those. + */ + refuseSubscriptionsListen?: number; }; /** * Optional server control for orderly shutdown (test HTTP server). diff --git a/test-servers/src/load-config.ts b/test-servers/src/load-config.ts index 05b2d3c6f..39599c5d6 100644 --- a/test-servers/src/load-config.ts +++ b/test-servers/src/load-config.ts @@ -166,6 +166,7 @@ export interface ConfigFile { legacy?: "stateless" | "reject"; injectSpecErrors?: boolean; neverAcknowledgeSubscriptions?: boolean | "after-first"; + refuseSubscriptionsListen?: number; }; }; } diff --git a/test-servers/src/test-server-http.ts b/test-servers/src/test-server-http.ts index c4afd51da..984a07edc 100644 --- a/test-servers/src/test-server-http.ts +++ b/test-servers/src/test-server-http.ts @@ -281,6 +281,30 @@ function createNeverAcknowledgeSubscriptionsInjector( }; } +/** + * Refuse every `subscriptions/listen` with a bare HTTP status (#2722) — no + * JSON-RPC body and no `WWW-Authenticate`, the shape a gateway or proxy in front + * of a server produces. Observed in the v2.11.0 smoke as a bare 403 from a + * logging proxy, which the Inspector re-sent until its eight-attempt cap and + * never reported. + * + * The status is the config value, so the same fixture serves both halves of + * the classification: a deterministic refusal (403, 400, 404, 405) the + * Inspector must not retry, and a transient one (503, 429) it must. + */ +function createRefuseSubscriptionsListenInjector( + status: number, +): express.RequestHandler { + return (req, res, next) => { + const body = req.body as JsonRpcListenBody | undefined; + if (body?.method !== "subscriptions/listen") { + next(); + return; + } + res.sendStatus(status); + }; +} + const specErrorInjector: express.RequestHandler = (req, res, next) => { const body = req.body as JsonRpcCallBody | undefined; const trigger = @@ -498,6 +522,16 @@ export class TestServerHttp { ); } + // Optional HTTP-refusal injector (#2722): answers `subscriptions/listen` + // with a bare HTTP status, so the Inspector's refusal classification has a + // server to reproduce it against. + const refuseListen = this.config.modern?.refuseSubscriptionsListen; + if (refuseListen !== undefined) { + extraMiddleware.push( + createRefuseSubscriptionsListenInjector(refuseListen), + ); + } + // Modern tasks extension (SEP-2663): the SDK's modern leg era-gates inbound // `tasks/*` spec methods, so serve them from a middleware ahead of the SDK // handler, backed by the shared runtime (also used by the tools/call task From 26a1b679f268892a7bcf0f9cca8385421914f6b4 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 20:47:14 -0400 Subject: [PATCH 08/44] fix(auth): carry a superseded namespace's client secret across a backend switch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An adoption's carry-over (#2630) reads superseded namespaces through the run's own secret store only. When a stripped stamp is re-adopted on a different backend than the one holding the old namespace's entries, those entries were never carried, and the next run on their backend saw a stamped file and purged the only copy of a confidential client's secret. Adoption now marks every record it could not read in the ledger as owed to the new namespace, with the client_id each dynamically registered client's credential fields belonged to. A run on that backend reads those fields, and its first locked save copies them into the stamped namespace before purging — only where the stamped namespace holds nothing for the field and the file still holds a client with the same client_id, so a re-registered client is never paired with the old secret and nothing cleared since is resurrected. Tokens and pre-registered clients' secrets are not carried: a deliberate clear of either is indistinguishable here. Closes #2672 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 211 +++++++++++++- .../auth/node/oauth-secrets-namespace.test.ts | 257 ++++++++++++++++++ core/auth/node/oauth-namespace-ledger.ts | 247 ++++++++++++++++- core/auth/node/oauth-persist-file.ts | 86 +++++- core/auth/node/oauth-secrets.ts | 29 ++ docs/secret-storage.md | 2 +- 6 files changed, 823 insertions(+), 9 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index fcb44702c..d7fa768f4 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -1,7 +1,8 @@ /** * Unit tests for the secrets-namespace ledger (#2560): its tolerant parse, * record-only-when-new writes, per-namespace purge bookkeeping, and the - * best-effort failure handling that keeps a ledger problem from ever + * pending carry-over a record an adoption could not read owes its + * successor (#2672), and the best-effort failure handling that keeps a ledger problem from ever * failing the OAuth save or removal it rides on. The end-to-end scenarios * (a stripped stamp re-adopted, a stripped file removed) live in the * integration suite's `oauth-secrets-namespace.test.ts`. @@ -30,7 +31,9 @@ vi.mock("@inspector/core/storage/store-io.js", async (importOriginal) => { import { writeStoreFile } from "@inspector/core/storage/store-io.js"; import { + markUncarriedNamespaces, namespaceLedgerPath, + pendingCarries, purgeSupersededNamespaces, recordedNamespaces, recordNamespaceKeys, @@ -62,6 +65,7 @@ class FakeKeyring extends KeyringSecretStore { import { LEGACY_TOKENS_FIELD, IDP_SESSION_FIELD, + issuerClientSecretField, oauthIdpSecretServerId, oauthSecretServerId, } from "@inspector/core/auth/node/oauth-secrets.js"; @@ -91,7 +95,14 @@ afterEach(() => { type LedgerJson = Record< string, - Record + Record< + string, + { + servers: string[]; + idpSessions: string[]; + carry?: { into: string; clients: Record> }; + } + > >; function readLedger(): LedgerJson { @@ -496,3 +507,199 @@ describe("purgeSupersededNamespaces", () => { ); }); }); + +describe("pending carry-over (#2672)", () => { + const NS3 = "33333333-3333-4333-8333-333333333333"; + const SECRET = issuerClientSecretField(ISSUER); + const owners = (clientId: string) => + new Map([[SERVER, new Map([[SECRET, clientId]])]]); + // Another run's store: marking is about the records it could not read. + const elsewhere = new KeyringSecretStore(); + + /** NS1's record in this run's store, owed to NS2 by a run elsewhere. */ + async function owed(clientId = "c", secret: string | null = "s3cret") { + await recordNamespaceKeys(stateFile, store, NS1, [SERVER], []); + if (secret !== null) { + await store.set(oauthSecretServerId(SERVER, NS1), SECRET, secret); + } + await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners(clientId)); + } + + describe("markUncarriedNamespaces", () => { + it("marks only other locations' records, never the successor's", async () => { + await recordNamespaceKeys(stateFile, store, NS1, [SERVER], []); + await recordNamespaceKeys(stateFile, elsewhere, NS1, [SERVER], []); + await recordNamespaceKeys(stateFile, store, NS2, [SERVER], []); + await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")); + const ledger = readLedger(); + expect(ledger[NS1].memory.carry).toEqual({ + into: NS2, + clients: { [SERVER]: { [SECRET]: "c" } }, + }); + expect(ledger[NS1].keyring.carry).toBeUndefined(); + expect(ledger[NS2].memory.carry).toBeUndefined(); + }); + + it("keeps the client a record was first marked with across a later re-adoption", async () => { + await owed("c"); + await markUncarriedNamespaces(stateFile, elsewhere, NS3, owners("d")); + expect(readLedger()[NS1].memory.carry).toEqual({ + into: NS3, + clients: { [SERVER]: { [SECRET]: "c" } }, + }); + }); + + it("leaves a record unmarked when the file holds no client for it", async () => { + await recordNamespaceKeys(stateFile, store, NS1, [SERVER], []); + vi.mocked(writeStoreFile).mockClear(); + await markUncarriedNamespaces(stateFile, elsewhere, NS2, new Map()); + expect(writeStoreFile).not.toHaveBeenCalled(); + expect(readLedger()[NS1].memory.carry).toBeUndefined(); + }); + + it("does not rewrite a ledger already marked the same way", async () => { + await owed(); + vi.mocked(writeStoreFile).mockClear(); + await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")); + expect(writeStoreFile).not.toHaveBeenCalled(); + }); + + it("warns instead of throwing when the ledger is unusable", async () => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + mkdirSync(ledgerFile); + await expect( + markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")), + ).resolves.toBeUndefined(); + expect(warn.mock.calls[0][0]).toContain( + "Could not update the OAuth secrets-namespace ledger", + ); + }); + }); + + describe("pendingCarries", () => { + it("lists what this store's records owe the current namespace, newest first", async () => { + await owed(); + await recordNamespaceKeys(stateFile, store, NS3, [SERVER], []); + await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")); + expect( + (await pendingCarries(stateFile, store, NS2)).map((c) => c.namespace), + ).toEqual([NS3, NS1]); + expect(await pendingCarries(stateFile, store, NS3)).toEqual([]); + expect(await pendingCarries(stateFile, elsewhere, NS2)).toEqual([]); + }); + + it("warns and reads as empty when the ledger cannot be read", async () => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + mkdirSync(ledgerFile); + expect(await pendingCarries(stateFile, store, NS2)).toEqual([]); + expect(warn).toHaveBeenCalledTimes(1); + }); + + it("ignores a malformed mark, and malformed clients inside a valid one", async () => { + writeFileSync( + ledgerFile, + JSON.stringify({ + namespaces: { + [NS1]: { memory: { servers: [SERVER], carry: { into: "bad+ns" } } }, + [NS3]: { + memory: { + servers: [SERVER], + carry: { + into: NS2, + clients: { + [SERVER]: { [SECRET]: 7, ok: "c" }, + "https://b.example": "nope", + "https://c.example": { only: 7 }, + }, + }, + }, + }, + "44444444-4444-4444-8444-444444444444": { + memory: { servers: [], carry: { into: NS2, clients: "nope" } }, + }, + }, + }), + ); + const carries = await pendingCarries(stateFile, store, NS2); + expect(carries.map((c) => [c.namespace, [...c.clients.keys()]])).toEqual([ + ["44444444-4444-4444-8444-444444444444", []], + [NS3, [SERVER]], + ]); + expect([...carries[1].clients.get(SERVER)!]).toEqual([["ok", "c"]]); + }); + }); + + describe("purgeSupersededNamespaces", () => { + const from = oauthSecretServerId(SERVER, NS1); + const to = oauthSecretServerId(SERVER, NS2); + + it("carries the owed field into the current namespace before purging", async () => { + await owed(); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("s3cret"); + expect(await store.get(from, SECRET)).toBeNull(); + // The carried key is recorded under the current namespace, like + // every other write the ledger tracks. + expect(readLedger()).toEqual({ + [NS2]: { memory: { servers: [SERVER], idpSessions: [] } }, + }); + }); + + it("records nothing new when the current namespace already lists the server", async () => { + await recordNamespaceKeys(stateFile, store, NS2, [SERVER], []); + await owed(); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("s3cret"); + }); + + it.each([ + ["a different client now", owners("other")], + ["no client for that server now", new Map()], + ])("does not carry with %s, and still purges", async (_label, now) => { + await owed(); + await purgeSupersededNamespaces(stateFile, store, NS2, now); + expect(await store.get(to, SECRET)).toBeNull(); + expect(await store.get(from, SECRET)).toBeNull(); + expect(existsSync(ledgerFile)).toBe(false); + }); + + it("keeps what the current namespace already holds", async () => { + await owed(); + await store.set(to, SECRET, "newer"); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("newer"); + expect(await store.get(from, SECRET)).toBeNull(); + }); + + it("has nothing to carry when the old namespace holds nothing", async () => { + await owed("c", null); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBeNull(); + expect(existsSync(ledgerFile)).toBe(false); + }); + + it("does not carry into a namespace the record is not owed to", async () => { + await owed(); + await purgeSupersededNamespaces(stateFile, store, undefined, owners("c")); + expect(await store.get(to, SECRET)).toBeNull(); + expect(await store.get(from, SECRET)).toBeNull(); + }); + + it("keeps the record unpurged, and warns, when the carry fails", async () => { + await owed(); + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + vi.spyOn(store, "set").mockRejectedValueOnce( + new Error("keychain locked"), + ); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(from, SECRET)).toBe("s3cret"); + expect(readLedger()[NS1].memory.carry?.into).toBe(NS2); + expect(warn.mock.calls[0][0]).toContain("keychain locked"); + + // The next locked save retries it. + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("s3cret"); + expect(await store.get(from, SECRET)).toBeNull(); + }); + }); +}); diff --git a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts index 3969a110f..9237e79d2 100644 --- a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts +++ b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts @@ -1027,3 +1027,260 @@ describe("a stripped stamp keeps what only its namespace holds (#2630)", () => { ).toBeUndefined(); }); }); + +describe("a stripped stamp re-adopted on another backend keeps what only the old one held (#2672)", () => { + const OTHER = "https://other.example/mcp"; + const SECRET_FIELD = issuerClientSecretField(ISSUER); + + function confidentialClient( + clientId: string, + secret?: string, + ): OAuthPersistSnapshot { + return { + servers: { + [SERVER]: { + activeIssuer: ISSUER, + byIssuer: { + [ISSUER]: { + clientInformation: { + client_id: clientId, + ...(secret !== undefined && { client_secret: secret }), + }, + clientRegistrationKind: "dcr", + tokens: tokensFor("one"), + }, + }, + }, + }, + idpSessions: {}, + }; + } + + async function stripStamp(filePath: string): Promise { + const parsed = JSON.parse(readFileSync(filePath, "utf8")) as Record< + string, + unknown + >; + delete parsed[SECRETS_NAMESPACE_KEY]; + await writeStoreFile(filePath, JSON.stringify(parsed)); + await flushStoreFileWrites(filePath); + } + + function clientOf(snapshot: OAuthPersistSnapshot | null) { + return snapshot?.servers[SERVER]?.byIssuer?.[ISSUER]?.clientInformation; + } + + function ledgerNamespaces(filePath: string): string[] { + const parsed = JSON.parse( + readFileSync(namespaceLedgerPath(filePath), "utf8"), + ) as { namespaces: Record }; + return Object.keys(parsed.namespaces); + } + + /** Steps 1–3 of the issue: keychain save, strip, re-adopt elsewhere. */ + async function readoptElsewhere( + keyring: SecretStore, + other: SecretStore, + ): Promise<{ ns1: string; ns2: string }> { + await writeOAuthSections( + fileA, + confidentialClient("dcr-client", "s3cret"), + undefined, + keyring, + ); + await flushStoreFileWrites(fileA); + const ns1 = namespaceOf(fileA); + await stripStamp(fileA); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "x" } }, idpSessions: {} }, + { servers: [OTHER] }, + other, + ); + await flushStoreFileWrites(fileA); + return { ns1, ns2: namespaceOf(fileA) }; + } + + it("carries the keychain's secret into the stamped namespace before purging it (the issue's repro)", async () => { + const keyring = new KeyringSecretStore(); + const { ns1, ns2 } = await readoptElsewhere( + keyring, + new InMemorySecretStore(), + ); + expect(ns2).not.toBe(ns1); + // The other backend could not see it, so the keychain still holds it. + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBe("s3cret"); + + // Step 4: a keychain run on the stamped file. Its read already sees + // the secret, so the save it makes from that state keeps it. + const loaded = await readOAuthStore(fileA, keyring); + expect(clientOf(loaded)?.client_secret).toBe("s3cret"); + await writeOAuthSections(fileA, loaded!, { servers: [SERVER] }, keyring); + await flushStoreFileWrites(fileA); + + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBe("s3cret"); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + expect(ledgerNamespaces(fileA)).toEqual([ns2]); + expect(clientOf(await readOAuthStore(fileA, keyring))?.client_secret).toBe( + "s3cret", + ); + }); + + it("carries it on a save that never read the file, too", async () => { + const keyring = new KeyringSecretStore(); + const { ns1, ns2 } = await readoptElsewhere( + keyring, + new InMemorySecretStore(), + ); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBe("s3cret"); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + }); + + it("does not pair the secret with a client registered since", async () => { + const keyring = new KeyringSecretStore(); + const other = new InMemorySecretStore(); + const { ns1, ns2 } = await readoptElsewhere(keyring, other); + // Re-registered on the other backend, as a public client this time. + await writeOAuthSections( + fileA, + confidentialClient("public-client"), + { servers: [SERVER] }, + other, + ); + await flushStoreFileWrites(fileA); + + expect( + clientOf(await readOAuthStore(fileA, keyring))?.client_secret, + ).toBeUndefined(); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBeNull(); + // Nothing owed any more, so the record is purged all the same. + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + expect(ledgerNamespaces(fileA)).toEqual([ns2]); + }); + + it("does not resurrect the tokens, only the client's credentials", async () => { + const keyring = new KeyringSecretStore(); + const { ns2 } = await readoptElsewhere(keyring, new InMemorySecretStore()); + const loaded = await readOAuthStore(fileA, keyring); + expect(loaded?.servers[SERVER]?.byIssuer?.[ISSUER]?.tokens).toBeUndefined(); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), `tokens:${ISSUER}`), + ).toBeNull(); + }); + + it("keeps a value the stamped namespace already holds", async () => { + const keyring = new KeyringSecretStore(); + const { ns2 } = await readoptElsewhere(keyring, new InMemorySecretStore()); + await keyring.set(oauthSecretServerId(SERVER, ns2), SECRET_FIELD, "newer"); + expect(clientOf(await readOAuthStore(fileA, keyring))?.client_secret).toBe( + "newer", + ); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBe("newer"); + }); + + it("carries nothing for a server removed from the file since", async () => { + const keyring = new KeyringSecretStore(); + const other = new InMemorySecretStore(); + const { ns1, ns2 } = await readoptElsewhere(keyring, other); + await writeOAuthSections( + fileA, + { servers: {}, idpSessions: {} }, + { servers: [SERVER] }, + other, + ); + await flushStoreFileWrites(fileA); + + expect((await readOAuthStore(fileA, keyring))?.servers[SERVER]).toBe( + undefined, + ); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBeNull(); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + }); + + it("still carries it after a second strip and re-adoption elsewhere", async () => { + const keyring = new KeyringSecretStore(); + const other = new InMemorySecretStore(); + const { ns1 } = await readoptElsewhere(keyring, other); + await stripStamp(fileA); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "z" } }, idpSessions: {} }, + { servers: [OTHER] }, + other, + ); + await flushStoreFileWrites(fileA); + const ns3 = namespaceOf(fileA); + + expect(clientOf(await readOAuthStore(fileA, keyring))?.client_secret).toBe( + "s3cret", + ); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns3), SECRET_FIELD), + ).toBe("s3cret"); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + }); +}); diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index c276e010f..ca1211278 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -32,6 +32,20 @@ * handed to the keychain record rather than dropped, because a * file-to-keychain hand-off may already have copied its entries there. * + * Why a record can carry a pending carry-over (#2672): an adoption reads + * the namespaces it supersedes through the run's own store only, so a + * record in another location was never carried into its successor — and + * the successor's stamp means no read falls back to it again. Purging it + * from its own location later would delete what may be the only copy of a + * confidential client's secret. So adoption marks every record it could + * not read with the namespace it was superseded by and the `client_id` + * each client-credential field belonged to; a run in that location then + * reads, and finally copies, those fields into the successor before the + * purge — but only where the state file still holds a client with that + * same `client_id`. A field the successor lacks is otherwise + * indistinguishable from one deliberately cleared since, and a different + * `client_id` means the secret would be paired with the wrong client. + * * Everything here is best-effort and never throws: the ledger is a cleanup * aid, not part of the credential path, so a failure warns once and the * save or removal it rides on carries on. Purging deletes credentials, so @@ -47,7 +61,11 @@ import { writeStoreFile, } from "../../storage/store-io.js"; import { serializeStore } from "../../storage/store-serialize.js"; -import { KeyringSecretStore, type SecretStore } from "./secret-store.js"; +import { + KeyringSecretStore, + secretStoreGetStrict, + type SecretStore, +} from "./secret-store.js"; import { FileSecretStore } from "./file-secret-store.js"; import { resolveConcreteSecretStore } from "./secret-store-selection.js"; import { @@ -110,10 +128,27 @@ function coveredLocations( .map(([recorded, keys]) => ({ location: recorded, keys })); } +/** + * Per server URL, the `client_id` each client-credential field belonged to + * (see `clientCredentialOwners`). Field names are keyed by issuer and come + * from persisted state, so these are maps rather than plain objects. + */ +export type ClientCredentialOwners = Map>; + +/** + * A record an adoption could not read (#2672): the namespace that + * superseded it, and what it may still hold that the state file indexes. + */ +interface PendingCarry { + into: string; + clients: ClientCredentialOwners; +} + /** The keys one namespace has had store entries written under. */ interface LedgerKeys { servers: Set; idpSessions: Set; + carry?: PendingCarry; } /** @@ -144,6 +179,24 @@ function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); } +function parseCarry(value: unknown): PendingCarry | undefined { + if (!isRecord(value) || !isValidSecretsNamespace(value.into)) { + return undefined; + } + const clients: ClientCredentialOwners = new Map(); + if (isRecord(value.clients)) { + for (const [url, fields] of Object.entries(value.clients)) { + if (!isRecord(fields)) continue; + const owners = new Map(); + for (const [field, clientId] of Object.entries(fields)) { + if (typeof clientId === "string") owners.set(field, clientId); + } + if (owners.size > 0) clients.set(url, owners); + } + } + return { into: value.into, clients }; +} + function stringsOf(value: unknown): string[] { return Array.isArray(value) ? value.filter((item): item is string => typeof item === "string") @@ -174,9 +227,11 @@ function parseLedger(raw: string | null): Ledger { const byLocation = new Map(); for (const [location, keys] of Object.entries(locations)) { if (!isRecord(keys)) continue; + const carry = parseCarry(keys.carry); byLocation.set(location, { servers: new Set(stringsOf(keys.servers)), idpSessions: new Set(stringsOf(keys.idpSessions)), + ...(carry && { carry }), }); } if (byLocation.size > 0) ledger.set(namespace, byLocation); @@ -191,7 +246,21 @@ function serializeLedger(ledger: Ledger): string { Object.fromEntries( [...byLocation].map(([location, keys]) => [ location, - { servers: [...keys.servers], idpSessions: [...keys.idpSessions] }, + { + servers: [...keys.servers], + idpSessions: [...keys.idpSessions], + ...(keys.carry && { + carry: { + into: keys.carry.into, + clients: Object.fromEntries( + [...keys.carry.clients].map(([url, owners]) => [ + url, + Object.fromEntries(owners), + ]), + ), + }, + }), + }, ]), ), ]), @@ -255,6 +324,158 @@ export async function recordNamespaceKeys( } } +/** + * Mark every record of a namespace other than `into` that lives somewhere + * other than `secretStore` as superseded by `into` without having been + * carried into it (#2672) — adoption's carry-over reads through this run's + * store only. `owners` is what the state file being adopted holds: for + * each server, the `client_id` each client-credential field belongs to. + * A record already marked keeps the owners it recorded first, since those + * name the client its secret was issued to. + * + * Call it before the store writes the adoption makes, like + * {@link recordNamespaceKeys}: a crash after them but before the mark + * would leave the record unmarked, and its secret purged once its + * location's run sees the stamp. + */ +export async function markUncarriedNamespaces( + stateFilePath: string, + secretStore: SecretStore, + into: string, + owners: ClientCredentialOwners, +): Promise { + try { + const ledger = parseLedger( + await readStoreFile(namespaceLedgerPath(stateFilePath)), + ); + const location = await secretStoreLocation(secretStore); + let changed = false; + for (const [namespace, byLocation] of ledger) { + if (namespace === into) continue; + for (const [recorded, keys] of byLocation) { + if (recorded === location) continue; + const clients: ClientCredentialOwners = + keys.carry?.clients ?? new Map(); + for (const url of keys.servers) { + const current = owners.get(url); + if (!current) continue; + const kept = clients.get(url) ?? new Map(); + for (const [field, clientId] of current) { + if (kept.has(field)) continue; + kept.set(field, clientId); + changed = true; + } + clients.set(url, kept); + } + // Nothing the file holds to carry: leave the record unmarked. + if (clients.size === 0) continue; + if (keys.carry?.into !== into) changed = true; + keys.carry = { into, clients }; + } + } + if (changed) await writeLedger(stateFilePath, ledger); + } catch (error) { + warnLedgerFailure("update", error); + } +} + +/** + * The records a run in `secretStore`'s location still owes `current` a + * carry-over from (#2672), newest first: each one's namespace and the + * client-credential fields it may hold, with the `client_id` each belonged + * to. A stamped file's reads fall back to these — for those fields only, + * and only where the file still holds a client with that `client_id` — so + * the state a save starts from does not drop a secret the purge is about + * to carry. + */ +export async function pendingCarries( + stateFilePath: string, + secretStore: SecretStore, + current: string, +): Promise> { + try { + const ledger = parseLedger( + await readStoreFile(namespaceLedgerPath(stateFilePath)), + ); + const location = await secretStoreLocation(secretStore); + const carries: Array<{ + namespace: string; + clients: ClientCredentialOwners; + }> = []; + for (const [namespace, byLocation] of ledger) { + for (const { keys } of coveredLocations(location, byLocation)) { + if (keys.carry?.into === current) { + carries.push({ namespace, clients: keys.carry.clients }); + } + } + } + return carries.reverse(); + } catch (error) { + warnLedgerFailure("read", error); + return []; + } +} + +/** + * Copy what a pending record holds into `current` before it is purged + * (#2672): each field it was marked with, into the same server's id under + * `current`, only where `current` holds nothing for it and the state file + * still holds a client with the `client_id` the field belonged to. Records + * the `current` key first, like every other write the ledger tracks. + * Returns whether every carry succeeded; a failure leaves the record for + * the next locked save. + */ +async function carryPending( + stateFilePath: string, + ledger: Ledger, + location: string, + secretStore: SecretStore, + namespace: string, + current: string, + carry: PendingCarry, + owners: ClientCredentialOwners, +): Promise { + let carried = true; + for (const [url, fields] of carry.clients) { + const now = owners.get(url); + if (!now) continue; + for (const [field, clientId] of fields) { + if (now.get(field) !== clientId) continue; + try { + const from = oauthSecretServerId(url, namespace); + const to = oauthSecretServerId(url, current); + const value = await secretStoreGetStrict(secretStore, from, field); + if (value === null) continue; + if ((await secretStoreGetStrict(secretStore, to, field)) !== null) { + continue; + } + let byLocation = ledger.get(current); + if (!byLocation) { + byLocation = new Map(); + ledger.set(current, byLocation); + } + const keys = byLocation.get(location) ?? { + servers: new Set(), + idpSessions: new Set(), + }; + byLocation.set(location, keys); + if (!keys.servers.has(url)) { + keys.servers.add(url); + await writeLedger(stateFilePath, ledger); + } + await secretStore.set(to, field, value); + } catch (error) { + carried = false; + warnLedgerFailure( + "carry over a superseded namespace recorded in", + error, + ); + } + } + } + return carried; +} + /** * Every namespace the ledger records, newest first (#2630). A namespace is * appended when it is first recorded and JSON keeps that order, so the last @@ -296,6 +517,11 @@ export async function recordedNamespaces( * locked save or removal retries it. Records for other locations are left * for a run that uses them. * + * A record an adoption could not read and marked as owed to `current` is + * carried into it first, for the clients `owners` — what the state file + * holds now — still matches (see {@link markUncarriedNamespaces}); a + * failed carry leaves the record unpurged, for the same retry. + * * Only call this under the real file lock: it deletes credentials, and an * unlocked caller could be racing an adopter whose namespace is recorded * but not yet stamped. @@ -304,6 +530,7 @@ export async function purgeSupersededNamespaces( stateFilePath: string, secretStore: SecretStore, current: string | undefined, + owners: ClientCredentialOwners = new Map(), ): Promise { let ledger: Ledger; let location: string; @@ -323,6 +550,22 @@ export async function purgeSupersededNamespaces( location, byLocation, )) { + if ( + current !== undefined && + keys.carry?.into === current && + !(await carryPending( + stateFilePath, + ledger, + location, + secretStore, + namespace, + current, + keys.carry, + owners, + )) + ) { + continue; + } const purges = [ ...[...keys.servers].map( (url) => () => oauthSecretServerId(url, namespace), diff --git a/core/auth/node/oauth-persist-file.ts b/core/auth/node/oauth-persist-file.ts index d8d0aed10..c347bca5a 100644 --- a/core/auth/node/oauth-persist-file.ts +++ b/core/auth/node/oauth-persist-file.ts @@ -35,6 +35,9 @@ * Until then, an unstamped file's reads and adoption fall back to those * namespaces for any secret the legacy ids lack (#2630), so what the * older Inspector could not see is carried over rather than purged. + * What an adoption could not read — a namespace's entries in another + * backend than the run's — is marked in the ledger, and carried over by + * a later run on that backend before it purges them (#2672). */ import { @@ -70,13 +73,17 @@ import { } from "./secret-store.js"; import { defaultSecretStore } from "./secret-store-selection.js"; import { + markUncarriedNamespaces, + pendingCarries, purgeSupersededNamespaces, recordedNamespaces, recordNamespaceKeys, + type ClientCredentialOwners, } from "./oauth-namespace-ledger.js"; import { MAX_WRITE_ATTEMPTS } from "./file-secret-store.js"; import { IDP_SESSION_FIELD, + clientCredentialOwners, getPersistTokensPolicy, isUsableStoredSecret, isValidSecretsNamespace, @@ -374,6 +381,14 @@ function warnNamespaceCleanupFailure(error: unknown): void { * Unlocked, the purge is skipped: a concurrent adopter's namespace can be * recorded before it is stamped, and purging it would delete live * credentials. + * + * The carry-over reads through this run's store only, so a superseded + * namespace's entries in another backend (a keychain this run could not + * unlock, say) are not carried. Adoption marks those records in the + * ledger instead, with the `client_id` each client credential belonged to + * (#2672); a later locked save on that backend carries them into the + * stamped namespace — where the file still holds the same client — before + * its purge deletes them, and its reads see them in the meantime. */ async function adoptSecretsNamespace( filePath: string, @@ -386,7 +401,12 @@ async function adoptSecretsNamespace( // An ordinary save also retries any superseded namespace whose purge // failed at adoption — the new stamp means adoption will not run again. if (locked) { - await purgeSupersededNamespaces(filePath, secretStore, existing); + await purgeSupersededNamespaces( + filePath, + secretStore, + existing, + snapshotClientOwners(parseOAuthPersistBlob(raw)), + ); } return existing; } @@ -424,6 +444,10 @@ async function adoptSecretsNamespace( Object.keys(snapshot.servers), Object.keys(snapshot.idpSessions), ); + // Before the copies too: what this run's store cannot show it is still + // owed to the new namespace by a run on the store that holds it. + const owners = snapshotClientOwners(snapshot); + await markUncarriedNamespaces(filePath, secretStore, namespace, owners); // Rollback baseline: the scoped ids are vacant before this call — the // namespace is a UUID minted moments ago, so nothing can already live // under it — which makes "restore" simply "delete what we copied". @@ -462,7 +486,7 @@ async function adoptSecretsNamespace( // Only now, with everything the file still indexes copied under the new // stamp, is every other recorded namespace safe to purge: purging first // would delete the only copy of what the copies above carried over. - await purgeSupersededNamespaces(filePath, secretStore, namespace); + await purgeSupersededNamespaces(filePath, secretStore, namespace, owners); const legacyIds = [ ...Object.keys(snapshot.servers).map((url) => oauthSecretServerId(url)), ...Object.keys(snapshot.idpSessions).map((issuer) => @@ -1197,6 +1221,21 @@ function fillMissingSecrets( } } +/** Per server, the `client_id` each client credential belongs to. */ +function snapshotClientOwners( + snapshot: OAuthPersistSnapshot | null, +): ClientCredentialOwners { + const owners: ClientCredentialOwners = new Map(); + for (const [url, state] of Object.entries(snapshot?.servers ?? {})) { + const fields = clientCredentialOwners(state); + if (fields.size > 0) owners.set(url, fields); + } + return owners; +} + +/** A superseded namespace a stamped file is still owed fields from. */ +type PendingCarry = Awaited>[number]; + /** * Read every secret the snapshot's entries could hold, from the ids * `namespace` gives them, then from each of `fallbacks` in order for the @@ -1206,6 +1245,11 @@ function fillMissingSecrets( * older Inspector wrote there since the strip wins, and the stripped * namespace supplies only what it alone still holds. * + * `carries` is non-empty only for a stamped file with records in this + * store an adoption elsewhere could not read (#2672, `pendingCarries`): + * each supplies only the client-credential fields it was marked with, + * and only for a client the file still holds under the same `client_id`. + * * Strict: this read hydrates the memory state that later sectioned writes * diff against, so a tolerant read during a store outage would present * every credential as absent — and the next save would *delete* them from @@ -1217,6 +1261,7 @@ async function readEntrySecrets( secretStore: SecretStore, namespace: string | undefined, fallbacks: string[], + carries: PendingCarry[] = [], ): Promise { const servers = new Map(); const idpSessions = new Map(); @@ -1238,6 +1283,32 @@ async function readEntrySecrets( ); } } + const owners = snapshotClientOwners(snapshot); + for (const { namespace: from, clients } of carries) { + const requests: SecretBulkRequest[] = []; + for (const [url, fields] of clients) { + const target = servers.get(url); + if (!target) continue; + const wanted = [...fields] + .filter( + ([field, clientId]) => + owners.get(url)?.get(field) === clientId && + !Object.hasOwn(target, field), + ) + .map(([field]) => field); + if (wanted.length > 0) { + requests.push({ + serverId: oauthSecretServerId(url, from), + fields: wanted, + }); + } + } + if (requests.length === 0) continue; + const values = await secretStoreGetManyStrict(secretStore, requests); + for (const [url, target] of servers) { + fillMissingSecrets(target, values[oauthSecretServerId(url, from)]); + } + } return { servers, idpSessions }; } @@ -1247,12 +1318,14 @@ async function joinSnapshot( secretStore: SecretStore, namespace: string | undefined, fallbacks: string[], + carries: PendingCarry[] = [], ): Promise { const values = await readEntrySecrets( snapshot, secretStore, namespace, fallbacks, + carries, ); return { servers: Object.fromEntries( @@ -1420,11 +1493,16 @@ export async function readOAuthStore( // until a write mints the namespace and moves them (#2549). An // unstamped file also falls back to the namespaces it used to carry, // so the state a save starts from still holds what only they have - // (#2630). + // (#2630). A stamped one reads only the client credentials a record in + // this store still owes it, until a locked save carries them (#2672). const namespace = parseSecretsNamespace(raw); const fallbacks = namespace === undefined ? await recordedNamespaces(filePath) : []; - return joinSnapshot(snapshot, secretStore, namespace, fallbacks); + const carries = + namespace === undefined + ? [] + : await pendingCarries(filePath, secretStore, namespace); + return joinSnapshot(snapshot, secretStore, namespace, fallbacks, carries); }); } diff --git a/core/auth/node/oauth-secrets.ts b/core/auth/node/oauth-secrets.ts index e62e99120..f77a154b5 100644 --- a/core/auth/node/oauth-secrets.ts +++ b/core/auth/node/oauth-secrets.ts @@ -574,6 +574,35 @@ export function serverSecretFields( return [...fields]; } +/** + * The `client_id` each dynamically registered client's credential fields + * belong to, keyed by field (#2672). A carried-over `client_secret` or + * registration token is only meaningful paired with the client it was + * issued to, and the store does not record that pairing, so a carry-over + * from a superseded namespace checks it against this. Pre-registered + * clients are left out: they are user-entered, and a user can clear one's + * secret while keeping its `client_id`, which no carry-over could tell + * apart from a secret that was never carried. + */ +export function clientCredentialOwners( + state: ServerOAuthState, +): Map { + const owners = new Map(); + const add = ( + info: { client_id?: unknown } | undefined, + fields: ClientInfoFields, + ): void => { + if (typeof info?.client_id !== "string") return; + owners.set(fields.clientSecret, info.client_id); + owners.set(fields.registrationToken, info.client_id); + }; + add(state.clientInformation, LEGACY_CLIENT_INFO_FIELDS); + for (const [issuer, slot] of Object.entries(state.byIssuer ?? {})) { + add(slot.clientInformation, issuerClientInfoFields(issuer)); + } + return owners; +} + /** Does this `clientInformation` object still carry a bearer-grade key? */ function clientInfoHasPlaintext( info: ClientInfoSecretKeys | undefined, diff --git a/docs/secret-storage.md b/docs/secret-storage.md index 0b1012e8e..ad32a85ca 100644 --- a/docs/secret-storage.md +++ b/docs/secret-storage.md @@ -17,7 +17,7 @@ These values are stored as secrets: They are kept out of `mcp.json` so that sharing, committing or syncing the file does not leak credentials (#1356). When the Inspector saves an entry to a durable store, it leaves each `env` key in `mcp.json` with an empty value and omits the client secret; the real values live in the store. `headers` are **not** moved: they are saved in `mcp.json` exactly as written, so a header that carries a credential stays in the file. [MCP server configuration](./mcp-server-configuration.md) describes what that means for other tools reading the same file. -Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). +Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace, noting the `client_id` each dynamically registered client's `client_secret` and registration token belonged to. The next run on their backend reads them, and its first locked save copies them into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the state file still holds a client with the same `client_id`, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). ## How the store is chosen From 43afcdbb67f3bd285738821a8079807eb7e57e60 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 21:51:58 -0400 Subject: [PATCH 09/44] fix(web): reject empty TLS PEM files; isolate the env-reading TLS test Node accepts an empty key and certificate and builds a server that can never handshake, so an empty file is refused with the pair error. The process.env test now saves and clears both variables, so a shell that exported them per docs/tls.md cannot decide its outcome. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/server/tls.ts | 7 +++- .../src/test/integration/server/tls.test.ts | 38 +++++++++++++++++-- 2 files changed, 39 insertions(+), 6 deletions(-) diff --git a/clients/web/server/tls.ts b/clients/web/server/tls.ts index 34c964790..9b17ce514 100644 --- a/clients/web/server/tls.ts +++ b/clients/web/server/tls.ts @@ -78,11 +78,14 @@ export function resolveTlsConfig( // Validate the pair now rather than at bind time, where the same OpenSSL // error surfaces from inside the server constructor with no hint of which // variable to fix. Catches a non-PEM file and a key that does not match the - // certificate alike. + // certificate alike. An empty file needs its own check: Node accepts + // `{ key: "", cert: "" }` and builds a server that can never complete a + // handshake. try { + if (!key.trim() || !cert.trim()) throw new Error("empty PEM file"); createSecureContext({ key, cert }); } catch (err) { - // createSecureContext only throws Error instances (OpenSSL failures). + // Both throwers above raise Error instances (OpenSSL failures or ours). const reason = (err as Error).message; throw new Error( `${TLS_KEY_PATH_ENV}="${keyPath}" and ${TLS_CERT_PATH_ENV}="${certPath}" are not a usable PEM key/certificate pair (${reason}).`, diff --git a/clients/web/src/test/integration/server/tls.test.ts b/clients/web/src/test/integration/server/tls.test.ts index d5a638628..2ddce7280 100644 --- a/clients/web/src/test/integration/server/tls.test.ts +++ b/clients/web/src/test/integration/server/tls.test.ts @@ -40,10 +40,30 @@ describe("resolveTlsConfig", () => { }); it("reads process.env by default", () => { - // The suite's own environment carries neither variable. - expect(process.env[TLS_KEY_PATH_ENV]).toBeUndefined(); - expect(process.env[TLS_CERT_PATH_ENV]).toBeUndefined(); - expect(resolveTlsConfig()).toBeUndefined(); + // Saved and cleared so a shell that exported the pair (as docs/tls.md + // suggests) cannot decide the outcome. + const saved = { + key: process.env[TLS_KEY_PATH_ENV], + cert: process.env[TLS_CERT_PATH_ENV], + }; + try { + delete process.env[TLS_KEY_PATH_ENV]; + delete process.env[TLS_CERT_PATH_ENV]; + expect(resolveTlsConfig()).toBeUndefined(); + process.env[TLS_KEY_PATH_ENV] = fixture.keyPath; + process.env[TLS_CERT_PATH_ENV] = fixture.certPath; + expect(resolveTlsConfig()).toEqual( + resolveTlsConfig(env(fixture.keyPath, fixture.certPath)), + ); + } finally { + for (const [name, value] of [ + [TLS_KEY_PATH_ENV, saved.key], + [TLS_CERT_PATH_ENV, saved.cert], + ] as const) { + if (value === undefined) delete process.env[name]; + else process.env[name] = value; + } + } }); it("refuses a key path without a cert path, naming both variables", () => { @@ -121,6 +141,16 @@ describe("resolveTlsConfig", () => { ); }); + it.each(["key", "cert"] as const)("refuses an empty %s file", (which) => { + const empty = join(fixture.dir, `empty-${which}.pem`); + writeFileSync(empty, "\n"); + const [key, cert] = + which === "key" ? [empty, fixture.certPath] : [fixture.keyPath, empty]; + expect(() => resolveTlsConfig(env(key, cert))).toThrow( + `${TLS_KEY_PATH_ENV}="${key}" and ${TLS_CERT_PATH_ENV}="${cert}" are not a usable PEM key/certificate pair (empty PEM file).`, + ); + }); + it("refuses a key that does not match the certificate", () => { const otherKey = join(fixture.dir, "other-key.pem"); const { privateKey } = generateKeyPairSync("rsa", { modulusLength: 2048 }); From 619b6201f30693cd66026ae952813f75c9c579c2 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 21:52:49 -0400 Subject: [PATCH 10/44] fix: let a folded JSON-RPC error body replace a failed send's clientError enrichProtocolEntries folds the server's JSON-RPC error body (e.g. -32601 on HTTP 404) into a request with no response. Since #2721 such a request already carries the failed send as a clientError, and keeping both made the entry read "Rejected by the Inspector" beside the server's own error. The body is the better record, so it now replaces the note. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../src/utils/correlateTransportErrors.test.ts | 15 +++++++++++++++ clients/web/src/utils/correlateTransportErrors.ts | 10 +++++++++- 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/clients/web/src/utils/correlateTransportErrors.test.ts b/clients/web/src/utils/correlateTransportErrors.test.ts index b285b63a4..6a6ff916f 100644 --- a/clients/web/src/utils/correlateTransportErrors.test.ts +++ b/clients/web/src/utils/correlateTransportErrors.test.ts @@ -74,6 +74,21 @@ describe("enrichProtocolEntries", () => { ).toBe(-32601); }); + it("replaces a failed send's clientError with the server's error body (#2721)", () => { + const failed: MessageEntry = { + ...requestEntry(1), + clientError: "HTTP 404 Not Found: Error POSTing to endpoint", + }; + const fetches = [transportFetch(1, errorBody(-32601))]; + const [out] = enrichProtocolEntries([failed], fetches); + expect( + out.response && "error" in out.response && out.response.error.code, + ).toBe(-32601); + expect(out).not.toHaveProperty("clientError"); + // The log's own entry is not mutated. + expect(failed.clientError).toBeDefined(); + }); + it("returns the SAME array reference when nothing matches", () => { const messages = [requestEntry(1)]; // A fetch for a different id, and one with no error body. diff --git a/clients/web/src/utils/correlateTransportErrors.ts b/clients/web/src/utils/correlateTransportErrors.ts index 3c2d5c023..7781a7aed 100644 --- a/clients/web/src/utils/correlateTransportErrors.ts +++ b/clients/web/src/utils/correlateTransportErrors.ts @@ -174,7 +174,15 @@ export function enrichProtocolEntries( const error = errorsById.get(String(id)); if (!error) return entry; changed = true; - return { ...entry, response: error }; + // A request whose send failed already carries the failure as a + // `clientError` (#2721). When the refusal also carried a JSON-RPC error + // body, that body is the better record of it — it is the server's own + // answer — so it replaces the note rather than sitting beside it, where + // a `clientError` on an answered entry would read as the Inspector + // rejecting a response. + const folded: MessageEntry = { ...entry, response: error }; + delete folded.clientError; + return folded; }); return changed ? enriched : messages; } From b4c352fc1a196f591e576d956808d8b71efda39a Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 21:54:17 -0400 Subject: [PATCH 11/44] fix(auth): record a credential's owner when written, and harden the carry-over MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-1 review of #2731: - Record the client_id each client credential belongs to when a save writes it, instead of inferring it from the file being adopted — an older Inspector may have re-registered in between. Records written before this carry no owner and are purged as before. - Fail adoption when the owed mark cannot be written, so a stamp never commits without it. - Throw from pendingCarries on an unreadable ledger, so a read never presents an owed secret as absent. - Visit records newest first in the purge, so the carried value matches what a read would have seen. - Keep owners and the owed mark through the secrets-file-to-keychain ledger hand-off. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 228 ++++++++++---- .../auth/node/oauth-secrets-namespace.test.ts | 56 ++++ core/auth/node/oauth-namespace-ledger.ts | 285 +++++++++--------- core/auth/node/oauth-persist-file.ts | 28 +- docs/secret-storage.md | 2 +- 5 files changed, 384 insertions(+), 215 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index d7fa768f4..2546a8ce6 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -100,7 +100,8 @@ type LedgerJson = Record< { servers: string[]; idpSessions: string[]; - carry?: { into: string; clients: Record> }; + owners?: Record>; + owedTo?: string; } > >; @@ -518,69 +519,126 @@ describe("pending carry-over (#2672)", () => { /** NS1's record in this run's store, owed to NS2 by a run elsewhere. */ async function owed(clientId = "c", secret: string | null = "s3cret") { - await recordNamespaceKeys(stateFile, store, NS1, [SERVER], []); + await recordNamespaceKeys( + stateFile, + store, + NS1, + [SERVER], + [], + owners(clientId), + ); if (secret !== null) { await store.set(oauthSecretServerId(SERVER, NS1), SECRET, secret); } - await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners(clientId)); + await markUncarriedNamespaces(stateFile, elsewhere, NS2); } + describe("recordNamespaceKeys", () => { + it("records each credential's owner, a later write's replacing it", async () => { + await recordNamespaceKeys( + stateFile, + store, + NS1, + [SERVER], + [], + owners("a"), + ); + vi.mocked(writeStoreFile).mockClear(); + await recordNamespaceKeys( + stateFile, + store, + NS1, + [SERVER], + [], + owners("a"), + ); + expect(writeStoreFile).not.toHaveBeenCalled(); + await recordNamespaceKeys( + stateFile, + store, + NS1, + [SERVER], + [], + owners("b"), + ); + expect(readLedger()[NS1].memory.owners).toEqual({ + [SERVER]: { [SECRET]: "b" }, + }); + }); + }); + describe("markUncarriedNamespaces", () => { - it("marks only other locations' records, never the successor's", async () => { - await recordNamespaceKeys(stateFile, store, NS1, [SERVER], []); - await recordNamespaceKeys(stateFile, elsewhere, NS1, [SERVER], []); - await recordNamespaceKeys(stateFile, store, NS2, [SERVER], []); - await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")); + it("marks only other locations' records that hold a credential, never the successor's", async () => { + await recordNamespaceKeys( + stateFile, + store, + NS1, + [SERVER], + [], + owners("c"), + ); + await recordNamespaceKeys( + stateFile, + elsewhere, + NS1, + [SERVER], + [], + owners("c"), + ); + await recordNamespaceKeys(stateFile, store, NS3, [SERVER], []); + await recordNamespaceKeys( + stateFile, + store, + NS2, + [SERVER], + [], + owners("c"), + ); + await markUncarriedNamespaces(stateFile, elsewhere, NS2); const ledger = readLedger(); - expect(ledger[NS1].memory.carry).toEqual({ - into: NS2, - clients: { [SERVER]: { [SECRET]: "c" } }, - }); - expect(ledger[NS1].keyring.carry).toBeUndefined(); - expect(ledger[NS2].memory.carry).toBeUndefined(); + expect(ledger[NS1].memory.owedTo).toBe(NS2); + expect(ledger[NS1].keyring.owedTo).toBeUndefined(); + // Written before owners were recorded: nothing to carry from it. + expect(ledger[NS3].memory.owedTo).toBeUndefined(); + expect(ledger[NS2].memory.owedTo).toBeUndefined(); }); - it("keeps the client a record was first marked with across a later re-adoption", async () => { + it("re-marks a record for a later re-adoption, keeping the owner it was written with", async () => { await owed("c"); - await markUncarriedNamespaces(stateFile, elsewhere, NS3, owners("d")); - expect(readLedger()[NS1].memory.carry).toEqual({ - into: NS3, - clients: { [SERVER]: { [SECRET]: "c" } }, + await markUncarriedNamespaces(stateFile, elsewhere, NS3); + expect(readLedger()[NS1].memory).toMatchObject({ + owedTo: NS3, + owners: { [SERVER]: { [SECRET]: "c" } }, }); }); - it("leaves a record unmarked when the file holds no client for it", async () => { - await recordNamespaceKeys(stateFile, store, NS1, [SERVER], []); - vi.mocked(writeStoreFile).mockClear(); - await markUncarriedNamespaces(stateFile, elsewhere, NS2, new Map()); - expect(writeStoreFile).not.toHaveBeenCalled(); - expect(readLedger()[NS1].memory.carry).toBeUndefined(); - }); - it("does not rewrite a ledger already marked the same way", async () => { await owed(); vi.mocked(writeStoreFile).mockClear(); - await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")); + await markUncarriedNamespaces(stateFile, elsewhere, NS2); expect(writeStoreFile).not.toHaveBeenCalled(); }); - it("warns instead of throwing when the ledger is unusable", async () => { - const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + it("throws when the ledger is unusable, so adoption cannot commit unmarked", async () => { mkdirSync(ledgerFile); await expect( - markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")), - ).resolves.toBeUndefined(); - expect(warn.mock.calls[0][0]).toContain( - "Could not update the OAuth secrets-namespace ledger", - ); + markUncarriedNamespaces(stateFile, elsewhere, NS2), + ).rejects.toThrow(); }); }); describe("pendingCarries", () => { it("lists what this store's records owe the current namespace, newest first", async () => { await owed(); - await recordNamespaceKeys(stateFile, store, NS3, [SERVER], []); - await markUncarriedNamespaces(stateFile, elsewhere, NS2, owners("c")); + await recordNamespaceKeys( + stateFile, + store, + NS3, + [SERVER], + [], + owners("c"), + ); + await markUncarriedNamespaces(stateFile, elsewhere, NS2); expect( (await pendingCarries(stateFile, store, NS2)).map((c) => c.namespace), ).toEqual([NS3, NS1]); @@ -588,44 +646,40 @@ describe("pending carry-over (#2672)", () => { expect(await pendingCarries(stateFile, elsewhere, NS2)).toEqual([]); }); - it("warns and reads as empty when the ledger cannot be read", async () => { - const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + it("throws when the ledger cannot be read, rather than hide an owed secret", async () => { mkdirSync(ledgerFile); - expect(await pendingCarries(stateFile, store, NS2)).toEqual([]); - expect(warn).toHaveBeenCalledTimes(1); + await expect(pendingCarries(stateFile, store, NS2)).rejects.toThrow(); }); - it("ignores a malformed mark, and malformed clients inside a valid one", async () => { + it("ignores a malformed mark, and malformed owners inside a valid one", async () => { writeFileSync( ledgerFile, JSON.stringify({ namespaces: { - [NS1]: { memory: { servers: [SERVER], carry: { into: "bad+ns" } } }, + [NS1]: { memory: { servers: [SERVER], owedTo: "bad+ns" } }, [NS3]: { memory: { servers: [SERVER], - carry: { - into: NS2, - clients: { - [SERVER]: { [SECRET]: 7, ok: "c" }, - "https://b.example": "nope", - "https://c.example": { only: 7 }, - }, + owedTo: NS2, + owners: { + [SERVER]: { [SECRET]: 7, ok: "c" }, + "https://b.example": "nope", + "https://c.example": { only: 7 }, }, }, }, "44444444-4444-4444-8444-444444444444": { - memory: { servers: [], carry: { into: NS2, clients: "nope" } }, + memory: { servers: [], owedTo: NS2, owners: "nope" }, }, }, }), ); const carries = await pendingCarries(stateFile, store, NS2); - expect(carries.map((c) => [c.namespace, [...c.clients.keys()]])).toEqual([ + expect(carries.map((c) => [c.namespace, [...c.owners.keys()]])).toEqual([ ["44444444-4444-4444-8444-444444444444", []], [NS3, [SERVER]], ]); - expect([...carries[1].clients.get(SERVER)!]).toEqual([["ok", "c"]]); + expect([...carries[1].owners.get(SERVER)!]).toEqual([["ok", "c"]]); }); }); @@ -638,20 +692,49 @@ describe("pending carry-over (#2672)", () => { await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); expect(await store.get(to, SECRET)).toBe("s3cret"); expect(await store.get(from, SECRET)).toBeNull(); - // The carried key is recorded under the current namespace, like - // every other write the ledger tracks. + // The carried key and its owner are recorded under the current + // namespace, like every other write the ledger tracks. expect(readLedger()).toEqual({ - [NS2]: { memory: { servers: [SERVER], idpSessions: [] } }, + [NS2]: { + memory: { + servers: [SERVER], + idpSessions: [], + owners: { [SERVER]: { [SECRET]: "c" } }, + }, + }, }); }); - it("records nothing new when the current namespace already lists the server", async () => { - await recordNamespaceKeys(stateFile, store, NS2, [SERVER], []); + it("records nothing new when the current namespace already lists the server and owner", async () => { + await recordNamespaceKeys( + stateFile, + store, + NS2, + [SERVER], + [], + owners("c"), + ); await owed(); await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); expect(await store.get(to, SECRET)).toBe("s3cret"); }); + it("carries the newest owed value when two records owe the same field", async () => { + await owed("c", "older"); + await recordNamespaceKeys( + stateFile, + store, + NS3, + [SERVER], + [], + owners("c"), + ); + await store.set(oauthSecretServerId(SERVER, NS3), SECRET, "newer"); + await markUncarriedNamespaces(stateFile, elsewhere, NS2); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("newer"); + }); + it.each([ ["a different client now", owners("other")], ["no client for that server now", new Map()], @@ -693,7 +776,7 @@ describe("pending carry-over (#2672)", () => { ); await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); expect(await store.get(from, SECRET)).toBe("s3cret"); - expect(readLedger()[NS1].memory.carry?.into).toBe(NS2); + expect(readLedger()[NS1].memory.owedTo).toBe(NS2); expect(warn.mock.calls[0][0]).toContain("keychain locked"); // The next locked save retries it. @@ -701,5 +784,32 @@ describe("pending carry-over (#2672)", () => { expect(await store.get(to, SECRET)).toBe("s3cret"); expect(await store.get(from, SECRET)).toBeNull(); }); + + it("hands a secrets file's owed record to the keychain, mark and owners included", async () => { + // Its entries may have been absorbed into the keychain before this + // file run, which then finds nothing to carry. + const fileStore = new FileSecretStore({ + filePath: join(tempDir, "secrets.json"), + passphrase: "", + }); + await recordNamespaceKeys( + stateFile, + fileStore, + NS1, + [SERVER], + [], + owners("c"), + ); + await markUncarriedNamespaces(stateFile, store, NS2); + await purgeSupersededNamespaces(stateFile, fileStore, NS2, owners("c")); + expect(readLedger()[NS1]).toEqual({ + keyring: { + servers: [SERVER], + idpSessions: [], + owners: { [SERVER]: { [SECRET]: "c" } }, + owedTo: NS2, + }, + }); + }); }); }); diff --git a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts index 9237e79d2..c2b495f6a 100644 --- a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts +++ b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts @@ -1186,6 +1186,62 @@ describe("a stripped stamp re-adopted on another backend keeps what only the old expect(ledgerNamespaces(fileA)).toEqual([ns2]); }); + it("does not pair the secret with a client the older version registered before re-adoption", async () => { + const keyring = new KeyringSecretStore(); + await writeOAuthSections( + fileA, + confidentialClient("dcr-client", "s3cret"), + undefined, + keyring, + ); + await flushStoreFileWrites(fileA); + const ns1 = namespaceOf(fileA); + // The ≤ 2.9.x save strips the stamp and re-registered meanwhile, so + // the file being adopted names a client the old secret is not for. + const parsed = JSON.parse(readFileSync(fileA, "utf8")) as { + servers: Record< + string, + { + byIssuer: Record< + string, + { clientInformation: { client_id: string } } + >; + } + >; + [SECRETS_NAMESPACE_KEY]?: string; + }; + delete parsed[SECRETS_NAMESPACE_KEY]; + parsed.servers[SERVER].byIssuer[ISSUER].clientInformation.client_id = + "reregistered"; + await writeStoreFile(fileA, JSON.stringify(parsed)); + await flushStoreFileWrites(fileA); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "x" } }, idpSessions: {} }, + { servers: [OTHER] }, + new InMemorySecretStore(), + ); + await flushStoreFileWrites(fileA); + const ns2 = namespaceOf(fileA); + + expect( + clientOf(await readOAuthStore(fileA, keyring))?.client_secret, + ).toBeUndefined(); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBeNull(); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + }); + it("does not resurrect the tokens, only the client's credentials", async () => { const keyring = new KeyringSecretStore(); const { ns2 } = await readoptElsewhere(keyring, new InMemorySecretStore()); diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index ca1211278..87ccc28b6 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -37,16 +37,20 @@ * record in another location was never carried into its successor — and * the successor's stamp means no read falls back to it again. Purging it * from its own location later would delete what may be the only copy of a - * confidential client's secret. So adoption marks every record it could - * not read with the namespace it was superseded by and the `client_id` - * each client-credential field belonged to; a run in that location then - * reads, and finally copies, those fields into the successor before the - * purge — but only where the state file still holds a client with that - * same `client_id`. A field the successor lacks is otherwise - * indistinguishable from one deliberately cleared since, and a different - * `client_id` means the secret would be paired with the wrong client. + * confidential client's secret. So every record also notes, as each save + * writes it, the `client_id` each client-credential field belongs to, and + * adoption marks every record it could not read as owed to the namespace + * that superseded it. A run in that location then reads, and finally + * copies, those fields into the successor before the purge — but only + * where the state file still holds a client with that same `client_id`. A + * field the successor lacks is otherwise indistinguishable from one + * deliberately cleared since, and a different `client_id` means the secret + * would be paired with the wrong client. The owner is the one recorded at + * write time, never inferred from the file being adopted: an older + * Inspector may have re-registered in between. A record written before + * owners were recorded has none, so nothing is carried from it. * - * Everything here is best-effort and never throws: the ledger is a cleanup + * Everything else here is best-effort and never throws: the ledger is a cleanup * aid, not part of the credential path, so a failure warns once and the * save or removal it rides on carries on. Purging deletes credentials, so * the callers in `oauth-persist-file.ts` run it only under the real file @@ -136,19 +140,39 @@ function coveredLocations( export type ClientCredentialOwners = Map>; /** - * A record an adoption could not read (#2672): the namespace that - * superseded it, and what it may still hold that the state file indexes. + * The keys one namespace has had store entries written under, the client + * each client credential written there belongs to, and — for a record an + * adoption could not read (#2672) — the namespace it is owed to. */ -interface PendingCarry { - into: string; - clients: ClientCredentialOwners; -} - -/** The keys one namespace has had store entries written under. */ interface LedgerKeys { servers: Set; idpSessions: Set; - carry?: PendingCarry; + owners: ClientCredentialOwners; + owedTo?: string; +} + +const emptyKeys = (): LedgerKeys => ({ + servers: new Set(), + idpSessions: new Set(), + owners: new Map(), +}); + +/** Merge `source` into `target`, the later owner winning; reports a change. */ +function mergeOwners( + target: ClientCredentialOwners, + source: ClientCredentialOwners, +): boolean { + let changed = false; + for (const [url, fields] of source) { + const kept = target.get(url) ?? new Map(); + for (const [field, clientId] of fields) { + if (kept.get(field) === clientId) continue; + kept.set(field, clientId); + changed = true; + } + if (kept.size > 0) target.set(url, kept); + } + return changed; } /** @@ -179,22 +203,18 @@ function isRecord(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); } -function parseCarry(value: unknown): PendingCarry | undefined { - if (!isRecord(value) || !isValidSecretsNamespace(value.into)) { - return undefined; - } - const clients: ClientCredentialOwners = new Map(); - if (isRecord(value.clients)) { - for (const [url, fields] of Object.entries(value.clients)) { - if (!isRecord(fields)) continue; - const owners = new Map(); - for (const [field, clientId] of Object.entries(fields)) { - if (typeof clientId === "string") owners.set(field, clientId); - } - if (owners.size > 0) clients.set(url, owners); +function parseOwners(value: unknown): ClientCredentialOwners { + const owners: ClientCredentialOwners = new Map(); + if (!isRecord(value)) return owners; + for (const [url, fields] of Object.entries(value)) { + if (!isRecord(fields)) continue; + const byField = new Map(); + for (const [field, clientId] of Object.entries(fields)) { + if (typeof clientId === "string") byField.set(field, clientId); } + if (byField.size > 0) owners.set(url, byField); } - return { into: value.into, clients }; + return owners; } function stringsOf(value: unknown): string[] { @@ -227,11 +247,11 @@ function parseLedger(raw: string | null): Ledger { const byLocation = new Map(); for (const [location, keys] of Object.entries(locations)) { if (!isRecord(keys)) continue; - const carry = parseCarry(keys.carry); byLocation.set(location, { servers: new Set(stringsOf(keys.servers)), idpSessions: new Set(stringsOf(keys.idpSessions)), - ...(carry && { carry }), + owners: parseOwners(keys.owners), + ...(isValidSecretsNamespace(keys.owedTo) && { owedTo: keys.owedTo }), }); } if (byLocation.size > 0) ledger.set(namespace, byLocation); @@ -249,17 +269,15 @@ function serializeLedger(ledger: Ledger): string { { servers: [...keys.servers], idpSessions: [...keys.idpSessions], - ...(keys.carry && { - carry: { - into: keys.carry.into, - clients: Object.fromEntries( - [...keys.carry.clients].map(([url, owners]) => [ - url, - Object.fromEntries(owners), - ]), - ), - }, + ...(keys.owners.size > 0 && { + owners: Object.fromEntries( + [...keys.owners].map(([url, fields]) => [ + url, + Object.fromEntries(fields), + ]), + ), }), + ...(keys.owedTo !== undefined && { owedTo: keys.owedTo }), }, ]), ), @@ -282,7 +300,9 @@ async function writeLedger( * `namespace` in `secretStore`. Call it *before* the store writes, so a * crash between the two leaves an over-recorded ledger rather than an * unrecorded entry. Rewrites the ledger only when a key is new, so a - * steady-state save costs one small read. + * steady-state save costs one small read. `owners` is, per server, the + * `client_id` each client credential being written belongs to (#2672); + * a later write's owner replaces an earlier one's, as its secret does. */ export async function recordNamespaceKeys( stateFilePath: string, @@ -290,6 +310,7 @@ export async function recordNamespaceKeys( namespace: string, servers: Iterable, idpSessions: Iterable, + owners: ClientCredentialOwners = new Map(), ): Promise { try { const ledger = parseLedger( @@ -304,7 +325,7 @@ export async function recordNamespaceKeys( let keys = byLocation.get(location); let changed = false; if (!keys) { - keys = { servers: new Set(), idpSessions: new Set() }; + keys = emptyKeys(); byLocation.set(location, keys); changed = true; } @@ -318,6 +339,7 @@ export async function recordNamespaceKeys( keys.idpSessions.add(issuer); changed = true; } + if (mergeOwners(keys.owners, owners)) changed = true; if (changed) await writeLedger(stateFilePath, ledger); } catch (error) { warnLedgerFailure("update", error); @@ -326,121 +348,94 @@ export async function recordNamespaceKeys( /** * Mark every record of a namespace other than `into` that lives somewhere - * other than `secretStore` as superseded by `into` without having been - * carried into it (#2672) — adoption's carry-over reads through this run's - * store only. `owners` is what the state file being adopted holds: for - * each server, the `client_id` each client-credential field belongs to. - * A record already marked keeps the owners it recorded first, since those - * name the client its secret was issued to. + * other than `secretStore`, and holds a client credential, as owed to + * `into` (#2672): adoption's carry-over reads through this run's store + * only, so those records were not carried into it. * - * Call it before the store writes the adoption makes, like - * {@link recordNamespaceKeys}: a crash after them but before the mark - * would leave the record unmarked, and its secret purged once its - * location's run sees the stamp. + * Unlike the rest of the ledger this throws, and adoption fails with it: + * call it before the stamp commits, because an adoption that commits + * without the mark leaves the record looking disposable, and the first + * save on its backend would purge the only copy of its secret. */ export async function markUncarriedNamespaces( stateFilePath: string, secretStore: SecretStore, into: string, - owners: ClientCredentialOwners, ): Promise { - try { - const ledger = parseLedger( - await readStoreFile(namespaceLedgerPath(stateFilePath)), - ); - const location = await secretStoreLocation(secretStore); - let changed = false; - for (const [namespace, byLocation] of ledger) { - if (namespace === into) continue; - for (const [recorded, keys] of byLocation) { - if (recorded === location) continue; - const clients: ClientCredentialOwners = - keys.carry?.clients ?? new Map(); - for (const url of keys.servers) { - const current = owners.get(url); - if (!current) continue; - const kept = clients.get(url) ?? new Map(); - for (const [field, clientId] of current) { - if (kept.has(field)) continue; - kept.set(field, clientId); - changed = true; - } - clients.set(url, kept); - } - // Nothing the file holds to carry: leave the record unmarked. - if (clients.size === 0) continue; - if (keys.carry?.into !== into) changed = true; - keys.carry = { into, clients }; - } + const ledger = parseLedger( + await readStoreFile(namespaceLedgerPath(stateFilePath)), + ); + const location = await secretStoreLocation(secretStore); + let changed = false; + for (const [namespace, byLocation] of ledger) { + if (namespace === into) continue; + for (const [recorded, keys] of byLocation) { + if (recorded === location || keys.owners.size === 0) continue; + if (keys.owedTo === into) continue; + keys.owedTo = into; + changed = true; } - if (changed) await writeLedger(stateFilePath, ledger); - } catch (error) { - warnLedgerFailure("update", error); } + if (changed) await writeLedger(stateFilePath, ledger); } /** * The records a run in `secretStore`'s location still owes `current` a - * carry-over from (#2672), newest first: each one's namespace and the - * client-credential fields it may hold, with the `client_id` each belonged - * to. A stamped file's reads fall back to these — for those fields only, - * and only where the file still holds a client with that `client_id` — so - * the state a save starts from does not drop a secret the purge is about - * to carry. + * carry-over from (#2672), newest first: each one's namespace and, per + * server, the `client_id` each client credential it holds belongs to. A + * stamped file's reads fall back to these — for those fields only, and + * only where the file still holds a client with that `client_id` — so the + * state a save starts from does not drop a secret the purge is about to + * carry. + * + * Throws on an unreadable ledger, like the strict store reads beside it: + * reading as empty would present an owed secret as absent, and the save + * that follows would delete it right after the purge carried it. */ export async function pendingCarries( stateFilePath: string, secretStore: SecretStore, current: string, -): Promise> { - try { - const ledger = parseLedger( - await readStoreFile(namespaceLedgerPath(stateFilePath)), - ); - const location = await secretStoreLocation(secretStore); - const carries: Array<{ - namespace: string; - clients: ClientCredentialOwners; - }> = []; - for (const [namespace, byLocation] of ledger) { - for (const { keys } of coveredLocations(location, byLocation)) { - if (keys.carry?.into === current) { - carries.push({ namespace, clients: keys.carry.clients }); - } +): Promise> { + const ledger = parseLedger( + await readStoreFile(namespaceLedgerPath(stateFilePath)), + ); + const location = await secretStoreLocation(secretStore); + const carries: Array<{ namespace: string; owners: ClientCredentialOwners }> = + []; + for (const [namespace, byLocation] of ledger) { + for (const { keys } of coveredLocations(location, byLocation)) { + if (keys.owedTo === current) { + carries.push({ namespace, owners: keys.owners }); } } - return carries.reverse(); - } catch (error) { - warnLedgerFailure("read", error); - return []; } + return carries.reverse(); } /** - * Copy what a pending record holds into `current` before it is purged - * (#2672): each field it was marked with, into the same server's id under - * `current`, only where `current` holds nothing for it and the state file - * still holds a client with the `client_id` the field belonged to. Records - * the `current` key first, like every other write the ledger tracks. - * Returns whether every carry succeeded; a failure leaves the record for - * the next locked save. + * Copy what an owed record holds into `current` before it is purged + * (#2672): each client credential it recorded, into the same server's id + * under `current`, only where `current` holds nothing for it and the state + * file (`now`) still holds a client with the `client_id` it belongs to. + * Records the key and its owner under `current` first, like every other + * write the ledger tracks. Returns whether every carry succeeded; a + * failure leaves the record for the next locked save. */ -async function carryPending( +async function carryOwed( stateFilePath: string, ledger: Ledger, location: string, secretStore: SecretStore, namespace: string, current: string, - carry: PendingCarry, - owners: ClientCredentialOwners, + keys: LedgerKeys, + now: ClientCredentialOwners, ): Promise { let carried = true; - for (const [url, fields] of carry.clients) { - const now = owners.get(url); - if (!now) continue; + for (const [url, fields] of keys.owners) { for (const [field, clientId] of fields) { - if (now.get(field) !== clientId) continue; + if (now.get(url)?.get(field) !== clientId) continue; try { const from = oauthSecretServerId(url, namespace); const to = oauthSecretServerId(url, current); @@ -454,13 +449,12 @@ async function carryPending( byLocation = new Map(); ledger.set(current, byLocation); } - const keys = byLocation.get(location) ?? { - servers: new Set(), - idpSessions: new Set(), - }; - byLocation.set(location, keys); - if (!keys.servers.has(url)) { - keys.servers.add(url); + const target = byLocation.get(location) ?? emptyKeys(); + byLocation.set(location, target); + const isNew = !target.servers.has(url); + target.servers.add(url); + const owner = new Map([[url, new Map([[field, clientId]])]]); + if (mergeOwners(target.owners, owner) || isNew) { await writeLedger(stateFilePath, ledger); } await secretStore.set(to, field, value); @@ -520,7 +514,9 @@ export async function recordedNamespaces( * A record an adoption could not read and marked as owed to `current` is * carried into it first, for the clients `owners` — what the state file * holds now — still matches (see {@link markUncarriedNamespaces}); a - * failed carry leaves the record unpurged, for the same retry. + * failed carry leaves the record unpurged, for the same retry. Records are + * visited newest first, so where two owe the same field the newer value + * is the one carried, as a read would have seen it. * * Only call this under the real file lock: it deletes credentials, and an * unlocked caller could be racing an adopter whose namespace is recorded @@ -544,7 +540,7 @@ export async function purgeSupersededNamespaces( return; } let changed = false; - for (const [namespace, byLocation] of ledger) { + for (const [namespace, byLocation] of [...ledger].reverse()) { if (namespace === current) continue; for (const { location: recorded, keys } of coveredLocations( location, @@ -552,15 +548,15 @@ export async function purgeSupersededNamespaces( )) { if ( current !== undefined && - keys.carry?.into === current && - !(await carryPending( + keys.owedTo === current && + !(await carryOwed( stateFilePath, ledger, location, secretStore, namespace, current, - keys.carry, + keys, owners, )) ) { @@ -589,14 +585,15 @@ export async function purgeSupersededNamespaces( if (!purged || recorded !== location) continue; byLocation.delete(recorded); // A file's entries may also have been copied into the keychain by a - // hand-off this ledger never saw; keep tracking them there. + // hand-off this ledger never saw; keep tracking them there — owners + // and any carry still owed included, since a copy absorbed before + // this run is one this run's carry could not find (#2672). if (recorded.startsWith(FILE_LOCATION_PREFIX)) { - const keyring = byLocation.get(KEYRING_LOCATION) ?? { - servers: new Set(), - idpSessions: new Set(), - }; + const keyring = byLocation.get(KEYRING_LOCATION) ?? emptyKeys(); for (const url of keys.servers) keyring.servers.add(url); for (const issuer of keys.idpSessions) keyring.idpSessions.add(issuer); + mergeOwners(keyring.owners, keys.owners); + keyring.owedTo ??= keys.owedTo; byLocation.set(KEYRING_LOCATION, keyring); } changed = true; diff --git a/core/auth/node/oauth-persist-file.ts b/core/auth/node/oauth-persist-file.ts index c347bca5a..247c2132b 100644 --- a/core/auth/node/oauth-persist-file.ts +++ b/core/auth/node/oauth-persist-file.ts @@ -385,10 +385,11 @@ function warnNamespaceCleanupFailure(error: unknown): void { * The carry-over reads through this run's store only, so a superseded * namespace's entries in another backend (a keychain this run could not * unlock, say) are not carried. Adoption marks those records in the - * ledger instead, with the `client_id` each client credential belonged to - * (#2672); a later locked save on that backend carries them into the - * stamped namespace — where the file still holds the same client — before - * its purge deletes them, and its reads see them in the meantime. + * ledger as owed to the new namespace instead (#2672), and fails if it + * cannot; a later locked save on that backend carries their client + * credentials into the stamped namespace — where the file still holds the + * client each was written for — before its purge deletes them, and its + * reads see them in the meantime. */ async function adoptSecretsNamespace( filePath: string, @@ -437,17 +438,20 @@ async function adoptSecretsNamespace( // Recorded before the copies, so an interrupted move's copies are still // findable if this namespace is later superseded. + const owners = snapshotClientOwners(snapshot); await recordNamespaceKeys( filePath, secretStore, namespace, Object.keys(snapshot.servers), Object.keys(snapshot.idpSessions), + owners, ); // Before the copies too: what this run's store cannot show it is still - // owed to the new namespace by a run on the store that holds it. - const owners = snapshotClientOwners(snapshot); - await markUncarriedNamespaces(filePath, secretStore, namespace, owners); + // owed to the new namespace by a run on the store that holds it. A + // failure here fails the adoption — committing without the mark would + // let that run purge it uncarried. + await markUncarriedNamespaces(filePath, secretStore, namespace); // Rollback baseline: the scoped ids are vacant before this call — the // namespace is a UUID minted moments ago, so nothing can already live // under it — which makes "restore" simply "delete what we copied". @@ -974,6 +978,7 @@ async function writeOAuthSectionsFrom( namespace, Object.keys(merged.servers), Object.keys(merged.idpSessions), + snapshotClientOwners(merged), ); for (const url of effective.servers ?? []) { @@ -1247,8 +1252,8 @@ type PendingCarry = Awaited>[number]; * * `carries` is non-empty only for a stamped file with records in this * store an adoption elsewhere could not read (#2672, `pendingCarries`): - * each supplies only the client-credential fields it was marked with, - * and only for a client the file still holds under the same `client_id`. + * each supplies only the client-credential fields it recorded an owner + * for, and only for a client the file still holds under that `client_id`. * * Strict: this read hydrates the memory state that later sectioned writes * diff against, so a tolerant read during a store outage would present @@ -1284,9 +1289,9 @@ async function readEntrySecrets( } } const owners = snapshotClientOwners(snapshot); - for (const { namespace: from, clients } of carries) { + for (const { namespace: from, owners: recorded } of carries) { const requests: SecretBulkRequest[] = []; - for (const [url, fields] of clients) { + for (const [url, fields] of recorded) { const target = servers.get(url); if (!target) continue; const wanted = [...fields] @@ -1388,6 +1393,7 @@ async function migratePlaintextSecrets( namespace, Object.keys(fresh.servers), Object.keys(fresh.idpSessions), + snapshotClientOwners(fresh), ); } const migrateEntrySecrets = async ( diff --git a/docs/secret-storage.md b/docs/secret-storage.md index ad32a85ca..d4fc3181b 100644 --- a/docs/secret-storage.md +++ b/docs/secret-storage.md @@ -17,7 +17,7 @@ These values are stored as secrets: They are kept out of `mcp.json` so that sharing, committing or syncing the file does not leak credentials (#1356). When the Inspector saves an entry to a durable store, it leaves each `env` key in `mcp.json` with an empty value and omits the client secret; the real values live in the store. `headers` are **not** moved: they are saved in `mcp.json` exactly as written, so a header that carries a credential stays in the file. [MCP server configuration](./mcp-server-configuration.md) describes what that means for other tools reading the same file. -Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace, noting the `client_id` each dynamically registered client's `client_secret` and registration token belonged to. The next run on their backend reads them, and its first locked save copies them into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the state file still holds a client with the same `client_id`, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). +Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also records in the ledger the `client_id` that every dynamically registered client's `client_secret` and registration token it writes belongs to. The next run on the owed entries' backend reads them, and its first locked save copies them into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the state file still holds a client with the `client_id` recorded when the secret was written, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. Entries written before this was recorded carry no owner, so they are purged as before. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). ## How the store is chosen From b37c6f60b1bfdd4756567d4b379b3f5668edcdfd Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 22:22:14 -0400 Subject: [PATCH 12/44] fix(auth): trust the keychain's owner, stop at a failed carry, record only written owners Round-2 review of #2731: - In a keychain run, a covered secrets file's owners are overlaid by the keychain record's own, since the hand-off keeps a value the keychain already holds; the file-to-keychain ledger fold keeps them that way. - A failed carry ends the purge pass, so an older owed record cannot fill the field the newer one still owes. - A save records owners only for the entries it writes, so another backend's registration cannot relabel a secret this store holds. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 59 +++++++++++++++++++ .../auth/node/oauth-secrets-namespace.test.ts | 44 ++++++++++++++ core/auth/node/oauth-namespace-ledger.ts | 56 ++++++++++++++---- core/auth/node/oauth-persist-file.ts | 13 +++- 4 files changed, 161 insertions(+), 11 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index 2546a8ce6..35d49e212 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -785,6 +785,65 @@ describe("pending carry-over (#2672)", () => { expect(await store.get(from, SECRET)).toBeNull(); }); + it("ends the pass at a failed carry, so an older record cannot take the newer one's field", async () => { + await owed("c", "older"); + await recordNamespaceKeys( + stateFile, + store, + NS3, + [SERVER], + [], + owners("c"), + ); + await store.set(oauthSecretServerId(SERVER, NS3), SECRET, "newer"); + await markUncarriedNamespaces(stateFile, elsewhere, NS2); + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + vi.spyOn(store, "set").mockRejectedValueOnce(new Error("busy")); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBeNull(); + expect(await store.get(from, SECRET)).toBe("older"); + expect(warn).toHaveBeenCalled(); + + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("newer"); + }); + + it("in a keychain run, trusts the keychain's owner over a secrets file's", async () => { + // The hand-off keeps a value the keychain already holds, so where + // both record one, the keychain's value is its own record's client's. + const keyring = new FakeKeyring(); + const fileStore = new FileSecretStore({ + filePath: join(tempDir, "secrets.json"), + passphrase: "", + }); + await recordNamespaceKeys( + stateFile, + keyring, + NS1, + [SERVER], + [], + owners("a"), + ); + await recordNamespaceKeys( + stateFile, + fileStore, + NS1, + [SERVER], + [], + owners("b"), + ); + await markUncarriedNamespaces(stateFile, store, NS2); + const [carry] = await pendingCarries(stateFile, keyring, NS2); + expect([...carry.owners.get(SERVER)!]).toEqual([[SECRET, "a"]]); + + // And the file run's hand-off of its record keeps the keychain's. + vi.spyOn(fileStore, "deleteAllForServer").mockResolvedValue(undefined); + await purgeSupersededNamespaces(stateFile, fileStore, NS2, owners("b")); + expect(readLedger()[NS1].keyring.owners).toEqual({ + [SERVER]: { [SECRET]: "a" }, + }); + }); + it("hands a secrets file's owed record to the keychain, mark and owners included", async () => { // Its entries may have been absorbed into the keychain before this // file run, which then finds nothing to carry. diff --git a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts index c2b495f6a..ff4a4b1ce 100644 --- a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts +++ b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts @@ -1242,6 +1242,50 @@ describe("a stripped stamp re-adopted on another backend keeps what only the old ).toBeNull(); }); + it("records owners only for the entries a save writes", async () => { + // A keychain save for client A, then a fallback-backend save that + // re-registers as B under the same stamp: a later keychain save of an + // unrelated server must not relabel the keychain's A secret as B's. + const keyring = new KeyringSecretStore(); + await writeOAuthSections( + fileA, + confidentialClient("client-a", "secret-a"), + undefined, + keyring, + ); + await flushStoreFileWrites(fileA); + const ns1 = namespaceOf(fileA); + await writeOAuthSections( + fileA, + confidentialClient("client-b", "secret-b"), + { servers: [SERVER] }, + new InMemorySecretStore(), + ); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + const ledger = JSON.parse( + readFileSync(namespaceLedgerPath(fileA), "utf8"), + ) as { + namespaces: Record< + string, + Record> }> + >; + }; + expect(ledger.namespaces[ns1].keyring.owners?.[SERVER]).toEqual({ + [SECRET_FIELD]: "client-a", + [`registration-token:${ISSUER}`]: "client-a", + }); + expect(ledger.namespaces[ns1].memory.owners?.[SERVER]).toEqual({ + [SECRET_FIELD]: "client-b", + [`registration-token:${ISSUER}`]: "client-b", + }); + }); + it("does not resurrect the tokens, only the client's credentials", async () => { const keyring = new KeyringSecretStore(); const { ns2 } = await readoptElsewhere(keyring, new InMemorySecretStore()); diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index 87ccc28b6..4a3780000 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -157,6 +157,28 @@ const emptyKeys = (): LedgerKeys => ({ owners: new Map(), }); +/** + * The owners a run in `location` should trust for one namespace's + * credentials (#2672): a covered secrets file's, overlaid by its own + * record's. The file-to-keychain hand-off keeps a value the keychain + * already holds, so where both record an owner for a field, the value the + * keychain holds is the keychain record's client's. + */ +function effectiveOwners( + location: string, + byLocation: Map, +): ClientCredentialOwners { + const owners: ClientCredentialOwners = new Map(); + for (const { location: recorded, keys } of coveredLocations( + location, + byLocation, + )) { + if (recorded !== location) mergeOwners(owners, keys.owners); + } + mergeOwners(owners, byLocation.get(location)?.owners ?? new Map()); + return owners; +} + /** Merge `source` into `target`, the later owner winning; reports a change. */ function mergeOwners( target: ClientCredentialOwners, @@ -404,10 +426,14 @@ export async function pendingCarries( const carries: Array<{ namespace: string; owners: ClientCredentialOwners }> = []; for (const [namespace, byLocation] of ledger) { - for (const { keys } of coveredLocations(location, byLocation)) { - if (keys.owedTo === current) { - carries.push({ namespace, owners: keys.owners }); - } + const owed = coveredLocations(location, byLocation).some( + ({ keys }) => keys.owedTo === current, + ); + if (owed) { + carries.push({ + namespace, + owners: effectiveOwners(location, byLocation), + }); } } return carries.reverse(); @@ -429,11 +455,11 @@ async function carryOwed( secretStore: SecretStore, namespace: string, current: string, - keys: LedgerKeys, + recorded: ClientCredentialOwners, now: ClientCredentialOwners, ): Promise { let carried = true; - for (const [url, fields] of keys.owners) { + for (const [url, fields] of recorded) { for (const [field, clientId] of fields) { if (now.get(url)?.get(field) !== clientId) continue; try { @@ -516,7 +542,9 @@ export async function recordedNamespaces( * holds now — still matches (see {@link markUncarriedNamespaces}); a * failed carry leaves the record unpurged, for the same retry. Records are * visited newest first, so where two owe the same field the newer value - * is the one carried, as a read would have seen it. + * is the one carried, as a read would have seen it — and a failed carry + * ends the pass, so an older record cannot fill the field the newer one + * still owes it. * * Only call this under the real file lock: it deletes credentials, and an * unlocked caller could be racing an adopter whose namespace is recorded @@ -540,8 +568,11 @@ export async function purgeSupersededNamespaces( return; } let changed = false; + let carryFailed = false; for (const [namespace, byLocation] of [...ledger].reverse()) { + if (carryFailed) break; if (namespace === current) continue; + const trusted = effectiveOwners(location, byLocation); for (const { location: recorded, keys } of coveredLocations( location, byLocation, @@ -556,11 +587,12 @@ export async function purgeSupersededNamespaces( secretStore, namespace, current, - keys, + trusted, owners, )) ) { - continue; + carryFailed = true; + break; } const purges = [ ...[...keys.servers].map( @@ -592,7 +624,11 @@ export async function purgeSupersededNamespaces( const keyring = byLocation.get(KEYRING_LOCATION) ?? emptyKeys(); for (const url of keys.servers) keyring.servers.add(url); for (const issuer of keys.idpSessions) keyring.idpSessions.add(issuer); - mergeOwners(keyring.owners, keys.owners); + // The keychain's own owner wins, as its value does in the hand-off. + const merged: ClientCredentialOwners = new Map(); + mergeOwners(merged, keys.owners); + mergeOwners(merged, keyring.owners); + keyring.owners = merged; keyring.owedTo ??= keys.owedTo; byLocation.set(KEYRING_LOCATION, keyring); } diff --git a/core/auth/node/oauth-persist-file.ts b/core/auth/node/oauth-persist-file.ts index 247c2132b..0b0549f25 100644 --- a/core/auth/node/oauth-persist-file.ts +++ b/core/auth/node/oauth-persist-file.ts @@ -978,7 +978,18 @@ async function writeOAuthSectionsFrom( namespace, Object.keys(merged.servers), Object.keys(merged.idpSessions), - snapshotClientOwners(merged), + // Only the entries this save writes: another entry's secret in + // this store was written for whatever client it was written for, + // and relabeling it from the file would misattribute it (#2672). + snapshotClientOwners({ + servers: Object.fromEntries( + (effective.servers ?? []).flatMap((url) => { + const entry = getOwnEntry(merged.servers, url); + return entry === undefined ? [] : [[url, entry]]; + }), + ), + idpSessions: {}, + }), ); for (const url of effective.servers ?? []) { From cde08e2ec6cf3e764631a721f91fda3af3dddf0e Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 22:38:26 -0400 Subject: [PATCH 13/44] fix(auth): record owners only for held values, and skip ambiguous hand-off owners MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-3 review of #2731: - A write records a client credential's owner only for a field that holds a value, so a later value filled into that field by the file-to-keychain hand-off is never labeled as the earlier client's. Adoption reads the values it will copy before recording them. - For a server the keychain record has been written for, only its own owners are trusted — in reads, carries and the ledger fold. A field it records no owner for may hold its own pre-owner value or one the hand-off filled in, so nothing is carried rather than guessed. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 28 +++++++ .../auth/node/oauth-secrets-namespace.test.ts | 21 ++++- core/auth/node/oauth-namespace-ledger.ts | 42 +++++++--- core/auth/node/oauth-persist-file.ts | 79 +++++++++++++------ docs/secret-storage.md | 2 +- 5 files changed, 134 insertions(+), 38 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index 35d49e212..c0d6f2a93 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -844,6 +844,34 @@ describe("pending carry-over (#2672)", () => { }); }); + it("in a keychain run, carries nothing a secrets file names an owner for when the keychain record's own is unknown", async () => { + // Written to the keychain before owners were kept (or for a client + // with no secret): the hand-off may have kept that value or filled + // the field from the file, and nothing says which. + const keyring = new FakeKeyring(); + const fileStore = new FileSecretStore({ + filePath: join(tempDir, "secrets.json"), + passphrase: "", + }); + await recordNamespaceKeys(stateFile, keyring, NS1, [SERVER], []); + await recordNamespaceKeys( + stateFile, + fileStore, + NS1, + [SERVER], + [], + owners("b"), + ); + await markUncarriedNamespaces(stateFile, store, NS2); + const [carry] = await pendingCarries(stateFile, keyring, NS2); + expect(carry.owners.size).toBe(0); + + // The file run's hand-off does not label the keychain's field either. + vi.spyOn(fileStore, "deleteAllForServer").mockResolvedValue(undefined); + await purgeSupersededNamespaces(stateFile, fileStore, NS2, owners("b")); + expect(readLedger()[NS1].keyring.owners).toBeUndefined(); + }); + it("hands a secrets file's owed record to the keychain, mark and owners included", async () => { // Its entries may have been absorbed into the keychain before this // file run, which then finds nothing to carry. diff --git a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts index ff4a4b1ce..45af9ce5b 100644 --- a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts +++ b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts @@ -1278,14 +1278,31 @@ describe("a stripped stamp re-adopted on another backend keeps what only the old }; expect(ledger.namespaces[ns1].keyring.owners?.[SERVER]).toEqual({ [SECRET_FIELD]: "client-a", - [`registration-token:${ISSUER}`]: "client-a", }); expect(ledger.namespaces[ns1].memory.owners?.[SERVER]).toEqual({ [SECRET_FIELD]: "client-b", - [`registration-token:${ISSUER}`]: "client-b", }); }); + it("records no owner for a client with no secret", async () => { + const keyring = new KeyringSecretStore(); + await writeOAuthSections( + fileA, + confidentialClient("public-client"), + undefined, + keyring, + ); + await flushStoreFileWrites(fileA); + const ledger = JSON.parse( + readFileSync(namespaceLedgerPath(fileA), "utf8"), + ) as { + namespaces: Record>; + }; + expect(ledger.namespaces[namespaceOf(fileA)].keyring.owners).toBe( + undefined, + ); + }); + it("does not resurrect the tokens, only the client's credentials", async () => { const keyring = new KeyringSecretStore(); const { ns2 } = await readoptElsewhere(keyring, new InMemorySecretStore()); diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index 4a3780000..16f77c604 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -157,26 +157,44 @@ const emptyKeys = (): LedgerKeys => ({ owners: new Map(), }); +/** + * Fold `other`'s owners into `own`'s, for a server `own` has never been + * written for only (#2672). Where `own` has, the value its store holds + * for a field may be its own write's or one a hand-off filled in, and its + * owners alone say which: an owner names the client of a value it wrote, + * and a missing one — a vacant field, or a record from before owners were + * kept — is ambiguous, so nothing is carried rather than guessed. + */ +function foldOwners( + own: LedgerKeys | undefined, + other: ClientCredentialOwners, +): ClientCredentialOwners { + const owners: ClientCredentialOwners = new Map(); + for (const [url, fields] of other) { + if (!own?.servers.has(url)) owners.set(url, new Map(fields)); + } + mergeOwners(owners, own?.owners ?? new Map()); + return owners; +} + /** * The owners a run in `location` should trust for one namespace's - * credentials (#2672): a covered secrets file's, overlaid by its own - * record's. The file-to-keychain hand-off keeps a value the keychain - * already holds, so where both record an owner for a field, the value the - * keychain holds is the keychain record's client's. + * credentials (#2672): its own record's, plus a covered secrets file's + * for servers its own record was never written for — a value the + * file-to-keychain hand-off can only have brought from that file. */ function effectiveOwners( location: string, byLocation: Map, ): ClientCredentialOwners { - const owners: ClientCredentialOwners = new Map(); + const fromFiles: ClientCredentialOwners = new Map(); for (const { location: recorded, keys } of coveredLocations( location, byLocation, )) { - if (recorded !== location) mergeOwners(owners, keys.owners); + if (recorded !== location) mergeOwners(fromFiles, keys.owners); } - mergeOwners(owners, byLocation.get(location)?.owners ?? new Map()); - return owners; + return foldOwners(byLocation.get(location), fromFiles); } /** Merge `source` into `target`, the later owner winning; reports a change. */ @@ -622,13 +640,11 @@ export async function purgeSupersededNamespaces( // this run is one this run's carry could not find (#2672). if (recorded.startsWith(FILE_LOCATION_PREFIX)) { const keyring = byLocation.get(KEYRING_LOCATION) ?? emptyKeys(); + // Before the servers merge, which would make every one look like + // the keychain's own. + keyring.owners = foldOwners(keyring, keys.owners); for (const url of keys.servers) keyring.servers.add(url); for (const issuer of keys.idpSessions) keyring.idpSessions.add(issuer); - // The keychain's own owner wins, as its value does in the hand-off. - const merged: ClientCredentialOwners = new Map(); - mergeOwners(merged, keys.owners); - mergeOwners(merged, keyring.owners); - keyring.owners = merged; keyring.owedTo ??= keys.owedTo; byLocation.set(KEYRING_LOCATION, keyring); } diff --git a/core/auth/node/oauth-persist-file.ts b/core/auth/node/oauth-persist-file.ts index 0b0549f25..090261a0d 100644 --- a/core/auth/node/oauth-persist-file.ts +++ b/core/auth/node/oauth-persist-file.ts @@ -436,16 +436,26 @@ async function adoptSecretsNamespace( ); } + // The legacy ids first, then the namespaces this file used to carry for + // whatever the legacy ids lack (#2630) — a stripped stamp leaves a + // confidential client's secret only there. Read before anything is + // written, so the owners recorded below are those of values that exist. + const values = await readEntrySecrets( + snapshot, + secretStore, + undefined, + await recordedNamespaces(filePath), + ); + const owners = snapshotClientOwners(snapshot); // Recorded before the copies, so an interrupted move's copies are still // findable if this namespace is later superseded. - const owners = snapshotClientOwners(snapshot); await recordNamespaceKeys( filePath, secretStore, namespace, Object.keys(snapshot.servers), Object.keys(snapshot.idpSessions), - owners, + writtenClientOwners(snapshot, (url) => values.servers.get(url) ?? {}), ); // Before the copies too: what this run's store cannot show it is still // owed to the new namespace by a run on the store that holds it. A @@ -457,15 +467,6 @@ async function adoptSecretsNamespace( // under it — which makes "restore" simply "delete what we copied". const touched: SecretFieldSnapshot[] = []; try { - // The legacy ids first, then the namespaces this file used to carry - // for whatever the legacy ids lack (#2630) — a stripped stamp leaves a - // confidential client's secret only there. - const values = await readEntrySecrets( - snapshot, - secretStore, - undefined, - await recordedNamespaces(filePath), - ); const copies = [ ...[...values.servers].map(([url, secrets]) => ({ scopedId: oauthSecretServerId(url, namespace), @@ -981,15 +982,18 @@ async function writeOAuthSectionsFrom( // Only the entries this save writes: another entry's secret in // this store was written for whatever client it was written for, // and relabeling it from the file would misattribute it (#2672). - snapshotClientOwners({ - servers: Object.fromEntries( - (effective.servers ?? []).flatMap((url) => { - const entry = getOwnEntry(merged.servers, url); - return entry === undefined ? [] : [[url, entry]]; - }), - ), - idpSessions: {}, - }), + writtenClientOwners( + { + servers: Object.fromEntries( + (effective.servers ?? []).flatMap((url) => { + const entry = getOwnEntry(merged.servers, url); + return entry === undefined ? [] : [[url, entry]]; + }), + ), + idpSessions: {}, + }, + splitSecretsOf, + ), ); for (const url of effective.servers ?? []) { @@ -1237,7 +1241,38 @@ function fillMissingSecrets( } } -/** Per server, the `client_id` each client credential belongs to. */ +/** + * Per server, the `client_id` each client credential a write stores + * belongs to (#2672) — only for fields that hold a value. A client with + * no secret records no owner for it, so a later value in that field from + * elsewhere (a hand-off filling a vacant keychain field) is never labeled + * as this client's. `secretsOf` gives the values the write stores. + */ +function writtenClientOwners( + snapshot: OAuthPersistSnapshot, + secretsOf: (url: string, state: ServerOAuthState) => OAuthSecretValues, +): ClientCredentialOwners { + const owners: ClientCredentialOwners = new Map(); + for (const [url, state] of Object.entries(snapshot.servers)) { + const secrets = secretsOf(url, state); + const fields = new Map( + [...clientCredentialOwners(state)].filter(([field]) => + Object.hasOwn(secrets, field), + ), + ); + if (fields.size > 0) owners.set(url, fields); + } + return owners; +} + +/** The store values a joined entry splits into; client secrets ignore the policy. */ +const splitSecretsOf = (_url: string, state: ServerOAuthState) => + splitServerOAuthState(state, "all").secrets; + +/** + * Per server, the `client_id` each client credential belongs to — what + * the state file holds now, for matching a carry against. + */ function snapshotClientOwners( snapshot: OAuthPersistSnapshot | null, ): ClientCredentialOwners { @@ -1404,7 +1439,7 @@ async function migratePlaintextSecrets( namespace, Object.keys(fresh.servers), Object.keys(fresh.idpSessions), - snapshotClientOwners(fresh), + writtenClientOwners(fresh, splitSecretsOf), ); } const migrateEntrySecrets = async ( diff --git a/docs/secret-storage.md b/docs/secret-storage.md index d4fc3181b..4d74ba33a 100644 --- a/docs/secret-storage.md +++ b/docs/secret-storage.md @@ -17,7 +17,7 @@ These values are stored as secrets: They are kept out of `mcp.json` so that sharing, committing or syncing the file does not leak credentials (#1356). When the Inspector saves an entry to a durable store, it leaves each `env` key in `mcp.json` with an empty value and omits the client secret; the real values live in the store. `headers` are **not** moved: they are saved in `mcp.json` exactly as written, so a header that carries a credential stays in the file. [MCP server configuration](./mcp-server-configuration.md) describes what that means for other tools reading the same file. -Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also records in the ledger the `client_id` that every dynamically registered client's `client_secret` and registration token it writes belongs to. The next run on the owed entries' backend reads them, and its first locked save copies them into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the state file still holds a client with the `client_id` recorded when the secret was written, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. Entries written before this was recorded carry no owner, so they are purged as before. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). +Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also records in the ledger the `client_id` that every dynamically registered client's `client_secret` and registration token it writes belongs to. The next run on the owed entries' backend reads them, and its first locked save copies them into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the state file still holds a client with the `client_id` recorded when the secret was written, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. Entries written before this was recorded carry no owner, so they are purged as before — as is a keychain value whose owner the ledger cannot tell apart from one a `secrets.json` hand-off filled in, rather than guessing. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). ## How the store is chosen From 57ee65955f7d63a21f73cf6efbe0ca36a7982ace Mon Sep 17 00:00:00 2001 From: olaservo Date: Mon, 19 Oct 2026 19:51:48 -0700 Subject: [PATCH 14/44] docs: retroactive DCO signoff for olaservo's unsigned commits (#2617) Signed-off-by: olaservo --- docs/dco-retroactive-signoffs.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/dco-retroactive-signoffs.md b/docs/dco-retroactive-signoffs.md index d6c975893..903d7b1c9 100644 --- a/docs/dco-retroactive-signoffs.md +++ b/docs/dco-retroactive-signoffs.md @@ -28,3 +28,17 @@ adds my `Signed-off-by: cliffhall ` to each of them. | `9fd5bc177cb9e40e651756c533fed14526043bc6` | #1983 | chore: drop the orphaned root override, and correct the TUI coverage note | | `94a227b9c12451ee7810f1177e4992ac4713742c` | #1983 | fix(tui): widen the root react range so an external ink can't split React | | `b70aed691418a7e2e98c897cba6ab44ad422f648` | #1987 | test(fix): probe a single-source route when waiting for the dead transport | + +## olaservo + +I, Ola Hungerford, am the author of the commits below. I certify that each of +them was contributed in accordance with the Developer Certificate of Origin, +version 1.1 (https://developercertificate.org/), and this signed-off commit +adds my `Signed-off-by: olaservo ` to each of them. + +| Commit | PR | Subject | +| ------------------------------------------ | ----- | --------------------------------------------------------------------------------------------------------------------- | +| `cf834633ba99a569ae766083bde166646fc13ac6` | #1994 | fix: add eol=lf policy to .gitattributes so Windows checkouts stay clean | +| `9f7895dcd976daca6bbe41efc1b381ec4e82b0c1` | #1995 | fix: invoke the registered SIGINT listener directly in run-method.test instead of emitting a process-wide signal | +| `9c6e4aeb4e827937abbef489ac5cd154a8919d6a` | #1996 | fix: settle ink input-handler re-subscription after frame waits in TUI App tests | +| `e3a026802bc8037afe025ecd8f42ce54c811f26d` | #1997 | fix: resolve node bins via package.json instead of spawning npx .cmd shims so the verify/smoke scripts run on Windows | From 17993dd4acefe64c7745f6b35d0279c30d7b1459 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 22:54:18 -0400 Subject: [PATCH 15/44] fix(auth): carry once per namespace before purging any location it covers Round-4 review of #2731: a keychain run purges a covered secrets file record's ids from the keychain too, so an unowed file record listed ahead of the owed keychain record deleted the values the carry was about to copy. The carry now runs once per namespace, before any of its covered locations is purged. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 38 +++++++++++++++++ core/auth/node/oauth-namespace-ledger.ts | 42 ++++++++++--------- 2 files changed, 60 insertions(+), 20 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index c0d6f2a93..0ae22d7c6 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -872,6 +872,44 @@ describe("pending carry-over (#2672)", () => { expect(readLedger()[NS1].keyring.owners).toBeUndefined(); }); + it("in a keychain run, carries before purging a covered file record listed ahead of the owed one", async () => { + // An unowed `file:` record ahead of the owed keychain record: its + // purge also deletes the server's ids from the keychain, so the + // carry has to run first. + class MapKeyring extends KeyringSecretStore { + readonly inner = new InMemorySecretStore(); + get = this.inner.get.bind(this.inner); + getStrict = this.inner.get.bind(this.inner); + set = this.inner.set.bind(this.inner); + deleteAllForServer = this.inner.deleteAllForServer.bind(this.inner); + } + const keyring = new MapKeyring(); + await recordNamespaceKeys( + stateFile, + new FileSecretStore({ + filePath: join(tempDir, "secrets.json"), + passphrase: "", + }), + NS1, + [SERVER], + [], + ); + await recordNamespaceKeys( + stateFile, + keyring, + NS1, + [SERVER], + [], + owners("c"), + ); + await keyring.set(from, SECRET, "s3cret"); + await markUncarriedNamespaces(stateFile, store, NS2); + + await purgeSupersededNamespaces(stateFile, keyring, NS2, owners("c")); + expect(await keyring.get(to, SECRET)).toBe("s3cret"); + expect(await keyring.get(from, SECRET)).toBeNull(); + }); + it("hands a secrets file's owed record to the keychain, mark and owners included", async () => { // Its entries may have been absorbed into the keychain before this // file run, which then finds nothing to carry. diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index 16f77c604..1d92b2f23 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -586,32 +586,34 @@ export async function purgeSupersededNamespaces( return; } let changed = false; - let carryFailed = false; for (const [namespace, byLocation] of [...ledger].reverse()) { - if (carryFailed) break; if (namespace === current) continue; - const trusted = effectiveOwners(location, byLocation); + // Carry once per namespace, before purging any location it covers: a + // keychain run purges a covered file record's ids from the keychain + // too, so purging an unowed record first would delete the very values + // an owed one is about to carry. + if ( + current !== undefined && + coveredLocations(location, byLocation).some( + ({ keys }) => keys.owedTo === current, + ) && + !(await carryOwed( + stateFilePath, + ledger, + location, + secretStore, + namespace, + current, + effectiveOwners(location, byLocation), + owners, + )) + ) { + break; + } for (const { location: recorded, keys } of coveredLocations( location, byLocation, )) { - if ( - current !== undefined && - keys.owedTo === current && - !(await carryOwed( - stateFilePath, - ledger, - location, - secretStore, - namespace, - current, - trusted, - owners, - )) - ) { - carryFailed = true; - break; - } const purges = [ ...[...keys.servers].map( (url) => () => oauthSecretServerId(url, namespace), From 24952c52c5dfd761837adea2c7d803c97d533236 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 23:29:52 -0400 Subject: [PATCH 16/44] fix(auth): carry a client credential only for the client its store stamp names A repeated secrets.json hand-off could pair a carried client secret with the wrong client (#2734). The carry trusted the owner the namespace ledger recorded per location, but a value reaches the keychain without the ledger seeing it: the hand-off keeps an existing keychain value and copies a file's otherwise, so after a re-registration between two hand-offs the file record named client B while the keychain held A's secret. The same gap made the keychain's cleanup index read as proof of a keychain write, and let several secrets.json files' records override one another in insertion order. Each DCR client credential now carries an owner stamp beside it in the store -- the client_id it was written for plus a digest of the value. The hand-off copies or keeps the stamp exactly as it does the value, so it names the client of the value actually there, and the digest refuses a stamp separated from its value. Carries (the pending read and the purge's copy) verify the stamp against the client the state file holds now; the ledger's owners only name which fields to look in. Adoption stamps the legacy values it moves, so they stay carriable. Closes #2734 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 198 +++++++++++++----- .../src/test/core/auth/oauth-secrets.test.ts | 120 ++++++++++- .../auth/node/oauth-secrets-namespace.test.ts | 157 ++++++++++++++ core/auth/node/oauth-namespace-ledger.ts | 151 +++++++------ core/auth/node/oauth-persist-file.ts | 49 +++-- core/auth/node/oauth-secrets.ts | 134 ++++++++++-- docs/secret-storage.md | 2 +- 7 files changed, 659 insertions(+), 152 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index 0ae22d7c6..13dc835e9 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -2,7 +2,8 @@ * Unit tests for the secrets-namespace ledger (#2560): its tolerant parse, * record-only-when-new writes, per-namespace purge bookkeeping, and the * pending carry-over a record an adoption could not read owes its - * successor (#2672), and the best-effort failure handling that keeps a ledger problem from ever + * successor (#2672) — carried only where the store's owner stamp vouches + * for the value (#2734) — and the best-effort failure handling that keeps a ledger problem from ever * failing the OAuth save or removal it rides on. The end-to-end scenarios * (a stripped stamp re-adopted, a stripped file removed) live in the * integration suite's `oauth-secrets-namespace.test.ts`. @@ -65,6 +66,8 @@ class FakeKeyring extends KeyringSecretStore { import { LEGACY_TOKENS_FIELD, IDP_SESSION_FIELD, + clientCredentialStamp, + clientCredentialStampField, issuerClientSecretField, oauthIdpSecretServerId, oauthSecretServerId, @@ -514,9 +517,31 @@ describe("pending carry-over (#2672)", () => { const SECRET = issuerClientSecretField(ISSUER); const owners = (clientId: string) => new Map([[SERVER, new Map([[SECRET, clientId]])]]); + const STAMP = clientCredentialStampField(SECRET); // Another run's store: marking is about the records it could not read. const elsewhere = new KeyringSecretStore(); + /** The keychain, as far as `instanceof` is concerned, over a map. */ + class MapKeyring extends KeyringSecretStore { + readonly inner = new InMemorySecretStore(); + get = this.inner.get.bind(this.inner); + getStrict = this.inner.get.bind(this.inner); + set = this.inner.set.bind(this.inner); + deleteAllForServer = this.inner.deleteAllForServer.bind(this.inner); + } + + /** Store `secret` under `namespace`, stamped as `clientId`'s, as a save would. */ + async function stamped( + target: { set: InMemorySecretStore["set"] }, + namespace: string, + secret: string, + clientId: string, + ) { + const id = oauthSecretServerId(SERVER, namespace); + await target.set(id, SECRET, secret); + await target.set(id, STAMP, clientCredentialStamp(clientId, secret)); + } + /** NS1's record in this run's store, owed to NS2 by a run elsewhere. */ async function owed(clientId = "c", secret: string | null = "s3cret") { await recordNamespaceKeys( @@ -527,9 +552,7 @@ describe("pending carry-over (#2672)", () => { [], owners(clientId), ); - if (secret !== null) { - await store.set(oauthSecretServerId(SERVER, NS1), SECRET, secret); - } + if (secret !== null) await stamped(store, NS1, secret, clientId); await markUncarriedNamespaces(stateFile, elsewhere, NS2); } @@ -675,11 +698,11 @@ describe("pending carry-over (#2672)", () => { }), ); const carries = await pendingCarries(stateFile, store, NS2); - expect(carries.map((c) => [c.namespace, [...c.owners.keys()]])).toEqual([ + expect(carries.map((c) => [c.namespace, [...c.fields.keys()]])).toEqual([ ["44444444-4444-4444-8444-444444444444", []], [NS3, [SERVER]], ]); - expect([...carries[1].owners.get(SERVER)!]).toEqual([["ok", "c"]]); + expect([...carries[1].fields.get(SERVER)!]).toEqual(["ok"]); }); }); @@ -691,6 +714,10 @@ describe("pending carry-over (#2672)", () => { await owed(); await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); expect(await store.get(to, SECRET)).toBe("s3cret"); + // With its stamp, so it still vouches for the value there. + expect(await store.get(to, STAMP)).toBe( + clientCredentialStamp("c", "s3cret"), + ); expect(await store.get(from, SECRET)).toBeNull(); // The carried key and its owner are recorded under the current // namespace, like every other write the ledger tracks. @@ -729,12 +756,32 @@ describe("pending carry-over (#2672)", () => { [], owners("c"), ); - await store.set(oauthSecretServerId(SERVER, NS3), SECRET, "newer"); + await stamped(store, NS3, "newer", "c"); await markUncarriedNamespaces(stateFile, elsewhere, NS2); await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); expect(await store.get(to, SECRET)).toBe("newer"); }); + it.each([ + ["no stamp", null], + ["a malformed stamp", "{not json"], + ["a stamp that is not an object", "7"], + ["a stamp naming no client", JSON.stringify({ sha256: "x" })], + // The hand-off keeps and copies per account: a kept value can meet a + // copied stamp, and an older Inspector can rewrite a value alone. + ["another value's stamp", clientCredentialStamp("c", "other")], + ])( + "does not carry a value with %s, and still purges", + async (_l, stamp) => { + await owed(); + if (stamp === null) await store.delete(from, STAMP); + else await store.set(from, STAMP, stamp); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBeNull(); + expect(await store.get(from, SECRET)).toBeNull(); + }, + ); + it.each([ ["a different client now", owners("other")], ["no client for that server now", new Map()], @@ -795,7 +842,7 @@ describe("pending carry-over (#2672)", () => { [], owners("c"), ); - await store.set(oauthSecretServerId(SERVER, NS3), SECRET, "newer"); + await stamped(store, NS3, "newer", "c"); await markUncarriedNamespaces(stateFile, elsewhere, NS2); const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); vi.spyOn(store, "set").mockRejectedValueOnce(new Error("busy")); @@ -808,14 +855,13 @@ describe("pending carry-over (#2672)", () => { expect(await store.get(to, SECRET)).toBe("newer"); }); - it("in a keychain run, trusts the keychain's owner over a secrets file's", async () => { - // The hand-off keeps a value the keychain already holds, so where - // both record one, the keychain's value is its own record's client's. + /** A `secrets.json` store at `name`, for its ledger location only. */ + const secretsFile = (name = "secrets.json") => + new FileSecretStore({ filePath: join(tempDir, name), passphrase: "" }); + + it("in a keychain run, looks in every field a covered record wrote, keeping the keychain's own owner", async () => { const keyring = new FakeKeyring(); - const fileStore = new FileSecretStore({ - filePath: join(tempDir, "secrets.json"), - passphrase: "", - }); + const fileStore = secretsFile(); await recordNamespaceKeys( stateFile, keyring, @@ -834,9 +880,9 @@ describe("pending carry-over (#2672)", () => { ); await markUncarriedNamespaces(stateFile, store, NS2); const [carry] = await pendingCarries(stateFile, keyring, NS2); - expect([...carry.owners.get(SERVER)!]).toEqual([[SECRET, "a"]]); + expect([...carry.fields.get(SERVER)!]).toEqual([SECRET]); - // And the file run's hand-off of its record keeps the keychain's. + // The file run's hand-off of its record keeps the keychain's owner. vi.spyOn(fileStore, "deleteAllForServer").mockResolvedValue(undefined); await purgeSupersededNamespaces(stateFile, fileStore, NS2, owners("b")); expect(readLedger()[NS1].keyring.owners).toEqual({ @@ -844,56 +890,103 @@ describe("pending carry-over (#2672)", () => { }); }); - it("in a keychain run, carries nothing a secrets file names an owner for when the keychain record's own is unknown", async () => { - // Written to the keychain before owners were kept (or for a client - // with no secret): the hand-off may have kept that value or filled - // the field from the file, and nothing says which. - const keyring = new FakeKeyring(); - const fileStore = new FileSecretStore({ - filePath: join(tempDir, "secrets.json"), - passphrase: "", - }); - await recordNamespaceKeys(stateFile, keyring, NS1, [SERVER], []); + it("in a keychain run, carries a value absorbed from a re-registered secrets file only for the client it was written for (#2734)", async () => { + // A's secret reached the keychain in one hand-off; the file then + // re-registered B, and the next hand-off kept A's value. The file's + // record says B, but the value is A's. + const keyring = new MapKeyring(); await recordNamespaceKeys( stateFile, - fileStore, + secretsFile(), NS1, [SERVER], [], owners("b"), ); + await stamped(keyring, NS1, "a-secret", "a"); await markUncarriedNamespaces(stateFile, store, NS2); - const [carry] = await pendingCarries(stateFile, keyring, NS2); - expect(carry.owners.size).toBe(0); - // The file run's hand-off does not label the keychain's field either. - vi.spyOn(fileStore, "deleteAllForServer").mockResolvedValue(undefined); - await purgeSupersededNamespaces(stateFile, fileStore, NS2, owners("b")); - expect(readLedger()[NS1].keyring.owners).toBeUndefined(); + await purgeSupersededNamespaces(stateFile, keyring, NS2, owners("b")); + expect(await keyring.get(to, SECRET)).toBeNull(); + expect(await keyring.get(from, SECRET)).toBeNull(); }); - it("in a keychain run, carries before purging a covered file record listed ahead of the owed one", async () => { - // An unowed `file:` record ahead of the owed keychain record: its - // purge also deletes the server's ids from the keychain, so the - // carry has to run first. - class MapKeyring extends KeyringSecretStore { - readonly inner = new InMemorySecretStore(); - get = this.inner.get.bind(this.inner); - getStrict = this.inner.get.bind(this.inner); - set = this.inner.set.bind(this.inner); - deleteAllForServer = this.inner.deleteAllForServer.bind(this.inner); - } + it("in a keychain run, still carries for the client the stamp names when a file record says otherwise", async () => { const keyring = new MapKeyring(); await recordNamespaceKeys( stateFile, - new FileSecretStore({ - filePath: join(tempDir, "secrets.json"), - passphrase: "", - }), + secretsFile(), NS1, [SERVER], [], + owners("b"), ); + await stamped(keyring, NS1, "a-secret", "a"); + await markUncarriedNamespaces(stateFile, store, NS2); + + await purgeSupersededNamespaces(stateFile, keyring, NS2, owners("a")); + expect(await keyring.get(to, SECRET)).toBe("a-secret"); + }); + + it("in a keychain run, carries an absorbed value its own record lists the server for but names no owner of (#2734)", async () => { + // An unrelated keychain save lists every merged server in its record + // but owners only for what it writes; that is no keychain write here. + const keyring = new MapKeyring(); + await recordNamespaceKeys(stateFile, keyring, NS1, [SERVER], []); + await recordNamespaceKeys( + stateFile, + secretsFile(), + NS1, + [SERVER], + [], + owners("c"), + ); + await stamped(keyring, NS1, "s3cret", "c"); + await markUncarriedNamespaces(stateFile, store, NS2); + + await purgeSupersededNamespaces(stateFile, keyring, NS2, owners("c")); + expect(await keyring.get(to, SECRET)).toBe("s3cret"); + }); + + it.each([ + ["the client whose file was absorbed", "a", "a-secret"], + ["the other file's client", "b", null], + ])( + "in a keychain run with two secrets files, carries only for %s (#2734)", + async (_label, now, carried) => { + // F1 holds A's secret and F2 B's under one namespace; the keychain + // absorbed F1's. Records are merged in no order that says which. + const keyring = new MapKeyring(); + await recordNamespaceKeys( + stateFile, + secretsFile("f1.json"), + NS1, + [SERVER], + [], + owners("a"), + ); + await recordNamespaceKeys( + stateFile, + secretsFile("f2.json"), + NS1, + [SERVER], + [], + owners("b"), + ); + await stamped(keyring, NS1, "a-secret", "a"); + await markUncarriedNamespaces(stateFile, store, NS2); + + await purgeSupersededNamespaces(stateFile, keyring, NS2, owners(now)); + expect(await keyring.get(to, SECRET)).toBe(carried); + }, + ); + + it("in a keychain run, carries before purging a covered file record listed ahead of the owed one", async () => { + // An unowed `file:` record ahead of the owed keychain record: its + // purge also deletes the server's ids from the keychain, so the + // carry has to run first. + const keyring = new MapKeyring(); + await recordNamespaceKeys(stateFile, secretsFile(), NS1, [SERVER], []); await recordNamespaceKeys( stateFile, keyring, @@ -902,7 +995,7 @@ describe("pending carry-over (#2672)", () => { [], owners("c"), ); - await keyring.set(from, SECRET, "s3cret"); + await stamped(keyring, NS1, "s3cret", "c"); await markUncarriedNamespaces(stateFile, store, NS2); await purgeSupersededNamespaces(stateFile, keyring, NS2, owners("c")); @@ -913,10 +1006,7 @@ describe("pending carry-over (#2672)", () => { it("hands a secrets file's owed record to the keychain, mark and owners included", async () => { // Its entries may have been absorbed into the keychain before this // file run, which then finds nothing to carry. - const fileStore = new FileSecretStore({ - filePath: join(tempDir, "secrets.json"), - passphrase: "", - }); + const fileStore = secretsFile(); await recordNamespaceKeys( stateFile, fileStore, diff --git a/clients/web/src/test/core/auth/oauth-secrets.test.ts b/clients/web/src/test/core/auth/oauth-secrets.test.ts index cdbb1539e..4ce2d521b 100644 --- a/clients/web/src/test/core/auth/oauth-secrets.test.ts +++ b/clients/web/src/test/core/auth/oauth-secrets.test.ts @@ -27,6 +27,10 @@ import { joinIdpSession, serverSecretFields, snapshotHasPlaintextSecrets, + clientCredentialStamp, + clientCredentialStampField, + restampClientCredentials, + stampedOwner, } from "@inspector/core/auth/node/oauth-secrets.js"; import type { ServerOAuthState } from "@inspector/core/auth/store.js"; import type { OAuthTokens } from "@modelcontextprotocol/client"; @@ -396,20 +400,126 @@ describe("serverSecretFields", () => { [ LEGACY_TOKENS_FIELD, LEGACY_CLIENT_SECRET_FIELD, + clientCredentialStampField(LEGACY_CLIENT_SECRET_FIELD), LEGACY_REGISTRATION_TOKEN_FIELD, + clientCredentialStampField(LEGACY_REGISTRATION_TOKEN_FIELD), PREREG_CLIENT_SECRET_FIELD, PREREG_REGISTRATION_TOKEN_FIELD, ].sort(), ); const issuer = "https://as.example"; - expect(serverSecretFields({ byIssuer: { [issuer]: {} } })).toContain( + const fields = serverSecretFields({ byIssuer: { [issuer]: {} } }); + for (const field of [ issuerTokensField(issuer), - ); - expect(serverSecretFields({ byIssuer: { [issuer]: {} } })).toContain( issuerClientSecretField(issuer), - ); - expect(serverSecretFields({ byIssuer: { [issuer]: {} } })).toContain( + clientCredentialStampField(issuerClientSecretField(issuer)), issuerRegistrationTokenField(issuer), + clientCredentialStampField(issuerRegistrationTokenField(issuer)), + ]) { + expect(fields).toContain(field); + } + }); +}); + +describe("client-credential owner stamps (#2734)", () => { + const issuer = "https://as.example"; + const SECRET = issuerClientSecretField(issuer); + const STAMP = clientCredentialStampField(SECRET); + const dcr = (clientId: string, secret?: string): ServerOAuthState => ({ + byIssuer: { + [issuer]: { + clientInformation: { + client_id: clientId, + ...(secret !== undefined && { client_secret: secret }), + }, + }, + }, + }); + + it("stamps each DCR client credential with its client, and never a pre-registered one", () => { + const { secrets } = splitServerOAuthState( + { + ...dcr("cid", "ics"), + clientInformation: { + client_id: "legacy", + client_secret: "cs", + registration_access_token: "rat", + } as ServerOAuthState["clientInformation"], + preregisteredClientInformation: { + client_id: "pre", + client_secret: "pre-cs", + }, + }, + "all", + ); + expect(stampedOwner(secrets[STAMP], "ics")).toBe("cid"); + expect( + stampedOwner( + secrets[clientCredentialStampField(LEGACY_CLIENT_SECRET_FIELD)], + "cs", + ), + ).toBe("legacy"); + expect( + stampedOwner( + secrets[clientCredentialStampField(LEGACY_REGISTRATION_TOKEN_FIELD)], + "rat", + ), + ).toBe("legacy"); + expect( + secrets[clientCredentialStampField(PREREG_CLIENT_SECRET_FIELD)], + ).toBeUndefined(); + // The join serves the values and ignores the stamps. + expect( + joinServerOAuthState( + { byIssuer: { [issuer]: { clientInformation: { client_id: "cid" } } } }, + secrets, + ).byIssuer![issuer]!.clientInformation, + ).toEqual({ client_id: "cid", client_secret: "ics" }); + }); + + it("stamps nothing for a client with no client_id", () => { + const { secrets } = splitServerOAuthState( + { + clientInformation: { + client_secret: "cs", + } as ServerOAuthState["clientInformation"], + }, + "all", + ); + expect(secrets).toEqual({ [LEGACY_CLIENT_SECRET_FIELD]: "cs" }); + }); + + it.each([ + ["no stamp", undefined], + ["a malformed stamp", "{"], + ["a non-object stamp", "null"], + ["a non-string client_id", JSON.stringify({ client_id: 7, sha256: "" })], + ["another value's stamp", clientCredentialStamp("cid", "other")], + ])("names no owner for %s", (_label, stamp) => { + expect(stampedOwner(stamp, "ics")).toBeUndefined(); + }); + + it("restamps only what no stamp vouches for, keeping a valid one whatever client it names", () => { + const valid = clientCredentialStamp("earlier", "ics"); + expect( + restampClientCredentials(dcr("cid"), { [SECRET]: "ics", [STAMP]: valid }), + ).toEqual({ [SECRET]: "ics", [STAMP]: valid }); + expect( + restampClientCredentials(dcr("cid"), { + [SECRET]: "ics", + [STAMP]: clientCredentialStamp("cid", "other"), + }), + ).toEqual({ + [SECRET]: "ics", + [STAMP]: clientCredentialStamp("cid", "ics"), + }); + expect(restampClientCredentials(dcr("cid"), { [SECRET]: "ics" })).toEqual({ + [SECRET]: "ics", + [STAMP]: clientCredentialStamp("cid", "ics"), + }); + // A stamp with no value beside it vouches for nothing. + expect(restampClientCredentials(dcr("cid"), { [STAMP]: valid })).toEqual( + {}, ); }); }); diff --git a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts index 45af9ce5b..505321bd5 100644 --- a/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts +++ b/clients/web/src/test/integration/auth/node/oauth-secrets-namespace.test.ts @@ -94,6 +94,8 @@ import { IDP_SESSION_FIELD, issuerClientSecretField, resetPersistTokensPolicyWarnings, + clientCredentialStampField, + stampedOwner, } from "@inspector/core/auth/node/oauth-secrets.js"; import { writeStoreFile, @@ -1303,6 +1305,161 @@ describe("a stripped stamp re-adopted on another backend keeps what only the old ); }); + /** + * The issue's (#2734) steps 1–4 and the strip of step 5: client A saved + * to `secrets.json` and absorbed, client B registered in the file under + * the same stamp, absorbed again — the keychain keeps A's value — then + * the stamp stripped and the file re-adopted on a memory backend. + */ + async function repeatedHandOff( + keyring: SecretStore, + reregister: boolean, + ): Promise<{ ns1: string; ns2: string }> { + const secretsFile = join(tempDir, "secrets.json"); + const fileStore = () => + new FileSecretStore({ filePath: secretsFile, passphrase: "" }); + const savedFileEnv = process.env[SECRET_FILE_ENV]; + process.env[SECRET_FILE_ENV] = secretsFile; + vi.spyOn(console, "warn").mockImplementation(() => {}); + try { + await writeOAuthSections( + fileA, + confidentialClient("client-a", "secret-a"), + undefined, + fileStore(), + ); + await flushStoreFileWrites(fileA); + await absorbFileSecretsIntoKeyring(keyring); + await writeOAuthSections( + fileA, + reregister + ? confidentialClient("client-b", "secret-b") + : { servers: { [OTHER]: { scope: "x" } }, idpSessions: {} }, + { servers: [reregister ? SERVER : OTHER] }, + fileStore(), + ); + await flushStoreFileWrites(fileA); + await absorbFileSecretsIntoKeyring(keyring); + expect(existsSync(secretsFile)).toBe(false); + } finally { + if (savedFileEnv === undefined) delete process.env[SECRET_FILE_ENV]; + else process.env[SECRET_FILE_ENV] = savedFileEnv; + } + const ns1 = namespaceOf(fileA); + await stripStamp(fileA); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + new InMemorySecretStore(), + ); + await flushStoreFileWrites(fileA); + return { ns1, ns2: namespaceOf(fileA) }; + } + + it("does not pair a secret a repeated hand-off kept with the client the file registered since (#2734)", async () => { + const keyring = new KeyringSecretStore(); + const { ns1, ns2 } = await repeatedHandOff(keyring, true); + // The keychain kept A's value; the file, and its record, say B. + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBe("secret-a"); + expect(clientOf(await readOAuthStore(fileA, keyring))).toEqual({ + client_id: "client-b", + }); + + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "z" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBeNull(); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + }); + + it("still carries a secret a repeated hand-off kept for the client it was written for (#2734)", async () => { + const keyring = new KeyringSecretStore(); + const { ns1, ns2 } = await repeatedHandOff(keyring, false); + expect(clientOf(await readOAuthStore(fileA, keyring))).toEqual({ + client_id: "client-a", + client_secret: "secret-a", + }); + + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "z" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns2), SECRET_FIELD), + ).toBe("secret-a"); + expect( + await keyring.get(oauthSecretServerId(SERVER, ns1), SECRET_FIELD), + ).toBeNull(); + }); + + it("stamps a legacy secret adoption moves, so a later re-adoption elsewhere can still carry it (#2734)", async () => { + // A pre-#2549 file: no stamp, and its client secret at the shared + // legacy id with no owner stamp beside it. + const keyring = new KeyringSecretStore(); + await writeStoreFile( + fileA, + JSON.stringify({ + servers: { + [SERVER]: { + activeIssuer: ISSUER, + byIssuer: { + [ISSUER]: { + clientInformation: { client_id: "dcr-client" }, + clientRegistrationKind: "dcr", + }, + }, + }, + }, + idpSessions: {}, + }), + ); + await flushStoreFileWrites(fileA); + await keyring.set(oauthSecretServerId(SERVER), SECRET_FIELD, "s3cret"); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "x" } }, idpSessions: {} }, + { servers: [OTHER] }, + keyring, + ); + await flushStoreFileWrites(fileA); + const ns1 = namespaceOf(fileA); + expect( + stampedOwner( + (await keyring.get( + oauthSecretServerId(SERVER, ns1), + clientCredentialStampField(SECRET_FIELD), + )) ?? undefined, + "s3cret", + ), + ).toBe("dcr-client"); + + await stripStamp(fileA); + await writeOAuthSections( + fileA, + { servers: { [OTHER]: { scope: "y" } }, idpSessions: {} }, + { servers: [OTHER] }, + new InMemorySecretStore(), + ); + await flushStoreFileWrites(fileA); + expect(clientOf(await readOAuthStore(fileA, keyring))?.client_secret).toBe( + "s3cret", + ); + }); + it("does not resurrect the tokens, only the client's credentials", async () => { const keyring = new KeyringSecretStore(); const { ns2 } = await readoptElsewhere(keyring, new InMemorySecretStore()); diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index 1d92b2f23..6c634a348 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -38,17 +38,25 @@ * the successor's stamp means no read falls back to it again. Purging it * from its own location later would delete what may be the only copy of a * confidential client's secret. So every record also notes, as each save - * writes it, the `client_id` each client-credential field belongs to, and - * adoption marks every record it could not read as owed to the namespace - * that superseded it. A run in that location then reads, and finally - * copies, those fields into the successor before the purge — but only - * where the state file still holds a client with that same `client_id`. A - * field the successor lacks is otherwise indistinguishable from one - * deliberately cleared since, and a different `client_id` means the secret - * would be paired with the wrong client. The owner is the one recorded at - * write time, never inferred from the file being adopted: an older - * Inspector may have re-registered in between. A record written before - * owners were recorded has none, so nothing is carried from it. + * writes it, which client-credential fields it wrote, and adoption marks + * every record it could not read as owed to the namespace that superseded + * it. A run in that location then reads, and finally copies, those fields + * into the successor before the purge — but only where the state file + * still holds the client the value was written for. A field the successor + * lacks is otherwise indistinguishable from one deliberately cleared + * since, and a different client means the secret would be paired with the + * wrong one. + * + * Which client a value was written for is read from the owner stamp stored + * beside it (#2734, `clientCredentialStampField`), never from the ledger. + * The ledger records each field's `client_id` too, but only as the writer + * saw it in its own location, and a value reaches the keychain without the + * ledger seeing it: a file-to-keychain hand-off copies a file's value only + * where the keychain has none, keeps an older one otherwise, and may be + * absorbing any of several `secrets.json` files. No combination of + * per-location records says which value it kept. The stamp moves with the + * value instead, so a value with no stamp that vouches for it — written + * before stamps existed, or separated from its stamp — is not carried. * * Everything else here is best-effort and never throws: the ledger is a cleanup * aid, not part of the credential path, so a failure warns once and the @@ -73,9 +81,11 @@ import { import { FileSecretStore } from "./file-secret-store.js"; import { resolveConcreteSecretStore } from "./secret-store-selection.js"; import { + clientCredentialStampField, isValidSecretsNamespace, oauthIdpSecretServerId, oauthSecretServerId, + stampedOwner, } from "./oauth-secrets.js"; /** The ledger's path, derived from the state file it belongs to. */ @@ -139,10 +149,15 @@ function coveredLocations( */ export type ClientCredentialOwners = Map>; +/** Per server URL, the client-credential fields a carry may copy (#2672). */ +export type CarryFields = Map>; + /** * The keys one namespace has had store entries written under, the client - * each client credential written there belongs to, and — for a record an - * adoption could not read (#2672) — the namespace it is owed to. + * each client credential written there was written for (which fields a + * carry should look in — the owner itself is read from the store's stamp, + * see the header), and — for a record an adoption could not read (#2672) — + * the namespace it is owed to. */ interface LedgerKeys { servers: Set; @@ -158,43 +173,40 @@ const emptyKeys = (): LedgerKeys => ({ }); /** - * Fold `other`'s owners into `own`'s, for a server `own` has never been - * written for only (#2672). Where `own` has, the value its store holds - * for a field may be its own write's or one a hand-off filled in, and its - * owners alone say which: an owner names the client of a value it wrote, - * and a missing one — a vacant field, or a record from before owners were - * kept — is ambiguous, so nothing is carried rather than guessed. + * The client-credential fields a run in `location` should look in for one + * namespace (#2672): every field any record it covers wrote, its own and a + * covered secrets file's alike, since a hand-off may have brought either + * here. Which of them holds a value worth carrying, and for which client, + * is the store's stamp to say (see the header), so a field listed here + * that was never written costs one empty read. */ -function foldOwners( - own: LedgerKeys | undefined, - other: ClientCredentialOwners, -): ClientCredentialOwners { - const owners: ClientCredentialOwners = new Map(); - for (const [url, fields] of other) { - if (!own?.servers.has(url)) owners.set(url, new Map(fields)); +function candidateFields( + location: string, + byLocation: Map, +): CarryFields { + const fields: CarryFields = new Map(); + for (const { keys } of coveredLocations(location, byLocation)) { + for (const [url, owned] of keys.owners) { + const kept = fields.get(url) ?? new Set(); + for (const field of owned.keys()) kept.add(field); + fields.set(url, kept); + } } - mergeOwners(owners, own?.owners ?? new Map()); - return owners; + return fields; } -/** - * The owners a run in `location` should trust for one namespace's - * credentials (#2672): its own record's, plus a covered secrets file's - * for servers its own record was never written for — a value the - * file-to-keychain hand-off can only have brought from that file. - */ -function effectiveOwners( - location: string, - byLocation: Map, -): ClientCredentialOwners { - const fromFiles: ClientCredentialOwners = new Map(); - for (const { location: recorded, keys } of coveredLocations( - location, - byLocation, - )) { - if (recorded !== location) mergeOwners(fromFiles, keys.owners); +/** Add to `target` each of `source`'s owners for a field it does not name. */ +function addMissingOwners( + target: ClientCredentialOwners, + source: ClientCredentialOwners, +): void { + for (const [url, fields] of source) { + const kept = target.get(url) ?? new Map(); + for (const [field, clientId] of fields) { + if (!kept.has(field)) kept.set(field, clientId); + } + if (kept.size > 0) target.set(url, kept); } - return foldOwners(byLocation.get(location), fromFiles); } /** Merge `source` into `target`, the later owner winning; reports a change. */ @@ -422,11 +434,10 @@ export async function markUncarriedNamespaces( /** * The records a run in `secretStore`'s location still owes `current` a * carry-over from (#2672), newest first: each one's namespace and, per - * server, the `client_id` each client credential it holds belongs to. A - * stamped file's reads fall back to these — for those fields only, and - * only where the file still holds a client with that `client_id` — so the - * state a save starts from does not drop a secret the purge is about to - * carry. + * server, the client-credential fields it may hold. A stamped file's reads + * fall back to these — for those fields only, and only where the value's + * stamp names a client the file still holds — so the state a save starts + * from does not drop a secret the purge is about to carry. * * Throws on an unreadable ledger, like the strict store reads beside it: * reading as empty would present an owed secret as absent, and the save @@ -436,13 +447,12 @@ export async function pendingCarries( stateFilePath: string, secretStore: SecretStore, current: string, -): Promise> { +): Promise> { const ledger = parseLedger( await readStoreFile(namespaceLedgerPath(stateFilePath)), ); const location = await secretStoreLocation(secretStore); - const carries: Array<{ namespace: string; owners: ClientCredentialOwners }> = - []; + const carries: Array<{ namespace: string; fields: CarryFields }> = []; for (const [namespace, byLocation] of ledger) { const owed = coveredLocations(location, byLocation).some( ({ keys }) => keys.owedTo === current, @@ -450,7 +460,7 @@ export async function pendingCarries( if (owed) { carries.push({ namespace, - owners: effectiveOwners(location, byLocation), + fields: candidateFields(location, byLocation), }); } } @@ -460,8 +470,9 @@ export async function pendingCarries( /** * Copy what an owed record holds into `current` before it is purged * (#2672): each client credential it recorded, into the same server's id - * under `current`, only where `current` holds nothing for it and the state - * file (`now`) still holds a client with the `client_id` it belongs to. + * under `current`, only where `current` holds nothing for it and the + * value's stamp names the client the state file (`now`) holds for that + * field (#2734) — the stamp is copied with it, so it still vouches there. * Records the key and its owner under `current` first, like every other * write the ledger tracks. Returns whether every carry succeeded; a * failure leaves the record for the next locked save. @@ -473,18 +484,24 @@ async function carryOwed( secretStore: SecretStore, namespace: string, current: string, - recorded: ClientCredentialOwners, + recorded: CarryFields, now: ClientCredentialOwners, ): Promise { let carried = true; for (const [url, fields] of recorded) { - for (const [field, clientId] of fields) { - if (now.get(url)?.get(field) !== clientId) continue; + for (const field of fields) { + const clientId = now.get(url)?.get(field); + if (clientId === undefined) continue; try { const from = oauthSecretServerId(url, namespace); const to = oauthSecretServerId(url, current); + const stampField = clientCredentialStampField(field); const value = await secretStoreGetStrict(secretStore, from, field); if (value === null) continue; + const stamp = await secretStoreGetStrict(secretStore, from, stampField); + if (stamp === null || stampedOwner(stamp, value) !== clientId) { + continue; + } if ((await secretStoreGetStrict(secretStore, to, field)) !== null) { continue; } @@ -502,6 +519,7 @@ async function carryOwed( await writeLedger(stateFilePath, ledger); } await secretStore.set(to, field, value); + await secretStore.set(to, stampField, stamp); } catch (error) { carried = false; warnLedgerFailure( @@ -604,7 +622,7 @@ export async function purgeSupersededNamespaces( secretStore, namespace, current, - effectiveOwners(location, byLocation), + candidateFields(location, byLocation), owners, )) ) { @@ -637,14 +655,15 @@ export async function purgeSupersededNamespaces( if (!purged || recorded !== location) continue; byLocation.delete(recorded); // A file's entries may also have been copied into the keychain by a - // hand-off this ledger never saw; keep tracking them there — owners - // and any carry still owed included, since a copy absorbed before - // this run is one this run's carry could not find (#2672). + // hand-off this ledger never saw; keep tracking them there — the + // fields a carry should look in and any carry still owed included, + // since a copy absorbed before this run is one this run's carry could + // not find (#2672). Where both name a field the keychain's owner is + // kept: neither decides a carry (the stamp does), but it is the one + // its own location wrote. if (recorded.startsWith(FILE_LOCATION_PREFIX)) { const keyring = byLocation.get(KEYRING_LOCATION) ?? emptyKeys(); - // Before the servers merge, which would make every one look like - // the keychain's own. - keyring.owners = foldOwners(keyring, keys.owners); + addMissingOwners(keyring.owners, keys.owners); for (const url of keys.servers) keyring.servers.add(url); for (const issuer of keys.idpSessions) keyring.idpSessions.add(issuer); keyring.owedTo ??= keys.owedTo; diff --git a/core/auth/node/oauth-persist-file.ts b/core/auth/node/oauth-persist-file.ts index 090261a0d..032487ef1 100644 --- a/core/auth/node/oauth-persist-file.ts +++ b/core/auth/node/oauth-persist-file.ts @@ -84,6 +84,7 @@ import { MAX_WRITE_ATTEMPTS } from "./file-secret-store.js"; import { IDP_SESSION_FIELD, clientCredentialOwners, + clientCredentialStampField, getPersistTokensPolicy, isUsableStoredSecret, isValidSecretsNamespace, @@ -92,10 +93,12 @@ import { newSecretsNamespace, oauthIdpSecretServerId, oauthSecretServerId, + restampClientCredentials, serverSecretFields, snapshotHasPlaintextSecrets, splitIdpSession, splitServerOAuthState, + stampedOwner, type OAuthSecretValues, } from "./oauth-secrets.js"; @@ -468,9 +471,14 @@ async function adoptSecretsNamespace( const touched: SecretFieldSnapshot[] = []; try { const copies = [ + // Restamped: a value read from the legacy ids predates stamps, and a + // fallback's stamp may belong to a value read from elsewhere (#2734). ...[...values.servers].map(([url, secrets]) => ({ scopedId: oauthSecretServerId(url, namespace), - secrets, + secrets: restampClientCredentials( + getOwnEntry(snapshot.servers, url) ?? {}, + secrets, + ), })), ...[...values.idpSessions].map(([issuer, secrets]) => ({ scopedId: oauthIdpSecretServerId(issuer, namespace), @@ -1298,8 +1306,10 @@ type PendingCarry = Awaited>[number]; * * `carries` is non-empty only for a stamped file with records in this * store an adoption elsewhere could not read (#2672, `pendingCarries`): - * each supplies only the client-credential fields it recorded an owner - * for, and only for a client the file still holds under that `client_id`. + * each supplies only the client-credential fields it recorded, and only + * where the value's owner stamp (#2734) names the client the file holds + * for that field now — the stamp, not the ledger, because a hand-off can + * have kept or copied a value no ledger record describes. * * Strict: this read hydrates the memory state that later sectioned writes * diff against, so a tolerant read during a store outage would present @@ -1335,29 +1345,42 @@ async function readEntrySecrets( } } const owners = snapshotClientOwners(snapshot); - for (const { namespace: from, owners: recorded } of carries) { + for (const { namespace: from, fields: recorded } of carries) { const requests: SecretBulkRequest[] = []; for (const [url, fields] of recorded) { const target = servers.get(url); if (!target) continue; - const wanted = [...fields] - .filter( - ([field, clientId]) => - owners.get(url)?.get(field) === clientId && - !Object.hasOwn(target, field), - ) - .map(([field]) => field); + const wanted = [...fields].filter( + (field) => + owners.get(url)?.has(field) === true && !Object.hasOwn(target, field), + ); if (wanted.length > 0) { requests.push({ serverId: oauthSecretServerId(url, from), - fields: wanted, + fields: wanted.flatMap((field) => [ + field, + clientCredentialStampField(field), + ]), }); } } if (requests.length === 0) continue; const values = await secretStoreGetManyStrict(secretStore, requests); for (const [url, target] of servers) { - fillMissingSecrets(target, values[oauthSecretServerId(url, from)]); + const found = values[oauthSecretServerId(url, from)] ?? {}; + for (const [field, clientId] of owners.get(url) ?? []) { + const value = Object.hasOwn(found, field) ? found[field] : undefined; + const stampField = clientCredentialStampField(field); + if ( + value === undefined || + Object.hasOwn(target, field) || + stampedOwner(found[stampField], value) !== clientId + ) { + continue; + } + setOwnEntry(target, field, value); + setOwnEntry(target, stampField, found[stampField]); + } } } return { servers, idpSessions }; diff --git a/core/auth/node/oauth-secrets.ts b/core/auth/node/oauth-secrets.ts index f77a154b5..2181dca85 100644 --- a/core/auth/node/oauth-secrets.ts +++ b/core/auth/node/oauth-secrets.ts @@ -20,7 +20,7 @@ * route); the browser round-trips full snapshots over the authed local API. */ -import { randomUUID } from "node:crypto"; +import { createHash, randomUUID } from "node:crypto"; import { OAuthTokensSchema } from "@modelcontextprotocol/core"; import type { OAuthTokens } from "@modelcontextprotocol/client"; import { setOwnEntry } from "../../storage/own-entry.js"; @@ -144,6 +144,64 @@ export const PREREG_REGISTRATION_TOKEN_FIELD = "prereg-registration-token"; /** One IdP issuer's session tokens (JSON `{ idToken?, refreshToken? }`). */ export const IDP_SESSION_FIELD = "idp-session"; +/** + * Field for the owner stamp stored beside a DCR client credential (#2734): + * which `client_id` the value in `field` was written for. The store does + * not otherwise record that pairing, and a carry-over from a superseded + * namespace needs it — a secret paired with the wrong client is useless. + * + * It lives in the store, beside the value, rather than in the namespace + * ledger because the value moves without the ledger seeing it: the + * file-to-keychain hand-off copies each account it finds unless the + * keychain already holds one, and knows nothing of state files. A stamp in + * the store is copied or kept exactly as its value is, so it names the + * client of the value actually there, however many hand-offs ran. + */ +export const clientCredentialStampField = (field: string): string => + `${field}#owner`; + +const credentialDigest = (value: string): string => + createHash("sha256").update(value).digest("hex"); + +/** + * The stamp naming `clientId` as the owner of `value`. It carries a digest + * of the value so a stamp separated from its value — the hand-off keeps + * and copies per account, so a kept value can meet a copied stamp, and an + * older Inspector can rewrite a value without its stamp — never vouches + * for another value. The digest is as protected as the value beside it. + */ +export function clientCredentialStamp(clientId: string, value: string): string { + return JSON.stringify({ + client_id: clientId, + sha256: credentialDigest(value), + }); +} + +/** + * The `client_id` a stamp names as `value`'s owner, or `undefined` when + * there is no stamp, it is malformed, or it was written for a different + * value — all of which make the owner unknown, and an unknown owner is + * never guessed. + */ +export function stampedOwner( + stamp: string | undefined, + value: string, +): string | undefined { + if (stamp === undefined) return undefined; + let parsed: unknown; + try { + parsed = JSON.parse(stamp); + } catch { + return undefined; + } + if (typeof parsed !== "object" || parsed === null) return undefined; + const { client_id: clientId, sha256 } = parsed as Record; + if (typeof clientId !== "string" || sha256 !== credentialDigest(value)) { + return undefined; + } + return clientId; +} + /** Secret values extracted from one entry, keyed by store field. */ export type OAuthSecretValues = Record; @@ -239,14 +297,20 @@ function splitTokens( * `StoredOAuthClientInformation` in `store.ts`). */ type ClientInfoSecretKeys = { + client_id?: unknown; client_secret?: string; registration_access_token?: string; }; -/** The pair of store fields one `clientInformation` slot splits into. */ +/** + * The pair of store fields one `clientInformation` slot splits into, and + * whether each value is stamped with its owner (DCR slots only — see + * {@link clientCredentialOwners} for why pre-registered ones are not). + */ interface ClientInfoFields { clientSecret: string; registrationToken: string; + stamped: boolean; } /** @@ -254,7 +318,8 @@ interface ClientInfoFields { * `secrets`, returning the public residue. Both keys are handled in one * place so a slot with a `registration_access_token` but no `client_secret` * still splits — key-by-key extraction is how the token used to slip - * through to plaintext. + * through to plaintext. A DCR slot's values are stamped with the client + * they belong to (#2734). */ function splitClientInformation( info: T, @@ -265,12 +330,18 @@ function splitClientInformation( if (client_secret === undefined && registration_access_token === undefined) { return info; } - if (client_secret !== undefined) { - secrets[fields.clientSecret] = client_secret; - } - if (registration_access_token !== undefined) { - secrets[fields.registrationToken] = registration_access_token; - } + const put = (field: string, value: string | undefined): void => { + if (value === undefined) return; + secrets[field] = value; + if (fields.stamped && typeof info.client_id === "string") { + secrets[clientCredentialStampField(field)] = clientCredentialStamp( + info.client_id, + value, + ); + } + }; + put(fields.clientSecret, client_secret); + put(fields.registrationToken, registration_access_token); return publicInfo as T; } @@ -301,14 +372,17 @@ function joinClientInformation( const issuerClientInfoFields = (issuer: string): ClientInfoFields => ({ clientSecret: issuerClientSecretField(issuer), registrationToken: issuerRegistrationTokenField(issuer), + stamped: true, }); const LEGACY_CLIENT_INFO_FIELDS: ClientInfoFields = { clientSecret: LEGACY_CLIENT_SECRET_FIELD, registrationToken: LEGACY_REGISTRATION_TOKEN_FIELD, + stamped: true, }; const PREREG_CLIENT_INFO_FIELDS: ClientInfoFields = { clientSecret: PREREG_CLIENT_SECRET_FIELD, registrationToken: PREREG_REGISTRATION_TOKEN_FIELD, + stamped: false, }; /** @@ -559,17 +633,25 @@ export function joinIdpSession( export function serverSecretFields( state: ServerOAuthState | undefined, ): string[] { + const stamped = (field: string): string[] => [ + field, + clientCredentialStampField(field), + ]; const fields = new Set([ LEGACY_TOKENS_FIELD, - LEGACY_CLIENT_SECRET_FIELD, - LEGACY_REGISTRATION_TOKEN_FIELD, + ...stamped(LEGACY_CLIENT_SECRET_FIELD), + ...stamped(LEGACY_REGISTRATION_TOKEN_FIELD), PREREG_CLIENT_SECRET_FIELD, PREREG_REGISTRATION_TOKEN_FIELD, ]); for (const issuer of Object.keys(state?.byIssuer ?? {})) { fields.add(issuerTokensField(issuer)); - fields.add(issuerClientSecretField(issuer)); - fields.add(issuerRegistrationTokenField(issuer)); + for (const field of stamped(issuerClientSecretField(issuer))) { + fields.add(field); + } + for (const field of stamped(issuerRegistrationTokenField(issuer))) { + fields.add(field); + } } return [...fields]; } @@ -603,6 +685,32 @@ export function clientCredentialOwners( return owners; } +/** + * `secrets` with each DCR client credential's stamp made to vouch for the + * value beside it (#2734), for a write that copies values it read rather + * than splitting them from state (adoption). A stamp that already vouches + * for its value is kept, whichever client it names — it records where that + * value came from. Any other value is stamped for the client `state` pairs + * it with, which is the pairing the copy itself asserts; a stamp whose + * value is absent is dropped. + */ +export function restampClientCredentials( + state: ServerOAuthState, + secrets: OAuthSecretValues, +): OAuthSecretValues { + const out: OAuthSecretValues = { ...secrets }; + for (const [field, clientId] of clientCredentialOwners(state)) { + const stampField = clientCredentialStampField(field); + const value = Object.hasOwn(out, field) ? out[field] : undefined; + if (value === undefined) { + delete out[stampField]; + } else if (stampedOwner(out[stampField], value) === undefined) { + setOwnEntry(out, stampField, clientCredentialStamp(clientId, value)); + } + } + return out; +} + /** Does this `clientInformation` object still carry a bearer-grade key? */ function clientInfoHasPlaintext( info: ClientInfoSecretKeys | undefined, diff --git a/docs/secret-storage.md b/docs/secret-storage.md index 4d74ba33a..c02ce9510 100644 --- a/docs/secret-storage.md +++ b/docs/secret-storage.md @@ -17,7 +17,7 @@ These values are stored as secrets: They are kept out of `mcp.json` so that sharing, committing or syncing the file does not leak credentials (#1356). When the Inspector saves an entry to a durable store, it leaves each `env` key in `mcp.json` with an empty value and omits the client secret; the real values live in the store. `headers` are **not** moved: they are saved in `mcp.json` exactly as written, so a header that carries a credential stays in the file. [MCP server configuration](./mcp-server-configuration.md) describes what that means for other tools reading the same file. -Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also records in the ledger the `client_id` that every dynamically registered client's `client_secret` and registration token it writes belongs to. The next run on the owed entries' backend reads them, and its first locked save copies them into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the state file still holds a client with the `client_id` recorded when the secret was written, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. Entries written before this was recorded carry no owner, so they are purged as before — as is a keychain value whose owner the ledger cannot tell apart from one a `secrets.json` hand-off filled in, rather than guessing. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). +Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also stores, beside every dynamically registered client's `client_secret` and registration token, an owner stamp naming the `client_id` it was written for, with a digest of the value so the stamp vouches for that value only (an extra `…#owner` entry per credential in the store). The stamp lives in the store rather than the ledger because moving `secrets.json` into the keychain copies or keeps each entry as it finds it, without consulting any ledger, so the stamp follows the value that was actually kept. The next run on the owed entries' backend reads them, and its first locked save copies them, stamp included, into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the value's stamp names the client the state file holds now, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. A value with no stamp that vouches for it (written before stamps existed, or left behind beside another value's stamp) is purged as before rather than guessed at. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). ## How the store is chosen From d43559b1aa50c190a760a8dc7454395d32f7a794 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 23:30:49 -0400 Subject: [PATCH 17/44] test(tui): pin MCP_INSPECTOR_LOG_DIR to a per-file temp dir The TUI suites that mount App reach getTuiLogger(), which with no override resolves to HOME/.mcp-inspector and appended test noise to the developer's real auth.log. The shared setup file now points the variable at a per-file mkdtemp directory, and a guard test asserts the pin is in place inside the OS temp directory. Closes #2723 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/tui/__tests__/logDirIsolation.test.ts | 21 +++++++++++++++++ clients/tui/__tests__/setup.ts | 23 +++++++++++++++++++ 2 files changed, 44 insertions(+) create mode 100644 clients/tui/__tests__/logDirIsolation.test.ts diff --git a/clients/tui/__tests__/logDirIsolation.test.ts b/clients/tui/__tests__/logDirIsolation.test.ts new file mode 100644 index 000000000..f37220495 --- /dev/null +++ b/clients/tui/__tests__/logDirIsolation.test.ts @@ -0,0 +1,21 @@ +import { describe, it, expect } from "vitest"; +import * as os from "node:os"; +import * as path from "node:path"; + +/** + * Guards the `__tests__/setup.ts` log-dir pin (#2723). Every TUI test file + * inherits that setup file, so asserting here that the override is in place, + * and points inside the temp directory, holds for the files that mount `App` + * too. Without it `getTuiLogger()` falls back to `HOME/.mcp-inspector` and the + * suite appends to the developer's real `auth.log`. + */ +describe("TUI test log directory", () => { + it("is pinned inside the OS temp directory by the shared setup file", () => { + const logDir = process.env.MCP_INSPECTOR_LOG_DIR; + expect(logDir).toBeDefined(); + const relative = path.relative(os.tmpdir(), logDir ?? ""); + expect(relative).not.toBe(""); + expect(relative.startsWith("..")).toBe(false); + expect(path.isAbsolute(relative)).toBe(false); + }); +}); diff --git a/clients/tui/__tests__/setup.ts b/clients/tui/__tests__/setup.ts index a1f29aaa0..22934ea34 100644 --- a/clients/tui/__tests__/setup.ts +++ b/clients/tui/__tests__/setup.ts @@ -10,3 +10,26 @@ // rather than per-file. process.stdout.setMaxListeners(100); process.stdin.setMaxListeners(100); + +import { afterAll } from "vitest"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; + +// Keeps the suite off the developer's real `~/.mcp-inspector/auth.log` (#2723). +// `getTuiLogger()` resolves `MCP_INSPECTOR_LOG_DIR ?? HOME/.mcp-inspector` on +// first use, and any test that mounts `App` reaches it — so with no override a +// test run appended "TUI started" lines to the real log, and concurrent +// worktree gates interleaved into the same file. The #2677 shape, one +// directory over: set unconditionally (a value inherited from the shell may +// well be the real directory), one directory per test file since a setup file +// runs once per file. `logger.test.ts` saves, deletes and restores the +// variable around each case, which this does not constrain. +const isolatedLogDir = fs.mkdtempSync( + path.join(os.tmpdir(), "mcp-inspector-tui-log-"), +); +process.env.MCP_INSPECTOR_LOG_DIR = isolatedLogDir; + +afterAll(() => { + fs.rmSync(isolatedLogDir, { recursive: true, force: true }); +}); From a781eb3d5f9832b3dfc75b9fbb962c65e5915413 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 23:36:01 -0400 Subject: [PATCH 18/44] ci: scan v2/main with a committed CodeQL advanced-setup workflow CodeQL default setup analyzes only the default branch and PRs into it, so no v2/main work was scanned until the milestone-merge PR (#2540). Add .github/workflows/codeql.yml: push and pull_request on main and v2/main plus the weekly schedule, same languages and default suite, timeout sized from 30 observed default-setup runs. The analyze job holds security-events: write, so verify:action-pins now counts that scope as credentialed and the job's actions are SHA-pinned. Closes #2545 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .github/workflows/codeql.yml | 77 +++++++++++++++++++++++++++++ docs/quality-gate.md | 2 + scripts/verify-action-pins.mjs | 17 ++++--- scripts/verify-action-pins.test.mjs | 10 +++- 4 files changed, 99 insertions(+), 7 deletions(-) create mode 100644 .github/workflows/codeql.yml diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 000000000..eb837b918 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,77 @@ +# CodeQL code scanning, as an advanced-setup workflow (#2545). +# +# Replaces CodeQL DEFAULT setup, which analyzes only the default branch +# (`main`) and PRs into it. We develop on `v2/main` and reach `main` only by +# milestone merge, so under default setup no v2 work was scanned until the +# release PR: the v2.9.0 ReDoS (#2540) surfaced on the milestone-merge PR and +# held the release, when it should have landed on the PR that introduced it. +# +# Same languages (`actions`, `javascript-typescript`) and the same default +# query suite as default setup had; what changes is only which refs run. +# +# - `push` to `main` and `v2/main` — the baseline each branch's alerts are +# tracked against. +# - `pull_request` into `main` and `v2/main` — a finding lands on the PR +# that introduced it. Runs from the PR's own ref, so this file is live the +# moment it is on `v2/main`, as `dco.yml` is. +# - the weekly `schedule`, which GitHub runs from the DEFAULT branch only — +# so it starts once a milestone merge carries this file to `main`. +# +# ⚠️ Default setup must be OFF (a repository setting — Settings → Code +# security → CodeQL analysis — not a file). GitHub refuses an advanced-setup +# upload while it is on, and with both on the two would double-report. +# +# `security-events: write` is the scope that uploads results — and therefore +# the scope that could suppress them — so `verify:action-pins` counts this job +# as credentialed and every action it runs is SHA-pinned (#2484). Pins are +# resolved with `npm run action:resolve-pin`. +name: CodeQL + +on: + push: + branches: [main, v2/main] + pull_request: + branches: [main, v2/main] + schedule: + - cron: "23 16 * * 6" # 16:23 UTC Saturday, default setup's old slot + +permissions: + contents: read + +jobs: + analyze: + name: Analyze (${{ matrix.language }}) + runs-on: ubuntu-latest + # A newer push to the same PR or branch supersedes the analysis. + concurrency: + group: codeql-${{ github.ref }}-${{ matrix.language }} + cancel-in-progress: true + # Observed under default setup over 30 runs (2026-09-02 – 2026-10-17): + # `actions` ~40s, `javascript-typescript` 1.5–4.7 min. Twice the slowest, + # rounded up to the next five minutes (#2333). + timeout-minutes: 10 + permissions: + contents: read + security-events: write + strategy: + fail-fast: false + matrix: + language: [actions, javascript-typescript] + + steps: + - name: Checkout + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Initialize CodeQL + uses: github/codeql-action/init@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0 + with: + languages: ${{ matrix.language }} + # Neither language needs a build; this is what default setup used. + build-mode: none + + - name: Analyze + uses: github/codeql-action/analyze@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0 + with: + category: "/language:${{ matrix.language }}" diff --git a/docs/quality-gate.md b/docs/quality-gate.md index 69e4e19be..fa0e07737 100644 --- a/docs/quality-gate.md +++ b/docs/quality-gate.md @@ -16,6 +16,8 @@ Each client self-validates from its own folder; the root scripts chain them. The One more CI check runs outside that table: **`.github/workflows/dco.yml`** fails on any commit that is not signed off ([#2566](https://github.com/modelcontextprotocol/inspector/issues/2566), [#2616](https://github.com/modelcontextprotocol/inspector/issues/2616)). It has two jobs. `DCO` runs on every *pull request with a `v2/**` base*, so `v2/main` and stacked v2 PRs, over the PR's own commits; v1 PRs and milestone PRs into `main` are out of its scope. `DCO (v2/main push)` re-checks every push that lands on `v2/main`, as a backstop for anything that merged without a passing PR check. The PR job runs on `pull_request`, so it is live as soon as the workflow is on `v2/main`. The trade-off is that a PR could edit its own check, and the push job could not catch that either, since a push run reads the workflow and the script from the pushed revision. It is accepted because PRs are maintainer-only, such an edit shows in the PR's diff, and the check exists to catch a *forgotten* signoff, not a forged one. The local gate runs the same script before a push as its `local:dco` stage (below), over `origin/v2/main..HEAD`. It replaced the probot DCO app, whose check vanished unnoticed when the app was suspended because it was never required. The replacement gates merges into `v2/main` as a **required** status check, set by the `v2/main - DCO` repository ruleset ([#2621](https://github.com/modelcontextprotocol/inspector/issues/2621)) because a workflow cannot declare itself required. The ruleset pins the check to GitHub Actions, so a hand-posted `DCO` status cannot satisfy it, and blocks deleting or force-pushing `v2/main`, which the push job's `before..after` range assumes never happens. +**`.github/workflows/codeql.yml`** runs CodeQL code scanning (`actions` and `javascript-typescript`, default query suite) on every push to and pull request into `main` and `v2/main`, plus a weekly schedule ([#2545](https://github.com/modelcontextprotocol/inspector/issues/2545)). It replaced CodeQL *default* setup, which scans only the default branch, so `v2/main` work went unscanned until the milestone-merge PR, which is where the v2.9.0 ReDoS ([#2540](https://github.com/modelcontextprotocol/inspector/issues/2540)) surfaced. Default setup is a repository setting and must stay **off**: GitHub rejects an advanced-setup upload while it is on. The job holds `security-events: write`, so `verify:action-pins` counts it as credentialed and its actions are SHA-pinned. It has no local-gate counterpart. + The local gate runs **every check** `main.yml` runs, and is not a mirror. One of its steps has no GitHub CI counterpart: | Local-only step | Why it is local-only | diff --git a/scripts/verify-action-pins.mjs b/scripts/verify-action-pins.mjs index 6706445a1..a8988d38a 100644 --- a/scripts/verify-action-pins.mjs +++ b/scripts/verify-action-pins.mjs @@ -13,9 +13,12 @@ // // A job counts as credentialed when it // -// 1. can mint an OIDC token or push a package — `id-token: write` or -// `packages: write` (or `write-all`), in its own `permissions:` or, absent -// one, the workflow's; +// 1. can mint an OIDC token, push a package or upload code-scanning results +// — `id-token: write`, `packages: write` or `security-events: write` (or +// `write-all`), in its own `permissions:` or, absent one, the workflow's. +// `security-events` joined in #2545: it is the scope a CodeQL job uploads +// its findings with, so a moved tag there could quietly drop every alert +// — the scan reporting clean is exactly what such a job is trusted for; // 2. is handed any secret other than `GITHUB_TOKEN`, its own or through the // workflow-level `env:` — in any spelling of the // expression (`secrets.X`, `secrets['X']`, `secrets[matrix.name]`), or as @@ -68,12 +71,14 @@ const DEFAULT_TOKEN = const EXPRESSION = /\$\{\{([\s\S]*?)\}\}/g; const EXACT_VERSION = /^v\d+\.\d+\.\d+$/; -/** Does this `permissions:` value let the job mint a token or push a package? */ +/** Does this `permissions:` value let the job mint a token, push a package or upload scan results? */ function mints(permissions) { if (permissions === "write-all") return true; if (permissions === null || typeof permissions !== "object") return false; return ( - permissions["id-token"] === "write" || permissions.packages === "write" + permissions["id-token"] === "write" || + permissions.packages === "write" || + permissions["security-events"] === "write" ); } @@ -233,7 +238,7 @@ export function main(root = repoRoot) { console.error( `verify:action-pins — ${problems.length} action ref(s) in a credentialed job are not SHA-pinned:\n` + problems.join("\n") + - "\n\nA job holding `id-token`/`packages: write`, a non-default secret, or building an" + + "\n\nA job holding `id-token`/`packages`/`security-events: write`, a non-default secret, or building an" + "\nartifact such a job consumes runs only immutable refs (#2484). Pin each as" + "\n uses: owner/repo@<40-hex sha> # vX.Y.Z" + "\nresolving the SHA and the exact release from the same tag lookup. The comment" + diff --git a/scripts/verify-action-pins.test.mjs b/scripts/verify-action-pins.test.mjs index 1fed79a4a..13f7bada9 100644 --- a/scripts/verify-action-pins.test.mjs +++ b/scripts/verify-action-pins.test.mjs @@ -30,11 +30,19 @@ test("a minting scope in a job's own permissions makes it credentialed", () => { " permissions:", " contents: read", " packages: write", + " scan:", + " permissions:", + " contents: read", + " security-events: write", " plain:", " permissions:", " contents: read", ); - assert.deepEqual([...credentialedJobs(yaml)].sort(), ["image", "publish"]); + assert.deepEqual([...credentialedJobs(yaml)].sort(), [ + "image", + "publish", + "scan", + ]); }); test("write-all counts as a minting scope", () => { From b0895155f7a279a418ef42b582576d8bc934c893 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 23:37:40 -0400 Subject: [PATCH 19/44] fix(ema): allow a public resource-AS client in leg 3 mintEmaResourceTokens refused to run without a resource-AS client secret, although redeemIdJagForAccessToken already redeems a public client with token_endpoint_auth_method "none". Drop the secret guard and keep the clientId check, so a resource AS that registers public clients works with EMA. The Resource AS Client Secret field now says it may stay empty for a public client. Tests cover the secretless leg-3 request (client_id in the body, no client_secret, no Authorization header), and the tests that used an empty secret to force a mint failure now use an empty clientId. Closes #2725 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../ServerSettingsForm.test.tsx | 5 ++ .../ServerSettingsForm/ServerSettingsForm.tsx | 7 ++- .../src/test/core/auth/ema/emaFlow.test.ts | 55 +++++++++++++++++-- core/auth/ema/emaFlow.ts | 8 +-- 4 files changed, 63 insertions(+), 12 deletions(-) diff --git a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx index d77095c26..48ef7f3ca 100644 --- a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx +++ b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx @@ -1550,6 +1550,11 @@ describe("ServerSettingsForm", () => { /resource authorization server's registered client credential/i, ).length, ).toBeGreaterThan(0); + // The secret is optional for a public resource-AS client (#2725), and the + // hint sits on the secret field only. + expect( + screen.getAllByText(/Leave it empty for a public client/i), + ).toHaveLength(1); }); it("documents the space-separated Scopes delimiter (#1692)", () => { diff --git a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.tsx b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.tsx index 18dd96ab2..b2551e946 100644 --- a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.tsx +++ b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.tsx @@ -548,6 +548,11 @@ export function ServerSettingsForm({ const resourceAsDescription = enterpriseManaged ? "The resource authorization server's registered client credential (EMA leg 3) — not the app client id/secret, which belong in Client Settings." : undefined; + // A resource AS that registers public clients issues no secret, and leg 3 + // redeems without one (#2725) — so say the field may stay empty. + const resourceAsSecretDescription = enterpriseManaged + ? `${resourceAsDescription} Leave it empty for a public client (token_endpoint_auth_method "none").` + : undefined; // The modern per-request "Log Level per Request" control is meaningless on a // legacy connection (#1629). Hide it when legacy is pinned, or when an `auto` // server negotiated legacy at connect time. A pinned `modern` server, and an @@ -1008,7 +1013,7 @@ export function ServerSettingsForm({ /> diff --git a/clients/web/src/test/core/auth/ema/emaFlow.test.ts b/clients/web/src/test/core/auth/ema/emaFlow.test.ts index 77af62b90..761279544 100644 --- a/clients/web/src/test/core/auth/ema/emaFlow.test.ts +++ b/clients/web/src/test/core/auth/ema/emaFlow.test.ts @@ -203,7 +203,7 @@ describe("emaFlow", () => { ).rejects.toThrow("Valid IdP ID Token required for EMA token mint"); }); - it("mintEmaResourceTokens throws when resource client secret is missing", async () => { + it("mintEmaResourceTokens throws when resource client id is missing", async () => { const exp = Math.floor(Date.now() / 1000) + 3600; idpSessions[IDP_ISSUER] = { idToken: jwtWithExp(exp) }; @@ -211,12 +211,55 @@ describe("emaFlow", () => { mintEmaResourceTokens( { ...baseConfig(storage), - resourceClientSecret: "", + resourceClientId: "", fetchFn: mockEmaFetch(jwtWithExp(exp)), }, resourceContext(), ), - ).rejects.toThrow(/resource authorization server client secret/); + ).rejects.toThrow(/resource authorization server clientId/); + }); + + it("mintEmaResourceTokens redeems leg 3 as a public client when the secret is empty (#2725)", async () => { + const exp = Math.floor(Date.now() / 1000) + 3600; + const idToken = jwtWithExp(exp); + idpSessions[IDP_ISSUER] = { idToken }; + const fetchFn = mockEmaFetch(idToken); + + const { tokens } = await mintEmaResourceTokens( + { ...baseConfig(storage), resourceClientSecret: "", fetchFn }, + resourceContext(), + ); + + expect(tokens.access_token).toBe("resource-access-token"); + const leg3 = fetchFn.mock.calls.find( + ([input]) => String(input) === `${AS_ISSUER}/token`, + ); + expect(leg3).toBeDefined(); + const body = new URLSearchParams(leg3?.[1]?.body as string); + expect(body.get("client_id")).toBe(EMA_MOCK_RESOURCE_CLIENT_ID); + expect(body.has("client_secret")).toBe(false); + expect(new Headers(leg3?.[1]?.headers).has("authorization")).toBe(false); + }); + + it("mintEmaResourceTokens falls back to the PRM resource when the context has no resourceUrl", async () => { + const exp = Math.floor(Date.now() / 1000) + 3600; + const idToken = jwtWithExp(exp); + idpSessions[IDP_ISSUER] = { idToken }; + const fetchFn = mockEmaFetch(idToken); + + await mintEmaResourceTokens( + { ...baseConfig(storage), fetchFn }, + { ...resourceContext(), resourceUrl: undefined }, + ); + + const bodyFor = (url: string) => + new URLSearchParams( + fetchFn.mock.calls.find(([input]) => String(input) === url)?.[1] + ?.body as string, + ); + // Leg 2 must carry `resource`, so it takes the PRM's; leg 3 omits it. + expect(bodyFor(`${IDP_ISSUER}/token`).get("resource")).toBe(MCP_RESOURCE); + expect(bodyFor(`${AS_ISSUER}/token`).has("resource")).toBe(false); }); it("trySilentEmaAuth saves tagged tokens when mint succeeds", async () => { @@ -250,7 +293,7 @@ describe("emaFlow", () => { const result = await trySilentEmaAuth({ ...baseConfig(storage), - resourceClientSecret: "", + resourceClientId: "", fetchFn: mockEmaFetch(idToken), }); @@ -260,7 +303,7 @@ describe("emaFlow", () => { /EMA legs 2–3 \(resource token mint\)/, ); expect(result.error.message).toMatch( - /resource authorization server client secret/, + /resource authorization server clientId/, ); } expect(storage.saveTokens).not.toHaveBeenCalled(); @@ -449,7 +492,7 @@ describe("emaFlow", () => { const fetchFn = mockEmaFetch(idToken); const config = { ...baseConfig(storage), - resourceClientSecret: "", + resourceClientId: "", fetchFn, }; diff --git a/core/auth/ema/emaFlow.ts b/core/auth/ema/emaFlow.ts index b1be981e7..9d0e48620 100644 --- a/core/auth/ema/emaFlow.ts +++ b/core/auth/ema/emaFlow.ts @@ -66,11 +66,9 @@ export async function mintEmaResourceTokens( "EMA requires resource authorization server clientId (per-server oauth.clientId)", ); } - if (!config.resourceClientSecret?.trim()) { - throw new Error( - "EMA requires resource authorization server client secret in server OAuth settings (Test Client secret from xaa.dev resource registration)", - ); - } + // No secret check: a resource AS that registers public clients + // (`token_endpoint_auth_method: none`) issues no secret, and leg 3 already + // redeems without one (#2725). const idToken = await getValidIdToken({ idp: config.idp, From c640056724d48dffe2b32f227804c7975858c8d3 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 23:47:49 -0400 Subject: [PATCH 20/44] feat(core): recognize HTTP 429 and surface the server's Retry-After A throttled request surfaced as the SDK's bare "Error POSTing to endpoint", indistinguishable from any other non-OK status. Annotate a 429 on the same request funnel the #2318 timeout diagnostics use, so all three clients say it is rate limiting, which request was throttled, and when the server asked to retry. The SDK's SdkHttpError drops response headers, so Retry-After comes from the fetch tracker. On the web path the upstream fetch runs in the backend, which relays the header on the typed send error rather than leaving the browser to race a fetch event against the send response. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../web/src/test/core/mcp/rateLimit.test.ts | 214 ++++++++++++++ .../mcp/inspectorClient-rate-limit.test.ts | 263 ++++++++++++++++++ core/mcp/inspectorClient.ts | 31 ++- core/mcp/rateLimit.ts | 173 ++++++++++++ core/mcp/remote/node/remote-session.ts | 18 ++ core/mcp/remote/node/server.ts | 7 + core/mcp/remote/remoteClientTransport.ts | 1 + core/mcp/remote/types.ts | 5 + 8 files changed, 705 insertions(+), 7 deletions(-) create mode 100644 clients/web/src/test/core/mcp/rateLimit.test.ts create mode 100644 clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts create mode 100644 core/mcp/rateLimit.ts diff --git a/clients/web/src/test/core/mcp/rateLimit.test.ts b/clients/web/src/test/core/mcp/rateLimit.test.ts new file mode 100644 index 000000000..3e2617463 --- /dev/null +++ b/clients/web/src/test/core/mcp/rateLimit.test.ts @@ -0,0 +1,214 @@ +import { describe, it, expect } from "vitest"; +import { + SdkError, + SdkErrorCode, + SdkHttpError, +} from "@modelcontextprotocol/client"; +import { + annotateRateLimit, + isRateLimitError, + parseRetryAfter, + rateLimitResponseOf, + retryAfterSince, + type AnnotatedRateLimitData, +} from "@inspector/core/mcp/rateLimit.js"; +import type { FetchRequestEntryBase } from "@inspector/core/mcp/types.js"; + +const NOW = Date.parse("2026-10-19T12:00:00Z"); + +function entry( + overrides: Partial = {}, +): FetchRequestEntryBase { + return { + id: "fetch-1", + timestamp: new Date(NOW), + method: "POST", + url: "http://localhost/mcp", + requestHeaders: {}, + responseStatus: 429, + responseHeaders: { "Retry-After": "30" }, + ...overrides, + }; +} + +function throttled(data: Record = {}): SdkHttpError { + return new SdkHttpError( + SdkErrorCode.ClientHttpNotImplemented, + "Error POSTing to endpoint: slow down", + { status: 429, statusText: "Too Many Requests", ...data }, + ); +} + +describe("rateLimitResponseOf", () => { + it("records a 429's Retry-After, whatever its casing, and when it was sent", () => { + expect(rateLimitResponseOf(entry())).toEqual({ + retryAfter: "30", + sentAt: NOW, + }); + }); + + it("records a 429 with no Retry-After", () => { + expect(rateLimitResponseOf(entry({ responseHeaders: {} }))).toEqual({ + sentAt: NOW, + }); + }); + + it("ignores anything that is not a 429", () => { + expect(rateLimitResponseOf(entry({ responseStatus: 503 }))).toBeUndefined(); + expect( + rateLimitResponseOf(entry({ responseStatus: undefined })), + ).toBeUndefined(); + }); +}); + +describe("retryAfterSince", () => { + it("returns the header of a 429 sent at or after the bound", () => { + const latest = { retryAfter: "30", sentAt: NOW }; + expect(retryAfterSince(latest, NOW)).toBe("30"); + expect(retryAfterSince(latest, NOW - 1)).toBe("30"); + }); + + it("drops a 429 that predates the request", () => { + expect(retryAfterSince({ retryAfter: "30", sentAt: NOW }, NOW + 1)).toBe( + undefined, + ); + }); + + it("returns nothing when no 429 was recorded", () => { + expect(retryAfterSince(undefined, NOW)).toBeUndefined(); + }); +}); + +describe("parseRetryAfter", () => { + it("reads delta-seconds", () => { + expect(parseRetryAfter("30", NOW)).toBe(30_000); + expect(parseRetryAfter(" 0 ", NOW)).toBe(0); + }); + + it("reads an HTTP-date as the delay from now", () => { + expect(parseRetryAfter("Mon, 19 Oct 2026 12:01:30 GMT", NOW)).toBe(90_000); + }); + + it("clamps a date already past to zero", () => { + expect(parseRetryAfter("Mon, 19 Oct 2026 11:00:00 GMT", NOW)).toBe(0); + }); + + it("treats a missing or malformed value as absent", () => { + expect(parseRetryAfter(undefined, NOW)).toBeUndefined(); + expect(parseRetryAfter("soon", NOW)).toBeUndefined(); + // `Date.parse` alone reads these as dates. + expect(parseRetryAfter("-5", NOW)).toBeUndefined(); + expect(parseRetryAfter("1.5", NOW)).toBeUndefined(); + expect(parseRetryAfter("2026-10-19T12:01:30Z", NOW)).toBeUndefined(); + // The right shape, but no real date. + expect(parseRetryAfter("Mon, 99 Foo 2026 12:00:00 GMT", NOW)).toBe( + undefined, + ); + }); +}); + +describe("isRateLimitError", () => { + it("matches only an SdkHttpError with status 429", () => { + expect(isRateLimitError(throttled())).toBe(true); + expect(isRateLimitError(throttled({ status: 503 }))).toBe(false); + expect( + isRateLimitError(new SdkError(SdkErrorCode.RequestTimeout, "timed out")), + ).toBe(false); + expect( + isRateLimitError(Object.assign(new Error("x"), { status: 429 })), + ).toBe(false); + expect(isRateLimitError(undefined)).toBe(false); + }); +}); + +describe("annotateRateLimit", () => { + it("names the method and the retry delay, keeping class, code, status and cause", () => { + const original = throttled(); + const annotated = annotateRateLimit(original, "tools/call", "30", NOW); + + expect(SdkHttpError.isInstance(annotated)).toBe(true); + const error = annotated as SdkHttpError; + expect(error.message).toBe( + "Rate limited by the server (HTTP 429) on tools/call. " + + "The server asked to retry after 30s. " + + "Error POSTing to endpoint: slow down", + ); + expect(error.code).toBe(SdkErrorCode.ClientHttpNotImplemented); + expect(error.status).toBe(429); + expect(error.statusText).toBe("Too Many Requests"); + expect(error.cause).toBe(original); + const data = error.data as AnnotatedRateLimitData; + expect(data.method).toBe("tools/call"); + expect(data.retryAfter).toBe("30"); + expect(data.retryAfterMs).toBe(30_000); + }); + + it("prefers a Retry-After the backend relayed on the error", () => { + const annotated = annotateRateLimit( + throttled({ retryAfter: "5" }), + "tools/list", + "30", + NOW, + ) as SdkHttpError; + expect(annotated.message).toContain("retry after 5s"); + expect((annotated.data as AnnotatedRateLimitData).retryAfter).toBe("5"); + }); + + it("ignores a relayed retryAfter that is not a string", () => { + const annotated = annotateRateLimit( + throttled({ retryAfter: 5 }), + "tools/list", + "30", + NOW, + ) as SdkHttpError; + expect(annotated.message).toContain("retry after 30s"); + }); + + it("says when the server sent no Retry-After", () => { + const annotated = annotateRateLimit( + throttled(), + "tools/list", + undefined, + NOW, + ) as SdkHttpError; + expect(annotated.message).toContain( + "The server sent no Retry-After header.", + ); + const data = annotated.data as AnnotatedRateLimitData; + expect(data.retryAfter).toBeUndefined(); + expect(data.retryAfterMs).toBeUndefined(); + }); + + it("reports an unparseable Retry-After verbatim rather than guessing", () => { + const annotated = annotateRateLimit( + throttled(), + "tools/list", + "soon", + NOW, + ) as SdkHttpError; + expect(annotated.message).toContain( + "The server sent an unparseable Retry-After (soon).", + ); + const data = annotated.data as AnnotatedRateLimitData; + expect(data.retryAfter).toBe("soon"); + expect(data.retryAfterMs).toBeUndefined(); + }); + + it("defaults the clock to now", () => { + const annotated = annotateRateLimit( + throttled(), + "tools/list", + "1", + ) as SdkHttpError; + expect(annotated.message).toContain("retry after 1s"); + }); + + it("returns anything that is not a 429 untouched", () => { + const unavailable = throttled({ status: 503 }); + expect(annotateRateLimit(unavailable, "tools/list", "30", NOW)).toBe( + unavailable, + ); + const timeout = new SdkError(SdkErrorCode.RequestTimeout, "timed out"); + expect(annotateRateLimit(timeout, "tools/list", "30", NOW)).toBe(timeout); + }); +}); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts new file mode 100644 index 000000000..0a0e1379e --- /dev/null +++ b/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts @@ -0,0 +1,263 @@ +import { describe, it, expect, afterEach } from "vitest"; +import { + createServer, + type IncomingMessage, + type Server, + type ServerResponse, +} from "node:http"; +import { serve, type ServerType } from "@hono/node-server"; +import { SdkErrorCode, SdkHttpError } from "@modelcontextprotocol/client"; +import { InspectorClient } from "@inspector/core/mcp/inspectorClient.js"; +import { createTransportNode } from "@inspector/core/mcp/node/transport.js"; +import { createRemoteTransport } from "@inspector/core/mcp/remote/createRemoteTransport.js"; +import { createRemoteApp } from "@inspector/core/mcp/remote/node/server.js"; +import { + eraToVersionNegotiation, + type CreateTransport, +} from "@inspector/core/mcp/types.js"; +import type { AnnotatedRateLimitData } from "@inspector/core/mcp/rateLimit.js"; + +/** + * Rate-limit recognition against a real connection (#2442). + * + * A server that answers `initialize` and then throttles `tools/list` with a + * `429`. The SDK's `SdkHttpError` keeps the status but drops the response + * headers, so the `Retry-After` in the message can only have come from the + * fetch tracker — directly on a Node transport, and relayed on the send error + * by the web backend on the remote one. Both are driven end to end here. + * + * Hand-rolled rather than a composable test server because the SDK's own + * server cannot be configured to answer one method with a 429. + */ + +interface ToolsListReply { + status: number; + headers?: Record; + body: string; +} + +interface ThrottlingServer { + url: string; + setToolsListReply: (reply: ToolsListReply) => void; + stop: () => Promise; +} + +const THROTTLED: ToolsListReply = { + status: 429, + headers: { "retry-after": "30" }, + body: "slow down", +}; + +function startThrottlingServer(): Promise { + let toolsListReply = THROTTLED; + + const handler = async (req: IncomingMessage, res: ServerResponse) => { + // No standalone notification stream; nothing here needs one. + if (req.method === "GET") { + res.writeHead(405).end(); + return; + } + if (req.method === "DELETE") { + res.writeHead(200).end(); + return; + } + const chunks: Buffer[] = []; + for await (const chunk of req) chunks.push(chunk as Buffer); + const body = JSON.parse(Buffer.concat(chunks).toString()) as { + id?: unknown; + method: string; + }; + if (body.id === undefined) { + res.writeHead(202).end(); + return; + } + if (body.method === "tools/list") { + res.writeHead(toolsListReply.status, { + "content-type": "text/plain", + ...toolsListReply.headers, + }); + res.end(toolsListReply.body); + return; + } + const result = + body.method === "initialize" + ? { + protocolVersion: "2025-06-18", + capabilities: { tools: {} }, + serverInfo: { name: "throttling-server", version: "1.0.0" }, + } + : {}; + res.writeHead(200, { "content-type": "application/json" }); + res.end(JSON.stringify({ jsonrpc: "2.0", id: body.id, result })); + }; + + const server: Server = createServer((req, res) => { + void handler(req, res); + }); + + return new Promise((resolve) => { + server.listen(0, "127.0.0.1", () => { + const address = server.address(); + /* v8 ignore next -- listen() on a fresh server always yields an AddressInfo */ + const port = typeof address === "object" && address ? address.port : 0; + resolve({ + url: `http://127.0.0.1:${port}/mcp`, + setToolsListReply: (reply) => { + toolsListReply = reply; + }, + stop: () => + new Promise((done) => { + server.closeAllConnections(); + server.close(() => done()); + }), + }); + }); + }); +} + +/** The web backend on an ephemeral port, as the browser's remote transport sees it. */ +function startRemoteBackend(): Promise<{ + server: ServerType; + transport: CreateTransport; +}> { + const { app, authToken } = createRemoteApp({ + initialConfig: { defaultEnvironment: {} }, + }); + return new Promise((resolve, reject) => { + const server = serve( + { fetch: app.fetch, port: 0, hostname: "127.0.0.1" }, + (info) => { + resolve({ + server, + transport: createRemoteTransport({ + baseUrl: `http://127.0.0.1:${info.port}`, + authToken, + }), + }); + }, + ); + server.on("error", reject); + }); +} + +async function rejectionOf(promise: Promise): Promise { + try { + await promise; + } catch (err) { + return err; + } + throw new Error("expected the request to reject"); +} + +describe("InspectorClient rate-limit recognition (#2442)", () => { + let client: InspectorClient | null = null; + let upstream: ThrottlingServer | null = null; + let backend: ServerType | null = null; + + async function connectTo( + url: string, + transport: CreateTransport = createTransportNode, + ): Promise { + client = new InspectorClient( + { type: "streamable-http", url }, + { + environment: { transport }, + // Pinned legacy so the connect is exactly `initialize`. + versionNegotiation: eraToVersionNegotiation("legacy"), + }, + ); + await client.connect(); + return client; + } + + afterEach(async () => { + try { + await client?.disconnect(); + } catch { + // Teardown only — the assertions already ran. + } + client = null; + await upstream?.stop(); + upstream = null; + if (backend) { + const closing = backend; + await new Promise((done) => closing.close(() => done())); + backend = null; + } + }); + + function expectThrottled(caught: unknown): AnnotatedRateLimitData { + expect(SdkHttpError.isInstance(caught)).toBe(true); + const error = caught as SdkHttpError; + // Same class, code and status as the SDK's own error: only the message + // and the data grew. + expect(error.code).toBe(SdkErrorCode.ClientHttpNotImplemented); + expect(error.status).toBe(429); + return error.data as AnnotatedRateLimitData; + } + + it("names the throttled request and the server's Retry-After", async () => { + upstream = await startThrottlingServer(); + const c = await connectTo(upstream.url); + + const caught = await rejectionOf(c.listTools()); + + const data = expectThrottled(caught); + expect((caught as Error).message).toBe( + "Rate limited by the server (HTTP 429) on tools/list. " + + "The server asked to retry after 30s. " + + "Error POSTing to endpoint: slow down", + ); + expect(data.method).toBe("tools/list"); + expect(data.retryAfter).toBe("30"); + expect(data.retryAfterMs).toBe(30_000); + expect(SdkHttpError.isInstance((caught as Error).cause)).toBe(true); + }); + + it("says so when the server sent no Retry-After", async () => { + upstream = await startThrottlingServer(); + upstream.setToolsListReply({ status: 429, body: "slow down" }); + const c = await connectTo(upstream.url); + + const caught = await rejectionOf(c.listTools()); + + const data = expectThrottled(caught); + expect((caught as Error).message).toContain( + "The server sent no Retry-After header.", + ); + expect(data.retryAfter).toBeUndefined(); + }); + + it("leaves a non-429 failure exactly as the SDK threw it", async () => { + upstream = await startThrottlingServer(); + upstream.setToolsListReply({ + status: 503, + headers: { "retry-after": "30" }, + body: "down for maintenance", + }); + const c = await connectTo(upstream.url); + + const caught = await rejectionOf(c.listTools()); + + expect(SdkHttpError.isInstance(caught)).toBe(true); + expect((caught as SdkHttpError).status).toBe(503); + expect((caught as Error).message).toBe( + "Error POSTing to endpoint: down for maintenance", + ); + }); + + it("relays the Retry-After through the web backend", async () => { + upstream = await startThrottlingServer(); + const started = await startRemoteBackend(); + backend = started.server; + const c = await connectTo(upstream.url, started.transport); + + const caught = await rejectionOf(c.listTools()); + + const data = expectThrottled(caught); + expect((caught as Error).message).toMatch( + /^Rate limited by the server \(HTTP 429\) on tools\/list\. The server asked to retry after 30s\. /, + ); + expect(data.retryAfter).toBe("30"); + }); +}); diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index 818c4f60e..10f6187ce 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -296,6 +296,12 @@ import { type NotificationStreamState, type OutstandingRequest, } from "./connectionDiagnostics.js"; +import { + annotateRateLimit, + rateLimitResponseOf, + retryAfterSince, + type RateLimitResponse, +} from "./rateLimit.js"; import { OAuthManager, type OAuthManagerConfig } from "./oauthManager.js"; import { RemoteClientTransport } from "./remote/remoteClientTransport.js"; @@ -555,6 +561,9 @@ export class InspectorClient extends InspectorClientEventTarget { // `GET` on Streamable HTTP). Both per-session, reset with the maps above. private lastResponse: LastResponse | undefined; private notificationStream: TrackedNotificationStream | undefined; + // The most recent `429` a transport fetch drew, kept for its `Retry-After` + // (#2442). Per-session, reset with the diagnostics above. + private lastRateLimitResponse: RateLimitResponse | undefined; /** True when the cached transport was built with an OAuth authProvider attached. */ private transportHasAuthProvider = false; /** Dedupes concurrent ambient auth challenges (reason + scopes). */ @@ -1071,7 +1080,7 @@ export class InspectorClient extends InspectorClientEventTarget { this.clientInfo, Object.keys(clientOptions).length > 0 ? clientOptions : undefined, ); - this.annotateSdkRequestTimeouts(this.client); + this.annotateSdkRequestErrors(this.client); if (this.tasksCapabilityAdvertised) { this.bindReceiverTasks(); } @@ -1080,7 +1089,8 @@ export class InspectorClient extends InspectorClientEventTarget { /** * Decorate the SDK Client's `request` so a per-request timeout rejects with * the connection's state at that moment instead of a bare `Request timed - * out` (#2318). + * out` (#2318), and a `429` rejects as rate limiting with the server's + * `Retry-After` instead of a bare `Error POSTing to endpoint` (#2442). * * `Protocol.request` is the one funnel every SDK verb goes through — * `listTools`, `callTool`, `readResource`, the handshake's `initialize`, @@ -1098,19 +1108,23 @@ export class InspectorClient extends InspectorClientEventTarget { * implementation signature serves both at runtime; the single cast on the * way back is what the overloads cost. */ - private annotateSdkRequestTimeouts(client: Client): void { + private annotateSdkRequestErrors(client: Client): void { type SdkRequest = Client["request"]; const sdkRequest: SdkRequest = client.request.bind(client); const decorated = async ( ...args: Parameters ): Promise => { + // A 429 is matched to this request by time: only one drawn by a fetch + // sent after the request started can be the response that failed it. + const sentAt = Date.now(); try { return await sdkRequest(...args); } catch (err) { - throw annotateRequestTimeout( - err, - args[0].method, - this.getConnectionDiagnostics(), + const method = args[0].method; + throw annotateRateLimit( + annotateRequestTimeout(err, method, this.getConnectionDiagnostics()), + method, + retryAfterSince(this.lastRateLimitResponse, sentAt), ); } }; @@ -1673,6 +1687,7 @@ export class InspectorClient extends InspectorClientEventTarget { this.taskProgress.clear(); this.lastResponse = undefined; this.notificationStream = undefined; + this.lastRateLimitResponse = undefined; this.dispatchConnectionDiagnosticsChange(); // Per-session for the same reason: both name entries of the PREVIOUS // server's list. Cleared here as well as in `disconnect()` because the @@ -1813,6 +1828,8 @@ export class InspectorClient extends InspectorClientEventTarget { onFetchRequest: (entry: FetchRequestEntryBase) => { this.dispatchFetchRequest({ ...entry, category: "transport" }); this.noteTransportStream(entry); + this.lastRateLimitResponse = + rateLimitResponseOf(entry) ?? this.lastRateLimitResponse; }, onFetchResponseBody: (id: string, body: string) => { this.dispatchFetchRequestBodyUpdate(id, body); diff --git a/core/mcp/rateLimit.ts b/core/mcp/rateLimit.ts new file mode 100644 index 000000000..c20235d7f --- /dev/null +++ b/core/mcp/rateLimit.ts @@ -0,0 +1,173 @@ +/** + * Rate-limit recognition: the one place that turns a server's `HTTP 429` into + * a message saying the server is throttling, and when it says to retry + * (#2442). + * + * Before this, a throttled request surfaced as the SDK's + * `Error POSTing to endpoint: ` — the same shape as every other non-OK + * status, so telling "the server is down" from "the server is pacing me" was + * left to the reader. Both facts needed to say more were already in-process: + * the error carries the status, and the fetch tracker recorded the response's + * `Retry-After` header. + * + * The header has to come from the tracker because the SDK's `SdkHttpError` + * keeps the status and the body but none of the response headers. Where the + * upstream fetch runs in the web backend, the backend reads the same header + * off its own tracked fetch and relays it on the send error as `retryAfter` + * (see `RemoteHttpErrorData`), so the browser does not depend on a fetch + * event racing the send's HTTP response across two connections. + * + * Everything here is pure. `InspectorClient` owns the bookkeeping, the same + * split `connectionDiagnostics.ts` makes for request timeouts. + * + * Deliberately narrow: only a 429 is recognized. A 503 also carries + * `Retry-After` but means "unavailable", which is exactly the case this exists + * to tell apart from throttling. + */ + +import { + SdkHttpError, + type SdkHttpErrorData, +} from "@modelcontextprotocol/client"; +import { findHeader } from "./fetchTracking.js"; +import { formatDuration } from "./connectionDiagnostics.js"; +import type { FetchRequestEntryBase } from "./types.js"; + +/** `429 Too Many Requests` (RFC 6585). */ +export const HTTP_TOO_MANY_REQUESTS = 429; + +/** A rate-limited response the fetch tracker recorded. */ +export interface RateLimitResponse { + /** The raw `Retry-After` header, when the server sent one. */ + retryAfter?: string; + /** Epoch ms at which the request that drew it went out. */ + sentAt: number; +} + +/** + * The rate-limit facts in a tracked fetch, or `undefined` when the response + * was not a 429 (or the fetch failed before any response arrived). + */ +export function rateLimitResponseOf( + entry: FetchRequestEntryBase, +): RateLimitResponse | undefined { + if (entry.responseStatus !== HTTP_TOO_MANY_REQUESTS) return undefined; + const retryAfter = findHeader(entry.responseHeaders, "retry-after"); + return { + ...(retryAfter !== undefined && { retryAfter }), + sentAt: entry.timestamp.getTime(), + }; +} + +/** + * The latest recorded 429's `Retry-After`, when that 429 answered a request + * sent at or after `since`. The bound is what keeps a header from a 429 + * minutes ago from being reported against a request it never answered. + */ +export function retryAfterSince( + latest: RateLimitResponse | undefined, + since: number, +): string | undefined { + if (!latest || latest.sentAt < since) return undefined; + return latest.retryAfter; +} + +/** + * The IMF-fixdate form of an HTTP-date (`Sun, 06 Nov 1994 08:49:37 GMT`), + * which RFC 9110 §5.6.7 requires senders to use. Matched before `Date.parse` + * because `Date.parse` is lenient past usefulness: it reads `-5` as the year + * 5 BC, which would report a malformed header as "retry now". + */ +const IMF_FIXDATE = + /^[A-Z][a-z]{2}, \d{2} [A-Z][a-z]{2} \d{4} \d{2}:\d{2}:\d{2} GMT$/; + +/** + * A `Retry-After` value as a delay in ms from `now` (RFC 9110 §10.2.3): either + * a non-negative integer count of seconds, or an HTTP-date. A date already in + * the past is a delay of `0`. Anything else is `undefined` — a malformed header + * is reported verbatim rather than guessed at. + */ +export function parseRetryAfter( + value: string | undefined, + now: number, +): number | undefined { + if (value === undefined) return undefined; + const trimmed = value.trim(); + if (/^\d+$/.test(trimmed)) return Number(trimmed) * 1000; + if (!IMF_FIXDATE.test(trimmed)) return undefined; + const at = Date.parse(trimmed); + // A well-formed shape can still name no real date (`Mon, 99 Foo 2026 …`). + if (Number.isNaN(at)) return undefined; + return Math.max(0, at - now); +} + +/** Whether `err` is the SDK's HTTP error for a `429 Too Many Requests`. */ +export function isRateLimitError(err: unknown): err is SdkHttpError { + return SdkHttpError.isInstance(err) && err.status === HTTP_TOO_MANY_REQUESTS; +} + +/** + * The data carried by an annotated rate-limit error: the SDK's own HTTP data, + * plus the method that was throttled and the retry timing when the server + * gave one. Exposed so a consumer that wants the structured form (a countdown, + * a CLI's JSON output) can read it off `error.data` instead of parsing the + * message. + */ +export interface AnnotatedRateLimitData extends SdkHttpErrorData { + method: string; + /** The raw `Retry-After` header, when the server sent one. */ + retryAfter?: string; + /** The parsed delay, when `retryAfter` was well-formed. */ + retryAfterMs?: number; +} + +function relayedRetryAfter(err: SdkHttpError): string | undefined { + const value = err.data.retryAfter; + return typeof value === "string" ? value : undefined; +} + +function describeRetry( + retryAfter: string | undefined, + retryAfterMs: number | undefined, +): string { + if (retryAfterMs !== undefined) { + return `The server asked to retry after ${formatDuration(retryAfterMs)}.`; + } + if (retryAfter !== undefined) { + return `The server sent an unparseable Retry-After (${retryAfter}).`; + } + return "The server sent no Retry-After header."; +} + +/** + * Rebuild a 429 `SdkHttpError` with a message that names it as rate limiting, + * says which request was throttled, and gives the retry timing when the server + * sent one. + * + * `retryAfter` is the header the caller tracked; a value the backend relayed on + * the error itself takes precedence, since it was read off the very response + * that produced this error. + * + * Anything that is not a 429 is returned untouched, so this is safe to apply + * blindly on the rejection path of every request. The result keeps the + * original's code and HTTP data, so a status or code check downstream sees + * what it saw before; the original is kept as `cause`. + */ +export function annotateRateLimit( + err: unknown, + method: string, + retryAfter: string | undefined, + now: number = Date.now(), +): unknown { + if (!isRateLimitError(err)) return err; + const header = relayedRetryAfter(err) ?? retryAfter; + const retryAfterMs = parseRetryAfter(header, now); + const data: AnnotatedRateLimitData = { + ...err.data, + method, + ...(header !== undefined && { retryAfter: header }), + ...(retryAfterMs !== undefined && { retryAfterMs }), + }; + const message = `Rate limited by the server (HTTP 429) on ${method}. ${describeRetry(header, retryAfterMs)} ${err.message}`; + return new SdkHttpError(err.code, message, data, { cause: err }); +} diff --git a/core/mcp/remote/node/remote-session.ts b/core/mcp/remote/node/remote-session.ts index cb0b16f89..c33765924 100644 --- a/core/mcp/remote/node/remote-session.ts +++ b/core/mcp/remote/node/remote-session.ts @@ -11,6 +11,11 @@ import { AuthChallengeError } from "../../../auth/challenge.js"; import type { RemoteAuthProviderHandle } from "./tokenAuthProvider.js"; import type { RemoteAuthState } from "../types.js"; import { progressTokenOf, waitForProgressToken } from "../progressToken.js"; +import { + rateLimitResponseOf, + retryAfterSince, + type RateLimitResponse, +} from "../../rateLimit.js"; export interface SessionEvent { type: RemoteEvent["type"]; @@ -47,6 +52,8 @@ export class RemoteSession { private readonly requestWaits = new Map(); private authProviderHandle: RemoteAuthProviderHandle | null = null; private appliedProtocolVersion: string | undefined; + /** The most recent `429` an upstream fetch drew, for its `Retry-After` (#2442). */ + private lastRateLimitResponse: RateLimitResponse | undefined; /** A protocol version is a dated token (`2025-11-25`, `2026-07-28`, …). */ private static readonly PROTOCOL_VERSION_PATTERN = /^[A-Za-z0-9._-]{1,64}$/; @@ -310,6 +317,8 @@ export class RemoteSession { } onFetchRequest(entry: FetchRequestEntryBase): void { + this.lastRateLimitResponse = + rateLimitResponseOf(entry) ?? this.lastRateLimitResponse; this.pushEvent({ type: "fetch_request", data: { @@ -322,6 +331,15 @@ export class RemoteSession { }); } + /** + * The `Retry-After` of the latest upstream `429` drawn by a fetch sent at or + * after `since` — read by the send route, so the header rides back on the + * send's own error rather than on a fetch event racing it (#2442). + */ + retryAfterSince(since: number): string | undefined { + return retryAfterSince(this.lastRateLimitResponse, since); + } + onFetchResponseBody(id: string, responseBody: string): void { this.pushEvent({ type: "fetch_request_body_update", diff --git a/core/mcp/remote/node/server.ts b/core/mcp/remote/node/server.ts index 0c677fb36..154ad507d 100644 --- a/core/mcp/remote/node/server.ts +++ b/core/mcp/remote/node/server.ts @@ -30,6 +30,7 @@ import { createTransportNode } from "../../node/transport.js"; import { createProxyFetch } from "../../node/proxyFetch.js"; import { OAUTH_TIMEOUT_WIRE_CODE } from "../../../auth/requestTimeout.js"; import { redactUrlQuery } from "../../fetchTracking.js"; +import { isRateLimitError } from "../../rateLimit.js"; import type { RemoteConnectRequest, RemoteSendRequest, @@ -1045,6 +1046,8 @@ export function createRemoteApp( // on this request and everything after it (#1935). session.applyProtocolVersion(protocolVersion); + // Bounds which upstream 429 can be this send's (#2442). + const sendStartedAt = Date.now(); session.beginSend(); const requestId = requestIdForSendWait(message); let responseWait: Promise | undefined; @@ -1115,6 +1118,9 @@ export function createRemoteApp( // browser rebuilds an `SdkHttpError` the era probe can classify (#2660). if (err instanceof SdkHttpError) { const text = err.data.text; + const retryAfter = isRateLimitError(err) + ? session.retryAfterSince(sendStartedAt) + : undefined; return c.json({ ok: false, kind: "transport_error", @@ -1126,6 +1132,7 @@ export function createRemoteApp( statusText: err.statusText, }), ...(typeof text === "string" && { text }), + ...(retryAfter !== undefined && { retryAfter }), }, }); } diff --git a/core/mcp/remote/remoteClientTransport.ts b/core/mcp/remote/remoteClientTransport.ts index 9f9a5462f..6339ebbda 100644 --- a/core/mcp/remote/remoteClientTransport.ts +++ b/core/mcp/remote/remoteClientTransport.ts @@ -182,6 +182,7 @@ function toSdkHttpError( status: http.status, ...(http.statusText !== undefined && { statusText: http.statusText }), ...(http.text !== undefined && { text: http.text }), + ...(http.retryAfter !== undefined && { retryAfter: http.retryAfter }), }); } diff --git a/core/mcp/remote/types.ts b/core/mcp/remote/types.ts index 006d16d04..6caa918d4 100644 --- a/core/mcp/remote/types.ts +++ b/core/mcp/remote/types.ts @@ -118,6 +118,11 @@ export interface RemoteHttpErrorData { statusText?: string; /** The upstream response body, which the probe parses for a JSON-RPC error. */ text?: string; + /** + * The upstream `Retry-After` header on a `429`, which the SDK's error does + * not carry; read off the backend's own tracked fetch (#2442). + */ + retryAfter?: string; } export type RemoteSendResponse = From 1a536d08f8d583ea310f96d84c0b61e57fcae381 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Mon, 19 Oct 2026 23:54:31 -0400 Subject: [PATCH 21/44] fix(oauth): refresh an expired token behind an insufficient_scope challenge Some resource servers answer an expired access token with 403 insufficient_scope. checkAuthChallengeSatisfied() tested only scope for that reason, so it reported the challenge satisfied, the transport replayed the same dead token, and recovery gave up without trying the refresh token: the gap #2051 closed for token_expired. insufficient_scope now also requires a provably unexpired token. Callers that have just completed a grant pass { scopeOnly: true }: the refresh, re-mint and forced re-authorization legs in handleAuthChallenge, the runner after its callback, and the web client on OAuth resume. A freshly issued opaque token has no readable exp, so the liveness test would reject it. An EMA challenge whose scope storage already covers skips the step-up confirm, since re-minting it broadens nothing. Closes #2724 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../web/src/hooks/useOAuthRecovery.test.tsx | 6 + clients/web/src/hooks/useOAuthRecovery.ts | 5 +- .../auth/runner-interactive-oauth.test.ts | 5 + .../src/test/core/mcp/oauthManager.test.ts | 113 +++++++++++++++++- core/auth/challenge.ts | 10 ++ core/auth/node/runner-interactive-oauth.ts | 15 ++- core/mcp/inspectorClient.ts | 7 +- core/mcp/oauthManager.ts | 54 +++++++-- specification/v2_auth_mid_session.md | 6 +- 9 files changed, 199 insertions(+), 22 deletions(-) diff --git a/clients/web/src/hooks/useOAuthRecovery.test.tsx b/clients/web/src/hooks/useOAuthRecovery.test.tsx index 6ba9f1807..e0f3f3bd3 100644 --- a/clients/web/src/hooks/useOAuthRecovery.test.tsx +++ b/clients/web/src/hooks/useOAuthRecovery.test.tsx @@ -1985,6 +1985,12 @@ describe("useOAuthRecovery", () => { await waitFor(() => expect(toastTitles()).toContain("Authorization complete"), ); + // #2724: the callback just minted the token, so only scope is checked — + // a liveness test would reject a fresh opaque token. + expect(client.checkAuthChallengeSatisfied).toHaveBeenCalledWith( + expect.objectContaining({ reason: "insufficient_scope" }), + { scopeOnly: true }, + ); }); }); diff --git a/clients/web/src/hooks/useOAuthRecovery.ts b/clients/web/src/hooks/useOAuthRecovery.ts index 313143a6a..7c47fe6f0 100644 --- a/clients/web/src/hooks/useOAuthRecovery.ts +++ b/clients/web/src/hooks/useOAuthRecovery.ts @@ -1478,7 +1478,10 @@ export function useOAuthRecovery({ : undefined; const stepUpScopesGranted = !stepUpChallenge || - (await client.checkAuthChallengeSatisfied(stepUpChallenge)); + // The callback just completed the grant, so only scope is in question. + (await client.checkAuthChallengeSatisfied(stepUpChallenge, { + scopeOnly: true, + })); if (stepUpChallenge && !stepUpScopesGranted) { notifications.show({ diff --git a/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts b/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts index e885e06d2..16145dec0 100644 --- a/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts +++ b/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts @@ -222,6 +222,11 @@ describe("runRunnerInteractiveOAuth", () => { }); expect(result).toEqual({ kind: "insufficient_scope", challenge }); + // #2724: the exchange just minted the token, so only scope is checked — a + // liveness test would reject a fresh opaque token. + expect(client.checkAuthChallengeSatisfied).toHaveBeenCalledWith(challenge, { + scopeOnly: true, + }); }); it("propagates completeOAuthFlow failures", async () => { diff --git a/clients/web/src/test/core/mcp/oauthManager.test.ts b/clients/web/src/test/core/mcp/oauthManager.test.ts index 96d1df80c..867d1d56d 100644 --- a/clients/web/src/test/core/mcp/oauthManager.test.ts +++ b/clients/web/src/test/core/mcp/oauthManager.test.ts @@ -15,6 +15,7 @@ import { } from "@inspector/core/auth/ema/clientConfigError.js"; import * as emaFlow from "@inspector/core/auth/ema/emaFlow.js"; import { mcpAuth } from "@inspector/core/auth/mcpAuth.js"; +import type { OAuthTokens } from "@modelcontextprotocol/client"; // Mock mcpAuth so OAuthManager tests do not hit the network. vi.mock("@inspector/core/auth/mcpAuth.js", async (importOriginal) => { @@ -1314,7 +1315,7 @@ describe("OAuthManager", () => { it("returns true when stored scope covers step-up union", async () => { const params = createMockParams(); storageOf(params).getTokens.mockResolvedValue({ - access_token: "tok", + access_token: jwtWithExp(Math.floor(Date.now() / 1000) + 3600), token_type: "Bearer", scope: "mcp tools:read tools:write", }); @@ -1383,6 +1384,112 @@ describe("OAuthManager", () => { ).toBe(false); }); + // #2724 — some resource servers answer an *expired* token with + // `insufficient_scope` rather than `invalid_token`. A scope-only check + // then calls the challenge satisfied, the transport replays the same dead + // token, and recovery gives up without ever trying the refresh token — + // the gap #2051 closed for `token_expired`. + describe("expired token behind an insufficient_scope challenge (#2724)", () => { + const COVERED = "mcp tools:read tools:write"; + const challenge = { + reason: "insufficient_scope" as const, + requiredScopes: ["tools:write"], + }; + const expiredJwt = () => jwtWithExp(Math.floor(Date.now() / 1000) - 60); + + it("is not satisfied by an expired JWT whose scope is covered", async () => { + const params = createMockParams(); + storageOf(params).getTokens.mockResolvedValue({ + access_token: expiredJwt(), + token_type: "Bearer", + scope: COVERED, + }); + storageOf(params).getScope.mockResolvedValue(COVERED); + const manager = new OAuthManager(params); + + expect(await manager.checkAuthChallengeSatisfied(challenge)).toBe( + false, + ); + }); + + it("is not satisfied by an opaque token whose scope is covered", async () => { + const params = createMockParams(); + storageOf(params).getTokens.mockResolvedValue({ + access_token: "opaque-tok", + token_type: "Bearer", + scope: COVERED, + }); + storageOf(params).getScope.mockResolvedValue(COVERED); + const manager = new OAuthManager(params); + + expect(await manager.checkAuthChallengeSatisfied(challenge)).toBe( + false, + ); + }); + + it("refreshes instead of replaying, and accepts the refreshed opaque token", async () => { + const params = createMockParams(); + let stored: OAuthTokens = { + access_token: expiredJwt(), + refresh_token: "r1", + token_type: "Bearer", + scope: COVERED, + }; + storageOf(params).getTokens.mockImplementation(async () => stored); + storageOf(params).getScope.mockResolvedValue(COVERED); + mockedMcpAuth.mockImplementation(async () => { + // A refresh grant that returns an opaque token: no readable `exp`, + // so only a scope-only post-grant check can call it satisfied. + stored = { + access_token: "opaque-refreshed", + refresh_token: "r2", + token_type: "Bearer", + scope: COVERED, + }; + return "AUTHORIZED"; + }); + const manager = new OAuthManager(params); + + const outcome = await manager.handleAuthChallenge(challenge); + + expect(outcome).toEqual({ kind: "satisfied" }); + expect(mockedMcpAuth).toHaveBeenCalledOnce(); + // Covered scope: an ordinary refresh, not a forced re-authorization. + expect(mockedMcpAuth.mock.calls[0]?.[1]).toEqual( + expect.objectContaining({ forceReauthorization: false }), + ); + }); + + it("re-mints an EMA token without a step-up prompt when nothing is broadened", async () => { + const params = createMockParams({ + enterpriseManagedAuth: { + idp: { + issuer: "https://idp.example.com", + clientId: "app-client", + clientSecret: "secret", + }, + }, + }); + storageOf(params).getTokens.mockResolvedValue({ + access_token: expiredJwt(), + token_type: "Bearer", + scope: COVERED, + }); + storageOf(params).getScope.mockResolvedValue(COVERED); + const silentSpy = vi + .spyOn(emaFlow, "trySilentEmaAuth") + .mockResolvedValue({ status: "success" }); + const manager = new OAuthManager(params); + manager.setOAuthConfig({ enterpriseManaged: true }); + + const outcome = await manager.handleAuthChallenge(challenge); + + expect(outcome).toEqual({ kind: "satisfied" }); + expect(silentSpy).toHaveBeenCalledOnce(); + silentSpy.mockRestore(); + }); + }); + // #2068 round 11 — the callback leg of an *ordinary* authorization. // `resolvePersistedScopeAfterGrant` falls back to the requested scope when // the token response omits `scope` (RFC 6749 §5.1), but only step-up ever @@ -1559,7 +1666,7 @@ describe("OAuthManager", () => { it("short-circuits handleAuthChallenge when scope already satisfied", async () => { const params = createMockParams(); storageOf(params).getTokens.mockResolvedValue({ - access_token: "tok", + access_token: jwtWithExp(Math.floor(Date.now() / 1000) + 3600), token_type: "Bearer", scope: "mcp tools:read tools:write", }); @@ -2390,7 +2497,7 @@ describe("OAuthManager", () => { scope: "mcp tools:read", }; const sufficientTokens = { - access_token: "a2", + access_token: jwtWithExp(Math.floor(Date.now() / 1000) + 3600), token_type: "Bearer", scope: "mcp tools:read weather:read", }; diff --git a/core/auth/challenge.ts b/core/auth/challenge.ts index 5f7ee5e9b..fcbfc002f 100644 --- a/core/auth/challenge.ts +++ b/core/auth/challenge.ts @@ -67,6 +67,16 @@ export interface HandleAuthChallengeOptions { confirmedStepUp?: boolean; } +export interface CheckAuthChallengeSatisfiedOptions { + /** + * `insufficient_scope` only: test scope coverage and skip the liveness test. + * For a caller that has just completed a grant — the grant is the evidence + * the token is live, and a freshly issued *opaque* token has no readable + * `exp` to prove it (#2724). + */ + scopeOnly?: boolean; +} + export interface ParseAuthChallengeContext { method?: string; toolName?: string; diff --git a/core/auth/node/runner-interactive-oauth.ts b/core/auth/node/runner-interactive-oauth.ts index 6e084d85f..d54681598 100644 --- a/core/auth/node/runner-interactive-oauth.ts +++ b/core/auth/node/runner-interactive-oauth.ts @@ -1,4 +1,7 @@ -import type { AuthChallenge } from "../challenge.js"; +import type { + AuthChallenge, + CheckAuthChallengeSatisfiedOptions, +} from "../challenge.js"; import { findIssuerBindingFailure } from "../issuerBinding.js"; import { issuerBindingFailureCopy } from "../oauthUx.js"; import { @@ -12,7 +15,10 @@ export interface RunnerInteractiveOAuthClient { authenticate(): Promise; beginInteractiveAuthorization(authorizationUrl: URL): Promise; completeOAuthFlow(authorizationCode: string, iss?: string): Promise; - checkAuthChallengeSatisfied(challenge: AuthChallenge): Promise; + checkAuthChallengeSatisfied( + challenge: AuthChallenge, + options?: CheckAuthChallengeSatisfiedOptions, + ): Promise; } export interface RunnerInteractiveOAuthRedirectProvider { @@ -204,7 +210,10 @@ export async function runRunnerInteractiveOAuth( if (options.authChallenge) { const satisfied = await racingSignals( - options.client.checkAuthChallengeSatisfied(options.authChallenge), + // The token exchange just completed, so only scope is in question. + options.client.checkAuthChallengeSatisfied(options.authChallenge, { + scopeOnly: true, + }), ); if (!satisfied) { return { diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index 818c4f60e..8ccb383ca 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -270,6 +270,7 @@ import { parseAuthChallengeFromError, type AuthChallenge, type AuthChallengeOutcome, + type CheckAuthChallengeSatisfiedOptions, type HandleAuthChallengeOptions, } from "../auth/challenge.js"; import { withOAuthEndpointOverrides } from "../auth/endpointOverrides.js"; @@ -6133,8 +6134,12 @@ export class InspectorClient extends InspectorClientEventTarget { */ async checkAuthChallengeSatisfied( challenge: AuthChallenge, + options?: CheckAuthChallengeSatisfiedOptions, ): Promise { - return this.ensureOAuthManager().checkAuthChallengeSatisfied(challenge); + return this.ensureOAuthManager().checkAuthChallengeSatisfied( + challenge, + options, + ); } /** diff --git a/core/mcp/oauthManager.ts b/core/mcp/oauthManager.ts index 0f8f919ae..bff565aed 100644 --- a/core/mcp/oauthManager.ts +++ b/core/mcp/oauthManager.ts @@ -42,6 +42,7 @@ import { EmaTransportOAuthProvider } from "../auth/ema/transportProvider.js"; import type { AuthChallenge, AuthChallengeOutcome, + CheckAuthChallengeSatisfiedOptions, HandleAuthChallengeOptions, } from "../auth/challenge.js"; import { @@ -597,16 +598,23 @@ export class OAuthManager { * Re-read persisted OAuth state and determine whether `challenge` is already * satisfied without an authorization-server round-trip. * - * Returns `true` for `insufficient_scope` when stored + token scope cover the - * SEP-2350 union. For `token_expired`, returns `true` only when storage holds - * a *provably* unexpired token — a JWT whose `exp` is still in the future. An - * opaque token carries no local expiry evidence, so the resource server's - * verdict stands and re-authorization proceeds. `invalid_token` and - * `unauthorized` always return `false` — the resource server explicitly - * rejected the credential. + * For `token_expired` and `insufficient_scope`, returns `true` only when + * storage holds a *provably* unexpired token — a JWT whose `exp` is still in + * the future. An opaque token carries no local expiry evidence, so the + * resource server's verdict stands and re-authorization proceeds. + * `insufficient_scope` additionally requires stored + token scope to cover + * the SEP-2350 union: some resource servers answer an *expired* token with + * `insufficient_scope`, and a scope-only check would replay that dead token + * instead of refreshing it (#2724, the same gap #2051 closed for + * `token_expired`). `invalid_token` and `unauthorized` always return + * `false` — the resource server explicitly rejected the credential. + * + * A caller that has just completed a grant passes `scopeOnly` — see + * {@link CheckAuthChallengeSatisfiedOptions}. */ async checkAuthChallengeSatisfied( challenge: AuthChallenge, + options?: CheckAuthChallengeSatisfiedOptions, ): Promise { const storage = this.oauthConfig.storage; if (!storage) { @@ -626,6 +634,19 @@ export class OAuthManager { ); } + if (!options?.scopeOnly && !isAccessTokenProvablyUnexpired(tokens)) { + return false; + } + return this.isStepUpScopeCovered(challenge, storage, tokens); + } + + /** Stored + token scope cover the challenge's SEP-2350 union. */ + private async isStepUpScopeCovered( + challenge: AuthChallenge, + storage: OAuthStorage, + tokens: OAuthTokens, + ): Promise { + const serverUrl = this.getServerUrl(); const enriched = await this.enrichChallengeWithAuthorizationScopes( challenge, tokens.scope, @@ -771,7 +792,14 @@ export class OAuthManager { const enriched = await this.enrichChallengeWithAuthorizationScopes(challenge); - if (enriched.reason === "insufficient_scope" && !options?.confirmedStepUp) { + // A challenge whose scope storage already covers reached here only because + // the stored token is not provably live (#2724) — that is an expiry, and + // re-minting it broadens nothing, so it needs no step-up consent. + if ( + enriched.reason === "insufficient_scope" && + !options?.confirmedStepUp && + !(await this.checkAuthChallengeSatisfied(enriched, { scopeOnly: true })) + ) { return { kind: "step_up_confirm", challenge: enriched }; } @@ -780,7 +808,9 @@ export class OAuthManager { if (enriched.reason === "insufficient_scope") { const silent = await trySilentEmaAuth(config); if (silent.status === "success") { - if (await this.checkAuthChallengeSatisfied(enriched)) { + if ( + await this.checkAuthChallengeSatisfied(enriched, { scopeOnly: true }) + ) { const minted = await this.requireStorage().getTokens( this.getServerUrl(), ); @@ -903,7 +933,9 @@ export class OAuthManager { if (result === "AUTHORIZED") { if (enriched.reason === "insufficient_scope") { - if (await this.checkAuthChallengeSatisfied(enriched)) { + if ( + await this.checkAuthChallengeSatisfied(enriched, { scopeOnly: true }) + ) { const freshTokens = await provider.tokens(); const scopeToPersist = resolvePersistedScopeAfterGrant( freshTokens?.scope, @@ -1044,7 +1076,7 @@ export class OAuthManager { if (result !== "AUTHORIZED") { return null; } - if (await this.checkAuthChallengeSatisfied(enriched)) { + if (await this.checkAuthChallengeSatisfied(enriched, { scopeOnly: true })) { const freshTokens = await provider.tokens(); const scopeToPersist = resolvePersistedScopeAfterGrant( freshTokens?.scope, diff --git a/specification/v2_auth_mid_session.md b/specification/v2_auth_mid_session.md index c022bedae..2c71d6d9c 100644 --- a/specification/v2_auth_mid_session.md +++ b/specification/v2_auth_mid_session.md @@ -208,7 +208,7 @@ Read-only check against **current storage** (and token expiry helpers). Used bef - **`token_expired`:** a **provably** unexpired access token in storage — a JWT whose `exp` is still in the future (60s skew). Storage records no absolute expiry for an opaque token (`expires_in` is relative to an issuance instant that is never persisted), so "expiry unknown" returns **false**: the resource server has said the credential is dead, and no local evidence contradicts it. Otherwise an opaque token would be replayed forever ([#2051](https://github.com/modelcontextprotocol/inspector/issues/2051)). - **`invalid_token` / `unauthorized`:** always **false** — the resource server explicitly rejected the credential. -- **`insufficient_scope`:** stored/token scope is a superset of required / union scopes. Empty scope on an `insufficient_scope` challenge returns **false** (do not short-circuit). +- **`insufficient_scope`:** a **provably** unexpired access token in storage (the same JWT `exp` test as `token_expired`) **and** stored/token scope that is a superset of required / union scopes. Some resource servers answer an _expired_ token with `insufficient_scope`; a scope-only check would call that challenge satisfied and replay the dead token instead of refreshing it ([#2724](https://github.com/modelcontextprotocol/inspector/issues/2724)). Empty scope on an `insufficient_scope` challenge returns **false** (do not short-circuit). A caller that has just completed a grant — the refresh, re-mint and forced re-authorization legs inside `handleAuthChallenge()`, the runner (CLI/TUI) after its callback, and the web client on OAuth resume — passes **`{ scopeOnly: true }`** to test scope only: the grant is the liveness evidence, and a freshly issued opaque token has no readable `exp`. ### Strategy by protocol @@ -217,7 +217,7 @@ Read-only check against **current storage** (and token expiry helpers). Used bef | Reason | Silent | Interactive | | ------------------------------------------------ | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `token_expired`, `invalid_token`, `unauthorized` | Refresh via `refresh_token` when supported | Authorization code flow (`authenticate()`) | -| `insufficient_scope` | N/A | Authorize with **`authorizationScopes`** = union(previous, challenge) via `mcpAuth({ forceReauthorization: true })`; navigation **deferred** until UI confirms (web modal / TUI Auth / CLI prompt) | +| `insufficient_scope` | Refresh via `refresh_token` when stored scope already covers the challenge (an expired token mislabelled as a scope failure, #2724) | Authorize with **`authorizationScopes`** = union(previous, challenge) via `mcpAuth({ forceReauthorization: true })`; navigation **deferred** until UI confirms (web modal / TUI Auth / CLI prompt) | Union scope is held in `pendingAuthorizationScope` until `completeOAuthFlow()` succeeds; cleared on failure. On success, `saveScope()` stores the AS-granted scope (explicit `scope` on the token response, or the requested union when `scope` is omitted per RFC 6749 §5.1). @@ -229,7 +229,7 @@ Per [v2_auth_ema.md](v2_auth_ema.md): **no** fallback to standard resource-OAuth | ------------------------------------- | ------------------------------------------------------------------------------------------------------ | -------------------------------------- | | Resource token expired / unauthorized | Legs 2–3 re-mint | — | | IdP session missing | — | Leg 1 IdP OIDC redirect, then legs 2–3 | -| `insufficient_scope` | After user confirms (web modal / TUI / CLI), re-mint legs 2–3 with union scopes when IdP session valid | IdP redirect if IdP session invalid | +| `insufficient_scope` | After user confirms (web modal / TUI / CLI), re-mint legs 2–3 with union scopes when IdP session valid; no confirm when stored scope already covers the challenge (nothing is broadened, #2724) | IdP redirect if IdP session invalid | On **web**, `handleAuthChallenge()` returns **`step_up_confirm`** for EMA `insufficient_scope` until the user clicks **Authorize** in `StepUpAuthModal`; it does **not** call `trySilentEmaAuth()` before that. After confirm, silent re-mint runs in-process when possible; otherwise IdP redirect + callback (same as before). From ea35ae36d6e286354f17ef25e8aefda264ca50e2 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:00:07 -0400 Subject: [PATCH 22/44] fix(core): correlate each 429's Retry-After by JSON-RPC id, not by time MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "The latest 429 since the request started" let a concurrent request's response overwrite another's: request A, whose 429 body was still in flight, reported request B's Retry-After. File each 429's header under the request id(s) in its tracked request body, and stamp it onto the error at the point that knows which message failed — trackSendFailure directly, the send route on the web backend. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../web/src/test/core/mcp/rateLimit.test.ts | 168 ++++++++++++------ .../mcp/inspectorClient-rate-limit.test.ts | 94 +++++++++- core/mcp/inspectorClient.ts | 25 ++- core/mcp/rateLimit.ts | 132 +++++++++----- core/mcp/remote/node/remote-session.ts | 23 +-- core/mcp/remote/node/server.ts | 14 +- 6 files changed, 330 insertions(+), 126 deletions(-) diff --git a/clients/web/src/test/core/mcp/rateLimit.test.ts b/clients/web/src/test/core/mcp/rateLimit.test.ts index 3e2617463..2cdf8870c 100644 --- a/clients/web/src/test/core/mcp/rateLimit.test.ts +++ b/clients/web/src/test/core/mcp/rateLimit.test.ts @@ -6,10 +6,12 @@ import { } from "@modelcontextprotocol/client"; import { annotateRateLimit, + attachRetryAfter, isRateLimitError, + MAX_UNCLAIMED_RATE_LIMITS, parseRetryAfter, - rateLimitResponseOf, - retryAfterSince, + RateLimitLedger, + requestIdsOf, type AnnotatedRateLimitData, } from "@inspector/core/mcp/rateLimit.js"; import type { FetchRequestEntryBase } from "@inspector/core/mcp/types.js"; @@ -25,6 +27,11 @@ function entry( method: "POST", url: "http://localhost/mcp", requestHeaders: {}, + requestBody: JSON.stringify({ + jsonrpc: "2.0", + id: 7, + method: "tools/list", + }), responseStatus: 429, responseHeaders: { "Retry-After": "30" }, ...overrides, @@ -39,43 +46,115 @@ function throttled(data: Record = {}): SdkHttpError { ); } -describe("rateLimitResponseOf", () => { - it("records a 429's Retry-After, whatever its casing, and when it was sent", () => { - expect(rateLimitResponseOf(entry())).toEqual({ - retryAfter: "30", - sentAt: NOW, - }); +describe("requestIdsOf", () => { + it("reads a single request's id, string or number", () => { + expect( + requestIdsOf('{"jsonrpc":"2.0","id":7,"method":"tools/list"}'), + ).toEqual([7]); + expect(requestIdsOf('{"jsonrpc":"2.0","id":"a","method":"x"}')).toEqual([ + "a", + ]); }); - it("records a 429 with no Retry-After", () => { - expect(rateLimitResponseOf(entry({ responseHeaders: {} }))).toEqual({ - sentAt: NOW, - }); + it("reads every request in a batch, skipping notifications and junk", () => { + expect( + requestIdsOf( + JSON.stringify([ + { jsonrpc: "2.0", id: 1, method: "a" }, + { jsonrpc: "2.0", method: "notifications/x" }, + { jsonrpc: "2.0", id: 2, result: {} }, + { jsonrpc: "2.0", id: null, method: "b" }, + null, + { jsonrpc: "2.0", id: 3, method: "c" }, + ]), + ), + ).toEqual([1, 3]); }); - it("ignores anything that is not a 429", () => { - expect(rateLimitResponseOf(entry({ responseStatus: 503 }))).toBeUndefined(); - expect( - rateLimitResponseOf(entry({ responseStatus: undefined })), - ).toBeUndefined(); + it("finds nothing in a missing or non-JSON body", () => { + expect(requestIdsOf(undefined)).toEqual([]); + expect(requestIdsOf("not json")).toEqual([]); + expect(requestIdsOf("42")).toEqual([]); }); }); -describe("retryAfterSince", () => { - it("returns the header of a 429 sent at or after the bound", () => { - const latest = { retryAfter: "30", sentAt: NOW }; - expect(retryAfterSince(latest, NOW)).toBe("30"); - expect(retryAfterSince(latest, NOW - 1)).toBe("30"); +describe("RateLimitLedger", () => { + it("files a 429's header, whatever its casing, under its request id, once", () => { + const ledger = new RateLimitLedger(); + ledger.record(entry()); + expect(ledger.take(7)).toBe("30"); + expect(ledger.take(7)).toBeUndefined(); }); - it("drops a 429 that predates the request", () => { - expect(retryAfterSince({ retryAfter: "30", sentAt: NOW }, NOW + 1)).toBe( - undefined, + it("keeps each overlapping request's own header", () => { + const ledger = new RateLimitLedger(); + ledger.record(entry()); + ledger.record( + entry({ + requestBody: JSON.stringify({ jsonrpc: "2.0", id: 8, method: "x" }), + responseHeaders: { "retry-after": "5" }, + }), ); + expect(ledger.take(7)).toBe("30"); + expect(ledger.take(8)).toBe("5"); }); - it("returns nothing when no 429 was recorded", () => { - expect(retryAfterSince(undefined, NOW)).toBeUndefined(); + it("files a 429 with no header as having none", () => { + const ledger = new RateLimitLedger(); + ledger.record(entry({ responseHeaders: {} })); + expect(ledger.take(7)).toBeUndefined(); + }); + + it("ignores anything that is not a 429", () => { + const ledger = new RateLimitLedger(); + ledger.record(entry({ responseStatus: 503 })); + ledger.record(entry({ responseStatus: undefined })); + expect(ledger.take(7)).toBeUndefined(); + }); + + it("drops the oldest unclaimed 429 past its cap", () => { + const ledger = new RateLimitLedger(); + for (let id = 0; id <= MAX_UNCLAIMED_RATE_LIMITS; id++) { + ledger.record( + entry({ + requestBody: JSON.stringify({ jsonrpc: "2.0", id, method: "x" }), + }), + ); + } + expect(ledger.take(0)).toBeUndefined(); + expect(ledger.take(1)).toBe("30"); + expect(ledger.take(MAX_UNCLAIMED_RATE_LIMITS)).toBe("30"); + }); + + it("forgets everything on clear", () => { + const ledger = new RateLimitLedger(); + ledger.record(entry()); + ledger.clear(); + expect(ledger.take(7)).toBeUndefined(); + }); +}); + +describe("attachRetryAfter", () => { + it("stamps the header onto a 429's data", () => { + const err = throttled(); + attachRetryAfter(err, "30"); + expect(err.data.retryAfter).toBe("30"); + }); + + it("keeps a header already relayed on the error", () => { + const err = throttled({ retryAfter: "5" }); + attachRetryAfter(err, "30"); + expect(err.data.retryAfter).toBe("5"); + }); + + it("leaves a non-429, or a missing header, alone", () => { + const unavailable = throttled({ status: 503 }); + attachRetryAfter(unavailable, "30"); + expect(unavailable.data.retryAfter).toBeUndefined(); + const err = throttled(); + attachRetryAfter(err, undefined); + expect(err.data).not.toHaveProperty("retryAfter"); + expect(() => attachRetryAfter(undefined, "30")).not.toThrow(); }); }); @@ -123,8 +202,8 @@ describe("isRateLimitError", () => { describe("annotateRateLimit", () => { it("names the method and the retry delay, keeping class, code, status and cause", () => { - const original = throttled(); - const annotated = annotateRateLimit(original, "tools/call", "30", NOW); + const original = throttled({ retryAfter: "30" }); + const annotated = annotateRateLimit(original, "tools/call", NOW); expect(SdkHttpError.isInstance(annotated)).toBe(true); const error = annotated as SdkHttpError; @@ -143,32 +222,21 @@ describe("annotateRateLimit", () => { expect(data.retryAfterMs).toBe(30_000); }); - it("prefers a Retry-After the backend relayed on the error", () => { - const annotated = annotateRateLimit( - throttled({ retryAfter: "5" }), - "tools/list", - "30", - NOW, - ) as SdkHttpError; - expect(annotated.message).toContain("retry after 5s"); - expect((annotated.data as AnnotatedRateLimitData).retryAfter).toBe("5"); - }); - - it("ignores a relayed retryAfter that is not a string", () => { + it("ignores a retryAfter on the data that is not a string", () => { const annotated = annotateRateLimit( throttled({ retryAfter: 5 }), "tools/list", - "30", NOW, ) as SdkHttpError; - expect(annotated.message).toContain("retry after 30s"); + expect(annotated.message).toContain( + "The server sent no Retry-After header.", + ); }); it("says when the server sent no Retry-After", () => { const annotated = annotateRateLimit( throttled(), "tools/list", - undefined, NOW, ) as SdkHttpError; expect(annotated.message).toContain( @@ -181,9 +249,8 @@ describe("annotateRateLimit", () => { it("reports an unparseable Retry-After verbatim rather than guessing", () => { const annotated = annotateRateLimit( - throttled(), + throttled({ retryAfter: "soon" }), "tools/list", - "soon", NOW, ) as SdkHttpError; expect(annotated.message).toContain( @@ -196,19 +263,16 @@ describe("annotateRateLimit", () => { it("defaults the clock to now", () => { const annotated = annotateRateLimit( - throttled(), + throttled({ retryAfter: "1" }), "tools/list", - "1", ) as SdkHttpError; expect(annotated.message).toContain("retry after 1s"); }); it("returns anything that is not a 429 untouched", () => { const unavailable = throttled({ status: 503 }); - expect(annotateRateLimit(unavailable, "tools/list", "30", NOW)).toBe( - unavailable, - ); + expect(annotateRateLimit(unavailable, "tools/list", NOW)).toBe(unavailable); const timeout = new SdkError(SdkErrorCode.RequestTimeout, "timed out"); - expect(annotateRateLimit(timeout, "tools/list", "30", NOW)).toBe(timeout); + expect(annotateRateLimit(timeout, "tools/list", NOW)).toBe(timeout); }); }); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts index 0a0e1379e..86e826732 100644 --- a/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts +++ b/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts @@ -27,7 +27,8 @@ import type { AnnotatedRateLimitData } from "@inspector/core/mcp/rateLimit.js"; * by the web backend on the remote one. Both are driven end to end here. * * Hand-rolled rather than a composable test server because the SDK's own - * server cannot be configured to answer one method with a 429. + * server cannot be configured to answer one method with a 429, let alone hold + * one 429's body back while another overtakes it. */ interface ToolsListReply { @@ -39,9 +40,19 @@ interface ToolsListReply { interface ThrottlingServer { url: string; setToolsListReply: (reply: ToolsListReply) => void; + /** + * The methods of the overlapping pair (`prompts/list`, `resources/list`), in + * the order they arrived. The first is answered `429` + `Retry-After: 30` + * with its body held back; the second `429` + `Retry-After: 5` at once. + */ + overlapArrivals: () => string[]; + /** Finish the held first response of the overlapping pair. */ + releaseFirstOverlap: () => void; stop: () => Promise; } +const OVERLAPPING = new Set(["prompts/list", "resources/list"]); + const THROTTLED: ToolsListReply = { status: 429, headers: { "retry-after": "30" }, @@ -50,6 +61,8 @@ const THROTTLED: ToolsListReply = { function startThrottlingServer(): Promise { let toolsListReply = THROTTLED; + const overlapArrivals: string[] = []; + let releaseFirstOverlap = () => {}; const handler = async (req: IncomingMessage, res: ServerResponse) => { // No standalone notification stream; nothing here needs one. @@ -79,11 +92,28 @@ function startThrottlingServer(): Promise { res.end(toolsListReply.body); return; } + if (OVERLAPPING.has(body.method)) { + overlapArrivals.push(body.method); + const first = overlapArrivals.length === 1; + res.writeHead(429, { + "content-type": "text/plain", + "retry-after": first ? "30" : "5", + }); + if (first) { + // Headers out now, so the 429 is tracked; the SDK's error waits on + // the body, which only the test releases. + res.flushHeaders(); + releaseFirstOverlap = () => res.end("first"); + } else { + res.end("second"); + } + return; + } const result = body.method === "initialize" ? { protocolVersion: "2025-06-18", - capabilities: { tools: {} }, + capabilities: { tools: {}, prompts: {}, resources: {} }, serverInfo: { name: "throttling-server", version: "1.0.0" }, } : {}; @@ -105,6 +135,8 @@ function startThrottlingServer(): Promise { setToolsListReply: (reply) => { toolsListReply = reply; }, + overlapArrivals: () => [...overlapArrivals], + releaseFirstOverlap: () => releaseFirstOverlap(), stop: () => new Promise((done) => { server.closeAllConnections(); @@ -140,6 +172,21 @@ function startRemoteBackend(): Promise<{ }); } +/** Poll until `predicate` holds; the failure names what never became true. */ +async function waitUntil( + what: string, + predicate: () => boolean, + timeoutMs = 5000, +): Promise { + const deadline = Date.now() + timeoutMs; + while (!predicate()) { + if (Date.now() > deadline) { + throw new Error(`timed out waiting for ${what}`); + } + await new Promise((r) => setTimeout(r, 10)); + } +} + async function rejectionOf(promise: Promise): Promise { try { await promise; @@ -260,4 +307,47 @@ describe("InspectorClient rate-limit recognition (#2442)", () => { ); expect(data.retryAfter).toBe("30"); }); + + /** + * Two requests throttled at once, the first one's response overtaken by + * the second's. Each must report its own response's `Retry-After`; picking + * "the latest 429" would hand the first request the second's 5s. + */ + async function expectOverlapCorrelated(c: InspectorClient): Promise { + const first = rejectionOf(c.listPrompts()); + await waitUntil( + "the first request to arrive", + () => upstream!.overlapArrivals().length === 1, + ); + const second = await rejectionOf(c.listResources()); + upstream!.releaseFirstOverlap(); + const firstCaught = await first; + + expect(upstream!.overlapArrivals()).toEqual([ + "prompts/list", + "resources/list", + ]); + expect(expectThrottled(firstCaught).retryAfter).toBe("30"); + expect((firstCaught as Error).message).toContain( + "on prompts/list. The server asked to retry after 30s.", + ); + expect(expectThrottled(second).retryAfter).toBe("5"); + expect((second as Error).message).toContain( + "on resources/list. The server asked to retry after 5s.", + ); + } + + it("gives each of two overlapping 429s its own Retry-After", async () => { + upstream = await startThrottlingServer(); + await expectOverlapCorrelated(await connectTo(upstream.url)); + }); + + it("gives each of two overlapping 429s its own Retry-After through the web backend", async () => { + upstream = await startThrottlingServer(); + const started = await startRemoteBackend(); + backend = started.server; + await expectOverlapCorrelated( + await connectTo(upstream.url, started.transport), + ); + }); }); diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index 10f6187ce..e9b026901 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -298,9 +298,8 @@ import { } from "./connectionDiagnostics.js"; import { annotateRateLimit, - rateLimitResponseOf, - retryAfterSince, - type RateLimitResponse, + attachRetryAfter, + RateLimitLedger, } from "./rateLimit.js"; import { OAuthManager, type OAuthManagerConfig } from "./oauthManager.js"; import { RemoteClientTransport } from "./remote/remoteClientTransport.js"; @@ -561,9 +560,10 @@ export class InspectorClient extends InspectorClientEventTarget { // `GET` on Streamable HTTP). Both per-session, reset with the maps above. private lastResponse: LastResponse | undefined; private notificationStream: TrackedNotificationStream | undefined; - // The most recent `429` a transport fetch drew, kept for its `Retry-After` - // (#2442). Per-session, reset with the diagnostics above. - private lastRateLimitResponse: RateLimitResponse | undefined; + // Each tracked `429`'s `Retry-After`, by the JSON-RPC id that drew it, until + // that id's failed send claims it (#2442). Per-session, reset with the + // diagnostics above. + private readonly rateLimits = new RateLimitLedger(); /** True when the cached transport was built with an OAuth authProvider attached. */ private transportHasAuthProvider = false; /** Dedupes concurrent ambient auth challenges (reason + scopes). */ @@ -1114,9 +1114,6 @@ export class InspectorClient extends InspectorClientEventTarget { const decorated = async ( ...args: Parameters ): Promise => { - // A 429 is matched to this request by time: only one drawn by a fetch - // sent after the request started can be the response that failed it. - const sentAt = Date.now(); try { return await sdkRequest(...args); } catch (err) { @@ -1124,7 +1121,6 @@ export class InspectorClient extends InspectorClientEventTarget { throw annotateRateLimit( annotateRequestTimeout(err, method, this.getConnectionDiagnostics()), method, - retryAfterSince(this.lastRateLimitResponse, sentAt), ); } }; @@ -1192,6 +1188,10 @@ export class InspectorClient extends InspectorClientEventTarget { // saw as unanswered, in every timeout message and Connection Info // row, until the session resets (#2318). if (isRequestTimeoutError(error)) return; + // The SDK's 429 error carries no headers; this is the one point that + // knows which request it failed, so it is where that request's own + // `Retry-After` goes onto it (#2442). + attachRetryAfter(error, this.rateLimits.take(message.id)); if (this.outstandingRequests.delete(message.id)) { this.dispatchConnectionDiagnosticsChange(); } @@ -1687,7 +1687,7 @@ export class InspectorClient extends InspectorClientEventTarget { this.taskProgress.clear(); this.lastResponse = undefined; this.notificationStream = undefined; - this.lastRateLimitResponse = undefined; + this.rateLimits.clear(); this.dispatchConnectionDiagnosticsChange(); // Per-session for the same reason: both name entries of the PREVIOUS // server's list. Cleared here as well as in `disconnect()` because the @@ -1828,8 +1828,7 @@ export class InspectorClient extends InspectorClientEventTarget { onFetchRequest: (entry: FetchRequestEntryBase) => { this.dispatchFetchRequest({ ...entry, category: "transport" }); this.noteTransportStream(entry); - this.lastRateLimitResponse = - rateLimitResponseOf(entry) ?? this.lastRateLimitResponse; + this.rateLimits.record(entry); }, onFetchResponseBody: (id: string, body: string) => { this.dispatchFetchRequestBodyUpdate(id, body); diff --git a/core/mcp/rateLimit.ts b/core/mcp/rateLimit.ts index c20235d7f..1bbfe637d 100644 --- a/core/mcp/rateLimit.ts +++ b/core/mcp/rateLimit.ts @@ -11,14 +11,20 @@ * `Retry-After` header. * * The header has to come from the tracker because the SDK's `SdkHttpError` - * keeps the status and the body but none of the response headers. Where the - * upstream fetch runs in the web backend, the backend reads the same header - * off its own tracked fetch and relays it on the send error as `retryAfter` - * (see `RemoteHttpErrorData`), so the browser does not depend on a fetch - * event racing the send's HTTP response across two connections. + * keeps the status and the body but none of the response headers. It is + * matched to its error by JSON-RPC id, not by time: a {@link RateLimitLedger} + * files each 429's header under the request id(s) in the tracked request body, + * and the code that sees the failed send — which knows the message it was + * sending — takes that id's header and stamps it onto the error's data as + * `retryAfter` ({@link attachRetryAfter}). Two requests throttled at once + * therefore each report their own server's word, whatever order their + * responses land in. * - * Everything here is pure. `InspectorClient` owns the bookkeeping, the same - * split `connectionDiagnostics.ts` makes for request timeouts. + * Where the upstream fetch runs in the web backend, the backend does the + * stamping from its own ledger and relays `retryAfter` on the send error (see + * `RemoteHttpErrorData`), so the browser does not depend on a fetch event + * racing the send's HTTP response across two connections. Either way, by the + * time {@link annotateRateLimit} sees the error, the header is on it. * * Deliberately narrow: only a 429 is recognized. A 503 also carries * `Retry-After` but means "unavailable", which is exactly the case this exists @@ -36,40 +42,72 @@ import type { FetchRequestEntryBase } from "./types.js"; /** `429 Too Many Requests` (RFC 6585). */ export const HTTP_TOO_MANY_REQUESTS = 429; -/** A rate-limited response the fetch tracker recorded. */ -export interface RateLimitResponse { - /** The raw `Retry-After` header, when the server sent one. */ - retryAfter?: string; - /** Epoch ms at which the request that drew it went out. */ - sentAt: number; +/** + * The JSON-RPC request ids in a tracked request body: one for a single + * request, several for a batch, none for a notification, a response, or a + * body that is not JSON. The tracker records the body redacted, but redaction + * masks only sensitive string fields, so `id` survives it. + */ +export function requestIdsOf(body: string | undefined): (string | number)[] { + if (body === undefined) return []; + let parsed: unknown; + try { + parsed = JSON.parse(body); + } catch { + return []; + } + const messages: unknown[] = Array.isArray(parsed) ? parsed : [parsed]; + const ids: (string | number)[] = []; + for (const message of messages) { + if (typeof message !== "object" || message === null) continue; + if (!("method" in message) || !("id" in message)) continue; + const { id } = message; + if (typeof id === "string" || typeof id === "number") ids.push(id); + } + return ids; } /** - * The rate-limit facts in a tracked fetch, or `undefined` when the response - * was not a 429 (or the fetch failed before any response arrived). + * How many unclaimed 429s a ledger holds before it drops the oldest. A 429 is + * normally claimed within the same tick its send fails; one that never is (a + * send aborted mid-flight is not reported as a failure) would otherwise stay + * for the rest of the session. */ -export function rateLimitResponseOf( - entry: FetchRequestEntryBase, -): RateLimitResponse | undefined { - if (entry.responseStatus !== HTTP_TOO_MANY_REQUESTS) return undefined; - const retryAfter = findHeader(entry.responseHeaders, "retry-after"); - return { - ...(retryAfter !== undefined && { retryAfter }), - sentAt: entry.timestamp.getTime(), - }; -} +export const MAX_UNCLAIMED_RATE_LIMITS = 100; /** - * The latest recorded 429's `Retry-After`, when that 429 answered a request - * sent at or after `since`. The bound is what keeps a header from a 429 - * minutes ago from being reported against a request it never answered. + * Each tracked 429's `Retry-After`, filed under the JSON-RPC id of the request + * that drew it, until the failed send for that id claims it. */ -export function retryAfterSince( - latest: RateLimitResponse | undefined, - since: number, -): string | undefined { - if (!latest || latest.sentAt < since) return undefined; - return latest.retryAfter; +export class RateLimitLedger { + // `undefined` is a real value here: a 429 that carried no header. + private readonly byRequestId = new Map(); + + /** File a tracked fetch, if it was a 429. Anything else is ignored. */ + record(entry: FetchRequestEntryBase): void { + if (entry.responseStatus !== HTTP_TOO_MANY_REQUESTS) return; + const retryAfter = findHeader(entry.responseHeaders, "retry-after"); + for (const id of requestIdsOf(entry.requestBody)) { + this.byRequestId.delete(id); + this.byRequestId.set(id, retryAfter); + if (this.byRequestId.size > MAX_UNCLAIMED_RATE_LIMITS) { + const [oldest] = this.byRequestId.keys(); + /* v8 ignore next -- size > 0 was just checked, so a first key exists */ + if (oldest !== undefined) this.byRequestId.delete(oldest); + } + } + } + + /** The `Retry-After` filed for `id`, removed as it is read. */ + take(id: string | number): string | undefined { + const retryAfter = this.byRequestId.get(id); + this.byRequestId.delete(id); + return retryAfter; + } + + clear(): void { + this.byRequestId.clear(); + } } /** @@ -121,11 +159,27 @@ export interface AnnotatedRateLimitData extends SdkHttpErrorData { retryAfterMs?: number; } -function relayedRetryAfter(err: SdkHttpError): string | undefined { +function retryAfterOn(err: SdkHttpError): string | undefined { const value = err.data.retryAfter; return typeof value === "string" ? value : undefined; } +/** + * Stamp a 429's own `Retry-After` onto its error's data, where + * {@link annotateRateLimit} reads it. A header already there wins: on the + * browser's remote transport the backend relayed it from the upstream + * response itself, which nothing on this side can know better. Anything that + * is not a 429, or a missing header, leaves the error untouched. + */ +export function attachRetryAfter( + err: unknown, + retryAfter: string | undefined, +): void { + if (!isRateLimitError(err) || retryAfter === undefined) return; + if (retryAfterOn(err) !== undefined) return; + err.data.retryAfter = retryAfter; +} + function describeRetry( retryAfter: string | undefined, retryAfterMs: number | undefined, @@ -144,9 +198,8 @@ function describeRetry( * says which request was throttled, and gives the retry timing when the server * sent one. * - * `retryAfter` is the header the caller tracked; a value the backend relayed on - * the error itself takes precedence, since it was read off the very response - * that produced this error. + * The header is read off the error's own data, where {@link attachRetryAfter} + * (or the backend relay) put it. * * Anything that is not a 429 is returned untouched, so this is safe to apply * blindly on the rejection path of every request. The result keeps the @@ -156,11 +209,10 @@ function describeRetry( export function annotateRateLimit( err: unknown, method: string, - retryAfter: string | undefined, now: number = Date.now(), ): unknown { if (!isRateLimitError(err)) return err; - const header = relayedRetryAfter(err) ?? retryAfter; + const header = retryAfterOn(err); const retryAfterMs = parseRetryAfter(header, now); const data: AnnotatedRateLimitData = { ...err.data, diff --git a/core/mcp/remote/node/remote-session.ts b/core/mcp/remote/node/remote-session.ts index c33765924..e81675d60 100644 --- a/core/mcp/remote/node/remote-session.ts +++ b/core/mcp/remote/node/remote-session.ts @@ -11,11 +11,7 @@ import { AuthChallengeError } from "../../../auth/challenge.js"; import type { RemoteAuthProviderHandle } from "./tokenAuthProvider.js"; import type { RemoteAuthState } from "../types.js"; import { progressTokenOf, waitForProgressToken } from "../progressToken.js"; -import { - rateLimitResponseOf, - retryAfterSince, - type RateLimitResponse, -} from "../../rateLimit.js"; +import { RateLimitLedger } from "../../rateLimit.js"; export interface SessionEvent { type: RemoteEvent["type"]; @@ -52,8 +48,8 @@ export class RemoteSession { private readonly requestWaits = new Map(); private authProviderHandle: RemoteAuthProviderHandle | null = null; private appliedProtocolVersion: string | undefined; - /** The most recent `429` an upstream fetch drew, for its `Retry-After` (#2442). */ - private lastRateLimitResponse: RateLimitResponse | undefined; + /** Each upstream `429`'s `Retry-After`, by the JSON-RPC id that drew it (#2442). */ + private readonly rateLimits = new RateLimitLedger(); /** A protocol version is a dated token (`2025-11-25`, `2026-07-28`, …). */ private static readonly PROTOCOL_VERSION_PATTERN = /^[A-Za-z0-9._-]{1,64}$/; @@ -317,8 +313,7 @@ export class RemoteSession { } onFetchRequest(entry: FetchRequestEntryBase): void { - this.lastRateLimitResponse = - rateLimitResponseOf(entry) ?? this.lastRateLimitResponse; + this.rateLimits.record(entry); this.pushEvent({ type: "fetch_request", data: { @@ -332,12 +327,12 @@ export class RemoteSession { } /** - * The `Retry-After` of the latest upstream `429` drawn by a fetch sent at or - * after `since` — read by the send route, so the header rides back on the - * send's own error rather than on a fetch event racing it (#2442). + * The `Retry-After` of the upstream `429` that answered request `id` — read + * by the send route, so the header rides back on the send's own error rather + * than on a fetch event racing it (#2442). */ - retryAfterSince(since: number): string | undefined { - return retryAfterSince(this.lastRateLimitResponse, since); + takeRetryAfter(id: string | number): string | undefined { + return this.rateLimits.take(id); } onFetchResponseBody(id: string, responseBody: string): void { diff --git a/core/mcp/remote/node/server.ts b/core/mcp/remote/node/server.ts index 154ad507d..576672445 100644 --- a/core/mcp/remote/node/server.ts +++ b/core/mcp/remote/node/server.ts @@ -1046,8 +1046,6 @@ export function createRemoteApp( // on this request and everything after it (#1935). session.applyProtocolVersion(protocolVersion); - // Bounds which upstream 429 can be this send's (#2442). - const sendStartedAt = Date.now(); session.beginSend(); const requestId = requestIdForSendWait(message); let responseWait: Promise | undefined; @@ -1118,9 +1116,15 @@ export function createRemoteApp( // browser rebuilds an `SdkHttpError` the era probe can classify (#2660). if (err instanceof SdkHttpError) { const text = err.data.text; - const retryAfter = isRateLimitError(err) - ? session.retryAfterSince(sendStartedAt) - : undefined; + // The 429's own header, by this send's JSON-RPC id (#2442). Read off + // the message rather than `requestId`, which is unset for a request + // the route deliberately does not wait on (`subscriptions/listen`). + const sentId = "id" in message ? message.id : undefined; + const retryAfter = + isRateLimitError(err) && + (typeof sentId === "string" || typeof sentId === "number") + ? session.takeRetryAfter(sentId) + : undefined; return c.json({ ok: false, kind: "transport_error", From 9285a965a0720b262d4c0f0bcdd3489b9b8329cf Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:09:08 -0400 Subject: [PATCH 23/44] fix(auth): write a carried credential's stamp before its value A failed stamp write after the value landed left an unstamped value the retry skipped as already held, letting the source be purged with no stamp left to vouch for the carried copy (#2734 review). Writing the stamp first makes either failure leave the field vacant and retryable. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 43 ++++++++++++------- core/auth/node/oauth-namespace-ledger.ts | 6 ++- 2 files changed, 33 insertions(+), 16 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index 13dc835e9..a58c9466b 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -815,22 +815,35 @@ describe("pending carry-over (#2672)", () => { expect(await store.get(from, SECRET)).toBeNull(); }); - it("keeps the record unpurged, and warns, when the carry fails", async () => { - await owed(); - const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); - vi.spyOn(store, "set").mockRejectedValueOnce( - new Error("keychain locked"), - ); - await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); - expect(await store.get(from, SECRET)).toBe("s3cret"); - expect(readLedger()[NS1].memory.owedTo).toBe(NS2); - expect(warn.mock.calls[0][0]).toContain("keychain locked"); + it.each([ + ["the stamp", 0], + ["the value", 1], + ])( + "keeps the record unpurged, and warns, when writing %s fails, and the retry carries both", + async (_label, failing) => { + await owed(); + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + const set = store.set.bind(store); + let calls = 0; + vi.spyOn(store, "set").mockImplementation(async (...args) => { + if (calls++ === failing) throw new Error("keychain locked"); + await set(...args); + }); + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(from, SECRET)).toBe("s3cret"); + expect(await store.get(to, SECRET)).toBeNull(); + expect(readLedger()[NS1].memory.owedTo).toBe(NS2); + expect(warn.mock.calls[0][0]).toContain("keychain locked"); - // The next locked save retries it. - await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); - expect(await store.get(to, SECRET)).toBe("s3cret"); - expect(await store.get(from, SECRET)).toBeNull(); - }); + // The next locked save retries it, value and stamp together. + await purgeSupersededNamespaces(stateFile, store, NS2, owners("c")); + expect(await store.get(to, SECRET)).toBe("s3cret"); + expect(await store.get(to, STAMP)).toBe( + clientCredentialStamp("c", "s3cret"), + ); + expect(await store.get(from, SECRET)).toBeNull(); + }, + ); it("ends the pass at a failed carry, so an older record cannot take the newer one's field", async () => { await owed("c", "older"); diff --git a/core/auth/node/oauth-namespace-ledger.ts b/core/auth/node/oauth-namespace-ledger.ts index 6c634a348..cfa6c3383 100644 --- a/core/auth/node/oauth-namespace-ledger.ts +++ b/core/auth/node/oauth-namespace-ledger.ts @@ -518,8 +518,12 @@ async function carryOwed( if (mergeOwners(target.owners, owner) || isNew) { await writeLedger(stateFilePath, ledger); } - await secretStore.set(to, field, value); + // Stamp first: a stamp left alone by a failed value write vouches + // for nothing, and the retry finds the field vacant and redoes + // both. The other order could leave an unstamped value the retry + // skips as already held, and no later carry would trust. await secretStore.set(to, stampField, stamp); + await secretStore.set(to, field, value); } catch (error) { carried = false; warnLedgerFailure( From f7e0377ab8077c5fe90cfba45d1164c7b0cf6bf4 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:15:48 -0400 Subject: [PATCH 24/44] fix: open the URL that was checked, not the raw input Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/server/web-server-config.ts | 3 +- .../web/src/test/core/node/openUrl.test.ts | 17 +++++++++++ .../server/server-auto-open.test.ts | 3 +- core/node/openUrl.ts | 28 +++++++++++++------ 4 files changed, 41 insertions(+), 10 deletions(-) diff --git a/clients/web/server/web-server-config.ts b/clients/web/server/web-server-config.ts index 81fbc6c46..008c66c60 100644 --- a/clients/web/server/web-server-config.ts +++ b/clients/web/server/web-server-config.ts @@ -265,7 +265,8 @@ export function openBrowser(url: string): void { // Through core's `openUrl`, not a bare `open(url)`: it bounds a stalled // opener with a timeout, refuses anything but http(s), and absorbs an // `'error'` on the child after launch, which unlistened would crash the - // server (#2533). Every one of those failures reaches the `.catch` below. + // server (#2533). That late `'error'` is swallowed there; the refusal, a + // spawn failure and the timeout all reach the `.catch` below. openUrl(url).catch((err: unknown) => { // The message only, on one line, matching the CLI's wording // (`core/cli/cli-oauth-navigation.ts`). Logging the error object printed diff --git a/clients/web/src/test/core/node/openUrl.test.ts b/clients/web/src/test/core/node/openUrl.test.ts index 5b53ad992..807942932 100644 --- a/clients/web/src/test/core/node/openUrl.test.ts +++ b/clients/web/src/test/core/node/openUrl.test.ts @@ -46,6 +46,23 @@ describe("openUrl", () => { ); }); + it.each([ + [" https:/a.example", "https://a.example/"], + ["\thttps://a.example", "https://a.example/"], + ["HTTPS:\\\\a.example\\x", "https://a.example/x"], + [ + "https:/../../mnt/c/Windows/System32/calc.exe", + "https://../mnt/c/Windows/System32/calc.exe", + ], + ])( + "opens the normalized form of %j, not the raw input", + async (raw, opened) => { + const { openUrl } = await import("@inspector/core/node/openUrl.js"); + await openUrl(raw); + expect(openMock).toHaveBeenCalledWith(opened); + }, + ); + it.each([ ["file:", "file:///C:/Windows/System32/calc.exe"], ["javascript:", "javascript:alert(1)"], diff --git a/clients/web/src/test/integration/server/server-auto-open.test.ts b/clients/web/src/test/integration/server/server-auto-open.test.ts index cf33383be..704b09dc6 100644 --- a/clients/web/src/test/integration/server/server-auto-open.test.ts +++ b/clients/web/src/test/integration/server/server-auto-open.test.ts @@ -113,7 +113,8 @@ describe("openBrowser", () => { openMock.mockImplementation(async () => fakeChild()); openBrowser("http://127.0.0.1:6274"); await settle(); - expect(openMock).toHaveBeenCalledWith("http://127.0.0.1:6274"); + // `openUrl` opens the URL's serialized form, which adds the root path. + expect(openMock).toHaveBeenCalledWith("http://127.0.0.1:6274/"); expect(warn).not.toHaveBeenCalled(); }); }); diff --git a/core/node/openUrl.ts b/core/node/openUrl.ts index faa362f6a..ae9301e58 100644 --- a/core/node/openUrl.ts +++ b/core/node/openUrl.ts @@ -37,17 +37,30 @@ export const OPEN_URL_TIMEOUT_MS = 5_000; */ const OPENABLE_PROTOCOLS: ReadonlySet = new Set(["http:", "https:"]); -/** Throw unless `href` parses as an `http:` or `https:` URL. */ -function assertOpenable(href: string): void { - let protocol: string; +/** + * Return the serialized form of `href`, or throw unless it parses as an + * `http:` or `https:` URL. + * + * The caller opens the **returned** string, never the input. The WHATWG parser + * is lenient — it accepts `" https:/a"`, `"HTTPS:\\a\\b"` or + * `"https:/../x"` as https — and the platform opener is not: on WSL, `open` + * treats a target as a URL only when it literally starts with `scheme://`, and + * otherwise converts it as a file path. Checking one string and opening + * another would let such an input pass as https and still reach the opener as + * a path. The serialization of an http(s) URL always starts with `http://` or + * `https://`, so what is checked is exactly what is opened. + */ +function toOpenableHref(href: string): string { + let parsed: URL; try { - protocol = new URL(href).protocol; + parsed = new URL(href); } catch { throw new Error("refusing to open an invalid URL"); } - if (!OPENABLE_PROTOCOLS.has(protocol)) { - throw new Error(`refusing to open a non-http(s) URL (${protocol})`); + if (!OPENABLE_PROTOCOLS.has(parsed.protocol)) { + throw new Error(`refusing to open a non-http(s) URL (${parsed.protocol})`); } + return parsed.href; } /** @@ -76,8 +89,7 @@ export async function openUrl( url: string | URL, timeoutMs: number = OPEN_URL_TIMEOUT_MS, ): Promise { - const href = typeof url === "string" ? url : url.href; - assertOpenable(href); + const href = toOpenableHref(typeof url === "string" ? url : url.href); let timer: ReturnType | undefined; const timeout = new Promise((_, reject) => { timer = setTimeout( From 0934e38eba8c91307097dcc08e9d289c77fccbdd Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:27:15 -0400 Subject: [PATCH 25/44] fix(oauth): judge step-up refresh coverage against persisted scope A token response may omit `scope` when it equals the request. The forceReauthorization decision compared the challenge only with the token's own scope, so a covered challenge on an expired token with no token scope forced interactive authorization instead of using the refresh token. Use resolveEffectiveGrantedScope, as checkAuthChallengeSatisfied does. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../src/test/core/mcp/oauthManager.test.ts | 64 +++++++++++-------- core/mcp/oauthManager.ts | 9 ++- 2 files changed, 44 insertions(+), 29 deletions(-) diff --git a/clients/web/src/test/core/mcp/oauthManager.test.ts b/clients/web/src/test/core/mcp/oauthManager.test.ts index 867d1d56d..43c4a009e 100644 --- a/clients/web/src/test/core/mcp/oauthManager.test.ts +++ b/clients/web/src/test/core/mcp/oauthManager.test.ts @@ -1427,38 +1427,46 @@ describe("OAuthManager", () => { ); }); - it("refreshes instead of replaying, and accepts the refreshed opaque token", async () => { - const params = createMockParams(); - let stored: OAuthTokens = { - access_token: expiredJwt(), - refresh_token: "r1", - token_type: "Bearer", - scope: COVERED, - }; - storageOf(params).getTokens.mockImplementation(async () => stored); - storageOf(params).getScope.mockResolvedValue(COVERED); - mockedMcpAuth.mockImplementation(async () => { - // A refresh grant that returns an opaque token: no readable `exp`, - // so only a scope-only post-grant check can call it satisfied. - stored = { - access_token: "opaque-refreshed", - refresh_token: "r2", + // A token response may omit `scope` when it equals the request (RFC + // 6749 §5.1); coverage then comes from the persisted scope. + it.each([ + ["carries its scope", COVERED], + ["omits scope", undefined], + ])( + "refreshes instead of replaying when the token %s, and accepts the refreshed opaque token", + async (_label, tokenScope) => { + const params = createMockParams(); + let stored: OAuthTokens = { + access_token: expiredJwt(), + refresh_token: "r1", token_type: "Bearer", - scope: COVERED, + scope: tokenScope, }; - return "AUTHORIZED"; - }); - const manager = new OAuthManager(params); + storageOf(params).getTokens.mockImplementation(async () => stored); + storageOf(params).getScope.mockResolvedValue(COVERED); + mockedMcpAuth.mockImplementation(async () => { + // A refresh grant that returns an opaque token: no readable `exp`, + // so only a scope-only post-grant check can call it satisfied. + stored = { + access_token: "opaque-refreshed", + refresh_token: "r2", + token_type: "Bearer", + scope: tokenScope, + }; + return "AUTHORIZED"; + }); + const manager = new OAuthManager(params); - const outcome = await manager.handleAuthChallenge(challenge); + const outcome = await manager.handleAuthChallenge(challenge); - expect(outcome).toEqual({ kind: "satisfied" }); - expect(mockedMcpAuth).toHaveBeenCalledOnce(); - // Covered scope: an ordinary refresh, not a forced re-authorization. - expect(mockedMcpAuth.mock.calls[0]?.[1]).toEqual( - expect.objectContaining({ forceReauthorization: false }), - ); - }); + expect(outcome).toEqual({ kind: "satisfied" }); + expect(mockedMcpAuth).toHaveBeenCalledOnce(); + // Covered scope: an ordinary refresh, not a forced re-authorization. + expect(mockedMcpAuth.mock.calls[0]?.[1]).toEqual( + expect.objectContaining({ forceReauthorization: false }), + ); + }, + ); it("re-mints an EMA token without a step-up prompt when nothing is broadened", async () => { const params = createMockParams({ diff --git a/core/mcp/oauthManager.ts b/core/mcp/oauthManager.ts index bff565aed..4982d543d 100644 --- a/core/mcp/oauthManager.ts +++ b/core/mcp/oauthManager.ts @@ -921,9 +921,16 @@ export class OAuthManager { fetchFn: this.params.effectiveAuthFetch, resourceMetadataUrl, ...(enriched.reason === "insufficient_scope" && { + // Judge coverage the way checkAuthChallengeSatisfied does: a token + // response that omitted `scope` granted what was stored, so a + // covered challenge on an expired token refreshes rather than + // forcing interactive authorization (#2724). forceReauthorization: isStrictScopeSuperset( scopeForAuth, - tokens?.scope, + resolveEffectiveGrantedScope( + await this.oauthConfig.storage?.getScope(serverUrl), + tokens?.scope, + ), ), }), }); From 3fbf2a9f20d0c5ae5b3a37b318f098e67ffa4a71 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:39:16 -0400 Subject: [PATCH 26/44] fix(auth): key in-flight PKCE verifiers by flow, not server URL Overlapping OAuth flows to the same server (two tabs, or several users of one deployed Inspector) shared one codeVerifier slot in the backend's oauth.json, so the later flow overwrote the earlier one's verifier and the earlier exchange failed PKCE with invalid_grant. The web client now keeps each flow's verifier in the initiating tab's sessionStorage, keyed by the OAuth `state` authId, and the callback exchange reads the verifier for the `state` it returned with. Keying inside servers[url] would not help, since the remote backend merges whole server entries. A callback whose `state` has no stored verifier fails with a clear error rather than borrowing another flow's. CLI/TUI set no flow store and keep the existing storage path; the runner now forwards the callback `state` too. Closes #2726 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/tui/__tests__/App.test.tsx | 15 +- clients/web/src/hooks/useOAuthRecovery.ts | 2 + clients/web/src/lib/environmentFactory.ts | 6 + .../test/core/auth/pkce-flow-store.test.ts | 99 +++++++ .../web/src/test/core/auth/providers.test.ts | 117 +++++++++ .../auth/runner-interactive-oauth.test.ts | 6 +- ...rClient-oauth-concurrent-flows-e2e.test.ts | 241 ++++++++++++++++++ core/auth/index.ts | 7 + core/auth/node/runner-interactive-oauth.ts | 12 +- core/auth/pkce-flow-store.ts | 98 +++++++ core/auth/providers.ts | 67 ++++- core/mcp/inspectorClient.ts | 15 +- core/mcp/oauthManager.ts | 13 + core/mcp/types.ts | 8 + specification/v2_storage.md | 2 + 15 files changed, 698 insertions(+), 10 deletions(-) create mode 100644 clients/web/src/test/core/auth/pkce-flow-store.test.ts create mode 100644 clients/web/src/test/integration/mcp/inspectorClient-oauth-concurrent-flows-e2e.test.ts create mode 100644 core/auth/pkce-flow-store.ts diff --git a/clients/tui/__tests__/App.test.tsx b/clients/tui/__tests__/App.test.tsx index 07f2ad9db..a7271ab24 100644 --- a/clients/tui/__tests__/App.test.tsx +++ b/clients/tui/__tests__/App.test.tsx @@ -108,7 +108,11 @@ const h = vi.hoisted(() => { // Captured options from the most recent callbackServer.start(), so a test can // drive the onCallback / onError handlers the OAuth flows register. interface CallbackOpts { - onCallback: (p: { code: string; iss?: string }) => Promise | void; + onCallback: (p: { + code: string; + iss?: string; + state?: string; + }) => Promise | void; onError: (p: { error?: string; error_description?: string }) => void; } const cb: { opts: CallbackOpts | null } = { opts: null }; @@ -1551,11 +1555,18 @@ describe("App (OAuth flows)", () => { const r = await mount(oneHttp()); await press(r, ["c"]); await waitUntil(() => h.cb.opts !== null); - await h.cb.opts!.onCallback({ code: "abc", iss: "https://as.example" }); + // #2726: the redirect's `state` selects the flow's PKCE verifier. + const state = "d".repeat(64); + await h.cb.opts!.onCallback({ + code: "abc", + iss: "https://as.example", + state, + }); await tick(); expect(h.clientSpies.completeOAuthFlow).toHaveBeenCalledWith( "abc", "https://as.example", + state, ); }); diff --git a/clients/web/src/hooks/useOAuthRecovery.ts b/clients/web/src/hooks/useOAuthRecovery.ts index 313143a6a..8a0bfd8b6 100644 --- a/clients/web/src/hooks/useOAuthRecovery.ts +++ b/clients/web/src/hooks/useOAuthRecovery.ts @@ -1381,6 +1381,8 @@ export function useOAuthRecovery({ await client.resumeAfterOAuth(params.code, { remoteSessionId: resumeSnapshot?.remoteSessionId, iss: params.iss, + // #2726: selects this flow's PKCE verifier from the tab's store. + state: stateParam ?? undefined, }); } catch (err) { connectStartRef.current = undefined; diff --git a/clients/web/src/lib/environmentFactory.ts b/clients/web/src/lib/environmentFactory.ts index 3b08460bb..c82959d67 100644 --- a/clients/web/src/lib/environmentFactory.ts +++ b/clients/web/src/lib/environmentFactory.ts @@ -6,6 +6,7 @@ import { } from "@inspector/core/mcp/remote/index.js"; import { BrowserNavigation } from "@inspector/core/auth/browser/index.js"; import type { RedirectUrlProvider } from "@inspector/core/auth/index.js"; +import { createWebStoragePkceFlowStore } from "@inspector/core/auth/pkce-flow-store.js"; import { getWebRemoteOAuthStorage } from "./remoteOAuthStorage.js"; export interface WebEnvironmentResult { @@ -20,6 +21,10 @@ export interface WebEnvironmentResult { * dev-backend wires this in `/api/*`). * - OAuth storage uses `RemoteOAuthStorage` (shared `oauth.json` via * `/api/storage/oauth`); navigation uses `BrowserNavigation`. + * - In-flight PKCE verifiers stay in this tab's `sessionStorage`, keyed by + * the flow's OAuth `state`, so overlapping flows for one server — two tabs, + * or two people on one deployed Inspector — cannot overwrite each other's + * in the shared store (#2726). * * Returns both the assembled environment and the logger so callers can share * the same pino instance for any direct logging they need to do, instead of @@ -69,6 +74,7 @@ export function createWebEnvironment( storage: getWebRemoteOAuthStorage(authToken), navigation: new BrowserNavigation(undefined, onBeforeOAuthRedirect), redirectUrlProvider, + flowStore: createWebStoragePkceFlowStore(), }, }; diff --git a/clients/web/src/test/core/auth/pkce-flow-store.test.ts b/clients/web/src/test/core/auth/pkce-flow-store.test.ts new file mode 100644 index 000000000..708a633dd --- /dev/null +++ b/clients/web/src/test/core/auth/pkce-flow-store.test.ts @@ -0,0 +1,99 @@ +import { describe, it, expect, afterEach, vi } from "vitest"; +import { + createWebStoragePkceFlowStore, + PKCE_FLOW_STORAGE_KEY_PREFIX, + type PkceFlowStorageArea, +} from "@inspector/core/auth/pkce-flow-store.js"; + +const SERVER = "https://mcp.example.com"; +const FLOW = "f".repeat(64); + +function makeArea(): PkceFlowStorageArea & { items: Map } { + const items = new Map(); + return { + items, + getItem: (key) => items.get(key) ?? null, + setItem: (key, value) => void items.set(key, value), + removeItem: (key) => void items.delete(key), + }; +} + +describe("createWebStoragePkceFlowStore (#2726)", () => { + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it("round-trips a verifier under a per-flow key", async () => { + const area = makeArea(); + const store = createWebStoragePkceFlowStore(() => area); + + await store.saveCodeVerifier(FLOW, SERVER, "cv"); + + expect([...area.items.keys()]).toEqual([ + `${PKCE_FLOW_STORAGE_KEY_PREFIX}${FLOW}`, + ]); + expect(await store.getCodeVerifier(FLOW, SERVER)).toBe("cv"); + }); + + it("keeps flows for the same server apart", async () => { + const otherTab = createWebStoragePkceFlowStore(() => makeArea()); + const area = makeArea(); + const shared = createWebStoragePkceFlowStore(() => area); + const other = "e".repeat(64); + + await shared.saveCodeVerifier(FLOW, SERVER, "cv-1"); + await shared.saveCodeVerifier(other, SERVER, "cv-2"); + + expect(await shared.getCodeVerifier(FLOW, SERVER)).toBe("cv-1"); + expect(await shared.getCodeVerifier(other, SERVER)).toBe("cv-2"); + // A separate storage area (another tab) sees neither. + expect(await otherTab.getCodeVerifier(FLOW, SERVER)).toBeUndefined(); + }); + + it("does not answer for a different server than the flow was started for", async () => { + const store = createWebStoragePkceFlowStore(() => makeArea()); + await store.saveCodeVerifier(FLOW, SERVER, "cv"); + + expect( + await store.getCodeVerifier(FLOW, "https://other.example.com"), + ).toBeUndefined(); + }); + + it("clears a flow's verifier", async () => { + const area = makeArea(); + const store = createWebStoragePkceFlowStore(() => area); + await store.saveCodeVerifier(FLOW, SERVER, "cv"); + + await store.clearCodeVerifier(FLOW); + + expect(area.items.size).toBe(0); + expect(await store.getCodeVerifier(FLOW, SERVER)).toBeUndefined(); + }); + + it.each([ + ["not JSON", "{"], + ["not an object", "42"], + ["null", "null"], + ["missing the verifier", JSON.stringify({ serverUrl: SERVER })], + [ + "a non-string verifier", + JSON.stringify({ serverUrl: SERVER, codeVerifier: 1 }), + ], + ])("treats a corrupt entry (%s) as absent", async (_label, raw) => { + const area = makeArea(); + area.items.set(`${PKCE_FLOW_STORAGE_KEY_PREFIX}${FLOW}`, raw); + const store = createWebStoragePkceFlowStore(() => area); + + expect(await store.getCodeVerifier(FLOW, SERVER)).toBeUndefined(); + }); + + it("defaults to sessionStorage", async () => { + const area = makeArea(); + vi.stubGlobal("sessionStorage", area); + const store = createWebStoragePkceFlowStore(); + + await store.saveCodeVerifier(FLOW, SERVER, "cv"); + + expect(area.items.has(`${PKCE_FLOW_STORAGE_KEY_PREFIX}${FLOW}`)).toBe(true); + }); +}); diff --git a/clients/web/src/test/core/auth/providers.test.ts b/clients/web/src/test/core/auth/providers.test.ts index 1992e7fcf..359e5c7cc 100644 --- a/clients/web/src/test/core/auth/providers.test.ts +++ b/clients/web/src/test/core/auth/providers.test.ts @@ -11,6 +11,10 @@ import { OAuthStorageBase } from "@inspector/core/auth/oauth-storage.js"; import { OAuthMemoryStore } from "@inspector/core/auth/store.js"; import type { OAuthPersistBackend } from "@inspector/core/auth/oauth-persist.js"; import { ensureCimdClientRegistration } from "@inspector/core/auth/cimd.js"; +import { + createWebStoragePkceFlowStore, + type PkceFlowStore, +} from "@inspector/core/auth/pkce-flow-store.js"; import { BrowserNavigation, BrowserOAuthClientProvider, @@ -1108,5 +1112,118 @@ describe("OAuthNavigation", () => { expect(storage.clearDiscoveryState).toHaveBeenCalledWith(SERVER); }); }); + + describe("per-flow PKCE verifier (#2726)", () => { + function makeFlowStore(): PkceFlowStore { + const store = new Map(); + return createWebStoragePkceFlowStore(() => ({ + getItem: (key) => store.get(key) ?? null, + setItem: (key, value) => void store.set(key, value), + removeItem: (key) => void store.delete(key), + })); + } + + it("keys the verifier by the state it generated, not the shared slot", async () => { + const storage = makeStorage(); + const flowStore = makeFlowStore(); + const provider = makeProvider(storage, vi.fn(), { flowStore }); + + const state = await provider.state(); + await provider.saveCodeVerifier("cv-a"); + + expect(storage.saveCodeVerifier).not.toHaveBeenCalled(); + expect(await flowStore.getCodeVerifier(state, SERVER)).toBe("cv-a"); + expect(await provider.codeVerifier()).toBe("cv-a"); + }); + + it("gives two overlapping flows for one server their own verifiers", async () => { + const storage = makeStorage(); + const flowStore = makeFlowStore(); + const providerA = makeProvider(storage, vi.fn(), { flowStore }); + const providerB = makeProvider(storage, vi.fn(), { flowStore }); + + const stateA = await providerA.state(); + await providerA.saveCodeVerifier("cv-a"); + const stateB = await providerB.state(); + await providerB.saveCodeVerifier("cv-b"); + + // The callback legs run on fresh providers, as after a page reload. + const callbackB = makeProvider(storage, vi.fn(), { flowStore }); + callbackB.resumeAuthorizationFlow(stateB); + expect(await callbackB.codeVerifier()).toBe("cv-b"); + await callbackB.clearAuthorizationFlow(); + + const callbackA = makeProvider(storage, vi.fn(), { flowStore }); + callbackA.resumeAuthorizationFlow(stateA); + expect(await callbackA.codeVerifier()).toBe("cv-a"); + }); + + it("fails clearly for a state with no stored verifier instead of using another flow's", async () => { + const storage = makeStorage(); + vi.mocked(storage.getCodeVerifier).mockResolvedValue("someone-elses"); + const flowStore = makeFlowStore(); + const other = makeProvider(storage, vi.fn(), { flowStore }); + await other.state(); + await other.saveCodeVerifier("cv-other"); + + const provider = makeProvider(storage, vi.fn(), { flowStore }); + provider.resumeAuthorizationFlow("a".repeat(64)); + + await expect(provider.codeVerifier()).rejects.toThrow( + /No code verifier saved for this authorization flow/, + ); + expect(storage.getCodeVerifier).not.toHaveBeenCalled(); + }); + + it("binds no flow for a state that is not one of ours", async () => { + const storage = makeStorage(); + const provider = makeProvider(storage, vi.fn(), { + flowStore: makeFlowStore(), + }); + + provider.resumeAuthorizationFlow("not-our-state"); + + await expect(provider.codeVerifier()).rejects.toThrow( + /No code verifier saved for this authorization flow/, + ); + }); + + it("invalidateCredentials('verifier') clears only the bound flow", async () => { + const storage = makeStorage(); + const flowStore = makeFlowStore(); + const provider = makeProvider(storage, vi.fn(), { flowStore }); + const state = await provider.state(); + await provider.saveCodeVerifier("cv"); + + await provider.invalidateCredentials("verifier"); + + expect(await flowStore.getCodeVerifier(state, SERVER)).toBeUndefined(); + expect(storage.clearCodeVerifier).not.toHaveBeenCalled(); + }); + + it("clearAuthorizationFlow() is a no-op without a flow store or a bound flow", async () => { + const storage = makeStorage(); + await makeProvider(storage).clearAuthorizationFlow(); + + const flowStore = makeFlowStore(); + const clear = vi.spyOn(flowStore, "clearCodeVerifier"); + await makeProvider(storage, vi.fn(), { + flowStore, + }).clearAuthorizationFlow(); + + expect(clear).not.toHaveBeenCalled(); + expect(storage.clearCodeVerifier).not.toHaveBeenCalled(); + }); + + it("keeps the shared slot when no flow store is configured", async () => { + const storage = makeStorage(); + const provider = makeProvider(storage); + + await provider.state(); + await provider.saveCodeVerifier("cv"); + + expect(storage.saveCodeVerifier).toHaveBeenCalledWith(SERVER, "cv"); + }); + }); }); }); diff --git a/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts b/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts index e885e06d2..9dc932014 100644 --- a/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts +++ b/clients/web/src/test/core/auth/runner-interactive-oauth.test.ts @@ -58,6 +58,8 @@ function createMockCallbackServer(handlers: { } const CALLBACK_ISS = "https://as.example"; +/** The OAuth `state` the redirect echoes; it selects the flow's verifier (#2726). */ +const CALLBACK_STATE = "c".repeat(64); /** * Drive the loopback callback the way a browser redirect would, including the @@ -71,7 +73,7 @@ async function simulateCallback( if (!handlers.onCallback) { throw new Error("onCallback not registered"); } - await handlers.onCallback({ code, iss }); + await handlers.onCallback({ code, iss, state: CALLBACK_STATE }); } describe("runRunnerInteractiveOAuth", () => { @@ -155,6 +157,7 @@ describe("runRunnerInteractiveOAuth", () => { expect(client.completeOAuthFlow).toHaveBeenCalledWith( "auth-code-123", CALLBACK_ISS, + CALLBACK_STATE, ); expect(redirectUrlProvider.redirectUrl).toBe( "http://127.0.0.1:6276/oauth/callback", @@ -192,6 +195,7 @@ describe("runRunnerInteractiveOAuth", () => { expect(client.completeOAuthFlow).toHaveBeenCalledWith( "step-up-code", CALLBACK_ISS, + CALLBACK_STATE, ); }); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-oauth-concurrent-flows-e2e.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-oauth-concurrent-flows-e2e.test.ts new file mode 100644 index 000000000..9d32e702d --- /dev/null +++ b/clients/web/src/test/integration/mcp/inspectorClient-oauth-concurrent-flows-e2e.test.ts @@ -0,0 +1,241 @@ +/** + * Overlapping OAuth flows for the same server through the shared remote store + * (#2726). + * + * Each "tab" is what the web client builds per page: its own + * `RemoteOAuthStorage` view over one backend `oauth.json`, and its own + * sessionStorage-like area for in-flight PKCE verifiers. Tab A starts a flow, + * tab B starts and finishes one for the same server, then tab A's callback page + * (a fresh page, so a fresh client and remote view, but the same tab storage) + * finishes. The test AS verifies PKCE S256, so A succeeds only if its exchange + * sends A's own verifier. + */ + +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { serve } from "@hono/node-server"; +import type { ServerType } from "@hono/node-server"; +import { InspectorClient } from "@inspector/core/mcp/inspectorClient.js"; +import { createRemoteTransport } from "@inspector/core/mcp/remote/createRemoteTransport.js"; +import { createRemoteFetch } from "@inspector/core/mcp/remote/createRemoteFetch.js"; +import { RemoteOAuthStorage } from "@inspector/core/auth/remote/storage-remote.js"; +import { createRemoteApp } from "@inspector/core/mcp/remote/node/server.js"; +import { + createWebStoragePkceFlowStore, + type PkceFlowStore, +} from "@inspector/core/auth/pkce-flow-store.js"; +import { ConsoleNavigation } from "@inspector/core/auth/providers.js"; +import { + TestServerHttp, + waitForOAuthWellKnown, + getDefaultServerConfig, + createOAuthTestServerConfig, + clearOAuthTestData, +} from "@modelcontextprotocol/inspector-test-server"; +import { + completeOAuthAuthorization, + createStaticRedirectUrlProvider, +} from "../helpers/oauth-client-fixtures.js"; +import type { MCPServerConfig } from "@inspector/core/mcp/types.js"; + +const REDIRECT_URL = "http://localhost:3001/oauth/callback"; +const CLIENT_ID = "concurrent-flows-client"; +const CLIENT_SECRET = "concurrent-flows-secret"; + +/** One tab's sessionStorage stand-in. */ +function createTabFlowStore(): PkceFlowStore { + const items = new Map(); + return createWebStoragePkceFlowStore(() => ({ + getItem: (key) => items.get(key) ?? null, + setItem: (key, value) => void items.set(key, value), + removeItem: (key) => void items.delete(key), + })); +} + +describe("InspectorClient overlapping OAuth flows to one server (#2726)", () => { + let mcpServer: TestServerHttp | undefined; + let remoteServer: ServerType | undefined; + let tempDir: string | undefined; + const clients: InspectorClient[] = []; + + beforeEach(() => { + clearOAuthTestData(); + tempDir = mkdtempSync(join(tmpdir(), "inspector-concurrent-oauth-")); + vi.spyOn(console, "log").mockImplementation(() => {}); + }); + + afterEach(async () => { + for (const client of clients.splice(0)) { + await client.disconnect(); + } + await mcpServer?.stop(); + mcpServer = undefined; + if (remoteServer) { + const server = remoteServer; + await new Promise((resolve, reject) => { + server.close((err) => (err ? reject(err) : resolve())); + }); + remoteServer = undefined; + } + if (tempDir) { + rmSync(tempDir, { recursive: true, force: true }); + tempDir = undefined; + } + vi.restoreAllMocks(); + }); + + async function startBackend(): Promise<{ + baseUrl: string; + authToken: string; + }> { + const { app, authToken } = createRemoteApp({ + storageDir: tempDir, + initialConfig: { defaultEnvironment: {} }, + }); + return new Promise((resolve, reject) => { + const server = serve( + { fetch: app.fetch, port: 0, hostname: "127.0.0.1" }, + (info) => { + resolve({ baseUrl: `http://127.0.0.1:${info.port}`, authToken }); + }, + ); + remoteServer = server; + server.on("error", reject); + }); + } + + async function startMcpServer(): Promise { + mcpServer = new TestServerHttp({ + ...getDefaultServerConfig(), + serverType: "streamable-http", + ...createOAuthTestServerConfig({ + requireAuth: true, + staticClients: [ + { + clientId: CLIENT_ID, + clientSecret: CLIENT_SECRET, + redirectUris: [REDIRECT_URL], + }, + ], + }), + }); + const port = await mcpServer.start(); + const serverUrl = `http://localhost:${port}`; + await waitForOAuthWellKnown(serverUrl); + return serverUrl; + } + + /** A client as one page load of one tab builds it. */ + function createPageClient( + backend: { baseUrl: string; authToken: string }, + serverUrl: string, + flowStore: PkceFlowStore | undefined, + ): InspectorClient { + const client = new InspectorClient( + { type: "streamable-http", url: `${serverUrl}/mcp` } as MCPServerConfig, + { + environment: { + transport: createRemoteTransport(backend), + fetch: createRemoteFetch(backend), + oauth: { + storage: new RemoteOAuthStorage(backend), + navigation: new ConsoleNavigation(), + redirectUrlProvider: createStaticRedirectUrlProvider(REDIRECT_URL), + flowStore, + }, + }, + oauth: { clientId: CLIENT_ID, clientSecret: CLIENT_SECRET }, + }, + ); + clients.push(client); + return client; + } + + /** Tab A starts, tab B starts and finishes, then tab A finishes. */ + async function runOverlappingFlows(withFlowStore: boolean): Promise<{ + finishA: () => Promise; + clientB: InspectorClient; + }> { + const backend = await startBackend(); + const serverUrl = await startMcpServer(); + const tabA = withFlowStore ? createTabFlowStore() : undefined; + const tabB = withFlowStore ? createTabFlowStore() : undefined; + + const urlA = await createPageClient( + backend, + serverUrl, + tabA, + ).authenticate(); + const urlB = await createPageClient( + backend, + serverUrl, + tabB, + ).authenticate(); + if (!urlA || !urlB) throw new Error("Expected two authorization URLs"); + + const approvedA = await completeOAuthAuthorization(urlA); + const approvedB = await completeOAuthAuthorization(urlB); + + const clientB = createPageClient(backend, serverUrl, tabB); + await clientB.completeOAuthFlow( + approvedB.code, + approvedB.iss, + urlB.searchParams.get("state") ?? undefined, + ); + await clientB.connect(); + + const finishA = async () => { + const clientA = createPageClient(backend, serverUrl, tabA); + await clientA.completeOAuthFlow( + approvedA.code, + approvedA.iss, + urlA.searchParams.get("state") ?? undefined, + ); + await clientA.connect(); + return clientA; + }; + return { finishA, clientB }; + } + + it("completes both flows, each with its own verifier", async () => { + const { finishA, clientB } = await runOverlappingFlows(true); + const clientA = await finishA(); + + expect(clientB.getStatus()).toBe("connected"); + expect(clientA.getStatus()).toBe("connected"); + }); + + it("without a flow store, the earlier flow fails PKCE (the #2726 defect)", async () => { + const { finishA, clientB } = await runOverlappingFlows(false); + + expect(clientB.getStatus()).toBe("connected"); + // A's exchange sends B's verifier and fails PKCE. The test AS consumes a + // code on first read, so the SDK's retry after that rejection reports the + // code as expired instead; either message is the same defect. + await expect(finishA()).rejects.toThrow( + /Invalid code_verifier|Invalid or expired authorization code/, + ); + }); + + it("a callback whose state has no stored verifier fails clearly", async () => { + const backend = await startBackend(); + const serverUrl = await startMcpServer(); + const tab = createTabFlowStore(); + + const url = await createPageClient(backend, serverUrl, tab).authenticate(); + if (!url) throw new Error("Expected an authorization URL"); + const approved = await completeOAuthAuthorization(url); + + // The callback lands in a different tab, which never saw this flow. + const client = createPageClient(backend, serverUrl, createTabFlowStore()); + await expect( + client.completeOAuthFlow( + approved.code, + approved.iss, + url.searchParams.get("state") ?? undefined, + ), + ).rejects.toThrow(/No code verifier saved for this authorization flow/); + }); +}); diff --git a/core/auth/index.ts b/core/auth/index.ts index 918eaac3d..3f2117de1 100644 --- a/core/auth/index.ts +++ b/core/auth/index.ts @@ -85,6 +85,13 @@ export { BaseOAuthClientProvider, } from "./providers.js"; +// Per-flow PKCE verifier store (#2726) +export type { PkceFlowStore, PkceFlowStorageArea } from "./pkce-flow-store.js"; +export { + createWebStoragePkceFlowStore, + PKCE_FLOW_STORAGE_KEY_PREFIX, +} from "./pkce-flow-store.js"; + // Utilities export { parseHttpUrl, diff --git a/core/auth/node/runner-interactive-oauth.ts b/core/auth/node/runner-interactive-oauth.ts index 6e084d85f..78f35c2fd 100644 --- a/core/auth/node/runner-interactive-oauth.ts +++ b/core/auth/node/runner-interactive-oauth.ts @@ -11,7 +11,11 @@ import type { RunnerOAuthCallbackConfig } from "./runner-oauth-callback.js"; export interface RunnerInteractiveOAuthClient { authenticate(): Promise; beginInteractiveAuthorization(authorizationUrl: URL): Promise; - completeOAuthFlow(authorizationCode: string, iss?: string): Promise; + completeOAuthFlow( + authorizationCode: string, + iss?: string, + state?: string, + ): Promise; checkAuthChallengeSatisfied(challenge: AuthChallenge): Promise; } @@ -142,7 +146,11 @@ export async function runRunnerInteractiveOAuth( path: options.callbackListen.pathname, onCallback: async (params) => { try { - await options.client.completeOAuthFlow(params.code, params.iss); + await options.client.completeOAuthFlow( + params.code, + params.iss, + params.state, + ); flowResolve(); } catch (err) { flowReject(toRunnerOAuthError(err)); diff --git a/core/auth/pkce-flow-store.ts b/core/auth/pkce-flow-store.ts new file mode 100644 index 000000000..29fc9abe5 --- /dev/null +++ b/core/auth/pkce-flow-store.ts @@ -0,0 +1,98 @@ +/** + * Per-flow PKCE verifier store (#2726). + * + * The web client's OAuth state lives in the backend's shared `oauth.json`, + * merged at whole `servers[url]` granularity, and keyed by server URL only. Two + * authorization flows for the same server that overlap in time — two tabs, or + * two people on one deployed Inspector — therefore share one `codeVerifier` + * slot: the later flow overwrites the earlier one's verifier, and the earlier + * flow's token exchange fails PKCE with `invalid_grant`. + * + * A verifier is in-flight state that belongs to exactly one flow, so this store + * keys it by the auth session id `generateOAuthState()` puts in the OAuth + * `state` parameter (the 64-hex `authId`), and the callback looks it up by the + * `state` it was redirected back with. Only the outcome of a completed flow + * (client information, tokens) goes to the shared store. + * + * Keying inside the shared `servers[url]` entry instead would not help: the + * remote backend merges whole server entries, so a second tab's write would + * still replace the first tab's map. The web client therefore backs this with + * the initiating tab's `sessionStorage`, which survives the full-page redirect + * to the authorization server and back and is never visible to another tab. + */ + +/** In-flight PKCE verifiers, keyed by the OAuth `state` authId of their flow. */ +export interface PkceFlowStore { + saveCodeVerifier( + flowId: string, + serverUrl: string, + codeVerifier: string, + ): Promise; + /** + * The verifier saved for `flowId`, or `undefined` when there is none — or + * when the flow was started for a different server, so a `state` replayed + * against another server can never borrow that server's verifier. + */ + getCodeVerifier( + flowId: string, + serverUrl: string, + ): Promise; + clearCodeVerifier(flowId: string): Promise; +} + +/** The subset of the Web Storage API the store uses. */ +export type PkceFlowStorageArea = Pick< + Storage, + "getItem" | "setItem" | "removeItem" +>; + +export const PKCE_FLOW_STORAGE_KEY_PREFIX = "mcp-inspector:pkce-flow:"; + +interface StoredPkceFlow { + serverUrl: string; + codeVerifier: string; +} + +function parseStoredFlow(raw: string | null): StoredPkceFlow | undefined { + if (raw === null) return undefined; + try { + const value: unknown = JSON.parse(raw); + if ( + typeof value === "object" && + value !== null && + "serverUrl" in value && + "codeVerifier" in value && + typeof value.serverUrl === "string" && + typeof value.codeVerifier === "string" + ) { + return { serverUrl: value.serverUrl, codeVerifier: value.codeVerifier }; + } + } catch { + // A corrupt entry is treated as absent: the callback then fails with the + // "no verifier for this flow" error rather than throwing a parse error. + } + return undefined; +} + +/** + * A {@link PkceFlowStore} over a Web Storage area — `sessionStorage` by + * default, so each verifier is scoped to the tab that started its flow. + */ +export function createWebStoragePkceFlowStore( + getStorage: () => PkceFlowStorageArea = () => sessionStorage, +): PkceFlowStore { + const key = (flowId: string) => `${PKCE_FLOW_STORAGE_KEY_PREFIX}${flowId}`; + return { + async saveCodeVerifier(flowId, serverUrl, codeVerifier) { + const entry: StoredPkceFlow = { serverUrl, codeVerifier }; + getStorage().setItem(key(flowId), JSON.stringify(entry)); + }, + async getCodeVerifier(flowId, serverUrl) { + const entry = parseStoredFlow(getStorage().getItem(key(flowId))); + return entry?.serverUrl === serverUrl ? entry.codeVerifier : undefined; + }, + async clearCodeVerifier(flowId) { + getStorage().removeItem(key(flowId)); + }, + }; +} diff --git a/core/auth/providers.ts b/core/auth/providers.ts index 92512c0a2..2ca5968e2 100644 --- a/core/auth/providers.ts +++ b/core/auth/providers.ts @@ -10,7 +10,8 @@ import type { OAuthDiscoveryState, } from "@modelcontextprotocol/client"; import type { OAuthStorage, SaveClientInformationOptions } from "./storage.js"; -import { generateOAuthState } from "./utils.js"; +import { generateOAuthState, parseOAuthState } from "./utils.js"; +import type { PkceFlowStore } from "./pkce-flow-store.js"; import { applyAuthorizationParams } from "./authorizationParams.js"; import { scopeForDeclinedRefreshGrant } from "./scopes.js"; @@ -136,6 +137,13 @@ export type OAuthProviderConfig = { * the request when {@link requestRefreshToken} is false. (#2068) */ configuredScope?: string; + /** + * Where the in-flight PKCE verifier lives (#2726). When set, the verifier is + * keyed by the OAuth `state` of the flow that created it instead of sharing + * the server-URL-keyed slot in {@link storage}, so overlapping flows for the + * same server cannot overwrite each other's. See `pkce-flow-store.ts`. + */ + flowStore?: PkceFlowStore; }; /** @@ -161,6 +169,13 @@ export class BaseOAuthClientProvider implements OAuthClientProvider { protected requestRefreshToken: boolean; /** Server-configured scope, for the #2068 `offline_access` distinction. */ protected configuredScope?: string; + /** Per-flow verifier store (#2726); absent → the shared storage slot. */ + protected flowStore?: PkceFlowStore; + /** + * The authId of the flow this provider is driving: set by {@link state} on the + * redirect leg and by {@link resumeAuthorizationFlow} on the callback leg. + */ + private flowId: string | undefined; constructor(serverUrl: string, oauthConfig: OAuthProviderConfig) { this.serverUrl = serverUrl; @@ -171,6 +186,7 @@ export class BaseOAuthClientProvider implements OAuthClientProvider { this.authorizationParams = oauthConfig.authorizationParams; this.requestRefreshToken = oauthConfig.requestRefreshToken ?? true; this.configuredScope = oauthConfig.configuredScope; + this.flowStore = oauthConfig.flowStore; } /** @@ -266,7 +282,28 @@ export class BaseOAuthClientProvider implements OAuthClientProvider { } state(): string | Promise { - return generateOAuthState(); + const state = generateOAuthState(); + // The SDK calls `state()` immediately before `saveCodeVerifier()`, so this + // is the flow the verifier about to be saved belongs to (#2726). + this.flowId = state; + return state; + } + + /** + * Bind this provider to the flow a callback's `state` names, so + * {@link codeVerifier} reads that flow's verifier and no other (#2726). A + * `state` that is not one of ours binds nothing — and, with a flow store, a + * flow-less read then fails rather than reaching for the shared slot. + */ + resumeAuthorizationFlow(state: string): void { + this.flowId = parseOAuthState(state)?.authId; + } + + /** Drop the bound flow's verifier once its exchange has completed. */ + async clearAuthorizationFlow(): Promise { + if (this.flowStore && this.flowId) { + await this.flowStore.clearCodeVerifier(this.flowId); + } } async clientInformation( @@ -471,10 +508,32 @@ export class BaseOAuthClientProvider implements OAuthClientProvider { } async saveCodeVerifier(codeVerifier: string): Promise { + if (this.flowStore && this.flowId) { + await this.flowStore.saveCodeVerifier( + this.flowId, + this.serverUrl, + codeVerifier, + ); + return; + } await this.storage.saveCodeVerifier(this.serverUrl, codeVerifier); } async codeVerifier(): Promise { + if (this.flowStore) { + // Never fall back to the shared slot here: whatever it holds belongs to + // some other flow, and exchanging with it is exactly the PKCE failure + // this store exists to prevent (#2726). + const verifier = this.flowId + ? await this.flowStore.getCodeVerifier(this.flowId, this.serverUrl) + : undefined; + if (!verifier) { + throw new Error( + "No code verifier saved for this authorization flow. It may have been started in another browser tab or already completed; please re-authenticate.", + ); + } + return verifier; + } const verifier = await this.storage.getCodeVerifier(this.serverUrl); if (!verifier) { throw new Error("No code verifier saved for session"); @@ -529,6 +588,10 @@ export class BaseOAuthClientProvider implements OAuthClientProvider { await this.storage.clearTokens(this.serverUrl); return; case "verifier": + if (this.flowStore) { + await this.clearAuthorizationFlow(); + return; + } await this.storage.clearCodeVerifier(this.serverUrl); return; case "discovery": diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index 818c4f60e..48ccb8711 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -6336,8 +6336,13 @@ export class InspectorClient extends InspectorClientEventTarget { async completeOAuthFlow( authorizationCode: string, iss?: string, + state?: string, ): Promise { - await this.ensureOAuthManager().completeOAuthFlow(authorizationCode, iss); + await this.ensureOAuthManager().completeOAuthFlow( + authorizationCode, + iss, + state, + ); if (this.usesDirectAuthRecovery()) { await this.reconnectAfterAuthRecovery(); } @@ -6365,9 +6370,13 @@ export class InspectorClient extends InspectorClientEventTarget { */ async resumeAfterOAuth( authorizationCode: string, - options?: { remoteSessionId?: string; iss?: string }, + options?: { remoteSessionId?: string; iss?: string; state?: string }, ): Promise { - await this.completeOAuthFlow(authorizationCode, options?.iss); + await this.completeOAuthFlow( + authorizationCode, + options?.iss, + options?.state, + ); const remoteSessionId = options?.remoteSessionId; const transport = this.baseTransport; diff --git a/core/mcp/oauthManager.ts b/core/mcp/oauthManager.ts index 0f8f919ae..5cd68a29b 100644 --- a/core/mcp/oauthManager.ts +++ b/core/mcp/oauthManager.ts @@ -194,6 +194,8 @@ export class OAuthManager { // #2068: lets the provider tell an `offline_access` the user asked for // from one inherited from a previous grant's persisted scope. configuredScope: this.oauthConfig.scope, + // #2726: in-flight PKCE verifiers keyed by flow, not by server URL. + flowStore: this.oauthConfig.flowStore, }); provider.setEventTarget(this.params.getEventTarget()); @@ -394,9 +396,15 @@ export class OAuthManager { authorizationUrl.searchParams.get("scope")?.trim() || requestedScope; } + /** + * `state` is the OAuth `state` the callback was redirected back with. It + * selects the flow whose PKCE verifier the exchange uses (#2726); without it + * a provider with a flow store has no verifier to send. + */ async completeOAuthFlow( authorizationCode: string, iss?: string, + state?: string, ): Promise { try { if (this.isEnterpriseManaged()) { @@ -426,6 +434,9 @@ export class OAuthManager { const provider = await this.createOAuthProvider(); const serverUrl = this.getServerUrl(); + if (state !== undefined) { + provider.resumeAuthorizationFlow(state); + } const result = await mcpAuth(provider, { serverUrl, @@ -439,6 +450,8 @@ export class OAuthManager { `Expected AUTHORIZED after providing authorization code, got: ${result}`, ); } + // The code is single-use, so the verifier that redeemed it is spent. + await provider.clearAuthorizationFlow(); const tokens = await provider.tokens(); if (!tokens) { diff --git a/core/mcp/types.ts b/core/mcp/types.ts index b9c5606bd..7cde32900 100644 --- a/core/mcp/types.ts +++ b/core/mcp/types.ts @@ -39,6 +39,7 @@ import type { RedirectUrlProvider, } from "../auth/providers.js"; import type { OAuthStorage } from "../auth/storage.js"; +import type { PkceFlowStore } from "../auth/pkce-flow-store.js"; import type { AuthChallenge } from "../auth/challenge.js"; /** @@ -1265,6 +1266,13 @@ export interface InspectorClientEnvironment { * - Browser: from window.location or callback route */ redirectUrlProvider?: RedirectUrlProvider; + + /** + * Where in-flight PKCE verifiers live, keyed by the OAuth `state` of their + * flow (#2726). Omit to keep the verifier in {@link storage}. + * - Browser: `createWebStoragePkceFlowStore()` (the tab's sessionStorage) + */ + flowStore?: PkceFlowStore; }; } diff --git a/specification/v2_storage.md b/specification/v2_storage.md index 79f3cac4f..485f55962 100644 --- a/specification/v2_storage.md +++ b/specification/v2_storage.md @@ -62,6 +62,8 @@ OAuth and EMA **runtime state** (access/refresh tokens, PKCE verifiers, granted **Stack (#1549):** `OAuthStorage` interface → `OAuthStorageBase` + `OAuthMemoryStore` + `OAuthPersistBackend` (`core/auth/oauth-storage.ts`, `core/auth/store.ts`, `core/auth/oauth-persist.ts`). Reads still promote legacy `{ state, version }` envelopes (migrate-on-write). All getters are async; setters auto-persist. Web shares one store via `getWebRemoteOAuthStorage()`. +**In-flight PKCE verifiers stay in the tab (#2726):** the web client does **not** put a flow's PKCE verifier in the shared store. Its environment sets an `oauth.flowStore` (`createWebStoragePkceFlowStore()`, `core/auth/pkce-flow-store.ts`) that keeps each verifier in the initiating tab's `sessionStorage`, keyed by the flow's OAuth `state` authId, and the `/oauth/callback` exchange reads the verifier for the `state` it came back with. A server-URL-keyed slot would be shared by every overlapping flow for that server — two tabs, or several people on one deployed Inspector — so the later flow would overwrite the earlier one's verifier and fail its exchange with `invalid_grant`; and keying inside `servers[url]` would not help, since the remote backend merges whole server entries. Only a completed flow's outcome (client information, tokens) reaches `oauth.json`. CLI / TUI set no flow store and keep the verifier in their store as before. + **Secret split (#2481):** `oauth.json` holds only the **non-secret residue** — server URLs, granted scopes, cached authorization-server metadata, and non-secret client registration fields. Secret material (access/refresh tokens, `client_secret`, `registration_access_token`, IdP session payloads) lives in the OS **secret store** (`core/auth/node/secret-store.ts`), keyed per server. Reads rejoin the residue with the stored secrets (lazily migrating any legacy plaintext into the store); writes are section-scoped merges under the state-file lock, so a writer never clobbers sections it did not read. A present-but-unrecognized `oauth.json` refuses mutation rather than being treated as empty. See [docs/secret-storage.md](../docs/secret-storage.md) for store selection and fallbacks. Details: [EMA auth](v2_auth_ema.md#oauth-persistence-1549--done), [Mid-session auth](v2_auth_mid_session.md). From 4c02f4b92ed7f1c87042740c857374b8d2c41556 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:45:00 -0400 Subject: [PATCH 27/44] fix(core): annotate 429s on the raw-wire path; reject impossible dates A modern-era tool call against a server advertising the tasks extension goes through ext-tasks' raw-wire channel, whose host hook applied only the timeout annotation, so its 429 kept the generic message. Both paths now share one annotateRequestError, and the hook is renamed annotateError. Date.parse rolls an impossible IMF-fixdate (31 Nov) over to a real one, so a Retry-After is accepted only if it prints back exactly as sent. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../extension/tasks/rawWireChannel.test.ts | 15 ++-- .../web/src/test/core/mcp/rateLimit.test.ts | 9 ++- .../mcp/inspectorClient-rate-limit.test.ts | 75 ++++++++++++++++++- core/extension/tasks/rawWireChannel.ts | 14 ++-- core/mcp/inspectorClient.ts | 23 ++++-- core/mcp/rateLimit.ts | 9 ++- 6 files changed, 117 insertions(+), 28 deletions(-) diff --git a/clients/web/src/test/core/extension/tasks/rawWireChannel.test.ts b/clients/web/src/test/core/extension/tasks/rawWireChannel.test.ts index eb2300cd4..af9a83169 100644 --- a/clients/web/src/test/core/extension/tasks/rawWireChannel.test.ts +++ b/clients/web/src/test/core/extension/tasks/rawWireChannel.test.ts @@ -23,7 +23,7 @@ function setup(overrides: Partial = {}) { transport: () => transport, defaultTimeoutMs: () => 1_000, resetTimeoutOnProgress: () => true, - annotateTimeout: (error) => error, + annotateError: (error) => error, ...overrides, }); return { channel, sent, transport }; @@ -100,8 +100,8 @@ describe("RawWireChannel", () => { it("times out with the annotated timeout as its cause, re-armed by progress", async () => { vi.useFakeTimers(); - const annotateTimeout = vi.fn((error: unknown) => error); - const { channel, sent } = setup({ annotateTimeout }); + const annotateError = vi.fn((error: unknown) => error); + const { channel, sent } = setup({ annotateError }); const promise = channel.dispatch({ method: "tools/call", params: { _meta: { progressToken: "p" } }, @@ -121,10 +121,7 @@ describe("RawWireChannel", () => { error instanceof DispatchError && (error.cause as { code?: unknown }).code !== undefined, ); - expect(annotateTimeout).toHaveBeenCalledWith( - expect.anything(), - "tools/call", - ); + expect(annotateError).toHaveBeenCalledWith(expect.anything(), "tools/call"); // stdio-style transport: the timeout reaches the server as a cancel. expect(sent[1]).toMatchObject({ method: "notifications/cancelled" }); }); @@ -166,8 +163,8 @@ describe("RawWireChannel", () => { it("annotates a send failure and rejects everything on rejectAll", async () => { const boom = new Error("closed"); - const annotateTimeout = vi.fn(() => boom); - const failing = setup({ annotateTimeout }); + const annotateError = vi.fn(() => boom); + const failing = setup({ annotateError }); failing.transport.send.mockRejectedValueOnce("raw failure"); await expect( failing.channel.dispatch({ method: "tasks/get" }), diff --git a/clients/web/src/test/core/mcp/rateLimit.test.ts b/clients/web/src/test/core/mcp/rateLimit.test.ts index 2cdf8870c..8e3b628b9 100644 --- a/clients/web/src/test/core/mcp/rateLimit.test.ts +++ b/clients/web/src/test/core/mcp/rateLimit.test.ts @@ -179,7 +179,14 @@ describe("parseRetryAfter", () => { expect(parseRetryAfter("-5", NOW)).toBeUndefined(); expect(parseRetryAfter("1.5", NOW)).toBeUndefined(); expect(parseRetryAfter("2026-10-19T12:01:30Z", NOW)).toBeUndefined(); - // The right shape, but no real date. + // The right shape, but no real date: `Date.parse` rolls an impossible + // day over (31 Nov → 1 Dec) and ignores a weekday that does not match. + expect(parseRetryAfter("Tue, 31 Nov 2026 12:00:00 GMT", NOW)).toBe( + undefined, + ); + expect(parseRetryAfter("Tue, 19 Oct 2026 12:01:30 GMT", NOW)).toBe( + undefined, + ); expect(parseRetryAfter("Mon, 99 Foo 2026 12:00:00 GMT", NOW)).toBe( undefined, ); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts index 86e826732..ff21cd261 100644 --- a/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts +++ b/clients/web/src/test/integration/mcp/inspectorClient-rate-limit.test.ts @@ -84,6 +84,14 @@ function startThrottlingServer(): Promise { res.writeHead(202).end(); return; } + if (body.method === "tools/call") { + res.writeHead(THROTTLED.status, { + "content-type": "text/plain", + ...THROTTLED.headers, + }); + res.end(THROTTLED.body); + return; + } if (body.method === "tools/list") { res.writeHead(toolsListReply.status, { "content-type": "text/plain", @@ -109,6 +117,30 @@ function startThrottlingServer(): Promise { } return; } + if (body.method === "server/discover") { + // Only a modern-era connect sends this; the legacy-pinned cases never do. + res.writeHead(200, { "content-type": "application/json" }); + res.end( + JSON.stringify({ + jsonrpc: "2.0", + id: body.id, + result: { + resultType: "complete", + ttlMs: 0, + cacheScope: "public", + supportedVersions: ["2026-07-28"], + // The tasks extension is what routes a modern `tools/call` onto + // ext-tasks' raw-wire channel instead of the SDK's `request`. + capabilities: { + tools: {}, + extensions: { "io.modelcontextprotocol/tasks": {} }, + }, + serverInfo: { name: "throttling-server", version: "1.0.0" }, + }, + }), + ); + return; + } const result = body.method === "initialize" ? { @@ -204,13 +236,15 @@ describe("InspectorClient rate-limit recognition (#2442)", () => { async function connectTo( url: string, transport: CreateTransport = createTransportNode, + era: "legacy" | "modern" = "legacy", ): Promise { client = new InspectorClient( { type: "streamable-http", url }, { environment: { transport }, - // Pinned legacy so the connect is exactly `initialize`. - versionNegotiation: eraToVersionNegotiation("legacy"), + // Pinned, so the connect is exactly `initialize` (legacy) or + // `server/discover` (modern) and nothing else. + versionNegotiation: eraToVersionNegotiation(era), }, ); await client.connect(); @@ -350,4 +384,41 @@ describe("InspectorClient rate-limit recognition (#2442)", () => { await connectTo(upstream.url, started.transport), ); }); + + /** + * Against a modern server advertising the tasks extension, a tool call + * bypasses the SDK's `Client.request` for the ext-tasks raw-wire channel, + * so this is the case that proves the annotation is applied on that path + * too, not only on the SDK funnel. (`tools/list` is no use here: it takes + * the SDK funnel in both eras, and so does `tools/call` without the + * extension.) + */ + async function expectModernThrottled(c: InspectorClient): Promise { + expect(c.getProtocolEra()).toBe("modern"); + const caught = await rejectionOf( + c.callTool({ name: "echo", inputSchema: { type: "object" } }, {}), + ); + const data = expectThrottled(caught); + expect((caught as Error).message).toMatch( + /^Rate limited by the server \(HTTP 429\) on tools\/call\. The server asked to retry after 30s\. /, + ); + expect(data.method).toBe("tools/call"); + expect(data.retryAfterMs).toBe(30_000); + } + + it("annotates a modern-era 429, which takes the raw-wire path", async () => { + upstream = await startThrottlingServer(); + await expectModernThrottled( + await connectTo(upstream.url, createTransportNode, "modern"), + ); + }); + + it("annotates a modern-era 429 through the web backend", async () => { + upstream = await startThrottlingServer(); + const started = await startRemoteBackend(); + backend = started.server; + await expectModernThrottled( + await connectTo(upstream.url, started.transport, "modern"), + ); + }); }); diff --git a/core/extension/tasks/rawWireChannel.ts b/core/extension/tasks/rawWireChannel.ts index 455062c5f..e2c401715 100644 --- a/core/extension/tasks/rawWireChannel.ts +++ b/core/extension/tasks/rawWireChannel.ts @@ -47,9 +47,9 @@ export interface RawWireChannelHost { defaultTimeoutMs(): number; /** Whether progress on a request's token re-arms its timeout. */ resetTimeoutOnProgress(): boolean; - /** Decorate a timeout the way an SDK request's timeout is (#2318); any - * other error is returned untouched. */ - annotateTimeout(error: unknown, method: string): unknown; + /** Decorate a failed request's error the way an SDK request's is — a + * timeout (#2318), a `429` (#2442); any other error is returned untouched. */ + annotateError(error: unknown, method: string): unknown; } interface PendingRawRequest { @@ -182,7 +182,7 @@ export class RawWireChannel { `Raw MCP request "${message.method}" timed out after ${timeoutMs} ms`, false, { - cause: this.host.annotateTimeout( + cause: this.host.annotateError( new SdkError(SdkErrorCode.RequestTimeout, "Request timed out", { timeout: timeoutMs, }), @@ -232,10 +232,10 @@ export class RawWireChannel { pending.cleanup(); // The browser's remote transport awaits the response inside `send`, // so its relay wait can expire here first, as the SDK's timeout - // shape; annotate it exactly as the local timer above does (a - // non-timeout error passes through untouched). + // shape; annotate it exactly as the local timer above does. A `429` + // fails the send here too, and is annotated the same way (#2442). reject( - this.host.annotateTimeout( + this.host.annotateError( error instanceof Error ? error : new Error(String(error)), message.method, ), diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index e9b026901..83dfc14ee 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -536,8 +536,7 @@ export class InspectorClient extends InspectorClientEventTarget { transport: () => this.transport, defaultTimeoutMs: () => this.requestTimeout ?? DEFAULT_REQUEST_TIMEOUT_MSEC, resetTimeoutOnProgress: () => this.resetTimeoutOnProgress, - annotateTimeout: (error, method) => - annotateRequestTimeout(error, method, this.getConnectionDiagnostics()), + annotateError: (error, method) => this.annotateRequestError(error, method), }); private readonly dispatchTaskRequest: RawClientDispatch = ( request, @@ -1117,16 +1116,26 @@ export class InspectorClient extends InspectorClientEventTarget { try { return await sdkRequest(...args); } catch (err) { - const method = args[0].method; - throw annotateRateLimit( - annotateRequestTimeout(err, method, this.getConnectionDiagnostics()), - method, - ); + throw this.annotateRequestError(err, args[0].method); } }; client.request = decorated as SdkRequest; } + /** + * The one place a failed request's error is decorated, shared by the SDK + * funnel above and the raw-wire channel that bypasses it (modern-era tool + * calls and task operations), so a caller sees the same error whichever + * path its request took: a timeout with the connection's state (#2318), a + * `429` named as rate limiting (#2442), anything else untouched. + */ + private annotateRequestError(err: unknown, method: string): unknown { + return annotateRateLimit( + annotateRequestTimeout(err, method, this.getConnectionDiagnostics()), + method, + ); + } + /** * @param base - the composed OAuth-path fetch, deadline innermost (#2319). * diff --git a/core/mcp/rateLimit.ts b/core/mcp/rateLimit.ts index 1bbfe637d..6696e79e4 100644 --- a/core/mcp/rateLimit.ts +++ b/core/mcp/rateLimit.ts @@ -134,8 +134,13 @@ export function parseRetryAfter( if (/^\d+$/.test(trimmed)) return Number(trimmed) * 1000; if (!IMF_FIXDATE.test(trimmed)) return undefined; const at = Date.parse(trimmed); - // A well-formed shape can still name no real date (`Mon, 99 Foo 2026 …`). - if (Number.isNaN(at)) return undefined; + // A well-formed shape can still name no real date. `Mon, 99 Foo 2026 …` is + // `NaN`, but `Tue, 31 Nov 2026 …` silently rolls over to 1 December, so the + // parse is accepted only if it prints back as exactly what was sent — + // `toUTCString` emits IMF-fixdate, so a real date always does. + if (Number.isNaN(at) || new Date(at).toUTCString() !== trimmed) { + return undefined; + } return Math.max(0, at - now); } From 3fd5b5b52faef509a89cb39ea79f988f395695e8 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 00:46:31 -0400 Subject: [PATCH 28/44] test(auth): keep one storage area in the cross-server flow-store test Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/src/test/core/auth/pkce-flow-store.test.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/clients/web/src/test/core/auth/pkce-flow-store.test.ts b/clients/web/src/test/core/auth/pkce-flow-store.test.ts index 708a633dd..6cec55c1c 100644 --- a/clients/web/src/test/core/auth/pkce-flow-store.test.ts +++ b/clients/web/src/test/core/auth/pkce-flow-store.test.ts @@ -51,9 +51,12 @@ describe("createWebStoragePkceFlowStore (#2726)", () => { }); it("does not answer for a different server than the flow was started for", async () => { - const store = createWebStoragePkceFlowStore(() => makeArea()); + const area = makeArea(); + const store = createWebStoragePkceFlowStore(() => area); await store.saveCodeVerifier(FLOW, SERVER, "cv"); + // Same area, so only the server binding can make this read miss. + expect(await store.getCodeVerifier(FLOW, SERVER)).toBe("cv"); expect( await store.getCodeVerifier(FLOW, "https://other.example.com"), ).toBeUndefined(); From 15de439c20dc5bef49613db1fbb43b77850606c3 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 01:07:55 -0400 Subject: [PATCH 29/44] fix(auth): bind an owner stamp with an HMAC keyed by the credential CodeQL flagged the stamp's plain SHA-256 of the client secret as an insufficient password hash (js/insufficient-password-hash, alert #82). The digest sat in the same store as the secret, so it revealed nothing that store does not, but the shape is still a fast unkeyed hash of a secret. The stamp now carries an HMAC over the client_id keyed by the credential: only a writer holding the value can produce it, it matches only that value, and it also refuses a stamp whose client_id was swapped. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/auth/oauth-namespace-ledger.test.ts | 2 +- .../src/test/core/auth/oauth-secrets.test.ts | 9 +++++- core/auth/node/oauth-secrets.ts | 30 ++++++++++++------- docs/secret-storage.md | 2 +- 4 files changed, 29 insertions(+), 14 deletions(-) diff --git a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts index a58c9466b..cf0150be7 100644 --- a/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts +++ b/clients/web/src/test/core/auth/oauth-namespace-ledger.test.ts @@ -766,7 +766,7 @@ describe("pending carry-over (#2672)", () => { ["no stamp", null], ["a malformed stamp", "{not json"], ["a stamp that is not an object", "7"], - ["a stamp naming no client", JSON.stringify({ sha256: "x" })], + ["a stamp naming no client", JSON.stringify({ mac: "x" })], // The hand-off keeps and copies per account: a kept value can meet a // copied stamp, and an older Inspector can rewrite a value alone. ["another value's stamp", clientCredentialStamp("c", "other")], diff --git a/clients/web/src/test/core/auth/oauth-secrets.test.ts b/clients/web/src/test/core/auth/oauth-secrets.test.ts index 4ce2d521b..ef629162a 100644 --- a/clients/web/src/test/core/auth/oauth-secrets.test.ts +++ b/clients/web/src/test/core/auth/oauth-secrets.test.ts @@ -493,8 +493,15 @@ describe("client-credential owner stamps (#2734)", () => { ["no stamp", undefined], ["a malformed stamp", "{"], ["a non-object stamp", "null"], - ["a non-string client_id", JSON.stringify({ client_id: 7, sha256: "" })], + ["a non-string client_id", JSON.stringify({ client_id: 7, mac: "" })], ["another value's stamp", clientCredentialStamp("cid", "other")], + [ + "a stamp whose client was swapped", + JSON.stringify({ + ...(JSON.parse(clientCredentialStamp("cid", "ics")) as object), + client_id: "other", + }), + ], ])("names no owner for %s", (_label, stamp) => { expect(stampedOwner(stamp, "ics")).toBeUndefined(); }); diff --git a/core/auth/node/oauth-secrets.ts b/core/auth/node/oauth-secrets.ts index 2181dca85..b728ca155 100644 --- a/core/auth/node/oauth-secrets.ts +++ b/core/auth/node/oauth-secrets.ts @@ -20,7 +20,7 @@ * route); the browser round-trips full snapshots over the authed local API. */ -import { createHash, randomUUID } from "node:crypto"; +import { createHmac, randomUUID } from "node:crypto"; import { OAuthTokensSchema } from "@modelcontextprotocol/core"; import type { OAuthTokens } from "@modelcontextprotocol/client"; import { setOwnEntry } from "../../storage/own-entry.js"; @@ -160,20 +160,28 @@ export const IDP_SESSION_FIELD = "idp-session"; export const clientCredentialStampField = (field: string): string => `${field}#owner`; -const credentialDigest = (value: string): string => - createHash("sha256").update(value).digest("hex"); +/** + * A MAC over `clientId` keyed by the credential itself: only a writer + * holding `value` can produce it, and it matches only that value and that + * client. Keyed rather than a plain digest of the value, so the credential + * is never the hashed input — a fast unkeyed hash of a secret is the shape + * of an offline-guessable password hash, even though this one sits in the + * same store as the value and so reveals nothing that store does not. + */ +const credentialMac = (clientId: string, value: string): string => + createHmac("sha256", value).update(clientId).digest("hex"); /** - * The stamp naming `clientId` as the owner of `value`. It carries a digest - * of the value so a stamp separated from its value — the hand-off keeps - * and copies per account, so a kept value can meet a copied stamp, and an - * older Inspector can rewrite a value without its stamp — never vouches - * for another value. The digest is as protected as the value beside it. + * The stamp naming `clientId` as the owner of `value`. It carries a MAC + * bound to the value so a stamp separated from its value — the hand-off + * keeps and copies per account, so a kept value can meet a copied stamp, + * and an older Inspector can rewrite a value without its stamp — never + * vouches for another value. */ export function clientCredentialStamp(clientId: string, value: string): string { return JSON.stringify({ client_id: clientId, - sha256: credentialDigest(value), + mac: credentialMac(clientId, value), }); } @@ -195,8 +203,8 @@ export function stampedOwner( return undefined; } if (typeof parsed !== "object" || parsed === null) return undefined; - const { client_id: clientId, sha256 } = parsed as Record; - if (typeof clientId !== "string" || sha256 !== credentialDigest(value)) { + const { client_id: clientId, mac } = parsed as Record; + if (typeof clientId !== "string" || mac !== credentialMac(clientId, value)) { return undefined; } return clientId; diff --git a/docs/secret-storage.md b/docs/secret-storage.md index c02ce9510..be20aa202 100644 --- a/docs/secret-storage.md +++ b/docs/secret-storage.md @@ -17,7 +17,7 @@ These values are stored as secrets: They are kept out of `mcp.json` so that sharing, committing or syncing the file does not leak credentials (#1356). When the Inspector saves an entry to a durable store, it leaves each `env` key in `mcp.json` with an empty value and omits the client secret; the real values live in the store. `headers` are **not** moved: they are saved in `mcp.json` exactly as written, so a header that carries a credential stays in the file. [MCP server configuration](./mcp-server-configuration.md) describes what that means for other tools reading the same file. -Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also stores, beside every dynamically registered client's `client_secret` and registration token, an owner stamp naming the `client_id` it was written for, with a digest of the value so the stamp vouches for that value only (an extra `…#owner` entry per credential in the store). The stamp lives in the store rather than the ledger because moving `secrets.json` into the keychain copies or keeps each entry as it finds it, without consulting any ledger, so the stamp follows the value that was actually kept. The next run on the owed entries' backend reads them, and its first locked save copies them, stamp included, into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the value's stamp names the client the state file holds now, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. A value with no stamp that vouches for it (written before stamps existed, or left behind beside another value's stamp) is purged as before rather than guessed at. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). +Acquired tokens follow the same rule for the OAuth state file: `oauth.json` keeps only non-secret state (flow bookkeeping, discovered metadata, public client ids), and the tokens and client secrets it used to hold live in the secret store. Each state file's store entries are scoped by a `secretsNamespace` UUID stamped into the file on its first write, so two state files (for example, per-profile `MCP_INSPECTOR_OAUTH_STATE_PATH` values) that authorize against the same server keep separate entries even in a shared store such as the OS keychain. A pre-namespace file is adopted transparently on its first save — its existing un-namespaced entries move under the new namespace, so the adopting file keeps its credentials. That migration deletes its sources, so it runs only under the real cross-process file lock; on a box where the lock cannot be taken, the save fails with a retryable error instead of racing a concurrent adopter — if you cannot make the lock directory writable there, the escape is to clear the file's stored OAuth state and re-authorize, since clearing does not migrate and a fresh state file mints its namespace without the lock. The namespace stamp is also one-way across versions: an Inspector older than the namespace (≤ 2.9.x) resolves only un-namespaced entries, so downgrading after adoption logs you out, and its next save strips the stamp — alternating old and new versions on one state file therefore re-adopts under a fresh namespace each time. The earlier namespace's entries are not stranded by this: each state file keeps a sidecar ledger, `.namespaces.json`, recording every namespace it has used and the servers and IdP issuers given store entries under each, and the next re-adoption (or removal of the state file) purges every recorded namespace the file no longer carries. Before re-adoption purges anything, it carries over what the older version could not see: for every server and IdP issuer the file still names, any secret the un-namespaced entries lack is copied from the namespaces the file used to carry, newest first, so a confidential client's `client_secret` (which the older version saved the `client_id` for, but could not reach) survives the round trip; reads of an unstamped file fall back the same way, so the state a save starts from already holds it. Only what nothing references any longer is purged — on every store backend, the OS keychain included, since the ledger names each entry and nothing has to be enumerated. The same purge cleans up after a state file deleted by hand, at the next save to that path. Entries are recorded per store backend (the keychain, or a particular `secrets.json`), and only a run using that backend purges them — so a run that falls back to a different store leaves the keychain's record for the next keychain run rather than "purging" an empty store and forgetting it. A keychain run also purges what was recorded against a `secrets.json`, since moving that file into the keychain once one is available copies its entries across; and when a `secrets.json` run purges its own record, the record passes to the keychain rather than being forgotten, so those copies are still cleaned up the next time the keychain is in use. Re-adoption's carry-over can only read the backend it runs on, so when it lands on a different one than the entries it supersedes (a locked keychain falling back to `secrets.json`, say), it marks those entries in the ledger as still owed to the new namespace (and fails the save if it cannot, rather than commit without the mark). Each save also stores, beside every dynamically registered client's `client_secret` and registration token, an owner stamp naming the `client_id` it was written for, with a MAC keyed by the value so the stamp vouches for that value only (an extra `…#owner` entry per credential in the store). The stamp lives in the store rather than the ledger because moving `secrets.json` into the keychain copies or keeps each entry as it finds it, without consulting any ledger, so the stamp follows the value that was actually kept. The next run on the owed entries' backend reads them, and its first locked save copies them, stamp included, into the new namespace before purging the old one — but only where the new namespace holds nothing for that field and the value's stamp names the client the state file holds now, since a re-registered client's secret is not the old one's, and anything else could resurrect a value cleared since. A value with no stamp that vouches for it (written before stamps existed, or left behind beside another value's stamp) is purged as before rather than guessed at. Tokens and pre-registered clients' secrets are not carried this way: either could have been cleared deliberately in the meantime with nothing to tell, and a token is recovered by re-authorizing. Like adoption's migration, the purge runs only under the real cross-process file lock; a namespace whose purge fails stays recorded and is retried on the next save, with a warning. One limit remains: if the ledger is deleted along with its state file, the entries it recorded cannot be found on the keyring backend, which cannot list its entries; on the `file` backend they remain visible in `secrets.json` (ids beginning `oauth++`) and can be removed by hand. If more than one pre-namespace state file was sharing a server's un-namespaced entry, the first one to adopt takes it with it, and each remaining pre-namespace profile re-authorizes that server once — its copy was already being overwritten by every other profile's saves, which is the bug the namespace fixes. Removing a profile that is still pre-namespace likewise purges the shared un-namespaced entries, as removal always has: the file being deleted is the store's only index of them, so leaving them would strand credentials nothing could find or clear again. A pre-existing `oauth.json` that still carries plaintext tokens is migrated on first read — the tokens move into the store and the file is rewritten without them — but only when the store is durable; under the `memory` store the file is left as-is, since it is still the only durable copy. A token entry so malformed it cannot be a credential (for example, a hand-edited value of the wrong type) is left in the file rather than migrated, so it stays visible and clearable. `MCP_INSPECTOR_PERSIST_TOKENS=all|access|none` controls which acquired tokens are persisted at all (see [Environment variables](./environment-variables.md#secret-store)). ## How the store is chosen From 25e5a2661d1d8907c834c88c8019e9601cd5935d Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 01:27:54 -0400 Subject: [PATCH 30/44] feat(cli): add --load mode for load-testing a tool `--load` drives one tool with concurrent, sustained tools/call traffic over the connection the CLI already sets up (transport, headers, OAuth), then prints a latency / error-rate summary in place of the single result. Bounds: --load-concurrency (default 10), --load-duration (default 10s when --load-requests is unset), --load-requests, and --load-rate (even spacing). The summary separates isError results (toolErrors) from calls that threw (errors), groups thrown messages (first ten kinds, rest as "(other)"), and reports nearest-rank latency percentiles and throughput. The engine lives in core/cli/handlers/load-test.ts so mcpdo can reuse it later; only the one-shot CLI exposes it for now. Closes #2439 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/cli/README.md | 70 +++- clients/cli/__tests__/load-test.test.ts | 525 ++++++++++++++++++++++++ clients/cli/src/cli.ts | 95 +++++ core/cli/handlers/load-test.ts | 321 +++++++++++++++ 4 files changed, 1010 insertions(+), 1 deletion(-) create mode 100644 clients/cli/__tests__/load-test.test.ts create mode 100644 core/cli/handlers/load-test.ts diff --git a/clients/cli/README.md b/clients/cli/README.md index db48f91e2..efda7034d 100644 --- a/clients/cli/README.md +++ b/clients/cli/README.md @@ -153,6 +153,11 @@ Options that specify the MCP server (catalog/config file, ad-hoc command/URL, en | `-q`, `--quiet` | Print only the result payload on stdout, or the error envelope on stderr on failure. Drops status lines, warnings, advisory summaries and a stdio server's own stderr; OAuth prompts a human must answer still print. See [Quiet output](#quiet-output---quiet). | | `--output ` | Write the result to a file **instead of** stdout. The parent directory must exist; an existing file is replaced. See [Saving a result to a file](#saving-a-result-to-a-file---output). | | `--output-format ` | Encoding of the `--output` file. `json` (default) is the whole result, pretty-printed. `raw` is the result's text, or the decoded bytes of its single image/audio/blob block; `tools/call` and `resources/read` only. | +| `--load` | Load-test one tool: call it repeatedly and concurrently, then print a latency / error-rate summary in place of a result. Requires `--method tools/call --tool-name `. See [Load testing a tool](#load-testing-a-tool---load). | +| `--load-concurrency ` | With `--load`: the most calls in flight at once (positive integer; default `10`). | +| `--load-duration ` | With `--load`: stop starting calls after this many seconds; calls already in flight still finish. Default `10` when `--load-requests` is not set. | +| `--load-requests ` | With `--load`: stop after starting this many calls. With `--load-duration` too, whichever is reached first ends the run. | +| `--load-rate ` | With `--load`: start at most this many calls per second across all workers, spaced evenly (default: unlimited). | | `--relogin` | Delete stored OAuth for this server URL from the shared store before connect; interactive login still only runs if the server requires auth. Requires an HTTP/SSE URL (rejected for stdio). Conflicts with `--stored-auth-only` / `--use-stored-auth` / `--wait-for-auth` / catalog short-circuits. | | `--no-revoke` | With `--relogin`, skip the [RFC 7009](https://datatracker.ietf.org/doc/html/rfc7009) revocation request that would otherwise end the grant at the authorization server when the local state is deleted. The per-server `oauth.revokeOnClear` setting is the persistent form of the same opt-out; either one is enough to skip it. See [Revoking on `--relogin`](#revoking-on---relogin). | | `--stored-auth-only` | **CI / non-interactive safe:** never start interactive OAuth / step-up (and never auto-open a browser); use the shared store if present, otherwise fail immediately with `auth_required`. Prefer this over a bare pipe/CI run that would otherwise attempt interactive login. | @@ -285,7 +290,70 @@ The flag is `--output-format` rather than a new `--format` value because `--form Exit codes are unchanged: a `tools/call` that returns `isError:true` is still written and still exits `5`. A file that cannot be written (missing directory, no permission) exits `1` with envelope code `output_write_failed`. -`--output` is rejected where there is no single result to write — `--app-info`, `--verify`, `--list-stored-auth`, `--print-handoff`, and `servers/list` / `servers/show` — and `--output-format` is rejected without `--output`, rather than either being silently ignored. +`--output` is rejected where there is no single result to write — `--app-info`, `--load`, `--verify`, `--list-stored-auth`, `--print-handoff`, and `servers/list` / `servers/show` — and `--output-format` is rejected without `--output`, rather than either being silently ignored. + +#### Load testing a tool (`--load`) + +`--load` points the Inspector's own connection — transport, headers, OAuth and +all — at one tool and keeps calling it, so seeing how a server behaves under +concurrent or sustained traffic does not mean rewriting that setup in a +separate script ([#2439](https://github.com/modelcontextprotocol/inspector/issues/2439)). + +```bash +# 20 calls in flight for 30 seconds +mcp-inspector --cli --method tools/call --tool-name search \ + --tool-arg query=mcp --load --load-concurrency 20 --load-duration 30 + +# exactly 500 calls, started at no more than 50 per second +mcp-inspector --cli --method tools/call --tool-name search \ + --tool-arg query=mcp --load --load-requests 500 --load-rate 50 +``` + +Every call sends the same arguments and `_meta`. When the run ends, one summary +is printed on stdout (wrapped in `{ "result": … }` with `--format json`): + +```json +{ + "tool": "search", + "concurrency": 20, + "durationMs": 30000, + "requests": 41210, + "ok": 41187, + "toolErrors": 0, + "errors": 23, + "errorRate": 0.0006, + "elapsedMs": 30012.4, + "throughput": 1373.12, + "latencyMs": { "min": 2.1, "mean": 14.5, "p50": 12.9, "p90": 22.4, "p99": 61.3, "max": 30004.8 }, + "errorMessages": [{ "message": "Request timed out", "count": 23 }] +} +``` + +- **`toolErrors`** counts results carrying `isError: true` — the tool ran and + reported a failure. **`errors`** counts calls that never produced a result: + protocol errors, request timeouts, a dropped connection. `errorRate` is both + together over `requests`, rounded to four places. +- **`errorMessages`** groups the thrown messages by frequency. The first ten + distinct messages are kept; any further ones are counted under `(other)`. +- **Latency** is per call, client-side, in milliseconds, with nearest-rank + percentiles. It includes everything the Inspector does around the request, + which is negligible next to a network round trip but not next to a + sub-millisecond stdio echo. +- **Bounds.** `--load-duration` stops *starting* calls; anything in flight is + allowed to finish, so `elapsedMs` can exceed the window by up to one call. + With `--load-rate`, a call whose slot falls past the window is never started. + +**All calls share one connection**, so this measures how a server copes with +concurrent requests on a single session — not with many sessions. A slow call +is bounded by the server's request timeout (`requestTimeout` in the +catalog/config file). + +The tool is looked up once before the clock starts, so a missing tool fails +the run with exit `5` (`tool_not_found`) instead of reporting a 100% error +rate. Otherwise a completed run exits `0` whatever its error rate — the summary +is the result. `--load` is rejected with any other method and with +`--app-info`, `--output` and `--output-format`, and each `--load-*` flag is +rejected without `--load`. #### Schema portability (`--strict`) diff --git a/clients/cli/__tests__/load-test.test.ts b/clients/cli/__tests__/load-test.test.ts new file mode 100644 index 000000000..b59b03cb3 --- /dev/null +++ b/clients/cli/__tests__/load-test.test.ts @@ -0,0 +1,525 @@ +import { describe, it, expect } from "vitest"; +import { runCli } from "./helpers/cli-runner.js"; +import { + expectCliSuccess, + expectCliFailure, + expectValidJson, +} from "./helpers/assertions.js"; +import { + getTestMcpServerCommand, + createTestServerHttp, + createTestServerInfo, + createSimpleTaskTool, + getCrashableTestMcpServerCommand, + CRASH_SERVER_TOOL_NAME, +} from "@modelcontextprotocol/inspector-test-server"; +import { + DEFAULT_LOAD_CONCURRENCY, + DEFAULT_LOAD_DURATION_MS, + MAX_ERROR_KINDS, + OTHER_ERRORS, + percentile, + positiveIntegerParser, + positiveNumberParser, + resolveLoadOptions, + runLoad, + summarizeLatency, + type LoadCallOutcome, + type LoadClock, +} from "@inspector/core/cli/handlers/load-test.js"; + +/** + * A clock that only moves when told to. `sleep` advances it by the requested + * amount, so a rate-limited run completes instantly while still observing + * the schedule it would keep in real time. + */ +function fakeClock(): LoadClock & { advance: (ms: number) => void } { + let t = 1000; + return { + now: () => t, + sleep: async (ms) => { + t += ms; + }, + advance: (ms) => { + t += ms; + }, + }; +} + +describe("load-test engine", () => { + describe("percentile / summarizeLatency", () => { + it("uses nearest-rank percentiles", () => { + const sorted = Array.from({ length: 100 }, (_, i) => i + 1); + expect(percentile(sorted, 50)).toBe(50); + expect(percentile(sorted, 90)).toBe(90); + expect(percentile(sorted, 99)).toBe(99); + expect(percentile([7], 99)).toBe(7); + expect(percentile([1, 2, 3], 0)).toBe(1); + }); + + it("summarizes unsorted samples and returns null for none", () => { + expect(summarizeLatency([])).toBeNull(); + expect(summarizeLatency([3, 1, 2])).toEqual({ + min: 1, + mean: 2, + p50: 2, + p90: 3, + p99: 3, + max: 3, + }); + }); + }); + + describe("flag parsers", () => { + it("accepts only plain positive integers for integer flags", () => { + const parse = positiveIntegerParser("--load-requests"); + expect(parse(" 25 ")).toBe(25); + for (const bad of ["0", "-1", "1.5", "1e3", "0x10", "", "abc"]) { + expect(() => parse(bad)).toThrow( + "--load-requests must be a positive integer.", + ); + } + }); + + it("accepts positive finite numbers for number flags", () => { + const parse = positiveNumberParser("--load-rate"); + expect(parse("2.5")).toBe(2.5); + for (const bad of ["0", "-1", "", " ", "Infinity", "abc"]) { + expect(() => parse(bad)).toThrow( + "--load-rate must be a positive number.", + ); + } + }); + }); + + describe("resolveLoadOptions", () => { + it("defaults to the default concurrency over the default window", () => { + expect(resolveLoadOptions({})).toEqual({ + concurrency: DEFAULT_LOAD_CONCURRENCY, + durationMs: DEFAULT_LOAD_DURATION_MS, + }); + }); + + it("drops the default window when a request count bounds the run", () => { + expect(resolveLoadOptions({ requests: 5, concurrency: 2 })).toEqual({ + concurrency: 2, + requests: 5, + }); + }); + + it("keeps both bounds and the rate when all are given", () => { + expect( + resolveLoadOptions({ durationSec: 1.5, requests: 5, rate: 3 }), + ).toEqual({ + concurrency: DEFAULT_LOAD_CONCURRENCY, + durationMs: 1500, + requests: 5, + rate: 3, + }); + }); + }); + + describe("runLoad", () => { + it("starts exactly the request count and never exceeds concurrency", async () => { + let inFlight = 0; + let peak = 0; + const call = async (): Promise => { + inFlight++; + peak = Math.max(peak, inFlight); + await new Promise((r) => setTimeout(r, 1)); + inFlight--; + return "ok"; + }; + const summary = await runLoad(call, { concurrency: 4, requests: 23 }); + expect(summary.requests).toBe(23); + expect(summary.ok).toBe(23); + expect(summary.requestLimit).toBe(23); + expect(peak).toBe(4); + expect(summary.errorRate).toBe(0); + expect(summary.latencyMs).not.toBeNull(); + expect(summary.throughput).toBeGreaterThan(0); + }); + + it("classifies ok, tool errors and thrown errors", async () => { + let n = 0; + const call = async (): Promise => { + const i = n++; + if (i % 4 === 1) return "tool_error"; + if (i % 4 === 2) throw new Error("boom"); + // A non-Error rejection is still counted, by its string form. + if (i % 4 === 3) throw "raw"; + return "ok"; + }; + const summary = await runLoad( + call, + { concurrency: 1, requests: 8 }, + fakeClock(), + ); + expect(summary).toMatchObject({ + requests: 8, + ok: 2, + toolErrors: 2, + errors: 4, + errorRate: 0.75, + errorMessages: [ + { message: "boom", count: 2 }, + { message: "raw", count: 2 }, + ], + }); + }); + + it("keeps a rare error visible in the error rate", async () => { + let n = 0; + const call = async (): Promise => + n++ === 0 ? "tool_error" : "ok"; + const summary = await runLoad( + call, + { concurrency: 1, requests: 3000 }, + fakeClock(), + ); + expect(summary.errorRate).toBe(0.0003); + }); + + it("folds error kinds past the cap into one (other) entry", async () => { + let n = 0; + const call = async (): Promise => { + throw new Error(`request ${n++} failed`); + }; + const summary = await runLoad( + call, + { concurrency: 1, requests: MAX_ERROR_KINDS + 3 }, + fakeClock(), + ); + expect(summary.errorMessages).toHaveLength(MAX_ERROR_KINDS + 1); + expect(summary.errorMessages.at(-1)).toEqual({ + message: OTHER_ERRORS, + count: 3, + }); + }); + + it("stops starting calls at the deadline and lets in-flight ones finish", async () => { + const clock = fakeClock(); + const call = async (): Promise => { + clock.advance(100); + return "ok"; + }; + const summary = await runLoad( + call, + { concurrency: 1, durationMs: 1000 }, + clock, + ); + // Calls start at 0, 100, …, 900ms; the one started at 900 ends at 1000. + expect(summary.requests).toBe(10); + expect(summary.durationMs).toBe(1000); + expect(summary.elapsedMs).toBe(1000); + expect(summary.throughput).toBe(10); + expect(summary.latencyMs).toMatchObject({ min: 100, max: 100 }); + }); + + it("spaces starts evenly at the rate and never sleeps past the window", async () => { + const clock = fakeClock(); + const starts: number[] = []; + const call = async (): Promise => { + starts.push(clock.now()); + return "ok"; + }; + const summary = await runLoad( + call, + { concurrency: 1, durationMs: 1000, rate: 4 }, + clock, + ); + expect(starts.map((t) => t - 1000)).toEqual([0, 250, 500, 750]); + expect(summary.rate).toBe(4); + expect(summary.requests).toBe(4); + }); + + it("does not sleep for a slot whose time has already passed", async () => { + const clock = fakeClock(); + const call = async (): Promise => { + clock.advance(500); + return "ok"; + }; + const summary = await runLoad( + call, + { concurrency: 1, requests: 3, rate: 10 }, + clock, + ); + // Each call outlasts its 100ms slot, so the schedule never holds one back. + expect(summary.elapsedMs).toBe(1500); + }); + + it("reports an empty run without dividing by zero", async () => { + const summary = await runLoad( + async () => "ok", + { concurrency: 2, durationMs: 0 }, + fakeClock(), + ); + expect(summary).toMatchObject({ + requests: 0, + errorRate: 0, + throughput: 0, + latencyMs: null, + errorMessages: [], + }); + }); + }); +}); + +describe("--load (CLI)", () => { + const { command, args } = getTestMcpServerCommand(); + const echoLoad = [ + command, + ...args, + "--cli", + "--method", + "tools/call", + "--tool-name", + "echo", + "--tool-arg", + "message=hi", + ]; + + it("prints a summary of the requested number of calls", async () => { + const result = await runCli([ + ...echoLoad, + "--load", + "--load-requests", + "30", + "--load-concurrency", + "5", + ]); + expectCliSuccess(result); + const json = expectValidJson(result); + expect(json).toMatchObject({ + tool: "echo", + concurrency: 5, + requestLimit: 30, + requests: 30, + ok: 30, + toolErrors: 0, + errors: 0, + errorRate: 0, + }); + expect(json.durationMs).toBeUndefined(); + expect(json.latencyMs.p99).toBeGreaterThanOrEqual(json.latencyMs.p50); + }); + + it("wraps the summary in the result envelope with --format json", async () => { + const result = await runCli([ + ...echoLoad, + "--load", + "--load-duration", + "0.2", + "--load-rate", + "20", + "--format", + "json", + ]); + expectCliSuccess(result); + const { result: summary } = expectValidJson(result); + expect(summary).toMatchObject({ tool: "echo", durationMs: 200, rate: 20 }); + expect(summary.requests).toBeGreaterThan(0); + expect(summary.requests).toBeLessThanOrEqual(4); + }); + + it("fails before loading when the tool does not exist", async () => { + const result = await runCli([ + command, + ...args, + "--cli", + "--method", + "tools/call", + "--tool-name", + "no_such_tool", + "--load", + "--load-requests", + "3", + ]); + expect(result.exitCode).toBe(5); + expect(result.stderr).toContain("Tool 'no_such_tool' not found on server."); + }); + + it("requires a tool name", async () => { + const result = await runCli([ + command, + ...args, + "--cli", + "--method", + "tools/call", + "--load", + ]); + expectCliFailure(result); + expect(result.stderr).toContain("--load requires --tool-name."); + }); + + it.each([ + [ + ["--method", "tools/list", "--load"], + "--load requires --method tools/call.", + ], + [ + ["--method", "tools/call", "--tool-name", "echo", "--load", "--app-info"], + "--load cannot be combined with --app-info.", + ], + [ + [ + "--method", + "tools/call", + "--tool-name", + "echo", + "--load", + "--output", + "out.json", + ], + "--load cannot be combined with --output or --output-format", + ], + [ + [ + "--method", + "tools/call", + "--tool-name", + "echo", + "--load-concurrency", + "2", + ], + "--load-concurrency requires --load.", + ], + [ + ["--method", "tools/call", "--tool-name", "echo", "--load-duration", "1"], + "--load-duration requires --load.", + ], + [ + ["--method", "tools/call", "--tool-name", "echo", "--load-requests", "1"], + "--load-requests requires --load.", + ], + [ + ["--method", "tools/call", "--tool-name", "echo", "--load-rate", "1"], + "--load-rate requires --load.", + ], + [ + ["--method", "tools/call", "--load", "--load-concurrency", "0"], + "--load-concurrency must be a positive integer.", + ], + ])("rejects %j", async (flags, message) => { + const result = await runCli([command, ...args, "--cli", ...flags]); + expect(result.exitCode).toBe(1); + expect(result.stderr).toContain(message); + }); + + it("counts a result carrying isError as a tool error", async () => { + let n = 0; + const server = createTestServerHttp({ + serverInfo: createTestServerInfo("load-mixed", "1.0.0"), + tools: [ + { + name: "flaky", + description: "Succeeds and fails in turn", + inputSchema: {}, + // The SDK answers a throwing handler with `isError: true`. + handler: async () => { + if (n++ % 2 === 1) throw new Error("flaky refused"); + return { content: [{ type: "text", text: "flaky" }] }; + }, + }, + ], + }); + try { + await server.start(); + const result = await runCli([ + server.url, + "--cli", + "--transport", + "http", + "--method", + "tools/call", + "--tool-name", + "flaky", + "--load", + "--load-requests", + "8", + "--load-concurrency", + "1", + ]); + expectCliSuccess(result); + expect(expectValidJson(result)).toMatchObject({ + requests: 8, + ok: 4, + toolErrors: 4, + errors: 0, + errorRate: 0.5, + errorMessages: [], + }); + } finally { + await server.stop(); + } + }); + + it("counts calls that throw, and keeps going, when the server dies", async () => { + const crashable = getCrashableTestMcpServerCommand(); + const result = await runCli([ + crashable.command, + ...crashable.args, + "--cli", + "--method", + "tools/call", + "--tool-name", + CRASH_SERVER_TOOL_NAME, + "--load", + "--load-requests", + "3", + "--load-concurrency", + "1", + ]); + // The run completed and reported; the failures are in the summary. + expectCliSuccess(result); + const json = expectValidJson(result); + expect(json).toMatchObject({ + requests: 3, + ok: 0, + toolErrors: 0, + errors: 3, + errorRate: 1, + }); + expect( + json.errorMessages.reduce( + (sum: number, e: { count: number }) => sum + e.count, + 0, + ), + ).toBe(3); + }); + + it("loads a taskSupport: required tool through the task path", async () => { + const server = createTestServerHttp({ + serverInfo: createTestServerInfo("load-tasks", "1.0.0"), + tools: [createSimpleTaskTool("simple_task", 10)], + tasks: { list: true, cancel: true }, + logging: true, + }); + try { + await server.start(); + const result = await runCli([ + server.url, + "--cli", + "--transport", + "http", + "--method", + "tools/call", + "--tool-name", + "simple_task", + "--load", + "--load-requests", + "3", + "--load-concurrency", + "3", + ]); + expectCliSuccess(result); + expect(expectValidJson(result)).toMatchObject({ + tool: "simple_task", + requests: 3, + ok: 3, + }); + } finally { + await server.stop(); + } + }); +}); diff --git a/clients/cli/src/cli.ts b/clients/cli/src/cli.ts index c9fa21593..f2a31adcb 100644 --- a/clients/cli/src/cli.ts +++ b/clients/cli/src/cli.ts @@ -57,6 +57,13 @@ import { import { getStateFilePath } from "@inspector/core/auth/node/storage-node.js"; import { SecretFileLockHeldError } from "@inspector/core/auth/node/secret-store.js"; import { consumeMethodOutcome } from "./handlers/consume-outcome.js"; +import { + positiveIntegerParser, + positiveNumberParser, + resolveLoadOptions, + runToolLoad, + type LoadOptions, +} from "@inspector/core/cli/handlers/load-test.js"; import { runMethod } from "@inspector/core/cli/handlers/run-method.js"; import { isOneShotMethod, @@ -137,6 +144,7 @@ async function callMethod( storedAuthOnly: boolean, relogin: boolean, revoke: boolean, + load?: LoadOptions, ): Promise { // Clear after parse-time validation so a bad flag combo never deletes store // entries. Deletes the shared URL-keyed OAuth entry (not "ignore for this run"). @@ -262,6 +270,25 @@ async function callMethod( { storedAuthOnly, autoOpenControl, quiet: args.quiet }, ); + // `--load` (#2439): many calls of one tool, then a single summary, in + // place of the one result a plain `tools/call` prints. + if (load) { + const summary = await withCliAuthRecoveryRetry( + inspectorClient, + serverConfig, + redirectUrlProvider, + callbackUrlConfig, + serverSettings, + () => runToolLoad(inspectorClient, args, load), + { storedAuthOnly, autoOpenControl, quiet: args.quiet }, + ); + await writeFormattedResult( + summary, + args.format === "json" ? "json" : "text", + ); + return; + } + const outcome = await withCliAuthRecoveryRetry( inspectorClient, serverConfig, @@ -789,6 +816,7 @@ type ParseResult = storedAuthOnly?: boolean; relogin?: boolean; revoke?: boolean; + load?: LoadOptions; } // Short-circuit modes (`--list-stored-auth`, `--print-handoff`) do their own // output and need no server connection; runCli returns immediately. @@ -997,6 +1025,30 @@ async function parseArgs(argv?: string[]): Promise { "--require-digests", 'With --verify: exit 9 when a skill advertises no digests (resources: "dynamic"), instead of reporting it as unverifiable and exiting 0.', ) + .option( + "--load", + "Load-test one tool: call it repeatedly and concurrently, then print a latency and error-rate summary instead of a result. Use with --method tools/call --tool-name . Bounded by --load-duration (default 10s) and/or --load-requests.", + ) + .option( + "--load-concurrency ", + "With --load: most calls in flight at once (positive integer; default 10).", + positiveIntegerParser("--load-concurrency"), + ) + .option( + "--load-duration ", + "With --load: stop starting calls after this many seconds; in-flight calls still finish (default 10 when --load-requests is not set).", + positiveNumberParser("--load-duration"), + ) + .option( + "--load-requests ", + "With --load: stop after starting this many calls (positive integer).", + positiveIntegerParser("--load-requests"), + ) + .option( + "--load-rate ", + "With --load: start at most this many calls per second, spread evenly (positive number; default unlimited).", + positiveNumberParser("--load-rate"), + ) .option( "--connect-timeout ", `Connection timeout in ms (default ${DEFAULT_CONNECT_TIMEOUT_MS} for ad-hoc --server-url / target invocations; 0 = no timeout).`, @@ -1185,6 +1237,11 @@ async function parseArgs(argv?: string[]): Promise { waitForAuth?: number; listStoredAuth?: boolean; printHandoff?: boolean; + load?: boolean; + loadConcurrency?: number; + loadDuration?: number; + loadRequests?: number; + loadRate?: number; }; if (options.relogin) { @@ -1272,6 +1329,34 @@ async function parseArgs(argv?: string[]): Promise { throw new Error("--skill-catalog-max-bytes requires --verify."); } + // `--load` (#2439) is checked here for the same reason as `--strict`: the + // short-circuit returns below would otherwise accept it and do nothing. + const loadBounds = [ + ["--load-concurrency", options.loadConcurrency], + ["--load-duration", options.loadDuration], + ["--load-requests", options.loadRequests], + ["--load-rate", options.loadRate], + ] as const; + for (const [flag, value] of loadBounds) { + if (value !== undefined && !options.load) { + throw new Error(`${flag} requires --load.`); + } + } + if (options.load) { + if (options.method !== "tools/call") { + throw new Error("--load requires --method tools/call."); + } + // Each of these shapes a single result; a load run prints a summary. + if (options.appInfo) { + throw new Error("--load cannot be combined with --app-info."); + } + if (options.output !== undefined || options.outputFormat !== undefined) { + throw new Error( + "--load cannot be combined with --output or --output-format; redirect stdout to keep the summary.", + ); + } + } + // `--advertise-apps` is checked here for the same reason: it shapes the // `initialize` handshake, and the short-circuit paths below never open an // MCP connection, so accepting it there would silently ignore it. @@ -1577,6 +1662,14 @@ async function parseArgs(argv?: string[]): Promise { // Commander's `--no-revoke` defaults this to true; only an explicit // `--no-revoke` makes it false. revoke: options.revoke !== false, + ...(options.load && { + load: resolveLoadOptions({ + concurrency: options.loadConcurrency, + durationSec: options.loadDuration, + requests: options.loadRequests, + rate: options.loadRate, + }), + }), }; } @@ -1611,6 +1704,7 @@ async function runParsedCli(argv?: string[]): Promise { storedAuthOnly, relogin, revoke, + load, } = parsed; const clientConfig = await loadRunnerClientConfig({ clientConfigPath }); // A bad --callback-url / MCP_OAUTH_CALLBACK_URL is a *usage* error, but its @@ -1641,5 +1735,6 @@ async function runParsedCli(argv?: string[]): Promise { storedAuthOnly === true, relogin === true, revoke !== false, + load, ); } diff --git a/core/cli/handlers/load-test.ts b/core/cli/handlers/load-test.ts new file mode 100644 index 000000000..95d089ef4 --- /dev/null +++ b/core/cli/handlers/load-test.ts @@ -0,0 +1,321 @@ +import type { Tool } from "@modelcontextprotocol/client"; +import type { InspectorClient } from "../../mcp/index.js"; +import { ManagedToolsState } from "../../mcp/state/index.js"; +import { CliExitCodeError, EXIT_CODES } from "../error-handler.js"; +import type { MethodArgs } from "./method-types.js"; + +/** + * `--load` (#2439): drive one tool with sustained, concurrent `tools/call` + * traffic over the connection the Inspector already set up — transport, OAuth + * and headers included — and report latency and error rate at the end. + * + * Lives in `core/cli/` rather than the CLI client so `mcpdo` can reuse it + * against a daemon connection without a second copy; only the one-shot CLI + * exposes it today. + * + * All calls share ONE connection, so this measures how a server handles + * concurrent requests on a session, not how it handles many sessions. + */ + +/** Bounds for one load run. Validated by the caller before connecting. */ +export type LoadOptions = { + /** Most calls in flight at once. */ + concurrency: number; + /** Stop starting calls after this long. In-flight calls still finish. */ + durationMs?: number; + /** Stop after this many calls have been started. */ + requests?: number; + /** Cap on calls started per second, across all workers. */ + rate?: number; +}; + +/** How one call ended. `tool_error` is a result carrying `isError: true`. */ +export type LoadCallOutcome = "ok" | "tool_error"; + +/** Latency distribution in milliseconds, nearest-rank percentiles. */ +export type LoadLatency = { + min: number; + mean: number; + p50: number; + p90: number; + p99: number; + max: number; +}; + +export type LoadSummary = { + tool?: string; + concurrency: number; + durationMs?: number; + requestLimit?: number; + rate?: number; + /** Calls that completed, in any outcome. */ + requests: number; + ok: number; + /** Calls that returned a result with `isError: true`. */ + toolErrors: number; + /** Calls that threw: protocol errors, timeouts, transport failures. */ + errors: number; + /** `(toolErrors + errors) / requests` to four places; 0 when none completed. */ + errorRate: number; + elapsedMs: number; + /** Completed calls per second of wall clock. */ + throughput: number; + /** `null` when no call completed. */ + latencyMs: LoadLatency | null; + /** Thrown messages by frequency, at most {@link MAX_ERROR_KINDS} of them. */ + errorMessages: { message: string; count: number }[]; +}; + +export const DEFAULT_LOAD_CONCURRENCY = 10; +/** Applied only when neither a duration nor a request count is given. */ +export const DEFAULT_LOAD_DURATION_MS = 10_000; + +/** + * Commander-style parser for a positive integer flag (`--load-concurrency`, + * `--load-requests`). Plain decimal digits only, so `1.5`, `1e3` and `0x10` + * are refused rather than coerced. + */ +export function positiveIntegerParser(flag: string): (value: string) => number { + return (value: string): number => { + const trimmed = value.trim(); + const n = /^\d+$/.test(trimmed) ? Number(trimmed) : NaN; + if (!Number.isSafeInteger(n) || n <= 0) { + throw new Error(`${flag} must be a positive integer.`); + } + return n; + }; +} + +/** Commander-style parser for a positive, finite number flag. */ +export function positiveNumberParser(flag: string): (value: string) => number { + return (value: string): number => { + const n = value.trim() === "" ? NaN : Number(value); + if (!Number.isFinite(n) || n <= 0) { + throw new Error(`${flag} must be a positive number.`); + } + return n; + }; +} + +/** + * Turn the raw flag values into {@link LoadOptions}, applying the defaults: + * {@link DEFAULT_LOAD_CONCURRENCY} workers, and a + * {@link DEFAULT_LOAD_DURATION_MS} window when no request count bounds the + * run instead. Given both, the run stops at whichever is reached first. + */ +export function resolveLoadOptions(flags: { + concurrency?: number; + durationSec?: number; + requests?: number; + rate?: number; +}): LoadOptions { + const durationMs = + flags.durationSec !== undefined + ? flags.durationSec * 1000 + : flags.requests === undefined + ? DEFAULT_LOAD_DURATION_MS + : undefined; + return { + concurrency: flags.concurrency ?? DEFAULT_LOAD_CONCURRENCY, + ...(durationMs !== undefined && { durationMs }), + ...(flags.requests !== undefined && { requests: flags.requests }), + ...(flags.rate !== undefined && { rate: flags.rate }), + }; +} + +/** Distinct error messages kept; the rest fold into one `(other)` entry. */ +export const MAX_ERROR_KINDS = 10; +export const OTHER_ERRORS = "(other)"; + +/** Injected so tests can drive the clock instead of waiting on it. */ +export type LoadClock = { + now: () => number; + sleep: (ms: number) => Promise; +}; + +const realClock: LoadClock = { + now: () => performance.now(), + sleep: (ms) => new Promise((resolve) => setTimeout(resolve, ms)), +}; + +/** Nearest-rank percentile over an ascending-sorted, non-empty array. */ +export function percentile(sorted: number[], p: number): number { + const rank = Math.ceil((p / 100) * sorted.length); + return sorted[Math.min(Math.max(rank, 1), sorted.length) - 1]!; +} + +function round(n: number, places = 2): number { + const scale = 10 ** places; + return Math.round(n * scale) / scale; +} + +export function summarizeLatency(latencies: number[]): LoadLatency | null { + if (latencies.length === 0) return null; + const sorted = [...latencies].sort((a, b) => a - b); + const total = sorted.reduce((sum, n) => sum + n, 0); + return { + min: round(sorted[0]!), + mean: round(total / sorted.length), + p50: round(percentile(sorted, 50)), + p90: round(percentile(sorted, 90)), + p99: round(percentile(sorted, 99)), + max: round(sorted[sorted.length - 1]!), + }; +} + +/** + * Count thrown messages, keeping the first {@link MAX_ERROR_KINDS} distinct + * ones seen. A failing server tends to fail the same way every time, but one + * that embeds a request id in each message would otherwise grow the map by + * one key per call for the whole run. + */ +class ErrorTally { + private readonly counts = new Map(); + private other = 0; + + add(message: string): void { + const count = this.counts.get(message); + if (count !== undefined) this.counts.set(message, count + 1); + else if (this.counts.size < MAX_ERROR_KINDS) this.counts.set(message, 1); + else this.other++; + } + + entries(): { message: string; count: number }[] { + const list = [...this.counts].map(([message, count]) => ({ + message, + count, + })); + list.sort((a, b) => b.count - a.count); + if (this.other > 0) list.push({ message: OTHER_ERRORS, count: this.other }); + return list; + } +} + +/** + * Run `call` under the given bounds and summarize what happened. Workers pull + * slots from a shared counter, so `concurrency` is a hard cap on calls in + * flight and `requests` a hard cap on calls started. With `rate`, slot `i` + * may not start before `i / rate` seconds into the run — an even schedule + * rather than a burst each second. + * + * A slot whose scheduled start falls past the duration is never started, so a + * rate-limited run does not overshoot its window by sleeping into it. + */ +export async function runLoad( + call: () => Promise, + options: LoadOptions, + clock: LoadClock = realClock, +): Promise { + const { concurrency, durationMs, requests, rate } = options; + const latencies: number[] = []; + const tally = new ErrorTally(); + let ok = 0; + let toolErrors = 0; + let errors = 0; + let started = 0; + + const start = clock.now(); + const deadline = durationMs === undefined ? Infinity : start + durationMs; + + const worker = async (): Promise => { + for (;;) { + if (requests !== undefined && started >= requests) return; + if (clock.now() >= deadline) return; + const slot = started++; + if (rate !== undefined) { + const scheduled = start + (slot * 1000) / rate; + if (scheduled >= deadline) return; + const wait = scheduled - clock.now(); + if (wait > 0) await clock.sleep(wait); + } + const t0 = clock.now(); + try { + const outcome = await call(); + if (outcome === "ok") ok++; + else toolErrors++; + } catch (err) { + errors++; + tally.add(err instanceof Error ? err.message : String(err)); + } + latencies.push(clock.now() - t0); + } + }; + + await Promise.all(Array.from({ length: concurrency }, worker)); + + const elapsedMs = clock.now() - start; + const completed = ok + toolErrors + errors; + return { + concurrency, + ...(durationMs !== undefined && { durationMs }), + ...(requests !== undefined && { requestLimit: requests }), + ...(rate !== undefined && { rate }), + requests: completed, + ok, + toolErrors, + errors, + // Four places, so a rare failure under heavy load (23 in 41k) still shows. + errorRate: + completed === 0 ? 0 : round((toolErrors + errors) / completed, 4), + elapsedMs: round(elapsedMs), + throughput: elapsedMs > 0 ? round((completed * 1000) / elapsedMs) : 0, + latencyMs: summarizeLatency(latencies), + errorMessages: tally.entries(), + }; +} + +/** + * Resolve `args.toolName` once, then load it with {@link runLoad}. The tool + * lookup happens before the clock starts, so a missing tool fails the run + * outright (exit 5, as a plain `tools/call` would) instead of being reported + * as a 100% error rate. + */ +export async function runToolLoad( + inspectorClient: InspectorClient, + args: MethodArgs, + options: LoadOptions, + clock?: LoadClock, +): Promise { + if (!args.toolName) { + throw new Error("--load requires --tool-name."); + } + const toolsState = new ManagedToolsState(inspectorClient); + let found: Tool | undefined; + try { + toolsState.setMetadata(args.metadata); + await toolsState.refresh(); + found = toolsState.getTools().find((t) => t.name === args.toolName); + } finally { + toolsState.destroy(); + } + if (!found) { + throw new CliExitCodeError( + EXIT_CODES.TOOL_ERROR, + `Tool '${args.toolName}' not found on server.`, + { code: "tool_not_found" }, + ); + } + const tool = found; + // Same rule as a single `tools/call` (#2649): a tool that requires task + // support can only be called task-augmented. + const useTask = tool.execution?.taskSupport === "required"; + const call = async (): Promise => { + const params = [ + tool, + args.toolArg ?? {}, + args.metadata, + args.toolMeta, + ] as const; + const invocation = useTask + ? await inspectorClient.callToolStream(...params) + : await inspectorClient.callTool(...params); + /* v8 ignore next 3 -- defensive: `ToolCallInvocation.result` is nullable, + but both call paths throw on failure rather than return a null result. */ + if (invocation.result === null) { + throw new Error(invocation.error ?? "Tool call failed"); + } + return invocation.result.isError === true ? "tool_error" : "ok"; + }; + const summary = await runLoad(call, options, clock); + return { tool: tool.name, ...summary }; +} From 9649799a7379e5497bc67988bcadda1b9ee0fb83 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 01:43:43 -0400 Subject: [PATCH 31/44] fix(cli): drop missed --load-rate slots and recheck the deadline after a wait The rate limiter scheduled slot i at start + i/rate, so a stall (one slow call at concurrency 1) left every missed slot owed and they all started at once afterwards. It now keys off the last slot actually handed out, so a missed slot is dropped and starts stay at least 1/rate apart. The deadline is also checked again after the rate wait, since a timer that wakes late under load would otherwise start a call past --load-duration. --load is now rejected with --list-stored-auth and --print-handoff, which return before the load runs. The README notes that a load run does not prompt for OAuth step-up. The engine tests' fake clock gains a timer queue so concurrent sleepers wake in order, as real timers do. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/cli/README.md | 13 ++- clients/cli/__tests__/load-test.test.ts | 103 ++++++++++++++++++++++-- clients/cli/src/cli.ts | 5 ++ core/cli/handlers/load-test.ts | 25 ++++-- 4 files changed, 130 insertions(+), 16 deletions(-) diff --git a/clients/cli/README.md b/clients/cli/README.md index efda7034d..dcd96363b 100644 --- a/clients/cli/README.md +++ b/clients/cli/README.md @@ -341,18 +341,25 @@ is printed on stdout (wrapped in `{ "result": … }` with `--format json`): sub-millisecond stdio echo. - **Bounds.** `--load-duration` stops *starting* calls; anything in flight is allowed to finish, so `elapsedMs` can exceed the window by up to one call. - With `--load-rate`, a call whose slot falls past the window is never started. + With `--load-rate`, starts are at least `1/rate` seconds apart, and a slot + nobody was free to take is dropped rather than made up later — a stall is + never followed by a catch-up burst. A slot that falls past the window is + never started. **All calls share one connection**, so this measures how a server copes with concurrent requests on a single session — not with many sessions. A slow call is bounded by the server's request timeout (`requestTimeout` in the catalog/config file). +A load run does not prompt for OAuth step-up: a tool that needs scopes the +session lacks fails every call, and the challenge shows in `errorMessages`. +Run one plain `tools/call` of the tool first to grant them, then load it. + The tool is looked up once before the clock starts, so a missing tool fails the run with exit `5` (`tool_not_found`) instead of reporting a 100% error rate. Otherwise a completed run exits `0` whatever its error rate — the summary -is the result. `--load` is rejected with any other method and with -`--app-info`, `--output` and `--output-format`, and each `--load-*` flag is +is the result. `--load` is rejected with any other method and with `--app-info`, +`--output`, `--output-format`, `--list-stored-auth` and `--print-handoff`, and each `--load-*` flag is rejected without `--load`. #### Schema portability (`--strict`) diff --git a/clients/cli/__tests__/load-test.test.ts b/clients/cli/__tests__/load-test.test.ts index b59b03cb3..d1955ce7b 100644 --- a/clients/cli/__tests__/load-test.test.ts +++ b/clients/cli/__tests__/load-test.test.ts @@ -29,17 +29,38 @@ import { } from "@inspector/core/cli/handlers/load-test.js"; /** - * A clock that only moves when told to. `sleep` advances it by the requested - * amount, so a rate-limited run completes instantly while still observing - * the schedule it would keep in real time. + * A clock that only moves when told to, with a timer queue: each `sleep` + * wakes at an absolute time, and timers fire one at a time in wake order, + * each on its own macrotask so the woken worker runs (up to its next await) + * before the clock moves again — as real timers behave. A rate-limited run + * therefore completes instantly while keeping the schedule it would in real + * time. */ function fakeClock(): LoadClock & { advance: (ms: number) => void } { let t = 1000; + const timers: { wakeAt: number; resolve: () => void }[] = []; + let pumping = false; + const pump = (): void => { + timers.sort((a, b) => a.wakeAt - b.wakeAt); + const next = timers.shift(); + if (!next) { + pumping = false; + return; + } + t = Math.max(t, next.wakeAt); + next.resolve(); + setImmediate(pump); + }; return { now: () => t, - sleep: async (ms) => { - t += ms; - }, + sleep: (ms) => + new Promise((resolve) => { + timers.push({ wakeAt: t + ms, resolve }); + if (!pumping) { + pumping = true; + setImmediate(pump); + } + }), advance: (ms) => { t += ms; }, @@ -233,6 +254,52 @@ describe("load-test engine", () => { expect(summary.requests).toBe(4); }); + it("drops missed rate slots instead of bursting to catch up", async () => { + const clock = fakeClock(); + const starts: number[] = []; + let n = 0; + const call = async (): Promise => { + starts.push(clock.now() - 1000); + if (n++ === 0) clock.advance(1000); + return "ok"; + }; + await runLoad(call, { concurrency: 1, requests: 5, rate: 10 }, clock); + // The 1s stall forfeits slots 100..900; spacing resumes from 1000. + expect(starts).toEqual([0, 1000, 1100, 1200, 1300]); + }); + + it("keeps concurrent workers spaced at the rate", async () => { + const clock = fakeClock(); + const starts: number[] = []; + const call = async (): Promise => { + starts.push(clock.now() - 1000); + return "ok"; + }; + await runLoad(call, { concurrency: 3, requests: 6, rate: 5 }, clock); + expect([...starts].sort((a, b) => a - b)).toEqual([ + 0, 200, 400, 600, 800, 1000, + ]); + }); + + it("does not start a call when its timer wakes past the deadline", async () => { + const clock = fakeClock(); + // Every timer fires 100ms late. + clock.sleep = async (ms) => clock.advance(ms + 100); + const starts: number[] = []; + const call = async (): Promise => { + starts.push(clock.now() - 1000); + return "ok"; + }; + const summary = await runLoad( + call, + { concurrency: 1, durationMs: 100, rate: 20 }, + clock, + ); + // Slot 50 wakes at 150, past the 100ms window, and is not started. + expect(starts).toEqual([0]); + expect(summary.requests).toBe(1); + }); + it("does not sleep for a slot whose time has already passed", async () => { const clock = fakeClock(); const call = async (): Promise => { @@ -361,6 +428,30 @@ describe("--load (CLI)", () => { ["--method", "tools/call", "--tool-name", "echo", "--load", "--app-info"], "--load cannot be combined with --app-info.", ], + [ + [ + "--method", + "tools/call", + "--tool-name", + "echo", + "--load", + "--list-stored-auth", + ], + "--load cannot be combined with --list-stored-auth or --print-handoff.", + ], + [ + [ + "--method", + "tools/call", + "--tool-name", + "echo", + "--load", + "--print-handoff", + "--server-url", + "http://localhost:1/mcp", + ], + "--load cannot be combined with --list-stored-auth or --print-handoff.", + ], [ [ "--method", diff --git a/clients/cli/src/cli.ts b/clients/cli/src/cli.ts index f2a31adcb..034e38c83 100644 --- a/clients/cli/src/cli.ts +++ b/clients/cli/src/cli.ts @@ -1350,6 +1350,11 @@ async function parseArgs(argv?: string[]): Promise { if (options.appInfo) { throw new Error("--load cannot be combined with --app-info."); } + if (options.listStoredAuth || options.printHandoff) { + throw new Error( + "--load cannot be combined with --list-stored-auth or --print-handoff.", + ); + } if (options.output !== undefined || options.outputFormat !== undefined) { throw new Error( "--load cannot be combined with --output or --output-format; redirect stdout to keep the summary.", diff --git a/core/cli/handlers/load-test.ts b/core/cli/handlers/load-test.ts index 95d089ef4..148cb6d19 100644 --- a/core/cli/handlers/load-test.ts +++ b/core/cli/handlers/load-test.ts @@ -194,12 +194,17 @@ class ErrorTally { /** * Run `call` under the given bounds and summarize what happened. Workers pull * slots from a shared counter, so `concurrency` is a hard cap on calls in - * flight and `requests` a hard cap on calls started. With `rate`, slot `i` - * may not start before `i / rate` seconds into the run — an even schedule - * rather than a burst each second. + * flight and `requests` a hard cap on calls started. With `rate`, starts are + * at least `1 / rate` seconds apart — an even schedule rather than a burst + * each second. * - * A slot whose scheduled start falls past the duration is never started, so a - * rate-limited run does not overshoot its window by sleeping into it. + * The rate limiter keys off the time the last slot was actually handed out, + * not off the run's start: a slot nobody was free to take is dropped rather + * than owed, so a stall (one slow call at concurrency 1) is never followed by + * a catch-up burst against a server that may just be recovering. + * + * A slot scheduled past the duration is never started, and the deadline is + * checked again after the wait, since a timer can wake late under load. */ export async function runLoad( call: () => Promise, @@ -216,17 +221,23 @@ export async function runLoad( const start = clock.now(); const deadline = durationMs === undefined ? Infinity : start + durationMs; + const interval = rate === undefined ? 0 : 1000 / rate; + // Earliest time the next slot may start. Claimed synchronously, so two + // workers can never be handed the same slot. + let nextStart = start; const worker = async (): Promise => { for (;;) { if (requests !== undefined && started >= requests) return; if (clock.now() >= deadline) return; - const slot = started++; + started++; if (rate !== undefined) { - const scheduled = start + (slot * 1000) / rate; + const scheduled = Math.max(nextStart, clock.now()); + nextStart = scheduled + interval; if (scheduled >= deadline) return; const wait = scheduled - clock.now(); if (wait > 0) await clock.sleep(wait); + if (clock.now() >= deadline) return; } const t0 = clock.now(); try { From bc6db3e1e98acbeaf4c597779759e26bb680a949 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 01:57:09 -0400 Subject: [PATCH 32/44] fix(cli): claim a --load-rate slot only after waking Workers reserved their slot before sleeping, so when the event loop stalled and several timers fired late together, every overdue worker started at once and --load-rate's spacing collapsed. A slot is now claimed only by a worker that is awake at its time, and each wake re-checks the bounds; the rest go back to waiting. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/cli/__tests__/load-test.test.ts | 24 +++++++++++- core/cli/handlers/load-test.ts | 51 +++++++++++++++---------- 2 files changed, 53 insertions(+), 22 deletions(-) diff --git a/clients/cli/__tests__/load-test.test.ts b/clients/cli/__tests__/load-test.test.ts index d1955ce7b..3a7c91ab5 100644 --- a/clients/cli/__tests__/load-test.test.ts +++ b/clients/cli/__tests__/load-test.test.ts @@ -36,8 +36,12 @@ import { * therefore completes instantly while keeping the schedule it would in real * time. */ -function fakeClock(): LoadClock & { advance: (ms: number) => void } { +function fakeClock( + options: { lateUntil?: number } = {}, +): LoadClock & { advance: (ms: number) => void } { let t = 1000; + // A stalled event loop: no timer fires before this many ms into the run. + const earliestWake = 1000 + (options.lateUntil ?? 0); const timers: { wakeAt: number; resolve: () => void }[] = []; let pumping = false; const pump = (): void => { @@ -55,7 +59,7 @@ function fakeClock(): LoadClock & { advance: (ms: number) => void } { now: () => t, sleep: (ms) => new Promise((resolve) => { - timers.push({ wakeAt: t + ms, resolve }); + timers.push({ wakeAt: Math.max(t + ms, earliestWake), resolve }); if (!pumping) { pumping = true; setImmediate(pump); @@ -281,6 +285,22 @@ describe("load-test engine", () => { ]); }); + it("keeps the rate when several workers' timers fire late together", async () => { + const clock = fakeClock({ lateUntil: 1000 }); + const starts: number[] = []; + const call = async (): Promise => { + starts.push(clock.now() - 1000); + return "ok"; + }; + await runLoad( + call, + { concurrency: 3, requests: 3, durationMs: 2000, rate: 5 }, + clock, + ); + // Workers 2 and 3 both wake at 1000; only one may start then. + expect(starts).toEqual([0, 1000, 1200]); + }); + it("does not start a call when its timer wakes past the deadline", async () => { const clock = fakeClock(); // Every timer fires 100ms late. diff --git a/core/cli/handlers/load-test.ts b/core/cli/handlers/load-test.ts index 148cb6d19..af35bbf5c 100644 --- a/core/cli/handlers/load-test.ts +++ b/core/cli/handlers/load-test.ts @@ -198,13 +198,11 @@ class ErrorTally { * at least `1 / rate` seconds apart — an even schedule rather than a burst * each second. * - * The rate limiter keys off the time the last slot was actually handed out, - * not off the run's start: a slot nobody was free to take is dropped rather - * than owed, so a stall (one slow call at concurrency 1) is never followed by - * a catch-up burst against a server that may just be recovering. - * - * A slot scheduled past the duration is never started, and the deadline is - * checked again after the wait, since a timer can wake late under load. + * The rate limiter keys off the time the last slot was actually taken, not + * off the run's start: a slot nobody was free to take is dropped rather than + * owed, so a stall — one slow call at concurrency 1, or timers firing late — + * is never followed by a catch-up burst against a server that may just be + * recovering. A slot past the duration is never started. */ export async function runLoad( call: () => Promise, @@ -222,23 +220,36 @@ export async function runLoad( const start = clock.now(); const deadline = durationMs === undefined ? Infinity : start + durationMs; const interval = rate === undefined ? 0 : 1000 / rate; - // Earliest time the next slot may start. Claimed synchronously, so two - // workers can never be handed the same slot. + // Earliest time the next slot may start. let nextStart = start; - const worker = async (): Promise => { + /** + * Wait for a slot and take it, or return false when the run is over. A + * slot is claimed only by a worker that is awake at its time — never + * reserved before a wait — and every wake re-checks the bounds. So if the + * event loop stalls and several workers' timers all fire late at once, the + * first to run takes the slot and the rest go back to waiting, rather than + * starting together. The claim itself is synchronous, so two workers can + * never take the same slot. + */ + const claimSlot = async (): Promise => { for (;;) { - if (requests !== undefined && started >= requests) return; - if (clock.now() >= deadline) return; - started++; - if (rate !== undefined) { - const scheduled = Math.max(nextStart, clock.now()); - nextStart = scheduled + interval; - if (scheduled >= deadline) return; - const wait = scheduled - clock.now(); - if (wait > 0) await clock.sleep(wait); - if (clock.now() >= deadline) return; + if (requests !== undefined && started >= requests) return false; + const now = clock.now(); + if (now >= deadline) return false; + const scheduled = Math.max(nextStart, now); + if (scheduled >= deadline) return false; + if (scheduled <= now) { + started++; + nextStart = now + interval; + return true; } + await clock.sleep(scheduled - now); + } + }; + + const worker = async (): Promise => { + while (await claimSlot()) { const t0 = clock.now(); try { const outcome = await call(); From 4e13034d3a1f6a9b30b3d1aa1c2d683105d14498 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 02:11:58 -0400 Subject: [PATCH 33/44] fix(cli): one timer per --load-rate slot, and hold the average rate Every idle worker armed a timer for the same next slot and all but one re-armed on waking: 9,900 timers for 100 calls at concurrency 100. Workers now queue on a FIFO turnstile and only the head waits on the clock, which brings that to 99. A claim also reset the schedule from the wake time, so ordinary timer lateness accumulated and the run fell short of the rate (176 of 200 calls in a 1s run at 200/s). A wake that is late by less than one interval now keeps the slot grid; only a slot missed entirely is dropped, so there is still no catch-up burst after a stall. The README notes the ~1ms timer ceiling that limits rates in the high hundreds per second. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/cli/README.md | 12 +++-- clients/cli/__tests__/load-test.test.ts | 40 +++++++++++++++-- core/cli/handlers/load-test.ts | 59 ++++++++++++++++--------- 3 files changed, 83 insertions(+), 28 deletions(-) diff --git a/clients/cli/README.md b/clients/cli/README.md index dcd96363b..2d2b11157 100644 --- a/clients/cli/README.md +++ b/clients/cli/README.md @@ -341,10 +341,14 @@ is printed on stdout (wrapped in `{ "result": … }` with `--format json`): sub-millisecond stdio echo. - **Bounds.** `--load-duration` stops *starting* calls; anything in flight is allowed to finish, so `elapsedMs` can exceed the window by up to one call. - With `--load-rate`, starts are at least `1/rate` seconds apart, and a slot - nobody was free to take is dropped rather than made up later — a stall is - never followed by a catch-up burst. A slot that falls past the window is - never started. + With `--load-rate`, starts follow an even `1/rate` grid; a timer that wakes + a little late keeps the grid, so the average rate holds. A slot missed + entirely — every worker busy, or a stalled event loop — is dropped rather + than made up later, so a stall is never followed by a catch-up burst. A + slot that falls past the window is never started. Node's timers resolve to + about a millisecond, so a rate in the high hundreds per second or above is + reached only approximately (1000/s measured ~700 against a local stdio + server); compare `requests` against what you asked for. **All calls share one connection**, so this measures how a server copes with concurrent requests on a single session — not with many sessions. A slow call diff --git a/clients/cli/__tests__/load-test.test.ts b/clients/cli/__tests__/load-test.test.ts index 3a7c91ab5..3d8932bf8 100644 --- a/clients/cli/__tests__/load-test.test.ts +++ b/clients/cli/__tests__/load-test.test.ts @@ -37,13 +37,14 @@ import { * time. */ function fakeClock( - options: { lateUntil?: number } = {}, -): LoadClock & { advance: (ms: number) => void } { + options: { lateUntil?: number; jitter?: number } = {}, +): LoadClock & { advance: (ms: number) => void; sleeps: () => number } { let t = 1000; // A stalled event loop: no timer fires before this many ms into the run. const earliestWake = 1000 + (options.lateUntil ?? 0); const timers: { wakeAt: number; resolve: () => void }[] = []; let pumping = false; + let sleepCount = 0; const pump = (): void => { timers.sort((a, b) => a.wakeAt - b.wakeAt); const next = timers.shift(); @@ -59,7 +60,11 @@ function fakeClock( now: () => t, sleep: (ms) => new Promise((resolve) => { - timers.push({ wakeAt: Math.max(t + ms, earliestWake), resolve }); + sleepCount++; + timers.push({ + wakeAt: Math.max(t + ms + (options.jitter ?? 0), earliestWake), + resolve, + }); if (!pumping) { pumping = true; setImmediate(pump); @@ -68,6 +73,7 @@ function fakeClock( advance: (ms) => { t += ms; }, + sleeps: () => sleepCount, }; } @@ -285,6 +291,34 @@ describe("load-test engine", () => { ]); }); + it("holds the average rate when every timer wakes a little late", async () => { + const clock = fakeClock({ jitter: 30 }); + const starts: number[] = []; + const call = async (): Promise => { + starts.push(clock.now() - 1000); + return "ok"; + }; + await runLoad( + call, + { concurrency: 1, durationMs: 1000, rate: 10 }, + clock, + ); + // 30ms late each time, but on the 100ms grid: ten starts, not eight. + expect(starts).toEqual([0, 130, 230, 330, 430, 530, 630, 730, 830, 930]); + }); + + it("arms one timer per slot however many workers are idle", async () => { + const clock = fakeClock(); + const summary = await runLoad( + async () => "ok", + { concurrency: 100, requests: 100, rate: 1000 }, + clock, + ); + expect(summary.requests).toBe(100); + // Slot 0 starts immediately; each of the other 99 needs one wait. + expect(clock.sleeps()).toBe(99); + }); + it("keeps the rate when several workers' timers fire late together", async () => { const clock = fakeClock({ lateUntil: 1000 }); const starts: number[] = []; diff --git a/core/cli/handlers/load-test.ts b/core/cli/handlers/load-test.ts index af35bbf5c..d02d9c911 100644 --- a/core/cli/handlers/load-test.ts +++ b/core/cli/handlers/load-test.ts @@ -194,9 +194,8 @@ class ErrorTally { /** * Run `call` under the given bounds and summarize what happened. Workers pull * slots from a shared counter, so `concurrency` is a hard cap on calls in - * flight and `requests` a hard cap on calls started. With `rate`, starts are - * at least `1 / rate` seconds apart — an even schedule rather than a burst - * each second. + * flight and `requests` a hard cap on calls started. With `rate`, starts + * follow an even `1 / rate` grid rather than a burst each second. * * The rate limiter keys off the time the last slot was actually taken, not * off the run's start: a slot nobody was free to take is dropped rather than @@ -223,28 +222,46 @@ export async function runLoad( // Earliest time the next slot may start. let nextStart = start; + // FIFO turnstile: workers wanting a slot queue on a promise chain, and only + // the one at the head waits on the clock. Without it, every idle worker + // armed its own timer for the same slot and all but one re-armed on waking + // — at concurrency 100 that is 99 wasted timers per call, enough to make + // the generator the bottleneck at a high --load-rate. + let turnstile: Promise = Promise.resolve(); + /** - * Wait for a slot and take it, or return false when the run is over. A - * slot is claimed only by a worker that is awake at its time — never - * reserved before a wait — and every wake re-checks the bounds. So if the - * event loop stalls and several workers' timers all fire late at once, the - * first to run takes the slot and the rest go back to waiting, rather than - * starting together. The claim itself is synchronous, so two workers can - * never take the same slot. + * Wait for a slot and take it, or return false when the run is over. Only + * the worker at the head of the turnstile is ever waiting on the clock, and + * it claims a slot only once it is awake at that slot's time, re-checking + * the bounds on every wake. So a timer that fires late — a stalled event + * loop — starts one call, never a burst of everything that was due. */ const claimSlot = async (): Promise => { - for (;;) { - if (requests !== undefined && started >= requests) return false; - const now = clock.now(); - if (now >= deadline) return false; - const scheduled = Math.max(nextStart, now); - if (scheduled >= deadline) return false; - if (scheduled <= now) { - started++; - nextStart = now + interval; - return true; + const ahead = turnstile; + let release!: () => void; + turnstile = new Promise((resolve) => (release = resolve)); + await ahead; + try { + for (;;) { + if (requests !== undefined && started >= requests) return false; + const now = clock.now(); + if (now >= deadline) return false; + const scheduled = Math.max(nextStart, now); + if (scheduled >= deadline) return false; + if (scheduled <= now) { + started++; + // A wake that is merely late — timers always are, by a fraction of + // a millisecond — keeps the slot grid, so the average rate holds. + // Only a slot missed entirely (a stall longer than one interval) + // is dropped, and spacing restarts from now: no catch-up burst. + nextStart = + now - nextStart < interval ? nextStart + interval : now + interval; + return true; + } + await clock.sleep(scheduled - now); } - await clock.sleep(scheduled - now); + } finally { + release(); } }; From 721d31426d80adabd0ec9843597944e8b610df97 Mon Sep 17 00:00:00 2001 From: olaservo Date: Tue, 20 Oct 2026 05:58:11 -0700 Subject: [PATCH 34/44] fix(skills): require ttlMs and cacheScope on modern resources/directory/read results (#2740) The stable ext-skills spec (ext-skills#157) makes ReadResourceDirectoryResult a CacheableResult, so a modern (2026-07-28+) result must carry ttlMs and cacheScope, as skills/list and skills/get already do. The Inspector parsed both eras with one permissive schema and reported a modern server omitting them as clean; the test server's fixture sent resultType alone and was itself non-conformant on the modern leg. - core/mcp/skillsSchemas.ts: add ModernDirectoryReadResultSchema and replace the "no modern variant" note with the new citation. - core/mcp/inspectorClient.ts: readResourceDirectory picks the schema from the negotiated era, as listSkills and getSkillResult do. - test-servers/src/skills.ts: readDirectoryPage and DirectoryReadResultShape send and type the full modern envelope. - Tests mirror the #2404 coverage for skills/get: schema unit cases, client era cases, and an integration case on both eras. - docs: test-servers.md and the roadmap no longer describe the old asymmetry. Co-Authored-By: Claude Fable 5.1 Signed-off-by: olaservo --- .../core/mcp/inspectorClient-skills.test.ts | 46 +++++++++ .../src/test/core/mcp/skillsSchemas.test.ts | 99 ++++++++++++++++--- .../mcp/inspectorClient-skills.test.ts | 12 +++ core/mcp/inspectorClient.ts | 17 +++- core/mcp/skillsSchemas.ts | 74 ++++++++------ docs/inspector-roadmap-2026-h2.md | 2 +- docs/test-servers.md | 20 ++-- test-servers/src/skills.ts | 25 +++-- 8 files changed, 218 insertions(+), 77 deletions(-) diff --git a/clients/web/src/test/core/mcp/inspectorClient-skills.test.ts b/clients/web/src/test/core/mcp/inspectorClient-skills.test.ts index bb94a5467..77eb4f0ca 100644 --- a/clients/web/src/test/core/mcp/inspectorClient-skills.test.ts +++ b/clients/web/src/test/core/mcp/inspectorClient-skills.test.ts @@ -327,6 +327,52 @@ describe("InspectorClient skills methods (#2234)", () => { // server; the live check is in the integration suite. const client = makeClient(); internals(client).protocolEra = "modern"; + stubRequest(client, { + resources: [CHILD], + ttlMs: 0, + cacheScope: "public", + }); + declareSkills(client, true); + await expect( + client.readResourceDirectory("skill://demo"), + ).resolves.toMatchObject({ resources: [CHILD] }); + }); + + it("requires the caching attributes on a modern resources/directory/read (#2740)", async () => { + // The stable ext-skills spec makes `ReadResourceDirectoryResult` a + // `CacheableResult`. The SDK codec checks neither attribute for a + // consumer-owned method, so without the modern schema this server would + // read as conforming. + const client = makeClient(); + internals(client).protocolEra = "modern"; + stubRequest(client, { resources: [CHILD] }); + declareSkills(client, true); + await expect( + client.readResourceDirectory("skill://demo"), + ).rejects.toBeDefined(); + }); + + it("returns the modern caching attributes whole — the CLI prints the result as received", async () => { + const client = makeClient(); + internals(client).protocolEra = "modern"; + stubRequest(client, { + resources: [CHILD], + ttlMs: 5, + cacheScope: "private", + }); + declareSkills(client, true); + await expect( + client.readResourceDirectory("skill://demo"), + ).resolves.toEqual({ + resources: [CHILD], + ttlMs: 5, + cacheScope: "private", + }); + }); + + it("does NOT require the caching attributes on a legacy connection", async () => { + // 2026-era attributes a legacy server has no business sending. + const client = makeClient(); stubRequest(client, { resources: [CHILD] }); declareSkills(client, true); await expect( diff --git a/clients/web/src/test/core/mcp/skillsSchemas.test.ts b/clients/web/src/test/core/mcp/skillsSchemas.test.ts index a9bb3c480..b6940910a 100644 --- a/clients/web/src/test/core/mcp/skillsSchemas.test.ts +++ b/clients/web/src/test/core/mcp/skillsSchemas.test.ts @@ -2,6 +2,7 @@ import { describe, it, expect } from "vitest"; import { DIRECTORY_MIME_TYPE, DirectoryReadResultSchema, + ModernDirectoryReadResultSchema, RESOURCES_DIRECTORY_READ_METHOD, DYNAMIC_RESOURCES, GetSkillEnvelopeSchema, @@ -237,30 +238,98 @@ describe("directory read schemas (#2248)", () => { DirectoryReadResultSchema.safeParse({ resources: "nope" }).success, ).toBe(false); }); +}); + +describe("resources/directory/read caching attributes (#2740)", () => { + const CHILD = { + uri: "skill://demo/templates/invoice.md", + name: "invoice.md", + mimeType: "text/markdown", + }; + const CACHE = { ttlMs: 0, cacheScope: "public" } as const; - it("does NOT require the caching attributes, unlike a modern skills/list page", () => { - // The deliberate asymmetry with `ModernListSkillsResultSchema`: SEP-2640 - // states `ttlMs`/`cacheScope` for a modern `skills/list` and says nothing - // of the kind for this method, whose only worked example omits them. - // Requiring them would fail a server that matched the spec's own example. + it("the LEGACY schema accepts a result with the caching attributes and one without", () => { + // They are 2026-era attributes: a legacy server must not be failed for + // their absence, nor for sending them. expect(DirectoryReadResultSchema.safeParse({ resources: [] }).success).toBe( true, ); - expect(ModernListSkillsResultSchema.safeParse({ skills: [] }).success).toBe( - false, - ); + expect( + DirectoryReadResultSchema.safeParse({ resources: [], ...CACHE }).success, + ).toBe(true); }); - it("still accepts the caching attributes when a server sends them", () => { - // Permitted, not mandated — a schema is not the place to reject an extra - // member the spec leaves open. + it("the MODERN schema requires ttlMs and cacheScope", () => { + // The stable ext-skills spec: `ReadResourceDirectoryResult` extends + // `CacheableResult`, so both are REQUIRED, as on `resources/list`. expect( - DirectoryReadResultSchema.safeParse({ + ModernDirectoryReadResultSchema.safeParse({ resources: [] }).success, + ).toBe(false); + expect( + ModernDirectoryReadResultSchema.safeParse({ resources: [], ttlMs: 0 }) + .success, + ).toBe(false); + expect( + ModernDirectoryReadResultSchema.safeParse({ resources: [], - ttlMs: 60, - cacheScope: "public", + cacheScope: "private", }).success, - ).toBe(true); + ).toBe(false); + }); + + it("the MODERN schema accepts the spec's directory-read example as the SDK codec delivers it (#2373)", () => { + // The stable spec's examples carry `ttlMs: 300000, cacheScope: "public"`. + // No `resultType`: the codec lifts it before any caller schema runs. + const parsed = ModernDirectoryReadResultSchema.parse({ + resources: [CHILD], + nextCursor: "7", + ttlMs: 300_000, + cacheScope: "public", + }); + expect(parsed.resources).toEqual([CHILD]); + expect(parsed.nextCursor).toBe("7"); + expect(parsed.ttlMs).toBe(300_000); + expect(parsed.cacheScope).toBe("public"); + }); + + it.each([-1, 0.5])( + "the MODERN schema rejects ttlMs %s — not a non-negative integer", + (ttlMs) => { + expect( + ModernDirectoryReadResultSchema.safeParse({ + resources: [], + ttlMs, + cacheScope: "public", + }).success, + ).toBe(false); + }, + ); + + it("the MODERN schema rejects a cacheScope outside public/private", () => { + expect( + ModernDirectoryReadResultSchema.safeParse({ + resources: [], + ttlMs: 0, + cacheScope: "shared", + }).success, + ).toBe(false); + }); + + it("the MODERN schema still requires the resources page", () => { + expect( + ModernDirectoryReadResultSchema.safeParse({ ...CACHE }).success, + ).toBe(false); + }); + + it("requires the attributes on every modern skills result alike", () => { + // The three modern schemas now agree; the asymmetry the earlier revision + // documented is gone. + expect(ModernListSkillsResultSchema.safeParse({ skills: [] }).success).toBe( + false, + ); + expect( + ModernDirectoryReadResultSchema.safeParse({ resources: [] }).success, + ).toBe(false); }); }); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-skills.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-skills.test.ts index e8322365a..aeab38bd5 100644 --- a/clients/web/src/test/integration/mcp/inspectorClient-skills.test.ts +++ b/clients/web/src/test/integration/mcp/inspectorClient-skills.test.ts @@ -326,6 +326,18 @@ describe("Skills extension over a real transport (#2234)", () => { ]); }); + it("carries the caching attributes on resources/directory/read (#2740)", async () => { + const started = await startSkillsServer(modern); + const connected = await connect(started.url, modern); + // On the modern leg `readResourceDirectory` selects + // `ModernDirectoryReadResultSchema`, so this resolving is itself the + // requirement; the fixture stamps both eras alike. + const page = await connected.readResourceDirectory( + "skill://data-analysis", + ); + expect(page).toMatchObject({ ttlMs: 0, cacheScope: "public" }); + }); + it("lists a dynamic skill's files, which is what the method is for", async () => { // `dynamic-report` advertises no manifest, so a directory read is the // only way its files are discoverable at all — the case SEP-2640 says diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index 5d535036a..5c918cd46 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -182,6 +182,7 @@ import { import { buildClientExtensions } from "./extensions.js"; import { DirectoryReadResultSchema, + ModernDirectoryReadResultSchema, GetSkillEnvelopeSchema, ModernGetSkillEnvelopeSchema, ListSkillsResultSchema, @@ -5056,16 +5057,22 @@ export class InspectorClient extends InspectorClientEventTarget { // for page one. ...(cursor !== undefined ? { cursor } : {}), }; - // One schema for both eras (#2373): the SDK codec enforces and lifts - // `resultType` on a modern connection, and SEP-2640 requires no caching - // attributes of this method — see `skillsSchemas.ts` for why that differs - // from `skills/list`. + // Era-aware, like `listSkills` and `getSkillResult` (#2740): the stable + // ext-skills spec makes `ReadResourceDirectoryResult` a `CacheableResult`, + // so a modern result must carry `ttlMs` / `cacheScope`. The SDK codec + // checks and lifts `resultType` (#2373) but, for a consumer-owned method, + // neither caching attribute — so this schema is the only thing that does. + // Legacy stays permissive. The result is returned whole: the CLI prints it + // as received, so the attributes reach its output. + const resultSchema = this.isModernEra() + ? ModernDirectoryReadResultSchema + : DirectoryReadResultSchema; try { return await this.invokeMcpClient( () => this.client!.request( { method: RESOURCES_DIRECTORY_READ_METHOD, params }, - DirectoryReadResultSchema, + resultSchema, this.getRequestOptions(this.progressTokenOf(metadata)), ), { method: RESOURCES_DIRECTORY_READ_METHOD }, diff --git a/core/mcp/skillsSchemas.ts b/core/mcp/skillsSchemas.ts index e0f2f36f5..e915f9100 100644 --- a/core/mcp/skillsSchemas.ts +++ b/core/mcp/skillsSchemas.ts @@ -24,8 +24,9 @@ * digest is a finding, not a parse error to swallow. But a settled shape is * enforced, because silently normalizing one past the checks would defeat the * point: `skills/get` requires its `{ skill }` envelope, and a modern - * `skills/list` or `skills/get` result requires the caching attributes (see - * `ModernListSkillsResultSchema` and `ModernGetSkillEnvelopeSchema`). + * `skills/list`, `skills/get` or `resources/directory/read` result requires + * the caching attributes (see `ModernListSkillsResultSchema`, + * `ModernGetSkillEnvelopeSchema` and `ModernDirectoryReadResultSchema`). */ import { z } from "zod/v4"; @@ -244,46 +245,55 @@ export type GetSkillResult = SkillEntry; export const DirectoryChildSchema = ResourceSchema; /** - * `resources/directory/read` result on **either** era: the directory's direct - * children plus the opaque cursor. On a modern connection this is the shape - * after the SDK codec has checked and removed `resultType`, which is why no - * modern variant exists — see the note below (#2373). + * `resources/directory/read` result on a **legacy** connection: the directory's + * direct children plus the opaque cursor, and nothing required beyond that. * - * `looseObject`, matching every other result schema here: a server that also - * sends the caching attributes is not wrong for doing so, and a schema is not - * the place to reject an extra member. + * Its modern counterpart is {@link ModernDirectoryReadResultSchema}. The + * caching attributes are 2026-era, so the legacy shape stays permissive, + * exactly as {@link ListSkillsResultSchema} and {@link GetSkillEnvelopeSchema} + * do: `looseObject`, so a legacy server that also sends them is not failed for + * doing so. */ export const DirectoryReadResultSchema = z.looseObject({ resources: z.array(DirectoryChildSchema), nextCursor: z.string().optional(), }); +/** + * The result as the SDK decoded it, on either era. Typed from the legacy + * schema, whose caching attributes are optional: the modern schema only + * narrows them to required, so either era's result fits — the same reason + * {@link GetSkillEnvelope} is typed from the legacy envelope. + */ export type DirectoryReadResult = z.infer; -/* - * There is deliberately **no modern variant** of the `resources/directory/read` - * schema, though `skills/list` and `skills/get` each have one (#2373, #2404). +/** + * `resources/directory/read` result on a **modern** (2026-07-28+) connection: + * the page plus `CacheableResult`'s required `ttlMs` / `cacheScope` (#2740). * - * On a modern (2026-07-28+) connection the SDK codec already enforces - * `resultType` — the base-protocol member SEP-2322 puts on every modern result - * — and lifts it off before any caller schema runs. What remains is the - * caching attributes, and only the two `skills/*` results are required to - * carry those: + * SEP-2640 as accepted said nothing about caching attributes on this method, + * and its one worked example carried `resultType` alone — so an earlier + * revision of this module deliberately had no modern variant here, on the + * reading that requiring more than the SEP's own example would fail a + * conforming server. The stable ext-skills specification settles it the way + * it settled `skills/get` (#2404): *"`ReadResourceDirectoryResult` extends + * `CacheableResult`, so `ttlMs` and `cacheScope` are **REQUIRED**, as they + * are on `resources/list`, and carry the same semantics. They describe the + * freshness of the directory listing, not of the held entry"* (ext-skills#157, + * recorded in its `docs/decisions.md`), and its directory-read examples now + * carry `"ttlMs": 300000, "cacheScope": "public"`. * - * - For `skills/list` the SEP states it outright — *"In protocol versions - * 2026-07-28 and later, the result also carries … `ttlMs` and - * `cacheScope`"* — so {@link ModernListSkillsResultSchema} requiring them is - * quoting the spec. - * - For `skills/get` the stable ext-skills specification states that - * `GetSkillResult` extends `CacheableResult`, so - * {@link ModernGetSkillEnvelopeSchema} requires them too. - * - For `resources/directory/read` it states **nothing of the kind**, and its - * one worked example carries `resultType: "complete"` and no caching - * attributes at all. Requiring them would fail a server that matched the - * SEP's own example, the failure direction this module works hardest to - * avoid. + * Same field shapes as {@link ModernListSkillsResultSchema}, and the same + * reason it exists: this is a consumer-owned method, so the SDK codec checks + * and lifts `resultType` (#2373) but checks neither caching attribute, and + * without this a modern server omitting them would be reported as clean. * - * So both eras parse that result with one schema. An earlier revision required - * `resultType` on "modern" variants; since the codec lifts it, they could never - * pass on a real modern connection. + * ⚠️ Picked by `InspectorClient.readResourceDirectory` from the negotiated + * era — a schema cannot know it on its own. */ +export const ModernDirectoryReadResultSchema = DirectoryReadResultSchema.extend( + { + ttlMs: z.int().min(0), + cacheScope: z.enum(["public", "private"]), + }, +); diff --git a/docs/inspector-roadmap-2026-h2.md b/docs/inspector-roadmap-2026-h2.md index 2d0aee923..3e4998c54 100644 --- a/docs/inspector-roadmap-2026-h2.md +++ b/docs/inspector-roadmap-2026-h2.md @@ -194,7 +194,7 @@ where the Inspector is thinnest over the SDK. Watch closely. | Feature | Confidence | Notes | | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Cache hint display** — `ttlMs` / `cacheScope` on the SEP-2549 surfaces (`tools/list`, `prompts/list`, `resources/list`, `resources/templates/list`, `resources/read`), plus modern (2026-07-28+) `skills/list` and `skills/get`, which the stable ext-skills spec requires to carry both fields (both validated as required on modern connections since [#2404](https://github.com/modelcontextprotocol/inspector/issues/2404)); legacy results do not require the fields but are shown when a server sends them, with freshness countdown and "stale" marking | 🟢 | SEP-2549 is Final. The runtime parses the hints everywhere and honors them through the SDK cache for the four `*/list` methods; `resources/read`, `skills/list` and `skills/get` go through plain requests that validate but do not honor them, so this item includes that plumbing as well as the display. | +| **Cache hint display** — `ttlMs` / `cacheScope` on the SEP-2549 surfaces (`tools/list`, `prompts/list`, `resources/list`, `resources/templates/list`, `resources/read`), plus modern (2026-07-28+) `skills/list`, `skills/get` and `resources/directory/read`, which the stable ext-skills spec requires to carry both fields (validated as required on modern connections since [#2404](https://github.com/modelcontextprotocol/inspector/issues/2404) and [#2740](https://github.com/modelcontextprotocol/inspector/issues/2740)); legacy results do not require the fields but are shown when a server sends them, with freshness countdown and "stale" marking | 🟢 | SEP-2549 is Final. The runtime parses the hints everywhere and honors them through the SDK cache for the four `*/list` methods; `resources/read`, `skills/list`, `skills/get` and `resources/directory/read` go through plain requests that validate but do not honor them, so this item includes that plumbing as well as the display. | | **Cache behavior observations** — note a re-fetch of a still-fresh result, and a list that changed inside its declared TTL, as diagnostics rather than errors | 🟢 | Inspector-shaped: nobody else observes both the hint and the reality. `ttlMs` is a freshness hint, so both are compliant. | | **Stateful-tool workflow investigation** — how to help a user carry an SEP-2567-style handle from one tool result into the next call | 🟡 | Replaces the first draft's "session lifecycle lane". The protocol has no concept of a handle (it is ordinary tool data), so a generic view would be inference; investigate before designing. | | **ETag support** — send `If-None-Match`, show 304s and version changes | 🔴 | Watch until a SEP reaches Draft with an SDK impl. | diff --git a/docs/test-servers.md b/docs/test-servers.md index 1b073f760..d3c7bea80 100644 --- a/docs/test-servers.md +++ b/docs/test-servers.md @@ -120,16 +120,16 @@ modern leg builds a fresh server per request, so there is no era to branch on when the handlers are registered, and on the legacy leg they are three extra members no codec inspects. -⚠️ **`resources/directory/read` deliberately carries `resultType` alone.** -SEP-2640 states the caching attributes for a modern `skills/list` in as many -words and says nothing of the kind for this method, whose one worked example -carries `resultType` and nothing else. A fixture sending more than the SEP shows -would make a client that wrongly *required* them look correct, which is the -opposite of what a conformance fixture is for — so `readDirectoryPage` stops -where the spec does. On the client, the SDK's modern codec enforces -`resultType` and lifts it off before the Inspector's own schema runs, so -`DirectoryReadResultSchema` serves both eras and requires no caching attributes -([#2373](https://github.com/modelcontextprotocol/inspector/issues/2373)). +`resources/directory/read` carries the same full envelope. SEP-2640 as accepted +showed `resultType` alone for this method, and the fixture once stopped where +the SEP did; the stable ext-skills spec makes `ReadResourceDirectoryResult` a +`CacheableResult`, so `ttlMs` / `cacheScope` are required there too +([#2740](https://github.com/modelcontextprotocol/inspector/issues/2740)). On +the client, the SDK's modern codec enforces `resultType` and lifts it off +before the Inspector's own schema runs +([#2373](https://github.com/modelcontextprotocol/inspector/issues/2373)), and +`ModernDirectoryReadResultSchema` requires the two caching attributes on a +modern connection, as the `skills/list` and `skills/get` schemas do. It works on **either era**: `skills/list`, `skills/get` and `resources/directory/read` are consumer-owned extension methods that neither diff --git a/test-servers/src/skills.ts b/test-servers/src/skills.ts index eddc7c988..e9228e163 100644 --- a/test-servers/src/skills.ts +++ b/test-servers/src/skills.ts @@ -600,13 +600,11 @@ export function readDirectoryPage( const from = Number.isFinite(start) && start > 0 ? start : 0; const next = from + DIRECTORY_PAGE_SIZE; return { - // `resultType` ONLY — deliberately not the full `MODERN_RESULT_ENVELOPE` - // the two `skills/*` results carry. SEP-2640 requires `ttlMs`/`cacheScope` - // of a modern `skills/list` in as many words and says nothing of the kind - // here, and its one worked example of a directory result carries - // `resultType` alone. A fixture that sent more than the SEP shows would - // make a client that wrongly required them look correct. - resultType: MODERN_RESULT_ENVELOPE.resultType, + // The full envelope, as the two `skills/*` results carry. SEP-2640 as + // accepted showed `resultType` alone here, so this fixture once sent only + // that; the stable ext-skills spec makes `ReadResourceDirectoryResult` a + // `CacheableResult`, with `ttlMs`/`cacheScope` REQUIRED (#2740). + ...MODERN_RESULT_ENVELOPE, resources: children.slice(from, next), ...(next < children.length ? { nextCursor: String(next) } : {}), }; @@ -681,15 +679,14 @@ const GetSkillResultShape = z.object({ }); /** - * The `resources/directory/read` result. Carries `resultType` from the modern - * envelope but **not** `ttlMs` / `cacheScope`: SEP-2640 states those for - * `skills/list` and says nothing about them here, and its one worked example of - * a directory result omits them. The fixture matches the SEP's example so a - * client that requires more than the spec asks for fails against it — which is - * the whole point of a conformance fixture. + * The `resources/directory/read` result: the full modern envelope, like the + * two `skills/*` results. The stable ext-skills spec makes + * `ReadResourceDirectoryResult` a `CacheableResult`, so `ttlMs` / + * `cacheScope` are required and a fixture sending `resultType` alone would be + * non-conformant on the modern leg (#2740). */ const DirectoryReadResultShape = z.object({ - resultType: ModernEnvelopeShape.resultType, + ...ModernEnvelopeShape, resources: z.array( z.object({ uri: z.string(), From 123998321d7da6583b09c972fb1525a7ab8ad55d Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 23:32:51 -0400 Subject: [PATCH 35/44] fix(web): validate the Host header on the production web server The prod Hono server now checks the Host header of every request (`/`, the SPA fallback, static assets and `/api/*`) against an allow-list and answers 403 before serving anything when it names a host that is not ours. Allowed: localhost and *.localhost, IP literals, the bind host, the hosts of ALLOWED_ORIGINS, and the new MCP_INSPECTOR_ALLOWED_HOSTS for reverse proxies that rewrite Host. The port is not compared, and /healthz stays exempt. Semantics follow Vite's server.allowedHosts, which already covers --dev. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/README.md | 15 +- clients/web/server/host-allow-list.ts | 204 +++++++++++++ clients/web/server/server.ts | 15 + clients/web/server/web-server-config.ts | 8 + .../server/host-allow-list.test.ts | 278 ++++++++++++++++++ .../server/server-host-allow-list.test.ts | 158 ++++++++++ .../server/web-server-config.test.ts | 13 + docs/docker.md | 4 +- docs/environment-variables.md | 1 + 9 files changed, 693 insertions(+), 3 deletions(-) create mode 100644 clients/web/server/host-allow-list.ts create mode 100644 clients/web/src/test/integration/server/host-allow-list.test.ts create mode 100644 clients/web/src/test/integration/server/server-host-allow-list.test.ts diff --git a/clients/web/README.md b/clients/web/README.md index 5a16451d7..2bcbe81f4 100644 --- a/clients/web/README.md +++ b/clients/web/README.md @@ -20,6 +20,7 @@ The `server/` directory holds the Node-only backend: - **`resolve-bind-host.ts`** — the shared bind-host guard (refuses an all-interfaces `HOST` unless `DANGEROUSLY_BIND_ALL_INTERFACES`), used by both bind points (`web-server-config.ts` + `vite.config.ts`); see [Host binding & the origin allow-list](#host-binding--the-origin-allow-list). - **`tls.ts`** — resolves the optional `INSPECTOR_TLS_KEY_PATH` / `INSPECTOR_TLS_CERT_PATH` pair both listeners serve `https` with, failing startup on a half-set or unusable pair (see [Serving the web UI over HTTPS](../../docs/tls.md)); **`node-request-bridge.ts`** — converts a dev-server Node request into the Fetch `Request` Hono takes, including the HTTP/2 form Vite serves under TLS. - **`inject-auth-token.ts`** — embeds the API token into the served `index.html` (see [Auth token](#auth-token)). +- **`host-allow-list.ts`** — the `Host`-header allow-list the prod server applies to every route but `/healthz`, refusing a request addressed to a name that is not ours before any page (and so any token) is served (see [The Host allow-list](#the-host-allow-list)). - **`health.ts`** — the unauthenticated `GET /healthz` liveness/readiness probe both backends answer (see [Health check](#health-check)). - **`sandbox-controller.ts`** — the MCP Apps sandbox HTTP server; **`app-origin-controller.ts`** — the dedicated app-origin server for `_meta.ui.domain` (see [MCP App dedicated origins](#mcp-app-dedicated-origins-metauidomain)); **`public-address.ts`** — validates `MCP_SANDBOX_FULL_ADDRESS` / `MCP_APP_ORIGIN_FULL_ADDRESS`, the public addresses those two advertise behind a reverse proxy; **`ensure-web-build.ts`** — builds `dist/` on demand for prod `--web`; **`vite-base-config.ts`** — shared `optimizeDeps` exclusions. - **`browser-externalized-builtin-gate.ts`** — Vite-agnostic build-gate logic that fails `vite build` when a Node built-in reaches the browser bundle (#1769); the thin Vite plugin wiring lives in `vite.config.ts`. It sits under `server/` (rather than `src/`) as the home for Node-only, build-time tooling — it's imported by the Vite config, never by the browser — alongside the other `vite-*` config helpers here. @@ -408,6 +409,18 @@ Both the prod backend (`server/web-server-config.ts`) and the dev Vite server (` The backend's `/api/*` routes also enforce an **origin allow-list** (`allowedOrigins`) as DNS-rebinding protection. When left to default on a loopback host, it expands to all three interchangeable loopback origin forms for the port — `http://localhost:PORT`, `http://127.0.0.1:PORT`, and `http://[::1]:PORT` — because `localhost` resolves to either IPv4 or IPv6 loopback and Node/Vite may bind the IPv6 form, so the browser can legitimately arrive at `http://[::1]:PORT`. Set `ALLOWED_ORIGINS` (comma-separated) to override; entries are canonicalized (`new URL(o).origin`), so a trailing slash / uppercase host / explicit `:80` still match. **Each entry must include the scheme** — `http://localhost:6274`, not `localhost:6274` (a scheme-less value is dropped with a warning). `ALLOWED_ORIGINS` **replaces** the default list (it does not merge), so **list every origin you'll browse from, including the loopback forms** you still want (`http://localhost:PORT`, `http://127.0.0.1:PORT`, `http://[::1]:PORT`) — otherwise local access stops working. A blank `ALLOWED_ORIGINS` does **not** disable the check — it falls back to the default (fail closed); there is no env knob to turn origin validation off. +### The Host allow-list + +The origin allow-list only sees requests that carry an `Origin` header, and a browser sends none on a same-origin `GET` — which is exactly how the page that carries the API token is loaded. So the prod server (`server/host-allow-list.ts`) also checks the **`Host` header** of every request — `/`, the SPA fallback, static assets and `/api/*` — and answers **403** before serving anything when it names a host that is not ours. A page loaded from some other DNS name that has been re-pointed at this machine still sends its own name in `Host`, so it is refused. It is the same defence Vite applies in `--dev` (`server.allowedHosts`), with the same rules: + +- **Always allowed:** `localhost`, any `*.localhost` name, and any IP literal (IPv4, or bracketed IPv6) — an address cannot be re-pointed, so it needs no listing. +- **Also allowed:** the bind host (`HOST`, unless it is a wildcard) and the host of every `ALLOWED_ORIGINS` entry, so setting `ALLOWED_ORIGINS=https://inspector.example.com` admits `Host: inspector.example.com` too. +- **`MCP_INSPECTOR_ALLOWED_HOSTS`** (comma-separated hostnames) admits anything else — chiefly a reverse proxy that **rewrites** `Host` to its upstream name (`proxy_pass http://inspector:6274` sends `Host: inspector` unless the proxy forwards the original). A leading dot admits a domain and its subdomains (`.corp.example`). Entries are bare hostnames: a scheme, port, path or wildcard is warned about and ignored. +- The **port is not compared**, so a remapped port (`-p 127.0.0.1:8080:6274`) or a proxy on `:443` still matches; the name is what a re-pointed page cannot change. A request with no `Host` header is refused. +- `GET /healthz` is exempt — it discloses nothing — so a probe that addresses the container by a service name keeps working. + +`MCP_INSPECTOR_ALLOWED_HOSTS` is read by the prod server only; under `--dev`, Vite's own `server.allowedHosts` check applies instead (see below). + **Running the Inspector behind a `*.localhost` proxy.** A common local-dev shape gives each service a friendly name — `my-api.localhost`, `my-app.localhost`, `inspector.localhost` — instead of a set of ports. The Inspector works there, but the origin allow-list does not include those names by default, so set it explicitly: ```sh @@ -426,7 +439,7 @@ Without this you get a confusing failure rather than an obvious one: the page lo The guard blocks only the **wildcard** all-interfaces addresses. Binding a **specific** IP or hostname is allowed with no opt-in — that's a single, deliberate exposure, unlike the wildcard which binds every interface at once (the pattern DNS-rebinding exploits). To serve the Inspector on a LAN or the internet: - **Bind a specific address.** `HOST=192.168.1.50` (a LAN IP) or a public IP works directly: the default origin allow-list follows the bind host, so `allowedOrigins` becomes `http://:PORT` and a browser hitting that address is accepted with no extra config. (The host is canonicalized the way a browser is — so `HOST=127.1` advertises `http://127.0.0.1:PORT`, an IPv6 bind host is bracketed as `http://[2001:db8::1]:PORT`, and the port is dropped when it's the http default `:80` — matching what the browser sends.) -- **Behind TLS or a reverse proxy**, the browser's `Origin` becomes the public origin (e.g. `https://inspector.example.com`, often without a port), which won't match the auto-derived `http://:PORT`. Set `ALLOWED_ORIGINS` to the real public origin(s): `ALLOWED_ORIGINS=https://inspector.example.com`. +- **Behind TLS or a reverse proxy**, the browser's `Origin` becomes the public origin (e.g. `https://inspector.example.com`, often without a port), which won't match the auto-derived `http://:PORT`. Set `ALLOWED_ORIGINS` to the real public origin(s): `ALLOWED_ORIGINS=https://inspector.example.com` That also admits `Host: inspector.example.com` to the [Host allow-list](#the-host-allow-list). If the proxy **rewrites** `Host` to some other name (its upstream's), every page load answers 403 until you add that name: `MCP_INSPECTOR_ALLOWED_HOSTS=inspector`, or configure the proxy to forward the original `Host` (nginx: `proxy_set_header Host $host;`). - **Using the `0.0.0.0` wildcard** (opt-in via `DANGEROUSLY_BIND_ALL_INTERFACES=true`, as the Docker image does): a wildcard bind also serves loopback, so the default allow-list is the loopback trio plus the canonical wildcard origins (`http://0.0.0.0:PORT`, `http://[::]:PORT`), and **local access works out of the box** — `docker run -p 127.0.0.1:6274:6274` browsed at `http://localhost:6274` connects with no extra config. Reaching it at a **non-loopback** address (a LAN IP, a public hostname) still needs `ALLOWED_ORIGINS` — but since that **replaces** the default, keep the loopback forms in the list if you also browse locally: `ALLOWED_ORIGINS=http://localhost:PORT,http://127.0.0.1:PORT,http://192.168.1.50:PORT,https://inspector.example.com`. The bind-host guard and the `ALLOWED_ORIGINS` allow-list apply to both the prod server and `--dev`. Note that in **`--dev`** the Vite dev server _additionally_ enforces its own `server.allowedHosts` Host-header check, whose default accepts loopback and IP-literal hosts. The host you **bind** is auto-allowed (Vite adds the resolved `server.host` — which this config sets from `HOST` — to the allow-list), so `HOST=` works out of the box under `--dev` too. What needs an explicit `server.allowedHosts` entry is reaching the dev server at a **different** name than the one bound — e.g. a wildcard bind reached by hostname, or a reverse-proxy domain. For those, prefer the prod server (`mcp-inspector --web`) or add the host to `server.allowedHosts`. diff --git a/clients/web/server/host-allow-list.ts b/clients/web/server/host-allow-list.ts new file mode 100644 index 000000000..9c6479dc8 --- /dev/null +++ b/clients/web/server/host-allow-list.ts @@ -0,0 +1,204 @@ +/** + * The `Host`-header allow-list the production web server applies to every + * request it answers. + * + * Why it exists: the origin allow-list on `/api/*` only sees requests that + * carry an `Origin` header, and a browser sends none on a same-origin `GET`. + * So a page on an attacker-controlled DNS name that is later re-pointed at + * this machine's address (DNS rebinding) is *same-origin* with the Inspector + * from the browser's point of view: it could `GET /`, read the API token + * `index.html` carries, and replay it against `/api/*`. What such a page + * cannot change is the `Host` header — the browser always sends the name the + * page was loaded from — so rejecting a `Host` that is not one of ours closes + * the whole chain before any HTML (and therefore any token) is served. This is + * the same defence Vite applies in `--dev` (`server.allowedHosts`), whose + * semantics this module follows. + * + * Allowed, compared case-insensitively and ignoring a single trailing dot: + * + * - `localhost` and any `*.localhost` name (RFC 6761: they always resolve to + * loopback, so no attacker can own one); + * - any IPv4 literal and any bracketed IPv6 literal — a rebinding attack needs + * a DNS name, so an address in `Host` cannot be one; + * - the bind host (`HOST`), when it is a specific name rather than a wildcard; + * - the host of every `allowedOrigins` entry, since an origin the API accepts + * is by definition a name the browser is meant to reach us at; + * - every entry of {@link ALLOWED_HOSTS_ENV}, for a reverse proxy that + * rewrites `Host` to a name none of the above covers. An entry starting with + * `.` (`.example.com`) also admits that domain and all of its subdomains, + * matching Vite's convention. + * + * **The port is not compared.** A rebinding page is rejected on its name + * alone, and the port the browser sends is legitimately different from the one + * bound whenever a port is remapped (`docker run -p 127.0.0.1:8080:6274`) or a + * proxy terminates on `:443`. It is still parsed, so a malformed `Host` is + * refused rather than guessed at. + * + * A request with **no** `Host` is refused: HTTP/1.1 requires it, and a client + * that omits it is not a browser this check needs to accommodate. + */ + +import { isIPv4, isIPv6 } from "node:net"; +import type { Context, MiddlewareHandler, Next } from "hono"; +import { + canonicalUrlHost, + isAllInterfacesHost, +} from "../../../core/node/hostUrl.ts"; + +/** Comma-separated extra hostnames the web server accepts in `Host`. */ +export const ALLOWED_HOSTS_ENV = "MCP_INSPECTOR_ALLOWED_HOSTS"; + +/** + * Normalize a hostname for comparison: trim, lowercase, and drop one trailing + * root dot (`localhost.` and `example.com.` name the same host as without it). + */ +function normalizeHostname(hostname: string): string { + return hostname.trim().toLowerCase().replace(/\.$/, ""); +} + +/** + * Extract the hostname from a `Host` header value (`name`, `name:port`, + * `[v6]`, `[v6]:port`), lowercased with any trailing dot removed. A bracketed + * IPv6 literal keeps its brackets. Returns `null` for a value that is not a + * well-formed host — an unterminated bracket, a bracketed non-IPv6 value, a + * non-numeric or out-of-range port, or an empty name. + */ +export function parseHostHeader(value: string): string | null { + const raw = value.trim().toLowerCase(); + let hostname: string; + let port: string | undefined; + if (raw.startsWith("[")) { + const end = raw.indexOf("]"); + if (end < 0) return null; + hostname = raw.slice(0, end + 1); + if (!isIPv6(hostname.slice(1, -1))) return null; + const rest = raw.slice(end + 1); + if (rest !== "") { + if (!rest.startsWith(":")) return null; + port = rest.slice(1); + } + } else { + const colon = raw.indexOf(":"); + hostname = colon < 0 ? raw : raw.slice(0, colon); + port = colon < 0 ? undefined : raw.slice(colon + 1); + } + if (port !== undefined) { + if (!/^\d{1,5}$/.test(port) || Number(port) > 65535) return null; + } + const name = hostname.startsWith("[") + ? hostname + : normalizeHostname(hostname); + return name === "" ? null : name; +} + +/** + * Parse {@link ALLOWED_HOSTS_ENV}: comma-separated hostnames, each optionally + * prefixed with `.` to admit a whole domain. Entries are lowercased and lose a + * trailing dot. An entry that is not a bare hostname — it carries a scheme, a + * port, a path, a wildcard or whitespace — is warned about and dropped, since + * it could never equal the hostname parsed out of a `Host` header and keeping + * it would leave the user believing a proxy name was admitted when it is not. + */ +export function parseAllowedHostsEnv(value: string | undefined): string[] { + if (!value) return []; + const out: string[] = []; + for (const entry of value.split(",")) { + const trimmed = entry.trim(); + if (trimmed === "") continue; + if (/[\s/:*@]/.test(trimmed) || trimmed === ".") { + console.warn( + `Ignoring invalid ${ALLOWED_HOSTS_ENV} entry: ${trimmed} (list bare hostnames, e.g. inspector.example.com or .example.com)`, + ); + continue; + } + out.push(normalizeHostname(trimmed)); + } + return out; +} + +export interface HostAllowListOptions { + /** The host the server binds (`HOST`). A wildcard adds nothing. */ + bindHost: string; + /** The API origin allow-list; each entry's hostname is admitted. */ + allowedOrigins: readonly string[]; + /** Extra hostnames, typically from {@link ALLOWED_HOSTS_ENV}. */ + extraHosts?: readonly string[]; +} + +/** + * The configured (non-implicit) hostnames a server accepts in `Host`, beyond + * the always-allowed loopback names and IP literals. Exported for tests and + * for anyone who needs to explain a rejection. + */ +export function configuredAllowedHosts( + options: HostAllowListOptions, +): string[] { + const hosts = new Set(); + if (!isAllInterfacesHost(options.bindHost)) { + hosts.add(normalizeHostname(canonicalUrlHost(options.bindHost))); + } + for (const origin of options.allowedOrigins) { + try { + const { hostname } = new URL(origin); + if (hostname !== "") hosts.add(normalizeHostname(hostname)); + } catch { + // The list is canonicalized before it gets here; an entry that still + // will not parse contributes no host rather than failing the server. + } + } + for (const extra of options.extraHosts ?? []) { + hosts.add(normalizeHostname(extra)); + } + return [...hosts]; +} + +/** + * Build the predicate that decides whether a `Host` header value is one this + * server answers. See the module comment for the rules. + */ +export function createHostAllowList( + options: HostAllowListOptions, +): (hostHeader: string | undefined) => boolean { + const configured = configuredAllowedHosts(options); + return (hostHeader) => { + if (hostHeader === undefined) return false; + const hostname = parseHostHeader(hostHeader); + if (hostname === null) return false; + if (hostname.startsWith("[")) return true; // parse validated it as IPv6 + if (isIPv4(hostname)) return true; + if (hostname === "localhost" || hostname.endsWith(".localhost")) { + return true; + } + for (const allowed of configured) { + if (allowed === hostname) return true; + if ( + allowed.startsWith(".") && + (allowed.slice(1) === hostname || hostname.endsWith(allowed)) + ) { + return true; + } + } + return false; + }; +} + +/** + * Hono middleware rejecting any request whose `Host` the allow-list refuses, + * with a plain-text 403 that names the host and the variable that admits it. + * Registered ahead of every route that serves HTML or the API. + */ +export function hostAllowListMiddleware( + options: HostAllowListOptions, +): MiddlewareHandler { + const isAllowed = createHostAllowList(options); + return async (c: Context, next: Next) => { + const host = c.req.header("host"); + if (!isAllowed(host)) { + return c.text( + `Forbidden: Host "${host ?? ""}" is not allowed. To serve the Inspector under this name, add it to ${ALLOWED_HOSTS_ENV} (or to ALLOWED_ORIGINS).`, + 403, + ); + } + await next(); + }; +} diff --git a/clients/web/server/server.ts b/clients/web/server/server.ts index cc0080d29..7c3cf4ea4 100644 --- a/clients/web/server/server.ts +++ b/clients/web/server/server.ts @@ -22,6 +22,7 @@ import { } from "./app-origin-controller.js"; import { injectAuthToken } from "./inject-auth-token.js"; import { HEALTH_PATH, healthResponse } from "./health.js"; +import { hostAllowListMiddleware } from "./host-allow-list.js"; import type { WebServerConfig } from "./web-server-config.js"; import { getSecretStorageInfo } from "../../../core/auth/node/secret-store-selection.ts"; import { @@ -102,6 +103,20 @@ export async function startHonoServer( // what it does and does not disclose. Hono serves HEAD from this GET route. app.get(HEALTH_PATH, (c) => healthResponse(c.req.method)); + // Refuse a request addressed to a `Host` that is not ours before anything + // below can answer it — `/`, the SPA fallback (both carry the API token), + // static assets and `/api/*` alike. Registered after the health route, which + // returns without calling `next()` and so stays reachable to a probe under + // any name; it discloses nothing. See host-allow-list.ts for the rules. + app.use( + "*", + hostAllowListMiddleware({ + bindHost: config.hostname, + allowedOrigins: config.allowedOrigins, + extraHosts: config.allowedHosts, + }), + ); + app.use("/api/*", async (c) => { return apiApp.fetch(c.req.raw); }); diff --git a/clients/web/server/web-server-config.ts b/clients/web/server/web-server-config.ts index 695fb0584..ed0dc1755 100644 --- a/clients/web/server/web-server-config.ts +++ b/clients/web/server/web-server-config.ts @@ -33,6 +33,7 @@ import { isAllInterfacesHost, } from "../../../core/node/hostUrl.ts"; import { resolveTlsConfig, webScheme, type TlsConfig } from "./tls.js"; +import { ALLOWED_HOSTS_ENV, parseAllowedHostsEnv } from "./host-allow-list.js"; // The single-source Inspector version (root package.json), read once at load. // The browser can't read the filesystem the way the CLI/TUI do, so the backend @@ -72,6 +73,12 @@ export interface WebServerConfig { initialServers: MCPConfig | null; storageDir: string | undefined; allowedOrigins: string[]; + /** + * Extra hostnames the prod server accepts in the `Host` header + * (`MCP_INSPECTOR_ALLOWED_HOSTS`), beyond loopback names, IP literals, the + * bind host and the hosts of {@link allowedOrigins}. See `host-allow-list.ts`. + */ + allowedHosts?: string[]; /** Sandbox port (0 = dynamic). */ sandboxPort: number; sandboxHost: string; @@ -552,6 +559,7 @@ export function buildWebServerConfig( initialServers, storageDir: process.env.MCP_STORAGE_DIR, allowedOrigins, + allowedHosts: parseAllowedHostsEnv(process.env[ALLOWED_HOSTS_ENV]), sandboxPort, sandboxHost: hostname, appOriginPort, diff --git a/clients/web/src/test/integration/server/host-allow-list.test.ts b/clients/web/src/test/integration/server/host-allow-list.test.ts new file mode 100644 index 000000000..af62141fd --- /dev/null +++ b/clients/web/src/test/integration/server/host-allow-list.test.ts @@ -0,0 +1,278 @@ +import { describe, it, expect, vi, afterEach } from "vitest"; +import { Hono } from "hono"; +import { + ALLOWED_HOSTS_ENV, + configuredAllowedHosts, + createHostAllowList, + hostAllowListMiddleware, + parseAllowedHostsEnv, + parseHostHeader, +} from "../../../../server/host-allow-list.js"; + +const LOOPBACK_ORIGINS = [ + "http://localhost:6274", + "http://127.0.0.1:6274", + "http://[::1]:6274", +]; + +describe("parseHostHeader", () => { + it.each([ + ["localhost", "localhost"], + ["localhost:6274", "localhost"], + ["LocalHost:6274", "localhost"], + [" example.com ", "example.com"], + ["example.com.", "example.com"], + ["example.com.:443", "example.com"], + ["127.0.0.1:6274", "127.0.0.1"], + ["[::1]", "[::1]"], + ["[::1]:6274", "[::1]"], + ["[2001:DB8::1]:80", "[2001:db8::1]"], + ["host:0", "host"], + ["host:65535", "host"], + ])("parses %j as %j", (value, expected) => { + expect(parseHostHeader(value)).toBe(expected); + }); + + it.each([ + "", + " ", + ":6274", + ".", + "[::1", + "[not-ipv6]", + "[::1]6274", + "[::1]:", + "[::1]:abc", + "host:", + "host:abc", + "host:65536", + "host:123456", + "host:80:80", + "::1", + ])("rejects %j as malformed", (value) => { + expect(parseHostHeader(value)).toBeNull(); + }); +}); + +describe("parseAllowedHostsEnv", () => { + afterEach(() => { + vi.restoreAllMocks(); + }); + + it("is empty when unset or blank", () => { + expect(parseAllowedHostsEnv(undefined)).toEqual([]); + expect(parseAllowedHostsEnv("")).toEqual([]); + expect(parseAllowedHostsEnv(" , ,")).toEqual([]); + }); + + it("trims, lowercases and drops a trailing dot", () => { + expect( + parseAllowedHostsEnv(" Inspector.Example.com , .Corp.Example. ,proxy"), + ).toEqual(["inspector.example.com", ".corp.example", "proxy"]); + }); + + it.each([ + "https://inspector.example.com", + "inspector.example.com:6274", + "inspector.example.com/path", + "*.example.com", + "*", + "user@host", + "two words", + ".", + ])("warns about and drops %j", (entry) => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + expect(parseAllowedHostsEnv(`good.example,${entry}`)).toEqual([ + "good.example", + ]); + expect(warn).toHaveBeenCalledTimes(1); + expect(warn.mock.calls[0][0]).toContain(ALLOWED_HOSTS_ENV); + expect(warn.mock.calls[0][0]).toContain(entry); + }); +}); + +describe("configuredAllowedHosts", () => { + it("adds the bind host, the origin hosts and the extras, deduplicated", () => { + expect( + configuredAllowedHosts({ + bindHost: "Inspector.LAN", + allowedOrigins: [ + "https://inspector.example.com", + "http://inspector.lan:6274", + "http://[2001:db8::1]:6274", + ], + extraHosts: ["Proxy.Internal.", ".corp.example"], + }), + ).toEqual([ + "inspector.lan", + "inspector.example.com", + "[2001:db8::1]", + "proxy.internal", + ".corp.example", + ]); + }); + + it("adds nothing for a wildcard bind", () => { + expect( + configuredAllowedHosts({ bindHost: "0.0.0.0", allowedOrigins: [] }), + ).toEqual([]); + expect( + configuredAllowedHosts({ bindHost: "::", allowedOrigins: [] }), + ).toEqual([]); + }); + + it("canonicalizes an IPv6 bind host to its bracketed form", () => { + expect( + configuredAllowedHosts({ bindHost: "::1", allowedOrigins: [] }), + ).toEqual(["[::1]"]); + }); + + it("skips an origin that does not parse or has no host", () => { + expect( + configuredAllowedHosts({ + bindHost: "0.0.0.0", + allowedOrigins: ["not a url", "file:///x"], + }), + ).toEqual([]); + }); +}); + +describe("createHostAllowList", () => { + const loopback = createHostAllowList({ + bindHost: "127.0.0.1", + allowedOrigins: LOOPBACK_ORIGINS, + }); + + it.each([ + "localhost", + "localhost:6274", + "LOCALHOST:6274", + "localhost.:6274", + "inspector.localhost", + "a.b.localhost:8080", + "127.0.0.1:6274", + "127.0.0.1", + "192.168.1.50:6274", + "10.0.0.1:8080", + "[::1]:6274", + "[::1]", + "[2001:db8::1]:6274", + "[::ffff:127.0.0.1]:6274", + ])("allows %j on a loopback bind", (host) => { + expect(loopback(host)).toBe(true); + }); + + it.each([ + "evil.example:6274", + "evil.example", + "localhost.evil.example", + "evillocalhost", + "127.0.0.1.nip.io:6274", + "2130706433:6274", + "", + "[::1", + "localhost:notaport", + ])("rejects %j on a loopback bind", (host) => { + expect(loopback(host)).toBe(false); + }); + + it("rejects a missing Host header", () => { + expect(loopback(undefined)).toBe(false); + }); + + it("does not compare the port (a remapped port still matches)", () => { + expect(loopback("localhost:8080")).toBe(true); + const named = createHostAllowList({ + bindHost: "127.0.0.1", + allowedOrigins: ["https://inspector.example.com"], + }); + expect(named("inspector.example.com:9999")).toBe(true); + }); + + it("allows the hosts named by allowedOrigins", () => { + const allow = createHostAllowList({ + bindHost: "0.0.0.0", + allowedOrigins: [...LOOPBACK_ORIGINS, "https://Inspector.Example.com"], + }); + expect(allow("inspector.example.com")).toBe(true); + expect(allow("INSPECTOR.EXAMPLE.COM:443")).toBe(true); + expect(allow("other.example.com")).toBe(false); + }); + + it("allows a specific named bind host", () => { + const allow = createHostAllowList({ + bindHost: "inspector.lan", + allowedOrigins: ["http://inspector.lan:6274"], + }); + expect(allow("inspector.lan:6274")).toBe(true); + const bindOnly = createHostAllowList({ + bindHost: "Inspector.LAN", + allowedOrigins: [], + }); + expect(bindOnly("inspector.lan")).toBe(true); + }); + + it("allows exact extra hosts and leading-dot domains", () => { + const allow = createHostAllowList({ + bindHost: "0.0.0.0", + allowedOrigins: LOOPBACK_ORIGINS, + extraHosts: ["inspector", ".corp.example"], + }); + expect(allow("inspector:6274")).toBe(true); + expect(allow("corp.example")).toBe(true); + expect(allow("a.corp.example:6274")).toBe(true); + expect(allow("a.b.corp.example")).toBe(true); + expect(allow("notcorp.example")).toBe(false); + expect(allow("inspector.evil")).toBe(false); + expect(allow("evil.example")).toBe(false); + }); +}); + +describe("hostAllowListMiddleware", () => { + function app(extraHosts?: string[]) { + const a = new Hono(); + a.use( + "*", + hostAllowListMiddleware({ + bindHost: "127.0.0.1", + allowedOrigins: LOOPBACK_ORIGINS, + extraHosts, + }), + ); + a.get("*", (c) => c.text("served")); + return a; + } + + it("passes an allowed host through to the route", async () => { + const res = await app().request("http://localhost:6274/", { + headers: { host: "localhost:6274" }, + }); + expect(res.status).toBe(200); + expect(await res.text()).toBe("served"); + }); + + it("answers 403 for a disallowed host, naming the escape hatch", async () => { + const res = await app().request("http://evil.example:6274/", { + headers: { host: "evil.example:6274" }, + }); + expect(res.status).toBe(403); + const body = await res.text(); + expect(body).not.toBe("served"); + expect(body).toContain("evil.example:6274"); + expect(body).toContain(ALLOWED_HOSTS_ENV); + }); + + it("answers 403 when the Host header is absent", async () => { + // `app.request` builds a Request with no `host` header unless one is set. + const res = await app().request("http://localhost/"); + expect(res.status).toBe(403); + expect(await res.text()).toContain('Host ""'); + }); + + it("admits a host named in the extra hosts", async () => { + const res = await app(["inspector"]).request("http://inspector/", { + headers: { host: "inspector:6274" }, + }); + expect(res.status).toBe(200); + }); +}); diff --git a/clients/web/src/test/integration/server/server-host-allow-list.test.ts b/clients/web/src/test/integration/server/server-host-allow-list.test.ts new file mode 100644 index 000000000..7d521af1e --- /dev/null +++ b/clients/web/src/test/integration/server/server-host-allow-list.test.ts @@ -0,0 +1,158 @@ +import { describe, it, expect, beforeAll, afterAll } from "vitest"; +import { createServer } from "node:net"; +import { request } from "node:http"; +import { mkdtemp, writeFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startHonoServer } from "../../../../server/server.js"; +import type { WebServerConfig } from "../../../../server/web-server-config.js"; +import type { WebServerHandle } from "../../../../server/types.js"; +import { INSPECTOR_API_TOKEN_GLOBAL } from "../../../../../../core/mcp/remote/constants.js"; + +// See server-token-injection.test.ts for why this pattern. +async function findFreePort(): Promise { + return new Promise((resolve, reject) => { + const srv = createServer(); + srv.unref(); + srv.on("error", reject); + srv.listen(0, "127.0.0.1", () => { + const addr = srv.address(); + if (addr && typeof addr === "object") { + const { port } = addr; + srv.close(() => resolve(port)); + } else { + srv.close(() => reject(new Error("Could not resolve a free port"))); + } + }); + }); +} + +interface RawResponse { + status: number; + body: string; +} + +/** + * Issue a GET over a real socket with an explicit `Host` header — `fetch` + * derives `Host` from the URL, so it cannot send the name a browser would + * after the name it loaded a page from is pointed at this address. + */ +function getWithHost( + port: number, + path: string, + host: string, + headers: Record = {}, +): Promise { + return new Promise((resolve, reject) => { + const req = request( + { + host: "127.0.0.1", + port, + path, + method: "GET", + headers: { ...headers, host }, + }, + (res) => { + let body = ""; + res.setEncoding("utf-8"); + res.on("data", (chunk: string) => { + body += chunk; + }); + res.on("end", () => resolve({ status: res.statusCode ?? 0, body })); + res.on("error", reject); + }, + ); + req.on("error", reject); + req.end(); + }); +} + +const TOKEN = "host-allow-list-token-1234567890"; +const INDEX_HTML = + "Inspector" + + '
'; + +describe("startHonoServer Host header allow-list", () => { + let handle: WebServerHandle; + let port: number; + let staticRoot: string; + + beforeAll(async () => { + staticRoot = await mkdtemp(join(tmpdir(), "inspector-host-")); + await writeFile(join(staticRoot, "index.html"), INDEX_HTML, "utf-8"); + await writeFile(join(staticRoot, "asset.js"), "1;\n", "utf-8"); + + port = await findFreePort(); + const config: WebServerConfig = { + port, + hostname: "127.0.0.1", + authToken: TOKEN, + dangerouslyOmitAuth: false, + initialMcpConfig: null, + mcpConfigPath: undefined, + writable: true, + initialServers: null, + storageDir: undefined, + allowedOrigins: [ + `http://localhost:${port}`, + `http://127.0.0.1:${port}`, + `http://[::1]:${port}`, + ], + allowedHosts: ["proxy.internal"], + sandboxPort: 0, + appOriginPort: 0, + sandboxHost: "127.0.0.1", + logger: undefined, + autoOpen: false, + staticRoot, + }; + handle = await startHonoServer(config); + }); + + afterAll(async () => { + await handle?.close(); + if (staticRoot) await rm(staticRoot, { recursive: true, force: true }); + }); + + it.each(["/", "/oauth/callback", "/asset.js", "/api/config"])( + "refuses %s under a foreign Host without serving the token", + async (path) => { + const res = await getWithHost(port, path, "evil.example:6274", { + "x-mcp-remote-auth": `Bearer ${TOKEN}`, + }); + expect(res.status).toBe(403); + expect(res.body).not.toContain(TOKEN); + expect(res.body).not.toContain(INSPECTOR_API_TOKEN_GLOBAL); + }, + ); + + // Hosts without the bound port: it.each runs before beforeAll picks one, + // and the port is not compared anyway (`:8080` stands in for a remap). + it.each(["localhost", "127.0.0.1", "[::1]", "localhost:8080"])( + "serves the token-injected page under Host %s", + async (host) => { + const res = await getWithHost(port, "/", host); + expect(res.status).toBe(200); + expect(res.body).toContain(TOKEN); + }, + ); + + it("serves the API under a loopback Host", async () => { + const res = await getWithHost(port, "/api/config", `localhost:${port}`, { + "x-mcp-remote-auth": `Bearer ${TOKEN}`, + }); + expect(res.status).toBe(200); + }); + + it("serves a host admitted through allowedHosts", async () => { + const res = await getWithHost(port, "/", "proxy.internal"); + expect(res.status).toBe(200); + expect(res.body).toContain(TOKEN); + }); + + it("keeps the health probe answering under any Host", async () => { + const res = await getWithHost(port, "/healthz", "evil.example"); + expect(res.status).toBe(200); + expect(res.body).toBe('{"status":"ok"}'); + }); +}); diff --git a/clients/web/src/test/integration/server/web-server-config.test.ts b/clients/web/src/test/integration/server/web-server-config.test.ts index 9262ab0e1..545bb9108 100644 --- a/clients/web/src/test/integration/server/web-server-config.test.ts +++ b/clients/web/src/test/integration/server/web-server-config.test.ts @@ -33,6 +33,7 @@ import { // beforeEach and restores it in afterEach so neither the test runner nor // sibling tests see leaked state. const MUTATED_ENV_KEYS = [ + "MCP_INSPECTOR_ALLOWED_HOSTS", API_SERVER_ENV_VARS.AUTH_TOKEN, LEGACY_AUTH_TOKEN_ENV, "CLIENT_PORT", @@ -230,6 +231,18 @@ describe("buildWebServerConfigFromEnv", () => { expect(cfg.authToken).toBe("primary"); }); + it("has no extra allowed hosts by default", () => { + expect(buildWebServerConfigFromEnv().allowedHosts).toEqual([]); + }); + + it("parses MCP_INSPECTOR_ALLOWED_HOSTS into allowedHosts", () => { + process.env.MCP_INSPECTOR_ALLOWED_HOSTS = " Inspector , .corp.example "; + expect(buildWebServerConfigFromEnv().allowedHosts).toEqual([ + "inspector", + ".corp.example", + ]); + }); + it("parses ALLOWED_ORIGINS, trimming entries and filtering empties", () => { process.env.ALLOWED_ORIGINS = "http://a:1, , http://b:2 "; const cfg = buildWebServerConfigFromEnv(); diff --git a/docs/docker.md b/docs/docker.md index 8115d84b7..a3528f4d8 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -28,9 +28,9 @@ docker run --rm -p 127.0.0.1:6274:6274 -p 127.0.0.1:6275:6275 -p 127.0.0.1:6278: Publish each on the **same port number** inside and out. The sandbox URL is handed to the browser via `/api/config` as `http://localhost:/sandbox`, so remapping it (`-p 9000:6275`) advertises a port the browser can't reach; use `-e MCP_SANDBOX_PORT=9000 -p 127.0.0.1:9000:9000` instead. The same holds for the app origin: the URL a published app document is served from is built from the port the container bound, so remap with `-e MCP_APP_ORIGIN_PORT=9001 -p 127.0.0.1:9001:9001` rather than with `-p` alone. -**Behind a reverse proxy or an ingress**, where the browser reaches the listeners at public hostnames rather than at `localhost`, the same-number rule is not enough: the advertised URLs name the container's bind. Set `MCP_SANDBOX_FULL_ADDRESS` (e.g. `https://inspector-sandbox.example.com/sandbox`) and, for apps declaring `_meta.ui.domain`, `MCP_APP_ORIGIN_FULL_ADDRESS` (e.g. `https://inspector-apps.example.com`), route each to its container port, and set `ALLOWED_ORIGINS` to the public Inspector origin. Each listener needs its own hostname or port — an address sharing the Inspector's origin is refused. See [Behind a reverse proxy](../clients/web/README.md#host-binding--the-origin-allow-list) for the full contract. +**Behind a reverse proxy or an ingress**, where the browser reaches the listeners at public hostnames rather than at `localhost`, the same-number rule is not enough: the advertised URLs name the container's bind. Set `MCP_SANDBOX_FULL_ADDRESS` (e.g. `https://inspector-sandbox.example.com/sandbox`) and, for apps declaring `_meta.ui.domain`, `MCP_APP_ORIGIN_FULL_ADDRESS` (e.g. `https://inspector-apps.example.com`), route each to its container port, and set `ALLOWED_ORIGINS` to the public Inspector origin. The web server also refuses a request whose `Host` header is not a name it knows (loopback names, IP literals, the hosts of `ALLOWED_ORIGINS`), so a proxy that forwards the public `Host` needs nothing more; one that rewrites `Host` to the container's service name (`Host: inspector`) needs `-e MCP_INSPECTOR_ALLOWED_HOSTS=inspector`, or every page load answers 403. Each listener needs its own hostname or port — an address sharing the Inspector's origin is refused. See [Behind a reverse proxy](../clients/web/README.md#host-binding--the-origin-allow-list) for the full contract. -**Keep the `127.0.0.1:` prefix on the published port.** A bare `-p 6274:6274` publishes on **every host interface**, putting the Inspector on your local network. The container's `HOST=0.0.0.0` is a separate concern — it governs the _container's_ interfaces, not the host's — so the `DANGEROUSLY_BIND_ALL_INTERFACES` opt-in that guards a wildcard bind outside a container does not cover this. It matters more here than for an ordinary web app: the backend spawns processes on request, `GET /` embeds the API token into the served HTML, and a request arriving with **no** `Origin` header skips the origin allow-list entirely — so for any non-browser client the API token is the only guard. Publishing wider needs a real access-control boundary in front of the Inspector — a reverse proxy that authenticates, an SSH tunnel, a private network. Setting your own `MCP_INSPECTOR_API_TOKEN` does **not** substitute: `GET /` discloses whatever token is in use, so a custom one is harvested exactly as easily as a generated one. +**Keep the `127.0.0.1:` prefix on the published port.** A bare `-p 6274:6274` publishes on **every host interface**, putting the Inspector on your local network. The container's `HOST=0.0.0.0` is a separate concern — it governs the _container's_ interfaces, not the host's — so the `DANGEROUSLY_BIND_ALL_INTERFACES` opt-in that guards a wildcard bind outside a container does not cover this. It matters more here than for an ordinary web app: the backend spawns processes on request, `GET /` embeds the API token into the served HTML, and a request arriving with **no** `Origin` header skips the origin allow-list entirely — so for any non-browser client the API token is the only guard. (A browser page is additionally held to the [Host allow-list](../clients/web/README.md#the-host-allow-list), which refuses a page loaded under a name that is not ours; it does nothing against a client that can reach the port and simply sends `Host: localhost`.) Publishing wider needs a real access-control boundary in front of the Inspector — a reverse proxy that authenticates, an SSH tunnel, a private network. Setting your own `MCP_INSPECTOR_API_TOKEN` does **not** substitute: `GET /` discloses whatever token is in use, so a custom one is harvested exactly as easily as a generated one. **Keeping the servers you add.** The Inspector saves your server list to `$HOME/.mcp-inspector/mcp.json`, which in the image is `/home/node/.mcp-inspector/mcp.json` — inside the container's writable layer, so `--rm` discards it and every run starts with an empty list. Mount a volume there to keep it: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 9387dff4f..106afd1a6 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -20,6 +20,7 @@ These guard the web backend, which spawns processes on request. Read [Host bindi | `HOST` | web, CLI | `127.0.0.1` | Address the web server binds. An all-interfaces host (`0.0.0.0`, `::`, an empty string, and equivalent spellings) is **refused** unless `DANGEROUSLY_BIND_ALL_INTERFACES` is enabled. The CLI reads it only to build its deep link. | | `DANGEROUSLY_BIND_ALL_INTERFACES` | web | off | Opts in to an all-interfaces `HOST`. Only `true` or `1` (case-insensitive) enable it, so `false` reads as off. The Docker image sets it. | | `ALLOWED_ORIGINS` | web | derived from `HOST` | Comma-separated origins allowed to call the API. Unset, the list follows `HOST` at `CLIENT_PORT`: the loopback origins for a loopback host, the loopback origins plus `http://0.0.0.0` and `http://[::]` for an all-interfaces bind, and otherwise only the configured host's own origin (so binding a LAN address does **not** also allow `localhost`). The default origins are `https://` when [TLS](#tls-for-the-web-ui) is on. **Replaces** the default list rather than adding to it, so list every form you browse from. Each entry must include the scheme (`http://localhost:6274`). The same list is the MCP Apps sandbox proxy's embedder allow-list (its `frame-ancestors` header and its referrer check), so a public Inspector origin must be listed here for the Apps tab to render. | +| `MCP_INSPECTOR_ALLOWED_HOSTS` | web | unset | Comma-separated hostnames the prod web server accepts in the `Host` header, on top of the ones it always accepts: `localhost`, `*.localhost`, IP literals, the `HOST` it binds and the hosts of `ALLOWED_ORIGINS`. Any other `Host` is answered with 403 before a page is served. Set it for a reverse proxy that rewrites `Host` to its upstream name. A leading `.` admits a domain and its subdomains (`.corp.example`); an entry with a scheme, port, path or wildcard is ignored with a warning. Not read under `--dev`, where Vite's `server.allowedHosts` applies. See [The Host allow-list](../clients/web/README.md#the-host-allow-list). | ## TLS for the web UI From 4c31c6b7eb0d6215ef41384de57ce357cdd8e017 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 23:39:58 -0400 Subject: [PATCH 36/44] fix(web): restrict Host names to hostname characters, punycode allowed-host entries Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/README.md | 4 +-- clients/web/server/host-allow-list.ts | 28 ++++++++++++++----- .../server/host-allow-list.test.ts | 12 ++++++++ docs/environment-variables.md | 2 +- 4 files changed, 36 insertions(+), 10 deletions(-) diff --git a/clients/web/README.md b/clients/web/README.md index 2bcbe81f4..af5f1bc29 100644 --- a/clients/web/README.md +++ b/clients/web/README.md @@ -419,7 +419,7 @@ The origin allow-list only sees requests that carry an `Origin` header, and a br - The **port is not compared**, so a remapped port (`-p 127.0.0.1:8080:6274`) or a proxy on `:443` still matches; the name is what a re-pointed page cannot change. A request with no `Host` header is refused. - `GET /healthz` is exempt — it discloses nothing — so a probe that addresses the container by a service name keeps working. -`MCP_INSPECTOR_ALLOWED_HOSTS` is read by the prod server only; under `--dev`, Vite's own `server.allowedHosts` check applies instead (see below). +`MCP_INSPECTOR_ALLOWED_HOSTS` is read by the prod server only; under `--dev`, Vite's own `server.allowedHosts` check applies instead (see below) — except over TLS, where Vite skips it because the certificate check already refuses a page loaded under another name. **Running the Inspector behind a `*.localhost` proxy.** A common local-dev shape gives each service a friendly name — `my-api.localhost`, `my-app.localhost`, `inspector.localhost` — instead of a set of ports. The Inspector works there, but the origin allow-list does not include those names by default, so set it explicitly: @@ -439,7 +439,7 @@ Without this you get a confusing failure rather than an obvious one: the page lo The guard blocks only the **wildcard** all-interfaces addresses. Binding a **specific** IP or hostname is allowed with no opt-in — that's a single, deliberate exposure, unlike the wildcard which binds every interface at once (the pattern DNS-rebinding exploits). To serve the Inspector on a LAN or the internet: - **Bind a specific address.** `HOST=192.168.1.50` (a LAN IP) or a public IP works directly: the default origin allow-list follows the bind host, so `allowedOrigins` becomes `http://:PORT` and a browser hitting that address is accepted with no extra config. (The host is canonicalized the way a browser is — so `HOST=127.1` advertises `http://127.0.0.1:PORT`, an IPv6 bind host is bracketed as `http://[2001:db8::1]:PORT`, and the port is dropped when it's the http default `:80` — matching what the browser sends.) -- **Behind TLS or a reverse proxy**, the browser's `Origin` becomes the public origin (e.g. `https://inspector.example.com`, often without a port), which won't match the auto-derived `http://:PORT`. Set `ALLOWED_ORIGINS` to the real public origin(s): `ALLOWED_ORIGINS=https://inspector.example.com` That also admits `Host: inspector.example.com` to the [Host allow-list](#the-host-allow-list). If the proxy **rewrites** `Host` to some other name (its upstream's), every page load answers 403 until you add that name: `MCP_INSPECTOR_ALLOWED_HOSTS=inspector`, or configure the proxy to forward the original `Host` (nginx: `proxy_set_header Host $host;`). +- **Behind TLS or a reverse proxy**, the browser's `Origin` becomes the public origin (e.g. `https://inspector.example.com`, often without a port), which won't match the auto-derived `http://:PORT`. Set `ALLOWED_ORIGINS` to the real public origin(s): `ALLOWED_ORIGINS=https://inspector.example.com`. That also admits `Host: inspector.example.com` to the [Host allow-list](#the-host-allow-list). If the proxy **rewrites** `Host` to some other name (its upstream's), every page load answers 403 until you add that name: `MCP_INSPECTOR_ALLOWED_HOSTS=inspector`, or configure the proxy to forward the original `Host` (nginx: `proxy_set_header Host $host;`). - **Using the `0.0.0.0` wildcard** (opt-in via `DANGEROUSLY_BIND_ALL_INTERFACES=true`, as the Docker image does): a wildcard bind also serves loopback, so the default allow-list is the loopback trio plus the canonical wildcard origins (`http://0.0.0.0:PORT`, `http://[::]:PORT`), and **local access works out of the box** — `docker run -p 127.0.0.1:6274:6274` browsed at `http://localhost:6274` connects with no extra config. Reaching it at a **non-loopback** address (a LAN IP, a public hostname) still needs `ALLOWED_ORIGINS` — but since that **replaces** the default, keep the loopback forms in the list if you also browse locally: `ALLOWED_ORIGINS=http://localhost:PORT,http://127.0.0.1:PORT,http://192.168.1.50:PORT,https://inspector.example.com`. The bind-host guard and the `ALLOWED_ORIGINS` allow-list apply to both the prod server and `--dev`. Note that in **`--dev`** the Vite dev server _additionally_ enforces its own `server.allowedHosts` Host-header check, whose default accepts loopback and IP-literal hosts. The host you **bind** is auto-allowed (Vite adds the resolved `server.host` — which this config sets from `HOST` — to the allow-list), so `HOST=` works out of the box under `--dev` too. What needs an explicit `server.allowedHosts` entry is reaching the dev server at a **different** name than the one bound — e.g. a wildcard bind reached by hostname, or a reverse-proxy domain. For those, prefer the prod server (`mcp-inspector --web`) or add the host to `server.allowedHosts`. diff --git a/clients/web/server/host-allow-list.ts b/clients/web/server/host-allow-list.ts index 9c6479dc8..6b382e96e 100644 --- a/clients/web/server/host-allow-list.ts +++ b/clients/web/server/host-allow-list.ts @@ -39,6 +39,7 @@ */ import { isIPv4, isIPv6 } from "node:net"; +import { domainToASCII } from "node:url"; import type { Context, MiddlewareHandler, Next } from "hono"; import { canonicalUrlHost, @@ -48,6 +49,13 @@ import { /** Comma-separated extra hostnames the web server accepts in `Host`. */ export const ALLOWED_HOSTS_ENV = "MCP_INSPECTOR_ALLOWED_HOSTS"; +/** + * The characters a hostname a browser sends can contain: it punycodes an IDN + * and refuses the rest, so anything else is not a browser's `Host` (a comma + * in particular is what two `Host` headers joined together look like). + */ +const HOSTNAME_RE = /^[a-z0-9._-]+$/; + /** * Normalize a hostname for comparison: trim, lowercase, and drop one trailing * root dot (`localhost.` and `example.com.` name the same host as without it). @@ -85,16 +93,16 @@ export function parseHostHeader(value: string): string | null { if (port !== undefined) { if (!/^\d{1,5}$/.test(port) || Number(port) > 65535) return null; } - const name = hostname.startsWith("[") - ? hostname - : normalizeHostname(hostname); - return name === "" ? null : name; + if (hostname.startsWith("[")) return hostname; + const name = normalizeHostname(hostname); + return HOSTNAME_RE.test(name) ? name : null; } /** * Parse {@link ALLOWED_HOSTS_ENV}: comma-separated hostnames, each optionally * prefixed with `.` to admit a whole domain. Entries are lowercased and lose a - * trailing dot. An entry that is not a bare hostname — it carries a scheme, a + * trailing dot; an internationalized name is converted to punycode. An entry + * that is not a bare hostname — it carries a scheme, a * port, a path, a wildcard or whitespace — is warned about and dropped, since * it could never equal the hostname parsed out of a `Host` header and keeping * it would leave the user believing a proxy name was admitted when it is not. @@ -105,13 +113,19 @@ export function parseAllowedHostsEnv(value: string | undefined): string[] { for (const entry of value.split(",")) { const trimmed = entry.trim(); if (trimmed === "") continue; - if (/[\s/:*@]/.test(trimmed) || trimmed === ".") { + // An IDN entry is converted to the punycode form a browser sends, so + // `bücher.example` matches rather than silently never matching. + const domain = trimmed.startsWith(".") ? "." : ""; + const ascii = /[\s/:*@]/.test(trimmed) + ? "" + : domainToASCII(normalizeHostname(trimmed.slice(domain.length))); + if (!HOSTNAME_RE.test(ascii)) { console.warn( `Ignoring invalid ${ALLOWED_HOSTS_ENV} entry: ${trimmed} (list bare hostnames, e.g. inspector.example.com or .example.com)`, ); continue; } - out.push(normalizeHostname(trimmed)); + out.push(domain + ascii); } return out; } diff --git a/clients/web/src/test/integration/server/host-allow-list.test.ts b/clients/web/src/test/integration/server/host-allow-list.test.ts index af62141fd..8373b8b58 100644 --- a/clients/web/src/test/integration/server/host-allow-list.test.ts +++ b/clients/web/src/test/integration/server/host-allow-list.test.ts @@ -49,6 +49,9 @@ describe("parseHostHeader", () => { "host:123456", "host:80:80", "::1", + "evil.example, x.localhost", + "evil.example x.localhost", + "bücher.example", ])("rejects %j as malformed", (value) => { expect(parseHostHeader(value)).toBeNull(); }); @@ -71,6 +74,13 @@ describe("parseAllowedHostsEnv", () => { ).toEqual(["inspector.example.com", ".corp.example", "proxy"]); }); + it("converts an internationalized entry to punycode", () => { + expect(parseAllowedHostsEnv("Bücher.example,.bücher.example")).toEqual([ + "xn--bcher-kva.example", + ".xn--bcher-kva.example", + ]); + }); + it.each([ "https://inspector.example.com", "inspector.example.com:6274", @@ -80,6 +90,8 @@ describe("parseAllowedHostsEnv", () => { "user@host", "two words", ".", + "..", + "a!b.example", ])("warns about and drops %j", (entry) => { const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); expect(parseAllowedHostsEnv(`good.example,${entry}`)).toEqual([ diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 106afd1a6..2adb8765e 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -20,7 +20,7 @@ These guard the web backend, which spawns processes on request. Read [Host bindi | `HOST` | web, CLI | `127.0.0.1` | Address the web server binds. An all-interfaces host (`0.0.0.0`, `::`, an empty string, and equivalent spellings) is **refused** unless `DANGEROUSLY_BIND_ALL_INTERFACES` is enabled. The CLI reads it only to build its deep link. | | `DANGEROUSLY_BIND_ALL_INTERFACES` | web | off | Opts in to an all-interfaces `HOST`. Only `true` or `1` (case-insensitive) enable it, so `false` reads as off. The Docker image sets it. | | `ALLOWED_ORIGINS` | web | derived from `HOST` | Comma-separated origins allowed to call the API. Unset, the list follows `HOST` at `CLIENT_PORT`: the loopback origins for a loopback host, the loopback origins plus `http://0.0.0.0` and `http://[::]` for an all-interfaces bind, and otherwise only the configured host's own origin (so binding a LAN address does **not** also allow `localhost`). The default origins are `https://` when [TLS](#tls-for-the-web-ui) is on. **Replaces** the default list rather than adding to it, so list every form you browse from. Each entry must include the scheme (`http://localhost:6274`). The same list is the MCP Apps sandbox proxy's embedder allow-list (its `frame-ancestors` header and its referrer check), so a public Inspector origin must be listed here for the Apps tab to render. | -| `MCP_INSPECTOR_ALLOWED_HOSTS` | web | unset | Comma-separated hostnames the prod web server accepts in the `Host` header, on top of the ones it always accepts: `localhost`, `*.localhost`, IP literals, the `HOST` it binds and the hosts of `ALLOWED_ORIGINS`. Any other `Host` is answered with 403 before a page is served. Set it for a reverse proxy that rewrites `Host` to its upstream name. A leading `.` admits a domain and its subdomains (`.corp.example`); an entry with a scheme, port, path or wildcard is ignored with a warning. Not read under `--dev`, where Vite's `server.allowedHosts` applies. See [The Host allow-list](../clients/web/README.md#the-host-allow-list). | +| `MCP_INSPECTOR_ALLOWED_HOSTS` | web | unset | Comma-separated hostnames the prod web server accepts in the `Host` header, on top of the ones it always accepts: `localhost`, `*.localhost`, IP literals, the `HOST` it binds and the hosts of `ALLOWED_ORIGINS`. Any other `Host` is answered with 403 before a page is served. Set it for a reverse proxy that rewrites `Host` to its upstream name. A leading `.` admits a domain and its subdomains (`.corp.example`); an entry with a scheme, port, path or wildcard is ignored with a warning. Not read under `--dev`, where Vite's `server.allowedHosts` applies (Vite skips that check under TLS, where the certificate already does the job). See [The Host allow-list](../clients/web/README.md#the-host-allow-list). | ## TLS for the web UI From 3d701a0baa5c9954c0717a4ad5c5425ae3cf566d Mon Sep 17 00:00:00 2001 From: cliffhall Date: Tue, 20 Oct 2026 23:44:56 -0400 Subject: [PATCH 37/44] fix(web): cap the /api/fetch response body The /api/fetch route buffered the proxied response with `res.text()`, which has no size bound, so the buffer held whatever the upstream chose to send. Read it through a byte-counting reader instead and refuse with a 502 once it passes MAX_PROXIED_RESPONSE_BYTES (4 MiB), cancelling the upstream stream so the connection is released. A Content-Length that already declares more than the cap is refused before any body is read. The route only carries OAuth work on the web client (metadata, token, DCR/CIMD and revocation responses, all kilobytes); MCP transport traffic rides /api/mcp/*. The cancel is started, not awaited, since a source's cancel promise may never settle. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../core/mcp/remote/node/cappedBody.test.ts | 190 ++++++++++++++++++ .../mcp/remote/server-extra-coverage.test.ts | 111 ++++++++++ core/mcp/remote/node/cappedBody.ts | 80 ++++++++ core/mcp/remote/node/server.ts | 47 ++++- 4 files changed, 427 insertions(+), 1 deletion(-) create mode 100644 clients/web/src/test/core/mcp/remote/node/cappedBody.test.ts create mode 100644 core/mcp/remote/node/cappedBody.ts diff --git a/clients/web/src/test/core/mcp/remote/node/cappedBody.test.ts b/clients/web/src/test/core/mcp/remote/node/cappedBody.test.ts new file mode 100644 index 000000000..345bd108e --- /dev/null +++ b/clients/web/src/test/core/mcp/remote/node/cappedBody.test.ts @@ -0,0 +1,190 @@ +/** + * `readCappedText` / `declaresOversizeBody` — the byte budget `/api/fetch` + * applies to a proxied response body. + * + * Driven with hand-built streams so the chunk boundaries, and the cancel the + * overflow path must start, are observable directly. + */ + +import { describe, expect, it } from "vitest"; +import { + declaresOversizeBody, + readCappedText, +} from "@inspector/core/mcp/remote/node/cappedBody.js"; + +const encoder = new TextEncoder(); + +interface TrackedStream { + stream: ReadableStream; + /** Number of chunks the source handed out. */ + pulled: () => number; + /** Whether the consumer cancelled the stream. */ + cancelled: () => boolean; +} + +/** A stream yielding `chunks` in order, then ending (or, with `endless`, never). */ +function streamOf( + chunks: Uint8Array[], + opts: { endless?: boolean; cancelNeverSettles?: boolean } = {}, +): TrackedStream { + let index = 0; + let wasCancelled = false; + const stream = new ReadableStream( + { + pull(controller) { + if (index < chunks.length) { + controller.enqueue(chunks[index++]); + return; + } + if (opts.endless) { + index++; + controller.enqueue(new Uint8Array(1024)); + return; + } + controller.close(); + }, + cancel() { + wasCancelled = true; + if (opts.cancelNeverSettles) return new Promise(() => {}); + return undefined; + }, + }, + { highWaterMark: 0 }, + ); + return { + stream, + pulled: () => index, + cancelled: () => wasCancelled, + }; +} + +describe("readCappedText", () => { + it("reads a body under the budget, decoding across chunk boundaries", async () => { + // "é" is two bytes in UTF-8; split it between chunks so a decoder that is + // not run in streaming mode would emit replacement characters. + const bytes = encoder.encode("café ok"); + const { stream, cancelled } = streamOf([ + bytes.subarray(0, 4), + bytes.subarray(4), + ]); + + const read = await readCappedText(stream, 1024); + + expect(read).toEqual({ kind: "complete", text: "café ok" }); + expect(cancelled()).toBe(false); + }); + + it("strips a leading BOM the way Response.text() does", async () => { + const { stream } = streamOf([ + new Uint8Array([0xef, 0xbb, 0xbf]), + encoder.encode("{}"), + ]); + + expect(await readCappedText(stream, 1024)).toEqual({ + kind: "complete", + text: "{}", + }); + }); + + it("accepts a body of exactly the budget", async () => { + const { stream, cancelled } = streamOf([ + encoder.encode("abcd"), + encoder.encode("efgh"), + ]); + + expect(await readCappedText(stream, 8)).toEqual({ + kind: "complete", + text: "abcdefgh", + }); + expect(cancelled()).toBe(false); + }); + + it("refuses one byte past the budget and cancels the stream", async () => { + const { stream, cancelled } = streamOf([ + encoder.encode("abcd"), + encoder.encode("efghi"), + ]); + + expect(await readCappedText(stream, 8)).toEqual({ kind: "overflow" }); + expect(cancelled()).toBe(true); + }); + + it("counts bytes, not chunks, so one oversized chunk overflows", async () => { + const { stream, cancelled } = streamOf([new Uint8Array(64)]); + + expect(await readCappedText(stream, 16)).toEqual({ kind: "overflow" }); + expect(cancelled()).toBe(true); + }); + + it("stops reading a body that never ends once it passes the budget", async () => { + const { stream, pulled, cancelled } = streamOf([], { endless: true }); + + expect(await readCappedText(stream, 4096)).toEqual({ kind: "overflow" }); + // 1 KiB per pull: the fifth crosses 4 KiB, and nothing is pulled after it. + expect(pulled()).toBe(5); + expect(cancelled()).toBe(true); + }); + + it("returns on overflow without waiting for a cancel that never settles", async () => { + const { stream, cancelled } = streamOf([new Uint8Array(32)], { + cancelNeverSettles: true, + }); + + // Would hang (and time the test out) if the cancel were awaited. + expect(await readCappedText(stream, 8)).toEqual({ kind: "overflow" }); + expect(cancelled()).toBe(true); + }); + + it("swallows a cancel that rejects", async () => { + const stream = new ReadableStream({ + start(controller) { + controller.enqueue(new Uint8Array(32)); + }, + cancel() { + return Promise.reject(new Error("cancel failed")); + }, + }); + + expect(await readCappedText(stream, 8)).toEqual({ kind: "overflow" }); + }); + + it("propagates a read that fails mid-body", async () => { + const failure = new Error("connection reset"); + let first = true; + const stream = new ReadableStream({ + pull(controller) { + if (first) { + first = false; + controller.enqueue(encoder.encode("partial")); + return; + } + controller.error(failure); + }, + }); + + await expect(readCappedText(stream, 1024)).rejects.toBe(failure); + }); +}); + +describe("declaresOversizeBody", () => { + it("is false with no Content-Length", () => { + expect(declaresOversizeBody(null, 10)).toBe(false); + }); + + it("is false for a length at or under the budget", () => { + expect(declaresOversizeBody("10", 10)).toBe(false); + expect(declaresOversizeBody(" 3 ", 10)).toBe(false); + }); + + it("is true for a length over the budget", () => { + expect(declaresOversizeBody("11", 10)).toBe(true); + expect(declaresOversizeBody("99999999999999999999", 10)).toBe(true); + }); + + it("ignores a malformed length and leaves enforcement to the read", () => { + expect(declaresOversizeBody("", 10)).toBe(false); + expect(declaresOversizeBody("12abc", 10)).toBe(false); + expect(declaresOversizeBody("-50", 10)).toBe(false); + expect(declaresOversizeBody("11, 11", 10)).toBe(false); + }); +}); diff --git a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts index bee1d3e33..e1dca0f24 100644 --- a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts +++ b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts @@ -31,6 +31,7 @@ import type pinoType from "pino"; import { closeAbortedStream, createRemoteApp, + MAX_PROXIED_RESPONSE_BYTES, requestIdForSendWait, mcpParamHeadersOnly, } from "@inspector/core/mcp/remote/node/server.js"; @@ -306,6 +307,10 @@ describe("server.ts supplemental coverage", () => { let upstreamHits: string[]; /** Resolves when the never-ending `/stream-open` response is closed. */ let openStreamClosed: Promise; + /** Resolves when the never-ending `/oversize-endless` response is closed. */ + let oversizeClosed: Promise; + /** Resolves when the `/oversize-declared` response is closed. */ + let declaredClosed: Promise; // Isolate the WHOLE suite from the developer's (or CI's) proxy environment. // Setting only uppercase HTTP_PROXY in the one proxy test would not be @@ -351,8 +356,60 @@ describe("server.ts supplemental coverage", () => { openStreamClosed = new Promise((resolve) => { markStreamClosed = resolve; }); + let markOversizeClosed: () => void = () => {}; + oversizeClosed = new Promise((resolve) => { + markOversizeClosed = resolve; + }); + let markDeclaredClosed: () => void = () => {}; + declaredClosed = new Promise((resolve) => { + markDeclaredClosed = resolve; + }); const srv = createServer((req, res) => { upstreamHits.push(req.url ?? ""); + if (req.url === "/at-cap") { + // Exactly the budget, declared and sent. + res.writeHead(200, { + "Content-Type": "application/json", + "Content-Length": String(MAX_PROXIED_RESPONSE_BYTES), + }); + res.end(Buffer.alloc(MAX_PROXIED_RESPONSE_BYTES, "a")); + return; + } + if (req.url?.startsWith("/over-cap-chunked")) { + // One byte past the budget, chunked, so no Content-Length warns of it + // and only the streamed count can catch it. + res.writeHead(200, { "Content-Type": "application/json" }); + res.write(Buffer.alloc(MAX_PROXIED_RESPONSE_BYTES, "a")); + res.end("a"); + return; + } + if (req.url === "/oversize-endless") { + // A body that never ends. Without a bound the route buffers it until + // memory runs out; with one it must stop and drop the connection. + res.writeHead(200, { "Content-Type": "application/json" }); + const chunk = Buffer.alloc(64 * 1024, "a"); + const pump = () => { + while (res.write(chunk)) { + /* fill until backpressure */ + } + }; + res.on("drain", pump); + res.on("close", () => markOversizeClosed()); + pump(); + return; + } + if (req.url === "/oversize-declared") { + // Declares more than the budget and sends only a sliver, then holds + // the connection: the route must refuse on the header alone, without + // waiting for a body that is not coming, and release the socket. + res.writeHead(200, { + "Content-Type": "application/json", + "Content-Length": String(MAX_PROXIED_RESPONSE_BYTES + 1), + }); + res.write("{"); + res.on("close", () => markDeclaredClosed()); + return; + } if (req.url === "/stream-hostile") { // Headers, one frame, and a connection whose teardown the route must // not wait on: `ReadableStream.cancel()` adopts the source's cancel @@ -709,6 +766,60 @@ describe("server.ts supplemental coverage", () => { expect(body.body).toBeUndefined(); }); + it("relays a body of exactly the response cap", async () => { + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/at-cap` }), + }); + expect(res.status).toBe(200); + const body = (await res.json()) as { status: number; body?: string }; + expect(body.status).toBe(200); + expect(body.body).toHaveLength(MAX_PROXIED_RESPONSE_BYTES); + }); + + it("answers 502 for a streamed body one byte past the response cap", async () => { + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/over-cap-chunked?code=x` }), + }); + expect(res.status).toBe(502); + const body = (await res.json()) as { error: string; body?: string }; + expect(body.body).toBeUndefined(); + expect(body.error).toContain(`${MAX_PROXIED_RESPONSE_BYTES}-byte limit`); + // The URL in the message is redacted like every other one this route + // hands back, since an OAuth query string can carry a code or a token. + expect(body.error).toContain(`${targetUrl}/over-cap-chunked`); + expect(body.error).not.toContain("code=x"); + }); + + it("stops reading a body that never ends and releases the upstream", async () => { + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/oversize-endless` }), + }); + expect(res.status).toBe(502); + // The cancel on overflow drops the connection. Without it the upstream + // keeps writing into a socket nobody reads and this never resolves. + await oversizeClosed; + }); + + it("answers 502 on an oversize Content-Length without waiting for the body", async () => { + // The upstream sends one byte of the length it declared and then + // stalls, so a route that read before checking would hang here. + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/oversize-declared` }), + }); + expect(res.status).toBe(502); + const body = (await res.json()) as { error: string }; + expect(body.error).toContain(`${MAX_PROXIED_RESPONSE_BYTES}-byte limit`); + await declaredClosed; + }); + it("returns 400 when url is missing", async () => { const res = await fetch(`${h.baseUrl}/api/fetch`, { method: "POST", diff --git a/core/mcp/remote/node/cappedBody.ts b/core/mcp/remote/node/cappedBody.ts new file mode 100644 index 000000000..225bfad41 --- /dev/null +++ b/core/mcp/remote/node/cappedBody.ts @@ -0,0 +1,80 @@ +/** + * Read a response body as text, refusing one larger than a byte budget. + * + * Exists for the web backend's `POST /api/fetch` route, which buffers the + * proxied response in full before answering — it returns the body inside a + * JSON envelope, so it cannot stream it through. `Response.text()` has no + * bound at all: whatever the upstream sends is held in memory until it ends. + * This reads chunk by chunk, counts bytes rather than chunks (a stream may hand + * back one chunk far larger than the budget), and stops at the first byte past + * the limit. + * + * Reject rather than truncate, unlike the 401/403 challenge read in + * `fetchTracking.ts` (#2297). That one records a body for display, where a + * prefix is still useful; this one hands the body to an OAuth client that will + * parse it, and a cut-off discovery document or token response is not a + * smaller valid one — it is a different, wrong one. + * + * Kept separate from the challenge read for the same reason: that helper + * truncates, reads a clone, and carries its own deadline, none of which apply + * here. The route already bounds the read in time through its abort signal. + */ + +/** Outcome of {@link readCappedText}. */ +export type CappedTextRead = + | { kind: "complete"; text: string } + | { kind: "overflow" }; + +/** + * Read `body` to the end as UTF-8, or stop once it exceeds `maxBytes`. + * + * Decodes the way `Response.text()` does (UTF-8, a leading BOM stripped, + * invalid sequences replaced), so a body under the budget reads identically. + * A body of exactly `maxBytes` is complete; one byte more is an overflow. + * + * On overflow the stream is cancelled so the upstream connection is released. + * A read that rejects — the route's abort signal firing mid-body, a reset + * connection — propagates to the caller unchanged. + */ +export async function readCappedText( + body: ReadableStream, + maxBytes: number, +): Promise { + const reader = body.getReader(); + const decoder = new TextDecoder(); + let text = ""; + let bytes = 0; + for (;;) { + const next = await reader.read(); + if (next.done) return { kind: "complete", text: text + decoder.decode() }; + bytes += next.value.byteLength; + if (bytes > maxBytes) { + // ⚠️ Started, not awaited. `cancel()` adopts the underlying source's + // cancel promise, which is permitted never to settle — awaiting it would + // hold the caller pending on exactly the upstream it is trying to drop. + // The callee owns its failure through the `catch`. + void reader.cancel().catch(() => {}); + return { kind: "overflow" }; + } + text += decoder.decode(next.value, { stream: true }); + } +} + +/** + * Whether a `Content-Length` header already declares more than `maxBytes`. + * + * Lets the route refuse before reading a byte. Only ever a shortcut: an absent, + * malformed or understated length says nothing, and the streamed count in + * {@link readCappedText} is what actually enforces the budget. A compressed + * body's length is its encoded size, which the decoded body can only match or + * exceed, so an oversize declaration is a safe refusal either way. + */ +export function declaresOversizeBody( + contentLength: string | null, + maxBytes: number, +): boolean { + if (contentLength === null || !/^\s*\d+\s*$/.test(contentLength)) { + return false; + } + return Number(contentLength) > maxBytes; +} diff --git a/core/mcp/remote/node/server.ts b/core/mcp/remote/node/server.ts index 576672445..c1ce1f622 100644 --- a/core/mcp/remote/node/server.ts +++ b/core/mcp/remote/node/server.ts @@ -28,6 +28,7 @@ import { bodyLimit } from "hono/body-limit"; import { watch as chokidarWatch, type FSWatcher } from "chokidar"; import { createTransportNode } from "../../node/transport.js"; import { createProxyFetch } from "../../node/proxyFetch.js"; +import { declaresOversizeBody, readCappedText } from "./cappedBody.js"; import { OAUTH_TIMEOUT_WIRE_CODE } from "../../../auth/requestTimeout.js"; import { redactUrlQuery } from "../../fetchTracking.js"; import { isRateLimitError } from "../../rateLimit.js"; @@ -357,6 +358,24 @@ const MAX_APP_CSP_CHARS = 8 * 1024; */ const MAX_STORAGE_BODY_BYTES = 4 * 1024 * 1024; +/** + * Upper bound on a response body `POST /api/fetch` will buffer, in bytes. + * + * The route reads the upstream body in full before answering, because it hands + * it back inside a JSON envelope, so without a bound the size of that buffer is + * whatever the upstream chooses to send. Enforced while reading (see + * `readCappedText`), and early from `Content-Length` when one is declared. + * + * Sized for what actually travels this route on the web client: OAuth work — + * authorization-server and protected-resource metadata, token and revocation + * responses, Dynamic Client Registration and client ID metadata documents. + * Those measure in kilobytes. MCP traffic does not come through here; it rides + * `/api/mcp/*`, whose transport ignores `environment.fetch`. 4 MiB leaves three + * orders of magnitude of headroom over a real OAuth document while keeping one + * request's buffer small, and matches {@link MAX_STORAGE_BODY_BYTES}. + */ +export const MAX_PROXIED_RESPONSE_BYTES = 4 * 1024 * 1024; + /** * Whether a string is safe to emit as an HTTP header VALUE. * @@ -1368,7 +1387,33 @@ export function createRemoteApp( contentType?.includes("application/x-ndjson"); let resBody: string | undefined; if (!isStream && res.body) { - resBody = await res.text(); + // 502: the upstream answered, and what it answered is more than this + // gateway will buffer. Distinct from the 500 an outbound failure gets + // and the 504 a deadline gets; the browser surfaces it through + // `createRemoteFetch` as a failed fetch carrying this message. + const refuseOversize = () => + c.json( + { + error: `response from ${safeUrl} exceeds the ${MAX_PROXIED_RESPONSE_BYTES}-byte limit for a proxied response`, + }, + 502, + ); + if ( + declaresOversizeBody( + res.headers.get("content-length"), + MAX_PROXIED_RESPONSE_BYTES, + ) + ) { + // Refused before a byte is read, so nothing else will release the + // body. Not awaited, for the same reason as the stream branch below; + // the callee owns its failure through the `catch`. + void res.body.cancel().catch(() => {}); + return refuseOversize(); + } + // Cancels its own reader on overflow. + const read = await readCappedText(res.body, MAX_PROXIED_RESPONSE_BYTES); + if (read.kind === "overflow") return refuseOversize(); + resBody = read.text; } else if (isStream) { // This route does not return the stream — it answers with JSON and no // body — so nobody downstream owns it and nothing will ever read it. From 53d1c62ef816f7f5c0dfc5b236fcb8ae4b551f2c Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 00:52:31 -0400 Subject: [PATCH 38/44] fix(web): relay an error status whose /api/fetch body is over the cap An oversize body on a non-2xx upstream response is now relayed as its own status and headers with no body, instead of a 502. The status is what an OAuth client acts on there: a 404 from a path-aware protected resource metadata URL is what lets discovery fall back to the root well-known URL. A 2xx response over the cap still answers 502. Also corrects the deadline comment in the route, which said the route carries MCP traffic (on the web client it carries OAuth work only), and the Content-Length note in cappedBody.ts, which overstated the relation between encoded and decoded size. Adds a test pinning that the cap counts decoded bytes for a gzip-encoded body. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../mcp/remote/server-extra-coverage.test.ts | 99 +++++++++++++++++++ core/mcp/remote/node/cappedBody.ts | 13 ++- core/mcp/remote/node/server.ts | 52 ++++++---- 3 files changed, 144 insertions(+), 20 deletions(-) diff --git a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts index e1dca0f24..95f9dbe45 100644 --- a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts +++ b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts @@ -25,6 +25,7 @@ import { import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { gzipSync } from "node:zlib"; import { serve } from "@hono/node-server"; import type { ServerType } from "@hono/node-server"; import type pinoType from "pino"; @@ -366,6 +367,45 @@ describe("server.ts supplemental coverage", () => { }); const srv = createServer((req, res) => { upstreamHits.push(req.url ?? ""); + if (req.url === "/not-found-oversize") { + // An error status with a body past the budget, chunked so only the + // streamed count can catch it. + res.writeHead(404, { + "Content-Type": "text/html", + "X-Probe": "relayed", + }); + res.write(Buffer.alloc(MAX_PROXIED_RESPONSE_BYTES, "a")); + res.end("a"); + return; + } + if (req.url === "/not-found-declared") { + // An error status declaring more than the budget, then stalling. + res.writeHead(404, { + "Content-Type": "text/html", + "Content-Length": String(MAX_PROXIED_RESPONSE_BYTES + 1), + }); + res.write("<"); + return; + } + if (req.url === "/gzip-over-cap" || req.url === "/gzip-under-cap") { + // A few KiB on the wire, so neither the declared length nor the + // encoded byte count comes near the budget; only the decoded size + // can exceed it. + const decoded = Buffer.alloc( + req.url === "/gzip-over-cap" + ? MAX_PROXIED_RESPONSE_BYTES + 1 + : 1024, + "a", + ); + const encoded = gzipSync(decoded); + res.writeHead(200, { + "Content-Type": "application/json", + "Content-Encoding": "gzip", + "Content-Length": String(encoded.byteLength), + }); + res.end(encoded); + return; + } if (req.url === "/at-cap") { // Exactly the budget, declared and sent. res.writeHead(200, { @@ -820,6 +860,65 @@ describe("server.ts supplemental coverage", () => { await declaredClosed; }); + it("relays an error status whose body overflows, with its headers and no body", async () => { + // The status is what an OAuth client acts on for an error: a 404 from a + // path-aware metadata URL is what lets discovery fall back to the root + // well-known URL, which a 502 in its place would stop. + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/not-found-oversize` }), + }); + expect(res.status).toBe(200); + const body = (await res.json()) as { + ok: boolean; + status: number; + headers: Record; + body?: string; + }; + expect(body.ok).toBe(false); + expect(body.status).toBe(404); + expect(body.headers["x-probe"]).toBe("relayed"); + expect(body.body).toBeUndefined(); + }); + + it("relays an error status that declares an oversize body, without waiting for it", async () => { + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/not-found-declared` }), + }); + expect(res.status).toBe(200); + const body = (await res.json()) as { status: number; body?: string }; + expect(body.status).toBe(404); + expect(body.body).toBeUndefined(); + }); + + it("counts decoded bytes, so a small gzip body that expands past the cap is refused", async () => { + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/gzip-over-cap` }), + }); + expect(res.status).toBe(502); + const body = (await res.json()) as { error: string }; + expect(body.error).toContain(`${MAX_PROXIED_RESPONSE_BYTES}-byte limit`); + }); + + it("relays a gzip body under the cap decoded", async () => { + // The control for the case above: the same encoding, under the budget, + // passes — so the refusal there is the decoded size, not the encoding. + const res = await fetch(`${h.baseUrl}/api/fetch`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ url: `${targetUrl}/gzip-under-cap` }), + }); + expect(res.status).toBe(200); + const body = (await res.json()) as { status: number; body?: string }; + expect(body.status).toBe(200); + expect(body.body).toBe("a".repeat(1024)); + }); + it("returns 400 when url is missing", async () => { const res = await fetch(`${h.baseUrl}/api/fetch`, { method: "POST", diff --git a/core/mcp/remote/node/cappedBody.ts b/core/mcp/remote/node/cappedBody.ts index 225bfad41..29159a064 100644 --- a/core/mcp/remote/node/cappedBody.ts +++ b/core/mcp/remote/node/cappedBody.ts @@ -65,9 +65,16 @@ export async function readCappedText( * * Lets the route refuse before reading a byte. Only ever a shortcut: an absent, * malformed or understated length says nothing, and the streamed count in - * {@link readCappedText} is what actually enforces the budget. A compressed - * body's length is its encoded size, which the decoded body can only match or - * exceed, so an oversize declaration is a safe refusal either way. + * {@link readCappedText} is what actually enforces the budget, and it counts + * decoded bytes. + * + * A compressed body's declared length is its encoded size, which is not a + * bound on the decoded size in either direction: decoding usually grows it, + * but gzip of incompressible data is a few bytes larger than the data itself. + * So a body declared just over the budget might decode to just under it. That + * edge is accepted rather than read through: the budget is three orders of + * magnitude above any real OAuth document, so a body that close to it is not + * one worth relaying, and refusing on the header avoids reading it at all. */ export function declaresOversizeBody( contentLength: string | null, diff --git a/core/mcp/remote/node/server.ts b/core/mcp/remote/node/server.ts index c1ce1f622..56b40c3bb 100644 --- a/core/mcp/remote/node/server.ts +++ b/core/mcp/remote/node/server.ts @@ -1327,13 +1327,15 @@ export function createRemoteApp( // hands a live stream to its caller, so there is nothing left for the timer // to protect and nothing it could sever. // ⚠️ Only the caller decides whether this request is bounded, and by how - // much. This route is the browser's way out to the network for MCP traffic - // as well as OAuth work, so an unconditional timer here would abort a - // Streamable HTTP tool call that legitimately withholds its response - // headers for longer than the budget — reintroducing on the backend exactly - // the regression `exemptMcpEndpoint` exists to prevent on the client, and - // reporting it as an OAuth timeout besides (Copilot). No deadline in the - // envelope means no timer at all. + // much. On the web client today this route carries OAuth work only — MCP + // transport traffic rides `/api/mcp/*`, since `createRemoteTransport` + // ignores the fetch it is handed (see `MAX_PROXIED_RESPONSE_BYTES`). + // But `createRemoteFetch` is a general-purpose fetch and the route proxies + // whatever it is sent, so an unconditional timer here would also bound a + // caller that legitimately waits longer than the OAuth budget — a request + // withholding its response headers, say — and report it as an OAuth + // timeout besides (Copilot). No deadline in the envelope means no timer at + // all. // Clamped to what `setTimeout` can schedule: past 2**31-1 the delay // overflows and Node falls back to 1ms, turning an over-large budget into // an immediate timeout. Non-finite is ignored outright rather than @@ -1387,17 +1389,33 @@ export function createRemoteApp( contentType?.includes("application/x-ndjson"); let resBody: string | undefined; if (!isStream && res.body) { - // 502: the upstream answered, and what it answered is more than this - // gateway will buffer. Distinct from the 500 an outbound failure gets - // and the 504 a deadline gets; the browser surfaces it through - // `createRemoteFetch` as a failed fetch carrying this message. + // An oversize body is never relayed. What replaces it depends on the + // upstream status: + // + // - A success becomes a 502: the upstream answered, and what it + // answered is more than this gateway will buffer, so there is no + // faithful way to hand it on. Distinct from the 500 an outbound + // failure gets and the 504 a deadline gets; the browser surfaces it + // through `createRemoteFetch` as a failed fetch carrying this message. + // - An error status is relayed as itself, with its headers and no body. + // The status is what an OAuth client acts on there — a 404 from a + // path-aware protected-resource metadata URL is what sends SDK + // discovery on to the root well-known URL — and a 502 in its place + // would end discovery instead of letting it fall back. const refuseOversize = () => - c.json( - { - error: `response from ${safeUrl} exceeds the ${MAX_PROXIED_RESPONSE_BYTES}-byte limit for a proxied response`, - }, - 502, - ); + res.ok + ? c.json( + { + error: `response from ${safeUrl} exceeds the ${MAX_PROXIED_RESPONSE_BYTES}-byte limit for a proxied response`, + }, + 502, + ) + : c.json({ + ok: res.ok, + status: res.status, + statusText: res.statusText, + headers: resHeaders, + }); if ( declaresOversizeBody( res.headers.get("content-length"), From 28eda4cb343de2d06f007dfa253ceccd133dfb63 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 00:54:04 -0400 Subject: [PATCH 39/44] docs(web): state the IP forms the Host allow-list accepts Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/web/README.md | 2 +- clients/web/server/host-allow-list.ts | 8 +++++--- docs/docker.md | 2 +- docs/environment-variables.md | 2 +- 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/clients/web/README.md b/clients/web/README.md index af5f1bc29..f69a7ff7f 100644 --- a/clients/web/README.md +++ b/clients/web/README.md @@ -413,7 +413,7 @@ The backend's `/api/*` routes also enforce an **origin allow-list** (`allowedOri The origin allow-list only sees requests that carry an `Origin` header, and a browser sends none on a same-origin `GET` — which is exactly how the page that carries the API token is loaded. So the prod server (`server/host-allow-list.ts`) also checks the **`Host` header** of every request — `/`, the SPA fallback, static assets and `/api/*` — and answers **403** before serving anything when it names a host that is not ours. A page loaded from some other DNS name that has been re-pointed at this machine still sends its own name in `Host`, so it is refused. It is the same defence Vite applies in `--dev` (`server.allowedHosts`), with the same rules: -- **Always allowed:** `localhost`, any `*.localhost` name, and any IP literal (IPv4, or bracketed IPv6) — an address cannot be re-pointed, so it needs no listing. +- **Always allowed:** `localhost`, any `*.localhost` name, and any IP address written as a dotted-quad IPv4 (`127.0.0.1`) or a bracketed IPv6 (`[::1]`) — an address cannot be re-pointed, so it needs no listing. Shorthand IPv4 spellings (`127.1`, `2130706433`, `0x7f000001`) are treated as names and refused; browsers always send the dotted-quad form, so only a hand-written client would notice. - **Also allowed:** the bind host (`HOST`, unless it is a wildcard) and the host of every `ALLOWED_ORIGINS` entry, so setting `ALLOWED_ORIGINS=https://inspector.example.com` admits `Host: inspector.example.com` too. - **`MCP_INSPECTOR_ALLOWED_HOSTS`** (comma-separated hostnames) admits anything else — chiefly a reverse proxy that **rewrites** `Host` to its upstream name (`proxy_pass http://inspector:6274` sends `Host: inspector` unless the proxy forwards the original). A leading dot admits a domain and its subdomains (`.corp.example`). Entries are bare hostnames: a scheme, port, path or wildcard is warned about and ignored. - The **port is not compared**, so a remapped port (`-p 127.0.0.1:8080:6274`) or a proxy on `:443` still matches; the name is what a re-pointed page cannot change. A request with no `Host` header is refused. diff --git a/clients/web/server/host-allow-list.ts b/clients/web/server/host-allow-list.ts index 6b382e96e..fe846169f 100644 --- a/clients/web/server/host-allow-list.ts +++ b/clients/web/server/host-allow-list.ts @@ -18,8 +18,10 @@ * * - `localhost` and any `*.localhost` name (RFC 6761: they always resolve to * loopback, so no attacker can own one); - * - any IPv4 literal and any bracketed IPv6 literal — a rebinding attack needs - * a DNS name, so an address in `Host` cannot be one; + * - any dotted-quad IPv4 literal and any bracketed IPv6 literal — a + * rebinding attack needs a DNS name, so an address in `Host` cannot be one. + * Shorthand IPv4 (`127.1`, `2130706433`) is treated as a name and refused; + * browsers always send the dotted-quad form; * - the bind host (`HOST`), when it is a specific name rather than a wildcard; * - the host of every `allowedOrigins` entry, since an origin the API accepts * is by definition a name the browser is meant to reach us at; @@ -141,7 +143,7 @@ export interface HostAllowListOptions { /** * The configured (non-implicit) hostnames a server accepts in `Host`, beyond - * the always-allowed loopback names and IP literals. Exported for tests and + * the always-allowed loopback names and IP addresses. Exported for tests and * for anyone who needs to explain a rejection. */ export function configuredAllowedHosts( diff --git a/docs/docker.md b/docs/docker.md index a3528f4d8..0b48f4ab9 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -28,7 +28,7 @@ docker run --rm -p 127.0.0.1:6274:6274 -p 127.0.0.1:6275:6275 -p 127.0.0.1:6278: Publish each on the **same port number** inside and out. The sandbox URL is handed to the browser via `/api/config` as `http://localhost:/sandbox`, so remapping it (`-p 9000:6275`) advertises a port the browser can't reach; use `-e MCP_SANDBOX_PORT=9000 -p 127.0.0.1:9000:9000` instead. The same holds for the app origin: the URL a published app document is served from is built from the port the container bound, so remap with `-e MCP_APP_ORIGIN_PORT=9001 -p 127.0.0.1:9001:9001` rather than with `-p` alone. -**Behind a reverse proxy or an ingress**, where the browser reaches the listeners at public hostnames rather than at `localhost`, the same-number rule is not enough: the advertised URLs name the container's bind. Set `MCP_SANDBOX_FULL_ADDRESS` (e.g. `https://inspector-sandbox.example.com/sandbox`) and, for apps declaring `_meta.ui.domain`, `MCP_APP_ORIGIN_FULL_ADDRESS` (e.g. `https://inspector-apps.example.com`), route each to its container port, and set `ALLOWED_ORIGINS` to the public Inspector origin. The web server also refuses a request whose `Host` header is not a name it knows (loopback names, IP literals, the hosts of `ALLOWED_ORIGINS`), so a proxy that forwards the public `Host` needs nothing more; one that rewrites `Host` to the container's service name (`Host: inspector`) needs `-e MCP_INSPECTOR_ALLOWED_HOSTS=inspector`, or every page load answers 403. Each listener needs its own hostname or port — an address sharing the Inspector's origin is refused. See [Behind a reverse proxy](../clients/web/README.md#host-binding--the-origin-allow-list) for the full contract. +**Behind a reverse proxy or an ingress**, where the browser reaches the listeners at public hostnames rather than at `localhost`, the same-number rule is not enough: the advertised URLs name the container's bind. Set `MCP_SANDBOX_FULL_ADDRESS` (e.g. `https://inspector-sandbox.example.com/sandbox`) and, for apps declaring `_meta.ui.domain`, `MCP_APP_ORIGIN_FULL_ADDRESS` (e.g. `https://inspector-apps.example.com`), route each to its container port, and set `ALLOWED_ORIGINS` to the public Inspector origin. The web server also refuses a request whose `Host` header is not a name it knows (loopback names, dotted-quad IPv4 and bracketed IPv6 addresses, the hosts of `ALLOWED_ORIGINS`), so a proxy that forwards the public `Host` needs nothing more; one that rewrites `Host` to the container's service name (`Host: inspector`) needs `-e MCP_INSPECTOR_ALLOWED_HOSTS=inspector`, or every page load answers 403. Each listener needs its own hostname or port — an address sharing the Inspector's origin is refused. See [Behind a reverse proxy](../clients/web/README.md#host-binding--the-origin-allow-list) for the full contract. **Keep the `127.0.0.1:` prefix on the published port.** A bare `-p 6274:6274` publishes on **every host interface**, putting the Inspector on your local network. The container's `HOST=0.0.0.0` is a separate concern — it governs the _container's_ interfaces, not the host's — so the `DANGEROUSLY_BIND_ALL_INTERFACES` opt-in that guards a wildcard bind outside a container does not cover this. It matters more here than for an ordinary web app: the backend spawns processes on request, `GET /` embeds the API token into the served HTML, and a request arriving with **no** `Origin` header skips the origin allow-list entirely — so for any non-browser client the API token is the only guard. (A browser page is additionally held to the [Host allow-list](../clients/web/README.md#the-host-allow-list), which refuses a page loaded under a name that is not ours; it does nothing against a client that can reach the port and simply sends `Host: localhost`.) Publishing wider needs a real access-control boundary in front of the Inspector — a reverse proxy that authenticates, an SSH tunnel, a private network. Setting your own `MCP_INSPECTOR_API_TOKEN` does **not** substitute: `GET /` discloses whatever token is in use, so a custom one is harvested exactly as easily as a generated one. diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 2adb8765e..5a12efd5e 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -20,7 +20,7 @@ These guard the web backend, which spawns processes on request. Read [Host bindi | `HOST` | web, CLI | `127.0.0.1` | Address the web server binds. An all-interfaces host (`0.0.0.0`, `::`, an empty string, and equivalent spellings) is **refused** unless `DANGEROUSLY_BIND_ALL_INTERFACES` is enabled. The CLI reads it only to build its deep link. | | `DANGEROUSLY_BIND_ALL_INTERFACES` | web | off | Opts in to an all-interfaces `HOST`. Only `true` or `1` (case-insensitive) enable it, so `false` reads as off. The Docker image sets it. | | `ALLOWED_ORIGINS` | web | derived from `HOST` | Comma-separated origins allowed to call the API. Unset, the list follows `HOST` at `CLIENT_PORT`: the loopback origins for a loopback host, the loopback origins plus `http://0.0.0.0` and `http://[::]` for an all-interfaces bind, and otherwise only the configured host's own origin (so binding a LAN address does **not** also allow `localhost`). The default origins are `https://` when [TLS](#tls-for-the-web-ui) is on. **Replaces** the default list rather than adding to it, so list every form you browse from. Each entry must include the scheme (`http://localhost:6274`). The same list is the MCP Apps sandbox proxy's embedder allow-list (its `frame-ancestors` header and its referrer check), so a public Inspector origin must be listed here for the Apps tab to render. | -| `MCP_INSPECTOR_ALLOWED_HOSTS` | web | unset | Comma-separated hostnames the prod web server accepts in the `Host` header, on top of the ones it always accepts: `localhost`, `*.localhost`, IP literals, the `HOST` it binds and the hosts of `ALLOWED_ORIGINS`. Any other `Host` is answered with 403 before a page is served. Set it for a reverse proxy that rewrites `Host` to its upstream name. A leading `.` admits a domain and its subdomains (`.corp.example`); an entry with a scheme, port, path or wildcard is ignored with a warning. Not read under `--dev`, where Vite's `server.allowedHosts` applies (Vite skips that check under TLS, where the certificate already does the job). See [The Host allow-list](../clients/web/README.md#the-host-allow-list). | +| `MCP_INSPECTOR_ALLOWED_HOSTS` | web | unset | Comma-separated hostnames the prod web server accepts in the `Host` header, on top of the ones it always accepts: `localhost`, `*.localhost`, dotted-quad IPv4 and bracketed IPv6 addresses, the `HOST` it binds and the hosts of `ALLOWED_ORIGINS`. Any other `Host` is answered with 403 before a page is served. Set it for a reverse proxy that rewrites `Host` to its upstream name. A leading `.` admits a domain and its subdomains (`.corp.example`); an entry with a scheme, port, path or wildcard is ignored with a warning. Not read under `--dev`, where Vite's `server.allowedHosts` applies (Vite skips that check under TLS, where the certificate already does the job). See [The Host allow-list](../clients/web/README.md#the-host-allow-list). | ## TLS for the web UI From 51c7ffec7fd4e7d47c3815bc73c4ea90deac15e6 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 03:17:20 -0400 Subject: [PATCH 40/44] chore: bump version to 2.12.0 npm audit --audit-level=high reports 0 vulnerabilities in the root and all five client installs, so no audit fixes precede the bump. Closes #2762 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- package-lock.json | 4 ++-- package.json | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/package-lock.json b/package-lock.json index 2e1908191..c2a4ef0a0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "@modelcontextprotocol/inspector", - "version": "2.11.0", + "version": "2.12.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@modelcontextprotocol/inspector", - "version": "2.11.0", + "version": "2.12.0", "hasInstallScript": true, "license": "SEE LICENSE IN LICENSE", "dependencies": { diff --git a/package.json b/package.json index 7b25f77d5..8243224cb 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@modelcontextprotocol/inspector", - "version": "2.11.0", + "version": "2.12.0", "description": "The Model Context Protocol Inspector", "keywords": [ "MCP", From 9edc173bacafb4d12471997a156c54f884b9018e Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 12:11:52 -0400 Subject: [PATCH 41/44] fix(cli): link https in --print-handoff when the web UI is served over TLS buildHandoff() hardcoded http:// for the deep link, so with both INSPECTOR_TLS_KEY_PATH and INSPECTOR_TLS_CERT_PATH set (#2446) the link pointed at a listener that serves https only. It now follows the same both-set, blank-is-unset rule as resolveTlsConfig(). A half pair stays http, since the web server refuses to start on one. Closes #2770 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- clients/cli/README.md | 2 +- clients/cli/__tests__/stored-auth.test.ts | 51 +++++++++++++++++++++++ clients/cli/src/cli.ts | 13 +++++- 3 files changed, 64 insertions(+), 2 deletions(-) diff --git a/clients/cli/README.md b/clients/cli/README.md index 2d2b11157..408977fc0 100644 --- a/clients/cli/README.md +++ b/clients/cli/README.md @@ -507,7 +507,7 @@ For the common case where OAuth was already completed in the **web inspector on | `--use-stored-auth` | Read the stored auth for `--server-url` and inject `Authorization: Bearer`. When a `refresh_token` is stored, the CLI runs the OAuth refresh grant first and injects the **fresh** access token (persisting the rotation); otherwise it injects the stored access token. Exits `3` (`no_stored_token`) — listing the stored keys — when nothing matches. Requires `--server-url`. | | `--wait-for-auth ` | Poll the OAuth state file (500 ms interval) until an access token for `--server-url` appears, then inject it. Times out at `` with exit `3` (`auth_wait_timeout`). Use after handing off to a human to complete OAuth in a browser. Unlike `--use-stored-auth`, this injects the freshly-landed access token directly (a token that just completed the browser flow is not expired), so it does **not** run the refresh grant. | | `--list-stored-auth` | Print `{ oauthStatePath, storedServerUrls }` (the server keys that currently have a token) and exit. No server connection is made. | -| `--print-handoff` | Print a JSON handoff block (`deepLink`, `portForwardCmd`, `oauthStatePath`, `apiToken`) for `--server-url` and exit — everything a script/remote VM needs to drive the browser-side OAuth dance. Requires `--server-url`. | +| `--print-handoff` | Print a JSON handoff block (`deepLink`, `portForwardCmd`, `oauthStatePath`, `apiToken`) for `--server-url` and exit — everything a script/remote VM needs to drive the browser-side OAuth dance. The `deepLink` is `https` when both `INSPECTOR_TLS_KEY_PATH` and `INSPECTOR_TLS_CERT_PATH` are set, matching a web UI served over TLS. Requires `--server-url`. | **State-file resolution** follows `MCP_INSPECTOR_OAUTH_STATE_PATH` → `/oauth.json` → `~/.mcp-inspector/storage/oauth.json` — the same precedence the rest of the Inspector uses, so the CLI and web backend agree on the file. Server keys are canonicalised with `new URL().href` (the scheme the web store writes), so a trailing-slash or case mismatch between the URL a human opened and the one the agent passed still resolves. diff --git a/clients/cli/__tests__/stored-auth.test.ts b/clients/cli/__tests__/stored-auth.test.ts index 27c05655c..fa272dca8 100644 --- a/clients/cli/__tests__/stored-auth.test.ts +++ b/clients/cli/__tests__/stored-auth.test.ts @@ -782,6 +782,57 @@ describe("--print-handoff", () => { expect(out.deepLink.startsWith("http://localhost:16274/?")).toBe(true); }); + // The web listener serves https only when both TLS paths are set (#2446), so + // a handoff for it must link https or the page cannot load (#2770). + it.each([ + [ + "both TLS paths set", + { INSPECTOR_TLS_KEY_PATH: "/k.pem", INSPECTOR_TLS_CERT_PATH: "/c.pem" }, + "https", + ], + [ + "only the key path set", + { INSPECTOR_TLS_KEY_PATH: "/k.pem", INSPECTOR_TLS_CERT_PATH: "" }, + "http", + ], + [ + "only the cert path set", + { INSPECTOR_TLS_KEY_PATH: "", INSPECTOR_TLS_CERT_PATH: "/c.pem" }, + "http", + ], + [ + "a blank key path", + { INSPECTOR_TLS_KEY_PATH: " ", INSPECTOR_TLS_CERT_PATH: "/c.pem" }, + "http", + ], + ])( + "links the web UI's scheme in the deep link with %s", + async (_case, tlsEnv, scheme) => { + const result = await runCli( + ["--print-handoff", "--server-url", "https://x.example/mcp"], + { + env: { + MCP_INSPECTOR_API_TOKEN: "tok123", + HOST: "127.0.0.1", + CLIENT_PORT: "16274", + ...tlsEnv, + }, + }, + ); + expectCliSuccess(result); + const out = JSON.parse(result.stdout) as { + deepLink: string; + portForwardCmd: string; + }; + expect(out.deepLink.startsWith(`${scheme}://127.0.0.1:16274/?`)).toBe( + true, + ); + // The sandbox and app-origin ports stay plain http under TLS, and the + // port-forward command forwards ports, not schemes, so it is unchanged. + expect(out.portForwardCmd).toContain("--tcp 16274:16274"); + }, + ); + it("classifies a bad --callback-url as a usage error, not auth_required", async () => { // The guard's message contains "OAuth"; without the explicit exit-code pin // the heuristic would map it to auth_required (exit 3) and tell an automated diff --git a/clients/cli/src/cli.ts b/clients/cli/src/cli.ts index 034e38c83..2a75a45f3 100644 --- a/clients/cli/src/cli.ts +++ b/clients/cli/src/cli.ts @@ -727,6 +727,17 @@ function buildHandoff( ? "localhost" : canonicalUrlHost(host); const clientPort = process.env.CLIENT_PORT || "6274"; + // The web UI listener serves https only when BOTH TLS paths are set (#2446; + // `resolveTlsConfig` in clients/web/server/tls.ts, where a blank value counts + // as unset). An http link to that listener cannot load (#2770). A half pair + // stays http: the web server refuses to start on one, so there is no https + // listener to point at. The sandbox and app-origin ports stay plain http + // under TLS by design, so only the deep link's scheme follows this. + const scheme = + process.env.INSPECTOR_TLS_KEY_PATH?.trim() && + process.env.INSPECTOR_TLS_CERT_PATH?.trim() + ? "https" + : "http"; const sandboxPort = process.env.MCP_SANDBOX_PORT || "6275"; // The dedicated app origin (#2056). Forwarded alongside the other two: an App // whose UI resource declares `_meta.ui.domain` is served from this port and @@ -749,7 +760,7 @@ function buildHandoff( if (apiToken) params.set("autoConnect", apiToken); return { serverUrl: normalizedUrl, - deepLink: `http://${linkHost}:${clientPort}/?${params.toString()}`, + deepLink: `${scheme}://${linkHost}:${clientPort}/?${params.toString()}`, portForwardCmd: `coder port-forward --tcp ${clientPort}:${clientPort} --tcp ${sandboxPort}:${sandboxPort} --tcp ${appOriginPort}:${appOriginPort}`, oauthStatePath: statePath, apiToken: apiToken ?? null, From cbb3c656698049c353e1c23c6b6da8a38482a29c Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 12:29:10 -0400 Subject: [PATCH 42/44] fix(web): drop the dropped body's framing headers from an oversize error relay When /api/fetch refuses an oversized body for an error status, it relays the status and headers with no body. The relayed headers still carried content-length / transfer-encoding / content-encoding for the dropped body, so the browser rebuilt a null-body Response claiming a 5 MiB length. Those three are now omitted on that path only. Closes #2771 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../mcp/remote/server-extra-coverage.test.ts | 13 +++++++++- core/mcp/remote/node/server.ts | 25 +++++++++++++++++-- 2 files changed, 35 insertions(+), 3 deletions(-) diff --git a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts index 95f9dbe45..71d2a7342 100644 --- a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts +++ b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts @@ -879,6 +879,9 @@ describe("server.ts supplemental coverage", () => { expect(body.ok).toBe(false); expect(body.status).toBe(404); expect(body.headers["x-probe"]).toBe("relayed"); + expect(body.headers["content-type"]).toBe("text/html"); + // The chunked framing described the dropped body, not this one (#2771). + expect(body.headers["transfer-encoding"]).toBeUndefined(); expect(body.body).toBeUndefined(); }); @@ -889,8 +892,16 @@ describe("server.ts supplemental coverage", () => { body: JSON.stringify({ url: `${targetUrl}/not-found-declared` }), }); expect(res.status).toBe(200); - const body = (await res.json()) as { status: number; body?: string }; + const body = (await res.json()) as { + status: number; + headers: Record; + body?: string; + }; expect(body.status).toBe(404); + expect(body.headers["content-type"]).toBe("text/html"); + // A declared length for a body that is not relayed would misdescribe the + // rebuilt Response (#2771). + expect(body.headers["content-length"]).toBeUndefined(); expect(body.body).toBeUndefined(); }); diff --git a/core/mcp/remote/node/server.ts b/core/mcp/remote/node/server.ts index 56b40c3bb..e9cdfbbec 100644 --- a/core/mcp/remote/node/server.ts +++ b/core/mcp/remote/node/server.ts @@ -376,6 +376,26 @@ const MAX_STORAGE_BODY_BYTES = 4 * 1024 * 1024; */ export const MAX_PROXIED_RESPONSE_BYTES = 4 * 1024 * 1024; +/** + * Headers that describe a response body's framing rather than the response. + * An oversize error is relayed with its status and headers but no body, and + * the browser rebuilds it as a `Response` with a null body, so these would + * claim a length or an encoding for a body that is not there (#2771). + */ +const BODY_FRAMING_HEADERS = new Set([ + "content-length", + "transfer-encoding", + "content-encoding", +]); + +function withoutBodyFraming( + headers: Record, +): Record { + return Object.fromEntries( + Object.entries(headers).filter(([k]) => !BODY_FRAMING_HEADERS.has(k)), + ); +} + /** * Whether a string is safe to emit as an HTTP header VALUE. * @@ -1397,7 +1417,8 @@ export function createRemoteApp( // faithful way to hand it on. Distinct from the 500 an outbound // failure gets and the 504 a deadline gets; the browser surfaces it // through `createRemoteFetch` as a failed fetch carrying this message. - // - An error status is relayed as itself, with its headers and no body. + // - An error status is relayed as itself, with its headers and no body + // (less the dropped body's framing headers, #2771). // The status is what an OAuth client acts on there — a 404 from a // path-aware protected-resource metadata URL is what sends SDK // discovery on to the root well-known URL — and a 502 in its place @@ -1414,7 +1435,7 @@ export function createRemoteApp( ok: res.ok, status: res.status, statusText: res.statusText, - headers: resHeaders, + headers: withoutBodyFraming(resHeaders), }); if ( declaresOversizeBody( From 3a174cb0382613821fc7e82c06e4f99a48f0b43d Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 13:06:31 -0400 Subject: [PATCH 43/44] fix(mcpdo): give resources/directory/read a URI and a cursor The subcommand registered neither, so the handler's 'URI is required' error was its only outcome: --uri was an unknown option and a positional URI was 'too many arguments'. It now takes [uri] / --uri like resources/read, plus --cursor, as the one-shot CLI does, so a script can page a listing. Closes #2767 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- .../mcpdo/__tests__/directory-read.test.ts | 177 ++++++++++++++++++ clients/mcpdo/src/connection/mcp.ts | 19 ++ 2 files changed, 196 insertions(+) create mode 100644 clients/mcpdo/__tests__/directory-read.test.ts diff --git a/clients/mcpdo/__tests__/directory-read.test.ts b/clients/mcpdo/__tests__/directory-read.test.ts new file mode 100644 index 000000000..1fc75dcd3 --- /dev/null +++ b/clients/mcpdo/__tests__/directory-read.test.ts @@ -0,0 +1,177 @@ +/** + * mcpdo `resources/directory/read` takes a URI and a cursor (#2767). + * + * The subcommand once registered neither, so the handler's "URI is required" + * error was the only outcome: `--uri` was an unknown option and a positional + * URI was "too many arguments". This drives it end to end against a real + * skills server — positional URI, `--uri`, and the `nextCursor` of the first + * page passed back through `--cursor`. + */ +import { describe, it, expect, afterEach } from "vitest"; +import * as fs from "node:fs"; +import * as os from "node:os"; +import * as path from "node:path"; +import { + createTestServerHttp, + createTestServerInfo, +} from "@modelcontextprotocol/inspector-test-server"; +import { runMcp, type McpResult } from "./helpers/mcp-runner.js"; +import { + createTestConfig, + deleteConfigFile, +} from "../../cli/__tests__/helpers/fixtures.js"; +import { expectCliSuccess } from "../../cli/__tests__/helpers/assertions.js"; +import { callDaemon } from "../src/daemon/client.js"; + +/** Budget for each mcpdo command, matching the sibling suites' `runMcp` calls. */ +const COMMAND_TIMEOUT_MS = 20_000; +/** + * Five commands run in sequence, so the test's own budget must exceed them all + * together, or Vitest cuts in before a slow command's own timer can name it + * (see AGENTS.md "Test-gate timeouts"). + */ +const TEST_TIMEOUT_MS = 5 * COMMAND_TIMEOUT_MS + 10_000; + +/** The skills fixture's directory resource (no trailing slash). */ +const DIRECTORY_URI = "skill://data-analysis"; + +interface DirectoryPage { + resources: { uri: string }[]; + nextCursor?: string; + ttlMs?: number; + cacheScope?: string; +} + +function pageOf(result: McpResult): DirectoryPage { + expectCliSuccess(result); + try { + return JSON.parse(result.stdout) as DirectoryPage; + } catch { + throw new Error( + `not a directory page (exit ${result.exitCode}):\n${result.output}`, + ); + } +} + +describe("mcpdo resources/directory/read (#2767)", () => { + let storageDir: string | undefined; + let configPath: string | undefined; + + afterEach(async () => { + if (storageDir) { + const socketPath = path.join(storageDir, "mcpdod.sock"); + if (fs.existsSync(socketPath)) { + try { + await callDaemon("daemon/stop", {}, { socketPath, timeoutMs: 2000 }); + } catch { + // already stopped + } + // `daemon/stop` answers before shutdown finishes, so wait for the + // socket to go before deleting the directory a live daemon holds. + const deadline = Date.now() + 2000; + while (fs.existsSync(socketPath) && Date.now() < deadline) { + await new Promise((r) => setTimeout(r, 50)); + } + } + fs.rmSync(storageDir, { recursive: true, force: true }); + storageDir = undefined; + } + if (configPath) { + deleteConfigFile(configPath); + configPath = undefined; + } + }); + + it( + "reads a directory by positional URI or --uri, and pages with --cursor", + async () => { + const server = createTestServerHttp({ + serverInfo: createTestServerInfo("skills-directory", "1.0.0"), + skills: true, + }); + try { + await server.start(); + configPath = createTestConfig({ + mcpServers: { + skills: { type: "streamable-http", url: server.url }, + }, + }); + storageDir = fs.mkdtempSync(path.join(os.tmpdir(), "mcpdo-dirread-")); + const env = { + MCP_STORAGE_DIR: storageDir, + MCP_INSPECTOR_DAEMON_DIR: storageDir, + }; + + expectCliSuccess( + await runMcp( + ["connect", "skills", "--config", configPath, "--format", "json"], + { env, timeout: COMMAND_TIMEOUT_MS }, + ), + ); + + const positional = pageOf( + await runMcp( + [ + "@skills", + "resources/directory/read", + DIRECTORY_URI, + "--format", + "json", + ], + { env, timeout: COMMAND_TIMEOUT_MS }, + ), + ); + expect(positional.resources.length).toBeGreaterThan(0); + expect( + positional.resources.every((r) => + r.uri.startsWith(`${DIRECTORY_URI}/`), + ), + ).toBe(true); + // The result is returned whole, so #2740's cacheable fields reach it. + expect(positional.ttlMs).toBeTypeOf("number"); + expect(positional.cacheScope).toBeTypeOf("string"); + + const flag = pageOf( + await runMcp( + [ + "@skills", + "resources/directory/read", + "--uri", + DIRECTORY_URI, + "--format", + "json", + ], + { env, timeout: COMMAND_TIMEOUT_MS }, + ), + ); + expect(flag.resources).toEqual(positional.resources); + + // The fixture pages its directory, so the first page hands back a + // cursor; passing it back must return a different page. + const cursor = positional.nextCursor; + if (cursor === undefined) { + throw new Error("the skills fixture's first page has no nextCursor"); + } + const next = pageOf( + await runMcp( + [ + "@skills", + "resources/directory/read", + DIRECTORY_URI, + "--cursor", + cursor, + "--format", + "json", + ], + { env, timeout: COMMAND_TIMEOUT_MS }, + ), + ); + expect(next.resources.length).toBeGreaterThan(0); + expect(next.resources).not.toEqual(positional.resources); + } finally { + await server.stop(); + } + }, + TEST_TIMEOUT_MS, + ); +}); diff --git a/clients/mcpdo/src/connection/mcp.ts b/clients/mcpdo/src/connection/mcp.ts index e4b200aa1..0d1c10a9d 100644 --- a/clients/mcpdo/src/connection/mcp.ts +++ b/clients/mcpdo/src/connection/mcp.ts @@ -1380,6 +1380,25 @@ function registerRpcCommands(program: CommandType): void { }); }); break; + case "resources/directory/read": + // The handler requires a URI and pages by cursor, so without these the + // command could never succeed (#2767). One page per call, as in the + // one-shot CLI: the caller descends and pages itself. + cmd + .argument("[uri]", "Directory resource URI") + .option("--uri ", "Directory resource URI") + .option( + "--cursor ", + "Opaque pagination cursor (pass back the nextCursor from the previous page)", + ); + cmd.action(async (uriPos: string | undefined, o) => { + await runRpc(program, method, { + uri: (o.uri as string | undefined) ?? uriPos, + cursor: o.cursor as string | undefined, + metadata: o.metadata, + }); + }); + break; case "skills/list": cmd.option( "--verify", From 6b108c3dfae0b11ec5e752cd43f2fe759698cd72 Mon Sep 17 00:00:00 2001 From: cliffhall Date: Wed, 21 Oct 2026 13:45:36 -0400 Subject: [PATCH 44/44] fix(core): keep the connection up when a request POST is HTTP-refused The SDK's StreamableHTTP _send reports a failed POST through onerror and then rethrows it to the sender. InspectorClient's onerror errored the whole connection, so on the Node transport a refused subscriptions/listen took the session down before #2722's refused stream status could show, and the TUI's 'Listen stream: refused by the server' was unreachable. A deterministic HTTP refusal (the #2722 classifier: ClientHttpNotImplemented with 400/403/404/405, a code the SDK throws only from its POST path) now leaves the status alone; the request's rejection already surfaces it. Everything else reported through onerror still errors the connection. Closes #2776 Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: cliffhall --- ...nspectorClient-peer-handler-timing.test.ts | 61 +++++++++++++++++++ .../inspectorClient-subscriptions-era.test.ts | 19 ++++-- core/mcp/inspectorClient.ts | 10 +++ 3 files changed, 86 insertions(+), 4 deletions(-) diff --git a/clients/web/src/test/core/mcp/inspectorClient-peer-handler-timing.test.ts b/clients/web/src/test/core/mcp/inspectorClient-peer-handler-timing.test.ts index c1ddd36c8..e0021b09c 100644 --- a/clients/web/src/test/core/mcp/inspectorClient-peer-handler-timing.test.ts +++ b/clients/web/src/test/core/mcp/inspectorClient-peer-handler-timing.test.ts @@ -1,4 +1,5 @@ import { describe, it, expect, vi } from "vitest"; +import { SdkErrorCode, SdkHttpError } from "@modelcontextprotocol/client"; import type { JSONRPCMessage, Root, @@ -1240,4 +1241,64 @@ describe("InspectorClient peer-handler timing (#1797)", () => { expect(silent.getRoots()).toEqual([{ uri: "file:///a" }]); await silent.disconnect(); }); + + /* + * The SDK's StreamableHTTP `_send` reports a failed POST through `onerror` + * and then rethrows it to the sender. A deterministic HTTP refusal is a + * refused request, which its rejection already surfaces, so it must not + * error the whole connection (#2776). Anything else reported here still does. + */ + describe("transport onerror (#2776)", () => { + async function connectedClient() { + const transport = new SampleAfterConnectTransport(); + const client = new InspectorClient( + { type: "stdio", command: "noop", args: [] }, + { environment: { transport: () => ({ transport }) } }, + ); + await client.connect(); + const errors: unknown[] = []; + client.addEventListener("error", (e) => errors.push(e.detail)); + return { client, transport, errors }; + } + + const postFailure = (status: number, statusText: string) => + new SdkHttpError( + SdkErrorCode.ClientHttpNotImplemented, + `Error POSTing to endpoint: ${statusText}`, + { status, statusText }, + ); + + it.each([ + [400, "Bad Request"], + [403, "Forbidden"], + [404, "Not Found"], + [405, "Method Not Allowed"], + ])( + "keeps the connection up for a POST refused with HTTP %i", + async (status, statusText) => { + const { client, transport, errors } = await connectedClient(); + transport.onerror?.(postFailure(status, statusText)); + expect(client.getStatus()).toBe("connected"); + expect(errors).toEqual([]); + await client.disconnect(); + }, + ); + + it.each([ + ["a transient POST failure (HTTP 503)", postFailure(503, "Unavailable")], + [ + "an auth failure", + new SdkHttpError(SdkErrorCode.ClientHttpAuthentication, "Forbidden", { + status: 403, + }), + ], + ["a non-HTTP transport error", new Error("stream broke")], + ])("still errors the connection for %s", async (_label, error) => { + const { client, transport, errors } = await connectedClient(); + transport.onerror?.(error); + expect(client.getStatus()).toBe("error"); + expect(errors).toEqual([error]); + await client.disconnect(); + }); + }); }); diff --git a/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts index 2d95734e0..4da2c92cb 100644 --- a/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts +++ b/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts @@ -1275,6 +1275,9 @@ describe("resource subscriptions era fork (#1630)", () => { refusal: { status: 403, statusText: "Forbidden" }, }); expect(internals(connected).modernReconnectAttempts).toBe(0); + // The Node transport reports the refused POST through `onerror` too; + // a refused request is not a failed connection (#2776). + expect(connected.getStatus()).toBe("connected"); }); it("reports the refusal even when the rollback empties the filter", async () => { @@ -1307,6 +1310,13 @@ describe("resource subscriptions era fork (#1630)", () => { status: "refused", refusal: { status: 403 }, }); + // This is the TUI's path to its refusal line: on the Node transport + // the session must survive the refused connect-time listen (#2776). + expect(connected.getStatus()).toBe("connected"); + expect(connected.getResourceSubscriptionStreamState().refusal).toEqual({ + status: 403, + statusText: "Forbidden", + }); }); it.each([400, 404, 405])( @@ -1333,10 +1343,11 @@ describe("resource subscriptions era fork (#1630)", () => { /* * The tests above drive the Node transport, which reports a failed POST - * through `onerror` as well as rejecting it — so the client is already - * `error` by the time the reconnect timer would fire, and the timer bails - * on the terminal status either way. The web client's remote transport - * rejects without `onerror`, which is the path the smoke observed the + * through `onerror` as well as rejecting it. Since #2776 that report no + * longer errors the client for a refusal, so the session stays connected + * there too; before it, the client was already `error` by the time the + * reconnect timer would fire. The web client's remote transport rejects + * without `onerror`, which is the path the smoke observed the * eight-attempt run on. These stub `listen()` with the error that * transport rebuilds (#2660) to model it: the session stays connected, * and only the classification stands between the refusal and the retry. diff --git a/core/mcp/inspectorClient.ts b/core/mcp/inspectorClient.ts index 5c918cd46..a202a748f 100644 --- a/core/mcp/inspectorClient.ts +++ b/core/mcp/inspectorClient.ts @@ -1345,6 +1345,16 @@ export class InspectorClient extends InspectorClientEventTarget { // from any non-"connecting" state captures the reason regardless of // ordering. if (this.status === "connecting") return; + // A deterministic HTTP refusal of a POST is a refused REQUEST, not a + // failed connection (#2776). The SDK's StreamableHTTP `_send` reports it + // here and then rethrows it to the sender, so the request's rejection + // already surfaces it, and #2722's listen refusal settles its own stream + // status from that rejection. Flipping the client to "error" here made + // that status unreachable on the Node transport. The classifier matches + // only the code the SDK throws from its POST path, so a failure with no + // other surface (an SSE GET stream, a dropped connection, a 5xx) and + // every auth failure still land below. + if (subscriptionListenRefusal(error) !== undefined) return; this.status = "error"; this.dispatchTypedEvent("statusChange", this.status); this.dispatchTypedEvent("error", error);