Skip to content

CLI --load: multi-connection mode (--load-connections) to test many sessions, not one #2746

Open

Description

@cliffhall

Follow-up to #2439 / #2743.

Problem

--load runs every call over the single connection the CLI already opened, so it measures concurrency within one session (documented in the CLI README). It cannot show how a server behaves with many clients at once: per-session state, session creation cost, connection limits, or contention that only shows up across sessions. For stdio this means one server process. For Streamable HTTP it means one Mcp-Session-Id.

Proposal

An opt-in --load-connections <n> (default 1, so today's behavior is unchanged). It opens n independent InspectorClient connections using the same resolved transport, headers and OAuth, spreads the workers across them, and reports one combined summary. It could optionally add a per-connection breakdown, or report connect failures separately from call errors.

Things to settle:

  • stdio: n connections means n spawned server processes. That is a different test from HTTP; decide whether to allow it or reject it for stdio.
  • OAuth: every connection reuses the stored token, so no extra prompts. Confirm that token refresh under concurrency doesn't race.
  • How --load-concurrency relates to it: total across all connections, or per connection.
  • Setup and teardown cost should stay out of the latency figures.

The engine (core/cli/handlers/load-test.ts, runLoad) already takes a call function, so the change is mostly in runToolLoad.

Priority score

Severity 3 (a real capability is missing; the workaround of running several CLI processes and merging their summaries by hand is partial) + urgency 3 (wanted this milestone) + 1 (milestoned) = 7 → Medium.

Activity

  1. added this to the v2.12.0 milestone on Oct 20, 2026
  2. modified the milestones: v2.12.0, v2.13.0 on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature requestv2Issues and PRs for v2

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions