Assert on structured output in the smoke tests

[olaservo]

Draft until the 2026-07-28 revision and SDK is released

What the test client now does

After listing tools it calls every tool that declares an outputSchema and checks the answer against it:

• the result must carry structuredContent, and
• a tool with an array-rooted schema must return a top-level JSON array, while an object-rooted one must return an object.

Protocol revision 2026-07-28 is what makes the array case possible: through 2025-11-25 an outputSchema had to be object-rooted. The client negotiates with mode: "auto" — one server/discover probe, falling back to the initialize handshake — so it keeps working against every example as it stands today while also being able to test a server that speaks the new revision.

Tool calls reach the live NWS API. An unreachable upstream comes back as an error result, which the test reports as a skip rather than a failure; someone else's outage should not fail the build.

Mock server

mock-mcp-server moves to the low-level Server and advertises two tools, one array-rooted and one object-rooted, with the schemas written out literally — what goes on the wire is exactly what is in the file.

Build output is no longer swallowed

ensure_built ran every build with >/dev/null 2>&1 unconditionally. A compile failure therefore left no binary behind and the test reported a downstream spawn ... ENOENT instead of the actual error, which is what makes a broken build hard to diagnose from CI logs alone — #143 is the current example. run_build now prints the compiler output when a build fails.

Two smaller fixes alongside it:

• check_dependency returns instead of calling exit 1, so a missing toolchain fails its own test and the suite still finishes and prints its summary.
• Binaries are resolved through a helper that accounts for the .exe suffix, so the suite is runnable on Windows as well as in CI.

CI

The test helpers move to the 2.0 MCP SDK packages, which is where the 2026-07-28 support lives. The examples themselves are untouched.

Those packages require Node 20 or newer, so CI moves off Node 18. It goes to Node 24, the Active LTS, rather than to the floor — Node 20 reached end of life in April 2026.

Still uncovered

The Go examples and the Rust client remain untested here. Each needs a change inside its own directory first — both clients abort when no .env file is present, so neither can be driven without credentials — so that coverage belongs with those changes rather than in this PR.

Verification

CI on this PR is green. Locally, run against main with these changes applied, all five covered examples pass:

✓ weather-server-python     ✓ get_alerts: structuredContent is an object, matching its schema
                            ✓ get_forecast: structuredContent is an object, matching its schema
✓ weather-server-typescript ✓ Verified structured output for 0 tools
✓ weather-server-rust       ✓ Verified structured output for 0 tools
✓ mcp-client-python
✓ mcp-client-typescript

The Python server already exercises the new assertion: its tools return -> str, from which the SDK derives an object-rooted {"result": ...} schema, and the test confirms the structured content matches it. The TypeScript and Rust servers declare no output schemas today, so the assertion correctly no-ops rather than failing them.

Related

First of a set bringing the examples onto 2026-07-28. This one is the prerequisite: it is what lets the smoke tests exercise a server that declares a non-object output schema. The rest of the set is #164 (Python), #165 (TypeScript), #166 (Go), #167 (Rust), with #163 as the shared prerequisite. The Ruby examples are deliberately not in the set: the mcp gem caps at protocol 2025-11-25.

---

🤖 Generated with Claude Code