Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 23 additions & 15 deletions docs/servers/media.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ The SDK ships two helpers for binary results (**`Image`** and **`Audio`**) and a

## Returning an image

Annotate the return type as `Image` and return one:
Annotate the return type as `Image`, point it at a file, and return it:

```python title="server.py" hl_lines="14 16"
```python title="server.py" hl_lines="8 12 14"
--8<-- "docs_src/media/tutorial001.py"
```

* `Image` takes exactly one of `data` (raw bytes) or `path` (a file to read).
* `format="png"` becomes the MIME type the client sees: `image/png`.
* The bytes here are a one-pixel placeholder so the file runs on its own. In a real server they come from Pillow, matplotlib, a headless browser, or anything else that hands you `bytes`.
* `Image` takes exactly one of `path` (a file to read) or `data` (raw bytes).
* The MIME type the client sees is guessed from the suffix: `logo.png` is announced as `image/png`.
* Nothing here is special about logos. Any PNG next to `server.py` works: a chart your code rendered, a diagram, a photo.

`Image` is an SDK convenience, not a protocol type. On the wire your return value becomes an **`ImageContent`** block (your bytes base64-encoded, plus the MIME type):
`Image` is an SDK convenience, not a protocol type. On the wire your return value becomes an **`ImageContent`** block (the file's bytes base64-encoded, plus the MIME type):

```python
result.content # [ImageContent(type="image", data="iVBORw0KGgoAAAANSUhEUg...", mime_type="image/png")]
Expand All @@ -25,7 +25,7 @@ result.structured_content # None

Two things to notice:

* `data` is base64. You returned raw `bytes`; the SDK did the encoding.
* `data` is base64. You never touched the bytes; the SDK read the file and did the encoding.
* `structured_content` is `None`. An `Image` is content for the model to look at, not data for the application to parse: there is no output schema. (Contrast **[Structured Output](structured-output.md)**, where the return annotation *is* the schema.)

!!! info
Expand All @@ -35,32 +35,40 @@ Two things to notice:

### Try it

Drop any PNG next to `server.py`, name it `logo.png`, and run:

```console
uv run mcp dev server.py
```

Open the **Tools** tab and call `logo`. The result is not a string: it is an `image` content block, and the Inspector renders it as a picture. You returned `bytes`; everything between that and the pixels on screen was the SDK.
Open the **Tools** tab and call `logo`. The result is not a string: it is an `image` content block, and the Inspector renders your picture. Everything between the file on disk and the pixels on screen was the SDK.

## Returning audio

`Audio` is the same shape:
`Audio` is the same shape. Keep `logo.png` where it was, and put any WAV beside it as `chime.wav`:

```python title="server.py" hl_lines="21-24"
```python title="server.py" hl_lines="18-21"
--8<-- "docs_src/media/tutorial002.py"
```

The result is an **`AudioContent`** block:

```python
result.content # [AudioContent(type="audio", data="UklGRjQAAABXQVZFZm1...", mime_type="audio/wav")]
result.content # [AudioContent(type="audio", data="UklGR...", mime_type="audio/wav")]
result.structured_content # None
```

Same deal: raw bytes in, base64 and a MIME type out, no output schema.
Same deal: a file on disk in, base64 and a MIME type out, no output schema.

## Bytes or a file

Both helpers also accept `path=` instead of `data=`. The file is read when the result is built, and the MIME type is guessed from the suffix:
Both helpers also accept `data=` (raw bytes) instead of `path=`. That is the mode for bytes that never came from a file of their own — a database column, an HTTP response, something Pillow just drew:

```python title="server.py" hl_lines="14 15"
--8<-- "docs_src/media/tutorial003.py"
```

With `path=` there is nothing to declare: the file is read when the result is built, and the MIME type is guessed from the suffix:

* `Image`: `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`.
* `Audio`: `.wav`, `.mp3`, `.ogg`, `.flac`, `.aac`, `.m4a`.
Expand All @@ -78,7 +86,7 @@ A suffix it doesn't recognise falls back to `application/octet-stream`.
An `Icon` is metadata, not content. It doesn't carry the image; it points at one with a URI, and a client may fetch it and show it next to your server's name, a tool, a resource, or a prompt.

```python title="server.py" hl_lines="5-6 8 11 17"
--8<-- "docs_src/media/tutorial003.py"
--8<-- "docs_src/media/tutorial004.py"
```

* `src` is a URI the client can resolve: `https:`, or a `data:` URI if you want the icon embedded with no extra fetch.
Expand All @@ -100,7 +108,7 @@ A tool's icons are on the `Tool` object from `tools/list`, a resource's on the `
## Recap

* Return an `Image` or `Audio` from a tool and the client receives an `ImageContent` / `AudioContent` block: your bytes base64-encoded, with a MIME type.
* Build one from in-memory `data=` plus an explicit `format=`, or from a `path=` and let the suffix decide.
* Build one from a `path=` and let the suffix decide the MIME type, or from in-memory `data=` plus an explicit `format=`.
* Media results carry no `structured_content` and no output schema.
* An `Icon` is a pointer: a `src` URI plus optional `mime_type`, `sizes`, and `theme`.
* `icons=[...]` works on the server, on tools, on resources, and on prompts, and clients find them on the matching objects.
Expand Down
8 changes: 3 additions & 5 deletions docs_src/media/tutorial001.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
import base64
from pathlib import Path

from mcp.server import MCPServer
from mcp.server.mcpserver import Image

mcp = MCPServer("Brand kit")

LOGO_PNG = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQ9bsBAAHPAURf8l/aAAAAAElFTkSuQmCC"
)
LOGO_FILE = Path(__file__).parent / "logo.png" # or the path to your file on disk


@mcp.tool()
def logo() -> Image:
"""The brand logo as a PNG."""
return Image(data=LOGO_PNG, format="png")
return Image(path=LOGO_FILE)
13 changes: 5 additions & 8 deletions docs_src/media/tutorial002.py
Original file line number Diff line number Diff line change
@@ -1,24 +1,21 @@
import base64
from pathlib import Path

from mcp.server import MCPServer
from mcp.server.mcpserver import Audio, Image

mcp = MCPServer("Brand kit")

LOGO_PNG = base64.b64decode(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQ9bsBAAHPAURf8l/aAAAAAElFTkSuQmCC"
)

CHIME_WAV = base64.b64decode("UklGRjQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YRAAAAAAAAAAAAAAAAAAAAAAAAAA")
LOGO_FILE = Path(__file__).parent / "logo.png"
CHIME_FILE = Path(__file__).parent / "chime.wav"


@mcp.tool()
def logo() -> Image:
"""The brand logo as a PNG."""
return Image(data=LOGO_PNG, format="png")
return Image(path=LOGO_FILE)


@mcp.tool()
def chime() -> Audio:
"""The notification chime as a WAV."""
return Audio(data=CHIME_WAV, format="wav")
return Audio(path=CHIME_FILE)
23 changes: 9 additions & 14 deletions docs_src/media/tutorial003.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
from mcp_types import Icon
from pathlib import Path

from mcp.server import MCPServer
from mcp.server.mcpserver import Image

LOGO = Icon(src="https://example.com/brand-kit.png", mime_type="image/png", sizes=["48x48"])
PALETTE = Icon(src="https://example.com/palette.svg", mime_type="image/svg+xml", sizes=["any"])
mcp = MCPServer("Brand kit")

mcp = MCPServer("Brand kit", icons=[LOGO])
LOGO_FILE = Path(__file__).parent / "logo.png"


@mcp.tool(icons=[PALETTE])
def palette() -> list[str]:
"""The brand colour palette as hex codes."""
return ["#1d4ed8", "#f59e0b", "#10b981"]


@mcp.resource("brand://guidelines", icons=[LOGO])
def guidelines() -> str:
"""How to use the brand assets."""
return "Use the primary colour for calls to action."
@mcp.tool()
def logo_from_bytes() -> Image:
"""The brand logo as a PNG."""
png = LOGO_FILE.read_bytes() # a database read, an HTTP response, Pillow output...
return Image(data=png, format="png")
20 changes: 20 additions & 0 deletions docs_src/media/tutorial004.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from mcp_types import Icon

from mcp.server import MCPServer

LOGO = Icon(src="https://example.com/brand-kit.png", mime_type="image/png", sizes=["48x48"])
PALETTE = Icon(src="https://example.com/palette.svg", mime_type="image/svg+xml", sizes=["any"])

mcp = MCPServer("Brand kit", icons=[LOGO])


@mcp.tool(icons=[PALETTE])
def palette() -> list[str]:
"""The brand colour palette as hex codes."""
return ["#1d4ed8", "#f59e0b", "#10b981"]


@mcp.resource("brand://guidelines", icons=[LOGO])
def guidelines() -> str:
"""How to use the brand assets."""
return "Use the primary colour for calls to action."
56 changes: 48 additions & 8 deletions tests/docs_src/test_media.py
Original file line number Diff line number Diff line change
@@ -1,28 +1,45 @@
"""`docs/servers/media.md`: every claim the page makes, proved against the real SDK."""

import base64
from pathlib import Path

import pytest
from mcp_types import AudioContent, Icon, ImageContent

from docs_src.media import tutorial001, tutorial002, tutorial003
from docs_src.media import tutorial001, tutorial002, tutorial003, tutorial004
from mcp import Client
from mcp.server.mcpserver import Audio, Image

# See test_index.py for why this is a per-module mark and not a conftest hook.
pytestmark = [pytest.mark.anyio, pytest.mark.filterwarnings("error::mcp.MCPDeprecationWarning")]


async def test_image_return_becomes_an_image_content_block() -> None:
"""tutorial001: `-> Image` reaches the client as a base64 `ImageContent` block, not text."""
@pytest.fixture
def logo_file(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
"""The PNG the tutorials expect next to `server.py`, fabricated on disk.

The SDK never parses the contents, so opaque bytes suffice.
"""
path = tmp_path / "logo.png"
path.write_bytes(b"fake png data")
monkeypatch.setattr(tutorial001, "LOGO_FILE", path)
monkeypatch.setattr(tutorial002, "LOGO_FILE", path)
monkeypatch.setattr(tutorial003, "LOGO_FILE", path)
return path


async def test_image_return_becomes_an_image_content_block(logo_file: Path) -> None:
"""tutorial001: `-> Image` reaches the client as a base64 `ImageContent` block, not text,
with the MIME type guessed from the `.png` suffix."""
async with Client(tutorial001.mcp) as client:
result = await client.call_tool("logo", {})
assert not result.is_error
assert result.content == [
ImageContent(type="image", data=base64.b64encode(tutorial001.LOGO_PNG).decode(), mime_type="image/png")
ImageContent(type="image", data=base64.b64encode(logo_file.read_bytes()).decode(), mime_type="image/png")
]


@pytest.mark.usefixtures("logo_file")
async def test_image_result_has_no_structured_content_and_no_output_schema() -> None:
"""tutorial001: media is content for the model, not data for the application."""
async with Client(tutorial001.mcp) as client:
Comment on lines +41 to 44

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 test_image_result_has_no_structured_content_and_no_output_schema asserts result.structured_content is None without first asserting not result.is_error — and now that this PR makes the logo tool fallible (Image(path=...) does file I/O), an error result also has structured_content=None, so the key assertion passes vacuously on the failure path. Add assert not result.is_error before the structured_content check, matching test_audio_return_becomes_an_audio_content_block added in this same PR.

Extended reasoning...

The bug. test_image_result_has_no_structured_content_and_no_output_schema (tests/docs_src/test_media.py:41-48) pins the doc claim "media is content for the model, not data for the application" by asserting tool.output_schema is None and result.structured_content is None. It never asserts not result.is_error. An error CallToolResult also has structured_content=None, so the structured-content assertion cannot distinguish the documented success behaviour from a failed tool call — it passes on both paths.

Why this PR introduced it. Before this PR, the logo tool returned inline bytes (Image(data=LOGO_PNG, format="png")) and could not fail. After it, the tool does file I/O: Image(path=LOGO_FILE) opens the file when the result is built, inside convert_result. Any exception there is caught by MCPServer._handle_call_tool (src/mcp/server/mcpserver/server.py:412-413), which returns CallToolResult(content=[TextContent(...)], is_error=True) with structured_content defaulting to None — it does not raise client-side. The PR touches this exact test (adding @pytest.mark.usefixtures("logo_file")), so the missing guard is squarely in scope.

Step-by-step proof of the vacuous pass. Suppose a future edit inlines Path(__file__).parent / "logo.png" in the tool body while leaving the LOGO_FILE constant in place (the logo_file fixture's monkeypatch.setattr(tutorial001, "LOGO_FILE", path) still succeeds silently — it patches a name nothing reads anymore):

  1. The test calls client.call_tool("logo", {}).
  2. The tool body evaluates Image(path=...); building the result opens the missing file → FileNotFoundError.
  3. _handle_call_tool catches it and returns CallToolResult(is_error=True, structured_content=None).
  4. assert tool.output_schema is None — passes (unrelated to the call). assert result.structured_content is None — passes, because error results carry no structured content.
  5. The test goes green while no longer proving the doc claim it exists to pin.

Why nothing else fully prevents it. The sibling test_image_return_becomes_an_image_content_block uses the same fixture and does assert not result.is_error plus exact content, so a fixture breakage would not be silent at the suite level — that is what keeps this at nit rather than normal. But this individual test's proof is still hollow, and the repo's own bar makes that a defect, not a style preference: .claude/skills/test-quality/SKILL.md's Hollow-proof check requires each claim to be proven by an assertion that cannot pass on an unrelated path, and AGENTS.md mandates conformance to that skill for test work.

The fix. One line, matching the pattern the same PR uses in test_audio_return_becomes_an_audio_content_block:

result = await client.call_tool("logo", {})
assert not result.is_error
assert result.structured_content is None

Expand All @@ -32,27 +49,50 @@ async def test_image_result_has_no_structured_content_and_no_output_schema() ->
assert result.structured_content is None


async def test_audio_return_becomes_an_audio_content_block() -> None:
async def test_audio_return_becomes_an_audio_content_block(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""tutorial002: `Audio` is the same shape as `Image`."""
chime_file = tmp_path / "chime.wav"
chime_file.write_bytes(b"fake wav data")
monkeypatch.setattr(tutorial002, "CHIME_FILE", chime_file)

async with Client(tutorial002.mcp) as client:
result = await client.call_tool("chime", {})
assert not result.is_error
assert result.content == [
AudioContent(type="audio", data=base64.b64encode(tutorial002.CHIME_WAV).decode(), mime_type="audio/wav")
AudioContent(type="audio", data=base64.b64encode(chime_file.read_bytes()).decode(), mime_type="audio/wav")
]
assert result.structured_content is None


async def test_in_memory_bytes_with_a_format_become_the_same_image_content_block(logo_file: Path) -> None:
"""tutorial003: `data=` plus `format=` produces the same wire block as `path=`."""
async with Client(tutorial003.mcp) as client:
result = await client.call_tool("logo_from_bytes", {})
assert not result.is_error
assert result.content == [
ImageContent(type="image", data=base64.b64encode(logo_file.read_bytes()).decode(), mime_type="image/png")
]


def test_path_file_is_read_when_the_result_is_built() -> None:
"""The page's `path=` claim (SDK-defined): the file is opened when the result is built,
not when the helper is constructed — `server.py` can name a file that appears later."""
image = Image(path="does-not-exist.png")
with pytest.raises(FileNotFoundError):
image.to_image_content()


def test_raw_data_without_a_format_falls_back_to_a_default_mime_type() -> None:
"""The `!!! check`: with `data=` there is no suffix to guess from, so `format=` decides."""
assert Image(data=b"\x89PNG\r\n\x1a\n", format="png").to_image_content().mime_type == "image/png"
assert Image(data=b"\x89PNG\r\n\x1a\n").to_image_content().mime_type == "image/png"
assert Audio(data=b"\xff\xfb", format="wav").to_audio_content().mime_type == "audio/wav"
assert Audio(data=b"\xff\xfb").to_audio_content().mime_type == "audio/wav"


async def test_icons_are_visible_where_they_were_declared() -> None:
"""tutorial003: server icons land on `server_info`, tool icons on the `Tool`, resource icons on the `Resource`."""
async with Client(tutorial003.mcp) as client:
"""tutorial004: server icons land on `server_info`, tool icons on the `Tool`, resource icons on the `Resource`."""
async with Client(tutorial004.mcp) as client:
assert client.server_info.icons == [
Icon(src="https://example.com/brand-kit.png", mime_type="image/png", sizes=["48x48"])
]
Expand Down
Loading