Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .sampo/changesets/mcp-client-conversation-session.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
pypi/posthog: minor
---

MCP analytics now groups tool calls into sessions by the client's own conversation id in `_meta`: ChatGPT's `openai/session`, Codex's `sessionId`, and VS Code's `vscode.conversationId`, each read only from the client its `User-Agent` or client name identifies. The client's id wins over a conversation handle and a transport session. ChatGPT gets no `conversation_id` argument and no handle, so its tool schemas and results stay unchanged. Events from `instrument()` now carry `$mcp_session_source` (`client_metadata`, `conversation`, `transport` or `generated`), so session fragmentation can be measured. `PostHogMCP.prepare_tool_list` and `prepare_tool_call` take the request's `headers` (and `client_info`), and `capture_tool_call` takes `request_meta`.
22 changes: 19 additions & 3 deletions posthog/mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,16 @@ Use a client dedicated to MCP analytics if the application also captures unrelat

A valid echoed conversation handle takes precedence over transport sessions. A request that already carries a transport session or a PostHog session token does not mint a new handle or append a prompt-back block. Requests carrying neither use the conversation fallback.

Some MCP clients name the conversation in each tool call's `params._meta`: ChatGPT's
`openai/session`, Codex's `sessionId`, and VS Code's `vscode.conversationId`. The SDK reads each key
only from a request whose `User-Agent` or client name identifies that client, and derives
`$session_id` from it ahead of any conversation handle or transport session. ChatGPT sends its id on
every call, so its `tools/list` has no `conversation_id` argument and its results get no handle, even
with `enable_conversation_id` on. Each event records which source grouped it in
`$mcp_session_source`: `client_metadata`, `conversation`, `transport`, or `generated`. Custom events
from `McpAnalytics.capture()` carry the instance's own source. The rules are the sdk-specs "Session
identity" requirement, and posthog-js `docs/adr/0014` records the reasoning.


Intent, model capture, conversation correlation, MCP exception capture, and `$mcp_initialize`
are on by default. Missing-capability reporting and feedback collection remain off.
Expand Down Expand Up @@ -233,9 +243,12 @@ Model and conversation injection copy tool objects instead of changing their ori
Always advertise the returned list and pass the original application tool to
`prepare_tool_call()`. Repeatedly preparing the original list preserves ownership.

Pass an existing transport or request session as `session_id`. A valid echoed
`conversation_id` takes precedence. Otherwise the existing session stays, and no new handle is
minted. `prepare_tool_result()` appends a new handle to the result's `content` and mirrors it
Pass `request_meta` whenever the request supplies `_meta`, and `headers` (plus `client_info` from a
handshake bound to the request, if you hold one) so the SDK can recognize the client; this enables
recognized client metadata and client conversation ids without changing the arguments sent to the
tool. Pass the same `headers` to `prepare_tool_list()`. Pass an existing transport or request
session as `session_id`. A client conversation id or a valid echoed `conversation_id` takes
precedence. Otherwise the existing session stays, and no new handle is minted. `prepare_tool_result()` appends a new handle to the result's `content` and mirrors it
into `structuredContent` when the tool declares an output schema. If the result has no channel
that can carry a new handle, `conversation_id` is `None` and the derived `session_id` is kept.
Set `enable_conversation_id=False` on `PostHogMCP` to keep the previous custom-dispatcher
Expand Down Expand Up @@ -447,6 +460,9 @@ Prefer it if you're on a recent client.
The `get_more_tools` and `send_feedback` virtual tools also use the conversation
handle when this option is enabled.

A recognized client's own conversation id in `params._meta` (such as ChatGPT's `openai/session`)
also groups calls on per-request servers, with or without this option, and wins over the handle.

### How the SDK tells you it's misconfigured

The failure used to be silent. It now surfaces two ways:
Expand Down
1 change: 1 addition & 0 deletions posthog/mcp/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ async def capture(self, event: str, properties: Optional[dict] = None) -> None:
data,
{
"session_id": data.session_id,
"session_source": data.session_source,
"event_type": MCPAnalyticsEventType.CUSTOM,
"event_name": event,
"timestamp": datetime.now(timezone.utc),
Expand Down
1 change: 1 addition & 0 deletions posthog/mcp/_capture.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ def capture_event(
full_event: Dict[str, Any] = {
"id": event_input.get("id") or "",
"session_id": session_id,
"session_source": event_input.get("session_source"),
"event_type": event_input.get("event_type") or MCPAnalyticsEventType.CUSTOM,
"event_name": event_input.get("event_name"),
"timestamp": timestamp,
Expand Down
173 changes: 173 additions & 0 deletions posthog/mcp/_client_conversation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
"""Client conversation ids: a value an MCP client puts in a ``tools/call``
request's ``_meta`` to name the conversation the call belongs to.

``CLIENT_CONVERSATION_ROWS`` is the table of the spec's "Session identity"
requirement, row for row. A row's key is read only from a request its recognition
rule matches, so a generic key such as Codex's ``sessionId`` is never read from a
client that means something else by it. posthog-js keeps the same table.
"""

from __future__ import annotations

import re
from typing import Any, Callable, NamedTuple, Optional, Sequence

from ._transport_identity import CLIENT_USER_AGENT_HEADER
from .request_headers import _to_header_bag, get_request_headers
from .session_token import decode_session_id, read_mcp_session_header

__all__ = [
"CLIENT_CONVERSATION_ROWS",
"ClientConversation",
"ClientConversationRow",
"meta_client_name",
"read_client_conversation",
"recognize_client",
"recognize_manual_client",
"request_user_agent",
]


class ClientConversation(NamedTuple):
"""A recognized client's conversation id, and the ``_meta`` key it came from."""

key: str
id: str


class ClientConversationRow(NamedTuple):
key: str
read_value: Callable[[Any], Optional[str]]
recognizes: Callable[[Optional[str], Optional[str]], bool]
# The client sends its id on every call, so the SDK advertises no
# ``conversation_id`` to it and mints it no handle. Rows whose clients have
# versions or kinds of conversation that send no id keep the handle.
handle_free: bool


_PRINTABLE_ASCII_UP_TO_128 = re.compile(r"[\x20-\x7e]{1,128}")
_UUID = re.compile(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}")
_NIL_UUID = "00000000-0000-0000-0000-000000000000"
_CHATGPT_USER_AGENT = re.compile(r"openai-mcp/[^\s(]*\s*\(\s*chatgpt\s*[;,)]", re.I)
_VSCODE_PRODUCT_NAMES = ("Visual Studio Code", "Visual Studio Code - Insiders")


def _read_printable_ascii(value: Any) -> Optional[str]:
if isinstance(value, str) and _PRINTABLE_ASCII_UP_TO_128.fullmatch(value):
return value
return None


def _read_uuid(value: Any) -> Optional[str]:
if not isinstance(value, str):
return None
lowered = value.lower()
return lowered if _UUID.fullmatch(lowered) and lowered != _NIL_UUID else None


def _user_agent_starts_with(user_agent: Optional[str], prefixes: Sequence[str]) -> bool:
lowered = (user_agent or "").lower()
return bool(lowered) and any(lowered.startswith(p.lower()) for p in prefixes)


def _client_name_is(client_name: Optional[str], names: Sequence[str]) -> bool:
lowered = (client_name or "").lower()
return bool(lowered) and any(lowered == name.lower() for name in names)


# Checked in this order; the first row that recognizes a request is its client's row.
CLIENT_CONVERSATION_ROWS = (
ClientConversationRow(
key="openai/session",
read_value=_read_printable_ascii,
recognizes=lambda user_agent, _: bool(
user_agent and _CHATGPT_USER_AGENT.match(user_agent)
),
handle_free=True,
),
ClientConversationRow(
key="sessionId",
read_value=_read_uuid,
recognizes=lambda user_agent, client_name: (
_user_agent_starts_with(user_agent, ["codex-mcp-client/"])
or _client_name_is(client_name, ["codex-mcp-client"])
),
handle_free=False,
),
ClientConversationRow(
key="vscode.conversationId",
read_value=_read_uuid,
recognizes=lambda user_agent, client_name: (
_user_agent_starts_with(
user_agent, [f"{name}/" for name in _VSCODE_PRODUCT_NAMES]
)
or _client_name_is(client_name, _VSCODE_PRODUCT_NAMES)
),
handle_free=False,
),
)


def recognize_client(
user_agent: Optional[str], client_name: Optional[str]
) -> Optional[ClientConversationRow]:
"""The row of the client a request's ``User-Agent`` or client name identifies.

The client name is ``$mcp_client_name`` as the adapters resolve it for the
request: its own ``_meta``, then a replayed session token, then a handshake."""
user_agent = user_agent if isinstance(user_agent, str) else None
client_name = client_name if isinstance(client_name, str) else None
for row in CLIENT_CONVERSATION_ROWS:
if row.recognizes(user_agent, client_name):
return row
return None


def read_client_conversation(
row: Optional[ClientConversationRow], meta: Any
) -> Optional[ClientConversation]:
"""The recognized client's conversation id in ``meta``, or ``None`` when there
is none or it is not the row's shape."""
if row is None or not isinstance(meta, dict):
return None
value = row.read_value(meta.get(row.key))
return ClientConversation(row.key, value) if value is not None else None


def request_user_agent(extra: Any) -> Optional[str]:
"""The request's ``User-Agent``, or ``None`` off HTTP. Never raises."""
try:
headers = get_request_headers(extra)
except Exception: # noqa: BLE001 - a header read must not break a request
return None
value = (headers or {}).get(CLIENT_USER_AGENT_HEADER)
return value if isinstance(value, str) else None


def recognize_manual_client(
headers: Any, request_meta: Any, handshake_client_name: Optional[str]
) -> Optional[ClientConversationRow]:
""":func:`recognize_client` for a custom dispatcher, from what it passes in:
the request's headers, its ``_meta``, and a handshake ``clientInfo`` name
bound to the request's transport session. The client name follows the
request's ``_meta``, then a replayed session token, then the handshake."""
bag = _to_header_bag(headers) if headers is not None else None
user_agent = (bag or {}).get(CLIENT_USER_AGENT_HEADER)
token = decode_session_id(read_mcp_session_header(bag)) if bag else None
return recognize_client(
user_agent if isinstance(user_agent, str) else None,
meta_client_name(request_meta)
or (token.client_name if token is not None else None)
or handshake_client_name,
)


def meta_client_name(meta: Any) -> Optional[str]:
"""The client name a 2026-07-28 request carries in its own ``_meta``."""
info = (
meta.get("io.modelcontextprotocol/clientInfo")
if isinstance(meta, dict)
else None
)
name = info.get("name") if isinstance(info, dict) else None
return name if isinstance(name, str) and name else None
25 changes: 21 additions & 4 deletions posthog/mcp/_instrument_fastmcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ async def wrapped(
# block on the minting response only.
delivered = False
if lifecycle.conversation_id:
if data.tool_output_instructions.get(name):
if lifecycle.mirrors_output_instructions:
result, delivered = mirror_instructions_into_structured_content(
result, lifecycle.conversation_id
)
Expand All @@ -203,7 +203,13 @@ async def wrapped(
# --- tools/list seam ---------------------------------------------------------


def _inject_tool_schemas(server: Any, data: MCPAnalyticsData, tools: list) -> None:
def _inject_tool_schemas(
server: Any,
data: MCPAnalyticsData,
tools: list,
*,
advertise_conversation_id: bool = True,
) -> None:
"""Advertise the analytics parameters on a listing's tools, in place.

Runs on both the client-facing listing and the SDK's internal cache-
Expand All @@ -218,6 +224,7 @@ def _inject_tool_schemas(server: Any, data: MCPAnalyticsData, tools: list) -> No
owns_context=_tool_owns_context(server, tool.name),
context_required=True,
is_sdk_virtual_tool=False,
advertise_conversation_id=advertise_conversation_id,
)


Expand Down Expand Up @@ -286,10 +293,20 @@ async def list_handler(req: Any) -> Any:
is_first_page=is_first_listing_page(getattr(req, "params", None)),
)

_inject_tool_schemas(server, data, tools)
_inject_tool_schemas(
server,
data,
tools,
advertise_conversation_id=lifecycle.advertises_conversation_id,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 · Keep the shared validation cache compatible with both client listings

On MCP 1.30.0, a ChatGPT listing after another client's listing replaces the shared _tool_cache output schema without _mcp_instructions. The other client's next call still mirrors that property, so tools with additionalProperties: false return an output validation error instead of their result. Keep the shared validation schema compatible with both clients' results.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. [P1] Mixed-client listings turn successful FastMCP results into validation errors

IMPORTANT · REQUIRED · Introduced regression
Location: posthog/mcp/_instrument_fastmcp.py:296–300. Related mirroring/state: :182–185, _instrumentation.py:1270–1273.

Ordinary sequential requests on one official MCP 1.x FastMCP instance turn a successful strict-schema tool result into an error:

1. Ordinary client: tools/list
2. ChatGPT client:  tools/list
3. Ordinary client: tools/call

Example tool:

class StrictResult(BaseModel):
    model_config = ConfigDict(extra="forbid")
    value: int

@server.tool()
def probe() -> StrictResult:
    return StrictResult(value=7)

Head returns:

{
  "content": [{
    "type": "text",
    "text": "Output validation error: Additional properties are not allowed ('_mcp_instructions' was unexpected)"
  }],
  "isError": true
}

Expected: the successful application result reaches the caller. Base does so.

The first listing sets shared tool_output_instructions["probe"]=True. The ChatGPT listing replaces the framework’s cached Tool with a strict schema lacking _mcp_instructions, but leaves that flag true. The next ordinary call mirrors the forbidden property; the outer framework validator rejects it.

Installed official MCP source confirms listings populate _tool_cache and call results are validated against it. PostHog’s FastMCP mirror executes inside that validator’s call path.

Canonical observer-only instrumentation forbids converting a successful result into failure. Optional mirroring still requires compatible declarations; official MCP requires structured results to conform to their output schema.

Executed regression: test_automatic_mixed_client_listings_keep_output_validation[other-then-chatgpt]; base passes, head fails:

assert not result.get("isError"), result
assert result["structuredContent"]["value"] == 7

The head’s ChatGPT-first-only control passes. The PR’s dismissal of that simpler ordering does not cover this failure.

Consider separating client-specific advertisements from a framework validation cache compatible with every result instrumentation can write, preserving ChatGPT’s unchanged listing. Retain both ordering cases.

Impact limits: reproduced on official MCP 1.28.1, shared instance, anchoring enabled and strict structured output. No new-regression execution claim is made for MCP 2.x or standalone FastMCP. This is actual response failure, not merely loss of optional mirroring.

Executed regression commands

Base: 5febc73aa74b38376a9d9cac895552686cefaca2; reviewed head: 3cbcd29b479ff2e47473a169c206b154b4c30378.

These are executed tests, not unexecuted sketches. Both pinned snapshots used Python 3.11.13 / MCP 1.28.1 with the same dependencies and sandbox. Local paths are represented by environment variables:

E="$REVIEW_EVIDENCE_DIR"
V="$REVIEW_VALIDATION_DIR"
PY="$REVIEW_PYTHON"
cd "$V"
for S in "$BASE_WORKTREE" "$HEAD_WORKTREE"; do
  for MODE in default bob-token; do
    env -i HOME="$V" TMPDIR="$V" PATH=/usr/bin:/bin PYTHONDONTWRITEBYTECODE=1 \
      /usr/bin/sandbox-exec -f "$E/sandbox.sb" \
      "$PY" -S "$E/regression_identity.py" "$S" "$MODE"
  done
  env -i HOME="$V" TMPDIR="$V" PATH=/usr/bin:/bin PYTHONDONTWRITEBYTECODE=1 \
    PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 /usr/bin/sandbox-exec -f "$E/sandbox.sb" \
    "$PY" -S "$E/run_pytest.py" "$S" "$E/test_followup_regressions.py" \
    --rootdir="$E" --confcutdir="$E" -o asyncio_mode=auto -q -s
done

Identity variants: base exit 0, head exit 1. Follow-up harness: base 6 passed, head 4 failed / 2 passed. No fixes were applied or tested.

)

result = apply_virtual_tool_injection(
result, injection, names, data, schema_field="inputSchema"
result,
injection,
names,
data,
schema_field="inputSchema",
advertise_conversation_id=lifecycle.advertises_conversation_id,
)

await lifecycle.record_result(
Expand Down
18 changes: 13 additions & 5 deletions posthog/mcp/_instrument_lowlevel.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ async def handler(req: Any) -> Any:
request = request_to_dict(req)
extra = {"session_id": mcp_session_id, "ctx": _request_context(server)}
try:
session_id = await prepare_request(
session = await prepare_request(
data,
mcp_session_id=mcp_session_id,
client_name=client_name,
Expand All @@ -152,7 +152,7 @@ async def handler(req: Any) -> Any:
except Exception as error:
await record_resource_request(
data,
session_id,
session,
event_type=event_type,
request=request,
error=error,
Expand All @@ -166,7 +166,7 @@ async def handler(req: Any) -> Any:

await record_resource_request(
data,
session_id,
session,
event_type=event_type,
request=request,
response=resource_listing_response(event_type, result),
Expand Down Expand Up @@ -305,7 +305,7 @@ async def handler(req: Any) -> Any:
# when the agent needs the handle, or the retry starts a fresh conversation.
delivered = False
if lifecycle.conversation_id:
if data.tool_output_instructions.get(name):
if lifecycle.mirrors_output_instructions:
call_result, delivered = mirror_instructions_into_structured_content(
call_result, lifecycle.conversation_id
)
Expand Down Expand Up @@ -351,6 +351,7 @@ def _inject_tool_schemas(
*,
context_required: bool,
high_level: Any = None,
advertise_conversation_id: bool = True,
) -> None:
"""Advertise the analytics parameters on a listing's tools, in place.

Expand All @@ -373,6 +374,7 @@ def _inject_tool_schemas(
context_required=context_required,
is_sdk_virtual_tool=False,
inject_model=inject_model,
advertise_conversation_id=advertise_conversation_id,
)
verdict = data.tool_model_parameter_injected.get(tool.name)
if verdict is None:
Expand Down Expand Up @@ -513,10 +515,16 @@ async def handler(req: Any) -> Any:
tools,
context_required=context_required,
high_level=high_level,
advertise_conversation_id=lifecycle.advertises_conversation_id,
)

result = apply_virtual_tool_injection(
result, injection, names, data, schema_field="inputSchema"
result,
injection,
names,
data,
schema_field="inputSchema",
advertise_conversation_id=lifecycle.advertises_conversation_id,
)

await lifecycle.record_result(
Expand Down
Loading
Loading