Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,699 changes: 2,699 additions & 0 deletions docs/mcp-contract-baseline.json

Large diffs are not rendered by default.

106 changes: 106 additions & 0 deletions docs/mcp-contract-drift.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# MCP schema compatibility digests

`MCP_CONTRACT_DIGESTS_ENABLED` defaults to false. Empty, `false`, and `0` disable it;
`true` and `1` enable it (case-insensitive, surrounding whitespace ignored). A malformed
value disables it and logs one warning per process or isolate, without the value.

When disabled, discovery and tool calls retain their existing output and a supplied
`X-MultiLLM-MCP-Contract` header is ignored. When enabled, discovery adds
`_meta.contract_digest` to each visible tool and to the tools/list result. The result
digest covers only tools authorized for that caller and selected toolsets. Filtering
and discovery order stay the same. Tool digests cover name, complete input/output
schemas, and contract version `1.0.0`; descriptions and annotations outside schemas
are excluded. Missing outputSchema is represented as null.

An enabled client may pin the **individual tool** digest with:

```http
X-MultiLLM-MCP-Contract: <64 lowercase hexadecimal characters from tool._meta.contract_digest>
```

The catalogue digest is for comparing discovery sets, not for pinning an individual
call. After authentication and tool authorization, a mismatch (including a malformed
or empty supplied pin) returns HTTP 409 with error code `mcp_contract_mismatch`
and message `The pinned MCP tool contract has changed. Refresh discovery.` before
execution, tool storage lookup, metering or provider dispatch. Unpinned calls keep working
when schemas change. No automatic retry is introduced. With the flag disabled, even
an invalid pin is ignored. A matching pin authorizes no additional permissions.

## Registered HTTP paths

Both the Flask gateway and the public edge Worker serve `POST /mcp`. Their
`tools/list` results include digests when enabled. `/mcp?toolsets=core,exa` restricts
discovery before hashing. Each tool includes `_meta.contract_digest`, and the
result includes `_meta.contract_digest` over its visible tools. Administrator,
reader and manager discovery follows the existing authorization rules.

`tools/call` checks the pin against the same contract used by discovery, after
authentication and tool authorization and before dispatch. A rejected pin returns
this JSON-RPC error with HTTP status 409 and `Cache-Control: no-store`:

```json
{"jsonrpc":"2.0","id":7,"error":{"code":"mcp_contract_mismatch","message":"The pinned MCP tool contract has changed. Refresh discovery."}}
```

The check covers every public Knowledge tool, including native provider tools and
tools forwarded to the private Knowledge service. Unauthorized calls keep the
existing scope-denial response and never expose a digest. Disabled calls keep
existing response bytes, status codes, headers and private dispatch payloads.
REST paths do not interpret this pin header. Status outputs report existing
operation names and native hash checks; they do not list tool schema definitions.
The offline catalogue and baseline remain independent of the runtime flag.

Set `MCP_CONTRACT_DIGESTS_ENABLED=true` on the public edge, Flask Container and
private Knowledge Worker consistently. The Container environment allowlist must
forward this key. Native Worker dispatch also checks an explicitly supplied
`options.contractPin` before authority, storage, metering and provider calls.
There are no new routes, response headers, migrations or runtime middleware hooks.

## Canonical format and bounds

`multillm-mcp-contract-v1` hashes UTF-8 compact JSON of typed nodes. Null is
`["null"]`, booleans and strings are `[type,value]`, arrays are `["array",nodes]`,
and objects are `["object",[[key,node],...]]` sorted by Unicode scalar key order.
Numbers use `["number",hex]`, where hex is the exact big-endian IEEE-754 binary64
representation. Equal integer/float values hash identically; negative zero becomes
positive zero. Tags prevent collisions between numbers and schema strings. Unknown
schema keywords remain intact and all arrays retain their order. This is a versioned
local canonical format, not RFC 8785. Changing the format requires new baselines.

Numbers must be finite with absolute value at most `2**53-1`; strings must contain
Unicode scalar values. Each canonical value is limited to 256 KiB, 64 nested levels,
and 65,536 nodes. These are bounds on trusted tool schemas and offline inputs, not
new limits on tool argument payloads. No hashing happens for disabled or unpinned
preflight. A digest identifies a declared schema, not a provider capability or a
promise that a provider result conforms to an output schema.

## Offline baseline and diff

From the repository root, using the installed Python environment:

```sh
python scripts/build_knowledge_mcp_catalogue.py --baseline docs/mcp-contract-baseline.json
python scripts/build_knowledge_mcp_catalogue.py --diff docs/mcp-contract-baseline.json
```

Both modes use local tool definitions and perform no network calls. The baseline
contains declared schemas, digests, and shared Python/Worker canonical vectors with
no timestamp, prompts, responses or credentials. Repeating a build yields identical
bytes. These modes do not regenerate `worker/knowledge-mcp-catalogue.json`; that
catalogue can be regenerated separately after tool schema changes. The existing
`--check` and no-argument catalogue build retain their prior behavior.

Diff output lists paths, change kinds and a combined classification. Tool/property
removal, required-field addition and type narrowing are breaking. Optional input
property additions and type widening are compatible. Unknown keyword changes,
constraint/composition/reference changes, output schema additions, version changes,
and ambiguous changes require review. Compatibility is a conservative structural
report, not a full JSON Schema implication proof. Tool renames appear as removal
plus addition. Existing schema fields remain in the digest even if the classifier
cannot interpret them. Baseline input is limited to 4 MiB. CLI exit status is 0 for
compatible, 1 for breaking/review-required, and 2 for an invalid baseline or operation.

This feature invokes no providers, spends no credits, stores no request content,
and adds no retention policy. Offline files contain public contract declarations;
protect them as source artifacts rather than request logs. Enabling the flag does
not change existing allowances, authentication, caching or retry permission.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"lint": "ruff check . --select E9,F63,F7,F82",
"typecheck": "mypy --ignore-missing-imports --follow-imports=skip --disable-error-code import-untyped --disable-error-code var-annotated --disable-error-code arg-type --disable-error-code assignment --disable-error-code operator app.py routes services providers streaming",
"test": "python -m pytest -q",
"test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs",
"test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_mcp_contract_drift_worker.mjs",
"db:migrate": "python scripts/validate_sqlite_schema.py --apply",
"db:validate": "python scripts/validate_sqlite_schema.py",
"security:secrets": "python scripts/check_static_secrets.py",
Expand Down
9 changes: 6 additions & 3 deletions routes/knowledge.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,10 @@ def _mcp_tool(identifier, params):
return _rpc_result(identifier, {"isError": True, "content": [{"type": "text", "text": json.dumps({
"error": {"code": "insufficient_scope", "message": "The key is not authorized for this Knowledge tool."},
})}]})
try:
mcp.check_contract_pin(tool_name, request.headers.get("X-MultiLLM-MCP-Contract"), definition=entry["definition"])
except KnowledgeError as error:
return _rpc_error(identifier, error.code, error.message, error.status)
arguments = params.get("arguments", {})
if not isinstance(arguments, dict):
return _rpc_error(identifier, -32602, "Tool arguments must be an object.")
Expand Down Expand Up @@ -256,9 +260,8 @@ def _mcp():
toolsets = mcp.requested_toolsets(request.args.get("toolsets"))
except ValueError:
return _rpc_error(identifier, -32602, f"Unknown toolset. Use any of: {', '.join(mcp.TOOLSETS)}.")
return _rpc_result(identifier, {"tools": [entry["definition"] for entry in _CATALOGUE["tools"]
if management.permits(g.authenticated_user, entry["scope"])
and (toolsets is None or entry["toolset"] in toolsets)]})
return _rpc_result(identifier, mcp.discovery_for_user(
g.authenticated_user, toolsets, entries=_CATALOGUE["tools"]))
if method != "tools/call":
return _rpc_error(identifier, -32601, "Method not found.")
return _mcp_tool(identifier, params)
Expand Down
25 changes: 25 additions & 0 deletions routes/knowledge_mcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from routes import knowledge_handoffs as handoffs
from routes import knowledge_skills as skills
from services.knowledge_native import NATIVE_OPERATIONS, NATIVE_TOOLS
from services import mcp_contract_drift as drift

CATALOGUE_PATH = Path(__file__).resolve().parents[1] / "worker" / "knowledge-mcp-catalogue.json"
# 2025-03-26 is not offered: it requires JSON-RPC batching, which this server does not accept.
Expand Down Expand Up @@ -130,3 +131,27 @@ def catalogue():

def catalogue_json():
return json.dumps(catalogue(), indent=2, ensure_ascii=False) + "\n"


def discovery_contract(scopes, toolsets=None, *, enabled=None, entries=None):
"""Runtime discovery hook: authorization filtering precedes digest metadata."""
active = drift.digests_enabled() if enabled is None else enabled
return drift.discovery_contract(catalogue()["tools"] if entries is None else entries, scopes, toolsets, enabled=active)


def discovery_for_user(user, toolsets=None, *, entries=None):
scopes = ["admin"] if user.get("is_admin") else user.get("scopes") or []
return discovery_contract(scopes, toolsets, entries=entries)


def check_contract_pin(tool_name, pin, *, enabled=None, definition=None):
"""Runtime preflight hook; invoke after tool authorization, before dispatch."""
active = drift.digests_enabled() if enabled is None else enabled
if not active or pin is None:
return
if definition is None:
definition = next((entry["definition"] for entry in catalogue()["tools"]
if entry["definition"]["name"] == tool_name), None)
if definition is None:
raise ValueError("Unknown Knowledge tool.")
drift.check_contract_pin(definition, pin, enabled=True)
29 changes: 27 additions & 2 deletions scripts/build_knowledge_mcp_catalogue.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,39 @@
"""Write the edge Worker's copy of the Knowledge MCP catalogue; --check only verifies it."""

import json
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parents[1]))

from routes.knowledge_mcp import CATALOGUE_PATH, catalogue_json
from routes.knowledge_mcp import CATALOGUE_PATH, catalogue, catalogue_json
from services.mcp_contract_drift import CONTRACT_VERSION, FORMAT, baseline_json, diff_contracts


def main(argv):
if len(argv) == 2 and argv[0] in {"--baseline", "--diff"}:
path = Path(argv[1]).resolve()
if path == CATALOGUE_PATH.resolve():
print("Contract baseline operations cannot overwrite the Worker catalogue.", file=sys.stderr)
return 2
definitions = [entry["definition"] for entry in catalogue()["tools"]]
try:
if argv[0] == "--baseline":
path.write_text(baseline_json(definitions), encoding="utf-8")
return 0
if path.stat().st_size > 4 * 1024 * 1024:
raise ValueError("Contract baseline exceeds the 4 MiB limit.")
previous = json.loads(path.read_text(encoding="utf-8"))
if (not isinstance(previous, dict) or previous.get("format") != FORMAT
or not isinstance(previous.get("tools"), list)):
raise ValueError("Unsupported contract baseline format.")
report = diff_contracts([entry["definition"] for entry in previous["tools"]], definitions,
old_version=previous.get("contract_version"), new_version=CONTRACT_VERSION)
except (OSError, ValueError, KeyError, TypeError) as error:
print(f"Contract baseline operation failed: {type(error).__name__}.", file=sys.stderr)
return 2
print(json.dumps(report, indent=2, ensure_ascii=False))
return 0 if report["classification"] == "compatible" else 1
expected = catalogue_json()
if argv == ["--check"]:
current = CATALOGUE_PATH.read_text(encoding="utf-8") if CATALOGUE_PATH.exists() else None
Expand All @@ -18,7 +43,7 @@ def main(argv):
return 1
return 0
if argv:
print("Usage: python scripts/build_knowledge_mcp_catalogue.py [--check]", file=sys.stderr)
print("Usage: python scripts/build_knowledge_mcp_catalogue.py [--check | --baseline PATH | --diff PATH]", file=sys.stderr)
return 2
CATALOGUE_PATH.write_text(expected, encoding="utf-8")
return 0
Expand Down
Loading
Loading