Skip to content

feat(gateway): add opt-in output schema validation - #36

Merged
ALikesToCode merged 1 commit into
integrate/gateway-waves-1-3from
feat/output-schema-validation
Oct 9, 2026
Merged

ALikesToCode merged 1 commit into
integrate/gateway-waves-1-3from
feat/output-schema-validation

Conversation

@ALikesToCode

Copy link
Copy Markdown
Owner

Summary

Adds explicit output schema validation behind OUTPUT_SCHEMA_VALIDATION_ENABLED, which defaults to false. While the flag is off, the multillm_output_validation request option is neither read nor removed, and every request and response is unchanged.

  • With the flag on, a managed Chat Completions, Messages or Responses request can include multillm_output_validation: {"mode": "strict", "schema": {...}}.
    • The gateway strips the option before dispatch.
    • It checks the non-streaming result as strict JSON against the schema. Chat checks every choice; Messages and Responses check the joined text blocks.
  • A valid response is returned with its original bytes and headers.
  • An invalid result returns HTTP 502 output_schema_violation.
    • The error lists bounded, content-free field paths (at most 100), never the model output.
    • There is no second generation and no repair. Tool repair stays separately opt-in.
  • A streaming request with the option returns 400 output_validation_requires_nonstream before dispatch.
  • Validation work is bounded:
    • response body ≤ 1 MiB and schema ≤ 64 KiB;
    • no remote $ref;
    • regex keywords, nested dialect or reference scopes, and Draft 3 schemas are rejected.
  • Raw passthrough and native Worker routes are unchanged.

Details are in docs/output-schema-validation.md.

Integration

The Wave 4 integration PR attaches with_managed_output_validation to the unified Chat and protocol dispatchers in routes/unified.py. Until then, the validator is inert even with the flag on.

Shared registrations

  • Added OUTPUT_SCHEMA_VALIDATION_ENABLED to the Container env keys.

Tests

  • tests/test_output_schema_validation.py: new. It covers Chat, translated and native Messages/Responses with the hook attached.
  • Full Python suite: 3,585 passed, 2 skipped. ruff, mypy and bandit are clean.
  • npm run test:worker: 1,318 passed.
  • cf:migrations:check, test:deploy, free routes, client headers and the wrangler deploy --dry-run bundle all pass.
  • tests/test_frontend_build.mjs fails locally because @tailwindcss/cli is not installed. It fails the same way on main.

This is part of Wave 4. It is based on integrate/gateway-waves-1-3, and the Wave 4 integration PR adds the README line.

Validate non-streaming managed results against a caller-supplied JSON schema behind OUTPUT_SCHEMA_VALIDATION_ENABLED (default off). Invalid results return 502 output_schema_violation with bounded, content-free paths; no regeneration or repair.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 87f352a5-3dfa-46b2-84eb-f05ad76585b9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
multillm-proxy 26d55ea Oct 08 2026, 10:56 PM

@ALikesToCode
ALikesToCode merged commit 4c24921 into integrate/gateway-waves-1-3 Oct 9, 2026
4 of 7 checks passed
@ALikesToCode
ALikesToCode deleted the feat/output-schema-validation branch October 26, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant