Repository navigation
feat(idempotency): add opt-in managed request idempotency - #47
Conversation
Accept Idempotency-Key on managed nonstreaming chat behind MANAGED_IDEMPOTENCY_ENABLED (default off). An atomic D1 claim (migration 0023) precedes admission and dispatch, binds the key to the principal, path, normalized body and routing policy, and records the provider handoff durably. Concurrent duplicates get 409 request_in_progress, a changed request under the same key gets 422, and a completed response replays from R2 with X-MultiLLM-Idempotency: replayed and no new provider call. With the flag off, the intelligence gateway keeps rejecting the header and raw provider routes are unchanged.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
multillm-proxy | 24d66c0 | Oct 09 2026, 12:42 AM |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 37552414 | Triggered | Generic Password | 9aea628 | worker/alert-delivery.mjs | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Summary
Adds opt-in idempotency for managed non-streaming chat behind
MANAGED_IDEMPOTENCY_ENABLED, which defaults to false.With the flag off:
Idempotency-Keywithidempotency_not_supported.Idempotency-Key.With the flag on:
0023) precedes admission and dispatch.A-Za-z0-9._:/+@-) is scoped to the authenticated principal, method and path.request_in_progresswithRetry-After: 1.idempotency_key_conflict.idempotency_stream_unsupportedbefore any claim.X-MultiLLM-Idempotency: replayed. Replay calls no provider and reserves no usage.outcome_unknown, and never starts a second generation.managed-idempotency/, up to 1 MiB with at most 32 headers, and expire 24 hours after the claim.idempotency_store_unavailablebefore generation. There is no process-local fallback.Details are in
docs/managed-idempotency.md.Worker
The private outbound handler mounts the
managed-statedispatcher (/v1/managed-state/*). Paths are fixed, only POST fromintelligence.internalis accepted, and unregistered domains return 404.The
idempotencydomain is wired here. Thereservationsandtool-grantsdomains accept injected handlers, but nothing is injected; those features use their own routes.Glue left for the Wave 4 integration PR
middleware.idempotency.register_idempotencythroughservices/gateway_extensions.py, with the namedidempotency_request_hookafter authentication, retention and secret policy and before admission. Until it is registered, turning the flag on makes keyed managed requests fail closed with 503idempotency_store_unavailable. Requests without a key are unaffected./v1/chat/completionsand connect usage settlement./v1/reservations(feat(budget): add opt-in durable ambiguous usage holds #44), and tool grants keep the model-statetool_grantsoperation (feat(mcp): add opt-in permission-aware deferred tool discovery #38). The managed-statereservationsandtoolGrantsinjection points stay unused, so there is only one route for each.Stacking
This PR is based on
feat/gateway-alerts(#46), so migrations0018–0023stay contiguous. Review only the top commit,feat(idempotency): add opt-in managed request idempotency; the merge commit brings in the lower layers.Deployment
Apply the migration before merging to main.
/readychecks the table, and a push to main does not apply D1 migrations:Operators should also add an R2 lifecycle rule for the
managed-idempotency/prefix to remove expired and orphaned objects.Shared registrations
tests/test_managed_idempotency_worker.mjstotest:worker.MANAGED_IDEMPOTENCY_ENABLEDto the Container env keys.managed_idempotencyto the D1 readiness tables.worker/idempotency-d1.mjsandworker/managed-state-dispatch.mjsto the secret firewall inventory.X-MultiLLM-Idempotencyto the exposed CORS headers in Flask and the Worker.Tests
tests/test_managed_idempotency.pyandtests/test_managed_idempotency_worker.mjs. They cover:npm run test:worker: 1,444 passed.npm run test:integration: 7 passed.cf:migrations:checkreports 23 contiguous, additive migrations.test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.CI notes
python/nodejobs: the dependency-audit steps fail on every branch, including main. fix(deps): clear the CI dependency audit advisories #42 fixes that.AdmissionCoordinatorDurable Object migration, whichwrangler versions uploadcannot apply. This is expected untilwrangler deployruns on main.This is part of Wave 4.