Repository navigation
feat(responses): add opt-in hosted Responses continuation state - #59
Conversation
With HOSTED_RESPONSES_ENABLED on, a managed /v1/responses request with gateway_state and store keeps the completed response under a gateway-issued ID. previous_response_id continues from it in order, and GET and DELETE act only on the caller's own IDs. State lives in D1 and R2 with a 24-hour TTL, and only classified complete successes are stored.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
multillm-proxy | a4630c5 | Oct 09 2026, 02:58 AM |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
Adds opt-in hosted Responses continuation state. With
HOSTED_RESPONSES_ENABLEDoff (the default),/v1/responsesrequests, response bytes, headers and storage are unchanged. That includesstore,previous_response_idand agateway_statefield.GETandDELETE /v1/responses/<id>keep their existing routing.POST /v1/responsesroute,gateway_state: truetogether with the literalstore: trueasks the gateway to keep the state.gateway_stateis removed before dispatch, and provider-side storage is turned off for that generation.gwresp_plus 32 hex characters.previous_response_idwith a gateway ID loads the caller's stored state. It rebuilds the original input, the completed output items and the new input in order, keeping tool call/result boundaries exact.gateway_stateruns once with the retained context and stores nothing new.GETandDELETE /v1/responses/<id>act only on gateway IDs owned by the caller (404 otherwise), withCache-Control: no-store.response_outcome_unknownwithout dispatching again. Storage errors return 503 and never trigger a provider retry.response.completedevent, a clean end of stream and accounting settlement. The terminal event is held until storage succeeds.retention_conflictbefore dispatch for hosted opt-in and gateway continuation. It also blocks retrieval, but explicit deletion still works.hosted_responses(migration0028) holds the owner, status, provider/model, parent, policy revision, expiry, body pointer, length and SHA-256.responses-state/in the existingmultillm_mediabucket./v1/managed-state/responsesoperation.Details are in
docs/responses-state.md.Wiring
services/gateway_extensions.py:routes/unified.py:dispatch_protocol_requestgains one decorator,@with_responses_state. The file is now exactly 1,000 lines.services/protocol_translation/responses.py:response_objectgets one small call site that rewrites the ID for hosted turns. It returns the same object otherwise.worker/managed-state-dispatch.mjsroutes/v1/managed-state/responsestohandleResponsesStateRequest.Shared registrations
tests/test_responses_state_worker.mjsintest:worker.worker/responses-state-d1.mjsin the edge module inventory;app.route('/v1/responses/<response_id>', methods=['GET', 'DELETE'])in the Flask route inventory.HOSTED_RESPONSES_ENABLEDin the Container env keys.hosted_responsesinREQUIRED_D1_TABLES.tests/test_wave4_registrar.pynow expectsresponses_state_hookbetween the retention and deadline hooks.Stacked on #57
Migration numbers must be contiguous in each PR, so this branch stacks on
feat/gateway-batches(#57, migration 0027). That one stacks on #56, #54 and #50. Merge those first.Tests
tests/test_responses_state.pyandtests/test_responses_state_worker.mjscover:npm run test:worker: 1,560 passed.npm run test:integration: 7 passed.migrations:check(28 contiguous, additive migrations),test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.tests/test_frontend_build.mjsfails locally because@tailwindcss/cliis not installed. It fails the same way on main.Apply the migration before enabling the flag:
npx wrangler d1 migrations apply multillm-intelligence --remote.CI notes
python/nodejobs: the dependency-audit steps fail on every branch, including main. fix(deps): clear the CI dependency audit advisories #42 fixes that.AdmissionCoordinatorDurable Object migration, whichwrangler versions uploadcannot apply. This is expected untilwrangler deployruns on main.This is part of Wave 5. The README line comes with the Wave 5 integration PR.