Repository navigation
feat(retention): add opt-in zero-content retention policy - #33
Conversation
Resolve an immutable inherit/zero retention policy per default, verified key, route and request header behind CONTENT_RETENTION_ENABLED, default off. Zero requests bypass the chat cache and shadow sampling, keep roleplay state request-local and skip Knowledge memo and handoff writes.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
multillm-proxy | 655eaba | Oct 08 2026, 10:09 PM |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 655eabaa2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| export function retentionKnowledgeOptions(policy, options = {}) { | ||
| if (!policy?.enabled) return options; | ||
| return { ...options, retentionPolicy: policy, | ||
| ...(!retentionAllowsContent(policy) ? { cache: NO_CONTENT_CACHE } : {}) }; |
There was a problem hiding this comment.
Apply zero retention to the production Knowledge cache
When the configured default policy is zero, production Knowledge searches still read and write globalThis.caches.default: dispatchKnowledge passes its original options to retrieveKnowledge, and this helper is never called outside tests. Consequently, private query bundles and excerpts can be replayed or persisted despite zero retention. Resolve the request policy and apply these options before entering retrieval.
Useful? React with 👍 / 👎.
| call(operation, payload, now = Date.now(), retentionPolicy = null) { | ||
| if (operation === "save" && !retentionAllowsContent(retentionPolicy)) fail("retention_forbidden", "Handoff content cannot be saved under zero retention.", 409); |
There was a problem hiding this comment.
Enforce the default policy on production handoff saves
With CONTENT_RETENTION_POLICY_JSON defaulting to zero, production handoff saves still reach this method with retentionPolicy === null: KnowledgeHandoffs.fetch calls this.handoffs.call(body.operation, body.payload) and the dispatcher transports no policy. Since null allows content, the save is parsed and written to SQL instead of returning retention_forbidden. Resolve the configured policy in this path or require the dispatcher to transport it.
Useful? React with 👍 / 👎.
| export function resolveRoleplayRetention(env, request) { | ||
| return resolveRetentionPolicy(env, { | ||
| keyId: request.headers.get("X-MultiLLM-Retention-Key-ID") ?? "", | ||
| keyHash: request.headers.get("X-MultiLLM-Retention-Key-Hash") ?? "", | ||
| route: request.headers.get("X-MultiLLM-Retention-Route") ?? new URL(request.url).pathname, | ||
| header: request.headers.get(RETENTION_HEADER) ?? "", |
There was a problem hiding this comment.
Forward the caller's retention opt-out to roleplay sessions
For public roleplay turns with an inheriting server default, X-MultiLLM-Retention: zero never reaches this resolver: handleRoleplayEdgeRequest rebuilds the internal headers with clientContextHeaders, whose allowlist omits the retention header. The durable object therefore resolves inherit and persists conversation state even though the authenticated caller opted out. The edge must validate and copy the retention header onto the internal request.
Useful? React with 👍 / 👎.
| const safe = structuredClone(this.records.filter(item => item.finishedAt && item.retention === "zero")); | ||
| this.writes = this.writes.catch(() => {}).then(() => this.storage.put("zero-turn-traces-v1", safe)); |
There was a problem hiding this comment.
Preserve zero-retention traces across object restarts
After a Durable Object hibernates, TurnTraceJournal reloads only operator_traces_v1, not zero-turn-traces-v1. The next zero-retention completion builds safe solely from the new in-memory records and overwrites the zero-trace key, discarding all previously persisted zero traces; those records also disappear from the timeline after restart. Load and merge the separate zero-trace journal before replacing it.
Useful? React with 👍 / 👎.
| const suppliedIdempotencyKey = request.headers.get("Idempotency-Key") ?? ""; | ||
| const idempotencyKey = retentionAllowsContent(retention) ? suppliedIdempotencyKey : await retentionRequestId(suppliedIdempotencyKey); | ||
| const duplicate = existingRoleplayRequest(state, idempotencyKey); |
There was a problem hiding this comment.
Preserve idempotency across retention-mode changes
When the same roleplay session first sends an idempotency key under zero and later retries it under inherit (or vice versa), the requests use incompatible identities and repositories: the zero path stores a SHA-256 digest in the counter record, while the normal path checks the raw key in conversation state. Neither lookup sees the other, so the second request reaches the provider and can bill twice despite using the same idempotency key. Record a common content-free identity that both policy modes consult.
Useful? React with 👍 / 👎.
| if policy.enabled: | ||
| g.multillm_retention_policy = {"mode": policy.mode, "revision": policy.revision} |
There was a problem hiding this comment.
Keep inherited retention compatible with the v1 cache
If CONTENT_RETENTION_ENABLED=true is combined with RESPONSE_CACHE_POLICY_REVISION=v1, even an effective inherit policy disables every otherwise eligible cache request. This assignment replaces the legacy snapshot with {mode: "inherit", ...}, while services/cache_policy._policy_snapshot accepts only mode === "legacy" and therefore returns None; the wrapper consequently bypasses both reads and writes. Extend the isolated-cache policy contract to represent enabled inherited retention rather than treating it as unavailable.
Useful? React with 👍 / 👎.
37a7eef
into
integrate/gateway-waves-1-2
Summary
Adds a zero-content retention policy behind
CONTENT_RETENTION_ENABLED, which defaults to false. While the flag is off, the policy JSON and theX-MultiLLM-Retentionheader are ignored, and caching, shadow sampling, roleplay, Knowledge memos and handoffs behave as today.CONTENT_RETENTION_POLICY_JSONsetsinheritorzeroper default, per verified key (account ID or key digest) and per exact route.X-MultiLLM-Retention: zerocan tighten the policy. No header value can loosen it.recovery_unavailable, and memory operations return 409retention_forbidden.retention_forbiddenbefore parsing or storage.Details are in
docs/zero-content-retention.md.Integration
The Wave 3 integration PR wires the rest:
Until then, the Worker paths apply only the configured default policy.
Shared registrations
tests/test_zero_content_retention_worker.mjstotest:worker.CONTENT_RETENTION_ENABLEDandCONTENT_RETENTION_POLICY_JSONto the Container env keys.worker/retention-policy.mjsto the secret firewall inventory.X-MultiLLM-Retentionrequest header;X-MultiLLM-RetentionandX-MultiLLM-Roleplay-Recovery.Tests
tests/test_zero_content_retention.pyandtests/test_zero_content_retention_worker.mjs: new.npm run test:worker: 1,241 passed, including the existing roleplay and Knowledge suites.cf:migrations:check,test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.tests/test_frontend_build.mjsfails locally because@tailwindcss/cliis not installed. It fails the same way on main.This is part of Wave 3. It is based on the Wave 2 integration branch (#25), and the Wave 3 integration PR adds the README line.