Skip to content

feat(retention): add opt-in zero-content retention policy - #33

Merged
ALikesToCode merged 1 commit into
integrate/gateway-waves-1-2from
feat/zero-content-retention
Oct 9, 2026
Merged

ALikesToCode merged 1 commit into
integrate/gateway-waves-1-2from
feat/zero-content-retention

Conversation

@ALikesToCode

Copy link
Copy Markdown
Owner

Summary

Adds a zero-content retention policy behind CONTENT_RETENTION_ENABLED, which defaults to false. While the flag is off, the policy JSON and the X-MultiLLM-Retention header are ignored, and caching, shadow sampling, roleplay, Knowledge memos and handoffs behave as today.

  • With the flag on, CONTENT_RETENTION_POLICY_JSON sets inherit or zero per default, per verified key (account ID or key digest) and per exact route.
  • X-MultiLLM-Retention: zero can tighten the policy. No header value can loosen it.
  • For zero requests:
    • The chat response cache is bypassed for reads and writes.
    • Shadow samples are never built or queued.
    • Roleplay turns use request-local state. Only numeric counters and content-free traces are written.
    • Recovery returns 409 recovery_unavailable, and memory operations return 409 retention_forbidden.
    • Knowledge memos and embeddings are skipped, and handoff saves return 409 retention_forbidden before parsing or storage.
  • Completion bytes and errors otherwise keep their meaning, and no retry permission is added.
  • This is a prospective gateway storage policy. It does not change provider-side retention or delete stored data.

Details are in docs/zero-content-retention.md.

Integration

The Wave 3 integration PR wires the rest:

  • Flask policy resolution after authentication;
  • the verified roleplay identity headers at the edge;
  • the Knowledge dispatcher transport.

Until then, the Worker paths apply only the configured default policy.

Shared registrations

  • Added tests/test_zero_content_retention_worker.mjs to test:worker.
  • Added CONTENT_RETENTION_ENABLED and CONTENT_RETENTION_POLICY_JSON to the Container env keys.
  • Added worker/retention-policy.mjs to the secret firewall inventory.
  • Flask and Worker CORS changes:
    • now allow the X-MultiLLM-Retention request header;
    • now expose X-MultiLLM-Retention and X-MultiLLM-Roleplay-Recovery.

Tests

  • tests/test_zero_content_retention.py and tests/test_zero_content_retention_worker.mjs: new.
  • Full Python suite: 3,198 passed, 2 skipped. ruff, mypy and bandit are clean.
  • npm run test:worker: 1,241 passed, including the existing roleplay and Knowledge suites.
  • cf:migrations:check, test:deploy, free routes, client headers and the wrangler deploy --dry-run bundle all pass.
  • tests/test_frontend_build.mjs fails locally because @tailwindcss/cli is not installed. It fails the same way on main.

This is part of Wave 3. It is based on the Wave 2 integration branch (#25), and the Wave 3 integration PR adds the README line.

Resolve an immutable inherit/zero retention policy per default, verified key, route and request header behind CONTENT_RETENTION_ENABLED, default off. Zero requests bypass the chat cache and shadow sampling, keep roleplay state request-local and skip Knowledge memo and handoff writes.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-08T22:13:17.680017Z 655eaba PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 09949db9-642c-4071-afd9-f5f6ef50d28f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
multillm-proxy 655eaba Oct 08 2026, 10:09 PM

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 655eabaa2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +64 to +67
export function retentionKnowledgeOptions(policy, options = {}) {
if (!policy?.enabled) return options;
return { ...options, retentionPolicy: policy,
...(!retentionAllowsContent(policy) ? { cache: NO_CONTENT_CACHE } : {}) };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Apply zero retention to the production Knowledge cache

When the configured default policy is zero, production Knowledge searches still read and write globalThis.caches.default: dispatchKnowledge passes its original options to retrieveKnowledge, and this helper is never called outside tests. Consequently, private query bundles and excerpts can be replayed or persisted despite zero retention. Resolve the request policy and apply these options before entering retrieval.

Useful? React with 👍 / 👎.

Comment on lines +63 to +64
call(operation, payload, now = Date.now(), retentionPolicy = null) {
if (operation === "save" && !retentionAllowsContent(retentionPolicy)) fail("retention_forbidden", "Handoff content cannot be saved under zero retention.", 409);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Enforce the default policy on production handoff saves

With CONTENT_RETENTION_POLICY_JSON defaulting to zero, production handoff saves still reach this method with retentionPolicy === null: KnowledgeHandoffs.fetch calls this.handoffs.call(body.operation, body.payload) and the dispatcher transports no policy. Since null allows content, the save is parsed and written to SQL instead of returning retention_forbidden. Resolve the configured policy in this path or require the dispatcher to transport it.

Useful? React with 👍 / 👎.

Comment on lines +46 to +51
export function resolveRoleplayRetention(env, request) {
return resolveRetentionPolicy(env, {
keyId: request.headers.get("X-MultiLLM-Retention-Key-ID") ?? "",
keyHash: request.headers.get("X-MultiLLM-Retention-Key-Hash") ?? "",
route: request.headers.get("X-MultiLLM-Retention-Route") ?? new URL(request.url).pathname,
header: request.headers.get(RETENTION_HEADER) ?? "",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Forward the caller's retention opt-out to roleplay sessions

For public roleplay turns with an inheriting server default, X-MultiLLM-Retention: zero never reaches this resolver: handleRoleplayEdgeRequest rebuilds the internal headers with clientContextHeaders, whose allowlist omits the retention header. The durable object therefore resolves inherit and persists conversation state even though the authenticated caller opted out. The edge must validate and copy the retention header onto the internal request.

Useful? React with 👍 / 👎.

Comment on lines +57 to +58
const safe = structuredClone(this.records.filter(item => item.finishedAt && item.retention === "zero"));
this.writes = this.writes.catch(() => {}).then(() => this.storage.put("zero-turn-traces-v1", safe));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve zero-retention traces across object restarts

After a Durable Object hibernates, TurnTraceJournal reloads only operator_traces_v1, not zero-turn-traces-v1. The next zero-retention completion builds safe solely from the new in-memory records and overwrites the zero-trace key, discarding all previously persisted zero traces; those records also disappear from the timeline after restart. Load and merge the separate zero-trace journal before replacing it.

Useful? React with 👍 / 👎.

Comment on lines +223 to 225
const suppliedIdempotencyKey = request.headers.get("Idempotency-Key") ?? "";
const idempotencyKey = retentionAllowsContent(retention) ? suppliedIdempotencyKey : await retentionRequestId(suppliedIdempotencyKey);
const duplicate = existingRoleplayRequest(state, idempotencyKey);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve idempotency across retention-mode changes

When the same roleplay session first sends an idempotency key under zero and later retries it under inherit (or vice versa), the requests use incompatible identities and repositories: the zero path stores a SHA-256 digest in the counter record, while the normal path checks the raw key in conversation state. Neither lookup sees the other, so the second request reaches the provider and can bill twice despite using the same idempotency key. Record a common content-free identity that both policy modes consult.

Useful? React with 👍 / 👎.

Comment on lines +84 to +85
if policy.enabled:
g.multillm_retention_policy = {"mode": policy.mode, "revision": policy.revision}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep inherited retention compatible with the v1 cache

If CONTENT_RETENTION_ENABLED=true is combined with RESPONSE_CACHE_POLICY_REVISION=v1, even an effective inherit policy disables every otherwise eligible cache request. This assignment replaces the legacy snapshot with {mode: "inherit", ...}, while services/cache_policy._policy_snapshot accepts only mode === "legacy" and therefore returns None; the wrapper consequently bypasses both reads and writes. Extend the isolated-cache policy contract to represent enabled inherited retention rather than treating it as unavailable.

Useful? React with 👍 / 👎.

@ALikesToCode
ALikesToCode merged commit 37a7eef into integrate/gateway-waves-1-2 Oct 9, 2026
5 of 7 checks passed
@ALikesToCode
ALikesToCode deleted the feat/zero-content-retention branch October 26, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant