Repository navigation
feat(retention): add opt-in zero-content retention policy #33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ALikesToCode
merged 1 commit into
integrate/gateway-waves-1-2
from
feat/zero-content-retention
Oct 9, 2026
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| # Zero-content retention | ||
|
|
||
| `CONTENT_RETENTION_ENABLED` defaults to `false`. While off, the retention header | ||
| and policy JSON are ignored. Cache, sampling, roleplay, memo and handoff behavior | ||
| stays unchanged. An empty flag means off; an empty policy means `{}`. | ||
| Malformed settings disable these controls and log a warning once per setting, | ||
| without logging its value. Invalid configuration therefore does not provide a | ||
| retention guarantee: validate settings before enabling the feature. | ||
|
|
||
| When enabled, `CONTENT_RETENTION_POLICY_JSON` accepts this shape: | ||
|
|
||
| ```json | ||
| { | ||
| "default": "inherit", | ||
| "keys": { "verified-key-id-or-sha256": "zero" }, | ||
| "routes": { "/v1/chat/completions": "zero" } | ||
| } | ||
| ``` | ||
|
|
||
| Each value is `inherit` or `zero`; route matching is exact. Key selectors are | ||
| server-authenticated account IDs (username when no ID exists) or SHA-256 digests | ||
| of the authenticated presented key. Use a key digest for individual keys within | ||
| one account. Never put raw keys in the policy. The tightest default, key, route | ||
| and header wins. `X-MultiLLM-Retention: zero` tightens the policy; | ||
| `X-MultiLLM-Retention: inherit` and unknown header values cannot loosen it. | ||
|
|
||
| For example, with the flag off, `X-MultiLLM-Cache: on` still caches an eligible | ||
| completion even if the caller sends `X-MultiLLM-Retention: zero`. With the flag | ||
| on and policy `{}`, that same retention header bypasses both cache reads and | ||
| writes, and bypasses shadow sample construction before a stream collector or | ||
| background queue can capture content. With a configured zero key or route, | ||
| sending `inherit` still bypasses retention. | ||
|
|
||
| Zero roleplay turns use fresh request-local conversation state. Callers must | ||
| supply the conversation history needed for each turn. They cannot recover a | ||
| durable conversation, import a branch or use operator memory under that policy. | ||
| Turn responses expose `X-MultiLLM-Retention: zero` and | ||
| `X-MultiLLM-Roleplay-Recovery: unavailable`. Recovery returns HTTP 409 with | ||
| `recovery_unavailable`; memory operations return HTTP 409 with | ||
| `retention_forbidden`. Errors and completion bytes otherwise keep their existing | ||
| meaning. No replacement completion or new retry permission is introduced. | ||
|
|
||
| Only numeric roleplay counters, bounded numeric model observations and hashed | ||
| request/model identities are written to a separate counter record. Content-free | ||
| traces exclude parameter receipts, retain at most 30 turns, and write separately | ||
| from existing traces. Zero turns neither populate the conversation cache nor | ||
| refresh the existing session deletion alarm. Old content, checkpoints, recovery | ||
| records and alarms are left in place; an already scheduled alarm can still run | ||
| under its original lifecycle. The feature does not scan or delete stored data. | ||
|
|
||
| Knowledge memo lookup, embeddings, bundle construction and background writes are | ||
| disabled for zero requests. A derived handoff save returns HTTP 409 | ||
| `retention_forbidden` before parsing or SQL mutation. It returns no saved ID. | ||
| Explicit source-corpus ingestion remains separately authorized and is unaffected. | ||
|
|
||
| The gateway must still hold and send content while serving a request, including | ||
| streaming and existing turn scheduling. Usage, cost, security and health counters | ||
| continue; provider calls can incur their normal cost. This is a prospective | ||
| gateway storage policy, not a provider-side retention agreement, secure memory | ||
| erasure, deletion API or prohibition on caller-authorized corpus storage. It does | ||
| not change provider prompt-cache settings or cover media storage and future | ||
| state/batch features; those features must adopt the same policy explicitly. | ||
|
|
||
| ## Where the policy applies | ||
|
|
||
| - Flask resolves the policy after authentication and before cache and dispatch | ||
| hooks, and exposes it as `g.multillm_retention_policy`. The chat response cache | ||
| and shadow sampling check it before reading, writing or queueing content. | ||
| - The Worker resolves the policy only from the verified key identity and the | ||
| public request path. It ignores identity in caller payloads and caller-set | ||
| `X-MultiLLM-Retention-Key-ID`, `X-MultiLLM-Retention-Key-Hash` and | ||
| `X-MultiLLM-Retention-Route` headers, and sets those headers itself for | ||
| roleplay turns, operator recovery and memory requests. | ||
| - Knowledge retrieval uses a non-storing cache for zero requests. Handoff saves | ||
| are rejected before content is serialized into a Durable Object call. | ||
| - Set `CONTENT_RETENTION_ENABLED` and `CONTENT_RETENTION_POLICY_JSON` on both the | ||
| Worker and the Container. No routes, D1 tables or migrations are added. | ||
| - Browser clients can send `X-MultiLLM-Retention` and read | ||
| `X-MultiLLM-Retention` and `X-MultiLLM-Roleplay-Recovery`. | ||
|
|
||
| Any feature that stores content must check the same policy before it builds a | ||
| record, queues content or persists it. | ||
|
|
||
| Tests use fake providers and storage. They do not verify a deployment or | ||
| provider-side retention. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| """Immutable, prospective retention decisions resolved after authentication.""" | ||
| from __future__ import annotations | ||
|
|
||
| import hashlib | ||
| import json | ||
| import logging | ||
| import os | ||
| from dataclasses import dataclass | ||
| from functools import lru_cache | ||
| from typing import Mapping | ||
|
|
||
| from flask import g, has_request_context, request | ||
|
|
||
| HEADER = "X-MultiLLM-Retention" | ||
| _MODES = {"inherit", "zero"} | ||
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class RetentionPolicy: | ||
| mode: str = "inherit" | ||
| enabled: bool = False | ||
| revision: str = "legacy" | ||
|
|
||
| @property | ||
| def allows_content(self) -> bool: | ||
| return not self.enabled or self.mode != "zero" | ||
|
|
||
|
|
||
| @lru_cache(maxsize=2) | ||
| def _warn_once(setting: str) -> None: | ||
| logger.warning("Invalid %s; content retention controls disabled", setting) | ||
|
|
||
|
|
||
| def _rules(value: object) -> bool: | ||
| if not isinstance(value, dict): | ||
| return False | ||
| return all(isinstance(key, str) and bool(key) and isinstance(mode, str) and mode in _MODES | ||
| for key, mode in value.items()) | ||
|
|
||
|
|
||
| def resolve_policy(env: Mapping[str, str] | None = None, *, key_id: str = "", | ||
| route: str = "", header: str = "", key_hash: str = "") -> RetentionPolicy: | ||
| """Tightest of default, authenticated key, exact route and caller opt-out.""" | ||
| env = os.environ if env is None else env | ||
| flag = str(env.get("CONTENT_RETENTION_ENABLED", "")).strip().lower() | ||
| if flag in {"", "0", "false", "no", "off"}: | ||
| return RetentionPolicy() | ||
| if flag not in {"1", "true", "yes", "on"}: | ||
| _warn_once("CONTENT_RETENTION_ENABLED") | ||
| return RetentionPolicy() | ||
| raw = env.get("CONTENT_RETENTION_POLICY_JSON", "").strip() or "{}" | ||
| try: | ||
| config = json.loads(raw) | ||
| if (not isinstance(config, dict) or set(config) - {"default", "keys", "routes"} | ||
| or not isinstance(config.get("default", "inherit"), str) | ||
| or config.get("default", "inherit") not in _MODES | ||
| or not _rules(config.get("keys", {})) or not _rules(config.get("routes", {}))): | ||
| raise ValueError("Invalid retention configuration") | ||
| except (ValueError, TypeError): | ||
| _warn_once("CONTENT_RETENTION_POLICY_JSON") | ||
| return RetentionPolicy() | ||
| modes = [config.get("default", "inherit"), config.get("routes", {}).get(route), | ||
| config.get("keys", {}).get(key_id), config.get("keys", {}).get(key_hash), | ||
| str(header).strip().lower()] | ||
| revision = hashlib.sha256(json.dumps(config, sort_keys=True, separators=(",", ":")).encode()).hexdigest() | ||
| return RetentionPolicy("zero" if "zero" in modes else "inherit", True, revision) | ||
|
|
||
|
|
||
| def request_policy() -> RetentionPolicy: | ||
| """Snapshot once per request; stream callbacks must capture the returned value.""" | ||
| if not has_request_context(): | ||
| return RetentionPolicy() | ||
| captured = getattr(g, "multillm_content_retention", None) | ||
| if isinstance(captured, RetentionPolicy): | ||
| return captured | ||
| user = getattr(g, "authenticated_user", None) or {} | ||
| from route_helpers import request_api_key | ||
| key = request_api_key() if user else "" | ||
| key_hash = hashlib.sha256(key.encode()).hexdigest() if key else "" | ||
| policy = resolve_policy(key_id=str(user.get("id") or user.get("username") or ""), | ||
| key_hash=key_hash, route=request.path, header=request.headers.get(HEADER, "")) | ||
| g.multillm_content_retention = policy | ||
| if policy.enabled: | ||
| g.multillm_retention_policy = {"mode": policy.mode, "revision": policy.revision} | ||
| return policy | ||
|
|
||
|
|
||
| def init_retention_policy(app) -> None: | ||
| """Coordinator hook: call after authentication and before managed dispatch.""" | ||
| @app.before_request | ||
| def capture_retention(): | ||
| if getattr(g, "authenticated_user", None): | ||
| request_policy() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If
CONTENT_RETENTION_ENABLED=trueis combined withRESPONSE_CACHE_POLICY_REVISION=v1, even an effectiveinheritpolicy disables every otherwise eligible cache request. This assignment replaces the legacy snapshot with{mode: "inherit", ...}, whileservices/cache_policy._policy_snapshotaccepts onlymode === "legacy"and therefore returnsNone; the wrapper consequently bypasses both reads and writes. Extend the isolated-cache policy contract to represent enabled inherited retention rather than treating it as unavailable.Useful? React with 👍 / 👎.