diff --git a/docs/pii-rehydration.md b/docs/pii-rehydration.md new file mode 100644 index 00000000..c6c6b574 --- /dev/null +++ b/docs/pii-rehydration.md @@ -0,0 +1,123 @@ +# PII redaction and streaming rehydration + +Opted-in managed generations replace detected PII before provider dispatch and +restore request-owned placeholders in JSON responses and SSE text. The credential +secret firewall runs first. Its findings, including values in secret-named fields, +never enter the reversible PII map, even when secret scanning observes rather than +redacts or blocks them. + +## Configuration + +The defaults leave traffic unchanged: + +```text +PII_REDACTION_ENABLED=false +PII_REDACTION_POLICY_JSON={} +``` + +Empty values mean these defaults. Invalid flags or policies disable redaction and +produce one warning per configuration field, without including its value. Enabling +the flag with an empty policy opts in nothing. Unknown policy fields are invalid. + +For example, opt in a managed route: + +```text +PII_REDACTION_ENABLED=true +PII_REDACTION_POLICY_JSON={"routes":["/v1/chat/completions"],"keys":[],"detectors":["email","phone","card"],"mode":"required"} +``` + +Or opt in an authenticated key scope, using its stable internal identifier rather +than the credential itself: + +```text +PII_REDACTION_POLICY_JSON={"routes":[],"keys":["key-7"],"detectors":["email"],"mode":"best_effort"} +``` + +Matching either an exact route or a key scope enables the selected detectors. An +empty detector list disables transformation. Flask scopes use the authenticated +user ID, falling back to the username. Native roleplay transport consumes the +authenticated scope supplied as `settings.piiKeyScope`; it never infers a scope +from a caller header. Route opt-in applies to all authenticated callers on that +route. Each list accepts at most 512 strings, each at most 256 characters; the +policy JSON accepts at most 64 KiB. + +Managed Flask routes are `/v1/chat/completions`, `/v1/messages`, `/v1/responses`, +and `/intelligence/v1/chat/completions`. Native roleplay uses +`/v1/roleplay/chat/completions`. Raw passthrough is never transformed. Register +the Flask request hook after authentication and retention policy, before +idempotency claims and generation-cache lookup. Forward both configuration +variables to the Flask Container when using it behind the Worker. + +## Detector coverage + +Detectors examine JSON string values recursively, preserving object keys and +non-string values. Email matching uses bounded ASCII local parts and domain +labels; it does not recognize every valid international email address. Phone +matching accepts 10–15 digits with a leading plus or common spacing, parentheses, +or hyphens. Bare digit strings are not treated as phones. Card matching accepts +13–19 digits passing Luhn, excluding repeated identical digits. Matches do not +prove that a phone or card exists. Long, ambiguous numeric runs and formats outside +these limits can remain unchanged. There is no remote or semantic classifier. + +## Request-local placeholders + +The format is `__MLPII_<64 lowercase hexadecimal characters>__` (74 ASCII bytes). +The hexadecimal portion is HMAC-SHA-256 of the original value under a random +32-byte secret generated for that request. Repeated values share a placeholder +within one request. Separate requests have separate secrets and maps; they cannot +restore each other's tokens. Only exact tokens in the current request's map are +restored. Unknown or altered complete tokens pass through unchanged. + +The map and secret remain in memory and are cleared on completion, error, or +disconnect, including cancellation before the first response chunk. They are not +logged, exported, cached, stored, or reused. A request permits at most 512 distinct +values and 1 MiB of serialized input, with a maximum JSON depth of 32 and 32,768 +visited nodes. A collision fails the entire transformation. + +## Streaming and errors + +Restoration handles arbitrary network chunk boundaries, UTF-8 splits, JSON +escapes, and placeholder fragments across successive text deltas. It carries at +most 128 bytes of unresolved token prefixes across at most 16 text lanes. SSE +frames are limited to 64 KiB, nonstream JSON bodies to 1 MiB, and buffered output +from a single transport read to 1 MiB. Parser traversal +has the same depth and node limits as request traversal. Restored values are +serialized with JSON escaping. SSE events can move withheld text into a later +delta; their concatenated text retains the original order. + +`required` is the default mode. A transformation or bound failure returns HTTP +422 with code `pii_redaction_failed` before dispatch. In `best_effort` mode, an +atomic transformation failure dispatches the unchanged firewall-checked payload +and records that redaction was skipped. A firewall refusal remains a refusal in +both modes. Policy validation errors disable the feature, rather than applying +required-mode rejection to existing traffic. + +Invalid response JSON, a parser bound failure, or an unresolved terminal prefix +returns HTTP 502 with code `pii_rehydration_failed` for nonstream responses. A +stream that has already started is terminated with an error and its map cleared; +it does not invent a completion or authorize replay. Provider output must echo an +issued placeholder exactly to restore it. Rewritten or invented tokens cannot be +recovered. A trailing incomplete prefix is never emitted as successful text. + +## Cache, replay, and operating limits + +Redacted Flask requests use request-local zero-content retention, which bypasses +the exact and shared generation caches. An `Idempotency-Key` on a request that +actually requires redaction returns HTTP 400, `pii_idempotency_unsupported`, before +dispatch or replay. Requests with no PII retain their existing behavior. + +Native roleplay transport reports `cacheable: false` and `replayable: false` +through `settings.onPIIDecision` for redacted requests. Its enclosing persistence +layer must honor that decision before storing any generation or replay body. +The transformed upstream request omits `Idempotency-Key`, and its request and +response carry `Cache-Control: no-store`. The request-local map is never part of +conversation state. Normal conversation retention is governed separately by the +content-retention policy. + +Redaction adds local scanning, HMAC, and parsing work without classifier charges. +Long placeholders can increase provider input tokens and cost. Nonstream +restoration buffers a bounded body; streaming buffers a bounded frame and may +delay text while a token prefix is unresolved. This feature neither certifies a +provider's privacy guarantees nor removes the need to select the appropriate +retention policy. Verify detector coverage and provider echo behavior for the +formats used by the application. diff --git a/package.json b/package.json index 4dc35c96..0a93fcdf 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,7 @@ "lint": "ruff check . --select E9,F63,F7,F82", "typecheck": "mypy --ignore-missing-imports --follow-imports=skip --disable-error-code import-untyped --disable-error-code var-annotated --disable-error-code arg-type --disable-error-code assignment --disable-error-code operator app.py routes services providers streaming", "test": "python -m pytest -q", - "test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs", + "test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs tests/test_pii_rehydration_worker.mjs", "db:migrate": "python scripts/validate_sqlite_schema.py --apply", "db:validate": "python scripts/validate_sqlite_schema.py", "security:secrets": "python scripts/check_static_secrets.py", diff --git a/services/managed_dispatch.py b/services/managed_dispatch.py index ad0490fc..d48426a8 100644 --- a/services/managed_dispatch.py +++ b/services/managed_dispatch.py @@ -23,13 +23,16 @@ def execute_managed_attempt(send, model, credential): from services.managed_turn import before_provider_submission before_provider_submission() scope = current_scope() - if scope is None: - return send() try: response = send() except BaseException: - scope.observer(model, credential)(classify_upstream_outcome(transport_failure="interrupted")) + if scope is not None: + scope.observer(model, credential)(classify_upstream_outcome(transport_failure="interrupted")) raise + from services.pii_redaction import current_context + context = current_context() + if context is not None: + response.multillm_pii_context = context return capture_managed_attempt(response, model, credential) @@ -167,6 +170,11 @@ def close(self): def observe_managed_response(downstream, upstream, *, stream=False): """Deliver classified final outcomes through the managed-dispatch callback.""" + from services.pii_redaction import current_context + from services.pii_stream import rehydrate_response + context = getattr(upstream, "multillm_pii_context", None) or current_context() + if context is not None: + downstream = rehydrate_response(downstream, context, stream=stream) from services.managed_turn import defer_affinity_response if defer_affinity_response(downstream, upstream, stream=stream): return downstream diff --git a/services/pii_redaction.py b/services/pii_redaction.py new file mode 100644 index 00000000..d167a55d --- /dev/null +++ b/services/pii_redaction.py @@ -0,0 +1,249 @@ +"""Deterministic, opt-in PII transformation with request-local ownership.""" + +from __future__ import annotations + +import hashlib +import hmac +import json +import logging +import os +import re +import secrets +from dataclasses import dataclass + +from services.secret_scan import scan_payload, scan_text + +MAX_INPUT_BYTES = 1024 * 1024 +MAX_VALUES = 512 +PREFIX = "__MLPII_" +TOKEN = re.compile(r"__MLPII_[0-9a-f]{64}__") +_EMAIL = re.compile(r"(? 65536: + raise ValueError + policy = json.loads(raw) + if not isinstance(policy, dict) or set(policy) - {"routes", "keys", "detectors", "mode"}: + raise ValueError + for name in ("routes", "keys", "detectors"): + items = policy.get(name, []) + if not isinstance(items, list) or len(items) > 512 or any( + not isinstance(item, str) or not 1 <= len(item) <= 256 for item in items + ): + raise ValueError + if set(policy.get("detectors", [])) - {"email", "phone", "card"}: + raise ValueError + mode = policy.get("mode", "required") + if mode not in {"required", "best_effort"}: + raise ValueError + if route not in policy.get("routes", []) and (not key_scope or key_scope not in policy.get("keys", [])): + return None + detectors = tuple(policy.get("detectors", [])) + return PIIPolicy(detectors, mode) if detectors else None + except (ValueError, TypeError, RecursionError): + _warn_once("PII_REDACTION_POLICY_JSON") + return None + + +def _luhn(digits): + total = 0 + for index, char in enumerate(reversed(digits)): + value = int(char) * (2 if index % 2 else 1) + total += value - 9 if value > 9 else value + return total % 10 == 0 and len(set(digits)) > 1 + + +def _findings(text, detectors): + protected = [(item["start"], item["end"]) for item in scan_text(text)] + found = [] + if "email" in detectors: + found.extend((match.start(), match.end()) for match in _EMAIL.finditer(text)) + if "phone" in detectors or "card" in detectors: + for match in _NUMBER.finditer(text): + value = match.group() + digits = "".join(char for char in value if char.isascii() and char.isdigit()) + card = 13 <= len(digits) <= 19 and _luhn(digits) + phone = 10 <= len(digits) <= 15 and (value.startswith("+") or any(char in value for char in " ()-")) + if "card" in detectors and card or "phone" in detectors and phone and len(digits) <= 15: + found.append((match.start(), match.end())) + end = -1 + for start, stop in sorted(found, key=lambda span: (span[0], -span[1])): + if start < end or any(start < right and stop > left for left, right in protected): + continue + yield start, stop + end = stop + + +class PIIContext: + """Only this object owns the ephemeral secret and reversible values.""" + + def __init__(self): + self.secret = secrets.token_bytes(32) + self.values: dict[str, str] = {} + self.closed = False + + def issue(self, value): + if self.closed: + raise PIIRedactionError("PII request context is closed") + token = PREFIX + hmac.new(self.secret, value.encode(), hashlib.sha256).hexdigest() + "__" + if token in self.values and self.values[token] != value: + raise PIIRedactionError("PII placeholder collision") + if token not in self.values and len(self.values) >= MAX_VALUES: + raise PIIRedactionError("PII request exceeds the value limit") + self.values[token] = value + return token + + def restore(self, text): + return TOKEN.sub(lambda match: self.values.get(match.group(), match.group()), text) + + def close(self): + self.values.clear() + self.secret = b"" + self.closed = True + + +def map_strings(value, transform, *, depth=0, counter=None, field="", skip_secret_fields=False): + counter = [0] if counter is None else counter + counter[0] += 1 + if depth > 32 or counter[0] > 32768: + raise PIIRedactionError("PII document exceeds the structure limit") + if isinstance(value, str): + if skip_secret_fields and scan_payload({field: value})["types"].get("secret_field"): + return value + return transform(value) + if isinstance(value, list): + return [map_strings(item, transform, depth=depth + 1, counter=counter, field=field, + skip_secret_fields=skip_secret_fields) for item in value] + if isinstance(value, dict): + return {key: map_strings(item, transform, depth=depth + 1, counter=counter, field=key, + skip_secret_fields=skip_secret_fields) for key, item in value.items()} + return value + + +def prepare_payload(payload, env=None, *, route="", key_scope="", raw=False, firewall=None): + env = os.environ if env is None else env + policy = None if raw else resolve_policy(env, route=route, key_scope=key_scope) + if policy is None: + return PreparedPayload(payload) + # A firewall refusal must not be caught by the best-effort PII policy. + checked = firewall(payload) if firewall is not None else payload + context = None + try: + if len(json.dumps(checked, ensure_ascii=False, allow_nan=False).encode()) > MAX_INPUT_BYTES: + raise PIIRedactionError("PII request exceeds the input limit") + context = PIIContext() + + def redact(text): + parts, end = [], 0 + for start, stop in _findings(text, policy.detectors): + parts.extend((text[end:start], context.issue(text[start:stop]))) + end = stop + return "".join(parts) + text[end:] + + updated = map_strings(checked, redact, skip_secret_fields=True) + if not context.values: + context.close() + return PreparedPayload(checked) + return PreparedPayload(updated, context) + except Exception: + if context is not None: + context.close() + if policy.mode == "best_effort": + return PreparedPayload(checked, skipped=True) + raise PIIRedactionError("Required PII transformation failed before provider dispatch") from None + + +def current_context(): + from flask import g, has_request_context + return getattr(g, "pii_context", None) if has_request_context() else None + + +def pii_request_hook(): + """After authentication/secret policy, before request identity and caches.""" + from flask import g, jsonify, request + from services.retention_policy import RetentionPolicy + from services.secret_firewall import protect_payload + + if request.method != "POST" or request.path not in { + "/v1/chat/completions", "/v1/responses", "/v1/messages", "/intelligence/v1/chat/completions" + }: + return None + user = getattr(g, "authenticated_user", None) or {} + if not user or current_context() is not None: + return None + key_scope = str(user.get("id") or user.get("username") or "") + if resolve_policy(os.environ, route=request.path, key_scope=key_scope) is None: + return None + payload = request.get_json(silent=True) + if not isinstance(payload, dict): + return None + try: + result = prepare_payload(payload, route=request.path, key_scope=key_scope, firewall=protect_payload) + except PIIRedactionError: + return jsonify({"error": {"code": "pii_redaction_failed", "message": "Required PII transformation failed before provider dispatch"}}), 422 + g.pii_redaction_skipped = result.skipped + if result.context is None: + return None + if request.headers.get("Idempotency-Key") is not None: + result.context.close() + return jsonify({"error": {"code": "pii_idempotency_unsupported", "message": "Redacted requests cannot use idempotency replay"}}), 400 + g.pii_context = result.context + g.pii_no_persistence = True + # Existing exact/shared caches already bypass requests under zero retention. + g.multillm_content_retention = RetentionPolicy("zero", True, "pii-request") + request._cached_json = (result.payload, result.payload) + return None + + +def register_pii_redaction(app): + app.extensions.setdefault("gateway_after_authentication", []).append(pii_request_hook) + + @app.after_request + def finish_pii(response): + from services.pii_stream import rehydrate_response + context = current_context() + if context is not None and not context.closed: + return rehydrate_response(response, context) + return response + + @app.teardown_request + def abandon_pii(error): + from flask import g + context = current_context() + if context is not None and not getattr(g, "pii_stream_owned", False): + context.close() diff --git a/services/pii_stream.py b/services/pii_stream.py new file mode 100644 index 00000000..c7812c06 --- /dev/null +++ b/services/pii_stream.py @@ -0,0 +1,206 @@ +"""Bounded JSON/SSE restoration; token prefixes never escape unresolved.""" + +from __future__ import annotations + +import json +import re +from collections import deque + +from services.pii_redaction import PREFIX, TOKEN + +MAX_FRAME_BYTES = 65536 +MAX_BODY_BYTES = 1024 * 1024 +_SEPARATOR = re.compile(rb"\r\n\r\n|\n\n|\r\r") + + +class PIIStreamError(ValueError): + """Restoration cannot continue safely; close without granting replay.""" + + +class TextCarry: + def __init__(self, context): + self.context = context + self.pending: dict[tuple, str] = {} + + def text(self, value, lane): + text = self.pending.pop(lane, "") + value + output, index = [], 0 + while index < len(text): + start = text.find("_", index) + if start < 0: + output.append(text[index:]) + break + output.append(text[index:start]) + rest = text[start:start + 74] + match = TOKEN.match(rest) + if match: + output.append(self.context.restore(match.group())) + index = start + len(match.group()) + continue + partial = PREFIX.startswith(rest) or ( + rest.startswith(PREFIX) and len(rest) < 74 and + re.fullmatch(r"[0-9a-f]{0,64}_{0,2}", rest[len(PREFIX):]) is not None + ) + if partial: + if lane not in self.pending and len(self.pending) >= 16: + raise PIIStreamError("PII stream exceeds the text lane limit") + self.pending[lane] = rest + if sum(len(item.encode()) for item in self.pending.values()) > 128: + raise PIIStreamError("PII stream exceeds the prefix limit") + break + output.append("_") + index = start + 1 + return "".join(output) + + def transform(self, value, path=(), depth=0, counter=None): + counter = [0] if counter is None else counter + counter[0] += 1 + if depth > 32 or counter[0] > 32768: + raise PIIStreamError("PII stream exceeds the structure limit") + if isinstance(value, str): + return self.text(value, path) + if isinstance(value, list): + return [self.transform(item, (*path, i), depth + 1, counter) for i, item in enumerate(value)] + if isinstance(value, dict): + return {key: self.transform(item, (*path, key), depth + 1, counter) for key, item in value.items()} + return value + + def finish(self): + if self.pending: + raise PIIStreamError("PII stream ended with an unresolved placeholder prefix") + + +class RehydratingIterator: + """Own both the source iterator and ephemeral map, including unstarted close.""" + + def __init__(self, source, context, *, event_stream): + self.source = iter(source) + self.context = context + self.event_stream = event_stream + self.buffer = b"" + self.ready: deque[bytes] = deque() + self.ready_bytes = 0 + self.carry = TextCarry(context) + self.closed = False + + def __iter__(self): + return self + + def _enqueue(self, output): + self.ready_bytes += len(output) + if self.ready_bytes > MAX_BODY_BYTES: + raise PIIStreamError("PII stream exceeds the output queue limit") + self.ready.append(output) + + def _frame(self, frame): + lines = frame.splitlines(keepends=True) + indices = [i for i, line in enumerate(lines) if line.startswith(b"data:")] + if not indices: + return frame + data = b"\n".join(lines[i][5:].strip() for i in indices) + if data == b"[DONE]": + self.carry.finish() + return frame + try: + value = json.loads(data) + restored = self.carry.transform(value) + except (ValueError, UnicodeError, RecursionError) as error: + if isinstance(error, PIIStreamError): + raise + raise PIIStreamError("PII stream contains invalid JSON data") from None + if restored == value: + return frame + ending = b"\r\n" if lines[indices[0]].endswith(b"\r\n") else b"\n" + lines[indices[0]] = b"data: " + json.dumps(restored, ensure_ascii=False, separators=(",", ":")).encode() + ending + for i in indices[1:]: + lines[i] = b"" + return b"".join(lines) + + def _feed(self, chunk): + maximum = MAX_FRAME_BYTES if self.event_stream else MAX_BODY_BYTES + for start in range(0, len(chunk), 4096): + self.buffer += chunk[start:start + 4096] + if self.event_stream: + while match := _SEPARATOR.search(self.buffer): + if match.end() > maximum: + raise PIIStreamError("PII stream frame exceeds the parser limit") + frame, self.buffer = self.buffer[:match.end()], self.buffer[match.end():] + self._enqueue(self._frame(frame)) + if len(self.buffer) > maximum: + raise PIIStreamError("PII response exceeds the parser limit") + + def __next__(self): + if self.closed and not self.ready: + raise StopIteration + try: + while not self.ready: + try: + chunk = next(self.source) + except StopIteration: + if self.event_stream: + if self.buffer: + self._enqueue(self._frame(self.buffer)) + self.carry.finish() + elif self.buffer: + body = self.carry.transform(json.loads(self.buffer)) + self.carry.finish() + self._enqueue(json.dumps(body, ensure_ascii=False, separators=(",", ":")).encode()) + self.buffer = b"" + self.close(clear_ready=False) + if not self.ready: + raise + break + self._feed(chunk.encode() if isinstance(chunk, str) else chunk) + output = self.ready.popleft() + self.ready_bytes -= len(output) + return output + except StopIteration: + raise + except BaseException: + self.ready.clear() + self.close() + raise + + def close(self, *, clear_ready=True): + if clear_ready: + self.ready.clear() + self.ready_bytes = 0 + if self.closed: + return + self.closed = True + self.buffer = b"" + self.carry.pending.clear() + self.context.close() + close = getattr(self.source, "close", None) + if close is not None: + close() + + +def rehydrate_response(response, context, *, stream=False): + if context.closed or getattr(response, "multillm_pii_rehydrated", False): + return response + response.multillm_pii_rehydrated = True + if stream or response.is_streamed or response.mimetype == "text/event-stream": + iterator = RehydratingIterator(response.response, context, event_stream=stream or response.mimetype == "text/event-stream") + response.response = iterator + response.call_on_close(iterator.close) + from flask import g, has_request_context + if has_request_context(): + g.pii_stream_owned = True + else: + try: + body = response.get_data() + if len(body) > MAX_BODY_BYTES: + raise PIIStreamError("PII response exceeds the body limit") + carry = TextCarry(context) + restored = carry.transform(json.loads(body)) + carry.finish() + response.set_data(json.dumps(restored, ensure_ascii=False, separators=(",", ":")).encode()) + except (ValueError, UnicodeError, RecursionError): + response.status_code = 502 + response.content_type = "application/json" + response.set_data(b'{"error":{"code":"pii_rehydration_failed","message":"Unable to restore provider response"}}') + finally: + context.close() + response.headers.pop("Content-Length", None) + return response diff --git a/services/upstream_transport.py b/services/upstream_transport.py index 3669c79e..5301bd1e 100644 --- a/services/upstream_transport.py +++ b/services/upstream_transport.py @@ -12,9 +12,13 @@ def close_retry_response(response: requests.Response) -> None: bind_cancellation(response).close() -def iter_stream_content(response: requests.Response) -> Iterator[bytes]: +def iter_stream_content(response: requests.Response, *, pii_context=None) -> Iterator[bytes]: """Yield decoded upstream bytes as soon as the socket exposes them.""" - return CancellationIterator(lambda: _stream_content(response), bind_cancellation(response)) + source = CancellationIterator(lambda: _stream_content(response), bind_cancellation(response)) + if pii_context is None: + return source + from services.pii_stream import RehydratingIterator + return RehydratingIterator(source, pii_context, event_stream=True) def _stream_content(response: requests.Response) -> Iterator[bytes]: diff --git a/tests/fixtures/secret_firewall_coverage.json b/tests/fixtures/secret_firewall_coverage.json index 3bc276f1..e6a1fa22 100644 --- a/tests/fixtures/secret_firewall_coverage.json +++ b/tests/fixtures/secret_firewall_coverage.json @@ -197,7 +197,7 @@ "guards": 3 }, "worker/roleplay/transport.mjs": { - "dispatches": 3, + "dispatches": 4, "guards": 1 }, "worker/knowledge-edge.mjs": { @@ -307,6 +307,8 @@ "worker/opencode/reasoning-policy.mjs", "worker/opencode/reasoning-request.mjs", "worker/opencode/reasoning-response.mjs", + "worker/pii-redaction.mjs", + "worker/pii-rehydration.mjs", "worker/prometheus.mjs", "worker/prompt-templates-d1.mjs", "worker/request-telemetry.mjs", diff --git a/tests/helpers/load_cloudflare_worker.mjs b/tests/helpers/load_cloudflare_worker.mjs index 677b0cc8..ece09ac7 100644 --- a/tests/helpers/load_cloudflare_worker.mjs +++ b/tests/helpers/load_cloudflare_worker.mjs @@ -396,6 +396,8 @@ export async function roleplayModuleUrl() { const transportDataUrl = dataModuleUrl( transportSource .replace('from "../upstream-cancellation.mjs";', `from "${new URL("../../worker/upstream-cancellation.mjs", import.meta.url)}";`) + .replace('from "../pii-rehydration.mjs";', `from "${new URL("../../worker/pii-rehydration.mjs", import.meta.url)}";`) + .replace('from "../pii-redaction.mjs";', `from "${new URL("../../worker/pii-redaction.mjs", import.meta.url)}";`) .replace('from "../secret-firewall.mjs";', `from "${new URL("../../worker/secret-firewall.mjs", import.meta.url)}";`) .replace('from "./compaction-budget.mjs";', `from "${compactionBudgetUrl}";`) .replace( diff --git a/tests/test_pii_rehydration.py b/tests/test_pii_rehydration.py new file mode 100644 index 00000000..9b957a36 --- /dev/null +++ b/tests/test_pii_rehydration.py @@ -0,0 +1,369 @@ +"""Request isolation, bounded parsing and managed dispatch with fake upstreams.""" + +import importlib +import json +import os +from unittest.mock import patch + +import pytest +from flask import Flask, Response, g, request +from tests.unified_api_test_case import UnifiedApiTestCase + + +ROUTE = "/v1/chat/completions" +EMAIL = "alice@school.org" +PHONE = "+1 (415) 555-2671" +CARD = "4111 1111 1111 1111" + + +def modules(): + return importlib.import_module("services.pii_redaction"), importlib.import_module("services.pii_stream") + + +def settings(mode="required", **extra): + return {"PII_REDACTION_ENABLED": "true", "PII_REDACTION_POLICY_JSON": json.dumps({ + "routes": [ROUTE], "detectors": ["email", "phone", "card"], "mode": mode, **extra})} + + +def prepare(text, env=None, **kwargs): + redaction, _ = modules() + return redaction.prepare_payload({"messages": [{"role": "user", "content": text}]}, + settings() if env is None else env, route=ROUTE, **kwargs) + + +def test_modules_are_available(): + redaction, stream = modules() + assert callable(redaction.prepare_payload) and callable(stream.rehydrate_response) + + +@pytest.mark.parametrize("env", [{}, {"PII_REDACTION_ENABLED": ""}, + {"PII_REDACTION_ENABLED": "true", "PII_REDACTION_POLICY_JSON": ""}, settings(routes=[]), + settings(keys=["other"], routes=[])]) +def test_disabled_and_unscoped_payload_is_same_object(env): + redaction, _ = modules() + payload = {"content": EMAIL} + prepared = redaction.prepare_payload(payload, env, route=ROUTE) + assert prepared.payload is payload and prepared.context is None + + +def test_raw_never_transforms_and_key_scope_opts_in(): + result = prepare(EMAIL, settings(routes=[], keys=["key-7"]), key_scope="key-7") + assert result.context.restore(result.payload["messages"][0]["content"]) == EMAIL + assert prepare(EMAIL, raw=True).context is None + + +@pytest.mark.parametrize("value", [EMAIL, PHONE, CARD]) +def test_detector_round_trip_and_request_isolation(value): + first, second = prepare(value), prepare(value) + token = first.payload["messages"][0]["content"] + assert value not in token and token.startswith("__MLPII_") and len(token) == 74 + assert first.context.restore(token) == value + assert second.context.restore(token) == token + forged = token[:-3] + ("1" if token[-3] == "0" else "0") + "__" + assert first.context.restore(forged) != value + first.context.close() + assert first.context.restore(token) == token and not first.context.values + + +@pytest.mark.parametrize("text", ["4111 1111 1111 1112", "123", "123456789012345678901", "bad@local", "x" * 65 + "@school.org"]) +def test_false_positives_remain_unchanged(text): + assert prepare(text).payload["messages"][0]["content"] == text + + +def test_secret_findings_are_never_placeholders_even_in_observe_mode(monkeypatch): + monkeypatch.setenv("SECRET_SCAN_DEFAULT", "observe") + text = "postgres://owner:S3cur3Password42@school.org/db " + EMAIL + result = prepare(text) + assert list(result.context.values.values()) == [EMAIL] + assert result.context.restore(result.payload["messages"][0]["content"]) == text + + +def test_required_limits_and_best_effort_are_atomic(): + redaction, _ = modules() + for text in ["x" * (1024 * 1024 + 1), " ".join(f"p{i}@school.org" for i in range(513))]: + with pytest.raises(redaction.PIIRedactionError): + prepare(text) + result = prepare(text, settings("best_effort")) + assert result.context is None and result.skipped and result.payload["messages"][0]["content"] == text + + +@pytest.mark.parametrize("env", [{"PII_REDACTION_ENABLED": "broken-value"}, + {"PII_REDACTION_ENABLED": "true", "PII_REDACTION_POLICY_JSON": "broken-value"}, + settings(detectors=["semantic"]), settings(mode="invalid")]) +def test_malformed_config_off_warns_once_without_value(env, caplog): + redaction, _ = modules() + redaction._warned.clear() + prepare(EMAIL, env) + prepare(EMAIL, env) + assert len(caplog.records) == 1 and "broken-value" not in caplog.text + + +def test_nonstream_restores_escaped_json_and_unknown_tokens(): + _, stream = modules() + result = prepare(EMAIL) + token = result.payload["messages"][0]["content"] + unknown = "__MLPII_" + "0" * 64 + "__" + response = Response(json.dumps({"content": "é " + token + " " + unknown}), content_type="application/json") + stream.rehydrate_response(response, result.context) + assert response.json == {"content": "é " + EMAIL + " " + unknown} + assert not result.context.values and "Content-Length" not in response.headers + + +def stream_bytes(chunks, context): + _, stream = modules() + return b"".join(stream.RehydratingIterator(iter(chunks), context, event_stream=True)) + + +def test_every_byte_split_utf8_and_json_escape(): + result = prepare(EMAIL) + token = result.payload["messages"][0]["content"] + data = ('data: {"choices":[{"delta":{"content":"é ' + token.replace("_", "\\u005f") + + '\\n"}}]}\r\n\r\ndata: [DONE]\r\n\r\n').encode() + for split in range(1, len(data)): + result = prepare(EMAIL) + current = result.payload["messages"][0]["content"] + wire = data.replace(token.encode(), current.encode()) if token.encode() in data else data.replace( + token.replace("_", "\\u005f").encode(), current.replace("_", "\\u005f").encode()) + output = stream_bytes([wire[:split], wire[split:]], result.context) + first = json.loads(output.splitlines()[0][6:]) + assert first["choices"][0]["delta"]["content"] == "é " + EMAIL + "\n" + assert not result.context.values + + +def test_placeholder_across_events_never_emits_partial_prefix(): + result = prepare(EMAIL) + token = result.payload["messages"][0]["content"] + chunks = [f'data: {{"choices":[{{"delta":{{"content":{json.dumps(part)}}}}}]}}\n\n'.encode() + for part in ["hello " + token[:13], token[13:60], token[60:] + "!"]] + output = stream_bytes(chunks, result.context) + texts = [json.loads(line[6:])["choices"][0]["delta"]["content"] for line in output.splitlines() if line.startswith(b"data:")] + assert texts == ["hello ", "", EMAIL + "!"] + assert b"__MLPII_" not in output + + +def test_terminal_partial_parser_overflow_and_disconnect_clear_map(): + _, stream = modules() + for wire in [b'data: {"content":"__MLPII_"}\n\ndata: [DONE]\n\n', b"data: " + b"x" * 65537]: + result = prepare(EMAIL) + with pytest.raises(stream.PIIStreamError): + stream_bytes([wire], result.context) + assert not result.context.values + result = prepare(EMAIL) + iterator = stream.RehydratingIterator(iter([b"data: "]), result.context, event_stream=True) + iterator.close() + assert not result.context.values + + +def test_secret_fields_are_not_reversible(): + redaction, _ = modules() + result = redaction.prepare_payload({"password": "A1ice42@school.org", "content": EMAIL}, settings(), route=ROUTE) + assert list(result.context.values.values()) == [EMAIL] + assert result.payload["password"] == "A1ice42@school.org" + + +def test_nonstream_terminal_prefix_is_a_real_error(): + _, stream = modules() + result = prepare(EMAIL) + response = Response(json.dumps({"content": "__MLPII_"}), content_type="application/json") + stream.rehydrate_response(response, result.context) + assert response.status_code == 502 and b"__MLPII_" not in response.data + assert not result.context.values + + +def test_nested_custom_text_also_withholds_terminal_prefix(): + _, stream = modules() + result = prepare(EMAIL) + response = Response(json.dumps({"custom": ["__MLPII_"]}), content_type="application/json") + stream.rehydrate_response(response, result.context) + assert response.status_code == 502 and b"__MLPII_" not in response.data + + +def test_stream_carry_bound_and_unknown_placeholder(): + _, stream = modules() + result = prepare(EMAIL) + token = result.payload["messages"][0]["content"] + frames = [json.dumps({"choices": [{"delta": {"content": token[:65]}}] * 2}).encode()] + with pytest.raises(stream.PIIStreamError): + stream_bytes([b"data: " + frames[0] + b"\n\n"], result.context) + assert not result.context.values + result = prepare(EMAIL) + unknown = "__MLPII_" + "0" * 64 + "__" + assert unknown.encode() in stream_bytes([f'data: {{"content":"{unknown}"}}\n\n'.encode()], result.context) + + +def test_collision_failure_is_atomic(monkeypatch): + redaction, _ = modules() + class Digest: + def hexdigest(self): + return "0" * 64 + monkeypatch.setattr(redaction.hmac, "new", lambda *args: Digest()) + with pytest.raises(redaction.PIIRedactionError): + prepare(EMAIL + " bob@school.org") + result = prepare(EMAIL + " bob@school.org", settings("best_effort")) + assert result.skipped and result.context is None + + +def test_random_secret_failure_obeys_policy(monkeypatch): + redaction, _ = modules() + def unavailable(size): + raise OSError("synthetic entropy failure") + monkeypatch.setattr(redaction.secrets, "token_bytes", unavailable) + with pytest.raises(redaction.PIIRedactionError): + prepare(EMAIL) + result = prepare(EMAIL, settings("best_effort")) + assert result.skipped and result.payload["messages"][0]["content"] == EMAIL + + +def test_parser_output_queue_is_bounded_and_close_drops_buffered_output(): + _, stream = modules() + result = prepare(EMAIL) + wire = b'data: {"content":"' + b"x" * 1000 + b'"}\n\n' + iterator = stream.RehydratingIterator(iter([wire * 1100]), result.context, event_stream=True) + with pytest.raises(stream.PIIStreamError): + next(iterator) + assert not result.context.values + result = prepare(EMAIL) + iterator = stream.RehydratingIterator(iter([wire * 2]), result.context, event_stream=True) + next(iterator) + iterator.close() + with pytest.raises(StopIteration): + next(iterator) + + +def test_restored_values_are_json_escaped(): + redaction, stream = modules() + context = redaction.PIIContext() + value = 'é "quoted"\\path\n' + token = context.issue(value) + response = Response(json.dumps({"content": token}), content_type="application/json") + stream.rehydrate_response(response, context) + assert response.json == {"content": value} + + +class RegisteredPIITests(UnifiedApiTestCase): + def setUp(self): + flags = {name: "false" for name in ( + "MANAGED_IDEMPOTENCY_ENABLED", "PROTOCOL_EXTRAS_ENABLED", "PROMPT_CACHE_AFFINITY_ENABLED", + "USAGE_RESERVATIONS_ENABLED", "CONTENT_RETENTION_ENABLED", "GENERATION_CACHE_SHARED_ENABLED", + "USAGE_LEDGER_ENABLED", "CONFIG_REVISION_SYNC_ENABLED")} + flags.update(settings()) + flags.update(SESSION_TIER_MODE="off", SECRET_SCAN_DEFAULT="off") + self.flags = patch.dict(os.environ, flags) + self.flags.start() + self.addCleanup(self.flags.stop) + with patch("config.load_runtime_env"): + super().setUp() + self.app.config["OUTPUT_SCHEMA_VALIDATION_ENABLED"] = False + redaction, _ = modules() + redaction.register_pii_redaction(self.app) + hooks = self.app.extensions["gateway_after_authentication"] + hooks.remove(redaction.pii_request_hook) + hooks.insert(1, redaction.pii_request_hook) + self.headers = {"Authorization": "Bearer admin-test-key"} + + def test_registered_unified_redacts_and_bypasses_exact_and_shared_cache(self): + body = {"model": "mimo:mimo-v2.5", "messages": [{"role": "user", "content": EMAIL}], "temperature": 0} + calls = [] + def provider(**kwargs): + payload = json.loads(kwargs["data"]) + calls.append(payload) + return self._chat_response(payload["messages"][0]["content"]) + with patch.object(self.app_module.ProxyService, "make_request", side_effect=provider): + for shared in ("false", "true"): + os.environ["GENERATION_CACHE_SHARED_ENABLED"] = shared + for _ in range(2): + response = self.client.post(ROUTE, json=body, headers={**self.headers, "X-MultiLLM-Cache": "on"}) + assert response.status_code == 200 + assert response.json["choices"][0]["message"]["content"] == EMAIL + assert response.headers["X-MultiLLM-Cache"] == "bypass" + assert len(calls) == 4 and EMAIL not in json.dumps(calls) + + def test_registered_required_failure_and_idempotency_do_not_dispatch(self): + body = {"model": "mimo:mimo-v2.5", "messages": [{"role": "user", "content": EMAIL}]} + with patch.object(self.app_module.ProxyService, "make_request", side_effect=AssertionError("unexpected dispatch")): + error = self.client.post(ROUTE, json=body, headers={**self.headers, "Idempotency-Key": "synthetic"}) + assert error.status_code == 400 and error.json["error"]["code"] == "pii_idempotency_unsupported" + body["messages"][0]["content"] = "x" * (1024 * 1024 + 1) + error = self.client.post(ROUTE, json=body, headers=self.headers) + assert error.status_code == 422 and error.json["error"]["code"] == "pii_redaction_failed" + + def test_registered_default_off_is_unchanged(self): + os.environ["PII_REDACTION_ENABLED"] = "false" + upstream = self._chat_response(EMAIL) + body = {"model": "mimo:mimo-v2.5", "messages": [{"role": "user", "content": EMAIL}]} + with patch.object(self.app_module.ProxyService, "make_request", return_value=upstream) as send: + response = self.client.post(ROUTE, json=body, headers=self.headers) + assert response.data == upstream.content + assert json.loads(send.call_args.kwargs["data"])["messages"] == body["messages"] + + def test_registered_stream_restores_fragments_and_disconnect_clears_map(self): + import io + import requests + contexts = [] + def provider(**kwargs): + token = json.loads(kwargs["data"])["messages"][0]["content"] + contexts.append(modules()[0].current_context()) + frames = [json.dumps({"choices": [{"delta": {"content": part}}]}).encode() + for part in (token[:13], token[13:], "!")] + wire = b"".join(b"data: " + frame + b"\n\n" for frame in frames) + b"data: [DONE]\n\n" + upstream = requests.Response() + upstream.status_code = 200 + upstream.headers["Content-Type"] = "text/event-stream" + upstream.raw = io.BytesIO(wire) + upstream.iter_content = lambda chunk_size=1, **options: iter(wire[i:i + 1] for i in range(len(wire))) + return upstream + body = {"model": "mimo:mimo-v2.5", "messages": [{"role": "user", "content": EMAIL}], "stream": True} + with patch.object(self.app_module.ProxyService, "make_request", side_effect=provider): + response = self.client.post(ROUTE, json=body, headers=self.headers) + wire = response.get_data() + pieces = [json.loads(line[6:])["choices"][0]["delta"]["content"] for line in wire.splitlines() + if line.startswith(b"data: {")] + assert "".join(pieces) == EMAIL + "!" and b"__MLPII_" not in wire + assert contexts[-1].closed and not contexts[-1].values + response = self.client.post(ROUTE, json=body, headers=self.headers, buffered=False) + response.close() + assert contexts[-1].closed and not contexts[-1].values + + +def test_registered_hook_dispatch_stream_and_cache_idempotency_guards(monkeypatch): + redaction, _ = modules() + from services.managed_dispatch import execute_managed_attempt, observe_managed_response + from services.retention_policy import request_policy + import requests + + for key, value in settings().items(): + monkeypatch.setenv(key, value) + monkeypatch.setenv("SECRET_SCAN_DEFAULT", "off") + app = Flask(__name__) + redaction.register_pii_redaction(app) + calls = [] + + @app.before_request + def authenticated(): + g.authenticated_user = {"id": "key-7"} + for hook in app.extensions["gateway_after_authentication"]: + error = hook() + if error is not None: + return error + + @app.post(ROUTE) + def completion(): + payload = request.get_json() + assert not request_policy().allows_content + token = payload["messages"][0]["content"] + def send(): + calls.append(payload) + upstream = requests.Response() + upstream.status_code = 200 + return upstream + upstream = execute_managed_attempt(send, "synthetic", "synthetic") + body = Response(json.dumps({"choices": [{"message": {"content": token}, "finish_reason": "stop"}]}), content_type="application/json") + return observe_managed_response(body, upstream) + + client = app.test_client() + body = {"model": "auto:synthetic", "messages": [{"role": "user", "content": EMAIL}]} + response = client.post(ROUTE, json=body) + assert response.status_code == 200 and response.json["choices"][0]["message"]["content"] == EMAIL + assert EMAIL not in json.dumps(calls) + response = client.post(ROUTE, json=body, headers={"Idempotency-Key": "synthetic"}) + assert response.status_code == 400 and len(calls) == 1 diff --git a/tests/test_pii_rehydration_worker.mjs b/tests/test_pii_rehydration_worker.mjs new file mode 100644 index 00000000..6b11ec5e --- /dev/null +++ b/tests/test_pii_rehydration_worker.mjs @@ -0,0 +1,256 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +const email = "alice@school.org", route = "/v1/roleplay/chat/completions"; +const config = (extra = {}) => ({ PII_REDACTION_ENABLED: "true", PII_REDACTION_POLICY_JSON: + JSON.stringify({ routes: [route], detectors: ["email", "phone", "card"], mode: "required", ...extra }) }); +const modules = () => Promise.all([import("../worker/pii-redaction.mjs"), import("../worker/pii-rehydration.mjs")]); +async function prepare(text = email, env = config(), options = {}) { + const [redaction] = await modules(); + return redaction.preparePayload({ messages: [{ role: "user", content: text }] }, env, { route, ...options }); +} + +test("PII modules are available", async () => { + const [redaction, stream] = await modules(); + assert.equal(typeof redaction.preparePayload, "function"); + assert.equal(typeof stream.rehydrateResponse, "function"); +}); + +test("disabled, empty, raw and unscoped keep original object", async () => { + const [redaction] = await modules(); + const payload = { content: email }; + for (const env of [{}, { PII_REDACTION_ENABLED: "" }, config({ routes: [] }), + { PII_REDACTION_ENABLED: "true", PII_REDACTION_POLICY_JSON: "" }]) { + const result = await redaction.preparePayload(payload, env, { route }); + assert.equal(result.payload, payload); + assert.equal(result.context, null); + } + assert.equal((await prepare(email, config(), { raw: true })).context, null); + assert.ok((await prepare(email, config({ routes: [], keys: ["key-7"] }), { keyScope: "key-7" })).context); +}); + +test("detectors, Luhn, authenticated tokens and concurrent maps", async () => { + for (const value of [email, "+1 (415) 555-2671", "4111 1111 1111 1111"]) { + const [a, b] = await Promise.all([prepare(value), prepare(value)]); + const token = a.payload.messages[0].content; + assert.match(token, /^__MLPII_[0-9a-f]{64}__$/); + assert.equal(a.context.restore(token), value); + assert.equal(b.context.restore(token), token); + a.context.close(); + assert.equal(a.context.values.size, 0); + assert.equal(a.context.restore(token), token); + } + for (const value of ["4111 1111 1111 1112", "123", "123456789012345678901", "bad@local", "x".repeat(65) + "@school.org"]) + assert.equal((await prepare(value)).payload.messages[0].content, value); +}); + +test("secret spans never enter PII map", async () => { + const result = await prepare("postgres://owner:S3cur3Password42@school.org/db " + email, + { ...config(), SECRET_SCAN_DEFAULT: "observe" }); + assert.deepEqual([...result.context.values.values()], [email]); + const [redaction] = await modules(); + const fields = await redaction.preparePayload({ password: "A1ice42@school.org", content: email }, config(), { route }); + assert.deepEqual([...fields.context.values.values()], [email]); +}); + +test("bounds fail closed and best effort is atomic", async () => { + for (const text of ["x".repeat(1024 * 1024 + 1), Array.from({ length: 513 }, (_, i) => `p${i}@school.org`).join(" ")]) { + await assert.rejects(prepare(text), { name: "PIIRedactionError" }); + const result = await prepare(text, config({ mode: "best_effort" })); + assert.equal(result.skipped, true); + assert.equal(result.context, null); + assert.equal(result.payload.messages[0].content, text); + } +}); + +test("nonstream JSON restores and drops map", async () => { + const [, stream] = await modules(), result = await prepare(); + const token = result.payload.messages[0].content; + const response = await stream.rehydrateResponse(Response.json({ content: "é " + token }), result.context); + assert.deepEqual(await response.json(), { content: "é " + email }); + assert.equal(result.context.values.size, 0); + const partial = await prepare(); + const error = await stream.rehydrateResponse(Response.json({ content: "__MLPII_" }), partial.context); + assert.equal(error.status, 502); + assert.equal(stream.isPIIFailure(error), true); + assert.equal(partial.context.values.size, 0); +}); + +test("every chunk split including UTF8 and JSON escapes", async () => { + const [, stream] = await modules(); + const sample = await prepare(), token = sample.payload.messages[0].content; + const wire = `data: ${JSON.stringify({ choices: [{ delta: { content: "é " + token + "\n" } }] }).replaceAll("_", "\\u005f")}\r\n\r\ndata: [DONE]\r\n\r\n`; + for (let split = 1; split < new TextEncoder().encode(wire).length; split += 1) { + const result = await prepare(); + const bytes = new TextEncoder().encode(wire.replaceAll(token.replaceAll("_", "\\u005f"), result.payload.messages[0].content.replaceAll("_", "\\u005f"))); + const body = new ReadableStream({ start(c) { c.enqueue(bytes.slice(0, split)); c.enqueue(bytes.slice(split)); c.close(); } }); + const response = await stream.rehydrateResponse(new Response(body, { headers: { "Content-Type": "text/event-stream" } }), result.context); + const text = await response.text(); + assert.equal(JSON.parse(text.split("\n")[0].slice(6)).choices[0].delta.content, "é " + email + "\n"); + assert.equal(result.context.values.size, 0); + } +}); + +test("event fragments, terminal prefixes, bounds and cancellation", async () => { + const [, stream] = await modules(), result = await prepare(); + const token = result.payload.messages[0].content; + const frames = [token.slice(0, 13), token.slice(13, 60), token.slice(60)].map(content => + `data: ${JSON.stringify({ choices: [{ delta: { content } }] })}\n\n`).join(""); + const response = await stream.rehydrateResponse(new Response(frames, { headers: { "Content-Type": "text/event-stream" } }), result.context); + const text = await response.text(); + assert.deepEqual(text.split("\n").filter(line => line.startsWith("data:")).map(line => JSON.parse(line.slice(6)).choices[0].delta.content), ["", "", email]); + for (const wire of ['data: {"content":"__MLPII_"}\n\ndata: [DONE]\n\n', "data: " + "x".repeat(65537)]) { + const item = await prepare(); + const output = await stream.rehydrateResponse(new Response(wire, { headers: { "Content-Type": "text/event-stream" } }), item.context); + await assert.rejects(output.text(), { name: "PIIStreamError" }); + assert.equal(item.context.values.size, 0); + } + let cancelled = false; + const item = await prepare(); + const upstream = new ReadableStream({ cancel() { cancelled = true; } }, { highWaterMark: 0 }); + const output = await stream.rehydrateResponse(new Response(upstream, { headers: { "Content-Type": "text/event-stream" } }), item.context); + await output.body.cancel(); + assert.equal(cancelled, true); + assert.equal(item.context.values.size, 0); +}); + +test("native roleplay transport redacts after firewall and restores", async () => { + const { attemptRoleplayCandidates } = await import("../worker/roleplay/transport.mjs"); + const previous = globalThis.fetch; + const calls = []; + globalThis.fetch = async (_url, init) => { + const payload = JSON.parse(init.body); + calls.push(payload); + return Response.json({ choices: [{ message: { content: payload.messages[0].content }, finish_reason: "stop" }] }); + }; + try { + const candidate = { key: "synthetic", provider: "nanogpt", model: "synthetic", token: "synthetic", endpoint: "https://example.invalid" }; + const decisions = []; + const result = await attemptRoleplayCandidates({ stats: {} }, [candidate], () => ({ model: "synthetic", messages: [{ role: "user", content: email }] }), + { ...config({ routes: [], keys: ["key-7"] }), SECRET_SCAN_DEFAULT: "off" }, + { upstreamHeaderTimeoutMs: 1000, piiKeyScope: "key-7", onPIIDecision: decision => decisions.push(decision) }, undefined, ""); + assert.equal((await result.response.json()).choices[0].message.content, email); + assert.ok(!JSON.stringify(calls).includes(email)); + assert.deepEqual(decisions, [{ redacted: true, skipped: false, cacheable: false, replayable: false }]); + result.cleanup(); + } finally { globalThis.fetch = previous; } +}); + +test("required failure never dispatches or falls back; best effort preserves payload", async () => { + const { attemptRoleplayCandidates } = await import("../worker/roleplay/transport.mjs"); + const previous = globalThis.fetch; + let calls = 0; + globalThis.fetch = async (_url, init) => { calls += 1; return Response.json({ content: JSON.parse(init.body).messages.map(message => message.content).join("") }); }; + const candidate = { key: "synthetic", provider: "nanogpt", model: "synthetic", token: "synthetic", endpoint: "https://example.invalid" }; + const payload = { model: "synthetic", messages: [{ content: "x".repeat(1024 * 1024 + 1) }] }; + try { + const result = await attemptRoleplayCandidates({ stats: {} }, [candidate, candidate], () => payload, + { ...config(), SECRET_SCAN_DEFAULT: "off" }, { upstreamHeaderTimeoutMs: 1000, preResponseFallbackEnabled: true }, undefined, ""); + assert.equal(result.terminalResponse.status, 422); + assert.equal(calls, 0); + const fallback = await attemptRoleplayCandidates({ stats: {} }, [candidate], () => payload, + { ...config({ mode: "best_effort" }), SECRET_SCAN_DEFAULT: "off" }, { upstreamHeaderTimeoutMs: 1000 }, undefined, ""); + assert.equal((await fallback.response.json()).content, payload.messages[0].content); + assert.equal(calls, 1); fallback.cleanup(); + } finally { globalThis.fetch = previous; } +}); + +test("abort during nonstream restoration clears state and cancels pending reader", async () => { + const [, stream] = await modules(), result = await prepare(); + let cancelled = false; + const controller = new AbortController(); + const body = new ReadableStream({ cancel() { cancelled = true; } }, { highWaterMark: 0 }); + const restoring = stream.rehydrateResponse(new Response(body, { headers: { "Content-Type": "application/json" } }), result.context, { signal: controller.signal }); + controller.abort(); + await assert.rejects(restoring, { name: "AbortError" }); + assert.equal(result.context.values.size, 0); + assert.equal(cancelled, true); +}); + +test("restoration failure after output cannot authorize another provider", async () => { + const { attemptRoleplayCandidates } = await import("../worker/roleplay/transport.mjs"); + const previous = globalThis.fetch; + let calls = 0; + globalThis.fetch = async () => { calls += 1; return Response.json({ content: "__MLPII_" }); }; + const candidate = { key: "synthetic", provider: "nanogpt", model: "synthetic", token: "synthetic", endpoint: "https://example.invalid" }; + try { + const result = await attemptRoleplayCandidates({ stats: {} }, [candidate, candidate], + () => ({ model: "synthetic", messages: [{ content: email }] }), + { ...config(), SECRET_SCAN_DEFAULT: "off" }, { upstreamHeaderTimeoutMs: 1000, preResponseFallbackEnabled: true }, undefined, ""); + assert.equal(calls, 1); + assert.equal(result.terminalResponse.status, 502); + assert.equal((await result.terminalResponse.json()).error.code, "pii_rehydration_failed"); + } finally { globalThis.fetch = previous; } +}); + +test("parser output batches are bounded", async () => { + const [, stream] = await modules(), result = await prepare(); + const parser = new stream.PIIStreamParser(result.context); + const wire = new TextEncoder().encode(('data: {"content":"' + "x".repeat(1000) + '"}\n\n').repeat(1100)); + assert.throws(() => parser.feed(wire), { name: "PIIStreamError" }); + parser.close(); + assert.equal(result.context.values.size, 0); +}); + +test("JSON escaping, unknown tokens and carry limits", async () => { + const [redaction, stream] = await modules(); + const context = new redaction.PIIContext(), value = 'é "quoted"\\path\n'; + const token = await context.issue(value), unknown = "__MLPII_" + "0".repeat(64) + "__"; + const response = await stream.rehydrateResponse(Response.json({ content: token + unknown }), context); + assert.deepEqual(await response.json(), { content: value + unknown }); + const result = await prepare(), parser = new stream.PIIStreamParser(result.context); + const partial = result.payload.messages[0].content.slice(0, 65); + assert.throws(() => parser.feed(new TextEncoder().encode(`data: ${JSON.stringify({ choices: [0, 1].map(() => ({ delta: { content: partial } })) })}\n\n`)), { name: "PIIStreamError" }); + parser.close(); +}); + +test("malformed policy warns once without its value", async () => { + const [redaction] = await modules(), previous = console.warn, warnings = []; + console.warn = message => warnings.push(message); + try { + const invalid = { PII_REDACTION_ENABLED: "true", PII_REDACTION_POLICY_JSON: "private-invalid-value" }; + assert.equal((await prepare(email, invalid)).context, null); + assert.equal((await prepare(email, invalid)).context, null); + assert.equal(warnings.length, 1); + assert.ok(!warnings[0].includes("private-invalid-value")); + } finally { console.warn = previous; } +}); + +test("secret creation and HMAC collisions obey atomic policy", async () => { + const original = globalThis.crypto; + try { + Object.defineProperty(globalThis, "crypto", { configurable: true, value: { + getRandomValues: values => original.getRandomValues(values), + subtle: { importKey: async () => ({}), sign: async () => new Uint8Array(32).buffer }, + } }); + await assert.rejects(prepare(email + " bob@school.org"), { name: "PIIRedactionError" }); + assert.equal((await prepare(email + " bob@school.org", config({ mode: "best_effort" }))).skipped, true); + Object.defineProperty(globalThis, "crypto", { configurable: true, value: { + getRandomValues() { throw new Error("synthetic entropy failure"); }, + } }); + await assert.rejects(prepare(), { name: "PIIRedactionError" }); + assert.equal((await prepare(email, config({ mode: "best_effort" }))).skipped, true); + } finally { Object.defineProperty(globalThis, "crypto", { configurable: true, value: original }); } +}); + +test("off, unscoped and raw transport preserves request and response identity", async () => { + const [, stream] = await modules(); + for (const [env, options] of [[{}, { route }], [config({ routes: [] }), { route }], [config(), { route, raw: true }]]) { + const request = new Request("https://example.invalid", { method: "POST", body: "{ invalid JSON }" }); + const response = new Response("unchanged é bytes", { headers: { "X-Synthetic": "unchanged" } }); + const output = await stream.piiFetch(request, env, options, async upstream => { + assert.equal(upstream, request); + assert.equal(await upstream.text(), "{ invalid JSON }"); + return response; + }); + assert.equal(output, response); + assert.equal(await output.text(), "unchanged é bytes"); + } +}); + +test("custom JSON text also withholds terminal prefixes", async () => { + const [, stream] = await modules(), result = await prepare(); + const response = await stream.rehydrateResponse(Response.json({ custom: ["__MLPII_"] }), result.context); + assert.equal(response.status, 502); + assert.ok(!(await response.text()).includes("__MLPII_")); +}); diff --git a/worker/container-env.mjs b/worker/container-env.mjs index 1a7994db..06b668e2 100644 --- a/worker/container-env.mjs +++ b/worker/container-env.mjs @@ -40,6 +40,8 @@ const DIRECT_ENV_KEYS = [ "CONNECTION_PROFILES_DB_PATH", "CONFIG_REVISION_SYNC_ENABLED", "CONFIG_SYNC_TTL_SECONDS", + "PII_REDACTION_ENABLED", + "PII_REDACTION_POLICY_JSON", "GATEWAY_ALERTS_ENABLED", "GATEWAY_ALERT_WEBHOOK_ALLOWLIST", "CONFIG_SECURITY_TTL_SECONDS", diff --git a/worker/pii-redaction.mjs b/worker/pii-redaction.mjs new file mode 100644 index 00000000..33d4e3fe --- /dev/null +++ b/worker/pii-redaction.mjs @@ -0,0 +1,132 @@ +/** Request-local deterministic PII findings; no reversible state is persisted. */ +import { scanPayload, scanText } from "./secret-scan.mjs"; + +export const PREFIX = "__MLPII_"; +export const TOKEN = /__MLPII_[0-9a-f]{64}__/g; +const MAX_INPUT_BYTES = 1024 * 1024, MAX_VALUES = 512; +const encoder = new TextEncoder(); +const warned = new Set(); +const EMAIL = /(? 65536) throw new Error(); + const policy = JSON.parse(raw); + if (!policy || typeof policy !== "object" || Array.isArray(policy) || + Object.keys(policy).some(key => !["routes", "keys", "detectors", "mode"].includes(key))) throw new Error(); + for (const name of ["routes", "keys", "detectors"]) { + const items = policy[name] ?? []; + if (!Array.isArray(items) || items.length > 512 || items.some(item => typeof item !== "string" || !item.length || item.length > 256)) throw new Error(); + } + if ((policy.detectors ?? []).some(name => !["email", "phone", "card"].includes(name))) throw new Error(); + const mode = policy.mode ?? "required"; + if (!["required", "best_effort"].includes(mode)) throw new Error(); + if (!(policy.routes ?? []).includes(route) && (!keyScope || !(policy.keys ?? []).includes(keyScope))) return null; + return policy.detectors?.length ? { detectors: policy.detectors, mode } : null; + } catch { warnOnce("PII_REDACTION_POLICY_JSON"); return null; } +} + +function luhn(digits) { + let total = 0; + [...digits].reverse().forEach((char, index) => { + const value = Number(char) * (index % 2 ? 2 : 1); + total += value > 9 ? value - 9 : value; + }); + return total % 10 === 0 && new Set(digits).size > 1; +} + +function findings(text, detectors) { + const protectedSpans = scanText(text), found = []; + if (detectors.includes("email")) for (const match of text.matchAll(EMAIL)) found.push([match.index, match.index + match[0].length]); + if (detectors.includes("phone") || detectors.includes("card")) { + for (const match of text.matchAll(NUMBER)) { + const value = match[0], digits = value.replace(/[^0-9]/g, ""); + const card = digits.length >= 13 && digits.length <= 19 && luhn(digits); + const phone = digits.length >= 10 && digits.length <= 15 && (value.startsWith("+") || /[ ()-]/.test(value)); + if (detectors.includes("card") && card || detectors.includes("phone") && phone) + found.push([match.index, match.index + value.length]); + } + } + found.sort((a, b) => a[0] - b[0] || b[1] - a[1]); + let end = -1; + return found.filter(([start, stop]) => { + if (start < end || protectedSpans.some(span => start < span.end && stop > span.start)) return false; + end = stop; return true; + }); +} + +export class PIIContext { + constructor() { + this.secret = crypto.getRandomValues(new Uint8Array(32)); + this.values = new Map(); this.closed = false; + } + async issue(value) { + if (this.closed) throw new PIIRedactionError("PII request context is closed"); + const key = await crypto.subtle.importKey("raw", this.secret, { name: "HMAC", hash: "SHA-256" }, false, ["sign"]); + const signature = new Uint8Array(await crypto.subtle.sign("HMAC", key, encoder.encode(value))); + if (this.closed) throw new PIIRedactionError("PII request context is closed"); + const token = PREFIX + [...signature].map(byte => byte.toString(16).padStart(2, "0")).join("") + "__"; + if (this.values.has(token) && this.values.get(token) !== value) throw new PIIRedactionError("PII placeholder collision"); + if (!this.values.has(token) && this.values.size >= MAX_VALUES) throw new PIIRedactionError("PII request exceeds the value limit"); + this.values.set(token, value); return token; + } + restore(text) { return text.replace(TOKEN, token => this.values.get(token) ?? token); } + close() { this.values.clear(); this.secret.fill(0); this.closed = true; } +} + +async function redactTree(value, context, detectors, depth = 0, counter = { nodes: 0 }, field = "") { + counter.nodes += 1; + if (depth > 32 || counter.nodes > 32768) throw new PIIRedactionError("PII document exceeds the structure limit"); + if (typeof value === "string") { + if (scanPayload({ [field]: value }).types.secret_field) return value; + const parts = []; let end = 0; + for (const [start, stop] of findings(value, detectors)) { + parts.push(value.slice(end, start), await context.issue(value.slice(start, stop))); end = stop; + } + return parts.join("") + value.slice(end); + } + if (Array.isArray(value)) { + const result = []; + for (const item of value) result.push(await redactTree(item, context, detectors, depth + 1, counter, field)); + return result; + } + if (value && typeof value === "object") { + const result = []; + for (const [key, item] of Object.entries(value)) result.push([key, await redactTree(item, context, detectors, depth + 1, counter, key)]); + return Object.fromEntries(result); + } + return value; +} + +export async function preparePayload(payload, env = {}, { route = "", keyScope = "", raw = false, firewall } = {}) { + const policy = raw ? null : resolvePIIPolicy(env, { route, keyScope }); + if (!policy) return { payload, context: null, skipped: false }; + const checked = firewall ? await firewall(payload) : payload; + let context; + try { + if (encoder.encode(JSON.stringify(checked)).length > MAX_INPUT_BYTES) throw new PIIRedactionError("PII request exceeds the input limit"); + context = new PIIContext(); + const updated = await redactTree(checked, context, policy.detectors); + if (!context.values.size) { context.close(); return { payload: checked, context: null, skipped: false }; } + return { payload: updated, context, skipped: false }; + } catch { + context?.close(); + if (policy.mode === "best_effort") return { payload: checked, context: null, skipped: true }; + throw new PIIRedactionError(); + } +} diff --git a/worker/pii-rehydration.mjs b/worker/pii-rehydration.mjs new file mode 100644 index 00000000..cdc91276 --- /dev/null +++ b/worker/pii-rehydration.mjs @@ -0,0 +1,208 @@ +/** Restore authenticated placeholders across bounded JSON/SSE frames. */ +import { PREFIX, preparePayload, resolvePIIPolicy } from "./pii-redaction.mjs"; + +const failures = new WeakSet(); +export const isPIIFailure = response => failures.has(response); + +const encoder = new TextEncoder(); +const MAX_FRAME_BYTES = 65536, MAX_BODY_BYTES = 1024 * 1024; +const COMPLETE_TOKEN = /^__MLPII_[0-9a-f]{64}__/; + +export class PIIStreamError extends Error { + constructor(message) { super(message); this.name = "PIIStreamError"; } +} + +class TextCarry { + constructor(context) { this.context = context; this.pending = new Map(); } + text(value, path) { + const lane = JSON.stringify(path); + const text = (this.pending.get(lane) ?? "") + value; + this.pending.delete(lane); + let output = "", index = 0; + while (index < text.length) { + const start = text.indexOf("_", index); + if (start < 0) { output += text.slice(index); break; } + output += text.slice(index, start); + const rest = text.slice(start, start + 74), token = COMPLETE_TOKEN.exec(rest)?.[0]; + if (token) { output += this.context.restore(token); index = start + token.length; continue; } + const partial = PREFIX.startsWith(rest) || (rest.startsWith(PREFIX) && rest.length < 74 && /^[0-9a-f]{0,64}_{0,2}$/.test(rest.slice(PREFIX.length))); + if (partial) { + if (this.pending.size >= 16) throw new PIIStreamError("PII stream exceeds the text lane limit"); + this.pending.set(lane, rest); + if ([...this.pending.values()].reduce((n, item) => n + encoder.encode(item).length, 0) > 128) + throw new PIIStreamError("PII stream exceeds the prefix limit"); + break; + } + output += "_"; index = start + 1; + } + return output; + } + transform(value, path = [], depth = 0, counter = { nodes: 0 }) { + counter.nodes += 1; + if (depth > 32 || counter.nodes > 32768) throw new PIIStreamError("PII stream exceeds the structure limit"); + if (typeof value === "string") return this.text(value, path); + if (Array.isArray(value)) return value.map((item, i) => this.transform(item, [...path, i], depth + 1, counter)); + if (value && typeof value === "object") return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, this.transform(item, [...path, key], depth + 1, counter)])); + return value; + } + finish() { if (this.pending.size) throw new PIIStreamError("PII stream ended with an unresolved placeholder prefix"); } +} + +export class PIIStreamParser { + constructor(context) { this.context = context; this.carry = new TextCarry(context); this.buffer = ""; this.decoder = new TextDecoder("utf-8", { fatal: true }); } + frame(frame) { + const lines = frame.match(/[^\r\n]*(?:\r\n|\r|\n|$)/g).filter(Boolean); + const indices = lines.flatMap((line, i) => line.startsWith("data:") ? [i] : []); + if (!indices.length) return frame; + const data = indices.map(i => lines[i].slice(5).trim()).join("\n"); + if (data === "[DONE]") { this.carry.finish(); return frame; } + let value; + try { value = JSON.parse(data); } catch { throw new PIIStreamError("PII stream contains invalid JSON data"); } + const restored = this.carry.transform(value); + if (JSON.stringify(restored) === JSON.stringify(value)) return frame; + const ending = lines[indices[0]].endsWith("\r\n") ? "\r\n" : "\n"; + lines[indices[0]] = "data: " + JSON.stringify(restored) + ending; + for (const i of indices.slice(1)) lines[i] = ""; + return lines.join(""); + } + feed(bytes, final = false) { + const output = []; + let outputBytes = 0; + const enqueue = frame => { + const restored = encoder.encode(this.frame(frame)); + outputBytes += restored.length; + if (outputBytes > MAX_BODY_BYTES) throw new PIIStreamError("PII stream exceeds the output queue limit"); + output.push(restored); + }; + for (let offset = 0; offset < bytes.length; offset += 4096) { + this.buffer += this.decoder.decode(bytes.subarray(offset, offset + 4096), { stream: true }); + let match; + while ((match = /\r\n\r\n|\n\n|\r\r/.exec(this.buffer))) { + const end = match.index + match[0].length, frame = this.buffer.slice(0, end); + if (encoder.encode(frame).length > MAX_FRAME_BYTES) throw new PIIStreamError("PII stream frame exceeds the parser limit"); + this.buffer = this.buffer.slice(end); enqueue(frame); + } + if (encoder.encode(this.buffer).length > MAX_FRAME_BYTES) throw new PIIStreamError("PII response exceeds the parser limit"); + } + if (final) { + this.buffer += this.decoder.decode(); + if (this.buffer) enqueue(this.buffer); + this.buffer = ""; this.carry.finish(); + } + return output; + } + close() { this.buffer = ""; this.carry.pending.clear(); this.context.close(); } +} + +async function readJSON(response, signal) { + const reader = response.body.getReader(), chunks = []; + let length = 0; + let rejectAbort; + const interrupted = new Promise((_, reject) => { rejectAbort = reject; }); + const abort = () => { + void reader.cancel().catch(() => {}); + rejectAbort(new DOMException("The request was aborted", "AbortError")); + }; + signal?.addEventListener("abort", abort, { once: true }); + try { + if (signal?.aborted) throw new DOMException("The request was aborted", "AbortError"); + while (true) { + const { done, value } = await Promise.race([reader.read(), interrupted]); + if (signal?.aborted) throw new DOMException("The request was aborted", "AbortError"); + if (done) break; + length += value.length; + if (length > MAX_BODY_BYTES) throw new PIIStreamError("PII response exceeds the body limit"); + chunks.push(value); + } + const bytes = new Uint8Array(length); let offset = 0; + for (const chunk of chunks) { bytes.set(chunk, offset); offset += chunk.length; } + return JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(bytes)); + } finally { + signal?.removeEventListener("abort", abort); + void reader.cancel().catch(() => {}); + reader.releaseLock(); + } +} + +export async function rehydrateResponse(response, context, { signal } = {}) { + if (!context || context.closed) return response; + const headers = new Headers(response.headers); headers.delete("content-length"); + if (!(headers.get("content-type") ?? "").toLowerCase().startsWith("text/event-stream")) { + try { + const carry = new TextCarry(context); + const value = carry.transform(await readJSON(response, signal)); + carry.finish(); + return new Response(JSON.stringify(value), { status: response.status, statusText: response.statusText, headers }); + } catch (error) { + if (error?.name === "AbortError") throw error; + headers.set("content-type", "application/json"); + const failure = new Response('{"error":{"code":"pii_rehydration_failed","message":"Unable to restore provider response"}}', { status: 502, headers }); + failures.add(failure); + return failure; + } finally { context.close(); } + } + const reader = response.body.getReader(), parser = new PIIStreamParser(context); + let closed = false, downstream; + function cleanup() { + if (closed) return; + closed = true; parser.close(); signal?.removeEventListener("abort", abort); + } + function abort() { + cleanup(); void reader.cancel().catch(() => {}); + downstream?.error(new DOMException("The request was aborted", "AbortError")); + } + const body = new ReadableStream({ + start(controller) { + downstream = controller; + signal?.addEventListener("abort", abort, { once: true }); + if (signal?.aborted) abort(); + }, + async pull(controller) { + if (closed) return; + try { + while (!closed) { + const { done, value } = await reader.read(); + if (closed) return; + const output = parser.feed(value ?? new Uint8Array(), done); + for (const chunk of output) controller.enqueue(chunk); + if (done) { cleanup(); reader.releaseLock(); controller.close(); } + if (done || output.length) return; + } + } catch (error) { + cleanup(); await reader.cancel().catch(() => {}); controller.error(error); + } + }, + async cancel(reason) { cleanup(); await reader.cancel(reason).catch(() => {}); }, + }, { highWaterMark: 0 }); + return new Response(body, { status: response.status, statusText: response.statusText, headers }); +} + +export async function piiFetch(request, env, options, fetchImpl) { + const policy = options.raw ? null : resolvePIIPolicy(env, options); + if (!policy) return fetchImpl(request); + let prepared; + try { + prepared = await preparePayload(await readJSON(request.clone(), request.signal), env, options); + } catch { + if (request.signal.aborted) throw new DOMException("The request was aborted", "AbortError"); + if (policy.mode === "best_effort") { + options.onDecision?.({ redacted: false, skipped: true, cacheable: true, replayable: true }); + return fetchImpl(request); + } + const response = Response.json({ error: { code: "pii_redaction_failed", message: "Required PII transformation failed before provider dispatch" } }, { status: 422 }); + failures.add(response); return response; + } + options.onDecision?.({ redacted: Boolean(prepared.context), skipped: prepared.skipped, + cacheable: !prepared.context, replayable: !prepared.context }); + if (!prepared.context) return fetchImpl(request); + const context = prepared.context; + const headers = new Headers(request.headers); + headers.delete("content-length"); headers.delete("idempotency-key"); + headers.set("cache-control", "no-store"); + try { + const upstream = await fetchImpl(new Request(request, { headers, body: JSON.stringify(prepared.payload), duplex: "half" })); + const response = await rehydrateResponse(upstream, context, { signal: request.signal }); + response.headers.set("cache-control", "no-store"); + return response; + } catch (error) { context.close(); throw error; } +} diff --git a/worker/roleplay/transport.mjs b/worker/roleplay/transport.mjs index 6ebbb863..32a53389 100644 --- a/worker/roleplay/transport.mjs +++ b/worker/roleplay/transport.mjs @@ -1,5 +1,7 @@ import { UpstreamCancellation, readBoundedUpstreamBytes } from "../upstream-cancellation.mjs"; import { firewallFetch, isSecretScanBlock } from "../secret-firewall.mjs"; +import { piiFetch, isPIIFailure } from "../pii-rehydration.mjs"; +import { resolvePIIPolicy } from "../pii-redaction.mjs"; import { buildProviderHeaders, isSafeFallbackStatus, @@ -578,22 +580,26 @@ async function fetchCandidate(candidate, payload, env, settings, signal, key) { signal: controller.signal, }; cancellation.handoff(); - const response = useOpenCodeContainer + const piiOptions = { route: "/v1/roleplay/chat/completions", keyScope: settings.piiKeyScope ?? "", onDecision: settings.onPIIDecision }; + const submit = checked => piiFetch(checked, env, piiOptions, async transformed => + useOpenCodeContainer ? containerNamespace.getByName("primary").fetch(transformed) + : fetch(transformed.url, { ...requestInit, headers: transformed.headers, body: await transformed.text() })); + const response = useOpenCodeContainer && !resolvePIIPolicy(env, piiOptions) ? await containerNamespace.getByName("primary").fetch( new Request( "https://roleplay.internal/opencode/v1/chat/completions", requestInit, ), ) - : await firewallFetch(new Request(candidate.endpoint, requestInit), env, { + : await firewallFetch(new Request(useOpenCodeContainer ? "https://roleplay.internal/opencode/v1/chat/completions" : candidate.endpoint, requestInit), env, { route: "/v1/roleplay/chat/completions", provider: candidate.provider, onDecision: settings.onSecretScan, principal: { id: env.ADMIN_USERNAME || "admin" }, - }, async checked => fetch(checked.url, { ...requestInit, headers: checked.headers, body: await checked.text() })); + }, submit); cancellation.headersReceived(); - const ownedResponse = isSecretScanBlock(response) + const ownedResponse = isSecretScanBlock(response) || isPIIFailure(response) ? response : cancellation.wrapResponse(response); - if (isSecretScanBlock(response)) cancellation.complete(); + if (isSecretScanBlock(response) || isPIIFailure(response)) cancellation.complete(); cancellation.throwIfAborted(); const headerMs = performance.now() - startedAt; return { @@ -777,7 +783,7 @@ export async function requestCompaction( } const { response } = attempted; - if (isSecretScanBlock(response)) { + if (isSecretScanBlock(response) || isPIIFailure(response)) { attempted.cleanup(); return { blockedResponse: response }; } @@ -908,7 +914,7 @@ export async function attemptRoleplayCandidates( }; } - if (isSecretScanBlock(attempted.response)) { + if (isSecretScanBlock(attempted.response) || isPIIFailure(attempted.response)) { attempted.cleanup(); return { state: nextState, terminalResponse: attempted.response }; }