Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 123 additions & 0 deletions docs/pii-rehydration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# PII redaction and streaming rehydration

Opted-in managed generations replace detected PII before provider dispatch and
restore request-owned placeholders in JSON responses and SSE text. The credential
secret firewall runs first. Its findings, including values in secret-named fields,
never enter the reversible PII map, even when secret scanning observes rather than
redacts or blocks them.

## Configuration

The defaults leave traffic unchanged:

```text
PII_REDACTION_ENABLED=false
PII_REDACTION_POLICY_JSON={}
```

Empty values mean these defaults. Invalid flags or policies disable redaction and
produce one warning per configuration field, without including its value. Enabling
the flag with an empty policy opts in nothing. Unknown policy fields are invalid.

For example, opt in a managed route:

```text
PII_REDACTION_ENABLED=true
PII_REDACTION_POLICY_JSON={"routes":["/v1/chat/completions"],"keys":[],"detectors":["email","phone","card"],"mode":"required"}
```

Or opt in an authenticated key scope, using its stable internal identifier rather
than the credential itself:

```text
PII_REDACTION_POLICY_JSON={"routes":[],"keys":["key-7"],"detectors":["email"],"mode":"best_effort"}
```

Matching either an exact route or a key scope enables the selected detectors. An
empty detector list disables transformation. Flask scopes use the authenticated
user ID, falling back to the username. Native roleplay transport consumes the
authenticated scope supplied as `settings.piiKeyScope`; it never infers a scope
from a caller header. Route opt-in applies to all authenticated callers on that
route. Each list accepts at most 512 strings, each at most 256 characters; the
policy JSON accepts at most 64 KiB.

Managed Flask routes are `/v1/chat/completions`, `/v1/messages`, `/v1/responses`,
and `/intelligence/v1/chat/completions`. Native roleplay uses
`/v1/roleplay/chat/completions`. Raw passthrough is never transformed. Register
the Flask request hook after authentication and retention policy, before
idempotency claims and generation-cache lookup. Forward both configuration
variables to the Flask Container when using it behind the Worker.

## Detector coverage

Detectors examine JSON string values recursively, preserving object keys and
non-string values. Email matching uses bounded ASCII local parts and domain
labels; it does not recognize every valid international email address. Phone
matching accepts 10–15 digits with a leading plus or common spacing, parentheses,
or hyphens. Bare digit strings are not treated as phones. Card matching accepts
13–19 digits passing Luhn, excluding repeated identical digits. Matches do not
prove that a phone or card exists. Long, ambiguous numeric runs and formats outside
these limits can remain unchanged. There is no remote or semantic classifier.

## Request-local placeholders

The format is `__MLPII_<64 lowercase hexadecimal characters>__` (74 ASCII bytes).
The hexadecimal portion is HMAC-SHA-256 of the original value under a random
32-byte secret generated for that request. Repeated values share a placeholder
within one request. Separate requests have separate secrets and maps; they cannot
restore each other's tokens. Only exact tokens in the current request's map are
restored. Unknown or altered complete tokens pass through unchanged.

The map and secret remain in memory and are cleared on completion, error, or
disconnect, including cancellation before the first response chunk. They are not
logged, exported, cached, stored, or reused. A request permits at most 512 distinct
values and 1 MiB of serialized input, with a maximum JSON depth of 32 and 32,768
visited nodes. A collision fails the entire transformation.

## Streaming and errors

Restoration handles arbitrary network chunk boundaries, UTF-8 splits, JSON
escapes, and placeholder fragments across successive text deltas. It carries at
most 128 bytes of unresolved token prefixes across at most 16 text lanes. SSE
frames are limited to 64 KiB, nonstream JSON bodies to 1 MiB, and buffered output
from a single transport read to 1 MiB. Parser traversal
has the same depth and node limits as request traversal. Restored values are
serialized with JSON escaping. SSE events can move withheld text into a later
delta; their concatenated text retains the original order.

`required` is the default mode. A transformation or bound failure returns HTTP
422 with code `pii_redaction_failed` before dispatch. In `best_effort` mode, an
atomic transformation failure dispatches the unchanged firewall-checked payload
and records that redaction was skipped. A firewall refusal remains a refusal in
both modes. Policy validation errors disable the feature, rather than applying
required-mode rejection to existing traffic.

Invalid response JSON, a parser bound failure, or an unresolved terminal prefix
returns HTTP 502 with code `pii_rehydration_failed` for nonstream responses. A
stream that has already started is terminated with an error and its map cleared;
it does not invent a completion or authorize replay. Provider output must echo an
issued placeholder exactly to restore it. Rewritten or invented tokens cannot be
recovered. A trailing incomplete prefix is never emitted as successful text.

## Cache, replay, and operating limits

Redacted Flask requests use request-local zero-content retention, which bypasses
the exact and shared generation caches. An `Idempotency-Key` on a request that
actually requires redaction returns HTTP 400, `pii_idempotency_unsupported`, before
dispatch or replay. Requests with no PII retain their existing behavior.

Native roleplay transport reports `cacheable: false` and `replayable: false`
through `settings.onPIIDecision` for redacted requests. Its enclosing persistence
layer must honor that decision before storing any generation or replay body.
The transformed upstream request omits `Idempotency-Key`, and its request and
response carry `Cache-Control: no-store`. The request-local map is never part of
conversation state. Normal conversation retention is governed separately by the
content-retention policy.

Redaction adds local scanning, HMAC, and parsing work without classifier charges.
Long placeholders can increase provider input tokens and cost. Nonstream
restoration buffers a bounded body; streaming buffers a bounded frame and may
delay text while a token prefix is unresolved. This feature neither certifies a
provider's privacy guarantees nor removes the need to select the appropriate
retention policy. Verify detector coverage and provider echo behavior for the
formats used by the application.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"lint": "ruff check . --select E9,F63,F7,F82",
"typecheck": "mypy --ignore-missing-imports --follow-imports=skip --disable-error-code import-untyped --disable-error-code var-annotated --disable-error-code arg-type --disable-error-code assignment --disable-error-code operator app.py routes services providers streaming",
"test": "python -m pytest -q",
"test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs",
"test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs tests/test_pii_rehydration_worker.mjs",
"db:migrate": "python scripts/validate_sqlite_schema.py --apply",
"db:validate": "python scripts/validate_sqlite_schema.py",
"security:secrets": "python scripts/check_static_secrets.py",
Expand Down
14 changes: 11 additions & 3 deletions services/managed_dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,16 @@ def execute_managed_attempt(send, model, credential):
from services.managed_turn import before_provider_submission
before_provider_submission()
scope = current_scope()
if scope is None:
return send()
try:
response = send()
except BaseException:
scope.observer(model, credential)(classify_upstream_outcome(transport_failure="interrupted"))
if scope is not None:
scope.observer(model, credential)(classify_upstream_outcome(transport_failure="interrupted"))
raise
from services.pii_redaction import current_context
context = current_context()
if context is not None:
response.multillm_pii_context = context
return capture_managed_attempt(response, model, credential)


Expand Down Expand Up @@ -167,6 +170,11 @@ def close(self):

def observe_managed_response(downstream, upstream, *, stream=False):
"""Deliver classified final outcomes through the managed-dispatch callback."""
from services.pii_redaction import current_context
from services.pii_stream import rehydrate_response
context = getattr(upstream, "multillm_pii_context", None) or current_context()
if context is not None:
downstream = rehydrate_response(downstream, context, stream=stream)
from services.managed_turn import defer_affinity_response
if defer_affinity_response(downstream, upstream, stream=stream):
return downstream
Expand Down
Loading
Loading