diff --git a/docs/learned-cooldown.md b/docs/learned-cooldown.md new file mode 100644 index 00000000..77411bf8 --- /dev/null +++ b/docs/learned-cooldown.md @@ -0,0 +1,91 @@ +# Learned provider cooldowns + +Cooldown tuning uses natural throttle and recovery outcomes already recorded by +the credential pools. It sends no probes or extra provider requests. It adjusts +only a cooldown duration; route order, credential preference and permission to +replay a request remain controlled by their existing policies. + +## Configuration + +The default is disabled: + +```sh +LEARNED_COOLDOWN_MODE=off +LEARNED_COOLDOWN_POLICY_JSON='{}' +``` + +Empty values use these defaults. A malformed mode or policy disables tuning and +logs one warning without the supplied value. Modes are `off`, `shadow` and +`apply`. Policies contain only a `buckets` array, with at most 128 entries and +32 KiB of JSON. Each entry requires exactly `provider`, `model` and +`quota_bucket`. Provider must be a nonempty string; model and quota bucket are +nonempty strings or `null`, and at least one must be nonnull. Names match +exactly, including null values. Wildcards, duplicate fields, duplicate scopes, +control characters and unknown fields are rejected. + +To observe a specific model without changing its existing cooldown: + +```sh +LEARNED_COOLDOWN_MODE=shadow +LEARNED_COOLDOWN_POLICY_JSON='{"buckets":[{"provider":"ce-gpt-pro","model":"gpt-5","quota_bucket":null}]}' +``` + +Shadow logs the suggested seconds, opaque bucket digest, confidence and step +count. It preserves existing cooldowns and selection. Change the mode to +`apply` to use a suggested duration after sufficient consistent evidence. +For a named quota bucket, specify its exact name and the model recorded by the +transport. A quota-only observation uses `model:null`. A configured name learns +nothing until a pool receives natural outcomes with that exact scope. + +`MODEL_COOLDOWN_ENABLED` still controls the existing model-scoped admission +policy. Tuning does not enable it. With model cooldowns disabled, the pool keeps +its legacy credential-wide rest semantics while learning separate scope records. + +## Evidence and bounds + +The interval starts at 1–3,600 seconds. A later natural throttle in the same +credential/model/quota bucket raises its lower bound from the failed wait. +A genuine successful request after a throttle lowers its upper bound from the +elapsed wait. Success without a preceding throttle provides no recovery sample. +Authentication rejection, caller errors, cancellation, transport errors and +catalog validation probes provide no learning evidence. + +Three consistent directional observations allow a midpoint trial. Each state +allows at most 12 trial steps. Contradictory observations or nonincreasing +timestamps widen the interval and remove confidence. Apply uses the existing +cooldown until confidence returns. The existing model cooldown cap also bounds +an applied trial. Trusted Retry-After or reset seconds supplied +by the transport are a hard minimum, even when longer than 3,600 seconds. Such +hints can prevent any shortening. Invalid or nonfinite hints are ignored. No +header is parsed or rewritten by the learner. + +Each credential, provider, model and quota bucket has separate evidence. Model +and quota names are never stored. Credential-keyed HMAC digests provide stable +opaque identities across processes; credentials and prefixes are never stored +in learned state. Rotating a credential creates a new identity. + +## Storage, errors and retention + +Standalone Flask uses process-local state, bounded to 1,024 entries. It resets +on process restart and is not a distributed quota authority. State expires +seven days after creation; observations do not extend its lifetime. + +With `INTELLIGENCE_STORAGE_BACKEND=d1`, learned state uses the private +`/v1/state/learned-cooldown` storage domain, backed by the additive +`0031_learned_cooldown.sql` migration. Its fixed operations are `get` and `put`; +puts use revision comparison and a transactional capacity check. Storage is +bounded to 1,024 rows. Reads ignore expired state; writes reclaim expired rows. +Physical deletion therefore depends on subsequent writes rather than a timer. + +A missing table or unavailable D1 store returns a sanitized JSON 503 from the +private storage operation. The pool falls back to its existing cooldown with +one warning and continues serving traffic. A stale revision, invalid stored +state or full capacity also prevents applying a suggestion. No public route or +new response header is introduced. Default-off mode performs no learned-state +storage operations or reports. + +Natural traffic can be sparse, concurrent or affected by changing provider +limits. The inferred interval is a suggestion, not proof of a provider's quota +or successful recovery. Shadow observation and D1 writes add storage work; +they add no provider usage, usage refreshes or paid generation calls. Existing +transport accounting and retry safety remain authoritative. diff --git a/intelligence-migrations/0031_learned_cooldown.sql b/intelligence-migrations/0031_learned_cooldown.sql new file mode 100644 index 00000000..10b6b35a --- /dev/null +++ b/intelligence-migrations/0031_learned_cooldown.sql @@ -0,0 +1,20 @@ +-- Opaque identities and bounded, expiring cooldown evidence. +CREATE TABLE IF NOT EXISTS learned_cooldown ( + bucket_digest TEXT PRIMARY KEY, + credential_digest TEXT NOT NULL, + lower_seconds INTEGER NOT NULL CHECK (lower_seconds BETWEEN 1 AND 3600), + upper_seconds INTEGER NOT NULL CHECK (upper_seconds BETWEEN lower_seconds AND 3600), + trial_seconds INTEGER NOT NULL CHECK (trial_seconds BETWEEN 1 AND 3600), + samples INTEGER NOT NULL CHECK (samples BETWEEN 0 AND 2147483647), + consistent INTEGER NOT NULL CHECK (consistent BETWEEN 0 AND 3), + steps INTEGER NOT NULL CHECK (steps BETWEEN 0 AND 12), + confident INTEGER NOT NULL CHECK (confident IN (0, 1)), + last_kind TEXT NOT NULL, + last_throttle REAL, + last_observed REAL NOT NULL, + floor_until REAL NOT NULL, + created_at REAL NOT NULL, + expires_at REAL NOT NULL, + revision INTEGER NOT NULL CHECK (revision BETWEEN 1 AND 2147483647) +); +CREATE INDEX IF NOT EXISTS learned_cooldown_expiry ON learned_cooldown (expires_at); diff --git a/package.json b/package.json index b10757f5..445e4732 100644 --- a/package.json +++ b/package.json @@ -9,7 +9,7 @@ "lint": "ruff check . --select E9,F63,F7,F82", "typecheck": "mypy --ignore-missing-imports --follow-imports=skip --disable-error-code import-untyped --disable-error-code var-annotated --disable-error-code arg-type --disable-error-code assignment --disable-error-code operator app.py routes services providers streaming", "test": "python -m pytest -q", - "test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs tests/test_responses_state_worker.mjs tests/test_gateway_batches_worker.mjs tests/test_semantic_generation_cache_worker.mjs tests/test_canary_traffic_worker.mjs tests/test_context_pages_worker.mjs tests/test_observability_exports_worker.mjs tests/test_prompt_injection_worker.mjs tests/test_pii_rehydration_worker.mjs tests/test_wave5_worker.mjs tests/test_stream_cost_breaker_worker.mjs tests/test_realtime_worker.mjs", + "test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_managed_idempotency_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_gateway_alerts_worker.mjs tests/test_session_tiers_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs tests/test_generation_deadline_worker.mjs tests/test_wave4_worker.mjs tests/test_responses_state_worker.mjs tests/test_gateway_batches_worker.mjs tests/test_semantic_generation_cache_worker.mjs tests/test_canary_traffic_worker.mjs tests/test_context_pages_worker.mjs tests/test_observability_exports_worker.mjs tests/test_prompt_injection_worker.mjs tests/test_pii_rehydration_worker.mjs tests/test_wave5_worker.mjs tests/test_learned_cooldown_worker.mjs tests/test_stream_cost_breaker_worker.mjs tests/test_realtime_worker.mjs", "db:migrate": "python scripts/validate_sqlite_schema.py --apply", "db:validate": "python scripts/validate_sqlite_schema.py", "security:secrets": "python scripts/check_static_secrets.py", diff --git a/services/credential_pool.py b/services/credential_pool.py index 6c1fbfc9..446b3e01 100644 --- a/services/credential_pool.py +++ b/services/credential_pool.py @@ -20,6 +20,7 @@ from services.auth_primitives import usable_credential from services.model_cooldown import ModelCooldownExhausted, model_cooldown, settings from services import pool_reset_schedule as reset_schedule +from services import learned_cooldown as learned from services.upstream_outcome import UpstreamOutcome, classify_upstream_outcome REFUSED_REST_SECONDS = 300 @@ -234,11 +235,17 @@ def record( ) return current = time.monotonic() if now is None else now + delay = learned.adjust_cooldown( + provider, key, outcome or classify_upstream_outcome(status), + model=model, quota_bucket=quota_bucket, now=now, + current_seconds=RATE_LIMITED_REST_SECONDS, + retry_after_seconds=retry_after_seconds, + ) with cls._lock: if status in REFUSED_STATUSES: cls._resting[(provider, key)] = current + REFUSED_REST_SECONDS elif status in RATE_LIMITED_STATUSES: - cls._resting[(provider, key)] = current + RATE_LIMITED_REST_SECONDS + cls._resting[(provider, key)] = current + delay elif 200 <= status < 300: cls._resting.pop((provider, key), None) diff --git a/services/learned_cooldown.py b/services/learned_cooldown.py new file mode 100644 index 00000000..9b88dcf4 --- /dev/null +++ b/services/learned_cooldown.py @@ -0,0 +1,320 @@ +"""Bounded passive cooldown trials from natural, scoped traffic outcomes.""" + +from __future__ import annotations + +import hashlib +import hmac +import json +import logging +import math +import os +import re +import threading +import time +from collections.abc import Mapping +from dataclasses import asdict, dataclass, replace + +from services.upstream_outcome import UpstreamOutcome + +logger = logging.getLogger(__name__) +TTL_SECONDS = 7 * 24 * 3600 +MAX_ENTRIES = 1024 +MAX_STEPS = 12 +_warned: set[str] = set() +_warning_lock = threading.Lock() +_HASH = re.compile(r"[0-9a-f]{64}") + + +def _warn_once(name: str) -> None: + with _warning_lock: + if name in _warned: + return + _warned.add(name) + logger.warning("%s; learned cooldown uses existing behaviour", name) + + +@dataclass(frozen=True) +class LearnedSettings: + mode: str = "off" + buckets: tuple[tuple[str, str | None, str | None], ...] = () + + +def _unique_object(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate_field") + result[key] = value + return result + + +def _scope_text(value: object) -> bool: + return isinstance(value, str) and 1 <= len(value) <= 256 and value == value.strip() and not any( + ord(character) < 32 or ord(character) == 127 or character == "*" for character in value + ) + + +def settings(environ: Mapping[str, str] | None = None) -> LearnedSettings: + source = os.environ if environ is None else environ + mode = (source.get("LEARNED_COOLDOWN_MODE") or "off").strip().lower() or "off" + if mode == "off": + return LearnedSettings() + if mode not in {"shadow", "apply"}: + _warn_once("Invalid LEARNED_COOLDOWN_MODE") + return LearnedSettings() + raw = (source.get("LEARNED_COOLDOWN_POLICY_JSON") or "{}").strip() or "{}" + try: + if len(raw.encode("utf-8")) > 32768: + raise ValueError("oversized_policy") + policy = json.loads(raw, object_pairs_hook=_unique_object) + if not isinstance(policy, dict) or set(policy) not in (set(), {"buckets"}): + raise ValueError("invalid_policy") + rules = policy.get("buckets", []) + if not isinstance(rules, list) or len(rules) > 128: + raise ValueError("invalid_buckets") + buckets = [] + for rule in rules: + if not isinstance(rule, dict) or set(rule) != {"provider", "model", "quota_bucket"}: + raise ValueError("invalid_scope") + provider, model, quota = rule["provider"], rule["model"], rule["quota_bucket"] + if not _scope_text(provider) or any(value is not None and not _scope_text(value) for value in (model, quota)): + raise ValueError("invalid_identity") + if model is None and quota is None: + raise ValueError("unscoped_policy") + bucket = (provider, model, quota) + if bucket in buckets: + raise ValueError("duplicate_scope") + buckets.append(bucket) + return LearnedSettings(mode, tuple(buckets)) + except (ValueError, TypeError, UnicodeError, RecursionError): + _warn_once("Invalid LEARNED_COOLDOWN_POLICY_JSON") + return LearnedSettings() + + +def _finite(value: object) -> bool: + if not isinstance(value, (int, float)) or isinstance(value, bool): + return False + try: + return math.isfinite(value) + except OverflowError: + return False + + +def _floor(value: float | None) -> float: + return float(value) if _finite(value) and value is not None and value > 0 else 0.0 + + +@dataclass(frozen=True) +class LearnedState: + credential_digest: str + lower_seconds: int = 1 + upper_seconds: int = 3600 + trial_seconds: int = 60 + samples: int = 0 + consistent: int = 0 + steps: int = 0 + confident: bool = False + last_kind: str = "none" + last_throttle: float | None = None + last_observed: float = 0 + floor_until: float = 0 + created_at: float = 0 + expires_at: float = 0 + revision: int = 0 + + +def valid_state(value: object, credential_digest: str, now: float) -> bool: + if not isinstance(value, dict) or set(value) != set(LearnedState.__dataclass_fields__): + return False + if value["credential_digest"] != credential_digest or not _HASH.fullmatch(credential_digest): + return False + ranges = {"lower_seconds": (1, 3600), "upper_seconds": (1, 3600), "trial_seconds": (1, 3600), + "samples": (0, 2_147_483_647), "consistent": (0, 3), "steps": (0, MAX_STEPS), "revision": (1, 2_147_483_647)} + if any(type(value[key]) is not int or not low <= value[key] <= high for key, (low, high) in ranges.items()): + return False + if value["lower_seconds"] > value["upper_seconds"] or type(value["confident"]) is not bool or value["last_kind"] not in {"none", "failure", "success"}: + return False + times = [value[key] for key in ("last_observed", "floor_until", "created_at", "expires_at")] + if any(not _finite(item) or item < 0 for item in times): + return False + throttle = value["last_throttle"] + return (throttle is None or _finite(throttle) and value["created_at"] <= throttle <= value["last_observed"]) and ( + value["created_at"] <= value["last_observed"] <= now < value["expires_at"] + and value["expires_at"] == value["created_at"] + TTL_SECONDS + ) + + +@dataclass(frozen=True) +class Suggestion: + bucket_digest: str + suggested_seconds: float + lower_seconds: int + upper_seconds: int + samples: int + steps: int + confident: bool + minimum_seconds: float + + +def _widen(state: LearnedState, observed: float) -> LearnedState: + return replace(state, lower_seconds=max(1, min(state.lower_seconds // 2, math.floor(observed))), + upper_seconds=min(3600, max(state.upper_seconds * 2, math.ceil(observed) + 1)), + consistent=0, confident=False, last_kind="none") + + +def advance(state: LearnedState, kind: str, now: float, floor: float) -> LearnedState: + """Infer delay bounds only when a real request followed a previous throttle.""" + result = state + direction = "none" + if state.samples and now <= state.last_observed: + result = _widen(state, max(1, state.lower_seconds)) + elif state.last_throttle is not None: + elapsed = now - state.last_throttle + if kind == "failure": + if elapsed >= state.upper_seconds: + result = _widen(state, elapsed) + else: + result = replace(state, lower_seconds=max(state.lower_seconds, min(3600, math.floor(elapsed) + 1))) + direction = kind + elif elapsed < state.lower_seconds or now < state.floor_until: + result = _widen(state, elapsed) + else: + result = replace(state, upper_seconds=min(state.upper_seconds, max(1, min(3600, math.ceil(elapsed))))) + direction = kind + if direction != "none": + count = min(3, result.consistent + 1) if result.last_kind == direction else 1 + result = replace(result, consistent=count, last_kind=direction) + if count >= 3 and result.steps < MAX_STEPS: + result = replace(result, trial_seconds=(result.lower_seconds + result.upper_seconds) // 2, + steps=result.steps + 1, consistent=0, confident=True) + floor_until = max(state.floor_until, now + floor) + minimum = min(3600, max(1, math.ceil(max(0, floor_until - now)))) + lower = max(minimum, result.lower_seconds) + upper = max(lower, result.upper_seconds) + return replace(result, lower_seconds=lower, upper_seconds=upper, + samples=min(2_147_483_647, state.samples + 1), revision=state.revision + 1, + last_observed=max(state.last_observed, now), floor_until=floor_until, + last_throttle=max(state.last_observed, now) if kind == "failure" else None) + + +class LearnedCooldown: + """Bounded local authority or fixed D1 compare-and-swap operations.""" + + def __init__(self, *, max_entries: int = MAX_ENTRIES): + self._max_entries = min(MAX_ENTRIES, max(1, max_entries)) + self._lock = threading.RLock() + self._entries: dict[str, LearnedState] = {} + + @staticmethod + def _identities(provider: str, credential: str, model: str | None, quota_bucket: str | None) -> tuple[str, str]: + # Keyed hashes are stable across Containers without storing a key or prefix. + key = credential.encode("utf-8") + credential_digest = hmac.new(key, json.dumps(["learned-credential-v1", provider]).encode(), hashlib.sha256).hexdigest() + bucket = json.dumps(["learned-bucket-v1", provider, model, quota_bucket], separators=(",", ":")).encode() + return hmac.new(key, bucket, hashlib.sha256).hexdigest(), credential_digest + + @staticmethod + def _call(operation: str, **values): + from services import control_state_d1 + return control_state_d1.call("learned-cooldown", operation, **values) + + def snapshot(self, *, now: float | None = None) -> dict[str, dict]: + current = time.time() if now is None else now + with self._lock: + self._prune(current) + return {key: asdict(value) for key, value in self._entries.items()} + + def _prune(self, now: float) -> None: + self._entries = {key: value for key, value in self._entries.items() if value.expires_at > now} + + def _read(self, bucket: str, credential: str, now: float, distributed: bool) -> LearnedState | None: + if not distributed: + self._prune(now) + return self._entries.get(bucket) + response = self._call("get", bucket_digest=bucket, credential_digest=credential, now=now) + if not isinstance(response, dict) or response.get("version") != 1 or "entry" not in response or response.get("error"): + raise ValueError("invalid_storage_response") + entry = response["entry"] + if entry is None: + return None + if not valid_state(entry, credential, now): + raise ValueError("invalid_storage_entry") + return LearnedState(**entry) + + def _write(self, bucket: str, state: LearnedState, expected: int, now: float, distributed: bool) -> bool: + if not distributed: + if bucket not in self._entries and len(self._entries) >= self._max_entries: + return False + self._entries[bucket] = state + return True + response = self._call("put", bucket_digest=bucket, credential_digest=state.credential_digest, + now=now, expected_revision=expected, entry=asdict(state)) + if not isinstance(response, dict) or response.get("version") != 1 or type(response.get("stored")) is not bool or response.get("error"): + raise ValueError("invalid_storage_response") + return response["stored"] + + def observe(self, provider: str, credential: str, outcome: UpstreamOutcome, *, + model: str | None = None, quota_bucket: str | None = None, now: float | None = None, + retry_after_seconds: float | None = None, default_seconds: float = 60, + config: LearnedSettings | None = None, + ) -> Suggestion | None: + config = settings() if config is None else config + if config.mode == "off" or (provider, model, quota_bucket) not in config.buckets: + return None + kind = "failure" if outcome.credential_health == "throttled" and outcome.reason in {"http_throttled", "quota_exhausted"} else ( + "success" if outcome.provider_health == "success" and outcome.credential_health == "accepted" and outcome.reason == "http_success" else None + ) + if kind is None: + return None + current = time.time() if now is None else now + if not _finite(current) or current < 0: + return None + bucket, digest = self._identities(provider, credential, model, quota_bucket) + distributed = os.environ.get("INTELLIGENCE_STORAGE_BACKEND", "").strip() == "d1" + with self._lock: + try: + state = self._read(bucket, digest, current, distributed) + if kind == "success" and (state is None or state.last_throttle is None): + return None + if state is None: + trial = min(3600, max(1, math.ceil(default_seconds))) + state = LearnedState(digest, trial_seconds=trial, created_at=current, expires_at=current + TTL_SECONDS) + if state.revision >= 2_147_483_647: + return None + updated = advance(state, kind, current, _floor(retry_after_seconds)) + if not self._write(bucket, updated, state.revision, current, distributed): + _warn_once("Learned cooldown capacity or revision conflict") + return None + except Exception: # noqa: BLE001 - optional storage cannot interrupt traffic. + _warn_once("Learned cooldown storage unavailable") + return None + minimum = max(_floor(retry_after_seconds), updated.floor_until - current, 0) + suggested = max(minimum, min(updated.upper_seconds, max(updated.trial_seconds, updated.lower_seconds))) + return Suggestion(bucket, suggested, updated.lower_seconds, updated.upper_seconds, + updated.samples, updated.steps, updated.confident, minimum) + + +learned_cooldown = LearnedCooldown() + + +def adjust_cooldown(provider: str, credential: str, outcome: UpstreamOutcome, *, + current_seconds: float, model: str | None = None, quota_bucket: str | None = None, + retry_after_seconds: float | None = None, now: float | None = None, + max_seconds: int = 3600, +) -> float: + """Change only the recorded duration; selection and replay policy stay outside.""" + config = settings() + if config.mode == "off" or (provider, model, quota_bucket) not in config.buckets: + return current_seconds + suggestion = learned_cooldown.observe(provider, credential, outcome, model=model, quota_bucket=quota_bucket, + retry_after_seconds=retry_after_seconds, now=now, default_seconds=current_seconds, config=config) + if suggestion is None: + return current_seconds + if config.mode == "shadow": + logger.info("Learned cooldown bucket=%s suggested_seconds=%s confidence=%s steps=%s", + suggestion.bucket_digest, suggestion.suggested_seconds, suggestion.confident, suggestion.steps) + return current_seconds + if outcome.credential_health != "throttled": + return current_seconds + proposed = min(max_seconds, 3600, suggestion.suggested_seconds) if suggestion.confident else current_seconds + return max(suggestion.minimum_seconds, proposed) diff --git a/services/model_cooldown.py b/services/model_cooldown.py index 462cd7df..f8389d55 100644 --- a/services/model_cooldown.py +++ b/services/model_cooldown.py @@ -16,6 +16,7 @@ from flask import jsonify from error_handlers import APIError +from services import learned_cooldown as learned from services.upstream_outcome import UpstreamOutcome logger = logging.getLogger(__name__) @@ -156,6 +157,11 @@ def record( scope = self._scope(model, quota_bucket) identity = self.credential_id(provider, credential) wide_auth = outcome.credential_health == "rejected" and credential_wide_auth + delay = learned.adjust_cooldown( + provider, credential, outcome, model=model, quota_bucket=quota_bucket, + current_seconds=_delay(retry_after_seconds, fallback_seconds, max_seconds), + retry_after_seconds=retry_after_seconds, now=now, max_seconds=max_seconds, + ) with self._lock: self._prune(current) if outcome.credential_health == "accepted": @@ -167,7 +173,7 @@ def record( ): return None entry = (identity, None if wide_auth else scope) - until = current + _delay(retry_after_seconds, fallback_seconds, max_seconds) + until = current + delay if entry not in self._entries and len(self._entries) >= self._max_entries: # Do not evict live state and silently dispatch a cooling key. self._overflow_until = max(self._overflow_until, until) diff --git a/services/nanogpt_key_pool.py b/services/nanogpt_key_pool.py index d3f1baf4..42786100 100644 --- a/services/nanogpt_key_pool.py +++ b/services/nanogpt_key_pool.py @@ -12,6 +12,7 @@ from services.auth_primitives import is_placeholder_credential from services.model_cooldown import ModelCooldownExhausted, model_cooldown, settings from services import pool_reset_schedule as reset_schedule +from services import learned_cooldown as learned from services.upstream_outcome import UpstreamOutcome, classify_upstream_outcome _DIRECT_KEY_NAMES = ("NANOGPT_API_KEY", "NANO_GPT_KEY") @@ -385,12 +386,18 @@ def record_result( if cls._active_key == key: cls._active_requests += 1 return + delay = learned.adjust_cooldown( + cls._model_cooldown_provider, key, outcome or classify_upstream_outcome(status), + model=model, quota_bucket=quota_bucket, now=now, + current_seconds=rejected_cooldown_seconds, + retry_after_seconds=retry_after_seconds, + ) if is_nanogpt_credential_rejection(status): cls.invalidate( key, status, check_ttl_seconds=check_ttl_seconds, - rejected_cooldown_seconds=rejected_cooldown_seconds, + rejected_cooldown_seconds=delay, now=now, ) return diff --git a/tests/fixtures/secret_firewall_coverage.json b/tests/fixtures/secret_firewall_coverage.json index c4696421..e48643c6 100644 --- a/tests/fixtures/secret_firewall_coverage.json +++ b/tests/fixtures/secret_firewall_coverage.json @@ -316,6 +316,7 @@ "worker/knowledge/skills-store.mjs", "worker/knowledge/skills-validation.mjs", "worker/knowledge/skills.mjs", + "worker/learned-cooldown-d1.mjs", "worker/log.mjs", "worker/managed-state-dispatch.mjs", "worker/media-files.mjs", diff --git a/tests/test_learned_cooldown.py b/tests/test_learned_cooldown.py new file mode 100644 index 00000000..18553bb9 --- /dev/null +++ b/tests/test_learned_cooldown.py @@ -0,0 +1,322 @@ +"""Passive cooldown learning with synthetic traffic and storage.""" + +import importlib +import json +import math +from concurrent.futures import ThreadPoolExecutor +from unittest.mock import Mock + +import pytest + +PROVIDER = "ce-gpt-pro" +KEYS = ["synthetic-first", "synthetic-second"] +RULE = {"provider": PROVIDER, "model": "A", "quota_bucket": None} + + +@pytest.fixture(autouse=True) +def isolated(monkeypatch): + global learned, credentials, nano, cooldown, classify, store + cooldown = importlib.import_module("services.model_cooldown") + learned = cooldown.learned + credentials = importlib.import_module("services.credential_pool") + nano = importlib.import_module("services.nanogpt_key_pool") + classify = importlib.import_module("services.upstream_outcome").classify_upstream_outcome + monkeypatch.setenv("LEARNED_COOLDOWN_MODE", "apply") + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", json.dumps({"buckets": [RULE]})) + monkeypatch.setenv("INTELLIGENCE_STORAGE_BACKEND", "local") + monkeypatch.setenv("MODEL_COOLDOWN_ENABLED", "true") + monkeypatch.setenv("MODEL_COOLDOWN_MAX_SECONDS", "3600") + monkeypatch.setenv("POOL_RESET_SCHEDULING_ENABLED", "false") + monkeypatch.setenv("PROMPT_CACHE_AFFINITY_ENABLED", "false") + store = learned.LearnedCooldown() + monkeypatch.setattr(learned, "learned_cooldown", store) + monkeypatch.setattr(credentials.CredentialPool, "keys", staticmethod(lambda provider: KEYS)) + for pool in (credentials.CredentialPool, nano.NanoGPTKeyPool, nano.NanoGPTUnifiedKeyPool): + pool.reset() + cooldown.model_cooldown.reset() + yield + for pool in (credentials.CredentialPool, nano.NanoGPTKeyPool, nano.NanoGPTUnifiedKeyPool): + pool.reset() + cooldown.model_cooldown.reset() + + +def observe(status, now, **kwargs): + return store.observe(PROVIDER, KEYS[0], classify(status), model="A", now=now, **kwargs) + + +def recoveries(count=3, start=100, delay=60): + result = None + for index in range(count): + observe(429, start + index * (delay + 1)) + result = observe(200, start + index * (delay + 1) + delay) + return result + + +def test_three_recoveries_choose_binary_trial_then_failures_raise_lower(): + first = recoveries(2) + assert not first.confident + assert first.steps == 0 + assert first.upper_seconds == 60 + result = recoveries(1, start=222) + assert result.confident and result.steps == 1 + assert result.suggested_seconds == 30 + observe(429, 300) + for now in (330, 361, 393): + result = observe(429, now) + assert result.lower_seconds == 33 + assert result.steps == 2 + assert result.suggested_seconds == 46 + + +def test_success_requires_later_throttle_in_same_bucket(): + assert observe(200, 100) is None + observe(429, 100) + other = store.observe(PROVIDER, KEYS[1], classify(200), model="A", now=160) + assert other is None + result = observe(200, 160) + assert result.upper_seconds == 60 and result.samples == 2 + assert observe(200, 200) is None + + +def test_contradictions_and_out_of_order_noise_widen_and_disable_apply(): + recoveries() + observe(429, 300) + result = observe(429, 400) + assert result.upper_seconds > 60 and not result.confident + width = result.upper_seconds - result.lower_seconds + result = observe(200, 399) + assert result.upper_seconds - result.lower_seconds >= width + assert not result.confident + + +def test_minimum_floor_including_outside_learning_bounds(monkeypatch): + recoveries() + delay = learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", + now=300, current_seconds=60, retry_after_seconds=90) + assert delay >= 90 + assert observe(200, 320).lower_seconds >= 70 + assert learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", + now=401, current_seconds=10, retry_after_seconds=7200) >= 7200 + + +@pytest.mark.parametrize("hint", [math.nan, math.inf, -1, True, "90"]) +def test_untrusted_hints_do_not_become_floors(hint): + recoveries() + assert learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", + now=300, current_seconds=60, retry_after_seconds=hint) == 30 + + +def test_max_steps_and_absolute_ttl(): + result = recoveries(45) + assert result.steps == 12 + assert 1 <= result.lower_seconds <= result.upper_seconds <= 3600 + entry = next(iter(store.snapshot(now=3000).values())) + assert learned.valid_state(entry, entry["credential_digest"], 3000) + assert len(store.snapshot(now=100 + learned.TTL_SECONDS)) == 0 + result = observe(429, 100 + learned.TTL_SECONDS) + assert result.steps == 0 and result.samples == 1 + + +def test_trusted_floor_persists_for_later_observations(): + recoveries() + learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", now=300, + current_seconds=60, retry_after_seconds=7200) + assert learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", now=400, + current_seconds=60) >= 7100 + + +def test_default_mode_warning_once_and_no_value(monkeypatch, caplog): + learned._warned.clear() + monkeypatch.setenv("LEARNED_COOLDOWN_MODE", "private-malformed-mode") + assert learned.settings().mode == "off" + assert learned.settings().mode == "off" + assert len(caplog.records) == 1 + assert "private-malformed-mode" not in caplog.text + + +def test_quota_only_opt_in_uses_exact_match(monkeypatch): + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", json.dumps({"buckets": [{**RULE, "model": None, "quota_bucket": "shared"}]})) + assert store.observe(PROVIDER, KEYS[0], classify(429), model="A", quota_bucket="shared", now=100) is None + assert store.observe(PROVIDER, KEYS[0], classify(429), quota_bucket="shared", now=100) is not None + + +def test_model_cooldown_cap_and_trusted_floor(monkeypatch): + recoveries() + monkeypatch.setenv("MODEL_COOLDOWN_MAX_SECONDS", "10") + credentials.CredentialPool.record(PROVIDER, KEYS[0], 429, model="A", now=300) + assert credentials.CredentialPool.available(PROVIDER, KEYS[:1], model="A", now=310) == KEYS[:1] + credentials.CredentialPool.record(PROVIDER, KEYS[0], 429, model="A", now=400, retry_after_seconds=90) + assert credentials.CredentialPool.available(PROVIDER, KEYS[:1], model="A", now=410) == [] + assert credentials.CredentialPool.available(PROVIDER, KEYS[:1], model="A", now=490) == KEYS[:1] + + +@pytest.mark.parametrize("status", [400, 401, 402, 403, 404, 422, 500, 503]) +def test_auth_caller_and_provider_failures_are_not_learned(status): + assert observe(status, 100) is None + assert store.snapshot(now=100) == {} + cancelled = classify(429, cancelled=True) + assert store.observe(PROVIDER, KEYS[0], cancelled, model="A", now=100) is None + + +@pytest.mark.parametrize("mode", ["off", "", "malformed"]) +def test_off_preserves_state_cooldowns_order_and_no_storage(monkeypatch, mode): + monkeypatch.setenv("LEARNED_COOLDOWN_MODE", mode) + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", "malformed-private-value") + monkeypatch.setattr(store, "observe", Mock(side_effect=AssertionError("storage touched"))) + credentials.CredentialPool.record(PROVIDER, KEYS[0], 429, model="A", now=100) + assert credentials.CredentialPool.select(PROVIDER, model="A", now=101) == KEYS[1] + assert credentials.CredentialPool.available(PROVIDER, KEYS, model="A", now=160) == KEYS + assert credentials.CredentialPool._resting == {} + assert store.snapshot(now=100) == {} + + +@pytest.mark.parametrize("seconds", [0, 60]) +def test_off_passes_nanogpt_rejection_cooldown_through(monkeypatch, seconds): + monkeypatch.setenv("LEARNED_COOLDOWN_MODE", "off") + monkeypatch.setenv("MODEL_COOLDOWN_ENABLED", "false") + invalidate = Mock() + monkeypatch.setattr(nano.NanoGPTKeyPool, "invalidate", invalidate) + nano.NanoGPTKeyPool.record_result(KEYS[0], 429, rejected_cooldown_seconds=seconds, now=100) + assert invalidate.call_args.kwargs["rejected_cooldown_seconds"] == seconds + + +def test_shadow_reports_suggestion_without_changing_cooldowns(monkeypatch, caplog): + monkeypatch.setenv("LEARNED_COOLDOWN_MODE", "shadow") + recoveries() + with caplog.at_level("INFO", logger=learned.__name__): + credentials.CredentialPool.record(PROVIDER, KEYS[0], 429, model="A", now=300) + assert credentials.CredentialPool.available(PROVIDER, KEYS[:1], model="A", now=330) == [] + assert credentials.CredentialPool.available(PROVIDER, KEYS[:1], model="A", now=360) == KEYS[:1] + assert "suggested_seconds=30" in caplog.text + assert all(key not in caplog.text for key in KEYS) + + +@pytest.mark.parametrize("scoped", [False, True]) +@pytest.mark.parametrize("kind", ["credential", "nano", "unified"]) +def test_registered_pool_record_paths_apply_only_duration(monkeypatch, scoped, kind): + monkeypatch.setenv("MODEL_COOLDOWN_ENABLED", "true" if scoped else "false") + provider = PROVIDER if kind == "credential" else ("nanogpt" if kind == "nano" else "nanogpt-unified") + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", json.dumps({"buckets": [{**RULE, "provider": provider}]})) + if kind == "credential": + record = lambda status, now: credentials.CredentialPool.record(provider, KEYS[0], status, model="A", now=now) + available = lambda now: credentials.CredentialPool.available(provider, KEYS, model="A", now=now) + else: + pool = nano.NanoGPTKeyPool if kind == "nano" else nano.NanoGPTUnifiedKeyPool + record = lambda status, now: pool.record_result(KEYS[0], status, model="A", now=now) + available = lambda now: [pool.select_available_key(KEYS, model="A", now=now)] + for start in (100, 161, 222): + record(429, start) + record(200, start + 60) + record(429, 300) + assert KEYS[0] not in available(329) + assert KEYS[0] in available(330) + assert not classify(429).replay_permission + + +def test_model_quota_credential_and_provider_isolation(monkeypatch): + rules = [RULE, {**RULE, "model": "B"}, {**RULE, "quota_bucket": "q"}, {**RULE, "provider": "ce-gpt-plus"}] + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", json.dumps({"buckets": rules})) + recoveries() + for changes in ({"model": "B"}, {"quota_bucket": "q"}, {"provider": "ce-gpt-plus"}, {"credential": KEYS[1]}): + args = {"provider": PROVIDER, "credential": KEYS[0], "model": "A", "outcome": classify(429), "now": 300, **changes} + result = store.observe(**args) + assert result.steps == 0 and not result.confident + encoded = json.dumps(store.snapshot(now=300)) + assert all(text not in encoded for text in [*KEYS, PROVIDER, "ce-gpt-plus"]) + + +@pytest.mark.parametrize("policy", ["[]", '{"extra":1}', '{"buckets":[{}]}', + '{"buckets":[{"provider":"*","model":"A","quota_bucket":null}]}', + '{"buckets":[{"provider":"ce-gpt-pro","model":true,"quota_bucket":null}]}', + '{"buckets":[{"provider":"ce-gpt-pro","model":null,"quota_bucket":null}]}', + '{"buckets":[],"buckets":[]}', '{"buckets":[NaN]}']) +def test_strict_policy_fails_off_once_without_values(monkeypatch, caplog, policy): + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", policy) + learned._warned.clear() + assert learned.settings().mode == "off" + assert learned.settings().mode == "off" + assert len(caplog.records) == 1 and policy not in caplog.text + + +def test_empty_defaults_no_opt_in_and_bounded_concurrency(monkeypatch): + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", "") + assert observe(429, 100) is None + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", json.dumps({"buckets": [RULE]})) + small = learned.LearnedCooldown(max_entries=2) + with ThreadPoolExecutor(max_workers=2) as executor: + results = list(executor.map(lambda index: small.observe(PROVIDER, f"synthetic-{index}", classify(429), model="A", now=100), range(10))) + assert sum(result is not None for result in results) == 2 + assert len(small.snapshot(now=100)) == 2 + + +def test_policy_budget_counts_utf8_bytes(monkeypatch): + rules = [{**RULE, "model": "界" * 250 + str(index)} for index in range(50)] + policy = json.dumps({"buckets": rules}, ensure_ascii=False) + assert len(policy) < 32768 < len(policy.encode("utf-8")) + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", policy) + assert learned.settings().mode == "off" + + +def test_d1_missing_table_falls_back_once_and_off_never_calls(monkeypatch, caplog): + monkeypatch.setenv("INTELLIGENCE_STORAGE_BACKEND", "d1") + call = Mock(side_effect=RuntimeError("sensitive storage detail")) + monkeypatch.setattr(store, "_call", call) + learned._warned.clear() + for now in (100, 200): + assert learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", now=now, current_seconds=60) == 60 + assert len(caplog.records) == 1 and "sensitive" not in caplog.text + monkeypatch.setenv("LEARNED_COOLDOWN_MODE", "off") + call.reset_mock() + assert observe(429, 300) is None + call.assert_not_called() + + +def test_fixed_d1_transport_uses_named_allowlist_endpoint(monkeypatch): + control = importlib.import_module("services.control_state_d1") + call = Mock(return_value={"version": 1, "entry": None}) + monkeypatch.setattr(control, "call", call) + digest = "a" * 64 + assert store._call("get", bucket_digest=digest, credential_digest=digest, now=100)["entry"] is None + call.assert_called_once_with("learned-cooldown", "get", bucket_digest=digest, credential_digest=digest, now=100) + + +def test_d1_contract_cas_conflict_and_invalid_rows_fail_to_existing(monkeypatch): + monkeypatch.setenv("INTELLIGENCE_STORAGE_BACKEND", "d1") + rows = {} + conflict = [False] + def call(operation, **values): + assert all(key not in json.dumps(values) for key in KEYS) + if operation == "get": + return {"version": 1, "entry": rows.get(values["bucket_digest"])} + if conflict[0]: + return {"version": 1, "stored": False} + rows[values["bucket_digest"]] = values["entry"] + return {"version": 1, "stored": True} + monkeypatch.setattr(store, "_call", call) + recoveries() + assert next(iter(rows.values()))["steps"] == 1 + conflict[0] = True + assert learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", now=300, current_seconds=60) == 60 + conflict[0] = False + next(iter(rows.values()))["lower_seconds"] = -1 + assert learned.adjust_cooldown(PROVIDER, KEYS[0], classify(429), model="A", now=400, current_seconds=60) == 60 + + +def test_catalog_probes_are_not_observations(monkeypatch): + monkeypatch.setenv("LEARNED_COOLDOWN_POLICY_JSON", json.dumps({"buckets": [{**RULE, "provider": "nanogpt"}]})) + probe = Mock(return_value=200) + assert nano.NanoGPTKeyPool.select_key(KEYS, probe, model="A", now=100) == KEYS[0] + assert probe.call_count == 1 and store.snapshot(now=100) == {} + + +def test_migration_additive_idempotent_preserves_old_rows(): + import sqlite3 + from pathlib import Path + db = sqlite3.connect(":memory:") + db.execute("CREATE TABLE old_state (value TEXT)") + db.execute("INSERT INTO old_state VALUES ('kept')") + sql = Path("intelligence-migrations/0031_learned_cooldown.sql").read_text() + db.executescript(sql) + db.executescript(sql) + assert db.execute("SELECT value FROM old_state").fetchone() == ("kept",) + assert "learned_cooldown" in {row[0] for row in db.execute("SELECT name FROM sqlite_master")} diff --git a/tests/test_learned_cooldown_worker.mjs b/tests/test_learned_cooldown_worker.mjs new file mode 100644 index 00000000..8212f7f0 --- /dev/null +++ b/tests/test_learned_cooldown_worker.mjs @@ -0,0 +1,127 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { readFileSync } from "node:fs"; +import { DatabaseSync } from "node:sqlite"; +import { handleLearnedCooldown, MAX_ENTRIES, TTL_SECONDS } from "../worker/learned-cooldown-d1.mjs"; + +const bucket = "a".repeat(64), credential = "b".repeat(64); +const env = { LEARNED_COOLDOWN_MODE: "apply" }; +const entry = (changes = {}) => ({ credential_digest: credential, lower_seconds: 1, upper_seconds: 60, + trial_seconds: 30, samples: 6, consistent: 0, steps: 1, confident: true, last_kind: "success", + last_throttle: null, last_observed: 100, floor_until: 0, created_at: 100, + expires_at: 100 + TTL_SECONDS, revision: 1, ...changes }); +const get = (changes = {}) => ({ version: 1, operation: "get", bucket_digest: bucket, + credential_digest: credential, now: 100, ...changes }); +const put = (changes = {}) => ({ ...get(), operation: "put", expected_revision: 0, entry: entry(), ...changes }); +function database(migrated = true) { + const sqlite = new DatabaseSync(":memory:"); + sqlite.exec("CREATE TABLE old_state (value TEXT); INSERT INTO old_state VALUES ('kept')"); + const migration = readFileSync(new URL("../intelligence-migrations/0031_learned_cooldown.sql", import.meta.url), "utf8"); + if (migrated) { sqlite.exec(migration); sqlite.exec(migration); } + const execute = (sql, args) => { + const numbered = /\?[1-9]/.test(sql); + const statement = sqlite.prepare(sql.replace(/\?(\d+)/g, (_, number) => `$v${number}`)); + const rows = numbered ? statement.all(Object.fromEntries(args.map((value, index) => [`v${index + 1}`, value]))) : statement.all(...args); + return { results: rows.map(row => ({ ...row })), meta: { changes: sqlite.prepare("SELECT changes() AS n").get().n } }; + }; + const db = { prepare(sql) { return { sql, args: [], bind(...args) { this.args = args; return this; }, + async first() { return execute(sql, this.args).results[0] ?? null; }, async all() { return execute(sql, this.args); }, + async run() { return execute(sql, this.args); } }; }, + async batch(items) { + sqlite.exec("BEGIN IMMEDIATE"); + try { const result = items.map(item => execute(item.sql, item.args)); sqlite.exec("COMMIT"); return result; } + catch (error) { sqlite.exec("ROLLBACK"); throw error; } + } }; + return { db, sqlite }; +} +async function rpc(db, body, settings = env) { + return handleLearnedCooldown(db, body, settings); +} + +test("off empty malformed mode leaves storage untouched", async () => { + let calls = 0; + const db = { prepare() { calls++; throw Error("storage touched"); } }; + for (const mode of [undefined, "", "off", "invalid"]) { + const result = await rpc(db, put(), { LEARNED_COOLDOWN_MODE: mode }); + assert.equal(result.status, 404); + } + assert.equal(calls, 0); +}); + +test("opaque entries round trip in apply and shadow without affecting old rows", async () => { + for (const mode of ["apply", "shadow"]) { + const { db, sqlite } = database(); + assert.equal((await (await rpc(db, put(), { LEARNED_COOLDOWN_MODE: mode })).json()).stored, true); + assert.deepEqual((await (await rpc(db, get())).json()).entry, entry()); + assert.equal(sqlite.prepare("SELECT value FROM old_state").get().value, "kept"); + assert.equal(sqlite.prepare("SELECT bucket_digest FROM learned_cooldown").get().bucket_digest, bucket); + assert.deepEqual((await (await rpc(db, get({ credential_digest: "c".repeat(64) }))).json()).entry, null); + } +}); + +test("compare and swap rejects simultaneous stale updates", async () => { + const { db } = database(); + const saved = await Promise.all([rpc(db, put()), rpc(db, put())]); + const results = await Promise.all(saved.map(response => response.json())); + assert.equal(results.filter(result => result.stored).length, 1); + const updated = put({ expected_revision: 1, entry: entry({ revision: 2, samples: 7, steps: 2 }) }); + assert.equal((await (await rpc(db, updated)).json()).stored, true); + assert.equal((await (await rpc(db, updated)).json()).stored, false); +}); + +test("strict fixed operations reject malformed payloads before D1", async () => { + let calls = 0; + const db = { prepare() { calls++; throw Error(); } }; + for (const body of [get({ version: 2 }), get({ bucket_digest: "raw-key" }), get({ now: NaN }), + get({ extra: "private" }), get({ operation: "sql" }), put({ expected_revision: true }), + put({ entry: entry({ lower_seconds: 0 }) }), put({ entry: entry({ upper_seconds: 3601 }) }), + put({ entry: entry({ steps: 13 }) }), put({ entry: entry({ consistent: 4 }) }), + put({ entry: entry({ confident: 1 }) }), put({ entry: entry({ revision: 2 }) }), + put({ entry: entry({ extra: "key-prefix" }) }), put({ entry: entry({ expires_at: 200 }) }), + put({ entry: entry({ last_throttle: 101 }) }), put({ entry: entry({ last_kind: "auth" }) })]) { + assert.equal((await rpc(db, body)).status, 400); + } + assert.equal(calls, 0); +}); + +test("missing table and storage outage return sanitized 503 with one warning", async () => { + const { db } = database(false); + const warnings = [], original = console.warn; + console.warn = (...values) => warnings.push(values.join(" ")); + try { + for (const body of [get(), put(), get()]) { + const response = await rpc(db, body); + assert.equal(response.status, 503); + assert.equal(response.headers.get("cache-control"), "no-store"); + assert.equal((await response.json()).error.code, "learned_cooldown_storage_unavailable"); + } + assert.equal(warnings.length, 1); + assert.doesNotMatch(warnings.join(), /SELECT|sqlite|no such table|synthetic/); + } finally { console.warn = original; } +}); + +test("absolute seven day expiry permits a new revision without reviving expired state", async () => { + const { db, sqlite } = database(); + await rpc(db, put()); + const now = 100 + TTL_SECONDS; + assert.equal((await (await rpc(db, get({ now }))).json()).entry, null); + assert.equal((await (await rpc(db, put({ now, entry: entry({ created_at: now, last_observed: now, expires_at: now + TTL_SECONDS }) }))).json()).stored, true); + assert.equal(sqlite.prepare("SELECT COUNT(*) AS n FROM learned_cooldown").get().n, 1); +}); + +test("global capacity is bounded and expired rows are reclaimed", async () => { + const { db, sqlite } = database(); + const seed = sqlite.prepare("INSERT INTO learned_cooldown (bucket_digest, credential_digest, lower_seconds, upper_seconds, trial_seconds, samples, consistent, steps, confident, last_kind, last_throttle, last_observed, floor_until, created_at, expires_at, revision) VALUES (?, ?, 1, 60, 30, 6, 0, 1, 1, 'success', NULL, 100, 0, 100, ?, 1)"); + for (let index = 0; index < MAX_ENTRIES; index++) seed.run(index.toString(16).padStart(64, "0"), credential, 100 + TTL_SECONDS); + assert.equal((await (await rpc(db, put())).json()).stored, false); + assert.equal(sqlite.prepare("SELECT COUNT(*) AS n FROM learned_cooldown").get().n, MAX_ENTRIES); + const now = 100 + TTL_SECONDS; + assert.equal((await (await rpc(db, put({ now, entry: entry({ created_at: now, last_observed: now, expires_at: now + TTL_SECONDS }) }))).json()).stored, true); +}); + +test("corrupt stored bounds never become an applied suggestion", async () => { + const { db, sqlite } = database(); + await rpc(db, put()); + sqlite.exec("UPDATE learned_cooldown SET last_kind='corrupt'"); + assert.equal((await rpc(db, get())).status, 503); +}); diff --git a/worker/container-env.mjs b/worker/container-env.mjs index d5116a3d..87b67ddd 100644 --- a/worker/container-env.mjs +++ b/worker/container-env.mjs @@ -40,6 +40,8 @@ const DIRECT_ENV_KEYS = [ "CONNECTION_PROFILES_DB_PATH", "CONFIG_REVISION_SYNC_ENABLED", "CONFIG_SYNC_TTL_SECONDS", + "LEARNED_COOLDOWN_MODE", + "LEARNED_COOLDOWN_POLICY_JSON", "REALTIME_ENABLED", "REALTIME_PROVIDERS_JSON", "STREAM_COST_BREAKER_ENABLED", diff --git a/worker/d1-schema.mjs b/worker/d1-schema.mjs index 5c1351dd..c9ce40f9 100644 --- a/worker/d1-schema.mjs +++ b/worker/d1-schema.mjs @@ -10,7 +10,7 @@ export const REQUIRED_D1_TABLES = Object.freeze(["intelligence_policy", "intelli "usage_events", "usage_daily", "usage_batches", "media_jobs", "media_job_items", "shadow_eval_samples", "shadow_eval_results", "shadow_eval_config", "shadow_eval_policy_backups", "prompt_templates", - "config_snapshot_revisions", "config_snapshots", "config_snapshot_applications", "control_revisions", "tool_grants", "generation_cache", "usage_reservation_budgets", "usage_reservations", "usage_reservation_transitions", "session_tiers", "gateway_alert_rules", "gateway_alert_events", "managed_idempotency", "context_pages", "canary_traffic", "semantic_generation_cache", "gateway_batch_files", "gateway_batches", "gateway_batch_items", "hosted_responses", "realtime_tickets", "realtime_sessions"]); + "config_snapshot_revisions", "config_snapshots", "config_snapshot_applications", "control_revisions", "tool_grants", "generation_cache", "usage_reservation_budgets", "usage_reservations", "usage_reservation_transitions", "session_tiers", "gateway_alert_rules", "gateway_alert_events", "managed_idempotency", "context_pages", "canary_traffic", "semantic_generation_cache", "gateway_batch_files", "gateway_batches", "gateway_batch_items", "hosted_responses", "realtime_tickets", "realtime_sessions", "learned_cooldown"]); /** * Readiness for the D1 schema. A deploy that skipped `wrangler d1 migrations apply` reports diff --git a/worker/learned-cooldown-d1.mjs b/worker/learned-cooldown-d1.mjs new file mode 100644 index 00000000..7f7f641a --- /dev/null +++ b/worker/learned-cooldown-d1.mjs @@ -0,0 +1,92 @@ +/** Private storage for passive cooldown evidence; no provider dispatch or retries. */ +export const TTL_SECONDS = 7 * 24 * 3600; +export const MAX_ENTRIES = 1024; +export const MAX_RPC_BYTES = 4096; +const HASH = /^[0-9a-f]{64}$/; +const FIELDS = ["credential_digest", "lower_seconds", "upper_seconds", "trial_seconds", "samples", "consistent", + "steps", "confident", "last_kind", "last_throttle", "last_observed", "floor_until", "created_at", "expires_at", "revision"]; +const COLUMNS = FIELDS.join(", "); +const object = value => value !== null && typeof value === "object" && !Array.isArray(value); +const exact = (value, fields) => object(value) && Object.keys(value).length === fields.length && fields.every(key => Object.hasOwn(value, key)); +const number = value => typeof value === "number" && Number.isFinite(value) && value >= 0; +const integer = (value, low, high) => Number.isSafeInteger(value) && value >= low && value <= high; +const digest = value => typeof value === "string" && HASH.test(value); +const warned = new Set(); +function warnOnce(name) { + if (warned.has(name)) return; + warned.add(name); + console.warn(`${name}; learned cooldown uses existing behaviour`); +} +const reply = (value, status = 200) => Response.json({ version: 1, ...value }, { status, headers: { "cache-control": "no-store" } }); +const failure = (code, status) => reply({ error: { code, message: "Learned cooldown state is unavailable." } }, status); + +export function learnedCooldownEnabled(env = {}) { + const mode = (env.LEARNED_COOLDOWN_MODE || "off").trim().toLowerCase() || "off"; + if (["shadow", "apply"].includes(mode)) return true; + if (mode !== "off") warnOnce("Invalid LEARNED_COOLDOWN_MODE"); + return false; +} + +export function validLearnedState(value, credential, now) { + if (!exact(value, FIELDS) || !digest(credential) || value.credential_digest !== credential + || !integer(value.lower_seconds, 1, 3600) || !integer(value.upper_seconds, value.lower_seconds, 3600) + || !integer(value.trial_seconds, 1, 3600) || !integer(value.samples, 0, 2147483647) + || !integer(value.consistent, 0, 3) || !integer(value.steps, 0, 12) || !integer(value.revision, 1, 2147483647) + || typeof value.confident !== "boolean" || !["none", "failure", "success"].includes(value.last_kind) + || !["last_observed", "floor_until", "created_at", "expires_at"].every(key => number(value[key]))) return false; + return (value.last_throttle === null || number(value.last_throttle) && value.last_throttle >= value.created_at + && value.last_throttle <= value.last_observed) && value.created_at <= value.last_observed + && value.last_observed <= now && now < value.expires_at && value.expires_at === value.created_at + TTL_SECONDS; +} + +const INSERT = `INSERT INTO learned_cooldown (bucket_digest, ${COLUMNS}) + SELECT ?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16 + WHERE ?17=0 AND (SELECT COUNT(*) FROM learned_cooldown) < ${MAX_ENTRIES} + ON CONFLICT(bucket_digest) DO NOTHING`; +const UPDATE = `UPDATE learned_cooldown SET ${FIELDS.map((field, index) => `${field}=?${index + 2}`).join(", ")} + WHERE bucket_digest=?1 AND credential_digest=?2 AND revision=?17 AND expires_at>?18 + AND created_at=?14 AND expires_at=?15 AND last_observed<=?12 + AND samples<=?6 AND steps<=?8`; + +async function read(db, body) { + const entry = await db.prepare(`SELECT ${COLUMNS} FROM learned_cooldown + WHERE bucket_digest=? AND credential_digest=? AND expires_at>?`) + .bind(body.bucket_digest, body.credential_digest, body.now).first(); + if (!entry) return reply({ entry: null }); + if (![0, 1].includes(entry.confident)) throw new Error("invalid_storage_entry"); + entry.confident = entry.confident === 1; + if (!validLearnedState(entry, body.credential_digest, body.now)) throw new Error("invalid_storage_entry"); + return reply({ entry }); +} + +async function write(db, body) { + const args = [body.bucket_digest, ...FIELDS.map(field => field === "confident" ? Number(body.entry[field]) : body.entry[field]), body.expected_revision]; + // The transactional batch serializes both capacity checks and stale revisions. + const results = await db.batch([ + db.prepare("DELETE FROM learned_cooldown WHERE expires_at<=?").bind(body.now), + db.prepare(body.expected_revision === 0 ? INSERT : UPDATE) + .bind(...args, ...(body.expected_revision === 0 ? [] : [body.now])), + ]); + return reply({ stored: results[1].meta.changes === 1 }); +} + +export async function handleLearnedCooldown(db, body, env = {}) { + if (!learnedCooldownEnabled(env)) return failure("not_found", 404); + const base = ["version", "operation", "bucket_digest", "credential_digest", "now"]; + if (!object(body) || body.version !== 1 || !["get", "put"].includes(body.operation) + || !digest(body.bucket_digest) || !digest(body.credential_digest) || !number(body.now) + || body.now > 8640000000000 || !exact(body, body.operation === "get" ? base : [...base, "expected_revision", "entry"])) { + return failure("invalid_request", 400); + } + if (body.operation === "put" && (!integer(body.expected_revision, 0, 2147483646) + || !validLearnedState(body.entry, body.credential_digest, body.now) || body.entry.revision !== body.expected_revision + 1)) { + return failure("invalid_request", 400); + } + try { + if (!db) throw new Error("storage_unavailable"); + return await (body.operation === "get" ? read(db, body) : write(db, body)); + } catch { + warnOnce("Learned cooldown storage unavailable"); + return failure("learned_cooldown_storage_unavailable", 503); + } +}