Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions docs/usage-reservations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Durable usage reservations

Dollar-budgeted keys can retain conservative usage holds after uncertain provider
handoff. The default is `USAGE_RESERVATIONS_ENABLED=false`; existing accounting,
responses, headers, and storage remain unchanged.

Enable with:

```text
USAGE_RESERVATIONS_ENABLED=true
USAGE_HOLD_REVIEW_AFTER_SECONDS=259200
```

An empty value uses the default. Invalid settings disable reservations and emit one
warning per setting, without its value. Review age must be 1–2147483647 seconds.
Review age marks an unresolved hold as `needs_review`; it never releases money.

A reservation moves from `reserved` to `dispatched`, then to `settled` with measured
provider cost or to `unknown` when completion or usage is uncertain. Disconnects after
handoff retain the estimate. Confirmed cancellation before dispatch and cache hits
before dispatch settle at zero with a release basis. A successful response with missing
or partial billable usage retains its hold; missing counters remain null rather than
becoming measured zero. Requests without a configured monetary budget are unchanged.

Admission and settlement use atomic storage transactions. All unresolved holds count
against both current UTC budget periods, even when created in an older period.
Settled charges belong to their admission day and month. Settlement and transition
identifiers reject conflicting replays; these identifiers grant no provider retry
permission. An estimate must fit every configured budget. Any unpriced eligible model
refuses admission with HTTP 503 `unpriced_reservation`. A priced request that exceeds
a budget returns HTTP 429 `budget_exceeded`. Holds round upwards and budget caps round downwards. Money is represented in integer units of
USD 0.0000000001 with a maximum representable amount of USD 900719.9254740991.
Measured spend can exceed the estimate; it is recorded without truncation to the budget.

`/v1/usage` includes content-free `budget.reservations` counters: `held_usd`, `reserved`,
`dispatched`, `unknown`, and `needs_review`. `in_flight_usd` includes unresolved durable
holds, and remaining budgets subtract them. Unavailable storage reports unavailable
totals and null remaining budgets, rather than reporting spendable money.

Unknown reservations require explicit reconciliation. A trusted administrator supplies
the reservation ID, current revision, measured cost, a provider evidence reference and
an audit reason. An explicitly authorized adjustment instead uses an adjustment basis
and audit reason. References and reasons are bounded identifiers, not provider response
bodies or free-form content. Concurrent reconciliations use compare-and-swap; one claim
wins and the other returns `reservation_conflict`. Reconciliation preserves previously
observed counters and retains transition history. No public reconciliation endpoint is
provided by the storage module; the caller must verify administrator authorization
before invoking the reconciliation contract. The private handler is not a public API.

SQLite uses the usage database path (`USAGE_DB_PATH`) and initializes additive tables.
Keep this file on durable shared storage for processes using that authority. D1 uses
`INTELLIGENCE_DB` and the additive `0020_usage_reservations.sql` migration. Its private
JSON endpoint is `http://intelligence.internal/v1/reservations`. Container access uses
an injected `D1ReservationStore` transport; it must not fall back to ephemeral SQLite
when D1 is configured. An absent D1 binding, missing tables, or transport failure returns
HTTP 503 `usage_reservations_unavailable` before provider submission. No new binding
or public route is needed. Rehearse and apply the migration before enabling the feature.

Request owners must persist handoff immediately before provider submission and finalize
with classified, measured accounting. Native owners use `createReservationLifecycle`
with verified identity, all eligible prices and initial legacy spend. They supply measured
cost and nullable counters through the existing flat finalization event (or its `usage`
field), plus the classified outcome and cancellation outcome.
Flask owners use `request_accounting.mark_dispatched` before submission; the existing
cancellation observer also records handoff when it binds an upstream response. Failed
finalization retains money, including a crashed reservation still in `reserved` or
`dispatched`. Review these stale records by an explicit unknown transition before
reconciliation; never assume that lack of a completion proves no provider charge.

Before first enabling, drain existing requests and flush the legacy usage ledger across
all replicas. The first reservation snapshots existing spend for each principal and UTC
period once; subsequent charges come from the durable reservation authority, while the
usage ledger continues to report measured costs without charging them again. Use one
authority and consistent settings on every replica. Do not change the flag during
in-flight requests or disable it with unresolved holds. Enabling after mixed legacy and
durable writers requires an operator-led reconciliation of the baseline. Automatic
external invoice matching, payments, and PostgreSQL reservation storage are not provided.

Hold and audit records contain no prompts, responses, credentials or provider bodies.
They are retained without automatic expiry or deletion, including after review age.
Their storage must be monitored and backed up. This accounting does not establish that
a provider invoice matches local prices, token estimates, or reported metering.
37 changes: 37 additions & 0 deletions intelligence-migrations/0020_usage_reservations.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
CREATE TABLE IF NOT EXISTS usage_reservation_budgets (
principal TEXT NOT NULL,
period TEXT NOT NULL,
baseline_units INTEGER NOT NULL CHECK (baseline_units >= 0),
PRIMARY KEY (principal, period)
);
CREATE TABLE IF NOT EXISTS usage_reservations (
id TEXT PRIMARY KEY,
principal TEXT NOT NULL,
day TEXT NOT NULL,
month TEXT NOT NULL,
amount_units INTEGER NOT NULL CHECK (amount_units >= 0),
charged_units INTEGER,
state TEXT NOT NULL CHECK (state IN ('reserved', 'dispatched', 'settled', 'unknown', 'reconciled')),
basis TEXT,
input_tokens INTEGER,
output_tokens INTEGER,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
handoff_at INTEGER,
revision INTEGER NOT NULL DEFAULT 0,
transition_id TEXT NOT NULL,
settlement_id TEXT UNIQUE
);
CREATE INDEX IF NOT EXISTS usage_reservations_principal ON usage_reservations (principal, state, day, month);
CREATE TABLE IF NOT EXISTS usage_reservation_transitions (
transition_id TEXT PRIMARY KEY,
reservation_id TEXT NOT NULL REFERENCES usage_reservations(id),
previous_state TEXT,
state TEXT NOT NULL,
at INTEGER NOT NULL,
basis TEXT,
evidence TEXT,
reason TEXT,
document TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS usage_reservation_transitions_history ON usage_reservation_transitions (reservation_id, at);
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"lint": "ruff check . --select E9,F63,F7,F82",
"typecheck": "mypy --ignore-missing-imports --follow-imports=skip --disable-error-code import-untyped --disable-error-code var-annotated --disable-error-code arg-type --disable-error-code assignment --disable-error-code operator app.py routes services providers streaming",
"test": "python -m pytest -q",
"test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_deferred_tools_worker.mjs",
"test:worker": "node --test tests/test_cloudflare_worker.mjs tests/test_cascades_worker.mjs tests/test_cascades_dashboard.mjs tests/test_image_quality_worker.mjs tests/test_secret_scan_worker.mjs tests/test_secret_firewall_worker.mjs tests/test_secret_firewall_coverage_worker.mjs tests/test_roleplay_compaction_budget_worker.mjs tests/test_roleplay_compaction_model_worker.mjs tests/test_roleplay_stream_validation_worker.mjs tests/test_aihubmix_worker.mjs tests/test_gpt_image_moderation_worker.mjs tests/test_dashboard_scripts.mjs tests/test_operations_dashboard.mjs tests/test_opencode_reasoning_worker.mjs tests/test_opencode_thinking_worker.mjs tests/test_nanogpt_reasoning_worker.mjs tests/test_nanogpt_speed_routing_worker.mjs tests/test_sse_heartbeat_worker.mjs tests/test_roleplay_worker.mjs tests/test_roleplay_config_worker.mjs tests/test_roleplay_glm_failover_worker.mjs tests/test_roleplay_intelligence_worker.mjs tests/test_roleplay_memory_worker.mjs tests/test_roleplay_compatibility_worker.mjs tests/test_roleplay_history_reconciliation_worker.mjs tests/test_roleplay_output_contract_analysis.mjs tests/test_roleplay_output_contract_cleanup_worker.mjs tests/test_roleplay_output_contract_nonstream_worker.mjs tests/test_roleplay_output_contract_worker.mjs tests/test_roleplay_performance_worker.mjs tests/test_roleplay_quality_routing_worker.mjs tests/test_roleplay_reasoning_worker.mjs tests/test_roleplay_session_storage_worker.mjs tests/test_roleplay_transport_failure_worker.mjs tests/test_roleplay_unlimited_worker.mjs tests/test_roleplay_workbench_worker.mjs tests/test_roleplay_candidate_context_worker.mjs tests/test_roleplay_operator_worker.mjs tests/test_workbench_stream.mjs tests/test_intelligence_worker.mjs tests/test_intelligence_d1_worker.mjs tests/test_intelligence_auth_worker.mjs tests/test_intelligence_outbound_worker.mjs tests/test_intelligence_operator.mjs tests/test_control_users_worker.mjs tests/test_control_audit_worker.mjs tests/test_access_sso_worker.mjs tests/test_d1_schema_worker.mjs tests/test_shadow_eval_worker.mjs tests/test_auto_routes_worker.mjs tests/test_ai_outbound_worker.mjs tests/test_knowledge_*.mjs tests/test_control_state_worker.mjs tests/test_route_health_worker.mjs tests/test_gateway_mcp_worker.mjs tests/test_usage_ledger_worker.mjs tests/test_media_outbound_worker.mjs tests/test_media_files_worker.mjs tests/test_media_jobs_worker.mjs tests/test_tool_repair_worker.mjs tests/test_prometheus_worker.mjs tests/test_evaluation_noise_floor_worker.mjs tests/test_prompt_templates_worker.mjs tests/test_config_snapshots_worker.mjs tests/test_usage_reservations_worker.mjs tests/test_wave3_hub_worker.mjs tests/test_wave3_transport_worker.mjs tests/test_admission_leases_worker.mjs tests/test_edge_request_metrics_worker.mjs tests/test_request_cancellation_worker.mjs tests/test_mcp_contract_drift_worker.mjs tests/test_zero_content_retention_worker.mjs tests/test_prompt_cache_cost_worker.mjs tests/test_config_revision_sync_worker.mjs tests/test_shared_generation_cache_worker.mjs tests/test_deferred_tools_worker.mjs",
"db:migrate": "python scripts/validate_sqlite_schema.py --apply",
"db:validate": "python scripts/validate_sqlite_schema.py",
"security:secrets": "python scripts/check_static_secrets.py",
Expand Down
Loading
Loading