Repository navigation
feat(routing): add opt-in reviewed configuration snapshots #24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # Control configuration snapshots | ||
|
|
||
| `CONFIG_SNAPSHOTS_ENABLED` defaults to false. All snapshot paths return 404 | ||
| while disabled, and existing auto-route saves keep their current payload, | ||
| storage and response behavior. No provider calls or generation are involved. | ||
|
|
||
| The public administration paths run in Flask for both direct administration | ||
| and Worker-forwarded Container traffic. They use the existing session login, | ||
| administrator check and CSRF protection. They are not gateway-key API routes. | ||
| The existing private `auto_routes` adapter stores configuration in D1; there is | ||
| no new public Worker dispatch or private outbound endpoint. | ||
|
|
||
| Before enabling, apply `intelligence-migrations/0015_config_snapshots.sql` | ||
| (`npx wrangler d1 migrations apply multillm-intelligence --remote`), and configure | ||
| `INTELLIGENCE_STORAGE_BACKEND=d1` for Flask. Enable the flag on both Worker | ||
| and Container. Enabled snapshots without durable D1 or any required table | ||
| fail closed with JSON HTTP 503. Enabled ordinary D1 route saves also fail | ||
| closed if the snapshot migration is missing. Local SQLite/PostgreSQL snapshot | ||
| storage is outside this initial private-D1 implementation boundary. | ||
|
|
||
| ## Review and apply | ||
|
|
||
| `GET /admin/config/snapshots` returns `current_revision`, snapshot metadata, | ||
| and the latest 100 application audit records. It never includes stored | ||
| configuration documents or account objects. The only supported domain is | ||
| `auto_routes`; its revision begins at zero when the feature is introduced. | ||
|
|
||
| Create a review with `POST /admin/config/snapshots`: | ||
|
|
||
| ```json | ||
| {"domain":"auto_routes","base_revision":0,"configuration":{"routes":[{"route_id":"auto:review","candidates":["openai:model-a","nanogpt:model-b"]}]}} | ||
| ``` | ||
|
|
||
| HTTP 201 returns the immutable snapshot metadata and ID. Creation requires | ||
| that `base_revision` still matches; it does not activate any route. | ||
| Configuration is a patch containing only the listed route IDs and their | ||
| ordered candidate model IDs. Omitted routes and seeded defaults remain active. | ||
| No deletion or wholesale replacement is performed. | ||
|
|
||
| `GET /admin/config/snapshots/<id>/diff` returns only route IDs and the approved | ||
| before/after model ID lists, plus the base and current revisions. Flask displays | ||
| the effective seeded defaults when there is no durable override. Diff pages | ||
| contain at most 20 routes; use `?offset=<next_offset>` until `next_offset` is | ||
| null. A review spanning pages must be restarted if `current_revision` changes. | ||
| An unknown snapshot returns 404. | ||
|
|
||
| Apply with `POST /admin/config/snapshots/<id>/apply`: | ||
|
|
||
| ```json | ||
| {"current_revision":0,"confirm":true} | ||
| ``` | ||
|
|
||
| Both literal `confirm=true` and the current integer revision are required. | ||
| A mismatch, stale review, repeat application, or patch that would exceed the | ||
| existing 200-row durable route read bound returns 409. The snapshot base | ||
| must equal the submitted revision. The transactional D1 batch compares the | ||
| revision and exact durable route state, writes the reviewed routes, increments | ||
| the revision, and appends an audit record with an opaque administrator digest. | ||
| A route write failure rolls back the revision, audit and route updates together. | ||
| Ordinary enabled D1 route saves advance the same revision. A fingerprint also | ||
| rejects a review if routes were saved while the flag was temporarily disabled. | ||
|
|
||
| A successful apply clears this Container’s route cache. Other Containers can | ||
| retain their last-good route copy for the existing 30-second cache interval; | ||
| an outage preserves their last-good routing. There is no cluster-wide immediate | ||
| cache invalidation. Seeded candidate orders retired by the existing auto-route | ||
| service are rejected at creation, rather than claiming an apply that the route | ||
| service would subsequently replace with current defaults. | ||
|
|
||
| To roll back, create a new snapshot containing the previous candidate order | ||
| at the current revision, review its diff and explicitly apply it. Historical | ||
| snapshots and application records are retained; rollback adds a revision. | ||
|
|
||
| ## Bounds, secrets and errors | ||
|
|
||
| There are at most 100 snapshots per domain. Each configuration is bounded to | ||
| 256 KiB, 200 routes and 16 distinct candidates per route; model IDs are bounded | ||
| to 256 characters. The existing private transport also bounds its complete | ||
| request envelope to 256 KiB, so configurations at that boundary require room | ||
| for metadata. Oversized requests return 413. Count exhaustion or a concurrent | ||
| revision change at creation returns 409. No automatic archival or deletion is | ||
| performed and there is no delete endpoint: once a domain has 100 snapshots, | ||
| creation returns 409 until unapplied snapshots are removed from D1 (applied | ||
| ones are referenced by their audit records). Application records are append-only and grow with | ||
| successful applies; list responses are bounded to the latest 100 records. | ||
|
|
||
| Only the exact route configuration schema is accepted. Provider keys, gateway | ||
| keys, headers, prompts, connections, environment fields and arbitrary metadata | ||
| are rejected, not redacted into storage. Known credential-shaped identifiers | ||
| are also rejected. Snapshots contain configuration, a durable-state digest and | ||
| opaque administrator attribution only; no runtime account or environment | ||
| objects are read into snapshot documents. | ||
|
|
||
| Malformed configuration returns 400, non-admin sessions return 403, and | ||
| unauthenticated sessions follow the existing login flow. Every snapshot response | ||
| uses `Cache-Control: no-store`. Storage failures return JSON 503 with | ||
| `config_snapshot_storage_unavailable`; they do not claim a completed apply. | ||
| Private submissions are never retried. If a response is lost after submission, | ||
| check the current revision and audit records before making a new review. | ||
|
|
||
| Snapshot operations consume D1 reads/writes and bounded private transport | ||
| capacity. They make no upstream generation calls, but D1 usage can have storage | ||
| and request costs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| -- Reviewed configuration only. No keys, headers, connection profiles or request content. | ||
| CREATE TABLE IF NOT EXISTS config_snapshot_revisions ( | ||
| domain TEXT PRIMARY KEY CHECK(domain = 'auto_routes'), | ||
| revision INTEGER NOT NULL CHECK(revision >= 0 AND revision <= 9007199254740991) | ||
| ); | ||
| CREATE TABLE IF NOT EXISTS config_snapshots ( | ||
| id TEXT PRIMARY KEY, | ||
| domain TEXT NOT NULL CHECK(domain = 'auto_routes'), | ||
| base_revision INTEGER NOT NULL CHECK(base_revision >= 0), | ||
| configuration TEXT NOT NULL CHECK(json_valid(configuration)), | ||
| base_fingerprint TEXT NOT NULL CHECK(length(base_fingerprint) = 64), | ||
| created_at TEXT NOT NULL, | ||
| created_by TEXT NOT NULL, | ||
| size_bytes INTEGER NOT NULL CHECK(size_bytes > 0 AND size_bytes <= 262144) | ||
| ); | ||
| CREATE INDEX IF NOT EXISTS config_snapshots_domain_created ON config_snapshots(domain, created_at, id); | ||
| CREATE TABLE IF NOT EXISTS config_snapshot_applications ( | ||
| id TEXT PRIMARY KEY, | ||
| domain TEXT NOT NULL CHECK(domain = 'auto_routes'), | ||
| snapshot_id TEXT NOT NULL REFERENCES config_snapshots(id), | ||
| base_revision INTEGER NOT NULL, | ||
| revision INTEGER NOT NULL, | ||
| applied_at TEXT NOT NULL, | ||
| applied_by TEXT NOT NULL, | ||
| UNIQUE(domain, revision) | ||
| ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| """Administrator-only snapshot review; existing session and CSRF protections apply.""" | ||
| from functools import wraps | ||
|
|
||
| from flask import jsonify, request | ||
|
|
||
| from error_handlers import APIError | ||
|
|
||
| from route_helpers import login_required | ||
| from routes.core import require_admin_dashboard_user | ||
| from services import config_snapshots | ||
|
|
||
|
|
||
| def json_errors(view): | ||
| @wraps(view) | ||
| def wrapped(*args, **kwargs): | ||
| try: | ||
| return view(*args, **kwargs) | ||
| except APIError as error: | ||
| return jsonify(error.to_dict()), error.status_code | ||
| return wrapped | ||
|
|
||
|
|
||
| def register_config_snapshot_routes(app): | ||
| def disabled_config_snapshots(): | ||
| if (request.path == "/admin/config/snapshots" or request.path.startswith("/admin/config/snapshots/")) and not config_snapshots.enabled(): | ||
| return jsonify({"error": "not_found"}), 404 | ||
|
|
||
| # Feature-off requests must be 404 even before session and CSRF checks. | ||
| app.before_request_funcs.setdefault(None, []).insert(0, disabled_config_snapshots) | ||
|
|
||
| @app.after_request | ||
| def private_config_snapshots(response): | ||
| if request.path == "/admin/config/snapshots" or request.path.startswith("/admin/config/snapshots/"): | ||
| response.headers["Cache-Control"] = "no-store" | ||
| return response | ||
|
|
||
| @app.route("/admin/config/snapshots", methods=["GET", "POST"]) | ||
| @json_errors | ||
| @login_required | ||
| def admin_config_snapshots(): | ||
| user = require_admin_dashboard_user() | ||
| if request.method == "GET": | ||
| return jsonify(config_snapshots.call("snapshot_list")) | ||
| return jsonify(config_snapshots.create(request.get_json(silent=True), user, app.config["API_BASE_URLS"])), 201 | ||
|
|
||
| @app.get("/admin/config/snapshots/<identifier>/diff") | ||
| @json_errors | ||
| @login_required | ||
| def admin_config_snapshot_diff(identifier): | ||
| require_admin_dashboard_user() | ||
| return jsonify(config_snapshots.diff(identifier, request.args.get("offset", "0"))) | ||
|
|
||
| @app.post("/admin/config/snapshots/<identifier>/apply") | ||
| @json_errors | ||
| @login_required | ||
| def admin_config_snapshot_apply(identifier): | ||
| user = require_admin_dashboard_user() | ||
| return jsonify(config_snapshots.apply(identifier, request.get_json(silent=True), user)) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The create handler reads and parses the entire request before installing any snapshot-specific body limit, so it inherits the application's 32 MiB default rather than the advertised 256 KiB bound. An authenticated request near that limit is materialized by
get_json, then serialized and encoded again byvalidate_configurationbefore it can return 413, which can consume several times the request size per worker; setrequest.max_content_lengthin the early snapshot gate, as the prompt-template routes do, so oversized snapshots are rejected before parsing.Useful? React with 👍 / 👎.