Skip to content

feat(routing): add opt-in role-scoped sticky session tiers - #45

Merged
ALikesToCode merged 3 commits into
feat/usage-reservationsfrom
feat/session-tiers
Oct 9, 2026
Merged

ALikesToCode merged 3 commits into
feat/usage-reservationsfrom
feat/session-tiers

Conversation

@ALikesToCode

Copy link
Copy Markdown
Owner

Summary

Adds opt-in role-scoped sticky session tiers behind SESSION_TIER_MODE=sticky. The default is off, and with it routing, response headers and storage are unchanged. A tier keeps an approved route stable across tool calls in the main, delegation and aux lanes, so a tool-using session doesn't switch models halfway through a tool exchange.

  • What carries the tier:
    • Managed requests carry session_tier metadata (session, lane, approved model and tier).
    • Roleplay requests carry it inside routing.session_tier.
    • Requests without tier metadata, explicit models and pinned roleplay routes keep their normal routing.
  • When the approval can change: only at a user-turn boundary, after a successful completion with all matching tool results.
    • Failures, cancellations, partial or unrelated results, and continuations without a new user turn cannot open a boundary.
    • Outstanding tool calls are tracked as hashed IDs (at most 128).
  • Policy checks still win: billing, entitlement, capability, context limits, credential availability and cooldown take precedence over stickiness.
    • A managed fallback must match both the approved native model and the reviewed quality_tier.
    • A roleplay fallback can use only the identical native model name on another eligible provider.
    • The actual fallback route is recorded separately from the approved route.
  • Invalidation and leases:
    • Expiry (SESSION_TIER_TTL_SECONDS, default 1,800), a policy revision change, or a confirmed credential failure invalidates only the affected lane.
    • An in-flight request holds a lease, and a concurrent request for that lane gets 409 session_tier_busy.
  • Opted-in roleplay turns: automatic continuation, refusal fallback and output-contract repair are off for that turn.
  • Storage:
    • Roleplay keeps the binding in its existing session Durable Object.
    • Python uses a process-local adapter (at most 10,000 lanes) or an injected SQLite adapter on the session_tiers schema.
    • Bindings hold hashed identities, route identifiers, expiry and a lease. They retain no prompt, response or tool content.
    • Unavailable storage returns 503 session_tier_storage_unavailable, and invalid metadata returns 400.

Details are in docs/session-tiers.md.

Glue left for the Wave 4 integration PR

  • The managed dispatcher must call prepare_session_tier with the authenticated principal and the current policy revision, strip session_tier before parsing and dispatch, and finalize after classified completion.
  • README line.

Follow-up

Migration 0021 creates session_tiers in D1, and the table is in the readiness list, but no D1 reader exists yet. Managed bindings are per process (or in the injected SQLite adapter). A D1-backed adapter through the private state endpoint, so that Containers share bindings, is a separate follow-up PR, not integration glue.

Stacking

This PR is based on feat/usage-reservations (#44), so migrations 0018–0021 stay contiguous. Review only feat(routing): add opt-in role-scoped sticky session tiers and the one-line follow-up test(session-tiers): keep the Worker module inventory sorted; the merge commit brings in the lower layers.

Deployment

Apply the migration before merging to main. /ready checks the table, and a push to main does not apply D1 migrations:

npx wrangler d1 migrations apply multillm-intelligence --remote

Shared registrations

  • Added tests/test_session_tiers_worker.mjs to test:worker.
  • Added SESSION_TIER_MODE and SESSION_TIER_TTL_SECONDS to the Container env keys.
  • Added session_tiers to the D1 readiness tables.
  • Added worker/roleplay/session-tier-policy.mjs to the secret firewall inventory.

Tests

  • New tests: tests/test_session_tiers.py and tests/test_session_tiers_worker.mjs.
  • Full Python suite: 3,637 passed, 2 skipped. ruff, mypy and bandit are clean.
  • npm run test:worker: 1,416 passed. npm run test:integration: 7 passed.
  • cf:migrations:check reports 21 contiguous, additive migrations.
  • test:deploy, free routes, client headers and the wrangler deploy --dry-run bundle all pass.

CI notes

  • GitHub python/node jobs: the dependency-audit steps fail on every branch, including main. fix(deps): clear the CI dependency audit advisories #42 fixes that.
  • Workers Builds: fails on this stack because the Wave 3 base adds the AdmissionCoordinator Durable Object migration, which wrangler versions upload cannot apply. This is expected until wrangler deploy runs on main.

This is part of Wave 4.

Bind an authenticated session and role lane (main, delegation or aux) to its approved model tier behind SESSION_TIER_MODE=sticky (default off), keep the tier through unresolved tool calls, and change it only at a safe user-turn boundary under existing policy. Bindings are hashed and stored in D1 (migration 0021) and roleplay Durable Object storage; expiry, revision or credential failure clears them.
# Conflicts:
#	package.json
#	worker/container-env.mjs
#	worker/d1-schema.mjs
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Oct 9, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5dc4e99f-0ff4-468b-8f33-eb5d196af408

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
multillm-proxy 2c5213c Oct 09 2026, 12:42 AM

@ALikesToCode
ALikesToCode merged commit d696f62 into feat/usage-reservations Oct 9, 2026
4 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant