Skip to content

feat(admission): add opt-in shared concurrency admission leases - #29

Merged
ALikesToCode merged 1 commit into
integrate/gateway-waves-1-2from
feat/admission-leases
Oct 9, 2026
Merged

ALikesToCode merged 1 commit into
integrate/gateway-waves-1-2from
feat/admission-leases

Conversation

@ALikesToCode

Copy link
Copy Markdown
Owner

Summary

Adds shared concurrency admission behind ADMISSION_ENABLED, which defaults to false. While the flag is off, no lease call or Durable Object access happens and dispatch, responses, headers and storage are unchanged. An empty ADMISSION_LIMITS_JSON means unlimited even when the flag is on; there is no default gate on any model.

  • A new AdmissionCoordinator Durable Object, using SQLite storage, keeps expiring leases across 64 shards. Native Worker and Flask callers therefore compete for the same capacity.
  • ADMISSION_LIMITS_JSON sets per-principal and per-model-group concurrency limits, for example {"principal": 4, "model_groups": {"heavy": 1}}.
  • Lease lifecycle:
    • Leases last 30 seconds, renew every 10 seconds and are bound to the request ID and the generation deadline.
    • Release happens exactly once on rejection, completion, disconnect or timeout.
  • Errors:
    • A denial is a real 429 with Retry-After, measured until the earliest blocking lease expires.
    • An unavailable authority while enabled is a 503. No success body is ever synthesized.
  • The private endpoint POST http://intelligence.internal/v1/admission is reachable only through Container egress, never from the public router.
  • Standalone Flask needs an explicit private ADMISSION_AUTHORITY_URL; a process-local semaphore is never presented as global.

Details are in docs/admission-leases.md.

Deployment

wrangler.jsonc adds the ADMISSION_COORDINATOR binding and an additive v3_admission Durable Object migration. wrangler deploy applies it on the next push to main. Existing bindings and migration tags are unchanged, and the class makes no storage writes and schedules no alarms while disabled. No D1 migration is needed.

Integration

This branch exports the class from cloudflare-worker.mjs and teaches the test loader the new import. The Wave 3 integration PR adds the remaining wiring:

  • mounting the Flask admission hook after authentication;
  • acquiring and releasing leases around native Worker generations.

Shared registrations

  • Added ADMISSION_ENABLED, ADMISSION_LIMITS_JSON and ADMISSION_AUTHORITY_URL to the Container env keys.
  • Added tests/test_admission_leases_worker.mjs to test:worker.
  • Added worker/admission-do.mjs to the secret firewall inventory.

Tests

  • tests/test_admission_leases.py and tests/test_admission_leases_worker.mjs: new.
  • Full Python suite: 3,207 passed, 2 skipped. ruff, mypy and bandit are clean.
  • npm run test:worker: 1,239 passed. cf:migrations:check, test:deploy, free routes, client headers and the wrangler deploy --dry-run bundle all pass.
  • tests/test_frontend_build.mjs fails locally because @tailwindcss/cli is not installed. It fails the same way on main.

This is part of Wave 3. It is based on the Wave 2 integration branch (#25), and the Wave 3 integration PR adds the README line.

Add a Durable Object admission authority with per-principal and per-model-group concurrency limits, renewable leases and Flask/Worker clients behind ADMISSION_ENABLED, default off. The class is inert while disabled.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-08T21:53:53.542039Z 4497f7a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 04ebb79b-609b-4d69-a456-ebb667a71055

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
multillm-proxy 4497f7a Oct 08 2026, 09:51 PM

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4497f7a056

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread worker/admission-do.mjs
Comment on lines +124 to +125
const [count] = this.rows(`SELECT COUNT(*) AS n, MIN(expires_at) AS next_expiry FROM admission_leases ${where}`, ...args);
if (count.n >= maximum) return failure("admission_denied", 429, Math.max(1, Math.ceil((count.next_expiry - now) / 1000)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Compute Retry-After for the lease that frees capacity

When a configured limit is lowered below the number of existing leases, MIN(expires_at) only identifies the first lease to expire, which may still leave the scope at or above its new limit. For example, reducing a principal from four active leases to a limit of one returns the first expiry even though three leases remain afterward, so clients following Retry-After are denied again. Select the expiry that reduces the count below maximum rather than always using the minimum.

Useful? React with 👍 / 👎.

Comment thread worker/admission-do.mjs
Comment on lines +124 to +125
const [count] = this.rows(`SELECT COUNT(*) AS n, MIN(expires_at) AS next_expiry FROM admission_leases ${where}`, ...args);
if (count.n >= maximum) return failure("admission_denied", 429, Math.max(1, Math.ceil((count.next_expiry - now) / 1000)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Compute Retry-After for the lease that actually frees capacity

When a configured limit is lowered below the number of existing leases, MIN(expires_at) only identifies the first lease to expire, which may still leave the scope at or above its new limit. For example, reducing a principal from four active leases to a limit of one returns the first expiry even though three leases remain afterward, so clients following Retry-After are denied again. Select the expiry that reduces the count below maximum rather than always using the minimum.

Useful? React with 👍 / 👎.

except AdmissionError as error:
self.failure = error
# Transport cancellation can wait for readers that call check(); never hold their lock.
self.release()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cancel upstream before waiting on release

When renewal fails because the authority connection is stalled, this synchronous release immediately makes another call to the same authority before invoking on_lost; that second call can consume the full connect/read timeout while the upstream generation continues running without receiving its cancellation signal. Invoke the cancellation collaborator before, or concurrently with, the best-effort release so an authority outage does not add another timeout of provider work.

Useful? React with 👍 / 👎.

@ALikesToCode
ALikesToCode merged commit 10c1c9b into integrate/gateway-waves-1-2 Oct 9, 2026
4 of 7 checks passed
@ALikesToCode
ALikesToCode deleted the feat/admission-leases branch October 26, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant