Repository navigation
[PR] Reserve output headroom on every wire (drop the Anthropic exemption) - #4184
Merged
Merged
Conversation
…ption) An over-full context degrades attention everywhere, so the safe default wins: Anthropic-wire sessions now reserve min(max_tokens, cap x window) like every other protocol. Wrapper providers on that wire (DeepSeek /anthropic) count completion against the window and 400'd at the wall (#3900); real-Claude routes can recover the reserved band once #3973 persists per-provider wall facts.
Contributor
📦 Merged Build PublishedThis merge to master was published as npm install -g [email protected] # this exact build
npm install -g billion-context@master # always the newest merged build
|
Contributor
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g [email protected]This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr4184.tgz
npm install -g packageInstall instructions are refreshed on each push. |
This was referenced Oct 21, 2026
This was referenced Oct 21, 2026
Merged
ranxianglei
added a commit
that referenced
this pull request
Oct 23, 2026
…on-caliber [PR] fix: reserve output headroom by route, not wire — caliber split (redo of reverted #4184)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
shouldReserveOutputHeadroom()now returnstruefor every wire protocol — the Anthropic exemption is dropped. Anthropic-wire sessions reservemin(max_tokens, 25% × window)from the context window before nudge/truncate bands are computed, same as OpenAI-family wires already did.Why
Decision made by the owner on 2026-10-21 after #3900's v0.1.203 verification: an over-full context costs attention everywhere, so paying up to a quarter of the window in reservation on Claude routes is worth removing the "provider counts completion against the window" class of 400s.
1,070,269 + 128,000 > 1,048,576. That provider counts output against the wall; real Claude does not. The old code decided by protocol string, so the wrapper inherited Claude's exemption and could blow through its wall on the first shot of every new baseline.max_tokensitself exceeds the 25% cap — and that case always existed on every other wire too (self-heal handles one overflow turn).Tradeoff (accepted)
Real-Claude routes lose up to
min(max_output, 25%)of usable history until #3973 lands per-provider wall facts and can skip the reservation where the input limit is provably independent. That is #3973's optimization space, not this PR's precondition.Deliberately not done
src/util.ts); call sites (src/server/handle.ts,src/server/side-request.ts) unchanged, they just gettruefrom the gate now.#3973untouched and still needed for the reverse direction (reclaiming the band on genuine Claude routes).Tests
tests/context-overflow.test.ts: gate test re-pinned ("every protocol reserves").tests/preflight-max-output-tokens.test.ts: the anthropic e2e (fix: Codex preflight summaries fail with unsupported max_output_tokens #663) ran knife-edge against the old no-reservation wall (+2,000 context restores its documented margin;12_000 − 1_024 ≈ 10_000).tests/anthropic-beta-window.test.ts: expected effective limits now include the reservation (1M−1024,200K−1024).tsc --noEmitclean. Full green per remote CI as usual.Refs #3900 #3973