Skip to content

[PR] Reserve output headroom on every wire (drop the Anthropic exemption) - #4184

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-21_always-reserve-headroom
Oct 21, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-21_always-reserve-headroom

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What changes

shouldReserveOutputHeadroom() now returns true for every wire protocol — the Anthropic exemption is dropped. Anthropic-wire sessions reserve min(max_tokens, 25% × window) from the context window before nudge/truncate bands are computed, same as OpenAI-family wires already did.

Why

Decision made by the owner on 2026-10-21 after #3900's v0.1.203 verification: an over-full context costs attention everywhere, so paying up to a quarter of the window in reservation on Claude routes is worth removing the "provider counts completion against the window" class of 400s.

Tradeoff (accepted)

Real-Claude routes lose up to min(max_output, 25%) of usable history until #3973 lands per-provider wall facts and can skip the reservation where the input limit is provably independent. That is #3973's optimization space, not this PR's precondition.

Deliberately not done

  • No new config surface — the toggle was behavioral (src/util.ts); call sites (src/server/handle.ts, src/server/side-request.ts) unchanged, they just get true from the gate now.
  • #3973 untouched and still needed for the reverse direction (reclaiming the band on genuine Claude routes).

Tests

  • tests/context-overflow.test.ts: gate test re-pinned ("every protocol reserves").
  • tests/preflight-max-output-tokens.test.ts: the anthropic e2e (fix: Codex preflight summaries fail with unsupported max_output_tokens #663) ran knife-edge against the old no-reservation wall (+2,000 context restores its documented margin; 12_000 − 1_024 ≈ 10_000).
  • tests/anthropic-beta-window.test.ts: expected effective limits now include the reservation (1M−1024, 200K−1024).
  • Local: 81/81 on the 7 core headroom/preflight suites, 58 pass / 0 fail across all 14 anthropic-named suites, tsc --noEmit clean. Full green per remote CI as usual.

Refs #3900 #3973

…ption)

An over-full context degrades attention everywhere, so the safe default wins:
Anthropic-wire sessions now reserve min(max_tokens, cap x window) like every
other protocol. Wrapper providers on that wire (DeepSeek /anthropic) count
completion against the window and 400'd at the wall (#3900); real-Claude
routes can recover the reserved band once #3973 persists per-provider wall
facts.
@ranxianglei ranxianglei changed the title Reserve output headroom on every wire (drop the Anthropic exemption) [PR] Reserve output headroom on every wire (drop the Anthropic exemption) Oct 21, 2026
@ranxianglei
ranxianglei merged commit ef59cf7 into master Oct 21, 2026
24 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.725-master.ef59cf7 (rolling master tag).

npm install -g [email protected]  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-21_always-reserve-headroom (4d50f9c)

Option A — Install this exact build (recommended)

npm install -g [email protected]

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4184.tgz
npm install -g package

Install instructions are refreshed on each push.

ranxianglei added a commit that referenced this pull request Oct 21, 2026
…room

[PR] Revert "#3900: reserve output headroom on every wire (drop the Anthropic exemption)" (#4184) — unblock v0.1.204
ranxianglei added a commit that referenced this pull request Oct 23, 2026
…on-caliber

[PR] fix: reserve output headroom by route, not wire — caliber split (redo of reverted #4184)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant