Skip to content

fix(usage): measure server-side fallback replies by the last run (#877) - #884

Open
iceteaSA wants to merge 2 commits into
cortexkit:masterfrom
iceteaSA:fix/server-fallback-usage
Open

iceteaSA wants to merge 2 commits into
cortexkit:masterfrom
iceteaSA:fix/server-fallback-usage

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Approved issue: #877

Harness coverage (required)

For each entry select: covered / not applicable — surface does not exist there / NOT covered.

  • OpenCode 1: NOT covered. The message and part events that reach the usage handler on OpenCode 1 do not carry the reasoning signature, so the marker is not available without another database read. The issue discussion agreed to keep OpenCode 1 out of this PR.
  • OpenCode 2: covered.
  • Pi: not applicable — Pi does not use server-side fallback, so Pi steps are never summed.
  • OMP: not applicable — same as Pi.
  • Rust module: NOT covered — this change is in the OpenCode 2 TypeScript usage path (v2/hooks/usage-reading.ts, called from v2/hooks/context.ts). The module was not changed or checked.

What changes

Anthropic's server-side fallback reruns a refused request on the fallback model and reports the usage of both runs added together. Magic Context recorded that sum as context pressure.

On OpenCode 2, resolveUsageReading now reads the reasoning signature of the reply:

Reply carries Recorded pressure
Per-run usage marker (anthropic-auth 2.5.8 and later) input_tokens + cache_read_input_tokens + cache_creation_input_tokens of the last fallback_message entry. A missing cache count is 0.
Boundary marker only (older anthropic-auth) No change to the previous reading. Only the response time moves, the same as a completed reply without input usage.
Per-run usage marker that is malformed, or that has no fallback_message entry Same as boundary only, with one log line.
No marker Unchanged.

The billed top-level usage is not changed, so cost stays correct. The signature is read from native.signature (OpenCode 2.2 and later) and from state.signature (OpenCode 2.0.x). The two prefixes are named constants in shared/provider-response-completion.ts.

Commits

  1. fix(usage): ignore summed usage from server-side fallback replies: the boundary-only rule.
  2. fix(usage): read fallback pressure from per-run usage: the per-run usage rule from the issue discussion.

Tests

  • Parser: the usage marker in native.signature and in state.signature, the last fallback_message entry wins, missing cache counts are 0, and five malformed payloads take the boundary-only path.
  • Persistence: a usage-marker reply records the last-run pressure and the map entry, not the sum. A boundary-only reply keeps the previous reading and moves the response time. The next normal reply records normally.
  • Actual OpenCode 2 hook (server-fallback-usage.test.ts): replies with the boundary marker, the usage marker in each location, invalid JSON and a missing fallback_message entry admit at the per-run pressure without execute. The control reply without a marker still enters the force band.
  • The new tests failed before the change: 14 failures in the focused set, 4 in the hook file. When the usage prefix was changed to a wrong value, 16 tests failed, including both hook cases with the usage marker.

Results on this branch (Bun 1.4.0):

  • provider-response-completion, usage-reading, usage-persist, server-fallback-usage: 34 pass, 0 fail.
  • src/v2/hooks and src/v2/fold: 605 pass, 5 skip, 0 fail.
  • bun run typecheck, Biome on the changed files, and git diff --check: clean.

Summary by cubic

Fixes #877: context pressure on OpenCode 2 coming from Anthropic server-side fallback replies now reflects the last run's usage instead of the billed sum, so sessions no longer jump into the force band.

Changes

  • On anthropic-auth 2.5.8 and later, the reply carries a per-run usage marker in native.signature (OpenCode 2.2+) or state.signature (2.0.x); the pressure is the last fallback_message entry's input + cache read + cache write, with missing cache counts defaulting to 0.
  • Replies with only the older boundary marker, or with a malformed usage marker, keep the previous pressure reading and move only the response time, logging one line.
  • Replies without a marker are unchanged, and the billed top-level usage is unchanged. OpenCode 1 is not covered because its message events lack the marker.

Written for commit e739317. Summary will update on new commits.

View guided diff

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

OpenCode 2 now measures fallback replies using the last fallback_message, rather than the billed total across runs.

  • OpenCode 2 measures fallback replies by the last run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Latest stored assistant reply] --> B{Reasoning signature}
  B -->|Valid per-run usage| C[Read last fallback_message]
  C --> D[Save pressure and reply time]
  B -->|Boundary or invalid usage| E[Keep pressure and update reply time]
  B -->|No marker| F[Use normal token counts]
  F --> D
Loading

Reviews (1) · Last reviewed commit: "fix(usage): read fallback pressure from ..." · Reviewed by Greptile

Anthropic's server-side fallback can run one request twice when a content filter refuses the first model. The reply then reports the usage of both runs added together. Magic Context read that sum as the size of the prompt, so a session at 58% could show 110% and enter the force band, hold the turn for the historian, refuse it and fold early.

A reply that carries the anthropic-auth fallback marker now keeps the previous pressure reading and only moves the response time, the same as a completed reply without input usage. OpenCode 1 is unchanged: its message events carry no reply content, so the marker does not reach the usage handler there.
anthropic-auth 2.5.8 and later put a per-run usage marker on every step
whose usage Anthropic summed: the handoff step and later steps that stay
on the fallback model. On OpenCode 2 the context size now comes from the
last fallback_message entry of that marker (input + cache read + cache
write, a missing count is 0). The billed top-level usage is not changed.

A reply with only the boundary marker (an older anthropic-auth) keeps
the previous reading and moves only the response time. A malformed usage
marker is treated the same way and logged. Replies without a marker are
unchanged. The signature is read from native.signature (OpenCode 2.2+)
and from state.signature (2.0.x).

Refs cortexkit#877
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Waiting for design-approved on #877.

This PR needs a linked issue with the design-approved label before it can be reviewed or merged. Link the approved issue with Approved issue: #<issue> (or Refs #<issue>) in the description. A maintainer will apply the design-approved label on the issue, or trivial on the pull request when there is genuinely no design to agree. The issue stays open until the change ships; maintainers take care of it then.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 8 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/plugin/src/v2/hooks/usage-reading.test.ts">

<violation number="1" location="packages/plugin/src/v2/hooks/usage-reading.test.ts:85">
P2: The invalid-marker tests never check the pressure reading they are meant to preserve. Add an assertion that `inputTokens` still comes from the top-level tokens (963_300) for each invalid payload.</violation>
</file>

<file name="packages/plugin/src/shared/provider-response-completion.test.ts">

<violation number="1" location="packages/plugin/src/shared/provider-response-completion.test.ts:7">
P3: The first test is titled "detects usage" but its usage marker payload `{"iterations":[]}` is invalid, so it only passes through the invalid-usage boundary path. Use a payload with a `fallback_message` entry so the test covers a valid usage marker.</violation>
</file>

Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack

View guided diff | Re-trigger cubic

],
limitFor,
});
expect(reading?.serverFallback).toBe(true);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The invalid-marker tests never check the pressure reading they are meant to preserve. Add an assertion that inputTokens still comes from the top-level tokens (963_300) for each invalid payload.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/plugin/src/v2/hooks/usage-reading.test.ts, line 85:

<comment>The invalid-marker tests never check the pressure reading they are meant to preserve. Add an assertion that `inputTokens` still comes from the top-level tokens (963_300) for each invalid payload.</comment>

<file context>
@@ -4,6 +4,88 @@ import { resolveUsageReading, usageReadingMatchesDraft } from "./usage-reading";
+            ],
+            limitFor,
+        });
+        expect(reading?.serverFallback).toBe(true);
+    });
+}
</file context>
Suggested change
expect(reading?.serverFallback).toBe(true);
expect(reading?.inputTokens).toBe(963_300);
expect(reading?.serverFallback).toBe(true);

for (const location of ["native", "state"]) {
test(`detects usage and boundary markers in ${location}.signature`, () => {
for (const signature of [
'cortexkit-server-fallback-usage-v1:{"iterations":[]}',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The first test is titled "detects usage" but its usage marker payload {"iterations":[]} is invalid, so it only passes through the invalid-usage boundary path. Use a payload with a fallback_message entry so the test covers a valid usage marker.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/plugin/src/shared/provider-response-completion.test.ts, line 7:

<comment>The first test is titled "detects usage" but its usage marker payload `{"iterations":[]}` is invalid, so it only passes through the invalid-usage boundary path. Use a payload with a `fallback_message` entry so the test covers a valid usage marker.</comment>

<file context>
@@ -0,0 +1,45 @@
+for (const location of ["native", "state"]) {
+    test(`detects usage and boundary markers in ${location}.signature`, () => {
+        for (const signature of [
+            'cortexkit-server-fallback-usage-v1:{"iterations":[]}',
+            "cortexkit-server-fallback-v1:primary|fallback",
+        ]) {
</file context>
Suggested change
'cortexkit-server-fallback-usage-v1:{"iterations":[]}',
'cortexkit-server-fallback-usage-v1:{"iterations":[{"type":"fallback_message","input_tokens":1}]}',

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Server-side fallback replies report summed usage, causing a false force band

1 participant