Repository navigation
fix(ai): take cumulative Anthropic stream usage from message_delta - #1074
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
Anthropic's message_delta usage is cumulative. With server tools (web search, web fetch, code execution) its input and cache counts exceed message_start's, but the stream accumulator only recorded message_start's input and cache counts, so streamed generations under-reported input tokens and cache writes. Keep the larger of the two for each count. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 Motivation and Context
Anthropic's
message_deltausage is cumulative. With server tools (web search, web fetch, code execution), itsinput_tokensand cache counts grow past themessage_startvalues during the response. The Anthropic stream accumulator only recorded input and cache counts frommessage_start(it readsoutput_tokensfrommessage_delta), so streamed generations under-reported$ai_input_tokensand$ai_cache_creation_input_tokens.Example from a recorded web-search stream:
message_starthas input 2247 and cache creation 0, whilemessage_deltahas input 2254 and cache creation 17269. Before this change the event reported 2247 and 0._AnthropicStreamAccumulator.consume, shared by the sync and async wrappers, now applies themessage_deltainput and cache counts after the per-event merge. It keeps the larger of the two values, since the totals never shrink, and leaves a field the event omits at themessage_startvalue.💚 How did you test it?
test_streaming_takes_cumulative_message_delta_usageand its_asynctwin. They use realanthropic.typesstream events with the values above and check the captured$ai_input_tokens,$ai_output_tokensand cache properties. Both fail on main.posthog/test/ai: 951 passed, 15 skipped; on main, only the two new tests fail.ruff format --checkpasses.ruff checkreports the same findings as on main for the touched files.mypy … | mypy-baseline filterreports nothing new.📝 Checklist
If releasing new changes
sampo addto generate a changeset file (added.sampo/changesets/cumulative-anthropic-stream-usage.mdin the same format)🤖 Agent context
Autonomy: Fully autonomous
The bug was found and the fix, tests and description were written by Claude Code; I reviewed the diff and the explanation before submitting.
Written with Claude Code (Claude Opus 5.5).
The fix keeps the larger of
message_startandmessage_deltafor each count, rather than switching the merge to "cumulative" mode. That mode would replace the mergedraw_usageand losemessage_start's fields, andmaxalso tolerates a delta that omits or under-reports a field.🤖 Generated with Claude Code