Repository navigation
fix(agent-platform): gateway-only generation events; never use pi cost estimates - #65457
Merged
Merged
Conversation
…t estimates On the ai-gateway path the gateway already emits a cost-bearing $ai_generation, but the runner emitted a second, cost-suppressed one too. The agent observability board reads the runner's events, so agent spend showed $0 while the gateway's cost-bearing events sat unattributed. Fix the split: - Runner forwards $agent_application_id + $agent_session_id to the gateway via X-PostHog-Properties, so the gateway's $ai_generation is attributable per agent (the gateway keeps $agent_* keys, strips $ai_*). - On the gateway path the runner suppresses its own $ai_generation (gateway emits it) but still emits $ai_span + $ai_trace. Off the gateway path it emits the generation without cost and lets PostHog ingestion price it. - accumulateUsage never uses pi-ai's client-side cost estimates anywhere; the session row's cost comes only from the gateway's settled /v1/usage figure. - Rename useGatewayCost -> gatewayEmitsGenerations to match the new role. Pairs with the code-repo board change that re-scopes the observability queries to $agent_application_id (PostHog/code#2837). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Contributor
|
Reviews (1): Last reviewed commit: "fix(agent-platform): gateway-only genera..." | Re-trigger Greptile |
accumulateUsage no longer folds pi-ai's conversation cost estimates, so the backfill walk rewrites tokens but leaves cost_total at zero. Update the stale assertion to match. Generated-By: PostHog Code Task-Id: 689bbb54-fe12-40cf-8ff1-2c2feda1ad23
dmarticus
approved these changes
Jun 23, 2026
Resolve conflict in agent-runner driver.ts: keep the gateway-only generation emit (guarded by deps.gatewayEmitsGenerations, no pi-ai cost) from this branch, drop master's removed deps.useGatewayCost reference, and adopt master's is_preview field on the event. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The agent observability board showed $0 spend / 7d (and a broken "Cost by model" panel) for agents even though sessions clearly had cost.
On the ai-gateway path there are two
$ai_generationevents per turn in the owning team's project:$ai_origin$agent_application_id$ai_total_cost_usdagent_platform_runnerThe board scoped on
$ai_origin = 'agent_platform_runner'and summed$ai_total_cost_usd, so it only ever matched the runner's cost-less event → $0. The gateway's cost-bearing event was unattributed to the agent. Pure double-reporting.Fix (runner side)
Make the gateway the single source of generation events on the gateway path:
$agent_application_id+$agent_session_idviaX-PostHog-Properties, so the gateway's$ai_generationis sliceable per agent. (The gateway keeps$agent_*keys and strips$ai_*.)$ai_generationon the gateway path (it still emits$ai_span+$ai_trace, which the gateway has no concept of). Off the gateway path the runner emits the generation without cost and lets PostHog ingestion price it from the catalog.accumulateUsageno longer trustsusage.cost.*; the session row'scost_totalcomes only from the gateway's settled/v1/usagefigure.useGatewayCost→gatewayEmitsGenerationsto match the new role.Paired change
The observability board re-scopes every panel from
$ai_originto$agent_application_id(present on generations, tool spans, and traces, on both gateway and direct paths) — PostHog/code#2837. Both are needed for the fix to land; each is independently safe (current runner events already carry$agent_application_id).Notes
gpt-5.5(via OpenRouter) negative on ~half its generations. This PR's passthrough cost sidesteps it for agent traffic; the catalog itself is out of scope here.Verification
agent-shared/agent-runner/agent-janitortypecheck clean; oxfmt + oxlint clean on touched files.summarize-conversationunit tests updated + green (pi cost never accumulated).agent-testsCI gate.🤖 Generated with Claude Code