fix(codex): port the #1078 pricing fixes (reasoning double-count + cache-write bucketing) to feat/core-extraction

CHANGELOG.md

@@ -22,6 +22,7 @@
 - **`codeburn sync` re-keys device, span, and trace identity.** Device, span, and trace ids are now domain-separated HMAC digests under the per-install privacy key rather than bare hashes. This is a one-time identity re-key: a sync backend sees a new device identity after upgrade, and spans pushed before the upgrade won't correlate with spans pushed after it. `sync push` also now hard-aborts — instead of degrading to an ephemeral, unstable key — when the config dir is unwritable or the on-disk privacy-key file is corrupt.
 
 ### Fixed
+- **Codex spend no longer counts reasoning tokens twice, and cache writes are priced only where OpenAI actually charges for them.** OpenAI bills reasoning tokens as *part of* `output_tokens`, not on top of it, but CodeBurn added `reasoning_output_tokens` to output when pricing a Codex call and again in the models and audit displays. Every Codex number was therefore too high: on a real corpus **cost by 3.5%** and **displayed Output tokens by 34.6%** ($4,713.12 -> $4,547.09; 22.6M -> 16.8M output tokens). The raw `reasoningTokens` figure is unchanged and still reported on its own; only the double-count is gone. Both places that price a Codex call — the host pricing pass and the cache-rehydration re-price — now go through one shared `billableOutputTokens` helper, so a cold run and a warm run can never disagree. Separately, Codex's `cache_write_input_tokens` (new in codex PR #33454) was never read and cache-creation tokens were hardcoded to 0; the core decoder now reads and clamps them so they can never exceed the uncached input, and the host carves them out of that bucket **only on models whose pricing source publishes a real cache-write rate** — gpt-5.6 and its terra/sol/luna variants charge 1.25x input for a cache write, everything before it charges nothing extra — because CodeBurn fabricates a 1.25x rate when a source omits one, and charging that would have invented a surcharge on gpt-5.5, gpt-5.4, gpt-5.3-codex and gpt-5. On models without an explicit rate the tokens stay in the plain input bucket and the price is unchanged to the cent. The field is new enough that today's impact is $0 on that corpus. Codex sessions re-parse once (codex results cache v13, codex parse version `-codex-pricing-v1`) and the daily cache takes v28 so finalized days re-derive; days whose Codex transcripts have since aged out are held by the same never-lose guard #1040 relies on and keep their pre-fix totals. No other provider's numbers move. This is the `feat/core-extraction` port of #1078 and its #1084 hardening follow-up, so the final branch→main merge keeps the fix instead of reverting it. Thanks @chr-evensen. (#1083, ports #1075/#1078/#1084)
 - Claude Desktop and Cowork sessions are discovered for Windows Microsoft Store (MSIX) installs. (#611)
 - **Nine providers served silently stale numbers after you pointed their env override at a different profile or root.** Kiro, Grok, Kimi, Mux, Mistral Vibe, Zerostack, Codebuff, Goose and Crush each honor an env var that relocates where discovery looks, but the var was never declared in the provider env fingerprint, so the cache section survived the change and kept reporting sessions parsed from the old root — with no diagnostic anywhere. The fix declares those vars, the adjacent OS-set path variables that resolve a discovery root for Claude, IBM Bob, Open Design and Kilo Code on Windows and Linux, Cursor's parse-budget override, and the Vercel AI Gateway credential — which must invalidate the fingerprint because a read-only refresh serves the cached report and would otherwise keep reporting the previous account's usage after a swap. Your next run re-parses the fourteen file-backed providers whose declarations changed — the nine above plus Claude, Cursor, Open Design, IBM Bob and Kilo Code — once, and only once; the Vercel AI Gateway declaration is a read-only-path correction, not a migration (its report is re-fetched on every writable run anyway); Copilot is deliberately NOT included because a discovery-root/account switch needs provider-specific namespace semantics — blindly carrying its durable section would combine old-root and new-root histories; `codeburn doctor` names deliberate overrides including the XDG_* vars, never the Windows ambient APPDATA / LOCALAPPDATA, and redacts credential values. (#920)
 - Sessions that were silently invisible now appear: Pi and Oh My Pi transcripts with an OMP title slot, Cline sessions under Code - Insiders or VSCodium roots, and OpenCode/kilo-code usage that silently read as zero now reports. (#930)

packages/cli/src/audit-report.ts

@@ -1,4 +1,4 @@
-import { getModelCosts, type ModelCosts } from './models.js'
+import { billableOutputTokens, getModelCosts, type ModelCosts } from './models.js'
 import { getProvider } from './providers/index.js'
 import { formatCost, formatTokens } from './format.js'
 import { renderTable, type TableColumn } from './text-table.js'
@@ -122,7 +122,7 @@ export async function aggregateAudit(projects: ProjectSummary[]): Promise<AuditR
     const meta = await resolveProvider(bucket.provider)
     const displayed = {
       inputTokens: bucket.raw.inputTokens,
-      outputTokens: bucket.raw.outputTokens + bucket.raw.reasoningTokens,
+      outputTokens: billableOutputTokens(bucket.provider, bucket.raw.outputTokens, bucket.raw.reasoningTokens),
       cacheWriteTokens: bucket.raw.cacheCreationInputTokens,
       cacheReadTokens: bucket.cacheReadDisplayed,
     }

packages/cli/src/codex-cache.ts

@@ -42,7 +42,13 @@ import type { ParsedProviderCall } from './providers/types.js'
 // has since reached 11 (#1078), and a shared version number on two different
 // payload shapes would let a cache written by either line be read as current by
 // the other.
-const CODEX_CACHE_VERSION = 12
+//
+// v13: the #1075/#1078 codex pricing fix, ported here (#1083). Every cached
+// call carries `costUSD` verbatim and its token buckets, so v12 entries hold
+// costs with reasoning double-counted and cache writes never carved out of
+// input; the stored `state` also lacks `prevCacheWrite`. Bump once and let
+// unchanged sessions re-decode.
+const CODEX_CACHE_VERSION = 13
 const CACHE_FILE = 'codex-results.json'
 
 type FileFingerprint = { mtimeMs: number; sizeBytes: number }

packages/cli/src/codex-credits.ts

@@ -36,9 +36,9 @@ export type CodexCreditTokens = {
   inputTokens: number
   /// Cache-read (cached input) tokens, billed at the cheaper cached rate.
   cachedReadTokens: number
+  /// Billable output tokens: reasoning is already included (billableOutputTokens
+  /// in models.ts), so callers must not add it on top here.
   outputTokens: number
-  /// Reasoning tokens are billed as output, matching CodeBurn's cost model.
-  reasoningTokens?: number
 }
 
 /// Credits consumed for one Codex usage record. Returns null when the model has
@@ -48,10 +48,9 @@ export function codexCredits(model: string, tokens: CodexCreditTokens): number |
   if (!rate) return null
   const safe = (n: number) => (Number.isFinite(n) && n > 0 ? n : 0)
   const PER_MILLION = 1_000_000
-  const output = safe(tokens.outputTokens) + safe(tokens.reasoningTokens ?? 0)
   return (
     (safe(tokens.inputTokens) / PER_MILLION) * rate.input +
     (safe(tokens.cachedReadTokens) / PER_MILLION) * rate.cachedInput +
-    (output / PER_MILLION) * rate.output
+    (safe(tokens.outputTokens) / PER_MILLION) * rate.output
   )
 }

packages/cli/src/daily-cache.ts

@@ -5,6 +5,16 @@ import { homedir } from 'os'
 import { join } from 'path'
 import type { DateRange, ProjectSummary } from './types.js'
 
+// Bumped to 28: the #1075/#1078 codex pricing fix, ported to this branch
+// (#1083). Reasoning tokens were billed on top of output (they are a subset of
+// it) and cache_write_input_tokens was ignored, so days finalized at v27 carry
+// codex costs overstated by ~3.5% and codex output tokens overstated by ~34.6%.
+// `usage-aggregator` serves every day before today from this cache and
+// retention is ten years, so nothing downstream would ever notice; raising
+// MIN_SUPPORTED_VERSION forces the one-time re-derivation. (main's own ladder
+// took 23 for the same fix; this branch reconciles at its final merge by
+// keeping the max.)
+//
 // Bumped to 27: the Cline CLI (npm `cline`, 3.x) is a NEW provider, so every
 // historical session under ~/.cline/data/sessions contributes usage that no
 // older rollup ever contained. Those files were never scanned before they were
@@ -109,8 +119,8 @@ import type { DateRange, ProjectSummary } from './types.js'
 // that older binaries skipped. v8 added local-model savings to the daily
 // rollup; the `savingsConfigHash` field is invalidated separately when the
 // user changes their `localModelSavings` mapping.
-export const DAILY_CACHE_VERSION = 27
-const MIN_SUPPORTED_VERSION = 27
+export const DAILY_CACHE_VERSION = 28
+const MIN_SUPPORTED_VERSION = 28
 // Version-suffixed so different binaries each own a distinct file and never
 // clobber an incompatible schema. Bumping the version mints a fresh filename;
 // adoptOlderDailyCaches then unions days out of every previous file (including

packages/cli/src/models-report.ts

@@ -3,6 +3,7 @@ import stripAnsi from 'strip-ansi'
 
 import { codexCredits } from './codex-credits.js'
 import { formatCost, formatTokens } from './format.js'
+import { billableOutputTokens } from './models.js'
 import { getProvider } from './providers/index.js'
 import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js'
 
@@ -119,7 +120,7 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat
             buckets.set(key, bucket)
           }
           bucket.inputTokens += call.usage.inputTokens
-          bucket.outputTokens += call.usage.outputTokens + call.usage.reasoningTokens
+          bucket.outputTokens += billableOutputTokens(provider, call.usage.outputTokens, call.usage.reasoningTokens)
           bucket.cacheWriteTokens += call.usage.cacheCreationInputTokens
           // cacheReadInputTokens (Anthropic vocab) and cachedInputTokens (OpenAI vocab)
           // are two names for the same thing. Providers populate one or set both to the
@@ -179,9 +180,10 @@ export async function aggregateModels(projects: ProjectSummary[], opts: Aggregat
       savingsUSD: bucket.savingsUSD,
       savingsBaselineModel: bucket.savingsBaselineModel,
       calls: bucket.calls,
-      // outputTokens already includes reasoning (folded in above), and for Codex
-      // inputTokens is non-cached with cacheReadTokens holding cached input, which
-      // is exactly what the credit rates expect.
+      // outputTokens is the billable output (for Codex that already contains
+      // reasoning, so nothing is added on top), and inputTokens is non-cached
+      // with cacheReadTokens holding cached input - exactly what the credit
+      // rates expect.
       credits: bucket.provider === 'codex'
         ? codexCredits(bucket.model, {
             inputTokens: bucket.inputTokens,

packages/cli/src/models.ts

@@ -12,6 +12,28 @@ export type ModelCosts = {
   cacheReadCostPerToken: number
   webSearchCostPerRequest: number
   fastMultiplier: number
+  /// True only when the pricing source carried a real cache-write rate. When
+  /// absent/false, `cacheWriteCostPerToken` is the fabricated `1.25 x input`
+  /// default, which is right for Anthropic-style pricing but would invent a
+  /// surcharge on providers that charge nothing extra to write cache. Callers
+  /// that decide WHICH bucket to put tokens in (rather than what to multiply
+  /// them by) must consult this before routing tokens to the cache-write
+  /// bucket. Optional so an incomplete literal defaults to the safe answer.
+  cacheWriteCostIsExplicit?: boolean
+}
+
+/// Providers whose reported `reasoningTokens` are a SUBSET of `outputTokens`
+/// rather than a separate bucket to add on top. OpenAI bills reasoning as part
+/// of output (every codex `token_count` event satisfies input + output ==
+/// total), and Anthropic folds thinking into output the same way, so summing
+/// the two double-counts both the cost and the displayed output tokens.
+const REASONING_INCLUDED_IN_OUTPUT = new Set(['claude', 'codex'])
+
+/// Output tokens to bill and display for one call. Single source of truth so
+/// the pricing sites and the display sums can never disagree about whether a
+/// provider's reasoning tokens are already inside its output count (#1075).
+export function billableOutputTokens(provider: string, outputTokens: number, reasoningTokens: number): number {
+  return REASONING_INCLUDED_IN_OUTPUT.has(provider) ? outputTokens : outputTokens + reasoningTokens
 }
 
 type PriceOverrideRates = {
@@ -36,6 +58,11 @@ type SnapshotEntry = [number, number, number | null, number | null, (number | nu
 
 const LITELLM_URL = 'https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json'
 const CACHE_TTL_MS = 24 * 60 * 60 * 1000
+// Bump whenever a ModelCosts field changes pricing behavior (cacheWriteCostIsExplicit,
+// added in #1075/#1078). A cache written under an older/missing version is treated as a
+// miss instead of read verbatim, so a stale on-disk file can't reintroduce a killed bug
+// for up to CACHE_TTL_MS after an upgrade.
+const CACHE_SCHEMA_VERSION = 2
 const WEB_SEARCH_COST = 0.01
 const ONE_HOUR_CACHE_WRITE_MULTIPLIER_FROM_FIVE_MINUTE_RATE = 1.6
 
@@ -70,6 +97,7 @@ function buildCosts(
     cacheReadCostPerToken: cacheRead ?? input * 0.1,
     webSearchCostPerRequest: WEB_SEARCH_COST,
     fastMultiplier: fast ?? 1,
+    cacheWriteCostIsExplicit: cacheWrite !== null && cacheWrite !== undefined,
   }
 }
 
@@ -200,6 +228,7 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
 
   await mkdir(getCacheDir(), { recursive: true })
   await writeFile(getCachePath(), JSON.stringify({
+    version: CACHE_SCHEMA_VERSION,
     timestamp: Date.now(),
     data: Object.fromEntries(pricing),
   }))
@@ -210,7 +239,8 @@ async function fetchAndCachePricing(): Promise<Map<string, ModelCosts>> {
 async function loadCachedPricing(): Promise<Map<string, ModelCosts> | null> {
   try {
     const raw = await readFile(getCachePath(), 'utf-8')
-    const cached = JSON.parse(raw) as { timestamp: number; data: Record<string, ModelCosts> }
+    const cached = JSON.parse(raw) as { version?: number; timestamp: number; data: Record<string, ModelCosts> }
+    if (cached.version !== CACHE_SCHEMA_VERSION) return null
     if (Date.now() - cached.timestamp > CACHE_TTL_MS) return null
     return new Map(Object.entries(cached.data))
   } catch {

packages/cli/src/parser.ts

@@ -1,7 +1,7 @@
 import { lstat, readFile, readdir, stat } from 'fs/promises'
 import { basename, dirname, join, resolve, sep } from 'path'
 import { readSessionLines } from './fs-utils.js'
-import { calculateCost, calculateLocalModelSavings, getShortModelName, isProxiedPath, getProxyPathsConfigHash } from './models.js'
+import { billableOutputTokens, calculateCost, calculateLocalModelSavings, getShortModelName, isProxiedPath, getProxyPathsConfigHash } from './models.js'
 import { resolveSubagentAttribution, sessionIdentity } from './sessions-report.js'
 import { discoverAllSessions, getProvider } from './providers/index.js'
 import { priceProviderCall } from './pricing-pass.js'
@@ -1272,9 +1272,10 @@ function mergeDurableCachedTurns(
 
 function cachedCallToApiCall(call: CachedCall): ParsedApiCall {
   const u = call.usage
-  const outputForCost = call.provider === 'claude'
-    ? u.outputTokens
-    : u.outputTokens + u.reasoningTokens
+  // Cache-rehydration twin of the fresh-parse pricing in pricing-pass.ts: both
+  // go through billableOutputTokens so a cached read and a cold parse can never
+  // disagree about whether reasoning is already inside output (#1075).
+  const outputForCost = billableOutputTokens(call.provider, u.outputTokens, u.reasoningTokens)
   const costUSD = calculateCost(
     call.model, u.inputTokens, outputForCost,
     u.cacheCreationInputTokens, u.cacheReadInputTokens,

packages/cli/src/pricing-pass.ts

@@ -1,4 +1,4 @@
-import { calculateCost } from './models.js'
+import { billableOutputTokens, calculateCost } from './models.js'
 import type { ParsedProviderCall } from './providers/types.js'
 
 // Host-side pricing pass (Phase 0 of the @codeburn/core extraction).
@@ -20,10 +20,14 @@ import type { ParsedProviderCall } from './providers/types.js'
 export function priceProviderCall(call: ParsedProviderCall): ParsedProviderCall {
   if (call.costBasis !== 'estimated') return call
 
-  // Mirror cachedCallToApiCall's non-claude branch: reasoning tokens are billed
-  // at the output rate. Provider calls never carry 1-hour cache tokens (the
-  // cache write path hardcodes them to 0), so the default 0 is correct here.
-  const outputForCost = call.outputTokens + call.reasoningTokens
+  // Mirror cachedCallToApiCall: reasoning tokens are billed at the output rate,
+  // except for the providers that already report reasoning INSIDE output
+  // (billableOutputTokens in models.ts), where adding it double-counts (#1075).
+  // Both sites go through that helper so a fresh parse and a cache-rehydrated
+  // read can never price the same call differently. Provider calls never carry
+  // 1-hour cache tokens (the cache write path hardcodes them to 0), so the
+  // default 0 is correct here.
+  const outputForCost = billableOutputTokens(call.provider, call.outputTokens, call.reasoningTokens)
   // Seam extension: price `pricingModel` when the decoder supplied one (its
   // display `model` differs from the model the price table is keyed by, e.g.
   // antigravity's suffix-stripped / aliased id). Falls back to `model` for

packages/cli/src/providers/codex.ts

@@ -8,6 +8,7 @@ import { decodeCodex, codexToolNameMap, countUnifiedDiffLoc } from '@codeburn/co
 import type { CodexDecodedCall, CodexDecodeState, CodexEntry } from '@codeburn/core/providers/codex'
 
 import { readSessionLines } from '../fs-utils.js'
+import { getModelCosts } from '../models.js'
 import { priceProviderCall } from '../pricing-pass.js'
 import {
   readCodexCacheEntry,
@@ -185,12 +186,22 @@ async function discoverSessionsInDir(codexDir: string): Promise<SessionSource[]>
 // marks the call so the pricing pass fills `costUSD` from the token buckets,
 // byte-identical to the two in-decoder pricing calls this retires (issue #809).
 function toPricedProviderCall(rich: CodexDecodedCall): ParsedProviderCall {
+  // Only move the decoder's cache-write candidate into the cache-write bucket
+  // when the pricing source publishes a real cache-write rate for this model
+  // (gpt-5.6+ charges 1.25x input; everything before it charges nothing extra).
+  // Otherwise buildCosts' fabricated 1.25x default would invent a surcharge
+  // OpenAI never billed, so the tokens stay where they already were - in plain
+  // input, priced exactly as before (#1075). The rate table is host-side, which
+  // is why this split lives here and not in the decoder.
+  const cacheWriteTokens = rich.cacheWriteCandidateTokens > 0 && getModelCosts(rich.model)?.cacheWriteCostIsExplicit
+    ? rich.cacheWriteCandidateTokens
+    : 0
   const call: ParsedProviderCall = {
     provider: 'codex',
     model: rich.model,
-    inputTokens: rich.inputTokens,
+    inputTokens: rich.inputTokens - cacheWriteTokens,
     outputTokens: rich.outputTokens,
-    cacheCreationInputTokens: rich.cacheCreationInputTokens,
+    cacheCreationInputTokens: rich.cacheCreationInputTokens + cacheWriteTokens,
     cacheReadInputTokens: rich.cacheReadInputTokens,
     cachedInputTokens: rich.cachedInputTokens,
     reasoningTokens: rich.reasoningTokens,