fix(codex): exclude harness startup and fix partial-dedup timing in Tok/s

CHANGELOG.md

@@ -40,6 +40,7 @@
 ### Fixed
 - **Mixed-version installs no longer thrash the Codex / Cursor / Antigravity result caches.** Daily and session caches already own a version-suffixed file so an old desktop binary and a newer CLI cannot clobber each other. The three per-provider result caches still used one unsuffixed filename with an internal version field, so a v10 and a v11 binary rewrote the same `codex-results.json` (and the Cursor / Antigravity siblings) on every run and each re-parsed its whole corpus. They now write `*-results.v<n>.json` the same way the daily cache does. The unsuffixed file is left for older binaries; a matching-version copy is adopted once and never overwritten. (#1082)
 - **Codex spend no longer counts reasoning tokens twice, and cache writes are priced only where OpenAI actually charges for them.** OpenAI bills reasoning tokens as *part of* `output_tokens`, not on top of it — on a 1,396-rollout corpus all 134,316 events carrying a total satisfy `input + output == total` — but CodeBurn added `reasoning_output_tokens` to output when pricing a Codex call and again in the models, audit and per-model displays. Every Codex number was therefore too high: on that corpus **cost by $166.03 (3.5%)** and **displayed Output tokens by 34.6%** ($4,713.12 -> $4,547.09; 22.6M -> 16.8M output tokens). The raw `reasoningTokens` figure is unchanged and still reported on its own; only the double-count is gone. Both places that price a Codex call — the parser and the cache-rehydration re-price — now go through one shared `billableOutputTokens` helper, so a cold run and a warm run can never disagree. Separately, Codex's `cache_write_input_tokens` (new in codex PR #33454) was never read and cache-creation tokens were hardcoded to 0; they are now carved out of the uncached-input bucket and clamped so they can never exceed it. That carve-out happens **only on models whose pricing source publishes a real cache-write rate** — gpt-5.6 and its terra/sol/luna variants charge 1.25x input for a cache write, everything before it charges nothing extra — because CodeBurn fabricates a 1.25x rate when a source omits one, and charging that would have invented a surcharge on gpt-5.5, gpt-5.4, gpt-5.3-codex and gpt-5. On models without an explicit rate the tokens stay in the plain input bucket and the price is unchanged to the cent. The field is new enough that today's impact is $0 on that corpus. Codex sessions re-parse once and the daily cache re-derives once off the warm session cache (a global re-derivation of every day and every provider, since it has no per-provider invalidation); no other provider's numbers move. Days whose Codex transcripts have since aged out are held by the same never-lose guard #1040 relies on: a re-derivation that finds fewer calls than the settled baseline keeps the older, pre-fix (double-counted) total rather than truncating it, so those days do not pick up the repricing until their sources are re-derived with equal or greater evidence. Long-context pricing tiers from the same report are tracked separately in #1076 and the missing `gpt-5.6-codex` snapshot rows in #1077. Thanks @chr-evensen. (#1075)
+- **Codex Tok/s no longer counts reasoning tokens twice or credits harness startup as model time.** Two distortions in the same metric, found and fixed together because they share the same cache-invalidation and test surface. (1) #1075 fixed the reasoning-token double-count for cost, but `activeGeneratedTokens`/`taskGeneratedTokens` in the Codex parser and `generatedTokens` in the `codex-tps` live-throughput reader still summed `outputTokens + reasoningTokens`; both now go through the same `billableOutputTokens('codex', …)` helper #1075 introduced, so the numerator can never drift from the billed one. (2) Codex fires `task_started` before it assembles the request, so the gap up to the first request-context event (`turn_context`, `world_state`, `event_msg/user_message`, or a `response_item/message`) was pure CLI/harness startup counted as active model time — the active window now starts at that first event instead, which matters most for one-shot `codex exec` sessions that pay the gap on every task. The duplicated tool-interval clip/merge/cap logic in `providers/codex.ts` and `codex-throughput.ts` is now one function (`mergeToolIntervals`, exported from `codex-throughput.ts`), which also closes a live trap where `task_complete`'s duration only parsed a plain number and silently dropped the `{secs,nanos}`/string forms `mcp_tool_call_end` already tolerated. (A third suspected distortion — fork-replay dedup dropping a token_count event's tokens from the numerator without shrinking the window to match — was investigated and retracted: the earlier `prevCumulativeTotal` guard already discards a repeated running total before dedup is ever reached, so a real Codex writer never produces a partial drop; the dedup site now carries a comment recording this so the trip isn't repeated.) Display only, no cost or token-count impact — verified byte-identical on the same real corpus. Combined effect on a real Codex corpus (original bug -> all fixes): GPT-5.5 37.8 -> 28.2 tok/s (-25.5%), Codex Auto Review 23.3 -> 20.0 (-14.2%), GPT-5.6 Sol 43.2 -> 33.9 (-21.4%), GPT-5.6 Luna 53.5 -> 49.0 (-8.5%), GPT-5.4 68.0 -> 43.6 (-35.9%), GPT-5.4 Mini 54.9 -> 55.6 (**+1.1%**, the harness-startup correction outweighing the reasoning-count correction for this model on this corpus). `activeGeneratedTokens`/`activeDurationMs`/`toolWaitMs` are stored verbatim in both the Codex result cache and the session cache rather than re-derived on read, so none of this self-heals: Codex sessions re-parse once (one cache-version bump covers both fixes, since they touch the same fields). The dashboard's per-model column stays labelled `Tok/s` — a wider label had zero room at the standard three-column layout, verified by breaking a real width-budget test — but the legend beneath it now reads "Effective Tok/s: generated tokens ÷ time the agent spent waiting on the model, tool execution excluded. Includes prefill, request assembly and reasoning. Not comparable to vendor decode-speed figures." (#1079, #1088)
 - **Codex calls attributed from session metadata no longer carry a stale model.** The Buffer fast path scanned `session_meta` for the first `"model"` string anywhere in the payload, so a nested `base_instructions.provenance.model` was read as if it were `payload.model` — and since the model is last-writer-wins state, that wrong value was credited to every call before the rollout's first `turn_context` and to every call after any mid-file `session_meta` (29 of 1380 rollouts on one real corpus carry a late `session_meta`, and 57 record usage before any `turn_context`). Direct payload fields are now read depth-aware, which is what the non-fast `JSON.parse` path always did. Codex sessions re-parse once (~9s on a 4 GB rollout corpus) and the daily cache re-derives once off the warm session cache, a global re-derivation of every day and every provider since it has no per-provider invalidation; it moves per-model attribution, and clears any rollup an earlier parse change had left stale. Days whose transcripts have partly aged out are held by the never-lose guard: on a real 110-day cache no day lost value and none disappeared — 100 days came back identical and 9 grok days rose by $19.80 in total. Thanks @timdp. (#1040)
 - **Codex `session_meta` cwd / session id / originator follow the same depth-1 window as `model`.** #1040 fixed nested `provenance.model`; the compact Buffer path still took the first `cwd`, `session_id`, `originator`, `name`, `forked_from_id` or `model_provider` anywhere in the payload, so a `dynamic_tools[].name` (or any same-named nested key) could steal the top-level field. Those strings now use the existing payload-depth-1 scan. Function-call `name` on other event types is unchanged. Codex sessions re-parse once. (#1045)
 - **Plan rows for sticker-price presets read as a budget instead of live provider quota.** There is no Grok quota endpoint, so a SuperGrok row was parsed API-equivalent spend divided by the plan's sticker price on a monthly reset — but the TUI labelled that math "plan" and "reset", which next to a client showing xAI's real weekly window read as CodeBurn being wrong. The bars and the arithmetic are unchanged; the words are not. Both the dashboard and the desktop app now say the number is an API-equivalent monthly budget and not a live provider window, in the same wording on both surfaces, and for every preset rather than as a SuperGrok special case. The window is anniversary-based (`plan.resetDay`, settable with `codeburn plan set --reset-day`), so it is called a budget reset rather than a calendar one. The row was also shortened to fit 80 columns: at that width the percentage and the projected month were being truncated away, including on custom plans, whose label carries the provider.

src/codex-cache.ts

@@ -27,7 +27,14 @@ import type { ParsedProviderCall } from './providers/types.js'
 // output, and cache_write_input_tokens is carved out of the input bucket. This
 // file stores each call's costUSD and token buckets verbatim, so entries
 // written by v10 carry the old (overstated) cost and must be re-derived.
-export const CODEX_CACHE_VERSION = 11
+// v13: codex throughput fix (#1079) - activeGeneratedTokens was summing
+// output + reasoning, the same double-count Fix A removed from cost. This
+// file stores activeGeneratedTokens/activeDurationMs/toolWaitMs verbatim (not
+// re-derived on read), so v11 entries carry the overstated numerator and must
+// re-parse. Not 12: v12 is claimed by feat/core-extraction's own port of this
+// throughput feature (PR #1086), so reusing it would let two incompatible
+// schemas share a filename.
+export const CODEX_CACHE_VERSION = 13
 export const CODEX_LEGACY_CACHE_FILE = 'codex-results.json'
 export function codexCacheFileName(version = CODEX_CACHE_VERSION): string {
   return `codex-results.v${version}.json`

src/codex-throughput.ts

@@ -1,6 +1,8 @@
 import { open, stat } from 'node:fs/promises'
 import { StringDecoder } from 'node:string_decoder'
 
+import { billableOutputTokens } from './models.js'
+
 export type CodexThroughputPoint = {
   timestamp: string
   model?: string
@@ -105,7 +107,12 @@ function durationMs(payload: RolloutLine['payload']): number | undefined {
   return undefined
 }
 
-function mergeToolIntervals(intervals: Array<[number, number]>, durationMs: number, taskStartedAt?: number, taskCompletedAt?: number): number {
+// Shared with src/providers/codex.ts (#1088 BUG-8): both clip a task's tool
+// intervals to its [taskStartedAt, taskStartedAt + durationMs] window, merge
+// overlaps, and cap the sum at durationMs. Was copy-pasted inline in
+// providers/codex.ts and had already drifted (duration parsing there accepted
+// only a plain `duration_ms` number); one copy now, called from both.
+export function mergeToolIntervals(intervals: Array<[number, number]>, durationMs: number, taskStartedAt?: number, taskCompletedAt?: number): number {
   const windowStart = taskStartedAt ?? (taskCompletedAt !== undefined ? taskCompletedAt - durationMs : undefined)
   const windowEnd = windowStart !== undefined ? windowStart + durationMs : undefined
   const clipped = intervals.map(([start, end]) => [
@@ -400,7 +407,9 @@ export class CodexThroughputReader {
       state.previousOutput = total?.output_tokens ?? state.previousOutput
       state.previousReasoning = total?.reasoning_output_tokens ?? state.previousReasoning
     }
-    const generatedTokens = outputTokens + reasoningTokens
+    // Reasoning is already inside output_tokens (#1075/#1078); same numerator
+    // as the cost path so live Tok/s can't drift from billed tokens (#1079).
+    const generatedTokens = billableOutputTokens('codex', outputTokens, reasoningTokens)
     if (generatedTokens <= 0) return
     const timestampMs = Date.parse(entry.timestamp)
     if (!Number.isFinite(timestampMs)) return

src/dashboard.tsx

@@ -746,6 +746,11 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
   const anyEstimated = Object.values(modelTotals).some(d => d.estimatedCostUSD > 0)
   const sorted = Object.entries(modelTotals).sort(([, a], [, b]) => b.costUSD - a.costUSD)
   const costLabels = sorted.map(([, data]) => markEstimated(formatCost(data.costUSD), data.estimatedCostUSD > 0))
+  // #1088: this column has zero width slack left at the standard 3-column
+  // breakpoint (verified: widening the header even one character clips
+  // 'cache'/'1-shot' and drops the value column entirely), so the header stays
+  // "Tok/s" -- the legend below carries the "Effective Tok/s" framing and the
+  // caveat that it is not a vendor decode-speed figure.
   const headers = ['cost', 'cache', 'calls', '1-shot', 'Tok/s']
   const values = sorted.map(([model, data], index) => {
     const totalInput = data.freshInput + data.cacheRead + data.cacheWrite
@@ -806,7 +811,7 @@ function ModelBreakdown({ projects, pw, bw }: { projects: ProjectSummary[]; pw:
       {anyEstimated && (
         <Text dimColor wrap="truncate-end">~ estimated cost (priced from estimated tokens)</Text>
       )}
-      <Text dimColor wrap="truncate-end">~ Tok/s: generated tokens / active time; tool wait excluded</Text>
+      <Text dimColor wrap="truncate-end">~ Effective Tok/s: generated tokens ÷ time the agent spent waiting on the model, tool execution excluded. Includes prefill, request assembly and reasoning. Not comparable to vendor decode-speed figures.</Text>
     </Panel>
   )
 }

src/parser.ts

@@ -1768,7 +1768,7 @@ function buildSessionSummary(
       modelBreakdown[modelKey].tokens.reasoningTokens += call.usage.reasoningTokens
       if (call.activeDurationMs !== undefined) {
         modelBreakdown[modelKey].activeDurationMs = (modelBreakdown[modelKey].activeDurationMs ?? 0) + call.activeDurationMs
-        modelBreakdown[modelKey].activeGeneratedTokens = (modelBreakdown[modelKey].activeGeneratedTokens ?? 0) + (call.activeGeneratedTokens ?? call.usage.outputTokens + call.usage.reasoningTokens)
+        modelBreakdown[modelKey].activeGeneratedTokens = (modelBreakdown[modelKey].activeGeneratedTokens ?? 0) + (call.activeGeneratedTokens ?? billableOutputTokens(call.provider, call.usage.outputTokens, call.usage.reasoningTokens))
         modelBreakdown[modelKey].toolWaitMs = (modelBreakdown[modelKey].toolWaitMs ?? 0) + (call.toolWaitMs ?? 0)
       }