diff --git a/CHANGELOG.md b/CHANGELOG.md index 878a5f77f..938788513 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,7 @@ - **The resident `codeburn serve` child.** The first real panel request is also the cache warm-up, so startup never runs an artificial warm-up query beside a duplicate one-shot child; each served command carries its own read-only option allowlist, and anything outside it falls back to a normal spawn; the child exits when its stdin closes, so it can never outlive the app. Requests whose response exceeds the 16 MiB frame limit still replace the child, but that deliberate kill no longer spends the resident's unexpected-death budget. (#972) ### Fixed +- **Subscription SKUs are classified from real product ids, and a false-positive built-in can be opted out.** `codex-auto-review` consumes ordinary Codex usage ([openai/codex#32224](https://github.com/openai/codex/issues/32224)) and is priced as GPT-5.5 on #1056, so treating it as $0 hid real spend — it left the flat-rate list. Warp's product id is `auto`, not the synthetic `warp`. `kimi-for-coding-highspeed` (the SKU #968 was filed around) is now honestly $0. `big-pickle` was dropped: it appears under OpenCode, not as a cited ClinePass codename. `codeburn model-flat-rate --remove` now opts out of a built-in, so a wrong classifier entry can warn again without waiting for a release. The daily-cache config hash now always includes the flat-rate section (even when empty), so the first run after upgrade re-derives every stored day once from the warm session cache. (#968, #1050) - **Codex MCP and skill usage is attributed from every shape Codex records a shell command in.** `mcp-cli call ` was only recognized when the command arrived as `function_call` arguments (#656). Codex has two other shapes for the same exec: its custom-tool transport records the shell tool as a `custom_tool_call` whose payload is an `input` program rather than `arguments`, and its item model repeats a finished command as `event_msg`/`item_completed` carrying a `CommandExecution` item with an argv `command`. Both reached the Bash counter and neither reached the matcher, so a CLI-wrapped MCP call stayed absent from the MCP breakdown exactly as before the fix. All three shapes now feed one classification pipeline. The same pipeline learns skills: Codex has no skill tool, so loading one is a shell read of the skill's `SKILL.md`, and those reads landed entirely under Bash with the Skills dimension empty. A read counts as a skill load only when the command segment starts with a file-reading binary (`cat`/`bat`/`sed`/`head`/`tail`/`less`/`more`) and the path it reads ends in `/SKILL.md`; the skill is ``, the same key `pi` derives for a native skill read (#588) and the same vocabulary the Claude parser records from the `Skill` tool. A `grep`/`rg`/`ls` that merely mentions a `SKILL.md` is a search near the file, not a skill load, and stays plain Bash. This is attribution only — no call, token or cost figure moves, and a command carried by both a response item and an item-model item is attributed once. On a 1,397-rollout corpus: Skills went from empty to 7 skills over 35 turns (55 attributions), Bash was unchanged at 42,170, and cost, calls, tokens, sessions, daily, models and projects came back identical. Cached Codex sessions re-parse once (`CODEX_CACHE_VERSION` 13 → 14 and the codex parse version both move; without them the fix is invisible on a warm cache). Thanks @chr-evensen. (#478) - **`gpt-5.6-codex` and `gpt-5.6-codex-max` now have their own pricing rows.** Neither id is in LiteLLM yet, and both were missing from the bundled snapshot — flagged during #1075 verification on a real corpus (285 sessions, 5,446 calls). `getModelCosts` already resolved both through the `gpt-5.6` prefix fallback, so live pricing was already correct once a session priced fresh; every prior Codex-suffixed id LiteLLM does carry bills identically to its bare-model sibling of the same generation (`gpt-5-codex` == `gpt-5`, `gpt-5.1-codex` == `gpt-5.1-codex-max` == `gpt-5.1`, `gpt-5.2-codex` == `gpt-5.2`, `gpt-5.3-codex` == `gpt-5.3`), which is the evidence both new rows mirror rather than inventing a rate. The gap that does not self-heal is the daily cache: it has no per-provider invalidation, so a day finalized while either id had no billable rate keeps that $0 forever. Raising `MIN_SUPPORTED_VERSION` (v23 -> v24) forces the one-time re-derivation, a lossless no-op for days already correct. (#1077) - **Mixed-version installs no longer thrash the Codex / Cursor / Antigravity result caches.** Daily and session caches already own a version-suffixed file so an old desktop binary and a newer CLI cannot clobber each other. The three per-provider result caches still used one unsuffixed filename with an internal version field, so a v10 and a v11 binary rewrote the same `codex-results.json` (and the Cursor / Antigravity siblings) on every run and each re-parsed its whole corpus. They now write `*-results.v.json` the same way the daily cache does. The unsuffixed file is left for older binaries; a matching-version copy is adopted once and never overwritten. (#1082) diff --git a/README.md b/README.md index b53a88b03..d929df01a 100644 --- a/README.md +++ b/README.md @@ -516,7 +516,7 @@ Sync sends token counts, costs, models, and projects, never prompts or code. Thi | `codeburn models --by-task` | Break each model into per-task-type rows | | `codeburn models --by-agent` | Break each model into per-agent rows: which agent drove which model's spend (`(main)` covers non-agent sessions; `--min-cost 0` shows sub-cent agents) | | `codeburn models --top 10` | Only the 10 most expensive models | -| `codeburn models --unpriced` | Only models with usage that currently price at $0 — the copyable form of the unpriced-models warning. Shows raw model IDs (not friendly names) so they can be pasted into `model-alias`; JSON keeps them exact | +| `codeburn models --unpriced` | Only models with usage that currently price at $0 — the copyable form of the unpriced-models warning. Shows raw model IDs (not friendly names). Per-token gaps go to `model-alias`; subscription / flat-rate SKUs go to `model-flat-rate`. JSON keeps IDs exact | | `codeburn models --format markdown` | Emit a paste-friendly markdown table | | `codeburn models --task feature` | Filter to feature-development work | | `codeburn models --provider claude` | Filter to a single provider | @@ -608,10 +608,11 @@ Aliases are stored in `~/.config/codeburn/config.json` and applied at runtime be ```bash codeburn price-override my-model --input 0.27 --output 1.10 # USD per 1M tokens codeburn model-savings "llama3.1:8b" gpt-4o # local model, counted as savings +codeburn model-flat-rate auto-genius # subscription SKU, $0 is correct codeburn proxy-path ~/work/copilot-repo # subscription-covered project ``` -`price-override` sets exact rates for any model (input, output, cache read, cache creation), useful for private deployments or models LiteLLM prices wrong. `model-savings` maps a free local model to a paid baseline: the local calls stay $0, and the dashboard shows what the same tokens would have cost on the baseline. `proxy-path` marks a project routed through a subscription-backed proxy (e.g. Claude Code over GitHub Copilot), so its API-rate cost is reported as subscription-covered and your net out-of-pocket stays honest. All three support `--list` and `--remove`. +`price-override` sets exact rates for any model (input, output, cache read, cache creation), useful for private deployments or models LiteLLM prices wrong. `model-savings` maps a free local model to a paid baseline: the local calls stay $0, and the dashboard shows what the same tokens would have cost on the baseline. `model-flat-rate` marks a subscription-billed product SKU so the unpriced warning stays quiet and `model-alias` is not suggested — aliasing those ids invents spend. `--remove` also opts out of a built-in SKU. `proxy-path` marks a project routed through a subscription-backed proxy (e.g. Claude Code over GitHub Copilot), so its API-rate cost is reported as subscription-covered and your net out-of-pocket stays honest. All four support `--list` and `--remove`. ### Filtering diff --git a/src/config.ts b/src/config.ts index b042749e8..742a7d650 100644 --- a/src/config.ts +++ b/src/config.ts @@ -44,6 +44,15 @@ export type CodeburnConfig = { // can show "saved $X by running locally". Distinct from modelAliases which // rewrites actual spend. localModelSavings?: Record + // Model ids whose $0 cost is correct because they are billed as a + // subscription / flat-rate product, not missing LiteLLM rows. Distinct from + // modelAliases (which invent per-token spend) and localModelSavings + // (counterfactual local baseline). See `codeburn model-flat-rate`. + flatRateModels?: string[] + // Opt-outs from the built-in flat-rate classifier. `model-flat-rate --remove` + // on a built-in SKU records the id here so a false positive can warn again + // without waiting for a release. + flatRateModelsRemoved?: string[] // Spend budgets are stored in the configured display currency, not USD. budget?: { daily?: number diff --git a/src/main.ts b/src/main.ts index 684f60160..67ba17ba9 100644 --- a/src/main.ts +++ b/src/main.ts @@ -2,7 +2,7 @@ import { isAbsolute } from 'path' import { Command, Option } from 'commander' import { installMenubarApp } from './menubar-installer.js' import { exportCsv, exportJson, type PeriodExport } from './export.js' -import { findUnpricedModels, loadPricing, sanitizeModelForDisplay, setModelAliases, setPriceOverrides, setLocalModelSavings, setProxyPaths, normalizeProxyPath } from './models.js' +import { findUnpricedModels, loadPricing, sanitizeModelForDisplay, setModelAliases, setPriceOverrides, setLocalModelSavings, setFlatRateModels, setFlatRateRemoved, setProxyPaths, normalizeProxyPath, unpricedModelHint, isBuiltInFlatRateModel, isSameFlatRateModel } from './models.js' import { parseAllSessions, filterProjectsByName, filterProjectsByDateRange, clearSessionCache, setInteractiveScanUI } from './parser.js' import { allProviderNames, getAllProviders } from './providers/index.js' import { getProvider } from './providers/index.js' @@ -465,6 +465,8 @@ program.hook('preAction', async (thisCommand) => { setModelAliases(config.modelAliases ?? {}) setPriceOverrides(config.priceOverrides ?? {}) setLocalModelSavings(config.localModelSavings ?? {}) + setFlatRateModels(config.flatRateModels ?? []) + setFlatRateRemoved(config.flatRateModelsRemoved ?? []) setProxyPaths(config.proxyPaths ?? []) if (thisCommand.opts<{ verbose?: boolean }>().verbose) { process.env['CODEBURN_VERBOSE'] = '1' @@ -1524,6 +1526,87 @@ program console.log(` Config: ${getConfigFilePath()}\n`) }) +program + .command('model-flat-rate [model]') + .description('Mark a model as subscription / flat-rate billed. $0 is the correct cost and the unpriced warning is silenced. Do not use model-alias for these — that maps them onto another model\'s per-token rate and invents spend (e.g. codeburn model-flat-rate auto-genius).') + .option('--remove ', 'Remove a flat-rate mark, including a built-in SKU') + .option('--list', 'List configured flat-rate models and built-in opt-outs') + .action(async (model?: string, opts?: { remove?: string; list?: boolean }) => { + const config = await readConfig() + const marked = [...(config.flatRateModels ?? [])] + const removed = [...(config.flatRateModelsRemoved ?? [])] + + if (opts?.list || (!model && !opts?.remove)) { + if (marked.length === 0 && removed.length === 0) { + console.log('\n No flat-rate models configured.') + console.log(` Config: ${getConfigFilePath()}`) + console.log(' Add one with: codeburn model-flat-rate \n') + } else { + if (marked.length > 0) { + console.log('\n Flat-rate / subscription models:') + for (const name of marked) { + console.log(` ${name}`) + } + } + if (removed.length > 0) { + console.log('\n Built-in flat-rate opt-outs (unpriced warning fires again):') + for (const name of removed) { + console.log(` ${name}`) + } + } + console.log(` Config: ${getConfigFilePath()}\n`) + } + return + } + + if (opts?.remove) { + const target = opts.remove + const idx = marked.indexOf(target) + const builtIn = isBuiltInFlatRateModel(target) + const alreadyOptedOut = removed.some(id => isSameFlatRateModel(id, target)) + if (idx < 0 && (!builtIn || alreadyOptedOut)) { + console.error(`\n No flat-rate mark found for: ${target}\n`) + process.exitCode = 1 + return + } + if (idx >= 0) { + marked.splice(idx, 1) + config.flatRateModels = marked.length > 0 ? marked : undefined + } + if (builtIn && !alreadyOptedOut) { + removed.push(target) + config.flatRateModelsRemoved = removed + } + await saveConfig(config) + console.log(`\n Removed flat-rate mark: ${target}`) + if (builtIn) { + console.log(' Built-in SKU opted out; the unpriced warning will fire again until you re-add it.') + } + console.log() + return + } + + if (!model) { + console.error('\n Usage: codeburn model-flat-rate \n') + process.exitCode = 1 + return + } + + if (!marked.includes(model)) marked.push(model) + config.flatRateModels = marked + const remainingOptOuts = removed.filter(id => !isSameFlatRateModel(id, model)) + config.flatRateModelsRemoved = remainingOptOuts.length > 0 ? remainingOptOuts : undefined + await saveConfig(config) + + if (config.modelAliases && Object.hasOwn(config.modelAliases, model)) { + console.log(`\n Note: ${model} is also in modelAliases (-> ${config.modelAliases[model]}).`) + console.log(' The alias still invents per-token spend. Remove it if $0 is the correct cost.') + } + + console.log(`\n Flat-rate mark saved: ${model}`) + console.log(` Config: ${getConfigFilePath()}\n`) + }) + program .command('proxy-path [path]') .description('Mark a project directory as routed through a subscription-backed LLM proxy (e.g. Claude Code over GitHub Copilot). Sessions whose canonical path is under it keep their full API-rate cost as the "would-be" figure, but that amount is reported as subscription-covered so the report can show net out-of-pocket (e.g. codeburn proxy-path ~/work/copilot-repo). Actual API-key sessions elsewhere are untouched.') @@ -2109,7 +2192,7 @@ program process.stdout.write(renderTable(renderRows, { byTask: !!opts.byTask, byAgent: !!opts.byAgent, showTotals: opts.totals !== false }) + '\n') // Never advise aliasing unconditionally: a subscription or flat-rate model // is correctly $0, and mapping it onto another model's rate invents spend. - if (opts.unpriced) process.stdout.write('If a model is billed per token, map it with: codeburn model-alias "" . Subscription or flat-rate models are correctly $0.\n') + if (opts.unpriced) process.stdout.write(unpricedModelHint() + '\n') } else { process.stderr.write(`codeburn: unknown --format "${opts.format}". Choose table, markdown, json, or csv.\n`) process.exit(1) diff --git a/src/mcp/tables.ts b/src/mcp/tables.ts index 9db92b722..c97110b9b 100644 --- a/src/mcp/tables.ts +++ b/src/mcp/tables.ts @@ -1,5 +1,6 @@ import { formatCost, formatTokens, markEstimated } from '../format.js' import type { MenubarPayload } from '../menubar-json.js' +import { unpricedModelHint } from '../models.js' const ESTIMATED_LEGEND = '_~ estimated cost (priced from estimated tokens)_' const isEstimated = (m: { estimatedCostUSD?: number }) => (m.estimatedCostUSD ?? 0) > 0 @@ -23,7 +24,7 @@ export function renderSummaryTable(p: MenubarPayload): string { `**${c.label}** — ${formatCost(c.cost)} · ${c.calls} calls · ${c.sessions} sessions`, `cache hit ${pct(c.cacheHitPercent)} · one-shot ${oneShot(c.oneShotRate)} · in ${formatTokens(c.inputTokens)} / out ${formatTokens(c.outputTokens)}`, ...(unpriced.length > 0 - ? [`⚠ ${unpriced.length} model${unpriced.length === 1 ? '' : 's'} unpriced, counted at $0: ${unpriced.map(u => `${u.model} (${u.calls} calls)`).join(', ')}. Cost above understates real spend; fix with \`codeburn model-alias\` or \`codeburn price-override\`.`] + ? [`⚠ ${unpriced.length} model${unpriced.length === 1 ? '' : 's'} unpriced, counted at $0: ${unpriced.map(u => `${u.model} (${u.calls} calls)`).join(', ')}. ${unpricedModelHint()}`] : []), '', '_Top models_', diff --git a/src/models.ts b/src/models.ts index 780cefe84..002eeb390 100644 --- a/src/models.ts +++ b/src/models.ts @@ -569,6 +569,117 @@ export function getLocalModelSavingsConfigHash(): string { return parts.join('\u0002') } +// Subscription / flat-rate product SKUs. $0 is the correct cost; aliasing +// them onto a per-token row fabricates spend (#968). Distinct from +// model-savings (counterfactual local baseline) and from a zero-rate +// price-override (user-declared free). Built-in families plus a user hatch. +let userFlatRateModels = new Set() +let userFlatRateLeaves = new Set() +let userFlatRateRemoved = new Set() +let userFlatRateRemovedLeaves = new Set() + +function flatRateLeaf(model: string): string { + const trimmed = model.trim().replace(/@.*$/, '').replace(/-\d{8}$/, '') + const leaf = trimmed.includes('/') ? trimmed.slice(trimmed.lastIndexOf('/') + 1) : trimmed + return leaf.toLowerCase() +} + +function fillFlatRateSet( + models: Iterable, +): { ids: Set; leaves: Set } { + const ids = new Set() + const leaves = new Set() + for (const model of models) { + if (!model || typeof model !== 'string') continue + ids.add(model) + const leaf = flatRateLeaf(model) + if (leaf) leaves.add(leaf) + } + return { ids, leaves } +} + +export function setFlatRateModels(models: Iterable): void { + const filled = fillFlatRateSet(models) + userFlatRateModels = filled.ids + userFlatRateLeaves = filled.leaves +} + +export function setFlatRateRemoved(models: Iterable): void { + const filled = fillFlatRateSet(models) + userFlatRateRemoved = filled.ids + userFlatRateRemovedLeaves = filled.leaves +} + +export function getFlatRateModelsConfigHash(): string { + const added = [...userFlatRateModels].sort().join('\u0002') + const removed = [...userFlatRateRemoved].sort().join('\u0002') + if (!removed) return added + return `${added}\u0003${removed}` +} + +export function getFlatRateModels(): string[] { + return [...userFlatRateModels] +} + +export function getFlatRateRemoved(): string[] { + return [...userFlatRateRemoved] +} + +export function isSameFlatRateModel(a: string, b: string): boolean { + if (!a || !b) return false + if (a === b) return true + const leaf = flatRateLeaf(a) + return leaf.length > 0 && leaf === flatRateLeaf(b) +} + +function isUserFlatRateModel(model: string): boolean { + if (userFlatRateModels.has(model)) return true + const leaf = flatRateLeaf(model) + return leaf.length > 0 && userFlatRateLeaves.has(leaf) +} + +function isFlatRateRemoved(model: string): boolean { + if (userFlatRateRemoved.has(model)) return true + const leaf = flatRateLeaf(model) + return leaf.length > 0 && userFlatRateRemovedLeaves.has(leaf) +} + +/// Product SKUs billed as a subscription, not missing LiteLLM rows. +/// Match raw ids and path-prefixed ids (`cline-pass/auto-genius`). Display +/// names from getShortModelName are matched only when the aggregation key +/// is not the raw leaf (Warp Auto *, Grok Composer *). +export function isBuiltInFlatRateModel(model: string): boolean { + const leaf = flatRateLeaf(model) + // Warp's product SKU is the bare id `auto`. Kiro rewrites its own `auto` + // to `kiro-auto` before pricing, so this leaf does not swallow Kiro. + if ( + leaf === 'auto' + || leaf === 'auto-genius' + || leaf === 'kimi-for-coding-highspeed' + ) return true + if (leaf.startsWith('grok-composer-')) return true + if (leaf.startsWith('warp-auto-')) return true + const display = model.trim() + if (/^grok composer\b/i.test(display)) return true + if (/^warp auto\b/i.test(display)) return true + return false +} + +export function isFlatRateModel(model: string): boolean { + if (!model) return false + if (isFlatRateRemoved(model)) return false + return isUserFlatRateModel(model) || isBuiltInFlatRateModel(model) +} + +/// Shared unpriced-warning copy. Never tell the user to alias unconditionally: +/// mapping a subscription SKU onto a priced row invents spend. Optional `model` +/// interpolates the sanitized id so the verbose calculateCost path names the +/// same two hatches. +export function unpricedModelHint(model = ''): string { + const safe = model.replace(/[\x00-\x1F\x7F-\x9F]/g, '?').slice(0, 200) + return `If a model is billed per token, map it with: codeburn model-alias "${safe}" . If $0 is correct (subscription / flat-rate): codeburn model-flat-rate "${safe}".` +} + /// Stable hash of the model-alias map, for the same staleness class as the /// hashes below: a resident process (codeburn serve) must not serve memoized /// parse results priced under aliases the user has since changed. @@ -910,14 +1021,18 @@ function exactPriceOverrideFor(model: string): ModelCosts | null { // correct cost, as are zero-rate USER overrides (explicitly declared free). /// Models whose $0 cost is CORRECT rather than a pricing gap, mirroring the /// exclusions findUnpricedModels applies: local-looking models, models mapped -/// to a local-savings baseline, and models an exact zero-rate user override -/// declares free. Used to keep their calls out of the pricing-coverage -/// denominator — otherwise a 95%-ollama user reads high coverage while every -/// genuinely cost-bearing call is unpriced. +/// to a local-savings baseline, subscription / flat-rate product SKUs, and +/// models an exact zero-rate user override declares free. Used to keep their +/// calls out of the pricing-coverage denominator — otherwise a 95%-ollama +/// user reads high coverage while every genuinely cost-bearing call is unpriced. export function isExpectedFreeModel(model: string): boolean { if (looksLikeLocalModel(model)) return true if (getLocalSavingsBaseline(model)) return true const costs = getModelCosts(model) + // A builtin/user alias can still attach a billable rate to a subscription + // SKU (warp-auto-* today). Those calls are priced, so they stay in the + // coverage denominator. Only the $0 / no-rate case is expected-free. + if (isFlatRateModel(model) && (!costs || !hasBillableRate(costs))) return true if (costs && !hasBillableRate(costs) && exactPriceOverrideFor(model)) return true return false } @@ -934,6 +1049,7 @@ export function findUnpricedModels( if (row.cost > 0) continue if (looksLikeLocalModel(model)) continue if (getLocalSavingsBaseline(model)) continue + if (isFlatRateModel(model)) continue const costs = getModelCosts(model) if (costs && hasBillableRate(costs)) continue if (costs && exactPriceOverrideFor(model)) continue @@ -950,6 +1066,7 @@ function shouldWarnAboutUnknownModel(name: string): boolean { // actively misleading there. Users who need cost visibility for local // inference can still set an alias via `codeburn model-alias`. if (looksLikeLocalModel(name)) return false + if (isFlatRateModel(name)) return false // The warning fired on every CLI invocation (including the default // dashboard) which made first launches look broken — three "no pricing // data" lines greet a user before the dashboard even draws. Now opt-in @@ -982,10 +1099,9 @@ export function calculateCost( // payloads written by external tools, so a hostile or corrupt file // could embed terminal escape sequences here. const safeName = sanitizeModelForDisplay(model) - const aliasHint = `Map it with: codeburn model-alias "${safeName}" , or track local-model savings with: codeburn model-savings "${safeName}" ` process.stderr.write( `codeburn: no pricing data for model "${safeName}" — costs for this model will show $0. ` + - `${aliasHint}, or update with: npx codeburn@latest.\n` + `${unpricedModelHint(safeName)} Or track local-model savings with: codeburn model-savings "${safeName}" , or update with: npx codeburn@latest.\n`, ) } return 0 @@ -1228,6 +1344,8 @@ export type PricingSnapshot = { aliases: Record priceOverrides: Record localModelSavings: Record + flatRateModels?: string[] + flatRateModelsRemoved?: string[] } export function snapshotPricingState(): PricingSnapshot { @@ -1236,6 +1354,8 @@ export function snapshotPricingState(): PricingSnapshot { aliases: userAliases, priceOverrides: userPriceOverridesConfig, localModelSavings: userLocalModelSavings, + flatRateModels: getFlatRateModels(), + flatRateModelsRemoved: getFlatRateRemoved(), } } @@ -1247,4 +1367,6 @@ export function restorePricingState(snapshot: PricingSnapshot): void { setModelAliases(snapshot.aliases) setPriceOverrides(snapshot.priceOverrides) setLocalModelSavings(snapshot.localModelSavings) + setFlatRateModels(snapshot.flatRateModels ?? []) + setFlatRateRemoved(snapshot.flatRateModelsRemoved ?? []) } diff --git a/src/overview.ts b/src/overview.ts index 4f6cabdc8..529924646 100644 --- a/src/overview.ts +++ b/src/overview.ts @@ -4,7 +4,7 @@ import { homedir } from 'os' import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory } from './types.js' import { formatCost as baseCost, getCurrency } from './currency.js' -import { findUnpricedModels, getShortModelName } from './models.js' +import { findUnpricedModels, getShortModelName, unpricedModelHint } from './models.js' import { markEstimated } from './format.js' import { dateKey } from './day-aggregator.js' import type { DailyEntry } from './daily-cache.js' @@ -224,7 +224,7 @@ export function renderOverview( .join(', ') const more = unpriced.length > 3 ? ` +${unpriced.length - 3} more` : '' out.push(kv('Unpriced', c.yellow(`${unpriced.length} model${unpriced.length === 1 ? '' : 's'} at $0: `) + shown + more)) - out.push(kv('', c.dim('Fix: codeburn model-alias "" '))) + out.push(kv('', c.dim(unpricedModelHint()))) } if (opts.budget) { const label = opts.budget.tier === 'daily' diff --git a/src/parser.ts b/src/parser.ts index 071bde686..13677c08e 100644 --- a/src/parser.ts +++ b/src/parser.ts @@ -3506,6 +3506,9 @@ function cacheKey(dateRange: DateRange | undefined, providerFilter: string | und // Pricing-affecting config participates so a memoized parse (exact-key or // burst-reused in a resident serve process) can never present costs priced // under aliases/overrides/savings the user has since changed. + // Flat-rate marks do not change parse-time cost (still $0 without a LiteLLM + // row); findUnpricedModels / coverage apply them at render time, so they + // stay out of this serve-memo key on purpose. return `${s}:${providerFilter ?? 'all'}:${claudeRoots}:${getProxyPathsConfigHash()}:${getModelAliasesConfigHash()}:${getPriceOverridesConfigHash()}:${getLocalModelSavingsConfigHash()}` } diff --git a/src/usage-aggregator.ts b/src/usage-aggregator.ts index 78b1d4830..cd27a1f74 100644 --- a/src/usage-aggregator.ts +++ b/src/usage-aggregator.ts @@ -2,7 +2,7 @@ import { homedir } from 'node:os' import { CATEGORY_LABELS, type ProjectSummary, type TaskCategory, type DateRange } from './types.js' import { type PeriodData, type ProviderCost, type BreakdownArrays, type MenubarPayload, type ClaudeConfigSelector, buildMenubarPayload } from './menubar-json.js' import { parseAllSessions, filterProjectsByName, filterProjectsByDays, filterProjectsByClaudeConfigSource, isSessionHydrationComplete } from './parser.js' -import { findUnpricedModels, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName, isExpectedFreeModel } from './models.js' +import { findUnpricedModels, getFlatRateModelsConfigHash, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getShortModelName, isExpectedFreeModel } from './models.js' import { getAllProviders, safeDiscoverSessions } from './providers/index.js' import { claude, getClaudeConfigDirs, getDesktopSessionsDirs } from './providers/claude.js' import { stat } from 'node:fs/promises' @@ -82,8 +82,8 @@ export function buildPeriodData(label: string, projects: ProjectSummary[]): Peri export function getDailyCacheConfigHash(): string { const savingsHash = getLocalModelSavingsConfigHash() const overridesHash = getPriceOverridesConfigHash() - if (!overridesHash) return savingsHash - return `localModelSavings=${savingsHash}\u0002priceOverrides=${overridesHash}` + const flatRateHash = getFlatRateModelsConfigHash() + return `localModelSavings=${savingsHash}\u0002priceOverrides=${overridesHash}\u0002flatRateModels=${flatRateHash}` } async function hydrateCache(): Promise { diff --git a/tests/cli-model-flat-rate.test.ts b/tests/cli-model-flat-rate.test.ts new file mode 100644 index 000000000..c883a6550 --- /dev/null +++ b/tests/cli-model-flat-rate.test.ts @@ -0,0 +1,105 @@ +import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { spawnSync } from 'node:child_process' + +import { describe, it, expect } from 'vitest' + +const CLI_TIMEOUT_MS = 30_000 + +function runCli(args: string[], home: string) { + return spawnSync(process.execPath, ['--import', 'tsx', 'src/cli.ts', ...args], { + cwd: process.cwd(), + env: { + ...process.env, + HOME: home, + USERPROFILE: home, + HOMEPATH: home, + HOMEDRIVE: '', + }, + encoding: 'utf-8', + }) +} + +function readConfig(home: string): Promise> { + return readFile(join(home, '.config', 'codeburn', 'config.json'), 'utf-8') + .then(raw => JSON.parse(raw) as Record) +} + +describe('codeburn model-flat-rate command', () => { + it('saves, lists, and removes a flat-rate mark', async () => { + const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-')) + try { + const set = runCli(['model-flat-rate', 'zz-my-pass-sku'], home) + expect(set.status).toBe(0) + expect(set.stdout).toContain('Flat-rate mark saved: zz-my-pass-sku') + + const saved = await readConfig(home) + expect(saved.flatRateModels).toEqual(['zz-my-pass-sku']) + + const list = runCli(['model-flat-rate', '--list'], home) + expect(list.status).toBe(0) + expect(list.stdout).toContain('zz-my-pass-sku') + + const remove = runCli(['model-flat-rate', '--remove', 'zz-my-pass-sku'], home) + expect(remove.status).toBe(0) + + const after = await readConfig(home) + expect(after.flatRateModels).toBeUndefined() + expect(after.flatRateModelsRemoved).toBeUndefined() + } finally { + await rm(home, { recursive: true, force: true }) + } + }, CLI_TIMEOUT_MS) + + it('warns when the same model is also configured in modelAliases', async () => { + const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-')) + try { + expect(runCli(['model-alias', 'auto-genius', 'gpt-4o'], home).status).toBe(0) + const set = runCli(['model-flat-rate', 'auto-genius'], home) + expect(set.status).toBe(0) + expect(set.stdout).toContain('also in modelAliases') + expect(set.stdout).toContain('invents per-token spend') + } finally { + await rm(home, { recursive: true, force: true }) + } + }, CLI_TIMEOUT_MS) + + it('rejects a remove for an unknown mark', async () => { + const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-')) + try { + const result = runCli(['model-flat-rate', '--remove', 'unknown-sku'], home) + expect(result.status).toBe(1) + expect(result.stderr).toContain('No flat-rate mark found') + } finally { + await rm(home, { recursive: true, force: true }) + } + }, CLI_TIMEOUT_MS) + + it('opts out of a built-in SKU so the unpriced warning can fire again', async () => { + const home = await mkdtemp(join(tmpdir(), 'codeburn-cli-flat-rate-')) + try { + const remove = runCli(['model-flat-rate', '--remove', 'auto-genius'], home) + expect(remove.status).toBe(0) + expect(remove.stdout).toContain('Removed flat-rate mark: auto-genius') + expect(remove.stdout).toContain('Built-in SKU opted out') + + const saved = await readConfig(home) + expect(saved.flatRateModels).toBeUndefined() + expect(saved.flatRateModelsRemoved).toEqual(['auto-genius']) + + const list = runCli(['model-flat-rate', '--list'], home) + expect(list.status).toBe(0) + expect(list.stdout).toContain('auto-genius') + expect(list.stdout).toContain('Built-in flat-rate opt-outs') + + const restore = runCli(['model-flat-rate', 'auto-genius'], home) + expect(restore.status).toBe(0) + const after = await readConfig(home) + expect(after.flatRateModels).toEqual(['auto-genius']) + expect(after.flatRateModelsRemoved).toBeUndefined() + } finally { + await rm(home, { recursive: true, force: true }) + } + }, CLI_TIMEOUT_MS) +}) diff --git a/tests/cli-models-unpriced.test.ts b/tests/cli-models-unpriced.test.ts index eb93fe903..72492dcc4 100644 --- a/tests/cli-models-unpriced.test.ts +++ b/tests/cli-models-unpriced.test.ts @@ -114,9 +114,7 @@ describe('codeburn models --unpriced public CLI', () => { expect(result.stdout).toContain('acme/unknown-beta-969') expect(result.stdout).not.toContain('claude-opus-4-6') expect(result.stdout).toContain('If a model is billed per token, map it with: codeburn model-alias "" ') - // #968: aliasing a subscription-billed model fabricates spend, so the - // hint must never read as an unconditional instruction. - expect(result.stdout).toContain('Subscription or flat-rate models are correctly $0.') + expect(result.stdout).toContain('codeburn model-flat-rate') expect(result.stdout).not.toContain('Fix: codeburn model-alias') }) }) diff --git a/tests/mcp-tables.test.ts b/tests/mcp-tables.test.ts index e0d32e49f..d4d328d5d 100644 --- a/tests/mcp-tables.test.ts +++ b/tests/mcp-tables.test.ts @@ -25,6 +25,14 @@ describe('tables', () => { expect(t).toContain('Opus 4.8') expect(t).toContain('| Model | Cost | Calls |') }) + it('unpriced warning names the flat-rate hatch instead of only alias', () => { + const p = payload() + p.current.unpricedModels = [{ model: 'zz-mystery-paid-model-999', calls: 3, tokens: 1200 }] + const t = renderSummaryTable(p) + expect(t).toContain('zz-mystery-paid-model-999') + expect(t).toContain('model-flat-rate') + expect(t).not.toContain('fix with `codeburn model-alias`') + }) it('breakdown by provider lists providers', () => { expect(renderBreakdownTable(payload(), 'provider', 20)).toContain('claude code') }) diff --git a/tests/models.test.ts b/tests/models.test.ts index ab4538e83..6338dabf4 100644 --- a/tests/models.test.ts +++ b/tests/models.test.ts @@ -13,10 +13,16 @@ import { setModelAliases, setPriceOverrides, setLocalModelSavings, + setFlatRateModels, + setFlatRateRemoved, + isExpectedFreeModel, + isFlatRateModel, getLocalModelSavingsConfigHash, getPriceOverridesConfigHash, getModelAliasesConfigHash, + getFlatRateModelsConfigHash, parseLiteLLMEntry, + unpricedModelHint, } from '../src/models.js' import { getDailyCacheConfigHash } from '../src/usage-aggregator.js' import snapshotData from '../src/data/litellm-snapshot.json' with { type: 'json' } @@ -29,6 +35,8 @@ afterEach(() => { setModelAliases({}) setPriceOverrides({}) setLocalModelSavings({}) + setFlatRateModels([]) + setFlatRateRemoved([]) }) describe('getModelCosts', () => { @@ -530,6 +538,17 @@ describe('user price overrides', () => { expect(secondCombined).not.toBe(baseline) expect(secondCombined).not.toBe(firstCombined) }) + + it('includes flat-rate marks in the daily cache config hash', () => { + setLocalModelSavings({}) + setPriceOverrides({}) + setFlatRateModels([]) + const baseline = getDailyCacheConfigHash() + setFlatRateModels(['zz-flat-hash']) + expect(getDailyCacheConfigHash()).not.toBe(baseline) + setFlatRateModels([]) + expect(getDailyCacheConfigHash()).toBe(baseline) + }) }) describe('calculateCost - OMP names produce non-zero cost', () => { @@ -1087,6 +1106,71 @@ describe('findUnpricedModels', () => { expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([]) }) + it('skips subscription / flat-rate product SKUs where $0 is correct', () => { + const rows = [ + { model: 'auto-genius', calls: 898, cost: 0, tokens: 35_300_000 }, + { model: 'cline-pass/auto-genius', calls: 4, cost: 0, tokens: 33_900 }, + { model: 'auto', calls: 449, cost: 0, tokens: 17_700_000 }, + { model: 'kimi-for-coding-highspeed', calls: 12, cost: 0, tokens: 3_400_000 }, + { model: 'moonshot/kimi-for-coding-highspeed', calls: 2, cost: 0, tokens: 80_000 }, + { model: 'grok-composer-2.5-fast', calls: 10, cost: 0, tokens: 1_900_000 }, + { model: 'Grok Composer 2.5 Fast', calls: 10, cost: 0, tokens: 1_900_000 }, + { model: 'Warp Auto (efficient)', calls: 3, cost: 0, tokens: 50_000 }, + { model: 'warp', calls: 449, cost: 0, tokens: 17_700_000 }, + { model: 'codex-auto-review', calls: 940, cost: 0, tokens: 7_200_000 }, + { model: 'Codex Auto Review', calls: 2, cost: 0, tokens: 100 }, + { model: 'big-pickle', calls: 4, cost: 0, tokens: 33_900 }, + { model: 'zz-mystery-paid-model-999', calls: 3, cost: 0, tokens: 1200 }, + ] + expect(findUnpricedModels(rows)).toEqual([ + { model: 'warp', calls: 449, tokens: 17_700_000 }, + // Note: NOT 'codex-auto-review' — #1056 aliases it to gpt-5.5, so it + // now resolves a billable rate and is filtered out here (a $0 row for + // it is stale data, not evidence of missing pricing). It still left + // the flat-rate list, verified separately in the "Codex activity ids + // (#1047)" describe block below. + { model: 'big-pickle', calls: 4, tokens: 33_900 }, + { model: 'zz-mystery-paid-model-999', calls: 3, tokens: 1200 }, + { model: 'Codex Auto Review', calls: 2, tokens: 100 }, + ]) + }) + + it('skips a user-declared flat-rate model, including path-prefixed siblings', () => { + const model = 'zz-my-pass-codename' + expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toHaveLength(1) + setFlatRateModels([model]) + expect(findUnpricedModels([{ model, calls: 1, cost: 0, tokens: 10 }])).toEqual([]) + expect(findUnpricedModels([{ model: `vendor/${model}`, calls: 1, cost: 0, tokens: 10 }])).toEqual([]) + expect(findUnpricedModels([{ model: 'zz-other-unknown', calls: 1, cost: 0, tokens: 10 }])).toHaveLength(1) + }) + + it('does not treat a priced sibling as expected-free just because a family is flat-rate', () => { + // warp-auto-* is a subscription SKU, but main already aliases it onto a + // billable row. Coverage must still count those priced calls. + expect(isFlatRateModel('warp-auto-efficient')).toBe(true) + expect(getModelCosts('warp-auto-efficient')).not.toBeNull() + expect(isExpectedFreeModel('warp-auto-efficient')).toBe(false) + expect(isExpectedFreeModel('auto-genius')).toBe(true) + expect(isExpectedFreeModel('auto')).toBe(true) + expect(isExpectedFreeModel('kimi-for-coding-highspeed')).toBe(true) + expect(isExpectedFreeModel('warp')).toBe(false) + expect(isExpectedFreeModel('codex-auto-review')).toBe(false) + expect(isExpectedFreeModel('zz-mystery-paid-model-999')).toBe(false) + }) + + it('lets --remove opt out of a built-in so a false positive can warn again', () => { + expect(findUnpricedModels([{ model: 'auto-genius', calls: 1, cost: 0, tokens: 10 }])).toEqual([]) + setFlatRateRemoved(['auto-genius']) + expect(isFlatRateModel('auto-genius')).toBe(false) + expect(findUnpricedModels([{ model: 'auto-genius', calls: 1, cost: 0, tokens: 10 }])).toEqual([ + { model: 'auto-genius', calls: 1, tokens: 10 }, + ]) + expect(findUnpricedModels([{ model: 'cline-pass/auto-genius', calls: 1, cost: 0, tokens: 10 }])).toEqual([ + { model: 'cline-pass/auto-genius', calls: 1, tokens: 10 }, + ]) + expect(isFlatRateModel('auto')).toBe(true) + }) + it('sorts by tokens, then calls', () => { const unpriced = findUnpricedModels([ { model: 'zz-small', calls: 9, cost: 0, tokens: 10 }, @@ -1126,3 +1210,97 @@ describe('getModelAliasesConfigHash', () => { setModelAliases({}) }) }) + +describe('getFlatRateModelsConfigHash', () => { + it('is empty for no marks, changes with content, ignores insertion order', () => { + setFlatRateModels([]) + expect(getFlatRateModelsConfigHash()).toBe('') + setFlatRateModels(['auto-genius']) + const one = getFlatRateModelsConfigHash() + expect(one).not.toBe('') + setFlatRateModels(['warp', 'auto-genius']) + const two = getFlatRateModelsConfigHash() + expect(two).not.toBe(one) + setFlatRateModels(['auto-genius', 'warp']) + expect(getFlatRateModelsConfigHash()).toBe(two) + setFlatRateModels([]) + }) + + it('changes when a built-in is opted out', () => { + setFlatRateModels([]) + setFlatRateRemoved([]) + const baseline = getFlatRateModelsConfigHash() + setFlatRateRemoved(['auto-genius']) + expect(getFlatRateModelsConfigHash()).not.toBe(baseline) + setFlatRateRemoved([]) + expect(getFlatRateModelsConfigHash()).toBe(baseline) + }) +}) + +describe('pricing snapshot carries flat-rate marks', () => { + it('restorePricingState reapplies user flat-rate marks', async () => { + const { snapshotPricingState, restorePricingState } = await import('../src/models.js') + setFlatRateModels(['zz-snapshot-flat']) + const snap = snapshotPricingState() + expect(snap.flatRateModels).toEqual(['zz-snapshot-flat']) + setFlatRateModels([]) + expect(isFlatRateModel('zz-snapshot-flat')).toBe(false) + restorePricingState(snap) + expect(isFlatRateModel('zz-snapshot-flat')).toBe(true) + setFlatRateModels([]) + }) + + it('restorePricingState reapplies built-in opt-outs', async () => { + const { snapshotPricingState, restorePricingState } = await import('../src/models.js') + setFlatRateRemoved(['auto-genius']) + const snap = snapshotPricingState() + expect(snap.flatRateModelsRemoved).toEqual(['auto-genius']) + setFlatRateRemoved([]) + expect(isFlatRateModel('auto-genius')).toBe(true) + restorePricingState(snap) + expect(isFlatRateModel('auto-genius')).toBe(false) + setFlatRateRemoved([]) + }) +}) + +describe('unpricedModelHint', () => { + it('never tells the user to alias unconditionally', () => { + expect(unpricedModelHint()).toContain('If a model is billed per token') + expect(unpricedModelHint()).toContain('model-flat-rate') + expect(unpricedModelHint()).not.toContain('Fix: codeburn model-alias') + }) + + it('names both hatches for a concrete unknown SKU', () => { + const hint = unpricedModelHint('zz-new-subscription-pass-sku') + expect(hint).toContain('codeburn model-alias "zz-new-subscription-pass-sku"') + expect(hint).toContain('codeburn model-flat-rate "zz-new-subscription-pass-sku"') + expect(hint).toContain('If a model is billed per token') + expect(hint).toContain('If $0 is correct') + }) +}) + +describe('calculateCost verbose unknown-model warning', () => { + it('does not present model-alias as the only fix for an unknown SKU', () => { + const previous = process.env['CODEBURN_VERBOSE'] + process.env['CODEBURN_VERBOSE'] = '1' + const chunks: string[] = [] + const originalWrite = process.stderr.write.bind(process.stderr) + process.stderr.write = ((chunk: string | Uint8Array, ...args: unknown[]) => { + chunks.push(typeof chunk === 'string' ? chunk : Buffer.from(chunk).toString()) + return (originalWrite as (chunk: string | Uint8Array, ...rest: unknown[]) => boolean)(chunk, ...args) + }) as typeof process.stderr.write + try { + expect(calculateCost('zz-new-subscription-pass-sku', 10, 10, 0, 0, 0)).toBe(0) + } finally { + process.stderr.write = originalWrite + if (previous === undefined) delete process.env['CODEBURN_VERBOSE'] + else process.env['CODEBURN_VERBOSE'] = previous + } + const text = chunks.join('') + expect(text).toContain('zz-new-subscription-pass-sku') + expect(text).toContain('If a model is billed per token') + expect(text).toContain('model-flat-rate') + expect(text).toContain('model-alias') + expect(text).not.toMatch(/Map it with: codeburn model-alias/) + }) +}) diff --git a/tests/overview.test.ts b/tests/overview.test.ts index 8fcb9525d..bdc38587f 100644 --- a/tests/overview.test.ts +++ b/tests/overview.test.ts @@ -240,6 +240,23 @@ describe('renderOverview unpriced models', () => { expect(out).toContain('1 model at $0') expect(out).toContain('zz-mystery-paid-model-999') expect(out).toContain('codeburn model-alias') + expect(out).toContain('model-flat-rate') + expect(out).not.toContain('Fix: codeburn model-alias') + }) + + it('stays silent for subscription SKUs whose $0 is correct', () => { + const out = renderOverview([makeProject({ + project: 'pass', + projectPath: '/Users/test/pass', + cost: 0, + calls: 4, + model: 'auto-genius', + provider: 'cline-cli', + tokens: { input: 1000, output: 200, cacheR: 0, cacheW: 0 }, + })], { label: 'June 2026', color: false }) + + expect(out).not.toContain('Unpriced') + expect(out).not.toContain('model-alias') }) it('stays silent when every model is priced', () => { diff --git a/tests/workflow-insights.test.ts b/tests/workflow-insights.test.ts index bb18d759f..f7908836a 100644 --- a/tests/workflow-insights.test.ts +++ b/tests/workflow-insights.test.ts @@ -350,6 +350,7 @@ describe('review-findings regressions', () => { expect(isExpectedFreeModel('qwen3.6:35b-a3b-bf16')).toBe(true) expect(isExpectedFreeModel('llama-3-8b-q4')).toBe(true) expect(isExpectedFreeModel('claude-opus-4-8')).toBe(false) + expect(isExpectedFreeModel('auto-genius')).toBe(true) // 95 local calls + 5 unpriced cloud calls: coverage must be 0, not 0.95. expect(computePricingCoverage(5, 5)).toBe(0) })