diff --git a/README.md b/README.md index c2edb3a..472b719 100644 --- a/README.md +++ b/README.md @@ -97,6 +97,7 @@ put the values under an `env` object. Empty or whitespace-only values are ignore { "env": { "ANTHROPIC_API_KEY": "", + "ANTHROPIC_AUTH_TOKEN": "", "ANTHROPIC_BASE_URL": "", "ANTHROPIC_MODEL": "" } @@ -105,7 +106,8 @@ put the values under an `env` object. Empty or whitespace-only values are ignore | Variable | Required | Default | Description | | --- | --- | --- | --- | -| `ANTHROPIC_API_KEY` | Yes | — | Your Anthropic API key, or the key for a third-party / proxy service. | +| `ANTHROPIC_API_KEY` | One credential required | — | Your Anthropic API key, or an API key accepted by a third-party service. | +| `ANTHROPIC_AUTH_TOKEN` | One credential required | — | A bearer token used instead of `ANTHROPIC_API_KEY`; OpenRouter recommends this mode for its Anthropic-compatible endpoint. | | `ANTHROPIC_BASE_URL` | No | `https://api.anthropic.com` | Point the SDK at a non-official / proxy endpoint. Leave it unset to use the official API — an empty value breaks requests. | | `ANTHROPIC_MODEL` | No | `claude-sonnet-4-6` | Override the model. An empty or whitespace-only value falls back to the default. | diff --git a/README.zh-CN.md b/README.zh-CN.md index 97b431d..a0e590a 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -92,6 +92,7 @@ uvx --from "git+https://github.com/minixalpha/nanoPyCodeAgent@v0.1.0" nanoPyCode { "env": { "ANTHROPIC_API_KEY": "", + "ANTHROPIC_AUTH_TOKEN": "", "ANTHROPIC_BASE_URL": "", "ANTHROPIC_MODEL": "" } @@ -100,7 +101,8 @@ uvx --from "git+https://github.com/minixalpha/nanoPyCodeAgent@v0.1.0" nanoPyCode | 变量 | 是否必填 | 默认值 | 说明 | | --- | --- | --- | --- | -| `ANTHROPIC_API_KEY` | 是 | 无 | 你的 Anthropic API key,或第三方/代理服务的 key。 | +| `ANTHROPIC_API_KEY` | 两种凭据任选其一 | 无 | 你的 Anthropic API key,或第三方服务接受的 API key。 | +| `ANTHROPIC_AUTH_TOKEN` | 两种凭据任选其一 | 无 | 代替 `ANTHROPIC_API_KEY` 使用的 bearer token;OpenRouter 的 Anthropic-compatible endpoint 推荐这种方式。 | | `ANTHROPIC_BASE_URL` | 否 | `https://api.anthropic.com` | 将 SDK 指向非官方/代理 endpoint。使用官方 API 时保持不设置;留空值会导致请求失败。 | | `ANTHROPIC_MODEL` | 否 | `claude-sonnet-4-6` | 覆盖默认模型。空值或纯空白会回退到默认值。 | diff --git a/docs/changelogs/0.8.x.md b/docs/changelogs/0.8.x.md index 08d62d9..b6b223a 100644 --- a/docs/changelogs/0.8.x.md +++ b/docs/changelogs/0.8.x.md @@ -37,6 +37,11 @@ All notable changes in the **0.8.x** release series are documented here. user/model steps, tool observations, token metrics, and terminal state, while preserving text stdout, refusing to overwrite an existing file, and explicitly marking fields shortened by Event Journal persistence limits. +- Provider-reported model costs in ATIF trajectories. Costs are captured + directly from response usage when available or reconciled asynchronously by + generation ID with bounded retries, support both API-key and bearer-token + authentication, and retain per-attempt diagnostics when reconciliation + remains incomplete. ### Fixed - Declare `httpx` as a direct runtime dependency so clean and containerized diff --git a/docs/dev_docs/en/event-journal-protocol-v1.md b/docs/dev_docs/en/event-journal-protocol-v1.md index ce1aef0..d71c0fc 100644 --- a/docs/dev_docs/en/event-journal-protocol-v1.md +++ b/docs/dev_docs/en/event-journal-protocol-v1.md @@ -492,6 +492,14 @@ stable user-output contract. External programs SHOULD NOT treat ## Versioning and compatibility +A terminal `run.completed` or `run.failed` payload may include an optional +`cost_reconciliation` array. Each item contains a `generation_id`, a final +`resolved` or `unresolved` status, and an `attempts` list. Attempts retain only +their ordinal, outcome classification, and, when applicable, an HTTP status or +exception type; credentials and response bodies are never stored. This adds +diagnostics without changing terminal-event semantics and is therefore a +v1-compatible optional payload extension. + A v1 reader fails closed on an unknown `schema_version` or event type. The compatibility rules are: diff --git a/docs/dev_docs/zh-CN/event-journal-protocol-v1.md b/docs/dev_docs/zh-CN/event-journal-protocol-v1.md index 8c963fd..cc726a0 100644 --- a/docs/dev_docs/zh-CN/event-journal-protocol-v1.md +++ b/docs/dev_docs/zh-CN/event-journal-protocol-v1.md @@ -128,6 +128,7 @@ v1 支持九种事件： | `model.started` | 一次模型调用开始。 | | `model.output_delta` | 模型流式产生一段文本。 | | `model.completed` | 一次模型调用成功完成，最终消息和 usage 已可用。 | +| `model.cost_resolved` | provider 的异步账单查询已解析一次模型调用的真实 cost。 | | `tool.started` | 一个工具调用开始。 | | `tool.completed` | 一个工具调用以正常结果、工具级错误或异常结束。 | | `run.completed` | Agent Run 正常结束，包括达到轮次上限。 | @@ -188,6 +189,7 @@ interactive 模式下，每次用户输入建立一个新的 Agent Run 和 Journ | `usage` | object 或 `null` | 本次模型调用的 token usage；schema 见下文。 | | `provider_response_id` | non-empty string 或 `null` | provider 原始 response/message ID。 | | `generation_id` | non-empty string 或 `null` | provider generation ID；当前从 `x-generation-id` response header 读取。 | +| `cost` | object（可选） | 本次调用的真实 cost 状态；schema 见下文。旧版 v1 entry 可以没有该字段。 | | `duration_ms` | non-negative number | 从开始请求到完整消息和响应头可用的耗时。 | `content` 支持以下 block： @@ -209,7 +211,24 @@ interactive 模式下，每次用户输入建立一个新的 Agent Run 和 Journ | `cache_read_input_tokens` | 否 | non-negative integer | 从 prompt cache 读取的输入 token 数。 | | `cache_creation_input_tokens` | 否 | non-negative integer | 写入 prompt cache 的输入 token 数。 | -provider 返回的其他 JSON usage 字段可以原样保留。v1 不根据 usage 或价格目录计算 cost。 +provider 返回的其他 JSON usage 字段可以原样保留。v1 不根据 token usage 或价格目录估算 cost。 + +`cost.status` 为 `resolved`、`pending` 或 `unknown`。响应直接带有真实 cost 时，`resolved` 同时记录 decimal string `amount`、`currency: "USD"`、`source: "provider_response.usage.cost"` 和 `kind: "provider_reported"`；有 generation ID 但响应未带 cost 时记录 `pending`；没有可补账身份时记录 `unknown`。未知 cost 不能写成零。 + +### `model.cost_resolved` + +| 字段 | 类型 | 含义 | +|---|---|---| +| `generation_id` | non-empty string | 与 `model.completed` 关联的 provider generation ID。 | +| `amount` | non-negative finite decimal string | provider 报告的实际金额。 | +| `currency` | non-empty string | 金额币种；OpenRouter 当前为 `USD`。 | +| `source` | non-empty string | 事实来源；generation 查询当前为 `provider_generation.total_cost`。 | +| `model` | non-empty string（可选） | generation 记录返回的实际模型。 | +| `provider_name` | non-empty string（可选） | generation 记录返回的上游 provider。 | + +该事件只追加、不修改先前的 `model.completed`。查询地址由当前 Anthropic SDK base URL 同源派生为 `v1/generation`，不会按 provider 名称把 credential 发送到另一个固定渠道。实现已用 OpenRouter 验证；其他兼容 provider 不提供 generation ID 或该查询接口时保持未知。查询失败不产生该事件，也不改变 run 的成功或失败终态。 + +`run.completed` 与 `run.failed` 可以带可选的 `cost_reconciliation` array。每项包含 `generation_id`、最终 `status`（`resolved` 或 `unresolved`）和 `attempts`；attempt 只保存序号、结果分类，以及适用时的 HTTP 状态码或异常类型，不保存 credential 或响应正文。该字段为现有终态增加诊断信息，不改变终态语义，因此属于 v1 兼容的可选 payload 扩展。 ### `tool.started` @@ -372,7 +391,7 @@ Journal 明确记录： - 本次用户输入； - 完整模型输出、流式文本和工具调用； - 完整工具输入和返回给模型的工具结果； -- provider message/generation ID、stop reason 和 usage； +- provider message/generation ID、stop reason、usage 和真实 cost； - 产生本次运行的程序名称和包版本； - 错误类型、错误消息和各阶段耗时。 @@ -384,7 +403,7 @@ v1 没有专门记录： - 完整 provider request、HTTP header 或 SDK 原始 response； - system prompt 和发给模型的完整历史快照； - spinner、ANSI 颜色、提示符、banner 等 stdout 表现细节； -- token cost 或价格目录解析结果； +- 价格目录或由 token 数估算的 cost； - session 身份、跨 run 父子关系； - ATIF trajectory 或 public `stream-json` 记录。 diff --git a/docs/dev_notes/en/0.8.x.md b/docs/dev_notes/en/0.8.x.md index 009a02b..2f951a7 100644 --- a/docs/dev_notes/en/0.8.x.md +++ b/docs/dev_notes/en/0.8.x.md @@ -210,3 +210,18 @@ uv run --project benchmarks/harbor \ ``` Development acceptance requires the validator to return zero, stdout to retain its existing text format, and the target to contain complete JSON. Harbor adapter collection and a real trial belong to the later Harbor integration acceptance work. + +#### Provider-reported cost collection and reconciliation + +**Goal:** trajectories record the actual model-call charges reported by the provider. The provider response is the source of truth; the Event Journal preserves the direct or deferred resolution state; and ATIF exposes per-step `metrics.cost_usd` plus a complete run total only when every billable call has been resolved. + +There are two collection paths: + +- **Synchronous response accounting.** OpenRouter's [Usage Accounting](https://openrouter.ai/docs/cookbook/administration/usage-accounting) defines a complete `usage` object for OpenAI-compatible Chat Completions and streaming responses. Its `usage.cost` is the total amount charged for the request, delivered in the complete non-streaming response or the final SSE message. When present, nano records it immediately as a resolved provider-reported USD cost. Token usage does not imply that cost is present: OpenRouter's [Anthropic Messages API](https://openrouter.ai/docs/api/api-reference/anthropic-messages/create-messages) preserves an Anthropic-compatible usage schema containing token, cache, service-tier, and speed fields but does not define `cost`. nano currently calls this endpoint through the Anthropic SDK, so a response may contain token counts and an extension such as `speed: "standard"` without a cost. +- **Asynchronous generation reconciliation.** When `usage.cost` is absent but `X-Generation-Id` is available, nano records the generation ID and marks the cost pending. This covers the current OpenRouter Anthropic Messages path and compatible APIs whose usage record becomes visible after the model response. OpenRouter's documented alternative is to retain the generation ID, then call [`GET /api/v1/generation?id=...`](https://openrouter.ai/docs/api/api-reference/generations/get-generation) and read `data.total_cost`. The endpoint documents 404, 429, and 5xx responses, so possession of an ID does not guarantee that its billing record is immediately queryable. + +The lookup endpoint is derived from the configured Anthropic SDK base URL and stays on the same origin. Reconciliation uses six bounded attempts with exponential delays of 1, 2, 4, 8, and 15 seconds, for about 30 seconds of total waiting. A missing record, transient HTTP status, network error, or HTTP 200 response without `total_cost` remains retryable; permanent client and authentication failures stop immediately. A successful lookup appends `model.cost_resolved` without rewriting the earlier `model.completed` entry. + +Reconciliation is observable rather than collapsing every failure into `None`. The terminal Journal event carries an optional `cost_reconciliation` list with the final status for each generation and each attempt classified as resolved, cost unavailable, HTTP error with its status code, request error with its exception type, or unsupported endpoint. Credentials and response bodies are never persisted. The ATIF projector copies the diagnostics into `extra.terminal.cost_reconciliation`. Cost enrichment remains best effort and never changes the task outcome; an unknown amount is never reported as zero. + +**Validation:** unit tests cover direct response extraction, pending and unknown states, successful deferred retry, permanent-error short circuiting, bounded failure, and attempt diagnostics. Agent event tests verify successful reconciliation before the terminal event and persistence of unresolved diagnostics. ATIF tests cover per-step costs, complete and partial totals, and terminal diagnostics. The complete acceptance command is `uv run pytest`. diff --git a/docs/dev_notes/zh-CN/0.8.x.md b/docs/dev_notes/zh-CN/0.8.x.md index a203f19..5caf3ae 100644 --- a/docs/dev_notes/zh-CN/0.8.x.md +++ b/docs/dev_notes/zh-CN/0.8.x.md @@ -166,8 +166,8 @@ journal writer 接收它以后，补上持久化所需的身份、顺序和记 实现不按技术层级拆成七个前后割裂的步骤，而是按可独立使用、可独立验收的能力组织。每项能力同时定义实现与验收，不把“验收”作为脱离具体功能的最后一步： 1. **运行事实与 Event Journal。** 建立 `Native Event` 与 `Journal Entry` 的版本化契约，让 agent loop 在 user、model、tool 与 run 边界只产生事实，并把事实追加写入内部 Event Journal。文本输出也从同一组事实投影，保持用户可见行为不变。 -2. **公开 ATIF trajectory。** 实现 Event Journal 到 ATIF-v1.7 的单向 projector，并用独立的 `--trajectory PATH` 在 headless run 中启用它。`PATH` 指定一份完整 ATIF JSON 快照的文件位置，不指向内部 Journal，也不改变 stdout。projector 映射 Journal 中已定义的 message、tool、timestamp、duration、token/cache usage 与 run 终态；cost 映射依赖独立的真实 cost 契约与补账机制。 -3. **真实 cost 补账。** 有 `usage.cost` 时直接记录；当前 OpenRouter Messages 路径则使用已保存的 `X-Generation-Id`，在 run 收尾时用有界重试查询 generation 的 `total_cost`，再追加 `model.cost_resolved`。查询失败不改变任务结果，projector 将已解析 cost 回填到对应 step 和 `final_metrics`，未知 cost 不写成 `0`。 +2. **公开 ATIF trajectory。** 实现 Event Journal 到 ATIF-v1.7 的单向 projector，并用独立的 `--trajectory PATH` 在 headless run 中启用它。`PATH` 指定一份完整 ATIF JSON 快照的文件位置，不指向内部 Journal，也不改变 stdout。projector 映射 Journal 中已定义的 message、tool、timestamp、duration、token/cache usage 与 run 终态；cost 映射依赖独立的供应商实际成本契约与对账机制。 +3. **供应商实际成本采集与对账。** 优先记录模型响应 `usage.cost` 中的供应商实际费用；响应未携带费用但提供 generation ID 时，先标记为 pending，再在 run 收尾阶段通过供应商的 Generation API 查询 `total_cost`。查询结果以追加事件保存，不修改既有 Journal Entry；查询失败不改变任务结果。projector 将已知费用映射到对应 step 和 run 汇总，并明确表示费用是否完整，绝不把未知费用记成零。 4. **Harbor 接入与端到端验收。** Harbor adapter 只负责传入 trajectory 路径、声明并读取 agent 生成的 ATIF，再回填 steps、tokens 和 cost，不再做 native trajectory 转换。先用 Harbor 契约测试验证采集与统计，再跑一次真实 trial 确认完整链路。 `--output-format` 与 `stream-json` 属于独立的 run output，不在上述 trajectory 能力中。 @@ -191,7 +191,7 @@ uv run pytest \ #### 公开 ATIF trajectory -**要做什么：**实现从 Event Journal 到 ATIF-v1.7 的单向 projector，把一次 headless Agent Run 输出为一份完整 ATIF JSON。CLI 增加独立的 `--trajectory PATH`：参数出现时启用 projector，`PATH` 指定文件位置，stdout 保持现有文本行为。内部 Event Journal 不对外暴露；真实 cost 补账、Harbor adapter 采集、`stream-json` 与交互会话的多 run trajectory 不属于这项能力。 +**要做什么：**实现从 Event Journal 到 ATIF-v1.7 的单向 projector，把一次 headless Agent Run 输出为一份完整 ATIF JSON。CLI 增加独立的 `--trajectory PATH`：参数出现时启用 projector，`PATH` 指定文件位置，stdout 保持现有文本行为。内部 Event Journal 不对外暴露；供应商实际成本采集与对账、Harbor adapter 采集、`stream-json` 与交互会话的多 run trajectory 不属于这项能力。 **协议依据：**目标格式以 Harbor 0.21.0 中的 ATIF-v1.7 为准： @@ -210,3 +210,34 @@ uv run --project benchmarks/harbor \ ``` 开发验收以 validator 返回 0、stdout 仍为原有文本输出、目标文件为完整 JSON 为准。Harbor adapter 读取与真实 trial 属于后续的 Harbor 接入验收。 + +#### 供应商实际成本采集与对账 + +**要做什么：**trajectory 记录供应商实际报告的模型调用费用。这里涉及三层数据，名称相似但用途不同： + +- **供应商响应**是成本的事实来源。`usage.cost` 指模型响应对象中 `usage` 对象的 `cost` 字段；`X-Generation-Id` 指模型响应 HTTP header 中的 generation ID；`data.total_cost` 指 Generation API 响应体中 `data` 对象的总费用字段。 +- **Event Journal**保存 Agent 采集到的运行事实。`model.completed` 是一次模型调用完成时产生的事件类型,该事件的 `payload.cost` 字段保存本次调用的成本状态。`model.cost_resolved` 是延迟查询成功后追加的另一种事件类型，其 `payload` 保存 generation ID、金额、币种和来源。 +- **ATIF trajectory**是从 Event Journal 投影出的公开结果。模型 step 的 `metrics.cost_usd` 表示单次调用费用，trajectory 顶层的 `final_metrics` 保存整次 run 的费用汇总及完整性状态。 + +费用有两条采集路径： + +- **响应内同步采集。** OpenRouter 的 [Usage Accounting](https://openrouter.ai/docs/cookbook/administration/usage-accounting) 对 OpenAI-compatible Chat Completions/streaming 响应定义了完整 `usage`，其中 `usage.cost` 是本次请求向账户收取的总额；非流式响应在完整响应中提供，流式响应在最后一个 SSE message 中提供。收到这个字段时，立即把它作为供应商报告的 USD 费用写入 `model.completed.payload.cost`，状态为 `resolved`。不能因为某个响应有 token usage 就推断它一定有 cost：OpenRouter 的 [Anthropic Messages API](https://openrouter.ai/docs/api/api-reference/anthropic-messages/create-messages) 为保持 Anthropic-compatible schema，其 `usage` 定义 token、cache、service tier、speed 等字段，但没有定义 `cost`；本项目当前通过 Anthropic SDK 调用该接口，所以实测 `usage` 可能只有 token 与 `speed: "standard"` 等扩展字段。 +- **按 generation ID 异步补账。** 响应没有 `usage.cost`、但 HTTP header 包含 `X-Generation-Id` 时，把 generation ID 写入 `model.completed.payload.generation_id`，并把 `payload.cost.status` 标记为 `pending`。这覆盖当前 OpenRouter Anthropic Messages 路径，也覆盖其他只先返回生成身份、稍后才完成 usage record 的兼容接口。OpenRouter 官方给出的替代方案是保存 generation ID，随后调用 [`GET /api/v1/generation?id=...`](https://openrouter.ai/docs/api/api-reference/generations/get-generation)，读取 `data.total_cost`；该接口文档明确列出 `404`、`429` 与 5xx，因此 generation ID 已存在不等于账单记录已在查询接口中可见。 + +对账请求从当前 Anthropic SDK 的 base URL 同源派生 `v1/generation` endpoint，不会按 hostname 绑定特定供应商，也不会把 API credential 发送到硬编码渠道；目前使用 OpenRouter 验证这套扩展协议。查询使用 6 次有界尝试和 `1、2、4、8、15` 秒指数退避，总等待上限约 30 秒：`404`、`408`、`409`、`429` 与常见 5xx、网络错误、HTTP 200 但尚无 `total_cost` 都可重试；`400`、`401`、`402`、`403` 等永久失败立即停止。成功后向 Event Journal 追加一条 `model.cost_resolved` 事件：金额写在 `payload.amount`，关联键写在 `payload.generation_id`，币种和来源分别写在 `payload.currency` 与 `payload.source`。实现不会回写已经持久化的 `model.completed` 事件，因此 Event Journal 仍保持 append-only。 + +对账的可观测性不能依赖 `None`。每次 run 的终态事件额外保存可选的 `cost_reconciliation` 列表，逐个 generation 记录最终 `resolved`/`unresolved` 状态，以及每次尝试的结果：`resolved`、HTTP 200 但 `cost_unavailable`、带状态码的 `http_error`、带异常类型的 `request_error`，或 base URL 无法导出查询端点时的 `unsupported_endpoint`。不保存 credential 或供应商响应正文。ATIF projector 将同一列表复制到 `extra.terminal.cost_reconciliation`，因此既可从内部 Journal 排障，也可从公开 trajectory 判断费用为何不完整。 + +成本采集属于 best-effort enrichment：HTTP 错误、账单记录尚未生成或响应无效，都不会改变 Agent 原任务的成功或失败结果。ATIF projector 通过两个事件的 `payload.generation_id` 将延迟对账结果关联回对应模型 step，并输出： + +- 单次模型调用费用写入该 ATIF step 的 `metrics.cost_usd`；成本来源和 generation ID 分别写入 `metrics.extra.cost_source` 与 `metrics.extra.generation_id`。 +- 所有应计费调用费用完整时，总费用写入 trajectory 顶层的 `final_metrics.total_cost_usd`。 +- 费用不完整时，已知费用小计和不完整标记分别写入 `final_metrics.extra.known_cost_usd` 与 `final_metrics.extra.cost_is_partial`；如果存在可识别的待对账调用，其 generation ID 列表写入 `final_metrics.extra.missing_generation_ids`。 + +未知费用始终保持未知，不会被写成零。 + +**自动化验收：**成本单元测试覆盖响应内直接采集、pending/unknown 状态、Generation API 的成功重试、永久 HTTP 错误短路、有界失败与逐次诊断；Agent 事件测试验证对账发生在 run 终态前且失败诊断被持久化；ATIF 测试验证 step 费用、完整汇总、部分汇总与终态诊断的映射。完整测试套件： + +```bash +uv run pytest +``` diff --git a/src/nanopycodeagent/agent.py b/src/nanopycodeagent/agent.py index 76c75ed..4de0df9 100644 --- a/src/nanopycodeagent/agent.py +++ b/src/nanopycodeagent/agent.py @@ -42,6 +42,11 @@ from .atif import project_atif, write_atif from .bash_tool import BASH_TOOL, run_bash +from .cost import ( + pending_cost, + resolve_generation_cost, + usage_cost, +) from .edit_tool import EDIT_TOOL, edit_preview, run_edit from .event_journal import ( EventEmitter, @@ -394,6 +399,7 @@ def _run_exchange( max_turns=max_turns, ) except BaseException as exc: + cost_reconciliation = _reconcile_costs(client, journal, emitter) emitter.emit( "run.failed", { @@ -401,17 +407,28 @@ def _run_exchange( "message": str(exc), "duration_ms": (time.perf_counter_ns() - run_started_ns) / 1_000_000, + **( + {"cost_reconciliation": cost_reconciliation} + if cost_reconciliation + else {} + ), "source_timestamp": utc_now(), }, ) raise else: + cost_reconciliation = _reconcile_costs(client, journal, emitter) emitter.emit( "run.completed", { "outcome": "completed" if finished else "max_turns_exhausted", "duration_ms": (time.perf_counter_ns() - run_started_ns) / 1_000_000, + **( + {"cost_reconciliation": cost_reconciliation} + if cost_reconciliation + else {} + ), "source_timestamp": utc_now(), }, ) @@ -492,6 +509,8 @@ def _run_model_loop( str(provider_response_id) if provider_response_id is not None else None ), "generation_id": generation_id, + "cost": usage_cost(usage if isinstance(usage, dict) else None) + or pending_cost(generation_id), "duration_ms": (model_completed_ns - model_started_ns) / 1_000_000, "source_timestamp": utc_now(), } @@ -515,6 +534,55 @@ def _run_model_loop( messages.append({"role": "user", "content": results}) +def _reconcile_costs( + client: anthropic.Anthropic, + journal: EventJournal, + emitter: EventEmitter, +) -> list[JsonObject]: + """Append resolved OpenRouter costs without affecting the run outcome.""" + base_url = getattr(client, "base_url", "") + credential = client.api_key or client.auth_token + if not isinstance(credential, str) or not credential: + return [] + outcomes: list[JsonObject] = [] + entries = EventJournal.replay(journal.path) + already_resolved = { + str(entry.payload["generation_id"]) + for entry in entries + if entry.type == "model.cost_resolved" + } + for entry in entries: + if entry.type != "model.completed": + continue + generation_id = entry.payload.get("generation_id") + cost = entry.payload.get("cost") + if ( + not isinstance(generation_id, str) + or generation_id in already_resolved + or not isinstance(cost, dict) + or cost.get("status") != "pending" + ): + continue + diagnostics: list[JsonObject] = [] + resolved = resolve_generation_cost( + base_url, + generation_id, + credential, + diagnostics=diagnostics, + ) + if resolved is not None: + resolved["source_timestamp"] = utc_now() + emitter.emit("model.cost_resolved", resolved) + outcomes.append( + { + "generation_id": generation_id, + "status": "resolved" if resolved is not None else "unresolved", + "attempts": diagnostics, + } + ) + return outcomes + + def run() -> int: """Start the read → ask → answer loop until the user types ``/exit``. diff --git a/src/nanopycodeagent/atif.py b/src/nanopycodeagent/atif.py index 869a6eb..0396bbb 100644 --- a/src/nanopycodeagent/atif.py +++ b/src/nanopycodeagent/atif.py @@ -6,6 +6,7 @@ import os import tempfile from collections.abc import Sequence +from decimal import Decimal from pathlib import Path from .event_journal import JsonObject, JsonValue, JournalEntry, SCHEMA_VERSION @@ -254,6 +255,12 @@ def project_atif(entries: Sequence[JournalEntry]) -> JsonObject: for entry in entries if entry.type == "tool.completed" } + resolved_costs = { + str(entry.payload["generation_id"]): entry.payload + for entry in entries + if entry.type == "model.cost_resolved" + } + cost_states: list[tuple[str | None, Decimal | None]] = [] terminal: JournalEntry | None = None for entry in entries[1:]: payload = entry.payload @@ -298,6 +305,33 @@ def project_atif(entries: Sequence[JournalEntry]) -> JsonObject: "extra": _step_extra(entry, model_starts.get(model_call_id)), } metrics = _metrics(usage) + generation_id = payload["generation_id"] + assert generation_id is None or isinstance(generation_id, str) + cost = payload.get("cost") + resolved_cost = ( + resolved_costs.get(generation_id) + if generation_id is not None + else None + ) + if ( + resolved_cost is None + and isinstance(cost, dict) + and cost.get("status") == "resolved" + ): + resolved_cost = cost + amount: Decimal | None = None + if isinstance(resolved_cost, dict): + amount = Decimal(str(resolved_cost["amount"])) + if metrics is None: + metrics = {} + metrics["cost_usd"] = float(amount) + metrics_extra = metrics.setdefault("extra", {}) + assert isinstance(metrics_extra, dict) + metrics_extra["cost_source"] = resolved_cost["source"] + if generation_id is not None: + metrics_extra["generation_id"] = generation_id + if isinstance(cost, dict) or resolved_cost is not None: + cost_states.append((generation_id, amount)) if metrics is not None: step["metrics"] = metrics tool_calls, observation = _tool_calls_and_observation( @@ -367,6 +401,9 @@ def project_atif(entries: Sequence[JournalEntry]) -> JsonObject: else: terminal_data["error_type"] = terminal_payload["error_type"] terminal_data["message"] = terminal_payload["message"] + cost_reconciliation = terminal_payload.get("cost_reconciliation") + if isinstance(cost_reconciliation, list): + terminal_data["cost_reconciliation"] = cost_reconciliation _add_journal_truncation(terminal_data, terminal) trajectory: JsonObject = { @@ -407,6 +444,25 @@ def project_atif(entries: Sequence[JournalEntry]) -> JsonObject: ) elif llm_steps: final_metrics["extra"] = {"usage_complete": False} + if cost_states: + known_cost = sum( + (amount for _, amount in cost_states if amount is not None), + Decimal(0), + ) + if all(amount is not None for _, amount in cost_states): + final_metrics["total_cost_usd"] = float(known_cost) + else: + final_extra = final_metrics.setdefault("extra", {}) + assert isinstance(final_extra, dict) + final_extra["known_cost_usd"] = float(known_cost) + final_extra["cost_is_partial"] = True + missing = [ + generation_id + for generation_id, amount in cost_states + if amount is None and generation_id is not None + ] + if missing: + final_extra["missing_generation_ids"] = missing return trajectory diff --git a/src/nanopycodeagent/cost.py b/src/nanopycodeagent/cost.py new file mode 100644 index 0000000..ea54ffb --- /dev/null +++ b/src/nanopycodeagent/cost.py @@ -0,0 +1,151 @@ +"""Resolve provider-reported costs for Anthropic-compatible model calls.""" + +from __future__ import annotations + +import time +from decimal import Decimal, InvalidOperation +from typing import Callable +from urllib.parse import urlsplit, urlunsplit + +import httpx + +from .event_journal import JsonObject + +DEFAULT_ATTEMPTS = 6 +DEFAULT_RETRY_DELAYS = (1.0, 2.0, 4.0, 8.0, 15.0) +RETRYABLE_HTTP_STATUSES = frozenset({404, 408, 409, 429, 500, 502, 503, 504}) + + +def generation_url(base_url: object) -> str | None: + """Build the provider-local generation endpoint from an SDK base URL.""" + try: + parsed = urlsplit(str(base_url)) + except ValueError: + return None + if parsed.scheme not in {"http", "https"} or not parsed.hostname: + return None + path = f"{parsed.path.rstrip('/')}/v1/generation" + return urlunsplit((parsed.scheme, parsed.netloc, path, "", "")) + + +def usage_cost(usage: JsonObject | None) -> JsonObject | None: + """Return a resolved cost when the provider included one in usage.""" + if usage is None or "cost" not in usage: + return None + try: + amount = Decimal(str(usage["cost"])) + except (InvalidOperation, ValueError): + return None + if not amount.is_finite() or amount < 0: + return None + return { + "status": "resolved", + "amount": str(amount), + "currency": "USD", + "source": "provider_response.usage.cost", + "kind": "provider_reported", + } + + +def pending_cost(generation_id: str | None) -> JsonObject: + """Describe whether a model call can be reconciled later.""" + if generation_id is None: + return {"status": "unknown"} + return {"status": "pending", "source": "provider_generation"} + + +def resolve_generation_cost( + base_url: object, + generation_id: str, + api_key: str, + *, + attempts: int = DEFAULT_ATTEMPTS, + request: Callable[..., httpx.Response] = httpx.get, + sleep: Callable[[float], None] = time.sleep, + diagnostics: list[JsonObject] | None = None, +) -> JsonObject | None: + """Query the configured provider's generation endpoint with bounded retry. + + Cost enrichment is best effort: transport errors, incomplete asynchronous + records, and malformed responses all remain unknown to the caller. + """ + endpoint = generation_url(base_url) + if endpoint is None: + if diagnostics is not None: + diagnostics.append({"attempt": 0, "status": "unsupported_endpoint"}) + return None + for attempt in range(attempts): + try: + response = request( + endpoint, + params={"id": generation_id}, + headers={"Authorization": f"Bearer {api_key}"}, + timeout=10.0, + ) + if response.status_code >= 400: + if diagnostics is not None: + diagnostics.append( + { + "attempt": attempt + 1, + "status": "http_error", + "http_status": response.status_code, + } + ) + if response.status_code not in RETRYABLE_HTTP_STATUSES: + return None + raise httpx.HTTPStatusError( + "retryable generation lookup response", + request=response.request, + response=response, + ) + body = response.json() + data = body.get("data") if isinstance(body, dict) else None + if isinstance(data, dict) and data.get("total_cost") is not None: + amount = Decimal(str(data["total_cost"])) + if amount.is_finite() and amount >= 0: + result: JsonObject = { + "generation_id": generation_id, + "amount": str(amount), + "currency": "USD", + "source": "provider_generation.total_cost", + } + for source, target in ( + ("model", "model"), + ("provider_name", "provider_name"), + ): + value = data.get(source) + if isinstance(value, str) and value: + result[target] = value + if diagnostics is not None: + diagnostics.append( + { + "attempt": attempt + 1, + "status": "resolved", + "http_status": response.status_code, + } + ) + return result + if diagnostics is not None: + diagnostics.append( + { + "attempt": attempt + 1, + "status": "cost_unavailable", + "http_status": response.status_code, + } + ) + except httpx.HTTPStatusError: + # Retryable HTTP failures were recorded before raising. + pass + except Exception as exc: + # Enrichment must never replace the agent's real task outcome. + if diagnostics is not None: + diagnostics.append( + { + "attempt": attempt + 1, + "status": "request_error", + "error_type": type(exc).__name__, + } + ) + if attempt < attempts - 1: + sleep(DEFAULT_RETRY_DELAYS[min(attempt, len(DEFAULT_RETRY_DELAYS) - 1)]) + return None diff --git a/src/nanopycodeagent/event_journal.py b/src/nanopycodeagent/event_journal.py index 08947b6..5957819 100644 --- a/src/nanopycodeagent/event_journal.py +++ b/src/nanopycodeagent/event_journal.py @@ -13,6 +13,7 @@ import threading from dataclasses import dataclass from datetime import UTC, datetime +from decimal import Decimal, InvalidOperation from pathlib import Path from typing import Callable, Mapping @@ -28,6 +29,7 @@ "model.started", "model.output_delta", "model.completed", + "model.cost_resolved", "tool.started", "tool.completed", "run.completed", @@ -83,6 +85,9 @@ "source_timestamp", } ), + "model.cost_resolved": frozenset( + {"generation_id", "amount", "currency", "source", "source_timestamp"} + ), "tool.started": frozenset( {"tool_call_id", "tool_name", "input", "source_timestamp"} ), @@ -120,6 +125,41 @@ def _validate_rfc3339_utc(value: str, field: str) -> None: raise ValueError(f"{field} must be RFC 3339 UTC") from exc +def _validate_cost_reconciliation(payload: JsonObject, event_type: str) -> None: + outcomes = payload.get("cost_reconciliation") + if outcomes is None: + return + if not isinstance(outcomes, list): + raise ValueError(f"{event_type}.cost_reconciliation must be a list") + for index, outcome in enumerate(outcomes): + field = f"{event_type}.cost_reconciliation[{index}]" + if not isinstance(outcome, dict): + raise ValueError(f"{field} must be an object") + generation_id = outcome.get("generation_id") + if not isinstance(generation_id, str) or not generation_id: + raise ValueError(f"{field}.generation_id must be a string") + if outcome.get("status") not in {"resolved", "unresolved"}: + raise ValueError(f"{field}.status is unsupported") + attempts = outcome.get("attempts") + if not isinstance(attempts, list): + raise ValueError(f"{field}.attempts must be a list") + for attempt_index, attempt in enumerate(attempts): + attempt_field = f"{field}.attempts[{attempt_index}]" + if not isinstance(attempt, dict): + raise ValueError(f"{attempt_field} must be an object") + number = attempt.get("attempt") + if not isinstance(number, int) or isinstance(number, bool) or number < 0: + raise ValueError(f"{attempt_field}.attempt must be non-negative") + if attempt.get("status") not in { + "resolved", + "cost_unavailable", + "http_error", + "request_error", + "unsupported_endpoint", + }: + raise ValueError(f"{attempt_field}.status is unsupported") + + def _validate_recorded_at(value: str) -> None: _validate_rfc3339_utc(value, "Journal Entry recorded_at") @@ -307,6 +347,53 @@ def _validate_native_payload(event_type: str, payload: JsonObject) -> None: value = payload[field] if value is not None and (not isinstance(value, str) or not value): raise ValueError(f"model.completed.{field} must be a string or null") + cost = payload.get("cost") + if cost is not None: + if not isinstance(cost, dict): + raise ValueError("model.completed.cost must be an object") + status = cost.get("status") + if status not in {"resolved", "pending", "unknown"}: + raise ValueError("model.completed.cost.status is unsupported") + if status == "resolved": + for field in ("amount", "currency", "source", "kind"): + value = cost.get(field) + if not isinstance(value, str) or not value: + raise ValueError( + f"model.completed.cost.{field} must be a string" + ) + try: + amount = Decimal(str(cost["amount"])) + except InvalidOperation as exc: + raise ValueError( + "model.completed.cost.amount must be decimal" + ) from exc + if not amount.is_finite() or amount < 0: + raise ValueError( + "model.completed.cost.amount must be non-negative" + ) + elif status == "pending": + source = cost.get("source") + if not isinstance(source, str) or not source: + raise ValueError( + "model.completed.cost.source must be a string" + ) + elif event_type == "model.cost_resolved": + _require_string(payload, "generation_id", event_type) + _require_string(payload, "amount", event_type) + _require_string(payload, "currency", event_type) + _require_string(payload, "source", event_type) + try: + amount = Decimal(str(payload["amount"])) + except InvalidOperation as exc: + raise ValueError("model.cost_resolved.amount must be decimal") from exc + if not amount.is_finite() or amount < 0: + raise ValueError("model.cost_resolved.amount must be non-negative") + for field in ("model", "provider_name"): + value = payload.get(field) + if value is not None and (not isinstance(value, str) or not value): + raise ValueError( + f"model.cost_resolved.{field} must be a string" + ) elif event_type == "tool.started": if not isinstance(payload["input"], dict): raise ValueError("tool.started.input must be an object") @@ -325,10 +412,12 @@ def _validate_native_payload(event_type: str, payload: JsonObject) -> None: elif event_type == "run.completed": if payload["outcome"] not in {"completed", "max_turns_exhausted"}: raise ValueError("run.completed.outcome is unsupported") + _validate_cost_reconciliation(payload, event_type) elif event_type == "run.failed": _require_string(payload, "error_type", event_type) if not isinstance(payload["message"], str): raise ValueError("run.failed.message must be a string") + _validate_cost_reconciliation(payload, event_type) @dataclass(frozen=True, slots=True) class NativeEvent: diff --git a/tests/helpers.py b/tests/helpers.py index cae014d..31572a4 100644 --- a/tests/helpers.py +++ b/tests/helpers.py @@ -121,10 +121,18 @@ def stream(self, **kwargs): class FakeClient: - def __init__(self, messages, *, api_key="sk-test", auth_token=None): + def __init__( + self, + messages, + *, + api_key="sk-test", + auth_token=None, + base_url="https://api.anthropic.com", + ): self.messages = messages self.api_key = api_key self.auth_token = auth_token + self.base_url = base_url def patch_client(monkeypatch, client): diff --git a/tests/test_agent_events.py b/tests/test_agent_events.py index 750fcdb..bdfb866 100644 --- a/tests/test_agent_events.py +++ b/tests/test_agent_events.py @@ -42,6 +42,11 @@ def test_headless_model_reply_is_journaled_without_changing_stdout( response_headers={"x-generation-id": "gen-1"}, ) patch_client(monkeypatch, FakeClient(FakeMessages([reply]))) + monkeypatch.setattr( + agent, + "resolve_generation_cost", + lambda *args, **kwargs: None, + ) assert agent.run_headless("fix it") == 0 @@ -116,6 +121,113 @@ def normalize_after_model_timing(value): assert completed.payload["duration_ms"] == 1 +@pytest.mark.parametrize( + ("api_key", "auth_token", "expected_credential"), + [ + ("sk-api", None, "sk-api"), + (None, "sk-auth", "sk-auth"), + ], +) +def test_openrouter_cost_is_reconciled_before_run_completion( + monkeypatch, api_key, auth_token, expected_credential +): + reply = FakeStream( + [text_block("done")], + usage=SimpleNamespace(input_tokens=10, output_tokens=2), + response_headers={"x-generation-id": "gen-cost-1"}, + ) + client = FakeClient( + FakeMessages([reply]), + base_url="https://openrouter.ai/api/v1", + api_key=api_key, + auth_token=auth_token, + ) + patch_client(monkeypatch, client) + credentials = [] + + def resolved(base_url, generation_id, credential, **kwargs): + credentials.append(credential) + return { + "generation_id": generation_id, + "amount": "0.00072", + "currency": "USD", + "source": "provider_generation.total_cost", + } + + monkeypatch.setattr( + agent, + "resolve_generation_cost", + resolved, + ) + + assert agent.run_headless("fix it") == 0 + + entries = EventJournal.replay(_only_journal_path()) + assert [entry.type for entry in entries[-3:]] == [ + "model.completed", + "model.cost_resolved", + "run.completed", + ] + assert entries[-3].payload["cost"] == { + "status": "pending", + "source": "provider_generation", + } + assert entries[-2].payload["amount"] == "0.00072" + assert credentials == [expected_credential] + assert entries[-1].payload["cost_reconciliation"] == [ + { + "generation_id": "gen-cost-1", + "status": "resolved", + "attempts": [], + } + ] + + +def test_unresolved_cost_diagnostics_are_persisted_on_run_terminal(monkeypatch): + reply = FakeStream( + [text_block("done")], + usage=SimpleNamespace(input_tokens=10, output_tokens=2), + response_headers={"x-generation-id": "gen-cost-1"}, + ) + patch_client( + monkeypatch, + FakeClient(FakeMessages([reply]), base_url="https://openrouter.ai/api"), + ) + + def unresolved(base_url, generation_id, api_key, *, diagnostics): + diagnostics.extend( + [ + {"attempt": 1, "status": "http_error", "http_status": 404}, + { + "attempt": 2, + "status": "request_error", + "error_type": "ReadTimeout", + }, + ] + ) + return None + + monkeypatch.setattr(agent, "resolve_generation_cost", unresolved) + + assert agent.run_headless("fix it") == 0 + + terminal = EventJournal.replay(_only_journal_path())[-1] + assert terminal.payload["cost_reconciliation"] == [ + { + "generation_id": "gen-cost-1", + "status": "unresolved", + "attempts": [ + {"attempt": 1, "status": "http_error", "http_status": 404}, + { + "attempt": 2, + "status": "request_error", + "error_type": "ReadTimeout", + }, + ], + } + ] + + def test_failed_tool_events_project_the_existing_tool_output( monkeypatch, capsys, tmp_path ): diff --git a/tests/test_atif.py b/tests/test_atif.py index 83d7abb..25bbdcc 100644 --- a/tests/test_atif.py +++ b/tests/test_atif.py @@ -178,6 +178,74 @@ def test_completed_journal_projects_atif_user_model_usage_and_terminal_state( } +def test_resolved_and_missing_costs_project_as_partial_metrics(tmp_path): + entries = _journal_entries(tmp_path) + completed = next(entry for entry in entries if entry.type == "model.completed") + completed.payload["cost"] = { + "status": "pending", + "source": "provider_generation", + } + terminal = entries.pop() + entries.append( + JournalEntry( + schema_version=1, + run_id=completed.run_id, + seq=terminal.seq, + recorded_at=terminal.recorded_at, + type="model.cost_resolved", + payload={ + "generation_id": "generation-1", + "amount": "0.00072", + "currency": "USD", + "source": "provider_generation.total_cost", + "source_timestamp": terminal.payload["source_timestamp"], + }, + ) + ) + entries.append( + JournalEntry( + schema_version=1, + run_id=terminal.run_id, + seq=terminal.seq + 1, + recorded_at=terminal.recorded_at, + type=terminal.type, + payload=terminal.payload, + ) + ) + + trajectory = project_atif(entries) + assert trajectory["steps"][1]["metrics"]["cost_usd"] == 0.00072 + assert trajectory["steps"][1]["metrics"]["extra"] == { + "cache_creation_input_tokens": 2, + "cost_source": "provider_generation.total_cost", + "generation_id": "generation-1", + } + assert trajectory["final_metrics"]["total_cost_usd"] == 0.00072 + + +def test_cost_reconciliation_diagnostics_project_to_terminal_extra(tmp_path): + entries = _journal_entries(tmp_path) + entries[-1].payload["cost_reconciliation"] = [ + { + "generation_id": "generation-1", + "status": "unresolved", + "attempts": [ + {"attempt": 1, "status": "http_error", "http_status": 404} + ], + } + ] + + assert project_atif(entries)["extra"]["terminal"]["cost_reconciliation"] == [ + { + "generation_id": "generation-1", + "status": "unresolved", + "attempts": [ + {"attempt": 1, "status": "http_error", "http_status": 404} + ], + } + ] + + def test_tool_lifecycle_is_folded_into_the_originating_agent_step(tmp_path): recorded_at = iter( [f"2026-08-26T09:00:00.00{index}Z" for index in range(1, 8)] diff --git a/tests/test_cli.py b/tests/test_cli.py index 016ca4e..db5df0c 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -145,7 +145,11 @@ def test_partial_model_usage_is_not_reported_as_complete_trajectory_totals( trajectory = json.loads(trajectory_path.read_text(encoding="utf-8")) assert trajectory["final_metrics"] == { "total_steps": 3, - "extra": {"usage_complete": False}, + "extra": { + "usage_complete": False, + "known_cost_usd": 0.0, + "cost_is_partial": True, + }, } diff --git a/tests/test_cost.py b/tests/test_cost.py new file mode 100644 index 0000000..d827378 --- /dev/null +++ b/tests/test_cost.py @@ -0,0 +1,154 @@ +"""Provider-reported cost extraction and OpenRouter reconciliation tests.""" + +import httpx +import pytest + +from nanopycodeagent.cost import ( + generation_url, + pending_cost, + resolve_generation_cost, + usage_cost, +) + + +@pytest.mark.parametrize( + ("base_url", "expected"), + [ + ("https://openrouter.ai/api", "https://openrouter.ai/api/v1/generation"), + ("https://provider.example/api", "https://provider.example/api/v1/generation"), + ("https://provider.example", "https://provider.example/v1/generation"), + ("not a URL", None), + ], +) +def test_generation_url_stays_on_the_configured_provider(base_url, expected): + assert generation_url(base_url) == expected + + +def test_usage_cost_preserves_provider_reported_decimal(): + assert usage_cost({"input_tokens": 1, "output_tokens": 2, "cost": "0.00072"}) == { + "status": "resolved", + "amount": "0.00072", + "currency": "USD", + "source": "provider_response.usage.cost", + "kind": "provider_reported", + } + assert pending_cost(None) == {"status": "unknown"} + assert pending_cost("gen-1") == { + "status": "pending", + "source": "provider_generation", + } + + +def test_generation_resolution_retries_until_cost_is_available(): + responses = iter( + [ + httpx.Response(404, request=httpx.Request("GET", "https://provider.example/api/v1/generation")), + httpx.Response( + 200, + request=httpx.Request("GET", "https://provider.example/api/v1/generation"), + json={ + "data": { + "total_cost": "0.00125", + "model": "anthropic/claude-sonnet-4", + "provider_name": "Anthropic", + } + }, + ), + ] + ) + calls = [] + sleeps = [] + diagnostics = [] + + def request(url, **kwargs): + calls.append((url, kwargs)) + return next(responses) + + assert resolve_generation_cost( + "https://provider.example/api", + "gen-1", "secret", request=request, sleep=sleeps.append, + diagnostics=diagnostics, + ) == { + "generation_id": "gen-1", + "amount": "0.00125", + "currency": "USD", + "source": "provider_generation.total_cost", + "model": "anthropic/claude-sonnet-4", + "provider_name": "Anthropic", + } + assert len(calls) == 2 + assert calls[0][1]["headers"] == {"Authorization": "Bearer secret"} + assert calls[0][1]["params"] == {"id": "gen-1"} + assert sleeps == [1.0] + assert diagnostics == [ + {"attempt": 1, "status": "http_error", "http_status": 404}, + {"attempt": 2, "status": "resolved", "http_status": 200}, + ] + + +def test_generation_resolution_failure_is_unknown_after_bounded_attempts(): + calls = [] + + def request(*args, **kwargs): + calls.append(None) + raise httpx.ReadError("not ready") + + assert resolve_generation_cost( + "https://provider.example/api", + "gen-1", "secret", request=request, sleep=lambda _: None + ) is None + assert len(calls) == 6 + + +def test_generation_resolution_does_not_retry_authentication_failure(): + diagnostics = [] + response = httpx.Response( + 401, + request=httpx.Request("GET", "https://provider.example/api/v1/generation"), + ) + + assert resolve_generation_cost( + "https://provider.example/api", + "gen-1", + "secret", + request=lambda *args, **kwargs: response, + sleep=lambda _: pytest.fail("must not retry a permanent failure"), + diagnostics=diagnostics, + ) is None + assert diagnostics == [ + {"attempt": 1, "status": "http_error", "http_status": 401} + ] + + +def test_generation_resolution_records_missing_cost_and_request_errors(): + responses = iter( + [ + httpx.Response( + 200, + request=httpx.Request("GET", "https://provider.example/api/v1/generation"), + json={"data": {}}, + ), + httpx.ReadTimeout("not ready"), + ] + ) + diagnostics = [] + + def request(*args, **kwargs): + result = next(responses) + if isinstance(result, Exception): + raise result + return result + + assert resolve_generation_cost( + "https://provider.example/api", + "gen-1", + "secret", + attempts=2, + request=request, + sleep=lambda _: None, + diagnostics=diagnostics, + ) is None + assert diagnostics == [ + {"attempt": 1, "status": "cost_unavailable", "http_status": 200}, + {"attempt": 2, "status": "request_error", "error_type": "ReadTimeout"}, + ]