[divyangchauhan]

Summary

DiffVouch now measures diff chunks in tokens and reports review token usage, making chunk limits and context consumption easier to inspect.

- Default chunk limit: 180,000 bytes
+ Default chunk limit: 50,000 locally counted tokens
+ Configure review.chunk_tokens or --chunk-tokens
+ Report per-chunk text counts, per-request usage, and peak input
- Native review deadline: 10 minutes per chunk
+ Native review deadline: 20 minutes per chunk

Existing chunk_bytes configurations retain byte-based behavior. Whole files and hunks remain the splitting boundaries. Native OpenAI API and ChatGPT subscription responses supply input, cached input, output, and reasoning usage when available; terminal and JSON output distinguish estimates and unavailable data. The original review prompt is unchanged.

Documentation covers configuration, usage interpretation, the decision to retain the original prompt, and the deferred chunk-size experiment.

Evidence

• Before: chunk defaults were byte-based and review output had no token report.
  After: configuration migration tests cover token defaults and legacy byte limits; chunk tests preserve multilingual diffs and enforce token limits; subscription and rendering tests verify usage, missing values, and peak versus cumulative input.
• Validation: go test -mod=readonly -race ./..., go vet -mod=readonly ./..., and release builds for Linux, macOS, and Windows on amd64 and arm64.

Merge Danger

Door: two-way

The change can be reverted without migrating persisted data. Existing explicit byte limits continue to work.

Blast Radius: reviews

The new default can change chunk boundaries and review behavior. Unknown model tokenizers use an explicitly labeled o200k_base estimate. The chunk limit covers the diff only, not the full model context. Longer native reviews can consume more time and subscription allowance. External CLI and Anthropic usage remains unavailable.