fix(runtime): classify provider capacity errors - #3365
Conversation
Astro-Han
left a comment
There was a problem hiding this comment.
Thanks for this — classifying on the provider's structured code/type rather than on message text is the right foundation, and matching the existing CONTEXT_OVERFLOW_PROVIDER_CODES shape means this slots in without inventing a new mechanism. The copy and recovery-hint plumbing through to the desktop is complete and the tests assert the classification and the retry metadata rather than restating the implementation.
No [P0]/[P1]. One [P2] that is really a question about the problem definition, and two [P3]s.
The [P2]: resource-exhausted / resource_exhausted is not a single meaning across providers. In gRPC and the Google API error model, RESOURCE_EXHAUSTED (status 8) is the standard code for quota exhaustion — per-minute, per-day, or per-project — not for "the server is busy right now". The user-facing copy this PR routes it to is 模型服务暂时满载,请等待几分钟或切换模型后重试 / Wait a few minutes or switch models, and that advice is actively wrong for a daily quota: waiting a few minutes will not help, and the correct action is different.
The 429 branch above catches the common case, since a Gemini quota error usually carries HTTP 429 and returns RateLimit before reaching this check. So this is not a blanket misclassification. But that also means the capacity branch is reached precisely when the status code is absent or non-429 — which is the case where the code alone is carrying the whole meaning, and where it is most ambiguous.
Could you say which providers you observed emitting these two codes, and with which meaning? If the set is narrow and "server at capacity" is what they actually mean, then pinning that in the comment next to PROVIDER_CAPACITY_CODES resolves it and I have no further concern. If it turns out the same code also arrives for quota exhaustion, the classification is fine but the recovery copy needs to not promise that waiting works.
I want to be explicit that I am asking rather than asserting: I have not seen your traces, and you may well have picked these two spellings from concrete provider payloads that mean exactly what you say.
Review assisted by AI (Claude Opus 5). Findings were verified against the files at this head; the reviewer is accountable for them.
| ]); | ||
|
|
||
| /** Provider codes meaning the model is temporarily at capacity. */ | ||
| const PROVIDER_CAPACITY_CODES: ReadonlySet<string> = new Set([ |
There was a problem hiding this comment.
[P2] See the review body. Short version: in the gRPC / Google API error model RESOURCE_EXHAUSTED is the standard code for quota exhaustion, not "server temporarily busy". This branch is only reached when the error did not already classify as RateLimit via 429, i.e. exactly when the code is carrying the meaning by itself.
If these two spellings came from concrete payloads that genuinely mean "at capacity", a one-line note here naming the providers would settle it permanently — the next person to add a code to this set will need the same reasoning, and right now the comment says what the code means but not who sends it or how you established that.
| const errorClass = classifyProviderFacts(facts); | ||
| const retryAfterMs = parseRetryAfterMs(facts.responseHeaders ?? {}); | ||
| if (errorClass === 'ProviderCapacity') { | ||
| if (retryAfterMs === null) return { retryable: false }; |
There was a problem hiding this comment.
[P3] This treats an absent Retry-After as more retryable than a malformed one, which inverts the usual information ordering.
parseRetryAfterMs returns undefined when neither header is present, and null when a header is present but unusable (unparseable, <= 0, or beyond MAX_SAFE_TIMER_DELAY_MS). So: no header at all → retryable: true and the caller backs off on its own; Retry-After: 0 or a garbage value → retryable: false and the turn does not retry at all.
A provider that sends a broken header ends up strictly worse off than one that sends none, even though the underlying condition is identically transient. The RateLimit branch below collapses both to non-retryable, which is defensible there because the server's own window is the whole point — but capacity is retried with local backoff, so the malformed case has a sensible fallback available and does not use it.
Not a blocker: it fails closed, and the user can retry by hand.
| ); | ||
|
|
||
| assert.equal(classifyError(capacity), 'ProviderCapacity'); | ||
| assert.deepEqual(providerRetryMetadata(capacity), { retryable: true }); |
There was a problem hiding this comment.
[P3] Object.assign(capacity, { responseHeaders: ... }) mutates capacity in place rather than deriving a new error, so after this line the object asserted on above no longer has the shape it was asserted with. It happens to be harmless in the current order, but it makes the two assertions look independent when they are not — if anyone later reorders them or adds a case in between, the earlier assert.deepEqual(providerRetryMetadata(capacity), { retryable: true }) starts failing for a reason that has nothing to do with the code under test.
{ ...capacity, responseHeaders: ... } would not work here since these are Error instances, but building a second error the same way you built the first would.
Worth saying that the test is otherwise well-shaped: it pins both code spellings, covers the top-level code carrier as well as the nested data.error.code one, and asserts retryAfterMs rather than only the boolean.
| case 'rate_limit': | ||
| case 'timeout': | ||
| return kind; | ||
| case 'provider_capacity': |
There was a problem hiding this comment.
[P3] The new class is collapsed back to provider_unavailable here, so everything downstream of ProviderRetryReason loses the distinction this PR just introduced — including the retry banner copy, which will say 模型服务暂时不可用 / Model service temporarily unavailable rather than anything about capacity.
That may well be deliberate: ProviderRetryReason is a narrower vocabulary than ModelFailureKind and widening it touches the retry banner in both locales. If so, it is worth one line of comment here saying the narrowing is intentional, because as written it reads like the case was added only to satisfy the switch.
(Credit where due — this is the one thing the other reviewer on our side and I independently landed on, so it does stand out to a reader.)
There was a problem hiding this comment.
Thanks for calling this out. The narrowing was not intentional; it did lose the user-visible distinction in the live retry banner.\n\nFixed in commit d473c5e: ProviderRetryReason now includes provider_capacity, the Runtime mapping and Runtime Host decoder preserve it, the compatibility epoch was bumped to 30 for the wire-contract change, and both locales have capacity-specific retry copy. Focused Runtime, Runtime Host protocol, and UI projection tests cover the full path.
|
Heads up — this is currently conflicting with One thing worth knowing: #3397 landed on 2026-08-22 and added ASF license headers across ~2685 files, so the rebase will touch more than you'd expect, and any file you add now needs a header ( Ping me once it's rebased and I'll pick it up. |
Fixes apache#3341 Generated-by: Codex
Keep xAI capacity errors distinct from quota exhaustion and fall back to bounded local retry when Retry-After is malformed. Generated-by: gpt-5.6-sol
Generated-by: gpt-5.6-sol
d473c5e to
4f00034
Compare
|
@Astro-Han PR #3365 已完成 rebase,当前基于最新 main(f19eede03),远端 head 为 4f00034。冲突已解决,构建和相关功能测试已通过;当前 PR 状态为 MERGEABLE,剩余 BLOCKED 原因为 REVIEW_REQUIRED,请继续处理。 |
Summary
Fixes #3341
Provider
resource-exhaustedfailures now retain a stableprovider_capacityclassification, use bounded retry metadata, and receive capacity-specific Desktop error and recovery guidance instead ofUnknown errorand direct-retry advice.Verification
npm installsucceeded and installed@ai-sdk/[email protected].npm run buildpassed for all workspaces.npm testwas attempted; it remains blocked by unrelated Windows environment failures including symlink permissions (EPERM), SQLite locks (EBUSY), missing Rive CLI, and workspace timeouts.AI use
Tool(s) and scope: Codex implemented the runtime classification, retry mapping, Desktop copy/recovery behavior, and regression tests. A
Generated-by: Codextrailer is present in the commit.Checklist
Does this PR entail a change in behavior?