Skip to content

fix(runtime): classify provider capacity errors - #3365

Open
CxHsin wants to merge 3 commits into
apache:mainfrom
CxHsin:fix/runtime-resource-exhausted
Open

fix(runtime): classify provider capacity errors#3365
CxHsin wants to merge 3 commits into
apache:mainfrom
CxHsin:fix/runtime-resource-exhausted

Conversation

@CxHsin

@CxHsin CxHsin commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #3341

Provider resource-exhausted failures now retain a stable provider_capacity classification, use bounded retry metadata, and receive capacity-specific Desktop error and recovery guidance instead of Unknown error and direct-retry advice.

Verification

  • npm install succeeded and installed @ai-sdk/[email protected].
  • npm run build passed for all workspaces.
  • Runtime provider classification tests passed: 11/11.
  • Desktop provider capacity presentation tests passed: 2/2.
  • Full npm test was attempted; it remains blocked by unrelated Windows environment failures including symlink permissions (EPERM), SQLite locks (EBUSY), missing Rive CLI, and workspace timeouts.

AI use

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Codex implemented the runtime classification, retry mapping, Desktop copy/recovery behavior, and regression tests. A Generated-by: Codex trailer is present in the commit.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — provider capacity failures now have stable classification and wait/switch-model guidance.
  • No

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — classifying on the provider's structured code/type rather than on message text is the right foundation, and matching the existing CONTEXT_OVERFLOW_PROVIDER_CODES shape means this slots in without inventing a new mechanism. The copy and recovery-hint plumbing through to the desktop is complete and the tests assert the classification and the retry metadata rather than restating the implementation.

No [P0]/[P1]. One [P2] that is really a question about the problem definition, and two [P3]s.

The [P2]: resource-exhausted / resource_exhausted is not a single meaning across providers. In gRPC and the Google API error model, RESOURCE_EXHAUSTED (status 8) is the standard code for quota exhaustion — per-minute, per-day, or per-project — not for "the server is busy right now". The user-facing copy this PR routes it to is 模型服务暂时满载,请等待几分钟或切换模型后重试 / Wait a few minutes or switch models, and that advice is actively wrong for a daily quota: waiting a few minutes will not help, and the correct action is different.

The 429 branch above catches the common case, since a Gemini quota error usually carries HTTP 429 and returns RateLimit before reaching this check. So this is not a blanket misclassification. But that also means the capacity branch is reached precisely when the status code is absent or non-429 — which is the case where the code alone is carrying the whole meaning, and where it is most ambiguous.

Could you say which providers you observed emitting these two codes, and with which meaning? If the set is narrow and "server at capacity" is what they actually mean, then pinning that in the comment next to PROVIDER_CAPACITY_CODES resolves it and I have no further concern. If it turns out the same code also arrives for quota exhaustion, the classification is fine but the recovery copy needs to not promise that waiting works.

I want to be explicit that I am asking rather than asserting: I have not seen your traces, and you may well have picked these two spellings from concrete provider payloads that mean exactly what you say.

Review assisted by AI (Claude Opus 5). Findings were verified against the files at this head; the reviewer is accountable for them.

]);

/** Provider codes meaning the model is temporarily at capacity. */
const PROVIDER_CAPACITY_CODES: ReadonlySet<string> = new Set([

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] See the review body. Short version: in the gRPC / Google API error model RESOURCE_EXHAUSTED is the standard code for quota exhaustion, not "server temporarily busy". This branch is only reached when the error did not already classify as RateLimit via 429, i.e. exactly when the code is carrying the meaning by itself.

If these two spellings came from concrete payloads that genuinely mean "at capacity", a one-line note here naming the providers would settle it permanently — the next person to add a code to this set will need the same reasoning, and right now the comment says what the code means but not who sends it or how you established that.

const errorClass = classifyProviderFacts(facts);
const retryAfterMs = parseRetryAfterMs(facts.responseHeaders ?? {});
if (errorClass === 'ProviderCapacity') {
if (retryAfterMs === null) return { retryable: false };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] This treats an absent Retry-After as more retryable than a malformed one, which inverts the usual information ordering.

parseRetryAfterMs returns undefined when neither header is present, and null when a header is present but unusable (unparseable, <= 0, or beyond MAX_SAFE_TIMER_DELAY_MS). So: no header at all → retryable: true and the caller backs off on its own; Retry-After: 0 or a garbage value → retryable: false and the turn does not retry at all.

A provider that sends a broken header ends up strictly worse off than one that sends none, even though the underlying condition is identically transient. The RateLimit branch below collapses both to non-retryable, which is defensible there because the server's own window is the whole point — but capacity is retried with local backoff, so the malformed case has a sensible fallback available and does not use it.

Not a blocker: it fails closed, and the user can retry by hand.

);

assert.equal(classifyError(capacity), 'ProviderCapacity');
assert.deepEqual(providerRetryMetadata(capacity), { retryable: true });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Object.assign(capacity, { responseHeaders: ... }) mutates capacity in place rather than deriving a new error, so after this line the object asserted on above no longer has the shape it was asserted with. It happens to be harmless in the current order, but it makes the two assertions look independent when they are not — if anyone later reorders them or adds a case in between, the earlier assert.deepEqual(providerRetryMetadata(capacity), { retryable: true }) starts failing for a reason that has nothing to do with the code under test.

{ ...capacity, responseHeaders: ... } would not work here since these are Error instances, but building a second error the same way you built the first would.

Worth saying that the test is otherwise well-shaped: it pins both code spellings, covers the top-level code carrier as well as the nested data.error.code one, and asserts retryAfterMs rather than only the boolean.

case 'rate_limit':
case 'timeout':
return kind;
case 'provider_capacity':

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] The new class is collapsed back to provider_unavailable here, so everything downstream of ProviderRetryReason loses the distinction this PR just introduced — including the retry banner copy, which will say 模型服务暂时不可用 / Model service temporarily unavailable rather than anything about capacity.

That may well be deliberate: ProviderRetryReason is a narrower vocabulary than ModelFailureKind and widening it touches the retry banner in both locales. If so, it is worth one line of comment here saying the narrowing is intentional, because as written it reads like the case was added only to satisfy the switch.

(Credit where due — this is the one thing the other reviewer on our side and I independently landed on, so it does stand out to a reader.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for calling this out. The narrowing was not intentional; it did lose the user-visible distinction in the live retry banner.\n\nFixed in commit d473c5e: ProviderRetryReason now includes provider_capacity, the Runtime mapping and Runtime Host decoder preserve it, the compatibility epoch was bumped to 30 for the wire-contract change, and both locales have capacity-specific retry copy. Focused Runtime, Runtime Host protocol, and UI projection tests cover the full path.

@Astro-Han

Copy link
Copy Markdown
Contributor

Heads up — this is currently conflicting with main, so I can't review or merge it as-is. Your CI is fully green, so it really is just the base that's behind.

One thing worth knowing: #3397 landed on 2026-08-22 and added ASF license headers across ~2685 files, so the rebase will touch more than you'd expect, and any file you add now needs a header (npm run write:asf-headers).

Ping me once it's rebased and I'll pick it up.

CxHsin added 3 commits August 23, 2026 08:38
Keep xAI capacity errors distinct from quota exhaustion and fall back to bounded local retry when Retry-After is malformed.

Generated-by: gpt-5.6-sol
@CxHsin
CxHsin force-pushed the fix/runtime-resource-exhausted branch from d473c5e to 4f00034 Compare August 23, 2026 00:58
@CxHsin

CxHsin commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

@Astro-Han PR #3365 已完成 rebase,当前基于最新 main(f19eede03),远端 head 为 4f00034。冲突已解决,构建和相关功能测试已通过;当前 PR 状态为 MERGEABLE,剩余 BLOCKED 原因为 REVIEW_REQUIRED,请继续处理。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

provider capacity errors (code=resource-exhausted) surface as "Unknown error" + misleading "retry directly" guidance

2 participants