Skip to content

fix(sdk): recover mid-stream provider 5xx/429 like severed connections - #1162

Closed
nordicnode wants to merge 9045 commits into
CodebuffAI:mainfrom
nordicnode:oss/recover-provider-5xx-midstream-1155
Closed

fix(sdk): recover mid-stream provider 5xx/429 like severed connections#1162
nordicnode wants to merge 9045 commits into
CodebuffAI:mainfrom
nordicnode:oss/recover-provider-5xx-midstream-1155

Conversation

@nordicnode

Copy link
Copy Markdown

Summary

  • a provider-reported 500/429 arriving mid-stream — the openai-compatible shim enqueues it as an error part with finishReason='error' — no longer ends the entire run with an error
  • extend classifyThrownStreamRecovery (already consulted for both the thrown-iterator and error-chunk shapes) so retryable APICallErrors (429, any 5xx) take the same capped recovery path as a severed body: note injected into the conversation, retry step forced, capped at MAX_CONSECUTIVE_STREAM_RECOVERIES
  • the recovery message names the HTTP status instead of claiming the connection dropped
  • client-error statuses (400/401/402/403) are deterministic — retrying cannot help — so they stay fatal and still propagate to the run's error handling, including the existing 402 rethrow
  • add regression tests: 500 and 429 recover with a status-naming message, a wrapped 503 behind a RetryError cause chain recovers, client-error statuses and post-cancel 5xx stay fatal

This addresses the run-killing part of #1155: the recoverable class was "the connection failed to speak" (isTransientNetworkError, a socket-level allowlist) and the fatal class was "the provider reported a failure", which is backwards for flaky endpoints, where both are the same transient event.

Validation

  • bun test sdk/src/__tests__/stream-interruption.test.ts (19 pass, 5 new)
  • bun test sdk (493 pass; the 100 failures are network/env-dependent and pre-exist on main — same count with this change stashed)
  • bunx tsc --noEmit -p sdk/tsconfig.json clean

Refs #1155

Source: CodebuffAI/freebuff-private@9cbbba39efda2071b8325bf2779b39170e35348c
Source: CodebuffAI/freebuff-private@8470da01520ea7c273efb6eca943fd37fb265189
Source: CodebuffAI/freebuff-private@884e8a41dd02bb5c0be09d8da8e465c994a6c6a4
Source: CodebuffAI/freebuff-private@b550c712e828f5211de07c5699a643fe0fa8f22e
Source: CodebuffAI/freebuff-private@ddfa6c89a27176e4e1afef0bd597077698e3d9e3
Source: CodebuffAI/freebuff-private@9b8fb92689a45c3d52b65f6d4177bb8b0e4a29db
Source: CodebuffAI/freebuff-private@e20dcb5811c89bc349a283c8fb8b474f03857eaa
Source: CodebuffAI/freebuff-private@1a9bb08ef45331c94cfa616a7e5f91dbc3375e7b
Source: CodebuffAI/freebuff-private@2204d7f013215d4a9746f87ccadb88422bdbf0ce
Source: CodebuffAI/freebuff-private@d85f7624f6df913150afd97028ce8034caf9401c
Source: CodebuffAI/freebuff-private@496a274da0db56d0465478154dbd368ead06dc33
Source: CodebuffAI/freebuff-private@2479764c0c4dc98e75a5c5778d29d10938b5997e
Source: CodebuffAI/freebuff-private@72cb80a710ca0b69c0ea1c92caa09449ae14ae23
Source: CodebuffAI/freebuff-private@00c76d2a5513197c7f5963463a1511ce8a4086d6
Source: CodebuffAI/freebuff-private@ae3b428c053013f6c40d893c785f6f383a1eefb7
Source: CodebuffAI/freebuff-private@1f928dbc1d0b1c4ae199f3b12111897300092931
Source: CodebuffAI/freebuff-private@23d94d0a7272b2c1460bb5576c0359cf0fe2765a
Source: CodebuffAI/freebuff-private@56076c9423fb9d6b185fb66d4ec7d7818c544aa4
Source: CodebuffAI/freebuff-private@7ac0799f05337a0df3eafe682e030544ce191d2f
Source: CodebuffAI/freebuff-private@2464fed8ca7d9730dfa2226e1c4267353c2b1e85
Source: CodebuffAI/freebuff-private@d6c79c09d66c9187499f37445a7f82a93ae674a1
Source: CodebuffAI/freebuff-private@db635212c3c95b5545a46afd2fffa3e0d345dd48
Source: CodebuffAI/freebuff-private@8602eeac5f86c63dcdf7051675bda726cc8e0b71
Source: CodebuffAI/freebuff-private@03bff2f48d6c40320acc1cb41ca389d222ac2baa
Source: CodebuffAI/freebuff-private@54215700121b04bef2a7857489eb2017d24a113f
Source: CodebuffAI/freebuff-private@4f4f9a6cd4e9c7aa793a32fcdc8438ced2a0f5a2
Source: CodebuffAI/freebuff-private@5f1f82c2fb2bcf95a01ffb6fc28ccfc25619a83e
Source: CodebuffAI/freebuff-private@9d8b240c6862ab9c5c720af7a1facbf85a48a877
Source: CodebuffAI/freebuff-private@1b960fe83deb21c87e4cb615d7789123361ac407
Source: CodebuffAI/freebuff-private@e8b8ef0c39f6e5fd0f1da7fe1d2704027dcbfb16
github-actions Bot and others added 26 commits August 31, 2026 00:22
Source: CodebuffAI/freebuff-private@f9c01eea0ce77cd368249f7c5d74ec024a7baf91
Source: CodebuffAI/freebuff-private@3e9ad036c6f9b7f71c2127403a89ce605c53c004
Source: CodebuffAI/freebuff-private@a98e28b9a11c0e821d5171d4fed11136ddfda460
Source: CodebuffAI/freebuff-private@5de60102491d5f19108aec59ab7a0bb41f3de60a
Source: CodebuffAI/freebuff-private@ade37e50551c2230d76dffc3fd29ca7ad02bdbfc
Source: CodebuffAI/freebuff-private@2d04668b529a04c2a90806272993d5f67a1afa1f
Source: CodebuffAI/freebuff-private@2f358ebf68b79b05acb81f18ee42ef4295edee17
Source: CodebuffAI/freebuff-private@e411009a76fbd9f689550bd51bc3b938209ec6f8
Source: CodebuffAI/freebuff-private@70a710b1d0941b7bd3a742dbc8e18f276b602850
Source: CodebuffAI/freebuff-private@3535770948dfde40d9b53936ad8487a3dbb90893
Source: CodebuffAI/freebuff-private@bba508a9f359d5e49230c932ef03fdb733c18bff
Source: CodebuffAI/freebuff-private@741dcfc4f0fc1e7c33d4544fc5ce7259e851c920
Source: CodebuffAI/freebuff-private@d9a3b75bd429ea44aca483addec9b5a3e3bcc79e
Source: CodebuffAI/freebuff-private@b76209dd2ec3f278ef28660c239c4e484572d7b4
Source: CodebuffAI/freebuff-private@ff62cf3506968799272123e6f637cdee6935e48f
Source: CodebuffAI/freebuff-private@d6591b0c4986740a517395811b4fe9d6e5db47b2
Source: CodebuffAI/freebuff-private@87a991308dfe74c4e68cf07c4206d0fc63db1c8a
Source: CodebuffAI/freebuff-private@5c9444e7ef7993a29e064220ef46c3a58af12b7d
Source: CodebuffAI/freebuff-private@e40b501c923f0bd2f33749cf6f081d9e74b42282
Source: CodebuffAI/freebuff-private@a82256a7abc33584a5bb1be29fbbd9c67c2dd8d7
Source: CodebuffAI/freebuff-private@42d9d1db9a7b2895732d49ab7d6ce84162fc81d4
Source: CodebuffAI/freebuff-private@61c0ffffb152a9b77e2ecfcd3d8adb0e4e4cfb87
Source: CodebuffAI/freebuff-private@a40168234a220d145b5ff7cf702b6364184d7296
Source: CodebuffAI/freebuff-private@91527d25716d4282b0e454e92ecf8b2fc8c1c9f1
Source: CodebuffAI/freebuff-private@09f98d52aaf28097b9509b34acdcd987ade63586
A provider-reported 500/429 arriving mid-stream — the openai-compatible
shim enqueues it as an error part with finishReason='error' — was thrown
straight out of the stream and ended the entire run with an error. The
same underlying transient event surfacing as a severed body instead
took the capped recovery path (note injected into the conversation,
retry step forced, capped at MAX_CONSECUTIVE_STREAM_RECOVERIES).

The recoverable class was 'the connection failed to speak' and the
fatal class was 'the provider reported a failure', which is backwards
for flaky endpoints, where both are the same transient event.

Route retryable APICallErrors (429, any 5xx) through the same capped
recovery path with a message naming the HTTP status. Client-error
statuses (400/401/402/403) are deterministic — retrying cannot help —
so they stay fatal and still propagate to the run's error handling.

Refs CodebuffAI#1155
@victorxheng

ghost commented Aug 31, 2026

Copy link
Copy Markdown

Apologies — this PR was auto-closed by GitHub when we force-pushed a history rewrite of this repository (repository maintenance; every commit SHA changed). That was not a judgment on this PR, and GitHub does not allow us to reopen it because the commits it was based on no longer exist in the new history.

If you'd like to continue with this change: rebase your branch onto the new main (or recreate it from a fresh clone) and open a new PR — feel free to link back to this one for context, and we'll pick up the review there.

Sorry for the churn, and thanks for contributing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants