From db2640ae7672515230badab9d864ff22246bb4eb Mon Sep 17 00:00:00 2001 From: b7r6 Date: Fri, 16 Oct 2026 11:52:26 -0400 Subject: [PATCH] Stop Azure retries after destination failure Treat a broken destination as terminal at the retry boundary, retaining the original error. Another origin download cannot repair the same writer. Preserve retries for transient failures with a usable writer. The preceding regressions now pass. Already-started SDK transfers remain visible in the byte ledger; cancelling those tasks is a separate change. --- nativelink-store/src/azure_blob_store.rs | 8 ++++++++ web/apps/docs/content/docs/how-to/stores/azure.mdx | 14 ++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/nativelink-store/src/azure_blob_store.rs b/nativelink-store/src/azure_blob_store.rs index f8c6d310bda..0b6712f1073 100644 --- a/nativelink-store/src/azure_blob_store.rs +++ b/nativelink-store/src/azure_blob_store.rs @@ -1192,6 +1192,14 @@ where self.note_read_ok(); Some((RetryResult::Err(e), writer)) } + // Retrying the origin cannot repair a failed destination. + // Preserve throttle feedback without crediting a successful read. + Err((e, wait)) if writer.is_pipe_broken() => { + if wait.is_some() { + self.note_read_throttled(); + } + Some((RetryResult::Err(e), writer)) + } Err((e, Some(wait))) => { self.note_read_throttled(); Some((RetryResult::RetryAfter(e, wait), writer)) diff --git a/web/apps/docs/content/docs/how-to/stores/azure.mdx b/web/apps/docs/content/docs/how-to/stores/azure.mdx index 562ed38085b..7f477674f48 100644 --- a/web/apps/docs/content/docs/how-to/stores/azure.mdx +++ b/web/apps/docs/content/docs/how-to/stores/azure.mdx @@ -136,6 +136,20 @@ described in [the shared options section](/how-to/stores/s3-and-compatible); `insecure_allow_http` and `disable_http2` are accepted but not read by the Azure store. + + When a download fails to send data because its destination has closed, the + read returns the delivery error without starting another retry. Transient + Azure errors remain retryable while the destination is usable. Requests + already started by the Azure SDK may still finish after the read returns. + + +The source link below shows the released implementation, which predates this +change. The regression tests in `nativelink-store/tests/azure_blob_store_test.rs` +and `azure_download_egress_test.rs` check that a failed destination stops retries +and repeated range downloads. + + + ## Steps [#steps]