Skip to content

Ask the original link for a fresh signed URL when one expires mid-download or while paused - #166

Open
Arpit-Khandelwal wants to merge 1 commit into
anmolkapil:mainfrom
Arpit-Khandelwal:fix/relink-expired-signed-url
Open

Arpit-Khandelwal wants to merge 1 commit into
anmolkapil:mainfrom
Arpit-Khandelwal:fix/relink-expired-signed-url

Conversation

@Arpit-Khandelwal

Copy link
Copy Markdown
Contributor

Fixes #160.

A link that redirects to a short-lived signed URL (Hugging Face resolve links, S3 presigned URLs) failed the download with "This link no longer works" once the signature expired, mid-download or while paused, even though the pasted link still worked.

What changes

  • The download keeps the link it was given (requestedUrl on the start request and its state). The dialog sets it from the probe, which also covers links from the browser extension. It's saved in the manifest and kept in history. The manifest stays at version 7: the field is optional, so downloads saved before this load as they did and just don't refresh.
  • When a range request gets 401/403/410 and requestedUrl differs from the URL in use, that link is probed again and checked exactly the way Fix checks a link: same size, range support, and bytes compared with what's on disk. I moved those checks out of replaceLink into adoptLink, so Fix and this share one path. Every stream then carries on with the fresh URL, with no strikes counted.
  • If the check fails, the 403 is handled as before and the download ends with "This link no longer works".
  • The probe runs at most once per refused URL. Every stream that hits the 403 waits on that one check, and a refreshed URL can only be refreshed again after a block has come in from it. Each run starts over, so a resume (including after a relaunch) gets another try.

Why a 403 isn't counted as a connection limit when the refresh works: concurrency.ts reads a 403 as "fewer connections, please". After a real expiry every new request is refused, and counting those would cut the stream count for minutes after each refresh. This only applies to downloads whose link redirects somewhere else; if the link redirects to the same URL, or the check fails, the 403 counts exactly as before.

How I checked it
e2e/signedUrl.spec.ts runs against a local origin where /start redirects to ?sig=1, and after "expiry" sig=1 answers 403 to range requests while /start points at sig=2. It covers four cases: expiry mid-download, expiry while paused then resumed, expiry across a quit and relaunch, and the link itself returning 410 (which still errors, with no loop). Every completed download is checked byte for byte.

  • On main all four fail; with this change they pass, 20/20 with --repeat-each=5.
  • download, recovery, savedProgress, integrity, lifecycle, downloadActions and probeRedirects pass. A download saved without requestedUrl still loads and completes.
  • Lint, typecheck and format are clean.

I didn't run a real hour-long Hugging Face or S3 download; the expiry is simulated by the local server.

…nload or while paused

A link that redirects to a short-lived signed URL (Hugging Face's resolve
links, S3 presigned ones) failed the download with "This link no longer
works" once the signature expired, although the link itself still worked.

The download now keeps the link it was given (requestedUrl, on the start
request and its state, so it's saved with it and kept in history). When a
range request is answered 401, 403 or 410 and that link differs from the
one in use, it is probed again once and checked exactly as Fix checks a
new link (size, range support, bytes compared with what's on disk). Every
stream then goes on with the fresh URL. If that fails, the download fails
as before. A resume, including after a relaunch, gets another try.

Fixes anmolkapil#160

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A download fails once its signed CDN URL expires, though the link it came from still works

1 participant