Skip to content

fix: keep extracts inside their time budget; flag YouTube IP blocks - #6

Merged
Adeolu05 merged 1 commit into
feat/export-formats-and-fixesfrom
fix/extract-reliability
Sep 28, 2026
Merged

Adeolu05 merged 1 commit into
feat/export-formats-and-fixesfrom
fix/extract-reliability

Conversation

@Adeolu05

@Adeolu05 Adeolu05 commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #5. Merge that first; GitHub will then retarget this PR to main.

Summary

Extract reliability

  • Title via oEmbed, fetched alongside the captions. Before, a watch-page scrape (up to 10s) ran before the caption request, all inside the 15s budget. Now the title comes from YouTube's oEmbed endpoint in parallel, and we wait at most 1.5s for it once the captions are in. oEmbed also returns titles with & and similar characters correctly, where the old regex left them HTML-escaped. Duration comes from the existing last-cue estimate in extract_guardrails.
  • Real deadline. asyncio.wait_for can't stop the worker thread, so a timed-out extract used to keep calling YouTube (and spending proxy bandwidth). The API and bot now pass a time.monotonic() deadline:
    • a requests.Session subclass caps each YouTube request at the time remaining,
    • IP-block retries never sleep past the deadline,
    • Vimeo's two requests are bounded the same way.
  • UPSTREAM_BLOCKED (503) for IpBlocked/RequestBlocked, so the metrics dashboard (which groups by error_code) separates proxy failures from videos that have no captions. Timeouts return UPSTREAM_TIMEOUT (504). Network errors get a clear message instead of a 500.

Housekeeping

  • CI now lints, type-checks and builds web/.
  • Removed seven unused web files left from an earlier homepage (Generator, Hero, BentoGrid, layout/Navbar, layout/Footer, ui/Button, types/index.ts). No page imports them.
  • Fixed the remaining lint errors. The new useHydrated hook (useSyncExternalStore) replaces the setState-in-effect "mounted" guards in Navbar and ThemeToggle.

Test plan

  • python -m unittest discover -s tests: 67 pass (11 new). Covers concurrent title fetch, a slow title not holding captions, deadline session behaviour, no retry sleep past the deadline, the 503/504 mapping, and the Vimeo deadline.
  • eslint .: 0 errors (1 existing font warning). tsc --noEmit passes.
  • Live: oEmbed returned the title in 0.87s. A 10ms budget failed with UPSTREAM_TIMEOUT after 0.09s without sending any request.
  • Live caption extraction: my home IP got rate-limited by YouTube during testing (the old code fails the same way, so it's not a regression). Retest once it clears.
  • next build: verified by the new CI job. It can't run locally until npm install restores the Windows lightningcss binary.

- Fetch the YouTube title via oEmbed, concurrently with captions, instead of
  a sequential watch-page scrape (up to 10s) before the caption request.
  Duration now comes from the guardrails' last-cue estimate.
- Carry a monotonic deadline from the API/bot into the fetch: every YouTube
  request is capped at the time left, IP-block retries never sleep past it,
  and Vimeo requests are bounded too. Timed-out extracts stop calling
  upstream instead of running on in the worker thread.
- New UPSTREAM_BLOCKED error code (503) for IpBlocked/RequestBlocked so the
  metrics dashboard separates proxy failures from videos without captions;
  YouTube timeouts map to UPSTREAM_TIMEOUT (504), network errors to a clear
  message instead of a 500.
- CI: lint, type-check, and build the web app.
- Web: remove seven unused components from an earlier homepage and fix the
  remaining lint errors (useHydrated replaces setState-in-effect guards).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
transcriptflow Ready Ready Preview Sep 28, 2026 5:45pm UTC

@Adeolu05
Adeolu05 merged commit 80a97b5 into feat/export-formats-and-fixes Sep 28, 2026
4 checks passed
Adeolu05 added a commit that referenced this pull request Oct 13, 2026

This branch was successfully deployed

1 active deployment
Preview — 6cccace8 Deployed Sep 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant