Skip to content

fix: recover LearningSuite media across portal layouts - #135

Open
swernerx wants to merge 2 commits into
mainfrom
codex/learningsuite-media-recovery
Open

swernerx wants to merge 2 commits into
mainfrom
codex/learningsuite-media-recovery

Conversation

@swernerx

@swernerx swernerx commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

LearningSuite sync could miss secondary videos, stall behind provider-specific dialogs, or report success after incomplete scans. I now read all loaded HLS players through their public metadata API, retain available subtitles, and report incomplete extraction or downloads as failures.

I made dialog handling optional and independent of a particular welcome screen. Registration does not wait for a dialog: handlers react to visible native/ARIA dialogs and common modal containers with explicit dismissal controls. I cover late controls, stacked overlays, localized labels, and ineffective close icons while excluding form submissions and required decisions. Unsupported overlays retain the normal navigation timeout.

I removed the requirement for a particular media URL path, preserve existing archive IDs, and derive stable fallback IDs without expiring access signatures. Subtitle extraction supports blob, inline VTT, and HTTP(S) tracks. Caption Markdown follows the CLI/configured language, then the provider's default track or track order; all available tracks are retained.

I added --refresh-media to inspect previously downloaded lessons while reusing video files. Additional videos receive distinct filenames, failed downloads keep their lesson in an error state, and failed navigation cleans up media listeners.

For Apple Speech's zero-duration-word error, I retry once with two seconds of silence in a temporary audio copy. I preserve the original file, record the padding in transcript metadata, exclude it from media duration, and retain the actual Cuttledoc error when progress output precedes it.

Validation:

  • I ran pnpm check: formatting, lint, type checking, 697 unit tests, and build passed.
  • I ran pnpm test:integration --coverage: 28 tests passed, including Chromium fixtures, real HLS downloads, and ffmpeg padding/cleanup on success and failure. I added integration coverage uploads alongside unit coverage in CI.
  • I covered dialog-free pages, late and stacked dialogs, native/ARIA/MUI/custom containers, multiple languages, unsafe form controls, alternative media URLs, subtitle failures, and a full multi-video lesson extraction without playback or seeking.
  • I verified the original recovery against 19 accessible courses in one live tenant: 1,671 videos and matching JSON/Markdown transcripts with no missing or failed entries. Other portal variants are covered by reproducible browser fixtures; I have not verified additional live tenants.

I preserved the automatic transcription behavior already on main and formatted the generated 1.6.0 changelog so the repository-wide format check passes.

@codecov-commenter

codecov-commenter commented Sep 25, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 74.74227% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.69%. Comparing base (e51e685) to head (af0b9a9).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/scraper/learningsuite/playerMedia.ts 18.91% 30 Missing ⚠️
src/scraper/learningsuite/extractor.ts 61.90% 8 Missing ⚠️
src/scraper/learningsuite/navigator.ts 0.00% 6 Missing ⚠️
src/cli/commands/syncLearningSuite.ts 89.58% 5 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #135      +/-   ##
==========================================
+ Coverage   50.74%   52.69%   +1.94%     
==========================================
  Files          53       58       +5     
  Lines        6010     6156     +146     
  Branches     1796     1805       +9     
==========================================
+ Hits         3050     3244     +194     
+ Misses       2958     2910      -48     
  Partials        2        2              
Files with missing lines Coverage Δ
src/scraper/learningsuite/captions.ts 100.00% <100.00%> (ø)
src/scraper/learningsuite/dialogs.ts 100.00% <100.00%> (ø)
src/scraper/learningsuite/mediaIdentity.ts 100.00% <100.00%> (ø)
src/transcription/audioPadding.ts 100.00% <100.00%> (ø)
src/transcription/cuttledoc.ts 91.89% <100.00%> (+1.89%) ⬆️
src/cli/commands/syncLearningSuite.ts 42.76% <89.58%> (+7.01%) ⬆️
src/scraper/learningsuite/navigator.ts 18.70% <0.00%> (+1.19%) ⬆️
src/scraper/learningsuite/extractor.ts 42.71% <61.90%> (+12.60%) ⬆️
src/scraper/learningsuite/playerMedia.ts 18.91% <18.91%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@swernerx swernerx changed the title fix: recover complete LearningSuite media and transcripts fix: recover LearningSuite media across portal layouts Sep 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants