Repository navigation
Optional captions from a faster-whisper container, with speaker turns - #18
Merged
Merged
Conversation
An optional transcription service (whisper/: faster-whisper behind one streaming HTTP endpoint, int8 on the CPU) and a `captions` job that sends each cut episode's download to it when PODBLOCK_WHISPER_URL is set and the feed has no timed transcript. What comes back is split into two-line captions and saved as WebVTT among the episode's transcripts, so it is shifted to fit the cut and served in the private feeds like a feed's own. Captions jobs name their episode in the payload rather than holding it, so the episode stays ready and editable, and they get a worker thread of their own so transcriptions don't hold up downloads. CI builds the image as ghcr.io/<repo>-whisper; deployment/whisper/ is its kustomization. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Prc7ntPTFgrdpi1Mv7kfFm
The transcription service now transcribes in batches of 8 by default (faster-whisper's batched pipeline: about twice as quick on a CPU), and, with WHISPER_SPEAKERS, finds who speaks when with sherpa-onnx (pyannote segmentation and a WeSpeaker voice model, on the CPU, no PyTorch or Hugging Face token). The audio is decoded once for both. The captions job saves the speaker turns beside the captions; they stay out of the captions and the cuts for now. `podblock speakers EPISODE_ID` sets them beside the episode's repeats (each voice's talk, its share inside the repeats, lone voices, voices mostly inside the repeats), and can ask the service's new /speakers endpoint again with other clustering options, to tune them without transcribing again. The example deployment uses large-v3-turbo with 6 threads. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Prc7ntPTFgrdpi1Mv7kfFm
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Prc7ntPTFgrdpi1Mv7kfFm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An optional second container that transcribes episodes, so the private feeds carry captions. pod-block works as before without it: the quick start, the main image and
kubectl apply -k deployment/are unchanged.The transcription service (
whisper/)int8), behind one streaming HTTP endpoint. It works on one episode at a time and stops when the client hangs up.WHISPER_BATCH): about twice as quick on a CPU, and it punctuates better.WHISPER_SPEAKERS=true) via sherpa-onnx: pyannote segmentation and a WeSpeaker voice model. No PyTorch or Hugging Face token; the models (about 35 MB) are fetched on first start.POST /speakersre-runs that step alone with other clustering options.avstays at 18.ghcr.io/<repo>-whisperbeside the app's image.In pod-block
PODBLOCK_WHISPER_URLturns it on. Each episode cut without a timed transcript from its feed gets acaptionsjob, which sends the uncut download to the service.rel="captions") among the episode's transcripts, so the existing code shifts them to fit the cut and serves them; a re-cut doesn't need a new transcript.podblock speakers EPISODE_IDcompares them with the episode's repeats: each voice's talk time, its share inside the repeats, voices heard mostly inside the repeats, and lone voices.--thresholdand--countre-run only the speaker step, for tuning.Deployment
deployment/whisper/is a separate kustomization with its own labels, so pod-block's Service never selects its pods. The example useslarge-v3-turbowith 6 threads.deployment/README.mdhas a new "Optional: captions" section covering adding and removing it.Measured on 4 cores (2.8 GHz Xeon), real episodes
small, one at a timesmall, batches of 8large-v3-turbo, batches of 8On three real shows, two read their own ads, so speaker turns had nothing to add. On a network show, threshold 0.65 separated the ad voices from the hosts (two voices 67% and 88% inside the repeats), while 0.8 merged most of them into the hosts. The default threshold needs more tuning before anything is cut by it.
Testing
Generated by Claude Code