Repository navigation
fix(capture): stop DeepFilterNet from eating the end of every utterance (#61) - #63
Merged
Merged
Conversation
Issue #61, found by the roadmap row 5 measurement. Both legs of the 16k -> 48k -> 16k round trip called soxr_process() exactly once. A polyphase resampler keeps its group delay (1024 output samples upsampling, 341 downsampling) in an internal buffer until it is told the input ended, and soxr reads that from in == NULL, not from ilen = 0 - the header is explicit. So a 16000-sample tone came back as 15170 through a length-preserving stub, and real speech lost 69-97 ms. deep-filter's own framing then drops a further constant 1440 samples at 48 kHz; a burst probe (lag 0, correlation 0.983) shows that is truncation rather than delay, so what disappeared was the end of the last word, on a pipeline whose VAD step keeps only 20 ms of trailing audio. Both legs now drain, and the model's 30 ms is appended back from the raw input: dfDenoise() preserves the buffer length (measured 0 samples lost across the benchmark corpus), trading the last 30 ms of denoising for no lost speech. A result more than 160 ms short is a malfunction, not a framing loss, so it is rejected and falls back to speexdsp instead of being padded into a half-denoised buffer - and results are assembled locally, so no failure path can leave a mutated partial in the caller's audio. test_denoise_fallback gains a chop.sh that reproduces the model's exact shape (zeroed body, 1440 samples short) and asserts the index where denoised audio hands over to the restored tail, plus gut.sh for the incomplete-result path. Four mutations checked: no drain, no compensation, no bound, and mutating the caller before the length checks - each turns a specific assertion red. Timing is unchanged within noise; three runs of the same corpus spanned RTF 0.131-0.163, so docs/benchmarks.md now states the spread instead of implying the fix was 15% faster. tests/test_bench_denoise.sh tightens samples_out from a 75-105% band to equality now that the length is an invariant.
Owner
Author
|
Extra evidence, this time through the real pipeline rather than the stubs or Three things to read out of that:
(The |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug (#61)
With
denoise: deepfilter, every utterance lost 69-97 ms off its end.AudioCapture::dfDenoise()resamples 16 kHz → 48 kHz, hands the file todeep-filter, and resamples back. Each leg calledsoxr_process()once. Apolyphase resampler keeps its group delay in an internal buffer — 1024 output
samples on the up leg, 341 on the down leg (measured,
soxr_delay(), enginecr32s, HQ quality which is the default) — and gives it back only when told theinput has ended. soxr does not read that from
ilen = 0; the header saysend-of-input is
in == NULL.Consequences, in the order they were measured:
16000 in → 15170 outdeep_filter 0.5.6deep-filter's own framing then drops a further constant 1440 samples at 48 kHz.
A burst probe answered whether that is delay or truncation: cross-correlating the
model's output against its input on real speech gave lag 0, correlation 0.983
with the file exactly 30 ms shorter — pure truncation. Since
applyDenoise()runs before
trimSilence(), and that step keeps onlykPadFrames = 1frame(20 ms) of trailing audio, 69–97 ms is 3–5× the protection. The part being lost
was the end of the last word, and README's "the last syllable is never clipped"
bullet was false on this path.
The fix
in = NULLuntil nothing more comes out (bounded at32 iterations). Takes the tone round trip to an exact
16000 → 16000.of
dfDenoise(); the trade is that the final 30 ms is un-denoised instead ofgone. Measured on the corpus: 0 samples lost on 5 of 7 clips, 2 samples
over on the 10 s ones (WAV-header rounding in the expected value).
kDfMaxTailRestore). A bigger hole is notframing, it is a broken result, so it is rejected and falls back to speexdsp
rather than being padded into a half-denoised utterance.
local vector and swapped in only on success; previously the giveUp paths
happened to be safe, and this makes it structural.
Tests
tests/test_denoise_fallback.cppgains two stubs that mimic measured shapes:chop.sh(zeroed body, 1440 samples short — the real model) andgut.sh(halfthe audio missing). The interesting assertion is positional: the body arrives as
zeros, so the first non-zero sample must be exactly at
kToneSamples - 480.That proves the drain and the compensation independently — without the drain the
handover index moves and the assertion fails even though the total length looks
right.
Four mutations, each red on a specific assertion:
tests/test_bench_denoise.shtightenssamples_outfrom a 75–105% band toequality, since length is now guaranteed. Suite: 21/21.
Docs
docs/benchmarks.md: post-fix row + committed artifacts, deficit tablebefore→after.
0.151, so the number is stable to a few percent". A third run gave 0.131, so
that agreement was luck. The file now quotes RTF 0.13–0.16 / ~1.0–1.3 s per
utterance and warns against reading 0.156 → 0.131 as a speed-up from this fix.
README §5 matches, in both languages.
Not changed
speexdspremains the default; this path still costs ~1 s/utterance and nowalso provably never changes audio length.