Skip to content

fix(capture): stop DeepFilterNet from eating the end of every utterance (#61) - #63

Merged
RevolutionLA merged 1 commit into
mainfrom
fix/df-resample-tail
Oct 11, 2026
Merged

RevolutionLA merged 1 commit into
mainfrom
fix/df-resample-tail

Conversation

@RevolutionLA

Copy link
Copy Markdown
Owner

The bug (#61)

With denoise: deepfilter, every utterance lost 69-97 ms off its end.

AudioCapture::dfDenoise() resamples 16 kHz → 48 kHz, hands the file to
deep-filter, and resamples back. Each leg called soxr_process() once. A
polyphase resampler keeps its group delay in an internal buffer — 1024 output
samples on the up leg, 341 on the down leg (measured, soxr_delay(), engine
cr32s, HQ quality which is the default) — and gives it back only when told the
input has ended. soxr does not read that from ilen = 0; the header says
end-of-input is in == NULL.

Consequences, in the order they were measured:

measurement before
1 s tone through a length-preserving stub 16000 in → 15170 out
benchmark speech clips (4.7–10 s), real deep_filter 0.5.6 deficit 1102–1558 samples = 69–97 ms

deep-filter's own framing then drops a further constant 1440 samples at 48 kHz.
A burst probe answered whether that is delay or truncation: cross-correlating the
model's output against its input on real speech gave lag 0, correlation 0.983
with the file exactly 30 ms shorter — pure truncation. Since applyDenoise()
runs before trimSilence(), and that step keeps only kPadFrames = 1 frame
(20 ms) of trailing audio, 69–97 ms is 3–5× the protection. The part being lost
was the end of the last word, and README's "the last syllable is never clipped"
bullet was false on this path.

The fix

  1. Drain both legs with in = NULL until nothing more comes out (bounded at
    32 iterations). Takes the tone round trip to an exact 16000 → 16000.
  2. Restore the model's 30 ms from the raw input. Length is now an invariant
    of dfDenoise(); the trade is that the final 30 ms is un-denoised instead of
    gone. Measured on the corpus: 0 samples lost on 5 of 7 clips, 2 samples
    over on the 10 s ones (WAV-header rounding in the expected value).
  3. Bound the restore at 160 ms (kDfMaxTailRestore). A bigger hole is not
    framing, it is a broken result, so it is rejected and falls back to speexdsp
    rather than being padded into a half-denoised utterance.
  4. No failure path may touch the caller's buffer. Results are assembled in a
    local vector and swapped in only on success; previously the giveUp paths
    happened to be safe, and this makes it structural.

Tests

tests/test_denoise_fallback.cpp gains two stubs that mimic measured shapes:
chop.sh (zeroed body, 1440 samples short — the real model) and gut.sh (half
the audio missing). The interesting assertion is positional: the body arrives as
zeros, so the first non-zero sample must be exactly at kToneSamples - 480.
That proves the drain and the compensation independently — without the drain the
handover index moves and the assertion fails even though the total length looks
right.

Four mutations, each red on a specific assertion:

mutation caught by
remove the drain loop compensation stops exactly at the model's 30 ms
remove the tail restore keeps the full utterance + the two index checks
make the restore bound unbounded a mostly-missing result is not a success
swap into the caller before the length checks a mostly-missing result is not a success

tests/test_bench_denoise.sh tightens samples_out from a 75–105% band to
equality, since length is now guaranteed. Suite: 21/21.

Docs

  • docs/benchmarks.md: post-fix row + committed artifacts, deficit table
    before→after.
  • Corrected a claim of my own: the pre-fix text said "a repeat run gave
    0.151, so the number is stable to a few percent". A third run gave 0.131, so
    that agreement was luck. The file now quotes RTF 0.13–0.16 / ~1.0–1.3 s per
    utterance and warns against reading 0.156 → 0.131 as a speed-up from this fix.
    README §5 matches, in both languages.
  • Roadmap row 5 marks DeepFilterNet truncates 69-97 ms off the end of every utterance (unflushed soxr round trip) #61 as found and fixed by the same measurement.

Not changed

speexdsp remains the default; this path still costs ~1 s/utterance and now
also provably never changes audio length.

Issue #61, found by the roadmap row 5 measurement. Both legs of the
16k -> 48k -> 16k round trip called soxr_process() exactly once. A polyphase
resampler keeps its group delay (1024 output samples upsampling, 341
downsampling) in an internal buffer until it is told the input ended, and soxr
reads that from in == NULL, not from ilen = 0 - the header is explicit. So a
16000-sample tone came back as 15170 through a length-preserving stub, and real
speech lost 69-97 ms. deep-filter's own framing then drops a further constant
1440 samples at 48 kHz; a burst probe (lag 0, correlation 0.983) shows that is
truncation rather than delay, so what disappeared was the end of the last word,
on a pipeline whose VAD step keeps only 20 ms of trailing audio.

Both legs now drain, and the model's 30 ms is appended back from the raw input:
dfDenoise() preserves the buffer length (measured 0 samples lost across the
benchmark corpus), trading the last 30 ms of denoising for no lost speech. A
result more than 160 ms short is a malfunction, not a framing loss, so it is
rejected and falls back to speexdsp instead of being padded into a
half-denoised buffer - and results are assembled locally, so no failure path can
leave a mutated partial in the caller's audio.

test_denoise_fallback gains a chop.sh that reproduces the model's exact shape
(zeroed body, 1440 samples short) and asserts the index where denoised audio
hands over to the restored tail, plus gut.sh for the incomplete-result path.
Four mutations checked: no drain, no compensation, no bound, and mutating the
caller before the length checks - each turns a specific assertion red.

Timing is unchanged within noise; three runs of the same corpus spanned RTF
0.131-0.163, so docs/benchmarks.md now states the spread instead of implying the
fix was 15% faster. tests/test_bench_denoise.sh tightens samples_out from a
75-105% band to equality now that the length is an invariant.
@RevolutionLA

Copy link
Copy Markdown
Owner Author

Extra evidence, this time through the real pipeline rather than the stubs or
the harness: tools/test_audio_file utt10s.wav out.wav deepfilter on a 10.05 s
clip with the genuine deep_filter 0.5.6 binary.

Nextless Capture [timer] df_denoise=1404ms samples=160850 restored=480
Nextless Capture: trimmed 10240 leading + 210 trailing samples (orig=160850 now=150400)
Nextless Pipeline [summary] loudness=-25.0 isBlank=0 denoiser=deepfilter time=1422ms samples=150400

Three things to read out of that:

  • samples=160850 equals the input frame count exactly, with restored=480 -
    the model's 30 ms came back from the raw tail, and nothing else was lost.
  • trimSilence removed only 210 trailing samples afterwards, so the audio
    the recogniser sees still ends where the recording ended. Before this fix the
    buffer was already 1325 samples shorter at that point, and the trim's own
    margin is 320 - that is how the last syllable went missing.
  • 1404 ms wall for 10 s of audio (RTF 0.139) sits inside the published 0.13-0.16
    band, so the drain and the tail copy did not move the cost.

(The VAD has been replaced by a hack line is libspeexdsp talking during
trimSilence, unrelated to this change.)

@RevolutionLA
RevolutionLA merged commit 524501d into main Oct 11, 2026
8 checks passed
@RevolutionLA
RevolutionLA deleted the fix/df-resample-tail branch October 11, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant