Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/guides/medialink.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,9 @@ intercom = medialink.Intercom(link, mic, speaker, voice=voice)
`audiofeatures`, which ignores a fan or road noise that stays on. Elsewhere,
CircuitPython among them, it uses a level gate: the room's quiet sets a
floor, and sound 10 dB over it counts. `voice.detector` says which you got.
The gate can't tell a voice from a door slamming.
The gate can't tell a voice from a door slamming. Both learn the room's quiet from
the first sound they hear, and pass over digital silence (frames in which
every sample is the same), which a microphone can send while it starts.

When both sides speak at once, one gets the floor and the other's words
are lost until the turn ends, as with two buttons. While the peer talks,
Expand Down
10 changes: 10 additions & 0 deletions lib/medialink/voice.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@

Either way a frame counts for ``hold_ms`` after the last one that held
speech, so the pauses between words don't end a turn.

Both learn the room's quiet from the first frames they hear, so a frame in
which every sample is the same is passed over: that is a microphone not yet
running, not a room. An ES7210 on a P4 board sends 100 to 280 ms of it the
first time it opens after power-up, and a detector that learned the quiet
from that took the room's own hiss for 3.4 s of speech.
"""

import struct
Expand Down Expand Up @@ -62,6 +68,10 @@ def speaking(self):
def feed(self, pcm):
"""Take a frame of signed 16-bit PCM. True while it is worth sending."""
self.level = _level(pcm, self.channels)
step = 2 * self.channels
if bytes(pcm[step:]) == bytes(pcm[:-step]):
# Digital silence, from a microphone still starting: not the room.
return self.speaking
if self._frontend is not None:
voiced = self._voiced_frontend(pcm)
else:
Expand Down
19 changes: 18 additions & 1 deletion tests/test_medialink_voice.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,23 @@ def test_a_steady_noise_becomes_the_floor(self):
heard = [voice.feed(f) for f in frames(loud)]
self.assertFalse(any(heard[-50:]))

def a_microphone_starting_up(self, detector):
"""A microphone's first open after power-up, as the P4 board's ES7210
gives it: 160 ms of one repeated sample, then 120 ms of zeros, then
the room's hiss."""
voice = medialink.VoiceActivity(detector=detector, hold_ms=1000)
start = frames(struct.pack("<h", -37) * (16 * 160) + bytes(2 * 16 * 120))
for frame in start:
voice.feed(frame)
return [voice.feed(f) for f in frames(hiss(16000 * 3, level=200, seed=7))]

def test_digital_silence_is_not_the_rooms_quiet(self):
self.assertFalse(any(self.a_microphone_starting_up("level")))

@unittest.skipUnless(HAVE_FEATURES, "needs audiofeatures with voice activity")
def test_digital_silence_is_not_the_rooms_quiet_to_audiofeatures(self):
self.assertFalse(any(self.a_microphone_starting_up("audiofeatures")))

@unittest.skipUnless(HAVE_FEATURES, "needs audiofeatures with voice activity")
def test_audiofeatures_is_the_default_where_there_is_one(self):
voice = medialink.VoiceActivity(hold_ms=0)
Expand Down Expand Up @@ -126,7 +143,7 @@ def listen_after_a_click(self, **kwargs):
again, as it does after each of the peer's turns, and the first
frame is a click."""
fmt = Format(16000, 1)
click = struct.pack("<h", 12000) * (FRAME // 2)
click = struct.pack("<hh", 12000, -12000) * (FRAME // 4)
sound = click + hiss(16000, seed=5)
network, a, b = pair()
voice = medialink.VoiceActivity(detector="level", hold_ms=40)
Expand Down
Loading