PhoneTranslate

Real-time translator for a live phone call on iphone alternative to iphones without apple intelligence for call assist; for translation.

The call is a normal cellular / FaceTime call answered on the Mac via Continuity, with audio bridged through BlackHole virtual devices.

• Downlink: the far end speaks German → you read English in the browser.
• Uplink: you speak English → they hear German (synthesized voice injected into the call).

The far end's live audio is also played to your headphones automatically, so you hear the call while reading the translation.

---

Requirements

• macOS on Apple Silicon (tested on macOS 15)
• Homebrew
• Python 3.12 (Argos Translate needs spaCy/thinc ≥ 3.10)

brew install [email protected] ffmpeg blackhole-2ch blackhole-16ch

Install

git clone <your-repo-url> PhoneTranslate && cd PhoneTranslate
/opt/homebrew/opt/[email protected]/bin/python3.12 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

Models (one-time)

• Whisper (STT) auto-downloads on first run and caches in ~/.cache.
• Argos English→German language package:

  python -c "import argostranslate.package as p; p.update_package_index(); \
  pkg=next(x for x in p.get_available_packages() if x.from_code=='en' and x.to_code=='de'); \
  p.install_from_path(pkg.download())"

• Piper German voice (Thorsten) not committed (63 MB); download into voices/:

  mkdir -p voices && cd voices
  BASE=https://huggingface.co/rhasspy/piper-voices/resolve/main/de/de_DE/thorsten/medium
  curl -sL -o de_DE-thorsten-medium.onnx      "$BASE/de_DE-thorsten-medium.onnx"
  curl -sL -o de_DE-thorsten-medium.onnx.json "$BASE/de_DE-thorsten-medium.onnx.json"
  cd ..

Audio routing (per call)

In FaceTime → Video menu:

• Microphone → BlackHole 2ch — the synthesized German is injected here; the far end hears it.
• Output → BlackHole 16ch — the far end's German lands here; the app captures it.

Speak into your real hardware mic (default MacBook Air Microphone, set in web_app.py). Use headphones, not speakers — otherwise the far end's German leaks into your mic and gets mistranslated. See WORKING_SETUP.md for why this routing works.

Run

Web UI (recommended):

source .venv/bin/activate
python web_app.py            # then open http://127.0.0.1:8000

Dial your call, press Start (first Start loads the models), and watch the transcript.

Terminal only:

python run_local.py          # Ctrl-C to quit

---

How it works

The pipeline only ever touches audio through one seam, AudioTransport, using a fixed format everywhere: 16 kHz, mono, 16-bit PCM, 20 ms frames.

• Downlink — one Whisper pass (task="translate", German → English text). Silero VAD segments utterances; Whisper runs on a background worker.
• Uplink — Whisper (small.en) transcribes your English → Argos translates en→de → Piper synthesizes German audio injected back into the call.

File	Role
audio_transport.py	The AudioTransport seam + fixed 16 kHz / 20 ms frame format
local_audio_transport.py	AudioTransport over BlackHole (capture 16ch, inject 2ch) + headphone monitor + MicCapture
inject_audio.py	Device helpers + standalone CLI to inject a file/tone into the call
downlink_stt.py	Downlink: Silero VAD + Whisper translate → English text
uplink_stt.py	Uplink: Whisper small.en transcribe + Argos en→de → German text
tts.py	Piper TTS: German text → 16 kHz / 20 ms frames
run_local.py	Terminal entry point (full-duplex)
web_app.py + web_ui.html	Local web UI (Start/Stop, live transcript over SSE)
voices/	Piper de_DE-thorsten-medium model (download separately)

Notes

• Accuracy vs latency: small runs ~1 s/utterance but is weak on names/numbers. Bump model_size in downlink_stt.py / uplink_stt.py to medium for accuracy (~3 s).
• Formality: Argos mixes du/Sie; forcing formal German (Sie) isn't handled yet.
• Cellular codec: narrowband/voice-optimized — speech is clear, music sounds tinny.