[johnmatthewtennant]

Summary

In a realtime (Expert-Spokesperson) call, the Spokesperson reads the Expert's replies aloud in its own words, so the user never hears the Expert's actual wording, and the transcript repeats it paraphrased. This change lets the Expert speak for itself. The Spokesperson keeps talking to the user in realtime, and the Expert's replies play through the call's ordinary text-to-speech backend, in the Expert's own voice.

• --expert-voice tts|spokesperson is a session option for Expert-Spokesperson calls only, and defaults to spokesperson. With tts, the session builds the configured TTS backend (Siri, OpenAI or Pocket, with its voice and rate) alongside the realtime connection.
• Replies: an admitted prepare_speak is queued as a reply rather than sent to the Spokesperson as a directive. Replies play in the Spokesperson's single playback slot. User speech or cancel stops a reply, and speak reports speech_completed, speech_interrupted (with spoken_through_utf8) or speech_failed. If a reply's audio fails without the host confirming it stopped, the session ends, as it does for conventional playback.
• Afterwards: the realtime session receives a silent context item, without a response.create, saying what the user heard. The Spokesperson can follow up without repeating it. Handoffs the reply answers are dismissed once any of it was heard, and released otherwise.
• Voices: the Spokesperson and the Expert get separate voice and speed controls in the menu bar, the CLI (settings --spokesperson-voice/--spokesperson-rate) and the protocol. speak --voice NAME [--language BCP47] speaks a single reply in another voice of the same backend, so each agent can sound different. A voice the session hasn't used yet is built on a worker thread, so building it never stalls audio or microphone input. Siri voices whose assets are missing are listed as not installed.

Related issue

None found.

Testing

Integration tests run against a fake realtime server and a fake OpenAI speech endpoint.

• expert_voice_speaks_replies_itself_and_tells_the_spokesperson_what_was_heard: a reply is admitted, the speech endpoint receives its exact text in the requested voice (coral), and speak gets speech_completed. The Spokesperson receives the note The Expert said this to the user in its own voice: I checked the repository., is never asked to say the reply itself, and starts no response. An unknown voice gets not_admitted / invalid_voice.

• user_speech_interrupts_an_expert_reply_where_it_was_heard and cancel_interrupts_an_expert_reply_where_it_was_heard: a reply is still playing after the host has played part of it. The user starting to speak, or cancel for that prepare, gets speech_interrupted, with spoken_through_utf8 short of the full text.

• expert_voice_reply_whose_audio_never_settles_ends_the_session: a host that never confirms the drain or the cancellation gets speech_failed, and then the session exits with failure. Before the fix, the session kept running.

• expert_voice_keeps_a_separate_spokesperson_voice_control: changing the Expert's voice to new-voice leaves the Spokesperson's realtime voice at cedar and its speed at 1.25.

• A unit test checks that a new voice is built off the session loop. The request comes back once the voice is ready, a voice already built is used immediately, and an unknown voice is refused. A request cancelled while its voice is being built gets cancel_result / cancelled and not_admitted / cancelled, and is not revived when the voice is ready.

• menu_reflects_live_call_state: an Expert-Spokesperson call's menu has Agent Voice: Aaron and Agent Speech Rate: 1.25× next to Spokesperson Voice: marin and Spokesperson Speech Rate: 1× (0.5× to 1.5×). Choosing cedar sends {"command":"spokesperson","voice":"cedar","rate":null}.

• In a live Buzz call, two agents replying with speak --voice were heard in two distinct voices.

[morgmart]

🤖 Automated code review

REQUEST_CHANGES. Fresh static Engineering review of the exact prepared three-dot comparison found two blocking lifecycle defects in Expert-owned TTS replies. Admitted replies can enter playback before the parent accepts output readiness, and playback failures discard recorded delivery progress and finalize partially heard replies as unheard. Supplied GitHub evidence is structurally valid: all ten reported check runs succeeded while the combined status remains pending; required checks still govern merge readiness. Final self-check covered every changed user flow; accessibility and localization completeness; navigation and consent guards; async, error, never-completes, lifecycle, and race behavior; test honesty; project and design-system rules; duplicate and overlapping findings; and concrete evidence plus user effect for blocking severity.

Deterministic publication result: 2 blocking and 0 non-blocking inline finding(s) publishable; 0 duplicate(s) suppressed; 0 blocking screenshot-evidence requirement(s) in this review body.

[johnmatthewtennant]

🤖 Combined into #397 with its commits kept, so the stack no longer needs restacking and review covers one diff. Open review findings carry over there. (AI-generated comment)