Skip to content

Chat view: add a TTS button to read agent replies aloud - #505

Merged
powerfooI merged 2 commits into
powerfooI:mainfrom
srp:agent-chat-tts
Oct 26, 2026
Merged

powerfooI merged 2 commits into
powerfooI:mainfrom
srp:agent-chat-tts

Conversation

@srp

@srp srp commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Agent History Chat with Copy and Speak revealed beneath a genuine Pi reply

chat-controls.mp4

The screenshot and silent clip show a genuine Pi reply after reading the public Northstar demo. The clip shows controls appearing on hover, hiding when the pointer leaves, and appearing again with keyboard focus. It demonstrates the UI, not audible playback. Captured from 90385be in Chromium 156 against a real Herdr backend; desktop viewport 1100 x 860, screenshot DPR 2, video DPR 1.

Summary

Sometimes a long agent reply is easier to hear than read, especially on a phone. This adds a small Speak button beside Copy in Agent History Chat, using the browser's native speech synthesis.

  • Convert Markdown with the existing marked lexer: retain prose, short identifiers, link labels, image descriptions, and small tables; announce omitted code blocks, diagrams, and oversized tables instead of reading their syntax.
  • Read the complete reply as a snapshot, in sequential chunks. Stop immediately when requested, when another reply is chosen, or when the chat closes or changes session. No automatic narration.
  • Reveal Speak on desktop hover or keyboard focus, whilst keeping Stop visible during playback. On mobile the control stays visible.
  • Save browser voice and speaking rate under Configuration > Behavior > Speech, with voice-list updates and graceful fallback when a saved voice is unavailable.

There are no new dependencies, server endpoints, or external speech integrations. The formatter remains separate from playback. Existing asset budgets are unchanged.

Compatibility and limitations

Speech depends on the browser and its installed voices. Unsupported browsers show a disabled control. Some voices use online services and may send reply text to their provider; Settings and the feature documentation state this. Background and locked-screen playback are not guaranteed. No microphone permission is required.

Verification

  • bun run precommit — formatting, Oxlint, TypeScript checks, frontend build, and the complete test suite: 3,929 passed, 4 optional tests skipped, 0 failed.
  • bun run typecheck:bun — all three projects passed.
  • bun run build:web — passed, including asset budgets.
  • Apple Silicon trial binary built with bun run build:darwin-arm64.
  • Formatter, mocked playback, preference persistence, mounted UI lifecycle, and compiled stylesheet tests cover the new behaviour.
  • Real Herdr-backed Chromium checks passed for desktop idle/hover visibility, keyboard focus, mobile layout, and saved rate changes. The headless capture environment has no installed voices, so it does not establish audible playback.
  • The contributor reports successful playback using the trial binary on an Apple Silicon Mac; the browser/version was not recorded. Physical iOS and Android playback, and Safari/Firefox/Brave-specific behaviour, remain unverified.

Oxlint reports one pre-existing TerminalView.tsx:689 warning. Vite also reports its existing large-chunk advisory; neither check fails.

Speech settings

Browser voice and speaking-rate preferences

Mobile layout

Speak and Copy remain visible in the mobile chat layout

Chromium viewport 390 x 844, DPR 2; this establishes layout only, not physical-device playback or keyboard behaviour.

Sometimes on mobile it is not convenient to read a reply, especially a
long one. Add a play button that uses the browser's native speech to read
the reply aloud.

Convert Markdown into speech-friendly text first, preserving prose and
short identifiers whilst announcing omitted code blocks, diagrams, and
large tables. Split long replies into smaller chunks for playback.

Let users choose a voice and speaking rate in Settings. Keep Stop visible
during playback, and stop reading when another reply is selected or the
chat is closed.
@powerfooI
powerfooI merged commit e5b9ca0 into powerfooI:main Oct 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants