feat(desktop): add message read-aloud playback - #3239
Closed
tellaho wants to merge 1 commit into
Closed
Conversation
Let a user explicitly play/stop a delivered agent response from the message action bar, reusing the local Pocket TTS stack while keeping read-aloud fully independent from huddle orchestration. Backend (src-tauri): - New app-scoped message_tts module with its own TtsPipeline instance and fresh active/cancel flags — zero STT/barge-in/huddle-phase coupling. Commands: speak_chat_message / stop_chat_message_speech. - Message-keyed lifecycle (preparing/playing/completed/stopped/ superseded/failed) emitted as message-tts-state-changed events. Generation tokens guarantee stale watcher completions can never overwrite a newer request; one playback app-wide, newer play supersedes older. - Huddle exclusion both ways: speak refuses while huddle phase is not Idle, and start/join_huddle stop any message playback before taking the audio device. - Long-message limit mirrors the huddle cap (2000 chars, char-counted) with a spoken suffix and a truncated flag so the cut is disclosed, not silent. Missing voice model kicks off the download and surfaces a retryable error. No persisted audio cache. Frontend (desktop/src): - MessageTtsProvider (mounted in AppShell inside HuddleProvider) folds backend events through a pure reducer (playbackReducer.mjs) so stale terminal events for superseded messages can't clear newer playback. Errors surface as toasts + per-message state; truncation is disclosed via toast at playback start. - ReadAloudButton in MessageActionBar for delivered agent text only (isAgent, not pending, has body), with aria-label/aria-pressed, tooltip, and disabled-with-reason during active huddles. Tests: 16 Rust unit tests (truncation UTF-8 safety, supersede/stop races, concurrent stops, event payload shape) and 13 node:test reducer tests (out-of-order supersede delivery, detached errors, rapid play/stop). Full suites green: cargo test (1576), pnpm test (3414), typecheck, biome + file-size checks, desktop build. Co-authored-by: Taylor Ho <taylorkmho@gmail.com> Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Category: new-feature
User Impact: Users can play or stop a spoken reading of delivered agent messages from the desktop message action bar.
Problem: Ordinary chat messages have no click-to-play speech path; Buzz's existing local TTS is coupled to active huddles. Solution: Add a separate app-scoped Pocket TTS controller with deterministic message-keyed playback, then expose it through the existing message action surface while preventing message and huddle audio from competing.
Note
This PR is stacked on #2467. Review only the changes above that PR; this remains an experiment until the base lands.
File changes
desktop/scripts/check-file-sizes.mjs
Adds the message TTS implementation to the repository's explicit large-file allowance.
desktop/src-tauri/src/app_state.rs
Owns message playback independently from huddle state at the application level.
desktop/src-tauri/src/huddle/mod.rs
Stops message playback before a huddle starts or joins so both flows cannot compete for audio.
desktop/src-tauri/src/lib.rs
Registers the message TTS module and its desktop commands.
desktop/src-tauri/src/message_tts.rs
Implements local Pocket synthesis, message-keyed lifecycle events, app-wide superseding playback, disclosed UTF-8-safe truncation, model recovery, and huddle exclusion.
desktop/src-tauri/src/message_tts_tests.rs
Covers truncation, lifecycle payloads, concurrent stop behavior, and stale completion races.
desktop/src/app/AppShell.tsx
Mounts message playback state around routed message surfaces while retaining access to huddle state.
desktop/src/features/message-tts/lib/playbackReducer.d.mts
Describes reducer event and state contracts for TypeScript consumers.
desktop/src/features/message-tts/lib/playbackReducer.mjs
Keeps frontend playback state deterministic when backend events arrive out of order.
desktop/src/features/message-tts/lib/playbackReducer.test.mjs
Exercises supersede, detached error, and rapid play/stop event sequences.
desktop/src/features/message-tts/ui/MessageTtsProvider.tsx
Bridges Tauri commands and events to message actions, truncation notices, huddle gating, and failure toasts.
desktop/src/features/messages/ui/MessageActionBar.tsx
Adds an accessible Play/Stop control for delivered agent text using the existing action-bar system.
Reproduction steps
Screenshot
Verification
pnpm test: 3,414 passedcargo test: 1,576 passed