Skip to content

feat(mobile): add Android system TTS fallback - #3297

Closed
johnmatthewtennant wants to merge 20 commits into
jtennant/mobile-pocket-ttsfrom
jtennant/mobile-android-system-tts-fallback
Closed

feat(mobile): add Android system TTS fallback#3297
johnmatthewtennant wants to merge 20 commits into
jtennant/mobile-pocket-ttsfrom
jtennant/mobile-android-system-tts-fallback

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Context

Pocket remains the preferred on-device voice backend, but voice currently becomes unavailable when its model is absent, cannot load, synthesis fails, or Android asks the app to release memory.

This PR is stacked on #3290 at exact head 4691cad0c0afa153c7a4ce4d2333340e26b3d9c2 (itself on #3303 at d9193add61f4246d0ebb408fbf64470c72aacd88).

Summary

Adds an Android-only fallback to the installed default TextToSpeech engine while preserving one ordered Dart-owned speech queue. Pocket is selected whenever it is ready; successful fallback is silent in the UI, while backend and fallback reason remain visible in controller state for diagnostics.

Changes

  • Adds generation-fenced Android system speech using the default engine, language, voice, and rate.
  • Preserves stop, disable, interruption, audio focus, noisy-route, background/foreground, cancellation, and cleanup behavior across backend transitions.
  • Preserves feat(mobile): add April INT8 Pocket TTS #3290's structured load/synthesis/playback failures and reports started only after a backend-and-generation-matched playback start.
  • Hands only unspoken Pocket chunks to system speech after a synthesis failure, preventing repeats.
  • Releases Pocket under real Android memory pressure, drains emitted PCM before fallback, and retries Pocket only at safe boundaries.
  • Splits long system utterances below Android's input limit without reordering the conversation.
  • Adds deterministic Dart and Kotlin coverage plus Android 15 instrumentation coverage for default-engine initialization, speech completion, and cancellation.

Related issue

No duplicate Android issue or PR found. The closest related work is the iOS-only system voice fallback in #3296.

Testing

  • just ci
  • just mobile-check
  • just mobile-test — 857 tests passed
  • flutter test test/shared/voice/pocket_voice_controller_test.dart — 18 focused tests passed
  • Android JVM tests: SystemTtsCallbackFenceTest
  • Isolated Android 15 AVD BuzzAndroidTTS-019fa858:
    • installed default Google TTS service discovered without an engine override
    • installedDefaultEngineInitializesWithItsDefaultLanguage passed
    • voiceBoundarySpeaksAndCancelsWithGenerationFencedCallbacks passed
    • 2 tests, 0 failures; the isolated AVD was shut down afterward
  • Directly affected restack reviews: NCMB keep, Stacksmith approve, minimize/dead-code clean

The prior Mobile CI failure was from the old stack lacking the aarch64-linux-android Rust target; #3290's current head installs the Android Rust targets before this child builds.

Manual audio gate: the headless emulator verified the real engine's completion callback and cancellation fence, but audible host output was not human-confirmed. On an Android device, leave Pocket unavailable, start voice, verify an assistant reply is audible through the system-default voice, then stop during a second reply and confirm it does not resume.

Screenshots

N/A — audio backend and lifecycle behavior; no visual UI change.

Reviewer-reproducible examples

  1. With Pocket ready, start voice and confirm replies use Pocket.
  2. Remove or disable the Pocket model, start voice, and confirm replies use the installed Android default TTS voice without an error toast.
  3. While system speech is active, press Stop or cause transient audio-focus loss; confirm speech stops and does not replay.
  4. Restore Pocket, foreground the app, and confirm the next queued reply returns to Pocket at an utterance boundary.
  5. Run SystemTextToSpeechInstrumentedTest from Android Studio on a named disposable Android 15 AVD.

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-pocket-tts branch from 199f931 to c0e3080 Compare July 28, 2026 13:17
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-android-system-tts-fallback branch from 335d754 to 535fa1a Compare July 28, 2026 13:54
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-pocket-tts branch from c0e3080 to e4ec043 Compare July 28, 2026 14:03
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-android-system-tts-fallback branch from 535fa1a to 46349bc Compare July 28, 2026 14:07
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-pocket-tts branch from e4ec043 to 4691cad Compare July 28, 2026 14:38
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-android-system-tts-fallback branch from 46349bc to d490d4f Compare July 28, 2026 14:44
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/mobile-pocket-tts branch from 4691cad to 6b1e6f1 Compare July 28, 2026 20:06
@johnmatthewtennant

Copy link
Copy Markdown
Contributor Author

🤖 Closing for now: Pocket TTS is the sole active voice backend, so the Android system TTS fallback is no longer in scope. The branch is preserved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant