Skip to content

feat(desktop): upgrade Pocket TTS to April model - #3266

Draft
johnmatthewtennant wants to merge 6 commits into
jt/buzz-voice-refactorfrom
jtennant/pocket-tts-2026-04
Draft

feat(desktop): upgrade Pocket TTS to April model#3266
johnmatthewtennant wants to merge 6 commits into
jt/buzz-voice-refactorfrom
jtennant/pocket-tts-2026-04

Conversation

@johnmatthewtennant

Copy link
Copy Markdown
Contributor

Context

Buzz Desktop currently installs the January 2026 Pocket TTS model. Pocket TTS v2.1.0 ships an April English model with a different SentencePiece tokenizer, learned BOS embedding, prompt policy, and recurrent-state manifest, so changing download URLs alone is not compatible.

Summary

This PR replaces the January Pocket TTS model with the pinned full-precision english_2026-04 export. Buzz keeps sherpa-onnx 1.13.4 as its shared ONNX runtime and adds the April-specific native frontend and state loop needed by the new bundle.

The PR remains draft pending manual listening of the raw, untrimmed onset-stress clip.

Changes

  • Pins the April ONNX bundle, Mary reference voice, CC BY 4.0 license, and SHA-256 checksums to immutable upstream commits.
  • Loads April bundle schema 2, SentencePiece tokens, learned BOS conditioning, and declared recurrent state tensors.
  • Applies April prompt behavior without the January model's leading-space prefix.
  • Splits oversized prompts at the bundle's exact 50-token boundary in the desktop worker, preserving per-chunk fades and barge-in checks.
  • Bumps the TTS model manifest from 3 to 4 and atomically replaces a verified January cache with April, while retaining the previous usable cache if installation is interrupted.
  • Adds a standalone onset-stress generator that preserves raw phrase beginnings without trimming, fade, gain, or normalization.
  • Removes the obsolete desktop probes that were hard-coded to January model filenames and precision comparisons.

Related issue

None found.

Testing

  • cargo test --locked -p buzz-voice --lib: 12 passed, 2 model-dependent tests ignored.
  • Model-dependent SentencePiece parity and exact 50-token splitting tests: passed, including quizzaciously xyzzy.
  • cargo test --locked --manifest-path desktop/src-tauri/Cargo.toml huddle::models::tests: 2 passed.
  • cargo test --locked --manifest-path desktop/src-tauri/Cargo.toml huddle::tts::tests: 43 passed.
  • cargo check --locked --manifest-path desktop/src-tauri/Cargo.toml: passed.
  • cargo check --locked --manifest-path desktop/src-tauri/Cargo.toml --all-targets: passed.
  • cargo clippy --locked -p buzz-voice --all-targets -- -D warnings: passed.
  • Release-mode synthesis loaded the pinned FP32 April bundle and produced 362,880 samples (15.120 seconds) at 24 kHz.
  • Raw clip SHA-256: 4002a93db353e8770ac6ceba4969b3dd5bc590f43bc533d3c9747e8e3f527e1b.

Manual listening is intentionally still pending. Automated checks do not satisfy that gate.

Screenshots

N/A. This changes model installation and speech synthesis, not a visual surface.

Reviewer-reproducible examples

Before and after model identity

From the repository root:

git show origin/jt/buzz-voice-refactor:desktop/src-tauri/src/huddle/models.rs \
  | sed -n '39,105p' \
  | grep -E 'sherpa-onnx-pocket-tts|TTS_MODEL_VERSION'

sed -n '39,105p' desktop/src-tauri/src/huddle/models.rs \
  | grep -E 'english_2026-04|TTS_MODEL_VERSION'

Target-branch output identifies the January bundle and manifest version 3:

"https://huggingface.co/csukuangfj2/sherpa-onnx-pocket-tts-2026-01-26/..."
const TTS_MODEL_VERSION: &str = "3";

PR-branch output identifies the pinned April bundle and manifest version 4:

"https://huggingface.co/KevinAHM/pocket-tts-onnx/resolve/58a6d00.../onnx/english_2026-04";
const TTS_MODEL_VERSION: &str = "4";

End-to-end raw synthesis

Install the Pocket model from Buzz Desktop Settings on this branch, then run:

. ./bin/activate-hermit
cargo run --locked --release -p buzz-voice --example pocket_april_onset -- \
  /path/to/installed/pocket-tts \
  /tmp/pocket-tts-april-onset-stress-raw.wav
shasum -a 256 /tmp/pocket-tts-april-onset-stress-raw.wav

The generator makes each sentence a separate model call and inserts only 250 ms of digital silence between outputs. It does not trim or post-process onset artifacts. A successful run prints all nine explicit boundaries and writes a mono 24 kHz PCM WAV.

Listen with headphones and record any affected boundary numbers:

  • Confirm no phrase start is missing.
  • Confirm no first phoneme is distorted or smeared.
  • Confirm no click, pop, or burst occurs at a generation boundary.
  • Pay particular attention to Hello, What, I'm, I've, Try, Please, Start, and Sounds.

The PR must remain draft until John records this listening result and separately authorizes ready-for-review.

Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-2026-04 branch from 51b1513 to ec97128 Compare July 28, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant