Skip to content

refactor(voice): extract reusable Pocket voice primitives - #2467

Draft
johnmatthewtennant wants to merge 9 commits into
block:mainfrom
johnmatthewtennant:jt/buzz-voice-refactor
Draft

refactor(voice): extract reusable Pocket voice primitives#2467
johnmatthewtennant wants to merge 9 commits into
block:mainfrom
johnmatthewtennant:jt/buzz-voice-refactor

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Context

Buzz Desktop owned Pocket TTS inference inside huddle, so other Buzz clients had to copy engine configuration and prompt behavior. The original extraction at 3e9dd6d3dacc6011838cec1302e4301039355378 passed host validation, but unchanged consumers failed for both iOS and Android because the crates.io sherpa-onnx-sys build script rejects mobile targets.

The mobile support here is a general buzz-voice packaging and API contract. It does not add a Buzz Mobile bridge, app behavior, model, or native binary.

Summary

This PR moves the existing Pocket engine and tests into a reusable buzz-voice workspace crate, preserves the existing synchronous API, and adds:

  • explicit static and shared native-link features, with static remaining the default;
  • a backwards-compatible cancellable synthesis callback;
  • a provenance-preserving local patch of sherpa-onnx-sys 1.13.4;
  • fail-closed mobile library resolution through caller-supplied SHERPA_ONNX_LIB_DIR.

Desktop keeps its existing huddle module surface through a compatibility re-export.

Changes

  • Adds crates/buzz-voice with the existing Pocket engine implementation and 13 tests.
  • Re-exports the shared engine through desktop/src-tauri/src/huddle/pocket.rs.
  • Adds synth_chunk_with_callback(text, lang, style, steps, Option<F>); synth_chunk remains unchanged and delegates with no callback.
  • Maps buzz-voice features to sherpa-onnx/static and sherpa-onnx/shared with upstream defaults disabled.
  • Carries the crates.io sherpa-onnx-sys 1.13.4 FFI sources, license, and Cargo.toml.orig; only its build script and a short README provenance note differ intentionally.
  • Links caller-supplied official iOS static archives/frameworks and Android shared libraries plus log.
  • Keeps desktop archive download, caching, runtime copying, and link behavior unchanged.
  • Keeps the patched sys crate at its published version during Docker dependency cooking instead of allowing cargo-chef to mask it as a workspace 0.0.1 package.
  • Runs the moved tests in both normal unit-test runner paths.

No native libraries or models are bundled.

Related issue

None found.

Testing

Red/green mobile evidence:

  • At 3e9dd6d3dacc6011838cec1302e4301039355378, the same downstream Berd bridge passed host tests but failed iOS and Android preparation with Unsupported target for sherpa-onnx prebuilt libs.
  • Berd Mobile draft PR Add desktop Home feed #12 then pinned the incorporated voice commits and produced fresh real-device results: iOS loaded in 413 ms and synthesized 170,880 non-silent samples in 1,859 ms; Android loaded in 1,527 ms and synthesized 184,320 non-silent samples in 4,210 ms. Both outputs were 24 kHz PCM.
  • Buzz Mobile draft PR feat(mobile): add on-device Pocket TTS #3290 is based directly on this canonical extraction after removing its duplicate API/package implementation; it retains only the app-facing C ABI, platform build integration, downloader, audio session, isolate, and UI behavior.
  • At 41b06a494ebd49ee0b4ea8435afab725d669b01f, all four Docker builds failed because cargo-chef 0.1.71 rewrote the workspace copy of sherpa-onnx-sys to version 0.0.1, conflicting with the registry package's unique links = "sherpa-onnx".
  • An intermediate fix recovered the relay-image builds but exposed the push-gateway's independent cargo-chef builder, which failed because the excluded patched crate was absent from its cook filesystem.
  • The final canonical head is 91eaee8804ec0c139a53782212df44a70aa994c1. Both Docker builders now keep the patch outside the masked workspace recipe and copy its real 1.13.4 source tree before dependency cooking. Downstream repins and platform revalidation are being coordinated against this SHA.

Local gates on the frozen voice/package diff (unchanged by the final Docker-only commits):

  • cargo fmt --all -- --check: passed.
  • cargo test --locked -p buzz-voice --all-targets: 13 passed.
  • Static and shared host cargo check: passed.
  • cargo clippy for buzz-voice static and vendored sherpa-onnx-sys with -D warnings: passed.
  • iOS device and arm64 simulator target checks with official 1.13.4 libraries: passed.
  • Android arm64-v8a and x86_64 target checks with official 1.13.4 libraries: passed.
  • Final staticlib/cdylib link probes for all four mobile targets: passed.
  • cargo package for buzz-voice and sherpa-onnx-sys: passed.
  • cargo deny check licenses: passed.
  • Pre-push Rust, desktop, and Tauri suites: passed, including 3,721 desktop tests and 1,804 Tauri/diagnostic tests.
  • The exact cargo-chef 0.1.71 failure was reproduced locally; with the real patched tree copied into the cook filesystem, resolution advanced past the sherpa-onnx-sys conflict. Final GitHub Docker builds are the authoritative image gate.
  • DCO trailers: present on every added commit.

Screenshots

N/A. This is a Rust API and native-link packaging change with no UI.

Reviewer-reproducible examples

Run the extracted engine tests:

cargo test --locked -p buzz-voice --all-targets

Exercise both host link modes:

cargo check --locked -p buzz-voice --all-targets \
  --no-default-features --features static
cargo check --locked -p buzz-voice --all-targets \
  --no-default-features --features shared

With an official sherpa-onnx 1.13.4 mobile library directory prepared by the caller:

SHERPA_ONNX_LIB_DIR=/absolute/path/to/iphoneos-arm64 \
  cargo check --locked -p buzz-voice --target aarch64-apple-ios \
  --no-default-features --features static

SHERPA_ONNX_LIB_DIR=/absolute/path/to/arm64-v8a \
  cargo check --locked -p buzz-voice --target aarch64-linux-android \
  --no-default-features --features shared

Cross-repository consumers must pin both the public crate and the patched sys crate to the same canonical SHA:

[dependencies]
buzz-voice = { git = "https://github.com/block/buzz", rev = "91eaee8804ec0c139a53782212df44a70aa994c1", default-features = false, features = ["shared"] }

[patch.crates-io]
sherpa-onnx-sys = { git = "https://github.com/block/buzz", rev = "91eaee8804ec0c139a53782212df44a70aa994c1" }

@johnmatthewtennant johnmatthewtennant changed the title Extract reusable Buzz voice primitives refactor(voice): extract reusable Buzz voice primitives Jul 27, 2026
@johnmatthewtennant johnmatthewtennant changed the title refactor(voice): extract reusable Buzz voice primitives refactor(desktop): extract reusable Buzz voice primitives Jul 27, 2026
@johnmatthewtennant johnmatthewtennant changed the title refactor(desktop): extract reusable Buzz voice primitives refactor(desktop): extract reusable Pocket voice primitives Jul 27, 2026
@@ -0,0 +1,654 @@
//! Pocket TTS engine wrapper around sherpa-onnx's `OfflineTts`.

@johnmatthewtennant johnmatthewtennant Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This implementation is mechanically moved from desktop/src-tauri/src/huddle/pocket.rs at exact parent d8f9d87c17131b952ea5b6c3767978c4637545fc. Reproduce the whitespace-tolerant comparison from the repository root:

diff -uw \
  <(git show d8f9d87c17131b952ea5b6c3767978c4637545fc:desktop/src-tauri/src/huddle/pocket.rs) \
  crates/buzz-voice/src/pocket.rs

Actual residual diff:

(no output)

Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
* origin/main:
  Publish symbol-bearing debug relay images (block#3250)
  feat(tracing): add datastore tracing plumbing (block#2760)
  fix(buzz-acp): accept id-keyed config options when resolving model switch (block#2795)
  fix(desktop): probe legacy Goose install dir on Windows (block#3248)
  refactor(desktop): extract install command execution into install_exec (block#3251)

Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant johnmatthewtennant changed the title refactor(desktop): extract reusable Pocket voice primitives refactor(voice): extract reusable Pocket voice primitives Jul 28, 2026
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant