Repository navigation
Speak with Chatterbox and Pocket TTS, with predefined voices - #1792
Merged
Merged
Conversation
Generated from llamadart-native main eb760895 (llama.cpp cd959bae) headers, packaged as the release workflow does, with native_header_archive.py and ffigen in a Linux container. The same procedure reproduces the committed v0.8.0-3 bindings byte for byte.
…type opt-in Drive llama_dart_tts_get_info_ext/init_ext/start_with_voice with the mask of types this core handles and the runtime builds, list predefined voices when the runtime exports the voice API, and add Lfm3AudioTtsAdapter, SopranoTtsAdapter, TextToSpeechRequest.voice and TextToSpeechCapabilities.supportsVoice/voices. TextToSpeechEngine.load asks the backend which models its runtime drives and refuses an adapter none of whose models it drives before loading anything, naming the required llamadart-native version. Refs #1615.
Chatterbox Turbo and Multilingual and Pocket TTS speak through the native model-type opt-in once llamadart-native v0.9.0 reports types 4 and 5. Pocket TTS lists its projector's voices; Chatterbox takes `default` or a speaker reference, and Multilingual a language and a two-sequence context. Older runtimes refuse before loading, naming v0.9.0. Refs #1783.
This was referenced Nov 8, 2026
…rano # Conflicts: # website/docs/platforms/support-matrix.md
…-pocket # Conflicts: # website/docs/platforms/support-matrix.md
13 of 14 tasks
…b backends The llama.cpp worker, its backend client, and the Web and WebGPU backends each answer the pre-load probe; nulling any of their answers now fails a test. Refs #1615.
…ox-pocket # Conflicts: # README.md # doc/testing_matrix.md # example/basic_app/README.md # example/basic_app/bin/llamadart_text_to_speech_example.dart # lib/src/backends/backend.dart # lib/src/backends/llama_cpp/llama_cpp_service.dart # lib/src/core/speech/text_to_speech_model.dart # skills/llamadart-speech/SKILL.md # test/e2e/backends/llama_cpp_exit_teardown_e2e_test.dart # test/e2e/backends/text_to_speech_models_e2e_test.dart # test/fixtures/llama_cpp_exit_probe.dart # test/integration/backends/llama_cpp_tts_model_types_test.dart # test/unit/core/speech/text_to_speech_model_test.dart # tool/testing/run_local_e2e.dart # tool/testing/test_matrix.dart # website/docs/getting-started/model-families.md # website/docs/guides/text-to-speech.md # website/docs/platforms/support-matrix.md
leehack
marked this pull request as ready for review
November 8, 2026 14:10
Contributor
|
Chat app preview removed for |
This was referenced Nov 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ChatterboxTtsAdapter,PocketTtsAdapter) through the native model-type opt-in, using llamadart-native v0.9.0 types 4 and 5. Refs Expose Chatterbox and Pocket TTS voices from native v0.9.0 #1783.default, the 25 Pocket TTS voices. Speaker-reference cloning works for Chatterbox (and for Pocket GGUFs that ship a voice encoder), and Multilingual takeslanguage.LlamaUnsupportedExceptionnamingllama_dart_tts_supported_model_typesand llamadart-native v0.9.0.mainnow that Speak with LFM3-Audio and Soprano through the native model-type opt-in #1791 (Expose LFM3 and Soprano speech output from native v0.8.0 #1615) is merged;mainpins llamadart-native v0.9.0. The diff againstmainis only this PR's families plus the probe tests below.worker_test.dart,llama_cpp_backend_test.dart), and the Web and WebGPU backends refuse non-Qwen3 models before loading and accept Qwen3-TTS (Chrome,web_backend_test.dart,webgpu_backend_test.dart). Each fails when its code path is nulled.Production-readiness scope
defaultor speaker-reference cloning, Multilinguallanguage; Pocket TTS speech with 25 predefined voices. The example CLI gains the two adapters plus--speakerand--sequences.llama_dart_tts_supported_model_types(llamadart-native v0.9.0 or later).loadthrowsLlamaUnsupportedExceptionnaming the symbol and v0.9.0.ModelParams.maxParallelSequencesof at least 2 (the classifier-free-guidance twin sequence).LlamaTextToSpeechExceptionbefore synthesis. Turbo and Pocket TTS:languagethrowsLlamaUnsupportedException. Pocket without a voice encoder: speaker reference throwsLlamaUnsupportedException.loadrefuses these families (Qwen3-TTS only).Completeness checklist
text-to-speech-models-smokeexemption reason from PR1.PR type guidance
Test Plan
dart run tool/prepare_workspace.dartdart format --output=none --set-exit-if-changed .dart analyzedart test -p vm -j 1 --exclude-tags local-only: full suite 11518 pass, 129 skip, 6 fail (allwindows_registry_qualification_test.dart: nopwshon this host) at 672b0de; at this head the TTS, worker, backend, entrypoint, structure and tooling suites pass (692 tests), and PR CI runs the full suite.dart test -p chrome --exclude-tags local-only: full suite 4626 pass, 1 skip before the retarget; at this head the Web, WebGPU and speech suites pass (391 tests), and PR CI runs the full suite.coverage-libmatrix row: not run locally;mainenforces it after merge.text-to-speech-models-smoke,llama-cpp-exit-teardown(quit-speaking), version skew on v0.8.0-3.Matrix Evidence
mainwith #1791vera; Multilingualfrgives "Bonjour, comment allez-vous aujourd'hui?".main(#1784 landed, pinv0.9.0); no local native overridemale, Soprano, Chatterbox Turbo default and cloned, Multilingual default and cloned, Pocket default andvera; Multilingualfrgives "Bonjour, comment allez-vous aujourd'hui?". Run at 672b0de, before the retarget.vera; Multilingualfrgives "Bonjour, comment allez-vous aujourd'hui?". Pocket lists 25 voices; no voice equals the first voice; Multilingual refuses a one-sequence context.LlamaUnsupportedExceptionnaming v0.9.0 before loading; LFM3/Soprano still speak.exit()mid-synthesis{"quit-speaking":[0,0,0]}for Chatterbox Multilingual and Pocket TTS._TtsApitest/integration/backends/llama_cpp_tts_model_types_test.dart,text_to_speech_model_test.dart.mainafter merge.Review Notes
99891bf160a1f2dbe7ab9de00812d128942495b6against basece8ddf674200297e5883d74e8888552c4af612b1; 0 P1/P2.high_risk_readiness.dartexits 2 (unverifiedPrerequisites).99891bf160(merge ofmainafter Speak with LFM3-Audio and Soprano through the native model-type opt-in #1791 landed as ce8ddf6), PR CI https://github.com/leehack/llamadart/actions/runs/42484282344 green: 34 checks pass, 9 skipped. The probe-test commit b161d90 also passed a dispatched run: https://github.com/leehack/llamadart/actions/runs/42483386669.High-risk regression review
backendRuntime,regressionPolicy).-j 4), Chrome 383, coverage and skills 48.99891bf160a1f2dbe7ab9de00812d128942495b6/ce8ddf674200297e5883d74e8888552c4af612b1.supportsModel, probe routing through worker, backend,_runtimeRefusal, Web, WebAutoBackend, WebGpu); real v0.8.0 refuses Chatterbox and Pocket before load. The integration suite fails if the type mask drops types 4/5 or the Multilingual sequence check is bypassed.maxParallelSequences: 2; Soprano was not run on real weights.