docs: fix Rime PRONOUNCE example and add bulk pronunciation lexicon guidance (T-3868) - #1245
Draft
jamsea wants to merge 2 commits into
Draft
docs: fix Rime PRONOUNCE example and add bulk pronunciation lexicon guidance (T-3868)#1245jamsea wants to merge 2 commits into
jamsea wants to merge 2 commits into
Conversation
Two documentation gaps found while working support ticket T-3868 (a customer needing 300 to 1000 medication pronunciation overrides). Rime TTS page: - PRONOUNCE and INLINE_SPEED are instance methods, but both examples called them on the class. As written they raise a TypeError. Both now call the method on the service instance. - Note that PRONOUNCE takes three arguments, so it needs a small wrapper function to be used as a text transform. - Note that PRONOUNCE only turns on bracket phonemization for the next message, and point to the Settings field that keeps it on for the whole session. - Link to the new bulk lexicon section for large pronunciation lists. Text-to-Speech guide, new "Bulk pronunciation overrides" section: - replace_text as the portable client side path, with a rough sense of its cost as the rule count grows. - Warn that text transforms run after aggregation, so multi-word replacements cannot match in TOKEN mode and need sentence aggregation. - Table of provider hosted lexicon options (AWS Polly, NVIDIA, Cartesia, ElevenLabs, Azure). - Warn that NVIDIA custom_dictionary entries cannot contain commas. - Note that Azure has no lexicon path, since it escapes text before building SSML, so an injected phoneme tag is read out literally. - Recommend respellings over IPA on ElevenLabs, since respellings work on every model. Regenerated llms.txt and llms-full.txt.
Contributor
|
🔍 Mintlify preview for this branch: https://daily-docs-t-3868-pronunciation-lexicons.mintlify.site |
Contributor
|
This seems like guide information, not in the learn section, as it's not required for all cases. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two documentation gaps found while working support ticket
All claims below were verified against
pipecatmain at0417f251.Gap A: the Rime
PRONOUNCEexample cannot runThe page showed
RimeTTSService.PRONOUNCE(text, "potato", "potato"), called on the class with 3 arguments.PRONOUNCEis an instance method taking(self, text, word, phoneme). The@staticmethoddecorator above it bindsPAUSE_TAG, notPRONOUNCE. As documented, the call bindsself=textand leavesphonememissing, so it raisesTypeError.INLINE_SPEEDis an instance method too, and its example on the same page had the identical problem. Fixed both.Also added:
PRONOUNCEtakes three arguments, so it needs a small wrapper function to satisfy the(text, aggregation_type)text transform contract.PRONOUNCEenables bracket phonemization for the next message only, with a pointer to theSettingsfield that keeps it on for the whole session. That is the better choice for anything beyond a one-off.Gap B: no guidance on bulk pronunciation lexicons
Nothing in the docs told you how to do a lexicon of hundreds of words, or which providers can host one. New "Bulk pronunciation overrides" section in the Text-to-Speech guide covering:
replace_textas the portable client side path. Rules are regexes compiled once at construction and applied as a loop per chunk, so cost grows linearly and stays small. Measured roughly 0.4 ms per sentence at 300 rules and 1.4 ms at 1000 on an Apple silicon Mac. Presented as an order of magnitude, with hardware variance called out.The footgun: transforms run after aggregation. In
TextAggregationMode.TOKEN,SimpleTextAggregatoryields each text immediately with no buffering, so a pattern spanning multiple words can never match. Multi-word lexicon entries require sentence aggregation. This is the single most useful thing on the page. Related: Token-mode text transforms cannot match across token boundaries — bounded lookback (LiveKit-style) instead of full sentence aggregation pipecat#5574.Provider hosted lexicon table, checked by grepping every
src/pipecat/services/*/tts.py:lexicon_namescustom_dictionary, with a warning that entries are joined into one comma separated string with no escaping, so a comma in an entry corrupts the dictionarypronunciation_dict_idpronunciation_dictionary_locators, deprecated in 1.6.0 and removed in 2.0.0_construct_ssmlescapes the input text before assembling SSML, so an injected<phoneme>tag is escaped and spoken literally, and there is no opt-out. Worth stating, since the page lists "SSML support: Fine-grained pronunciation control" as an advanced feature and people try it on Azure.ElevenLabs specifics: prefer respellings over IPA, since respellings work on every model while IPA
<phoneme>tags needenable_ssml_parsing=Trueand only work on v2 models.Checks run
npx mint broken-links --check-anchors --check-redirects: no broken linksnode scripts/check-imports.mjs --pipecat ../pipecat: all imports resolvenode scripts/docs-meta-lint.mjs: the one error and the llms.txt budget warning both reproduce onmainunchanged, so neither comes from this branchllms.txtandllms-full.txtregeneratedNote:
npm run formatis not a no-op onmainright now, it reformats about 30 unrelated files. I reverted those so this diff stays scoped.Opened as a draft for review.