speaker_helper.lexicon module

Pronunciation lexicon: per-word overrides applied before synthesis.

Module summary

Some words a document contains are not words the engine’s own grapheme-to-phoneme stage gets right: proper nouns (“Grasse” the town, not “grasse” the adjective), acronyms (“ONU”), anglicisms in French text, product names. This module lets whoever owns the text state, once, how such a word is pronounced — and applies that statement automatically on every synthesis.

The statement can take two forms, because the engines take two forms:

IPA (ipa:) — exact phonemes. Deterministic, but only an engine with a phoneme input can honour it. The kokoro backend does: it already phonemises text itself (espeak-ng → IPA) and hands the model phonemes, so a lexicon entry is spliced into that stream verbatim (see phonemise_with_lexicon()).
Respelling (respelling:) — an orthographic rewrite (“GitHub” → “guite-heube”) that an engine without phoneme access reads correctly. Fragile by nature but universal: it works by rewriting the text itself, so Chatterbox, Kyutai, and any future end-to-end model benefit.

When an entry states neither, the respelling is derived once by a local LLM — resolved through best-engine-ai-helper (the suite rule: no hardcoded model names) and served by Ollama at temperature 0 — then persisted to a sidecar cache next to the lexicon file. After that first derivation the whole feature is deterministic and fully offline.

Who applies which tier is decided by the backend’s LEXICON_TIER class attribute: "ipa" means “the backend splices phonemes itself, leave the text alone”; anything else (the default) means Speaker rewrites the text with Lexicon.respell() before calling the engine.

Usage example

>>> lex = Lexicon.from_mapping({"entries": [
...     {"word": "GitHub", "language": "*", "respelling": "guite-heube"},
... ]})
>>> lex.respell("Va sur GitHub ce soir.", "fr")
'Va sur guite-heube ce soir.'

Author

Warith HARCHAOUI — https://linkedin.com/in/warith-harchaoui

speaker_helper.lexicon.ANY_LANGUAGE = '*'

an entry carrying it applies to every language. Useful for brand names that are mangled the same way everywhere.

Type:: The wildcard language

speaker_helper.lexicon.CACHE_SUFFIX = '.llm-cache.yaml': Sidecar suffix for LLM-derived respellings. A separate file (not a rewrite of the user’s lexicon) because the lexicon is hand-edited and commented — regenerating it through a YAML dumper would destroy those comments.

class speaker_helper.lexicon.Lexicon(entries, *, path=None, llm_enabled=True)[source]

Bases: object

A set of pronunciation overrides, applied by word-boundary matching.

Parameters:

entries (list of LexiconEntry) – The overrides, in file order (later entries win on duplicate words).
path (Path or None) – Where the lexicon was loaded from — used to place the LLM sidecar cache. None keeps everything in memory (tests, ad-hoc use).
llm_enabled (bool) – Allow deriving missing respellings with the local LLM. Off means a word without a respelling is simply left as written.

ensure_respelling(entry, language)[source]

Return the entry’s respelling, deriving it with the LLM if needed.

Parameters:

entry (LexiconEntry) – The entry to complete. Mutated in place when a derivation succeeds (and persisted to the sidecar cache).
language (str) – Language of the text being synthesised — used when the entry is a wildcard, because a respelling is written in a language’s orthography even when the word applies to all of them.

Returns:

The respelling, or "" when none exists and none could be derived.

Return type:

str

entry_for(word, language)[source]

Return the entry matching word in language, if any.

Parameters:

word (str) – The written form as found in the text (any casing).
language (str) – Language of the surrounding text.

Returns:

The exact-language entry when one exists, else the "*" entry, else None.

Return type:

LexiconEntry or None

classmethod from_mapping(mapping, *, path=None)[source]

Build a lexicon from a plain mapping (the YAML document shape).

Parameters:

mapping (dict) – {"entries": [{word, language, ipa, respelling}, ...], "llm": bool} — unknown keys are ignored.
path (Path or None) – Origin file, forwarded for sidecar-cache placement.

Returns:

The populated lexicon.

Return type:

Lexicon

classmethod load(path)[source]

Load a lexicon from a YAML file.

Parameters:: path (str or Path) – The lexicon file (see lexicon.yaml.example at the repo root).
Returns:: The populated lexicon.
Return type:: Lexicon
Raises:: FileNotFoundError – When path does not exist — a configured-but-missing lexicon is a mistake worth hearing about, unlike an unconfigured one.

respell(text, language)[source]

Rewrite text so an engine without phoneme input says it right.

Every lexicon word found in text is replaced by its respelling — the hand-written one when the entry has it, otherwise one derived once by the local LLM and cached. A word whose respelling cannot be obtained (no LLM available) is left exactly as written.

Parameters:

text (str) – The text about to be synthesised.
language (str) – Language of the text.

Returns:

The rewritten text (identical to text when nothing matched).

Return type:

str

split(text, language)[source]

Cut text into plain chunks and lexicon matches, in order.

This is the primitive the IPA tier builds on: the phonemiser runs on the plain chunks, and each matched entry contributes its phonemes.

Parameters:

text (str) – The text to scan.
language (str) – Language of the text (decides which entries apply).

Returns:

Pairs in reading order. (chunk, None) is ordinary text; (matched_word, entry) is a lexicon hit, with the word as it appeared in the text.

Return type:

list of (str, LexiconEntry or None)

Examples

>>> lex = Lexicon.from_mapping(
...     {"entries": [{"word": "ONU", "language": "fr", "ipa": "oɛny"}]})
>>> lex.split("Vive l'ONU !", "fr")
[("Vive l'", None), ('ONU', LexiconEntry(word='ONU', ...)), (' !', None)]

class speaker_helper.lexicon.LexiconEntry(word, language='*', ipa='', respelling='', source='user')[source]

Bases: object

One pronunciation override: a word and how to say it.

Parameters:

word (str) – The written form to match (matched case-insensitively, on word boundaries).
language (str) – ISO-639-1 code the entry applies to, or "*" for every language.
ipa (str) – IPA phonemes for the word (no surrounding slashes). Honoured verbatim by phoneme-input backends; used as a hint when deriving a respelling.
respelling (str) – Orthographic rewrite for engines without phoneme access. Empty means “derive one with the local LLM on first use”.
source (str) – "user" for hand-written entries, "llm" for a derived respelling — kept so a cached derivation is distinguishable from a human decision.

ipa: str = ''

language: str = '*'

respelling: str = ''

source: str = 'user'

word: str

speaker_helper.lexicon.builtin_entries()[source]

The packaged per-language entries, language stamped from the filename.

Returns:: Every entry of every lexicons/<code>.yaml, with language set to <code> (the files do not repeat it) and source set to "builtin" so a merged lexicon stays auditable.
Return type:: list of LexiconEntry

speaker_helper.lexicon.load_lexicon(settings)[source]

Resolve and load the lexicon the settings point at, if any.

Resolution order for the file path: settings.lexicon["path"], then the SPEAKER_HELPER_LEXICON environment variable (already folded into settings by Settings), then a lexicon.yaml sitting in the current directory — mirroring how settings.yaml itself is found.

Parameters:: settings (Settings) – The configuration whose lexicon mapping drives this (keys: path, enabled, llm).
Returns:: The loaded lexicon, or None when the feature is disabled or no file exists (the no-lexicon path costs nothing per synthesis).
Return type:: Lexicon or None

speaker_helper.lexicon.phonemise_with_lexicon(text, language, lexicon, phonemise)[source]

Phonemise text, splicing lexicon IPA in place of matched words.

The deterministic tier: the engine’s own phonemiser runs on everything the lexicon does not claim, and each claimed word contributes, in order of preference, its ipa verbatim, the phonemisation of its respelling, or the phonemisation of the word itself (a no-op override).

Parameters:

text (str) – The text to phonemise.
language (str) – Language of the text (decides which lexicon entries apply).
lexicon (Lexicon or None) – The active lexicon; None degrades to a plain phonemise(text).
phonemise (callable) – str -> str, the backend’s own phonemiser for this language — passed in so this module depends on no backend.

Returns:

One phoneme string for the whole text.

Return type:

str

Examples

>>> lex = Lexicon.from_mapping(
...     {"entries": [{"word": "Grasse", "language": "fr", "ipa": "ɡʁas"}]})
>>> phonemise_with_lexicon("Grasse", "fr", lex, lambda s: f"<{s}>")
'ɡʁas'