speaker_helper.backends.kyutai module
Kyutai TTS: native French speech from a Paris lab, offline, voice by embedding.
Module summary
Kyutai’s tts-1.6b-en_fr (delayed-streams modeling) is the engine this
package reaches for when the text is French and has to sound French: the
model is trained by a French lab on French data, and the 2026-10 study
(STUDY.md on the research branch) found it the best-documented native
French open model — where Chatterbox renders French through an English-first
pipeline and Kokoro’s single French voice is a stated grade B-.
Voices are precomputed embeddings from the kyutai/tts-voices
repository (CC-BY-4.0 / CC0 banks), fetched by name on first use. That is a
deliberate anti-abuse policy by Kyutai: the speaker encoder is unreleased, so
this backend selects voices and does not clone them —
KyutaiEngine.clone_voice() says so instead of pretending.
Two implementations, one backend
The same weights run through two runtimes, picked by the device knob:
MLX (
moshi-mlx, Kyutai’s own Apple-Silicon implementation) — the Metal GPU path.device: "auto"(the default) selects it whenos-helpersays this is a Mac andmoshi_mlxis importable, because the CPU fallback measured ~8× slower than real time on an M2 Max while the MLX path is the operating point Kyutai themselves ship for Macs.PyTorch (
moshi) — everywhere else, on the device named by the knob ("cpu","cuda"…).
Both load the same Hugging Face files (~3 GB, cached once, offline after).
The weights download on the first synthesis and are cached; after that the engine is fully offline. Sampling uses a temperature (the MLX path pins the upstream seed, so it is reproducible run to run; the PyTorch path is not).
This backend has no phoneme input (plain text in), so pronunciation-lexicon
entries reach it through the respelling tier applied by
Speaker — see speaker_helper.lexicon.
Usage example
>>> from speaker_helper import Speaker, Settings
>>> # spk = Speaker(Settings.from_mapping({"backend": "kyutai", "language": "fr"}))
>>> # await spk.say("Bonjour depuis Paris.")
- speaker_helper.backends.kyutai.DEFAULT_REPO: str = 'kyutai/tts-1.6b-en_fr'
Kyutai’s 1.6B English+French TTS.
- Type:
The default model repository
- speaker_helper.backends.kyutai.DEFAULT_VOICE_BY_LANG: dict[str, str] = {'en': 'unmute-prod-website/p329_022.wav', 'fr': 'cml-tts/fr/10087_11650_000028-0002_enhanced.wav'}
Default female voice per language, as names in the voice repository (the
.safetensorsembedding suffix is appended by the runtime).The French default is a CML-TTS speaker (CC BY 4.0), picked by measurement in the speak A/B study (2026-10-26): screened against four other female CML voices and unmute’s
developpeuse-3, it scored highest on UTMOSv2 by a wide margin (≈3.4 vs ≈2.0 for developpeuse-3 on the same sentences, F0 ≈ 226 Hz — female).developpeuse-3(unmute-prod-website/developpeuse-3.wav, CC0) remains a fine lighter -licence alternative viavoice=.p329_022is VCTK speaker p329, a female English voice.
- speaker_helper.backends.kyutai.DEFAULT_VOICE_REPO: str = 'kyutai/tts-voices'
The default voice-embedding repository.
- class speaker_helper.backends.kyutai.KyutaiEngine(settings)[source]
Bases:
objectThe Kyutai TTS backend: text in, native French (or English) out.
- Parameters:
settings (Settings) –
voicemay name a voice from thekyutai/tts-voicesrepository (or a local.safetensorsembedding path); empty picks the female default for the language. Thekyutaimapping supplies the knobs:repo— model repository (defaultDEFAULT_REPO).voice_repo— voice-embedding repository (defaultDEFAULT_VOICE_REPO).device—"auto"(default: Kyutai’s MLX runtime on a Mac when installed, torch CPU elsewhere),"mlx", or a torch device.dtype— torch path only:"float32"(default) or"bfloat16".cfg_coef— classifier-free-guidance strength (default2.0; higher follows the voice/text more strictly, at some audio-quality cost).temp— sampling temperature (default0.6, the upstream default).quantize— MLX path only: weight-quantization bits (default8, Kyutai’s own on-device setting — measured RTF 0.84 vs 7.5 unquantized on an M2 Max;nullkeeps full bf16).
- async aclose()[source]
Drop the implementation so its runtime can free the weights.
- Return type:
None
- async clone_voice(name, samples, *, language=None)[source]
Refuse, and explain Kyutai’s deliberate policy.
The speaker encoder that turns a recording into a voice embedding is unreleased by design (Kyutai’s anti-abuse stance): voices come from the published bank or their voice-donation program. Saying so beats a generic NotImplementedError, because the caller’s next question is always “then how do I get a voice”.
- Parameters:
name (str)
samples (list[VoiceSample])
language (str | None)
- Return type:
- async health()[source]
Whether this engine could synthesise, without loading the model.
- Return type:
- async list_voices(engine=None)[source]
The curated female defaults — the full bank lives on Hugging Face.
The
kyutai/tts-voicesrepository holds hundreds of embeddings; listing them all would need a network call this method must not make. The two defaults are real, immediately usable names; any other repository path works the same way.
- async synthesize(text, *, language=None, voice=None)[source]
Synthesise
textinto oneAudioResult.No
speedparameter on purpose: the model has no rate control, and resampling would shift pitch with pace.narrate()detects the missing keyword and withdraws the knob instead of failing.- Parameters:
text (str) – What to say (must be non-empty).
language (str or None) –
"fr"or"en"; anything else is warned about and read with the model’s bilingual tongue. Defaults to the configured language.voice (str or None) – A voice name from
kyutai/tts-voices(e.g."unmute-prod-website/developpeuse-3.wav") or a local.safetensorsembedding path. Empty picks the female default for the language.
- Returns:
The synthesised audio and its metadata.
- Return type:
- Raises:
ValueError – If
textis empty or whitespace-only.
- speaker_helper.backends.kyutai.SAMPLE_RATE_HZ: int = 24000
Mimi (Kyutai’s codec) decodes at 24 kHz mono — the same rate as kokoro and chatterbox, which is why a narration can mix engines without resampling.
- speaker_helper.backends.kyutai.SUPPORTED_LANGUAGES: tuple[str, ...] = ('fr', 'en')
The languages the 1.6B model actually speaks. Anything else is reported (then synthesised anyway): the model will read it with a French/English tongue, and whoever asked should know rather than discover it by ear.
- speaker_helper.backends.kyutai.check_install()[source]
Return 0 when a Kyutai runtime (
moshiormoshi_mlx) is importable.- Return type:
- speaker_helper.backends.kyutai.resolve_implementation(device)[source]
Turn the
deviceknob into a concrete implementation name.- Parameters:
device (str) –
"auto"(default),"mlx", or a torch device ("cpu","cuda", …).- Returns:
"mlx"or"torch:<device>".- Return type:
Examples
>>> resolve_implementation("cpu") 'torch:cpu' >>> resolve_implementation("mlx") 'mlx'