speaker_helper.backends.kyutai module

Kyutai TTS: native French speech from a Paris lab, offline, voice by embedding.

Module summary

Kyutai’s tts-1.6b-en_fr (delayed-streams modeling) is the engine this package reaches for when the text is French and has to sound French: the model is trained by a French lab on French data, and the 2026-10 study (STUDY.md on the research branch) found it the best-documented native French open model — where Chatterbox renders French through an English-first pipeline and Kokoro’s single French voice is a stated grade B-.

Voices are precomputed embeddings from the kyutai/tts-voices repository (CC-BY-4.0 / CC0 banks), fetched by name on first use. That is a deliberate anti-abuse policy by Kyutai: the speaker encoder is unreleased, so this backend selects voices and does not clone them — KyutaiEngine.clone_voice() says so instead of pretending.

Two implementations, one backend

The same weights run through two runtimes, picked by the device knob:

  • MLX (moshi-mlx, Kyutai’s own Apple-Silicon implementation) — the Metal GPU path. device: "auto" (the default) selects it when os-helper says this is a Mac and moshi_mlx is importable, because the CPU fallback measured ~8× slower than real time on an M2 Max while the MLX path is the operating point Kyutai themselves ship for Macs.

  • PyTorch (moshi) — everywhere else, on the device named by the knob ("cpu", "cuda"…).

Both load the same Hugging Face files (~3 GB, cached once, offline after).

The weights download on the first synthesis and are cached; after that the engine is fully offline. Sampling uses a temperature (the MLX path pins the upstream seed, so it is reproducible run to run; the PyTorch path is not).

This backend has no phoneme input (plain text in), so pronunciation-lexicon entries reach it through the respelling tier applied by Speaker — see speaker_helper.lexicon.

Usage example

>>> from speaker_helper import Speaker, Settings
>>> # spk = Speaker(Settings.from_mapping({"backend": "kyutai", "language": "fr"}))
>>> # await spk.say("Bonjour depuis Paris.")

Author

Warith HARCHAOUI — https://linkedin.com/in/warith-harchaoui

speaker_helper.backends.kyutai.DEFAULT_REPO: str = 'kyutai/tts-1.6b-en_fr'

Kyutai’s 1.6B English+French TTS.

Type:

The default model repository

speaker_helper.backends.kyutai.DEFAULT_VOICE_BY_LANG: dict[str, str] = {'en': 'unmute-prod-website/p329_022.wav', 'fr': 'cml-tts/fr/10087_11650_000028-0002_enhanced.wav'}

Default female voice per language, as names in the voice repository (the .safetensors embedding suffix is appended by the runtime).

The French default is a CML-TTS speaker (CC BY 4.0), picked by measurement in the speak A/B study (2026-10-26): screened against four other female CML voices and unmute’s developpeuse-3, it scored highest on UTMOSv2 by a wide margin (≈3.4 vs ≈2.0 for developpeuse-3 on the same sentences, F0 ≈ 226 Hz — female). developpeuse-3 (unmute-prod-website/developpeuse-3.wav, CC0) remains a fine lighter -licence alternative via voice=. p329_022 is VCTK speaker p329, a female English voice.

speaker_helper.backends.kyutai.DEFAULT_VOICE_REPO: str = 'kyutai/tts-voices'

The default voice-embedding repository.

class speaker_helper.backends.kyutai.KyutaiEngine(settings)[source]

Bases: object

The Kyutai TTS backend: text in, native French (or English) out.

Parameters:

settings (Settings) –

voice may name a voice from the kyutai/tts-voices repository (or a local .safetensors embedding path); empty picks the female default for the language. The kyutai mapping supplies the knobs:

  • repo — model repository (default DEFAULT_REPO).

  • voice_repo — voice-embedding repository (default DEFAULT_VOICE_REPO).

  • device — "auto" (default: Kyutai’s MLX runtime on a Mac when installed, torch CPU elsewhere), "mlx", or a torch device.

  • dtype — torch path only: "float32" (default) or "bfloat16".

  • cfg_coef — classifier-free-guidance strength (default 2.0; higher follows the voice/text more strictly, at some audio-quality cost).

  • temp — sampling temperature (default 0.6, the upstream default).

  • quantize — MLX path only: weight-quantization bits (default 8, Kyutai’s own on-device setting — measured RTF 0.84 vs 7.5 unquantized on an M2 Max; null keeps full bf16).

async aclose()[source]

Drop the implementation so its runtime can free the weights.

Return type:

None

async clone_voice(name, samples, *, language=None)[source]

Refuse, and explain Kyutai’s deliberate policy.

The speaker encoder that turns a recording into a voice embedding is unreleased by design (Kyutai’s anti-abuse stance): voices come from the published bank or their voice-donation program. Saying so beats a generic NotImplementedError, because the caller’s next question is always “then how do I get a voice”.

Parameters:
Return type:

str

async health()[source]

Whether this engine could synthesise, without loading the model.

Return type:

dict

async list_voices(engine=None)[source]

The curated female defaults — the full bank lives on Hugging Face.

The kyutai/tts-voices repository holds hundreds of embeddings; listing them all would need a network call this method must not make. The two defaults are real, immediately usable names; any other repository path works the same way.

Parameters:

engine (str | None)

Return type:

VoiceList

async synthesize(text, *, language=None, voice=None)[source]

Synthesise text into one AudioResult.

No speed parameter on purpose: the model has no rate control, and resampling would shift pitch with pace. narrate() detects the missing keyword and withdraws the knob instead of failing.

Parameters:
  • text (str) – What to say (must be non-empty).

  • language (str or None) – "fr" or "en"; anything else is warned about and read with the model’s bilingual tongue. Defaults to the configured language.

  • voice (str or None) – A voice name from kyutai/tts-voices (e.g. "unmute-prod-website/developpeuse-3.wav") or a local .safetensors embedding path. Empty picks the female default for the language.

Returns:

The synthesised audio and its metadata.

Return type:

AudioResult

Raises:

ValueError – If text is empty or whitespace-only.

speaker_helper.backends.kyutai.SAMPLE_RATE_HZ: int = 24000

Mimi (Kyutai’s codec) decodes at 24 kHz mono — the same rate as kokoro and chatterbox, which is why a narration can mix engines without resampling.

speaker_helper.backends.kyutai.SUPPORTED_LANGUAGES: tuple[str, ...] = ('fr', 'en')

The languages the 1.6B model actually speaks. Anything else is reported (then synthesised anyway): the model will read it with a French/English tongue, and whoever asked should know rather than discover it by ear.

speaker_helper.backends.kyutai.check_install()[source]

Return 0 when a Kyutai runtime (moshi or moshi_mlx) is importable.

Return type:

int

speaker_helper.backends.kyutai.resolve_implementation(device)[source]

Turn the device knob into a concrete implementation name.

Parameters:

device (str) – "auto" (default), "mlx", or a torch device ("cpu", "cuda", …).

Returns:

"mlx" or "torch:<device>".

Return type:

str

Examples

>>> resolve_implementation("cpu")
'torch:cpu'
>>> resolve_implementation("mlx")
'mlx'