speaker_helper.eval.priors module

Quality priors and Pareto selection for choosing an engine.

Module summary

Two ideas make engine choice operational:

  1. A quality axis even without a reference. When no transcriber is wired in, fidelity is estimated from per-engine priors — order-of-magnitude scores in [0, 1], recalibrated from real runs. This keeps every evaluation reporting both speed and quality.

  2. Pareto selection. To pick an engine/voice you want the non-dominated set on the quality↔RTF plane — high quality and fast. pareto_front() extracts it.

Together these turn a batch of EvalReport (one per engine) into an actionable “which engine should I ship?” answer.

Usage example

>>> from speaker_helper.eval.priors import engine_quality_prior
>>> engine_quality_prior("kokoro")
0.75

Author

Warith HARCHAOUI — https://linkedin.com/in/warith-harchaoui

speaker_helper.eval.priors.PUBLIC_ELO_AS_OF: str = '2026-11-04'

As-of date of the PUBLIC_ELO pull — Elo moves with every vote, so the date is part of the claim (same rule as LANDSCAPE.md’s licence table).

speaker_helper.eval.priors.elo_to_unit(elo)[source]

Map a public arena Elo onto the router’s [0, 1] quality axis.

A transparent, order-preserving linear map — (elo - 1000) / 1000, clamped. The absolute value means little (Elo scales are arena-specific); only the ORDER matters to the router’s argmax, and a linear map keeps the printed justification traceable back to the public number.

Parameters:

elo (float) – An Elo rating from PUBLIC_ELO’s arena.

Returns:

The rating rescaled to [0, 1].

Return type:

float

Examples

>>> elo_to_unit(1465.0)
0.465
>>> elo_to_unit(2200.0)
1.0
speaker_helper.eval.priors.engine_quality_prior(engine, language=None)[source]

Return the prior quality score in [0, 1] for a TTS engine.

Parameters:
  • engine (str) – Engine id (e.g. "kokoro").

  • language (str or None) – ISO-639 code (regional variants accepted, e.g. "fr-fr"). When a per-language refinement exists for the engine it wins over the base prior; None keeps the language-blind base.

Returns:

The most specific prior available, or 0.80 for an unknown engine.

Return type:

float

Examples

>>> engine_quality_prior("kyutai", "fr")
0.92
>>> engine_quality_prior("kyutai")
0.88
speaker_helper.eval.priors.pareto_front(points)[source]

Return the non-dominated points: maximise quality, minimise RTF.

A point A dominates B if A is at least as good on both axes (higher or equal quality, lower or equal mean_rtf) and strictly better on one.

Parameters:

points (sequence) – Candidates, each exposing quality and mean_rtf (e.g. EvalReport).

Returns:

The Pareto-optimal subset, in input order.

Return type:

list