speaker_helper.eval.priors module
Quality priors and Pareto selection for choosing an engine.
Module summary
Two ideas make engine choice operational:
A quality axis even without a reference. When no transcriber is wired in, fidelity is estimated from per-engine priors — order-of-magnitude scores in
[0, 1], recalibrated from real runs. This keeps every evaluation reporting both speed and quality.Pareto selection. To pick an engine/voice you want the non-dominated set on the quality↔RTF plane — high quality and fast.
pareto_front()extracts it.
Together these turn a batch of EvalReport
(one per engine) into an actionable “which engine should I ship?” answer.
Usage example
>>> from speaker_helper.eval.priors import engine_quality_prior
>>> engine_quality_prior("kokoro")
0.75
- speaker_helper.eval.priors.PUBLIC_ELO_AS_OF: str = '2026-11-04'
As-of date of the PUBLIC_ELO pull — Elo moves with every vote, so the date is part of the claim (same rule as LANDSCAPE.md’s licence table).
- speaker_helper.eval.priors.elo_to_unit(elo)[source]
Map a public arena Elo onto the router’s
[0, 1]quality axis.A transparent, order-preserving linear map —
(elo - 1000) / 1000, clamped. The absolute value means little (Elo scales are arena-specific); only the ORDER matters to the router’s argmax, and a linear map keeps the printed justification traceable back to the public number.- Parameters:
elo (float) – An Elo rating from
PUBLIC_ELO’s arena.- Returns:
The rating rescaled to
[0, 1].- Return type:
Examples
>>> elo_to_unit(1465.0) 0.465 >>> elo_to_unit(2200.0) 1.0
- speaker_helper.eval.priors.engine_quality_prior(engine, language=None)[source]
Return the prior quality score in
[0, 1]for a TTSengine.- Parameters:
- Returns:
The most specific prior available, or
0.80for an unknown engine.- Return type:
Examples
>>> engine_quality_prior("kyutai", "fr") 0.92 >>> engine_quality_prior("kyutai") 0.88
- speaker_helper.eval.priors.pareto_front(points)[source]
Return the non-dominated points: maximise quality, minimise RTF.
A point A dominates B if A is at least as good on both axes (higher or equal
quality, lower or equalmean_rtf) and strictly better on one.- Parameters:
points (sequence) – Candidates, each exposing
qualityandmean_rtf(e.g.EvalReport).- Returns:
The Pareto-optimal subset, in input order.
- Return type: