"""
vocal_helper.router
===================
The **aiguilleur**: the study-grounded diarization backend router.
Diarization is the one pipeline stage with a genuine backend fork, and there is
**no single best backend**: the right one depends on whether the audio is a
**live stream** or a **batch file**, on its **duration**, on the **speaker
count**, and on whether the deployment can afford a PyTorch install. This module
turns those scenario conditions into one explicit, testable decision that
carries **both quality (DER) and speed (RTF)** so the CLIs and downstream code
never hard-code a backend, re-derive the trade-off, or hide the cost.
Quality × speed per scenario (the whole point)
----------------------------------------------
Numbers were **re-validated on this machine** (2026-07-19,
``studies/router_profile_validation.py``, ``pyannote.metrics`` collar 0.25, median
DER + RTF) against ground truth: bagarre (30 short mixes) + AMI dev-slice (2 real
meetings). The ``sherpa`` DER is from ADR 0002; its ONNX models (pyannote-3.0
segmentation + TitaNet-large embedding) now ship in the diarization-engines
bundle, so the torch-free path runs with no HuggingFace and no PyTorch. DER =
quality (lower is better); RTF = speed (``< 1`` = faster than real time):
======== ========= ============ ========== =================================================
mode backend DER RTF when the router picks it
======== ========= ============ ========== =================================================
offline nemo **0.142** 0.051 short (≤300 s), ≤4 speakers; dense interleaved turns
offline pyannote **0.122** 0.067 long / unknown length / >4 speakers; the robust default
offline sherpa 0.174 0.58 torch-free deployment (no PyTorch); ADR 0002
online nemo 0.586 0.030 any live stream (the default online embedder)
online sherpa 0.174 0.58 torch-free streaming = periodic offline re-diarization
======== ========= ============ ========== =================================================
What the router will never do
-----------------------------
It never routes to a **paid** backend. ``pyannoteai`` exists as an offline
diarization backend — it is the whole of the hybrid mode, hosted speakers with
local words — but it is reachable only by asking for it by name
(``diar={"backend": "pyannoteai"}``).
This is not an oversight, it is the rule that makes the rest safe. A router that
could pick a billed engine would mean the same command sometimes costs money and
sometimes does not, decided by a duration threshold nobody remembers. And since
cost is not on the same scale as DER or RTF, there is no honest way to put it in
the table below and let a comparison decide: the two local columns trade quality
against speed, which are commensurable; adding euros is a different kind of
quantity, and the person spending them is the one who should choose.
So: the router arbitrates among the free local backends on measurement. Paying is
a decision, made by name, by a human.
Why a router, not a default
---------------------------
Two independent findings, both measured on this machine:
1. **Offline: a length crossover.** On short dense turns (bagarre, ~30 s) NeMo
Sortformer wins by ~2.3x (offline DER **0.142** vs pyannote **0.330**): its
end-to-end slot attribution drives speaker-confusion to ~0. On long meetings
(AMI) the verdict *reverses*: pyannote median DER **0.122** (inside Bredin
2023's 0.188 band), and Sortformer *hangs* past ~25 min (no output on a 27-min
meeting), since its 90 s / 4-speaker cap puts long/crowded form out of distribution.
So offline needs a router: "ship nemo" or "ship pyannote" is wrong for one
common workload.
2. **Online: nemo, always — but online is not the only live option, and it is not
the good one.** The streaming stage's error rate is three to four times the
offline one (0.586 against 0.122) because it cannot model overlapping speech,
and no choice of embedder repairs that: the limitation is the latency.
When the speaker labels matter as much as the words, the answer is not a better
online clusterer, it is :mod:`vocal_helper.rediar` — run the **offline**
diarizer periodically on a bounded window and correct the labels already shown.
**Measured, it depends on the length, and on whole meetings it loses.** On
five minutes of AMI IS1008a it halved the error: online alone 0.666, windowed
re-diarization 0.348, offline whole-file 0.149
(``studies/asr_acceleration_rtf.md``). On the whole meetings it did *worse*
than the online stage it was meant to correct: 0.679 against 0.597 on IS1008a
(944 s), 0.817 against 0.616 on ES2011a (1114 s), offline 0.124 and 0.188
(``studies/rediar_live_der.json``, from ``studies/rediar_live_der.py``). An
earlier version of this docstring claimed the full 0.122, then a plain halving;
both were one measurement read as a rule. The growing buffer
(:mod:`vocal_helper.growing`) replaced it as the live default for that reason.
Why it stops halfway is structural and worth knowing: the published rows are the
**voice-activity detector's** segments, and relabelling cannot repair a segment
that contains a speaker change. The offline pass gets to choose its own
boundaries; this one inherits them. Emitting the offline segmentation instead of
relabelling the VAD's is the next lever, not a better clusterer.
The online stage remains the right choice for the opposite priority — a label
within 500 ms and no retroactive correction, which an interactive agent needs
and a transcript does not.
The rest of this finding still stands for the online stage itself: vocal-helper's ``OnlineDiarStage`` is a
latency-bound cosine-clustering *approximation*, inherently ~3-4x the offline
DER (it cannot model overlap). Across lengths the NeMo TitaNet embedder is the
best online backend (bagarre 0.586, AMI 0.497) and beats pyannote/embedding
online (0.590 / **0.844**), matching the 2026-06-30 embedding sweep that made
it the default. There is **no online length crossover**: streaming routes to
nemo (``refine_on_close`` roughly halves the DER on meetings that over-segment,
a stage knob the router leaves to the stage).
The torch-free ``sherpa`` (ONNX TitaNet-large, DER 0.174/0.148, *beats* NeMo
Sortformer 0.267, FR+EN validated, ADR 0002) is the portability pick either way.
Scope
-----
This module decides the **diarization** backend (mode + backend), the one stage
with a real fork. The other stages are single-backend by study verdict and are
not routed: VAD is Silero v5 (32 ms cadence), ASR is pywhispercpp
``large-v3-turbo`` (no faster-whisper win on CPU/Apple in the study), the
analyst is Gemma via Ollama. Language is *discovered*, never routed to a default
(see :mod:`vocal_helper.lid`).
Usage example
-------------
>>> from vocal_helper.router import select_diarization
>>> plan = select_diarization(live=False, duration_s=45.0, max_speakers=3)
>>> plan.backend, plan.expected_der, plan.expected_rtf
('nemo', 0.142, 0.051)
>>> select_diarization(live=False, duration_s=1800.0).backend
'pyannote'
Author
------
Warith HARCHAOUI, https://linkedin.com/in/warith-harchaoui
"""
from __future__ import annotations
from dataclasses import dataclass
# Offline NeMo Sortformer stays reliable up to ~300 s (it chunks internally at
# its 60 s ideal duration) but *hangs* on very long meetings — the pdbms study
# saw no output on a 27-min AMI file. Past this ceiling, only pyannote is safe.
NEMO_MAX_DURATION_S: float = 300.0
# Sortformer is trained/capped at 4 speakers. Beyond that it silently mislabels,
# so a short clip with a known >4 speaker count must still go to pyannote.
SORTFORMER_MAX_SPEAKERS: int = 4
# Representative (DER, RTF) per (mode, backend), re-validated on this machine
# (2026-07-19; studies/router_profile_validation.py; median DER + RTF). Keyed so
# a decision only names (mode, backend) and the quality + speed numbers follow —
# they can never drift from the reason. DER = quality (lower better); RTF = speed
# (< 1 faster than real time). Only the (mode, backend) pairs the router can emit
# are listed — online never routes to pyannote (it loses to nemo online at every
# length: 0.590/0.844 vs 0.586/0.497).
_PROFILE: dict[tuple[str, str], tuple[float, float]] = {
# Synthetic mix with exact ground truth (studies/diar_far_field.py), anechoic;
# 0.252 at 0.6 s RT60. Both measured at the 1.04 s schedule that
# ``NemotronOnnxDiar`` defaults to, whose CPU RTF is 0.258 (0.166 is the
# 6.4 s context, which is not the default).
("offline", "nemotron"): (0.090, 0.258),
("offline", "nemo"): (0.142, 0.051), # bagarre n=30, offline Sortformer
("offline", "pyannote"): (0.122, 0.067), # AMI dev-slice n=2 median
("offline", "sherpa"): (0.174, 0.58), # ADR 0002, ES2011a, TitaNet-large ONNX
("online", "nemo"): (0.586, 0.030), # bagarre n=30; AMI long 0.497 — latency-bound ~4x offline
("online", "sherpa"): (0.174, 0.58), # periodic offline re-diarization (ADR 0002)
}
[docs]
@dataclass(frozen=True)
class BackendPlan:
"""One routing decision: which diarizer, and its quality + speed.
Attributes
----------
mode : str
``"online"`` (streaming :class:`~vocal_helper.OnlineDiarStage`) or
``"offline"`` (whole-buffer :class:`~vocal_helper.OfflineDiarStage`).
backend : str
Diarization backend — ``"nemotron"``, ``"pyannote"``, ``"nemo"`` or ``"sherpa"`` — to
pass straight to the stage's ``diar={"backend": ...}`` config.
expected_der : float
Representative diarization error rate (quality — lower is better) for
this scenario, from the pdbms study / ADR 0002.
expected_rtf : float
Representative real-time factor (speed — ``< 1`` is faster than real
time) for this scenario.
reason : str
Human-readable justification citing the deciding measurement, surfaced
to the operator so the choice is never a black box.
sherpa_num_clusters : int or None
When the backend is ``sherpa`` and the exact speaker count is known, the
count to pin its clustering to (``FastClusteringConfig(num_clusters=...)``).
``None`` leaves sherpa on auto-clustering, so meeting-audio behaviour is
unchanged. This is the lever that collapses sherpa's 2-party telephony
over-segmentation (pdbms diar-study §12).
"""
mode: str
backend: str
expected_der: float
expected_rtf: float
reason: str
sherpa_num_clusters: int | None = None
@property
def rationale(self) -> str:
"""Alias for :attr:`reason` (some callers log ``plan.rationale``)."""
return self.reason
def _plan(
mode: str, backend: str, reason: str, *, sherpa_num_clusters: int | None = None
) -> BackendPlan:
"""Assemble a :class:`BackendPlan`, attaching quality + speed from ``_PROFILE``.
Parameters
----------
mode : str
``"online"`` or ``"offline"``.
backend : str
``"pyannote"``, ``"nemo"`` or ``"sherpa"``.
reason : str
The justification string for this decision.
Returns
-------
BackendPlan
The plan with ``expected_der`` / ``expected_rtf`` looked up for
``(mode, backend)`` so the numbers can never contradict the choice.
"""
# Single source of truth for the scenario's quality/speed — no hand-typed
# numbers at the call sites that could drift from the reason.
der, rtf = _PROFILE[(mode, backend)]
return BackendPlan(
mode=mode,
backend=backend,
expected_der=der,
expected_rtf=rtf,
reason=reason,
sherpa_num_clusters=sherpa_num_clusters,
)
[docs]
def select_diarization(
*,
live: bool,
duration_s: float | None = None,
max_speakers: int | None = None,
num_speakers: int | None = None,
torch_free: bool = False,
pyannote_available: bool = True,
nemo_available: bool = True,
nemotron_available: bool = False,
) -> BackendPlan:
"""Route to the diarization backend that the experiments justify.
Encodes the pdbms unified-study crossover (see the module docstring) as a
single, testable decision over the conditions that actually move DER
(quality) and RTF (speed): live-vs-batch, duration, speaker count, and
torch-availability. The returned plan carries both axes explicitly.
Parameters
----------
live : bool
``True`` for a live stream (streaming diarizer), ``False`` for a batch
file (whole-buffer offline diarizer — the reliable default for files).
duration_s : float or None, optional
Audio duration in seconds when known (a file's length is cheap to
read). ``None`` means "unknown" and is treated as long-form — the safe,
robust branch. Default ``None``.
max_speakers : int or None, optional
Known upper bound on the number of speakers. Used only to keep audio
with more than :data:`SORTFORMER_MAX_SPEAKERS` speakers off NeMo
Sortformer (its hard 4-speaker cap). ``None`` = unknown. Default
``None``.
num_speakers : int or None, optional
Exact known speaker count (distinct from the ``max_speakers`` ceiling).
When the router picks ``sherpa`` and this is set, the plan carries it as
``sherpa_num_clusters`` to pin sherpa's clustering — the fix for its
2-party telephony over-segmentation (pdbms diar-study §12.1). Leaving it
``None`` keeps sherpa on auto-clustering, so meeting behaviour is
unchanged. Default ``None``.
torch_free : bool, optional
``True`` when the deployment cannot install PyTorch — routes to the
``sherpa`` onnxruntime backend regardless of length. Default ``False``.
pyannote_available : bool, optional
Whether the pyannote backend can actually run (extra installed + bundle
present). When a rule would pick ``pyannote`` but it is unavailable, the
router falls back rather than choosing an unrunnable backend. Default
``True``.
nemo_available : bool, optional
Whether the NeMo Sortformer backend can actually run (the ``nemo`` extra
is importable). When the short/dense rule would pick ``nemo`` but it is
not installed, the router falls through to the robust ``pyannote`` branch
rather than emitting an unrunnable backend. Default ``True``.
nemotron_available : bool, optional
Whether the Nemotron 3 ONNX graph is in the diarization-engines bundle.
When it is, every offline run routes to ``nemotron``: it leads
:data:`vocal_helper.diar.OFFLINE_PREFERENCE` on error rate alone, is
torch-free, and has neither Sortformer's length hang nor its 4-speaker
cap. Default ``False``, so a caller that does not probe keeps the
duration crossover; the CLI and the HTTP surface probe.
Returns
-------
BackendPlan
The chosen ``mode`` + ``backend``, its ``expected_der`` /
``expected_rtf``, and the ``reason``.
Examples
--------
>>> select_diarization(live=False, duration_s=45.0, max_speakers=3).backend
'nemo'
>>> select_diarization(live=False, duration_s=1800.0).backend
'pyannote'
>>> select_diarization(live=True, torch_free=True).backend
'sherpa'
>>> # a short clip still routes to pyannote when the nemo extra is absent
>>> select_diarization(live=False, duration_s=45.0, nemo_available=False).backend
'pyannote'
>>> # torch-free with a known 2-party count pins sherpa's clustering
>>> select_diarization(live=False, torch_free=True, num_speakers=2).sherpa_num_clusters
2
>>> # without a known count, sherpa stays on auto-clustering (meeting default)
>>> select_diarization(live=False, torch_free=True).sherpa_num_clusters is None
True
Notes
-----
The router decides *which* diarizer and reports its scenario quality/speed;
the online/offline *stages* keep their own tuned knobs (join threshold,
refine pass, chunk ceiling). Numbers were re-validated on this machine
(``studies/router_profile_validation.py``, 2026-07-19) against ground truth;
sherpa is from ADR 0002.
"""
mode = "online" if live else "offline"
# 0. OFFLINE, Nemotron present → Nemotron. It used to be reachable only from
# the library's ``backend="auto"``, which reads OFFLINE_PREFERENCE, while
# the CLI and HTTP paths came here and got a duration crossover that did
# not know it existed. It is torch-free, so the torch_free override below
# does not outrank it. Live is not routed here: the streaming Pipeline
# swaps Nemotron in itself when the graph is present.
if not live and nemotron_available:
der, rtf = _PROFILE[("offline", "nemotron")]
return _plan(
"offline",
"nemotron",
f"graph in the bundle → nemotron (torch-free ONNX, end-to-end; DER {der} "
f"on an exact-truth mix, 0.252 reverberated; RTF {rtf})",
)
# 1. Portability override — no PyTorch available. The torch-free ONNX
# TitaNet (sherpa) is the only runnable backend; quality beats NeMo
# Sortformer, so it is a safe forced pick. For streaming, sherpa runs as
# periodic offline re-diarization (ADR 0002: per-segment online sherpa is
# a dead end), which is why online sherpa carries the offline DER 0.174.
if torch_free:
pin = (
f"; speaker count known ({num_speakers}) → pinned clustering "
"(collapses 2-party over-segmentation, pdbms §12.1)"
if num_speakers is not None
else ""
)
return _plan(
mode,
"sherpa",
"torch-free deployment → sherpa (onnxruntime TitaNet-large, no PyTorch); "
"DER 0.174 ES2011a / 0.148 held-out IS1008a, beats NeMo Sortformer 0.267, "
"FR+EN validated (ADR 0002)"
+ ("; streaming = periodic offline re-diarization" if live else "")
+ pin,
sherpa_num_clusters=num_speakers,
)
# 2. STREAMING → nemo, always. vocal-helper's OnlineDiarStage is a
# latency-bound cosine-clustering approximation (~3-4x the offline DER);
# the NeMo TitaNet embedder is the best online backend at *every* length
# measured here (bagarre 0.586, AMI 0.497) and beats pyannote/embedding
# online (0.590 / 0.844) — the 2026-06-30 default. There is no online
# length crossover, and the 4-speaker cap is Sortformer/offline-only (the
# online path uses TitaNet embeddings, uncapped). refine_on_close (a stage
# knob) roughly halves the DER on meetings that over-segment.
if live:
der, rtf = _PROFILE[("online", "nemo")]
return _plan(
"online",
"nemo",
f"live stream → nemo TitaNet embedder (best online backend at every "
f"length; DER {der}, RTF {rtf} — online is a latency-bound ~3-4x-offline "
f"approximation, refine_on_close helps long meetings)",
)
# 3. OFFLINE short / dense regime — NeMo Sortformer wins by ~2.3x on short
# interleaved turns (DER 0.142 vs pyannote 0.330 on bagarre). Its 4-speaker
# cap means a *known* larger count must skip it, and "unknown duration" is
# deliberately NOT short — without a length we take the robust branch.
too_many_speakers = max_speakers is not None and max_speakers > SORTFORMER_MAX_SPEAKERS
short_enough = duration_s is not None and duration_s <= NEMO_MAX_DURATION_S
if short_enough and not too_many_speakers and nemo_available:
der, rtf = _PROFILE[("offline", "nemo")]
return _plan(
"offline",
"nemo",
f"≤{NEMO_MAX_DURATION_S:.0f}s, ≤{SORTFORMER_MAX_SPEAKERS} speakers → "
f"nemo Sortformer (end-to-end, confusion ~0; DER {der}, RTF {rtf})",
)
# 4. OFFLINE long-form / unknown / many-speaker → pyannote, the robust
# default (AMI median DER 0.122); NeMo hangs past ~25 min and caps at 4
# speakers, so pyannote is the only safe choice here.
if pyannote_available:
# Name the *actual* reason nemo was skipped so the operator nudge is
# honest — a short clip can reach pyannote purely because the nemo extra
# is absent, not because it is long or crowded.
why_long = (
"unknown duration (treated as long-form)"
if duration_s is None
else f">{NEMO_MAX_DURATION_S:.0f}s"
if not short_enough
else f">{SORTFORMER_MAX_SPEAKERS} speakers"
if too_many_speakers
else "nemo extra not installed"
)
der, rtf = _PROFILE[("offline", "pyannote")]
return _plan(
"offline",
"pyannote",
# Backend, not checkpoint: this read "pyannote 3.1" while
# _PyannoteOfflineDiar defaulted to community-1. Which checkpoint
# the pyannote backend loads is its own business, measured under
# its DEFAULT_CHECKPOINT; naming one here only created a second
# copy to go stale, and this module stays import-free on purpose
# so it cannot read the real one. test_router.py guards it.
f"{why_long} → pyannote (robust default; DER {der}, RTF {rtf}; "
f"nemo hangs past ~25 min / >4 speakers)",
)
# 5. pyannote was the right call but is not installed/bundled — fall back to
# the torch-free sherpa rather than an unrunnable backend. (nemo is unsafe
# here: this branch is exactly the long/many-speaker case it fails on.)
pin = (
f"; speaker count known ({num_speakers}) → pinned clustering (pdbms §12.1)"
if num_speakers is not None
else ""
)
return _plan(
"offline",
"sherpa",
"pyannote unavailable on the long-form/robust branch → sherpa "
"(onnxruntime TitaNet); nemo is unsafe past ~25 min / >4 speakers" + pin,
sherpa_num_clusters=num_speakers,
)