vocal_helper.router module

vocal_helper.router

The aiguilleur: the study-grounded diarization backend router.

Diarization is the one pipeline stage with a genuine backend fork, and there is no single best backend: the right one depends on whether the audio is a live stream or a batch file, on its duration, on the speaker count, and on whether the deployment can afford a PyTorch install. This module turns those scenario conditions into one explicit, testable decision that carries both quality (DER) and speed (RTF) so the CLIs and downstream code never hard-code a backend, re-derive the trade-off, or hide the cost.

Quality × speed per scenario (the whole point)

Numbers were re-validated on this machine (2026-07-19, studies/router_profile_validation.py, pyannote.metrics collar 0.25, median DER + RTF) against ground truth: bagarre (30 short mixes) + AMI dev-slice (2 real meetings). The sherpa DER is from ADR 0002; its ONNX models (pyannote-3.0 segmentation + TitaNet-large embedding) now ship in the diarization-engines bundle, so the torch-free path runs with no HuggingFace and no PyTorch. DER = quality (lower is better); RTF = speed (< 1 = faster than real time):

mode

backend

DER

RTF

when the router picks it

offline

nemo

0.142

0.051

short (≤300 s), ≤4 speakers; dense interleaved turns

offline

pyannote

0.122

0.067

long / unknown length / >4 speakers; the robust default

offline

sherpa

0.174

0.58

torch-free deployment (no PyTorch); ADR 0002

online

nemo

0.586

0.030

any live stream (the default online embedder)

online

sherpa

0.174

0.58

torch-free streaming = periodic offline re-diarization

What the router will never do

It never routes to a paid backend. pyannoteai exists as an offline diarization backend — it is the whole of the hybrid mode, hosted speakers with local words — but it is reachable only by asking for it by name (diar={"backend": "pyannoteai"}).

This is not an oversight, it is the rule that makes the rest safe. A router that could pick a billed engine would mean the same command sometimes costs money and sometimes does not, decided by a duration threshold nobody remembers. And since cost is not on the same scale as DER or RTF, there is no honest way to put it in the table below and let a comparison decide: the two local columns trade quality against speed, which are commensurable; adding euros is a different kind of quantity, and the person spending them is the one who should choose.

So: the router arbitrates among the free local backends on measurement. Paying is a decision, made by name, by a human.

Why a router, not a default

Two independent findings, both measured on this machine:

  1. Offline: a length crossover. On short dense turns (bagarre, ~30 s) NeMo Sortformer wins by ~2.3x (offline DER 0.142 vs pyannote 0.330): its end-to-end slot attribution drives speaker-confusion to ~0. On long meetings (AMI) the verdict reverses: pyannote median DER 0.122 (inside Bredin 2023’s 0.188 band), and Sortformer hangs past ~25 min (no output on a 27-min meeting), since its 90 s / 4-speaker cap puts long/crowded form out of distribution. So offline needs a router: “ship nemo” or “ship pyannote” is wrong for one common workload.

  2. Online: nemo, always — but online is not the only live option, and it is not the good one. The streaming stage’s error rate is three to four times the offline one (0.586 against 0.122) because it cannot model overlapping speech, and no choice of embedder repairs that: the limitation is the latency.

    When the speaker labels matter as much as the words, the answer is not a better online clusterer, it is vocal_helper.rediar — run the offline diarizer periodically on a bounded window and correct the labels already shown.

    Measured, it depends on the length, and on whole meetings it loses. On five minutes of AMI IS1008a it halved the error: online alone 0.666, windowed re-diarization 0.348, offline whole-file 0.149 (studies/asr_acceleration_rtf.md). On the whole meetings it did worse than the online stage it was meant to correct: 0.679 against 0.597 on IS1008a (944 s), 0.817 against 0.616 on ES2011a (1114 s), offline 0.124 and 0.188 (studies/rediar_live_der.json, from studies/rediar_live_der.py). An earlier version of this docstring claimed the full 0.122, then a plain halving; both were one measurement read as a rule. The growing buffer (vocal_helper.growing) replaced it as the live default for that reason.

    Why it stops halfway is structural and worth knowing: the published rows are the voice-activity detector’s segments, and relabelling cannot repair a segment that contains a speaker change. The offline pass gets to choose its own boundaries; this one inherits them. Emitting the offline segmentation instead of relabelling the VAD’s is the next lever, not a better clusterer.

    The online stage remains the right choice for the opposite priority — a label within 500 ms and no retroactive correction, which an interactive agent needs and a transcript does not.

    The rest of this finding still stands for the online stage itself: vocal-helper’s OnlineDiarStage is a latency-bound cosine-clustering approximation, inherently ~3-4x the offline DER (it cannot model overlap). Across lengths the NeMo TitaNet embedder is the best online backend (bagarre 0.586, AMI 0.497) and beats pyannote/embedding online (0.590 / 0.844), matching the 2026-06-30 embedding sweep that made it the default. There is no online length crossover: streaming routes to nemo (refine_on_close roughly halves the DER on meetings that over-segment, a stage knob the router leaves to the stage).

The torch-free sherpa (ONNX TitaNet-large, DER 0.174/0.148, beats NeMo Sortformer 0.267, FR+EN validated, ADR 0002) is the portability pick either way.

Scope

This module decides the diarization backend (mode + backend), the one stage with a real fork. The other stages are single-backend by study verdict and are not routed: VAD is Silero v5 (32 ms cadence), ASR is pywhispercpp large-v3-turbo (no faster-whisper win on CPU/Apple in the study), the analyst is Gemma via Ollama. Language is discovered, never routed to a default (see vocal_helper.lid).

Usage example

>>> from vocal_helper.router import select_diarization
>>> plan = select_diarization(live=False, duration_s=45.0, max_speakers=3)
>>> plan.backend, plan.expected_der, plan.expected_rtf
('nemo', 0.142, 0.051)
>>> select_diarization(live=False, duration_s=1800.0).backend
'pyannote'

Author

Warith HARCHAOUI, https://linkedin.com/in/warith-harchaoui

class vocal_helper.router.BackendPlan(mode, backend, expected_der, expected_rtf, reason, sherpa_num_clusters=None)[source]

Bases: object

One routing decision: which diarizer, and its quality + speed.

Parameters:

  • mode (str)

  • backend (str)

  • expected_der (float)

  • expected_rtf (float)

  • reason (str)

  • sherpa_num_clusters (int | None)

mode

"online" (streaming OnlineDiarStage) or "offline" (whole-buffer OfflineDiarStage).

Type:

str

backend

Diarization backend — "nemotron", "pyannote", "nemo" or "sherpa" — to pass straight to the stage’s diar={"backend": ...} config.

Type:

str

expected_der

Representative diarization error rate (quality — lower is better) for this scenario, from the pdbms study / ADR 0002.

Type:

float

expected_rtf

Representative real-time factor (speed — < 1 is faster than real time) for this scenario.

Type:

float

reason

Human-readable justification citing the deciding measurement, surfaced to the operator so the choice is never a black box.

Type:

str

sherpa_num_clusters

When the backend is sherpa and the exact speaker count is known, the count to pin its clustering to (FastClusteringConfig(num_clusters=...)). None leaves sherpa on auto-clustering, so meeting-audio behaviour is unchanged. This is the lever that collapses sherpa’s 2-party telephony over-segmentation (pdbms diar-study §12).

Type:

int or None

backend: str
expected_der: float
expected_rtf: float
mode: str
property rationale: str

Alias for reason (some callers log plan.rationale).

reason: str
sherpa_num_clusters: int | None = None
vocal_helper.router.select_diarization(*, live, duration_s=None, max_speakers=None, num_speakers=None, torch_free=False, pyannote_available=True, nemo_available=True, nemotron_available=False)[source]

Route to the diarization backend that the experiments justify.

Encodes the pdbms unified-study crossover (see the module docstring) as a single, testable decision over the conditions that actually move DER (quality) and RTF (speed): live-vs-batch, duration, speaker count, and torch-availability. The returned plan carries both axes explicitly.

Parameters:

  • live (bool) – True for a live stream (streaming diarizer), False for a batch file (whole-buffer offline diarizer — the reliable default for files).

  • duration_s (float or None, optional) – Audio duration in seconds when known (a file’s length is cheap to read). None means “unknown” and is treated as long-form — the safe, robust branch. Default None.

  • max_speakers (int or None, optional) – Known upper bound on the number of speakers. Used only to keep audio with more than SORTFORMER_MAX_SPEAKERS speakers off NeMo Sortformer (its hard 4-speaker cap). None = unknown. Default None.

  • num_speakers (int or None, optional) – Exact known speaker count (distinct from the max_speakers ceiling). When the router picks sherpa and this is set, the plan carries it as sherpa_num_clusters to pin sherpa’s clustering — the fix for its 2-party telephony over-segmentation (pdbms diar-study §12.1). Leaving it None keeps sherpa on auto-clustering, so meeting behaviour is unchanged. Default None.

  • torch_free (bool, optional) – True when the deployment cannot install PyTorch — routes to the sherpa onnxruntime backend regardless of length. Default False.

  • pyannote_available (bool, optional) – Whether the pyannote backend can actually run (extra installed + bundle present). When a rule would pick pyannote but it is unavailable, the router falls back rather than choosing an unrunnable backend. Default True.

  • nemo_available (bool, optional) – Whether the NeMo Sortformer backend can actually run (the nemo extra is importable). When the short/dense rule would pick nemo but it is not installed, the router falls through to the robust pyannote branch rather than emitting an unrunnable backend. Default True.

  • nemotron_available (bool, optional) – Whether the Nemotron 3 ONNX graph is in the diarization-engines bundle. When it is, every offline run routes to nemotron: it leads vocal_helper.diar.OFFLINE_PREFERENCE on error rate alone, is torch-free, and has neither Sortformer’s length hang nor its 4-speaker cap. Default False, so a caller that does not probe keeps the duration crossover; the CLI and the HTTP surface probe.

Returns:

The chosen mode + backend, its expected_der / expected_rtf, and the reason.

Return type:

BackendPlan

Examples

>>> select_diarization(live=False, duration_s=45.0, max_speakers=3).backend
'nemo'
>>> select_diarization(live=False, duration_s=1800.0).backend
'pyannote'
>>> select_diarization(live=True, torch_free=True).backend
'sherpa'
>>> # a short clip still routes to pyannote when the nemo extra is absent
>>> select_diarization(live=False, duration_s=45.0, nemo_available=False).backend
'pyannote'
>>> # torch-free with a known 2-party count pins sherpa's clustering
>>> select_diarization(live=False, torch_free=True, num_speakers=2).sherpa_num_clusters
2
>>> # without a known count, sherpa stays on auto-clustering (meeting default)
>>> select_diarization(live=False, torch_free=True).sherpa_num_clusters is None
True

Notes

The router decides which diarizer and reports its scenario quality/speed; the online/offline stages keep their own tuned knobs (join threshold, refine pass, chunk ceiling). Numbers were re-validated on this machine (studies/router_profile_validation.py, 2026-07-19) against ground truth; sherpa is from ADR 0002.