vocal_helper.live_metrics module

vocal_helper.live_metrics

What a live session actually costs the person watching it: how long after the words were spoken the text appeared, how often what appeared was later rewritten, and where in the producer/consumer chain the wait accumulated.

Why this module exists

The suite already has a precise latency model — operator-assistance-streaming’s CADENCE.md gives per-stage constants and a table of expected on-screen delays, down to “a 3-second line appears at t ≈ 3.85 s”. It is an excellent model and it is predicted, never observed. Nothing in the live path records what really happened, so a regression in a queue, a model swap that doubles a round-trip, or a scheduler change that starves a consumer, all land silently on the one axis a live product is judged on.

Offline quality has WER and DER. Live has these three, and they are not the same question:

latency

Seconds between the end of a piece of speech and the moment its text reached the consumer. Measured from the end of the utterance, not the start, because that is when a listener starts waiting. A long line is not late because it was long.

revision (“scintillement”)

The share of emitted items later rewritten. Text that jumps is unusable at any word error rate, and a pipeline that emits early and corrects often can post a flattering latency while being worse to watch than one that waits.

queue wait

Where the latency came from. CADENCE.md’s own framing: user-visible latency is the sum of waits in the queues plus each stage’s compute. Without the split, a latency number says something is slow and never which thing.

The invariant

CADENCE.md states one rule the whole design rests on: no UI update appears before the audio that produced it. LiveMetrics.violations() turns that from prose into something a test can fail on — an event whose audio had not been captured yet when it was emitted means a clock, a buffer or a mock is lying, and every number computed next to it is meaningless.

Nothing here touches audio, models or the network: it records timestamps a caller provides, so it is exercised in tests without a microphone and costs a dictionary insert in production.

Author

Warith HARCHAOUI — https://linkedin.com/in/warith-harchaoui

class vocal_helper.live_metrics.LiveMetrics(started_at=<factory>, _emissions=<factory>, _latest=<factory>, _revisions=0, _queue_waits=<factory>, _violations=<factory>)

Bases: object

Collects live-session timings. One instance per session.

Parameters:

• started_at (float, optional) – Monotonic clock reading at the moment audio capture began. Defaults to now, which is right when the collector is created alongside the source and wrong if it is created later — pass it explicitly when replaying a file, where “now” and “the start of the audio” are different instants.

• _emissions (list[_Emission])

• _latest (dict[tuple[str, float], _Emission])

• _revisions (int)

• _queue_waits (dict[str, list[float]])

• _violations (list[dict])

Examples

>>> m = LiveMetrics(started_at=0.0)
>>> m.record(speaker="S0", t0=0.0, t1=3.0, text="bonjour", now_s=3.85)
>>> round(m.report()["latency_p50_s"], 2)
0.85

latencies()

Per-item seconds between the end of the speech and its delivery.

From the END of the utterance: that is when a listener starts waiting. Measuring from the start would charge a long sentence for being long, and would reward a pipeline that chops speech into fragments.

Return type:

list[float]

record(*, speaker, t0, t1, text, now_s=None)

Record one item reaching the consumer.

t0/t1 are positions in the AUDIO timeline; now_s is the monotonic clock when the item was handed over (defaults to now). Re-recording the same (speaker, t0) with different text counts as a revision — which is what flicker is — while identical text is a duplicate delivery and is not.

Parameters:

• speaker (str)

• t0 (float)

• t1 (float)

• text (str)

• now_s (float | None)

Return type:

None

record_queue_wait(stage, seconds)

Record how long one item waited in stage’s inbox before being picked up.

This is the half of latency a total cannot show. CADENCE.md: user-visible latency is the sum of queue waits plus per-stage compute, so a slow session with no queue wait is a slow model, and the same number with a large queue wait is a starved consumer. Opposite fixes.

Parameters:

• stage (str)

• seconds (float)

Return type:

None

report()

One flat dict: latency quantiles, revisions, per-queue waits, violations.

Return type:

dict

revision_rate()

Share of deliveries that rewrote something already shown, in [0, 1].

Zero is not automatically good: a pipeline that only emits finalised text cannot flicker and pays for it in latency instead. The pair is the measurement, never either alone.

Return type:

float

started_at: float

summary()

A one-line human reading of report(), for a log or a CLI tail.

Return type:

str

property violations: list[dict]

Items emitted before their audio existed. Non-empty means do not trust the rest.