discussion_helper.remote module

The microphone at the other end of a socket.

Module summary

A browser has a microphone and a speaker; a server has the model. That is the ordinary shape of a voice agent, and it is a real product rather than a compromise — every hosted voice agent works this way. The audio crosses the wire as PCM, which is what WebRTC and WebSocket transports carry underneath their own framing.

So the two device-bound parts get a second implementation each. The RemoteListener reads frames a client pushes in instead of opening a microphone; the RemoteVoice hands synthesised audio back to the client instead of a sound card. The loop does not know the difference — which is the point of it being written against protocols.

What crossing a network changes, and what it does not

It does not change the floor, the interruption, or the budget’s shape. It changes two things, and both are honest to state:

The budget grows a network leg, 30–80 ms each way on a good link and a tail that is much worse than its median. On a local-first deployment — the browser and the server on one machine — it is nothing. Across the internet it is the line that decides whether 800 ms is reachable at all.

Echo moves to the client. The server never sees the room, so it cannot tell the agent’s voice from the user’s. The browser can: getUserMedia has had echoCancellation for years, and it is better than anything this package could do at this end. A client that turns it off gets the self-interruption described in KNOWLEDGE.md §4, and no server-side grace period fully rescues it.

Author

Warith HARCHAOUI — https://linkedin.com/in/warith-harchaoui/

class discussion_helper.remote.RemoteListener(*, recogniser=None, sample_rate=16000)

Bases: object

Hears whatever a client pushes in.

Two ways to push, because two kinds of client exist and pretending otherwise would force one of them to lie:

• push_text() — the client already has a transcript. A browser using the Web Speech API, a phone system with its own recogniser, a test.

• push_audio() — the client has PCM and wants this end to listen. Frames are handed to a recogniser the caller supplies; without one, this raises rather than silently dropping the audio.

speech_started() is driven by mark_speaking(), which the transport calls when its own voice activity detection fires. It is kept separate from the audio path on purpose: a barge-in has to travel on the first syllable, and waiting for a transcript would make it a sentence late — on a network, a sentence plus a round trip.

Parameters:

• recogniser (callable or None) – (pcm, sample_rate) -> str, awaited. Required only for push_audio().

• sample_rate (int) – What the client is sending. 16 kHz is what every recogniser in this suite wants and what capture-helper produces.

async aclose()

Return type:

None

end()

No more turns are coming; the conversation may finish.

Return type:

None

mark_speaking()

The client’s VAD says somebody started talking.

Called on the first syllable, not on the first word: this is what releases a barge-in, and a barge-in that waited for a word would be late by a word plus a network leg.

Return type:

None

async push_audio(pcm)

Hand in one completed utterance’s worth of PCM.

A completed utterance, not a frame: deciding where a turn ends is turn detection, and this package does none of it (KNOWLEDGE.md §3). A transport that has raw frames should run them through vocal-helper’s VAD and end-of-turn stages and call this with what comes out.

Parameters:

pcm (bytes)

Return type:

None

push_text(text)

Hand in a finished turn the client already transcribed.

Parameters:

text (str)

Return type:

None

async speech_started()

Return when the client’s VAD reports speech.

Return type:

None

async utterances()

Yield each turn the client pushed, in order.

Return type:

AsyncIterator[Utterance]

class discussion_helper.remote.RemoteVoice(send, *, on_stop=None, synthesise=None, voice='', language='')

Bases: object

Sends the audio back instead of playing it.

The sentence is synthesised here — the model is on this end — and the samples go out over whatever the transport hands in. What makes this a voice rather than a function call is stop(): it fires a cancellation the transport is expected to honour by discarding whatever it has buffered, not by letting it drain. A client that plays out its buffer reproduces, at the far end, exactly the deafness KNOWLEDGE.md §4 is about — and from the server’s logs the interruption will look like it worked.

Parameters:

• send (callable) – (wav_bytes, sentence) -> None, awaited, called once per sentence.

• on_stop (callable or None) – Awaited when an interruption lands, so the transport can tell its client to drop what it is holding.

• synthesise (callable or None) – (sentence) -> (wav_bytes, duration_s), awaited. Defaults to speaker-helper.

• voice (str)

• language (str)

async aclose()

Release the engine.

Return type:

None

async speak(sentence)

Synthesise one sentence and push it to the client.

Parameters:

sentence (str)

Return type:

SpokenSentence

async stop()

Stop sending, and tell the client to drop what it holds.

Return type:

None