vocal_helper.accel module

vocal_helper.accel

Picking the fastest transcription path this machine can actually run.

Transcription is the one stage whose speed decides whether a live session is possible at all. In batch, a slow engine is an inconvenience; live, the sum of the stages’ real-time factors must stay under 1, or the pipeline takes on a delay it will never recover and the latency grows without bound.

So this module does not accept a default. It probes the hardware, picks the best accelerator compiled into the installed whisper.cpp, sizes the thread pool to the cores that are actually fast, and says what it chose — because an unexplained real-time factor is the hardest kind of performance problem to chase.

The ladder, measured and sourced

Measured on 60 s of real French speech, Apple Silicon, large-v3-turbo-q5 (2026-11-01, studies/asr_acceleration_rtf.md):

path	RTF	vs CPU
Apple Neural Engine (ANEForge)	—	~1.0–1.3× over Metal, ~5× less energy
Metal (GPU)	0.033	4.9× faster
CPU + BLAS	0.164	baseline

The ANE row carries no local measurement because the installed build has no ANE and no CoreML compiled in; its figures are the ones reported upstream, where the backend was merged on 2026-09-18 (ggml-org/whisper.cpp#3905). CoreML sits between the two at roughly 2× the Metal encoder, and is likewise absent here.

Two findings worth keeping, both counter-intuitive

MLX is not the fastest thing on Apple’s own hardware. whisper.cpp with Metal beat mlx-whisper by 1.85× on the same model and audio. Where the products still use mlx-whisper, it is for its word-onset accuracy, not its speed.

“Faster-whisper” is 20× slower here. CTranslate2 has no Metal path, so it stays on the CPU: real-time factor 0.657, leaving 1.5× of headroom, which does not hold a live session. The published “up to 4× faster than whisper.cpp” claims do not survive contact with Apple Silicon.

Threads: the fast cores, not all of them

Apple Silicon has performance and efficiency cores. Handing whisper.cpp a thread count that spills onto the efficiency cores makes it slower, because the fast cores then wait at every synchronisation point for the slow ones. The default was a hard-coded 6, which on a 12-core machine with 8 performance cores leaves two of them idle, and on a 4-performance-core laptop oversubscribes by half.

Usage example

>>> hw = probe()
>>> hw.performance_threads >= 1
True
>>> plan = resolve("auto")
>>> plan.accelerator in ("ane", "coreml", "metal", "cpu")
True
>>> plan.describe().startswith(plan.accelerator)
True
>>> resolve("cpu").accelerator
'cpu'

vocal_helper.accel.ACCELERATORS = ('auto', 'ane', 'coreml', 'metal', 'cpu'): Accepted values. "auto" resolves to the best the machine can actually run.

class vocal_helper.accel.AccelPlan(accelerator, threads, model_kwargs=<factory>, env=<factory>, why='')[source]

Bases: object

A resolved decision: which accelerator, how many threads, and how to get it.

Parameters:

accelerator (str)
threads (int)
model_kwargs (dict)
env (dict[str, str])
why (str)

accelerator: str

apply_env()[source]

Set this plan’s environment variables, without clobbering a caller’s.

A variable already set by whoever started the process wins: an operator who exported something did it on purpose, and silently overriding that is how a benchmark measures the wrong thing.

Return type:: None

describe()[source]

One line, for a log that has to explain itself later.

Examples

>>> line = resolve("cpu").describe()
>>> line.startswith("cpu, ") and line.endswith("asked for explicitly")
True

Return type:: str

env: dict[str, str]: Environment variables that must be set for this path to engage.

model_kwargs: dict: Extra keyword arguments for the whisper.cpp model constructor.

threads: int

why: str = '': Why this one, in one clause — so a surprising real-time factor is explainable.

class vocal_helper.accel.Hardware(system, machine, apple_silicon, performance_threads, whisper_backends=<factory>)[source]

Bases: object

What this machine offers, as far as transcription is concerned.

Parameters:

system (str)
machine (str)
apple_silicon (bool)
performance_threads (int)
whisper_backends (frozenset[str])

apple_silicon: bool

describe()[source]

One line naming the machine and what it can run.

Returns:: e.g. "Darwin/arm64, 8 fast threads, backends: metal".
Return type:: str

Examples

>>> "threads" in probe().describe()
True

machine: str: Processor architecture, e.g. "arm64".

performance_threads: int: Cores worth giving to whisper.cpp — see the module docstring on why not all.

system: str: "Darwin", "Linux", "Windows".

whisper_backends: frozenset[str]: Which of {"metal", "coreml", "ane", "cuda"} the installed build carries.

vocal_helper.accel.probe()[source]

Describe this machine once, and remember it.

Returns:: Cached for the process: the answer cannot change while it runs, and the probe shells out to sysctl, which is not free in a hot loop.
Return type:: Hardware

Examples

>>> probe() is probe()
True
>>> probe().system == platform.system()
True

vocal_helper.accel.resolve(accelerator='auto', *, threads=None)[source]

Decide the transcription acceleration path for this machine.

Parameters:

accelerator (str, optional) – One of ACCELERATORS. "auto" (default) takes the best the installed build can run. An explicit value is honoured even when the probe says it is unavailable, because an override exists precisely for the case where the probe is wrong — but the reason recorded on the plan says so, so a disappointing result is traceable.
threads (int, optional) – Override the thread count. None sizes it to the fast cores.

Returns:

What to do, and why.

Return type:

AccelPlan

Raises:

ValueError – On an unknown accelerator name, at decision time rather than deep inside a model constructor.

Examples

>>> resolve("metal", threads=4).threads
4
>>> resolve("cpu").model_kwargs
{'context_params': {'use_gpu': False}}
>>> resolve("nvidia")
Traceback (most recent call last):
    ...
ValueError: unknown accelerator 'nvidia'; choose from ane, auto, coreml, cpu, metal