AI Helpers

Warith Harchaoui, Mohamed Chelali, Bachir Zerroug

Since April 2024
17 Python libraries and 6 Rust rewrites — BSD-3-Clause — local-first

⭐ AI Helpers on GitHub

The three AI Helpers authors: Warith Harchaoui, Mohamed Chelali and Bachir Zerroug

Introduction

AI Helpers is a suite of 17 Python libraries, licensed under BSD-3-Clause (same as scikit-learn / numpy / scipy), designed to simplify AI development across audio/video processing, live speech transcription, media acquisition, capture, secure storage, caching, vector search, and hardware-aware local-model selection. The suite is organized into groups, so you install only the corner you need: Core (OS Helper), Audio & voice (Audio, Vocal, Speaker, Discussion), Video & capture (Video, Capture), Media acquisition (YouTube, Podcast), Storage & transfer (Bucket, SFTP), Documents (md2star), Cache (Wallet), and Misc (Standpoint, Best Engine AI, ANN Router, Elbow Helper).

Every bundled helper exposes the same surfaces: a Python API, a CLI built with click (except os-helper, the foundation, which ships none), a FastAPI HTTP server, an MCP tool set mounted on that same app, and an installable Claude Code / OpenCode skill. Where it earns its place, it also adds a minimal browser GUI at /gui. The MCP surface is never a second implementation: fastapi-mcp mounts /mcp on the FastAPI app the helper already serves, so every tool is a route and the two cannot drift apart. Where a helper is meant to run as a service, the HTTP surface ships with a Dockerfile paired with a docker-compose.yml — 13 of the 17 — carrying the port, the environment and a health probe, plus the companion each one actually needs: MinIO for Bucket, a throwaway SFTP server for SFTP, Ollama for Standpoint and Best Engine AI, Qdrant and pgvector for ANN Router. The suite is local-first by design: your files, audio, camera/mic and documents are processed on your machine with open-source tooling, with no SaaS, no telemetry, no account. The two remote-transfer helpers, Bucket and SFTP, are the honest exception; their job is to move data to storage / servers you choose.

Install just the group you need — every helper is its own package and all of them are on PyPI, so you install by name (each pulls in os-helper automatically):

# 🔊 Audio & voice
pip install audio-helper vocal-helper

# 🎬 Video & capture
pip install video-helper capture-helper

# 🌐 Media acquisition
pip install youtube-helper podcast-helper

# 🗄️ Storage & transfer
pip install bucket-helper sftp-helper

# 🧩 Misc (docs, cache, positioning maps, local-model
#         selection, vector-search routing, knee detection)
pip install md2star wallet-helper standpoint best-engine-ai-helper ann-router elbow-helper

# 🧱 Core only (foundation utilities)
pip install os-helper

You still need ffmpeg on PATH for the media helpers (audio / video / youtube / podcast / capture / vocal); brew install ffmpeg on macOS 🍎.

🦀 Six of the seventeen also have an independent Rust rewrite — a separate repository with a smaller surface, not a drop-in replacement. They exist so the suite can run where Python cannot, phones included, and each takes its own platform's fast path: see 🦀 Rust ports for why that matters and how each one is accelerated. All six are published on crates.io rather than PyPI, and each is linked from its helper below:

# libraries
cargo add audio-helper-rs capture-helper-rs podcast-helper-rs youtube-helper-rs
cargo add vocal-helper-rs --features onnx

# binaries (md2star-rs, md2docx, md2pptx) — no Pandoc, no Python
cargo install md2star-rs

Contents

🔒 The Promise

The tools you build AI with should run on your machine, keep your data under your control, and never make you rent access to your own workflow.

No SaaS, no telemetry, no account, no cloud lock-in, and every line is BSD-3-Clause, so you can read it, fork it, and trust it. We hold ourselves to that honestly, and we are explicit about exactly where it does (and does not) fully apply.

How we applied it, concretely. Every bundled helper ships the same surfaces so you own the whole stack: a Python API, a click CLI, a FastAPI HTTP server, an MCP tool set for agents, an agent skill, and (where auditing the output in a browser genuinely helps) a minimal GUI at /gui, served by the same local FastAPI app with no build step, no framework, and no external calls. Each helper documents its exact trigger conditions in a TRIGGERS.md — and that catalogue is enforced, not decorative: it is mirrored into the skill's references/triggers.md, and in sixteen of the seventeen a scripts/check_triggers.py gate fails the test suite when the skill stops naming a capability or drops its SKIP clause. The two remote-transfer helpers wear the exception openly instead of pretending otherwise.

🦀 Rust ports

The same promise, on every machine. Local-first means the work happens on your machine — and a person's machines are not only laptops: they are a phone in a pocket, a tablet on a counter, a small box in a studio. A suite written only in Python keeps that promise on computers and quietly breaks it everywhere else. Closing that gap is what the Rust crates are for. They are not a side experiment: they are the same promise, carried to the devices people actually hold.

Why Python makes this hard

Not a criticism of Python — a consequence of how it ships. Five things stand between a Python library and an arbitrary device:

Rust inverts all five. Cross-compiling is a boring, first-class operation (cargo build --target aarch64-apple-ios), dependencies resolve at build time on the developer's machine, and the output is one static library or binary that runs with nothing installed. The same source tree produces artefacts for macOS, Linux, Windows, iOS and Android without the code knowing which one it is headed for.

Hardware acceleration, crate by crate

« Cross-device » is only worth something if each crate reaches the fast path its platform offers. So here is where each one actually gets its speed — including the four that have no arithmetic of their own to accelerate, which is worth saying plainly rather than dressing up.

CrateWhere its speed comes from
vocal-helper-rs The real accelerator story. Diarization and transcription run ONNX graphs, so each session picks an ONNX Runtime execution provider. Since 0.2.0 that choice is wired: the caller's choice first, then VOCAL_HELPER_PROVIDER, then a measurement on the device (a short warm-up, then 30 s of synthetic audio per provider), the verdict kept under the graph's digest. One feature per provider: cuda and tensorrt on NVIDIA, coreml on Apple, directml on Windows, openvino on Intel. A provider that fails to register is skipped with a warning, down to the CPU, so opening never fails for want of an accelerator. On Ubuntu with NVIDIA it needs CUDA 13 and cuDNN 9. CI builds cuda,tensorrt and proves the fallback, but no NVIDIA GPU has actually run it yet.
capture-helper-rs The operating system's own audio stack, through cpal: CoreAudio on macOS, WASAPI on Windows, ALSA / PulseAudio / JACK / PipeWire on Linux. No PortAudio, no ffmpeg, no extra install — the microphone path is the native one on each system.
audio-helper-rs · podcast-helper-rs Both lean on ffmpeg, and both measured before picking a fast path. audio-helper-rs chooses the encoder of a compressed output on the machine, by trial at the real sample rate and channel count; a platform encoder must do the job in under two-thirds of the time to displace the preferred one, and on Apple Silicon AudioToolbox did not. podcast-helper-rs found AudioToolbox decoders slower than ffmpeg's own, so the default stays, with --decoder to override it. A GPU build of ffmpeg changes nothing for either: NVENC, NVDEC, VAAPI and Quick Sync handle pictures, not audio.
youtube-helper-rs No arithmetic of its own: it drives yt-dlp and waits on the network. The Rust gain here is the single static binary and the typed interface, not speed, and claiming otherwise would be dishonest.
md2star-rs Markdown to DOCX, PPTX and PDF, and DOCX or PPTX back to Markdown (twin), plus the deck audit, with citations, native equations and md2star's templates, all inside the process: no Pandoc, no LibreOffice, no LaTeX, no subprocess. Conversion is XML, text measurement and typesetting, so no accelerator applies. Its hardware story is the absence of prerequisites: CI type-checks the library for iOS and Android on every push, PDF writer and Mermaid included, and the CLI is one static binary (6.4 MB, 50 MB with the in-process PDF writer). What stays with the Python original: PDF back to Markdown, full CSL styles, the GUI and the visual judge.

And one cautionary measurement, because the obvious reasoning about accelerators turned out to be wrong. ONNX Runtime has no Metal provider, so on Apple Silicon acceleration goes through CoreML — putting it first looked self-evident. Measured on the diarization graph, CoreML gives a real-time factor of 1.257, slower than real time and therefore unusable, against 0.528 for CPU alone: it takes 1193 of the graph's 2827 nodes and splits them into 316 partitions, so every streaming step pays that many crossings between two engines. Forcing eight threads on an eight-performance-core machine was wrong too (0.953 against 0.528 when the runtime chooses). The lesson is not that CoreML is bad — it will be excellent on another graph — but that a hardcoded order is wrong somewhere, which is why the Python helper measures instead of assuming. The crate does the same since 0.2.0: on an M2 Max, its first open of the diarization graph found the CPU ahead again (0.74 against 1.12 for CoreML), while on the transcription graph the two came out level (0.084 against 0.086).

And the speed? Honestly, that is not the reason

A little, sometimes. But well-accelerated Python does not need rescuing, because it already delegates the heavy work to native code: numpy computes in C, onnxruntime in C++, ffmpeg in its own binary. A Rust port calls those very same libraries, so where the work is already native, rewriting changes nothing. That was measured rather than assumed: profiling vocal-helper's diarization path, 99.8 % of the wall time is inside ONNX Runtime — the feature front end accounts for 0.1 % and the bookkeeping around the model for another 0.1 %. The ceiling on a Rust port's speed gain there is 0.2 %: a real-time factor of 0.754 instead of 0.755. Python does lose on tight per-item loops that resist vectorisation, but that is usually a reason to reshape the loop rather than change language. Anyone promising a Rust rewrite as a performance fix should be asked for the profile first.

Each crate is an independent rewrite of its helper's core promise — not a binding, not a line-by-line translation — and each is verified against the Python it mirrors rather than assumed to agree. vocal-helper-rs, for instance, compares its feature front end and its streaming bookkeeping against fixtures generated by the Python, because both feed the same ONNX graph: a silent disagreement there would produce different answers on mobile, and no unit test on either side would notice. All six crates are on crates.io today. vocal-helper-rs arrived last and is at 0.2.0: it runs diarization and transcription end to end through ONNX Runtime, though the graphs weigh hundreds of megabytes and stay outside the crate, so you pass their path. True to the section above, speed is not its argument: on the CPU of an M2 Max its transcription runs at a real-time factor of 0.05, against 0.022 to 0.028 for the Python twin.

Composed example

The helpers are designed to chain. Here the suite turns a YouTube talk into a shareable Word document and PDF — acquisition with 🌐 youtube-helper, decoding with 🔊 audio-helper, speech-to-text with 🗣️ vocal-helper, and typesetting with 📄 md2star:

import youtube_helper as yth      # 🌐 acquisition
import audio_helper as ah          # 🔊 decode to PCM
import vocal_helper as voh         # 🗣️ speech-to-text (Whisper)

URL = "https://www.youtube.com/watch?v=YE7VzlLtp-4"

# 1) Acquire: pull the talk's audio (16 kHz mono is ideal for ASR).
yth.download_audio(URL, "talk.mp3", target_sample_rate=16000)

# 2) Transcribe: Whisper on the decoded PCM (to_numpy=True → float32 array).
pcm, sr = ah.load_audio("talk.mp3", target_sample_rate=16000, to_mono=True, to_numpy=True)
transcript = voh.transcribe_pcm(pcm, sr, language="en")

# 3) Hand off to md2star: write a titled Markdown file for typesetting.
with open("talk.md", "w", encoding="utf-8") as fh:
    fh.write(f"# Talk transcript\n\n_Source: {URL}_\n\n{transcript}\n")
# 4) Typeset the transcript as a Word document and a PDF (md2star CLIs).
md2docx talk.md      # → talk.docx
md2pdf  talk.md      # → talk.pdf

Because md2star's md → docx → pdf render is faithful and reversible (see the md2star round-trip identity), the resulting documents read straight back to Markdown without losing the transcript's text.

🧱 Core

OS Helper

OS Helper logo OS Helper is the foundation every other helper builds on: OS and hardware detection, process execution, paths and files, temporary files, hashing, config loading, wall, CPU and GPU timers, and gettext catalogues. It ships no command line of its own. An optional Tree Radar page (os-helper-gui) draws disk usage as a treemap coloured by age, duplicates or file type; since 4.1.1 it carries its own copy of D3 and works offline.

🔊 Audio & voice

Audio Helper

Audio Helper logo Audio Helper decodes through ffmpeg and covers the everyday audio chores: loading with resampling and mono downmix, format conversion, splitting and concatenation, silence and room tone, MFCC-based similarity and Mel/MFCC features, plus optional source separation through Demucs. A recipe canvas at /gui chains them in the browser.

Vocal Helper

Vocal Helper logo Vocal Helper turns audio into diarized, transcribed utterances, with an optional rolling summary from a local model. It runs as an async producer/consumer pipeline, live or over a whole recording. Since 5.0.0 the defaults are Nemotron 3 for diarization and Nemotron 3.5 for transcription when the language is to be discovered (whisper.cpp when it is given), both through ONNX without torch. The diarization graph downloads once from deraison.ai, and live diarization answers with 1.04 s of latency instead of 6.4 s. pyannote, NeMo and sherpa-onnx stay available on request.

Speaker Helper

Speaker Helper logo Speaker Helper is the inverse of Vocal Helper: text in, speech out, synthesised in your own process rather than behind a server. Say a sentence, stream a reply, or narrate a whole document and get back the timed track that lets a page highlight along with the voice. Seven engines (Kokoro, OpenVoice, Chatterbox, Kyutai, audio8, qwen3tts, ElevenLabs) sit behind one router that picks from measured evidence: Kokoro when streaming, Chatterbox offline. OpenVoice, Chatterbox, audio8 and qwen3tts clone a voice from a short reference; ElevenLabs is the one paid cloud engine, used only when asked for by name.

Discussion Helper

Vocal Helper hears, Speaker Helper speaks, a local model answers — and none of them knows whose turn it is. Discussion Helper is the loop in between: listen, answer, speak, and stop the moment the user cuts in. One floor holder at a time, barge-in under 150 ms that discards queued audio rather than draining it, and a per-exchange latency breakdown that names which stage to fix. It listens through Vocal Helper 5: whisper.cpp when the language is named, Nemotron 3.5 otherwise. It runs entirely on your machine, and the whole loop can be rehearsed with no microphone, no model and no sound card.

🎬 Video & capture

Video Helper

Video Helper logo Video Helper wraps ffmpeg for video work: validation, conversion and trimming, frame extraction over several backends (VidGear, PyAV, ffmpeg pipe), optical flow, compression, concatenation, overlays and burned-in subtitles, SRT/VTT conversion, and matching on-screen faces to speakers (YuNet, SFace, Light-ASD).

Capture Helper

Capture Helper logo Capture Helper is the live input layer for the AI Helpers stack: cross-platform camera and microphone enumeration and capture, yielding (H, W, 3) BGR frames and PCM. It ships a live multi-source scene configurator GUI at /gui — pick cameras and mics, preview them live in the browser, arrange a scene, save/load it as JSON. Camera and microphone data never leave your machine.

🌐 Media acquisition

YouTube Helper

YouTube Helper logo YouTube Helper downloads video, audio and thumbnails through yt-dlp from YouTube, Vimeo, DailyMotion, Twitch and the many other sites yt-dlp knows, with ffmpeg post-processing. It browses stream catalogues, resolves direct URLs, and pulls engagement metadata and comments without any platform API. A failed download is retried in steps, browser cookies and then Tor among them.

Podcast Helper

Podcast Helper logo Podcast Helper is universal audio stream consumption: URL-in to PCM-out. It accepts local files, direct audio URLs, RSS/Atom feed URLs (auto-picks the latest episode), and any yt-dlp-supported source. Built on youtube-helper + ffmpeg + feedparser + podcastparser, with an anti-aliasing filter before every downsampling. Spotify and Apple Podcasts are refused, with a hint to use the RSS feed instead.

🗄️ Storage & transfer

Bucket Helper

Bucket Helper logo Bucket Helper covers AWS S3 and any S3-compatible object storage (MinIO, Backblaze B2, DigitalOcean Spaces, Cloudflare R2, Wasabi) via boto3. Same shape as SFTP Helper: credentials loader, upload / download / delete / exists / list_prefix, and a remote_tempfile context manager.

SFTP Helper

SFTP Helper logo SFTP Helper drives the system OpenSSH sftp client: upload and download files or whole directories (a directory travels as one tar.gz stream), resume, list, stat, delete, mkdir, plus a remote_tempfile helper. Host-key checking is always strict, and authentication goes through your SSH key.

📄 Documents

md2star

md2star logo md2star is a Markdown → DOCX / PPTX / PDF bridge built on Pandoc, with curated branded templates, Mermaid diagrams, LaTeX math and BibTeX bibliographies. twin reads a finished document back into editable Markdown, audit checks a deck for overlaps, off-canvas shapes and text that is too small, and --smart-layout places slides on a designer template's own layouts, where --min-pt sets a floor on text size. Since 3.13 an image's title is printed as its photo credit inside the frame.

♻️ Cache

Wallet Helper

Wallet Helper logo Wallet Helper makes sure you never run the same heavy call twice — a paid API request, a slow model, any expensive function. Results are persistent and content-addressed, served from a local store instead of recomputed, even across process restarts. Two identical calls in flight collapse into one via single-flight, in-process or across processes/hosts via a fenced SQLite lease or an optional HTTP dedup server.

🧩 Misc

Standpoint

Standpoint logo Standpoint turns a comparison table (options as rows, criteria as columns) into a labelled 2D positioning map, a short written analysis, and a YAML of coordinates, in a single command. Ordinary PCA under the hood; Standpoint adds the hand-work — orients the map around a reference option, names the axes in plain words, colours and labels the points.

Best Engine AI Helper

Best Engine AI Helper logo Best Engine AI Helper picks (and pulls) the best local LLM or VLM for the machine it runs on. It detects available memory (Apple Silicon unified memory, NVIDIA VRAM, system RAM), consults a bundled model catalog, selects the highest-scoring model that fits a safety headroom, pulls it via Ollama, and writes an env file downstream projects can source. It also plans text-to-image runs, sizes and streams models too large to load, and serves an OpenAI-compatible /v1 endpoint where model="best" resolves per request.

ANN Router

ANN Router logo ANN Router is the vector-search sibling of Best Engine AI Helper: measure the criteria → select the engine → return a discussable rationale, applied to approximate-nearest-neighbour search. It selects, justifies, and can instantiate the right backend among exact, turbovec, HNSW, FAISS, Annoy, Qdrant and pgvector — an indispensable component for RAG.

Elbow Helper

Elbow Helper logo Elbow Helper answers a harder question than "where might a knee or elbow be?" — an algorithm asked that will always return something, even on noise. Instead: is any candidate knee strong, unique, persistent, reproducible, and unlikely under a no-knee model? If not, it abstains. A from-scratch, NumPy-only knee locator; robust_knees covers the multi-knee case.

Frequently asked questions

Is it really free?

Yes. Every helper ships under BSD-3-Clause, the license of numpy and scikit-learn: read the code, fork it, ship it inside a product you sell. There is no account to create and no trial that runs out on a Friday evening.

Do I have to install all seventeen?

Install the one you need. Each helper is its own package on PyPI: pip install youtube-helper brings youtube-helper, its dependencies and os-helper, the foundation every helper shares. Nobody hands out a badge for collecting all seventeen.

Does anything leave my machine?

Nine helpers run entirely on your machine: os, audio, video, capture, vocal, speaker, discussion, md2star and wallet. A few of them download a model or a template once, then work offline. Two reach the outside world only where you point them: youtube-helper and podcast-helper contact the sites and feeds you name, nothing else. The two storage helpers, bucket-helper and sftp-helper, send your files to the server you pick; that is their whole job. Vocal and Speaker also support a few paid hosted engines, but call one only when you name it and paste its key. The automatic routing never picks them.

What do I need besides pip?

Python 3.10 to 3.13. The media helpers (audio, video, capture, vocal, youtube, podcast) call ffmpeg, so install it first: brew install ffmpeg on macOS. md2star also needs Pandoc; its Rust rewrite, md2star-rs, does not.

Do I need a GPU?

No. The defaults run on an ordinary laptop. vocal-helper runs its speech models through ONNX, without torch. Before trusting an accelerator, it benchmarks it on your machine. That habit has already saved us from ourselves: on an M2 Max, CoreML ran the diarization graph slower than real time (real-time factor 1.257), while the plain CPU managed 0.528.

Can my AI agent use them?

Yes. Each helper serves an MCP tool set at /mcp, on the same FastAPI server as its HTTP API, plus a skill for Claude Code or OpenCode. Its TRIGGERS.md tells the agent when to reach for the helper and when to leave it alone, which spares you an agent trying to transcribe a spreadsheet.

Are the Rust versions faster?

Rarely. We checked before anyone got their hopes up. In vocal-helper's diarization, 99.8% of the time goes to ONNX Runtime, which Rust calls exactly as Python does: rewriting the remaining 0.2% saves at most 0.2%. The six crates exist for phones, tablets and single-file installs. vocal-helper-rs even transcribes more slowly than its Python twin (real-time factor 0.05 against 0.022 to 0.028). We'd rather say so here than in the small print.

Where do I start?

Start from a job you already have. Turning a recorded talk into a Word document takes four helpers: youtube-helper fetches it, audio-helper decodes it, vocal-helper transcribes it, md2star typesets the result. The composed example shows the whole chain in about twenty lines.

Who is behind it?

Three people: Warith Harchaoui, Mohamed Chelali and Bachir Zerroug, who have been releasing the suite since April 2024. Write to warithmetics@deraison.ai about a bug, an idea or a project that needs the suite wired in. A human reads every message.

How do I say thanks?

Star the suite on GitHub: one click, no newsletter in return. It tells the next developer who lands there that the detour is worth it. Did one helper save your afternoon? It deserves a star of its own: the ⭐ in its section leads straight to its repository. Those stars tell us which helpers to polish first.

⭐ AI Helpers on GitHub · GitHub (the suite) · PyPI · 📃 Documentation · 📡 Releases feed (Atom) · llms.txt · sitemap.xml

BSD-3-Clause License — Warith Harchaoui, Mohamed Chelali & Bachir Zerroug.
Contact : warithmetics@deraison.ai