youtube_helper package

Submodules

Module contents

YouTube Helper

Utilities for downloading videos, audio, thumbnails, resolving direct media URLs, and pulling public engagement metadata — from YouTube, Vimeo, DailyMotion, Twitch VOD, and any other site yt-dlp supports.

Modules

  • maindownload helpers (videos / audio / thumbnails) and

    metadata extraction.

  • streamingURL resolver that turns a user-facing page URL into a

    direct media URL (with the right headers / cookies) so an ffmpeg-based reader can consume it without going through yt-dlp again. Pure resolution — no PCM decoding (that’s podcast_helper’s job).

  • brandingno-API engagement helpers (channel info, video lists

    with normalised metrics, subtitles, comments). Built on yt-dlp’s public metadata — no Google Data API, no Vimeo API, no OAuth, no quota.

Multi-surface exposure

The same functions are also available as:

  • youtube_helper.cli — the command line (click), entry point youtube-helper. No extra dependency.

  • youtube_helper.api — FastAPI HTTP surface. Requires the [api] extra.

Dependencies - yt-dlp - ffmpeg (on PATH) - os-helper / audio-helper / video-helper

Usage Example

>>> import youtube_helper as yth
>>> meta = yth.video_url_meta_data("https://www.youtube.com/watch?v=YE7VzlLtp-4")
>>> yth.download_audio("https://www.youtube.com/watch?v=YE7VzlLtp-4", "out.mp3")

Author

Warith Harchaoui, Ph.D. — https://linkedin.com/in/warith-harchaoui/

class youtube_helper.DirectMediaURL[source]

Bases: TypedDict

Resolved direct-media URL ready for ffmpeg.

Keys

urlstr

The direct media URL. Can be an HLS .m3u8 (live), DASH .mpd, or progressive .mp4 / .webm — whichever yt-dlp selected.

containerstr

Container format string ("hls", "dash", "mp4", "webm", "audio", …). Useful for ffmpeg’s -f hint if the URL extension is ambiguous.

is_livebool

True if the source is a live stream. Callers may want to skip -re (real-time pacing) since live is already real-time.

headersdict[str, str]

HTTP headers the player must send with the request. YouTube often requires a specific User-Agent and Referer to avoid 403s; pass them to ffmpeg via -headers.

has_audiobool

Whether this single URL already carries sound. prefer="audio" always yields True; prefer="video" yields False whenever the site only offers video-only streams, which is now YouTube’s normal answer. Callers needing both tracks resolve a second URL with prefer="audio" and let ffmpeg take one input from each.

container: str
has_audio: bool
headers: dict[str, str]
is_live: bool
url: str
class youtube_helper.VideoStreamInfo[source]

Bases: TypedDict

One video format’s metadata, normalised across extractors.

Keys

format_idstr

yt-dlp’s stable identifier for this format on this URL.

urlstr

The direct media URL — ready for ffmpeg / PyAV.

headersdict[str, str]

HTTP headers ffmpeg must send with the request (User-Agent, Referer, Cookie, …). Pass through to video_helper.extract_frames(http_headers=...) or to ffmpeg’s -headers flag.

extstr

File extension reported by yt-dlp ("mp4" / "webm" / "m3u8" / …).

containerstr

Normalised container hint ("mp4" / "webm" / "hls" / "dash") — useful when the ext is ambiguous.

vcodecstr

Video codec ("h264" / "vp9" / "av01" / "none" if audio-only — filtered out of this list).

acodecstr

Audio codec ("none" for video-only formats — common on YouTube DASH, paired with a separate audio-only format).

width, heightint

Pixels. 0 if unknown (rare).

fpsfloat

Frames per second. 0.0 if unknown.

vbr_kbpsfloat

Estimated video bitrate in kbit/s. 0.0 if unknown.

filesize_bytesint

Estimated file size in bytes. 0 if unknown (always 0 for live).

is_livebool

True if this is a live stream (URL is an HLS / DASH manifest).

protocolstr

yt-dlp’s protocol string ("https" / "m3u8_native" / "http_dash_segments" / …).

languagestr | None

Language code if the format carries an explicit one (rare for video; common for audio).

notestr

yt-dlp’s format_note — typically a human-readable label ("1080p60", "Premium", …).

acodec: str
container: str
ext: str
filesize_bytes: int
format_id: str
fps: float
headers: dict[str, str]
height: int
is_live: bool
language: str | None
note: str
protocol: str
url: str
vbr_kbps: float
vcodec: str
width: int
youtube_helper.channel_info(url, verbose=False)[source]

Return one normalised dict describing the channel / creator page.

Works with YouTube channel URLs (/@handle, /channel/UC…), Vimeo user pages, DailyMotion creators. Pulls subscriber count, channel title, description, total view count when available.

Parameters:
Return type:

dict[str, Any]

youtube_helper.channel_videos(url, max_videos=200, include_shorts=True, include_lives=False, verbose=False)[source]

List videos of a channel / creator page with per-video engagement metadata.

For YouTube channels, walks the uploads playlist; for Vimeo / DailyMotion, walks the user page. Each entry is normalised to the same engagement schema (id, url, title, duration_seconds, view_count, like_count, comment_count, upload_date, kind, …).

Two-pass strategy: a fast flat listing first (one call per channel tab walked — see _channel_listing_urls()), then per-video metadata fetch (1 call per video). The second pass is what makes view_count / like_count reliable — extract_flat returns IDs and sometimes view_count, but rarely likes / comments.

Parameters:
Return type:

list[dict[str, Any]]

youtube_helper.default_ytdlp_options(overwrites=True, audio=False, video=False, cookie_dir=None, max_height=None, player_clients=None)[source]

Generate default yt-dlp options based on provided parameters.

Parameters:

  • overwrites (bool, optional) – Whether to overwrite existing files. Default is True.

  • audio (bool, optional) – Whether to download audio only. Default is False.

  • video (bool, optional) – Whether to download video only (overrides audio). Default is False.

  • cookie_dir (str or None, optional) – Directory in which yt-dlp’s session cookie jar is written. The download helpers pass their temporary_folder here so the jar disappears when the with block exits; when omitted it falls back to the system temp directory. It is never written to the current working directory.

  • max_height (int or str or None, optional) – Cap the video height, e.g. 720. None falls back to the YOUTUBE_HELPER_MAX_HEIGHT environment variable, and if that is empty too, yt-dlp picks best-available as before. See height_capped_format() for why the generated selector also prefers non-HLS.

  • player_clients (sequence of str or None, optional) – Ask YouTube as these clients instead of yt-dlp’s defaults (web_embedded, tv, android_vr, mweb, …). Mostly useful for pinning one explicitly; callers normally leave this alone and let the retry ladder try the clients in _PLAYER_CLIENT_FALLBACKS on failure.

Returns:

A dictionary of options for yt-dlp.

Return type:

dict

Notes

The function sets various options for yt-dlp. If audio is set to True, the format is set to download the best audio available. If video is set to True, it overrides the audio flag and downloads video+audio.

youtube_helper.download_audio(url, output_path=None, target_sample_rate=44100)[source]

Download the best quality audio from a given URL and save it to the specified output path.

Parameters:

  • url (str) – The URL of the video to download the audio from.

  • output_path (str, optional) – The path where the downloaded audio should be saved.

  • target_sample_rate (int, optional) – The sample rate of the output audio file. Defaults to 44100.

Returns:

Path of downloaded audio

Return type:

str

Notes

This function uses yt-dlp to download the best quality audio from the given URL. When the extension yt-dlp delivers (.webm / .m4a on YouTube) differs from output_path’s, the file is converted through audio_helper.sound_converter: target_sample_rate is applied there and the result is mono. When the extensions match, the file is copied untouched and keeps the source’s rate and channels.

youtube_helper.download_thumbnail(url, output_path=None)[source]

Download the thumbnail of a video from a given URL and save it to the specified output path.

Parameters:

  • url (str) – The URL of the video from which to download the thumbnail.

  • output_path (str, optional) – The path where the downloaded thumbnail should be saved.

Returns:

Path of downloaded thumbnail

Return type:

str

Notes

This function uses yt-dlp to download the thumbnail of the specified video. It handles different output formats and ensures the thumbnail is saved in the desired format using PIL.

youtube_helper.download_video(url, output_path=None, max_height=None)[source]

Download video from a given URL and save it to the specified output path.

Parameters:

  • url (str) – The URL of the video to be downloaded.

  • output_path (str, optional) – The path where the downloaded video should be saved.

  • max_height (int or str or None, optional) – Cap the video height, e.g. 720. None falls back to YOUTUBE_HELPER_MAX_HEIGHT, and if that is empty too, yt-dlp picks best-available as it always has. Worth setting when the destination has a size limit, or when speed matters more than resolution: the capped selector also prefers direct https formats over fragmented HLS, which YouTube throttles heavily (see height_capped_format()).

Returns:

Path of downloaded video

Return type:

str

Notes

This function downloads the best quality video from the given URL using yt-dlp. It handles different formats and ensures that the video is saved in the desired format using ffmpeg if necessary. The function checks whether the downloaded video is valid and converts it to the specified output format if needed.

youtube_helper.engagement_batch(urls, verbose=False)[source]

Map a list of URLs to normalised engagement dicts, one per URL.

Tolerant: bad URLs / unavailable videos are reported as {“url”: …, “_error”: “…”} rather than raising.

Parameters:
Return type:

list[dict[str, Any]]

youtube_helper.ensure_recent_ytdlp(min_version=None)[source]

Return the installed yt-dlp version. Warn if older than min_version.

YouTube changes its signature scheme roughly every quarter; an outdated yt-dlp is the #1 cause of failures. Callers that mind freshness can surface a warning early.

Parameters:

min_version (str | None)

Return type:

str

youtube_helper.extract_frames_stream(url, *, prefer_codec=None, prefer_format=None, max_fps=None, language=None, include_video_only=True, include_combined=True, cookies_from_browser=None, verbose=False, **extract_frames_kwargs)[source]

Stream frames from any yt-dlp-supported URL through video_helper.extract_frames().

One-call composition of:

  1. pick_video_stream() — resolves url to a single VideoStreamInfo matching the codec / format / fps / language constraints.

  2. video_helper.extract_frames() — opens the resolved direct media URL with the right http_headers and yields frames under the destination of your choice (numpy BGR, torch RGB, or PIL).

Why this exists

The two-step pattern is the published recipe (see ai-helpers/EXAMPLES.md and youtube-helper/EXAMPLES.md):

>>> stream = yth.pick_video_stream(url, prefer_codec="h264")
>>> for frame in vh.extract_frames(
...     stream["url"], http_headers=stream["headers"],
...     destination="torch", device="auto", batch_size=32,
... ):
...     model(frame)

extract_frames_stream collapses both calls into one, plumbs the headers automatically, and lets the caller forward any extract_frames kwarg (destination, device, batch_size, output_width, frame_step, …) verbatim.

param url:

Any URL yt-dlp can extract (YouTube / Vimeo / DailyMotion / Twitch VOD / …). Live streams are accepted as well — the returned iterator runs until the live source ends.

type url:

str

param prefer_codec:

Forwarded to pick_video_stream(). See its docstring for the matching semantics. Defaults pick yt-dlp’s best candidate.

type prefer_codec:

optional

param prefer_format:

Forwarded to pick_video_stream(). See its docstring for the matching semantics. Defaults pick yt-dlp’s best candidate.

type prefer_format:

optional

param max_fps:

Forwarded to pick_video_stream(). See its docstring for the matching semantics. Defaults pick yt-dlp’s best candidate.

type max_fps:

optional

param language:

Forwarded to pick_video_stream(). See its docstring for the matching semantics. Defaults pick yt-dlp’s best candidate.

type language:

optional

param include_video_only:

Forwarded to pick_video_stream(). Defaults keep both.

type include_video_only:

bool, optional

param include_combined:

Forwarded to pick_video_stream(). Defaults keep both.

type include_combined:

bool, optional

param cookies_from_browser:

Forwarded to pick_video_stream(). Required for age-gated / members-only content.

type cookies_from_browser:

str, optional

param verbose:

Forwarded to pick_video_stream().

type verbose:

bool, optional

param **extract_frames_kwargs:

Any keyword accepted by video_helper.extract_frames(): start_index / end_index / start_instant / end_instant / stabilize / frame_step / frame_interval / frame_indices / frame_times / backend / hwaccel / output_width / output_height / pad_color / destination / device / batch_size / layout.

Note: http_headers is wired automatically from the resolved stream — passing it yourself overrides the resolver’s headers (rare; mostly useful when the caller already authored a stable header dict).

returns:

The same iterator type that video_helper.extract_frames() returns — numpy / torch / PIL frames or batches per the passed-in destination and batch_size.

rtype:

Iterator

raises RuntimeError:

If yt-dlp can’t extract the URL (private / removed / geo-blocked).

raises ValueError:

If no stream matches the codec / format / fps / language constraints (the catalog itself was non-empty, but every entry got filtered out).

Examples

>>> import youtube_helper as yth
>>> # Sequential decode at 1 frame per second, 224×224 letterboxed.
>>> for frame in yth.extract_frames_stream(
...     "https://www.youtube.com/watch?v=YE7VzlLtp-4",
...     prefer_codec="h264", prefer_format="mp4", max_fps=30,
...     frame_interval=1.0, output_width=224, output_height=224,
... ):
...     model(frame)

>>> # Batched torch tensors on a GPU device — same downstream code as
>>> # the file-based path.
>>> for batch in yth.extract_frames_stream(
...     "https://vimeo.com/123",
...     destination="torch", device="auto", batch_size=32,
... ):
...     logits = model(batch.float() / 255.0)
Parameters:

  • url (str)

  • prefer_codec (str | None)

  • prefer_format (str | None)

  • max_fps (float | None)

  • language (str | None)

  • include_video_only (bool)

  • include_combined (bool)

  • cookies_from_browser (str | None)

  • verbose (bool)

  • extract_frames_kwargs (Any)

Return type:

Any

youtube_helper.height_capped_format(max_height)[source]

yt-dlp format selector for “no taller than max_height, and avoid HLS if possible”.

Non-HLS first on purpose. YouTube throttles fragmented HLS far more aggressively than a direct https format: on one measured episode, 75-300 KiB/s across 1507 fragments against ~2 MiB/s for the equivalent https stream. protocol^=http keeps https and http_dash_segments while excluding m3u8_native.

Every rung degrades gracefully: height-capped non-HLS, then height-capped anything, then best available, so a video that simply has no small format still downloads instead of failing with “Requested format is not available”.

Parameters:

max_height (int) – Maximum video height in pixels, e.g. 720.

Returns:

A yt-dlp format string.

Return type:

str

youtube_helper.is_short(meta)[source]

Heuristic — true if the entry looks like a Short / vertical clip.

Looks at duration (≤ 60s), webpage_url (/shorts/), and #Shorts in title.

Parameters:

meta (dict[str, Any])

Return type:

bool

youtube_helper.is_valid_video_url(url)[source]

Check if a given URL is a valid video URL using yt-dlp.

Parameters:

url (str) – The URL to check.

Returns:

True if the URL is valid, False otherwise.

Return type:

bool

Notes

This function extracts metadata from the URL using yt-dlp to determine if the URL points to a valid video.

youtube_helper.list_video_streams(url, *, include_video_only=True, include_combined=True, cookies_from_browser=None, verbose=False)[source]

Enumerate every video format yt-dlp finds for this URL.

Results are ordered by yt-dlp’s quality ranking (best last). Pure video-only formats and combined (video + audio) formats are both included by default — adjust with the boolean flags.

Parameters:

  • url (str) – Any URL yt-dlp can extract.

  • include_video_only (bool, optional) – Keep formats with acodec="none" (DASH-style streams that need a separate audio mux). Default True.

  • include_combined (bool, optional) – Keep formats with both video and audio (legacy progressive streams; common on Vimeo, rare on YouTube high-quality). Default True.

  • cookies_from_browser (str, optional) – Pass through to yt-dlp’s --cookies-from-browser (e.g. "firefox" / "chrome" / "safari") for age-gated or login-required content. Tried on every retry tier below, ahead of whichever browser YOUTUBE_HELPER_COOKIES_FROM_BROWSER names.

  • verbose (bool, optional) – Echo yt-dlp’s output to stderr (off by default).

Returns:

Possibly empty if the URL has no video (audio-only podcast, for example).

Return type:

list[VideoStreamInfo]

Notes

Goes through the same browser-UA / cookies-from-browser / Tor retry chain as youtube_helper.main.video_url_meta_data() (see the README’s “Download resilience” section) — a bot-check or rate-limit on the first attempt is not the end of the story.

Raises:

RuntimeError – If yt-dlp could not extract info at all (private video, geo-blocked without bypass, removed, …).

Parameters:

  • url (str)

  • include_video_only (bool)

  • include_combined (bool)

  • cookies_from_browser (str | None)

  • verbose (bool)

Return type:

list[VideoStreamInfo]

youtube_helper.pick_video_stream(url, *, prefer_codec=None, prefer_format=None, max_fps=None, language=None, include_video_only=True, include_combined=True, cookies_from_browser=None, verbose=False)[source]

Pick a single best video stream matching the given constraints.

Among the catalog returned by list_video_streams(), applies each non-None constraint as a hard filter, then returns the highest-ranked remaining candidate (yt-dlp’s native order — typically sorts by height × bitrate).

Parameters:

  • url (str) – Source URL (any yt-dlp-supported site).

  • prefer_codec (str, optional) – Substring match against vcodec ("h264" matches both "avc1.640028" and "h264" once normalised; "vp9" matches "vp09.00.40.08"; "av1" matches "av01...").

  • prefer_format (str, optional) – Equality match against ext ("mp4" / "webm").

  • max_fps (float, optional) – Drop formats with fps > max_fps.

  • language (str, optional) – Equality match against the format’s language (rarely set for video — primarily useful when the source carries multi-track video).

  • include_video_only (bool, optional) – See list_video_streams().

  • include_combined (bool, optional) – See list_video_streams().

  • cookies_from_browser (same as list_video_streams().)

  • verbose (same as list_video_streams().)

Returns:

The chosen stream.

Return type:

VideoStreamInfo

Raises:

  • RuntimeError – If yt-dlp can’t extract the URL.

  • ValueError – If no stream matches the constraints (the catalog itself was non-empty, but every entry got filtered out).

youtube_helper.resolve_direct_url(url, *, prefer='audio', live='auto')[source]

Resolve url to a direct, ffmpeg-readable media URL.

Parameters:

  • url (str) – Any URL yt-dlp can extract (YouTube / Vimeo / DailyMotion / Twitch / …).

  • prefer ("audio" | "video") – Format preference. "audio" selects bestaudio — cheapest path for ASR + diarization. "video" selects the best stream carrying pictures, for the /video view (Phase 7) that needs the visual track too. A progressive stream (video and audio in one file) is preferred, but YouTube no longer publishes one for most videos, so the usual answer is a video-only stream: check has_audio on the result if you need sound in the same URL, and resolve the audio separately with prefer="audio" when it is False.

  • live ("auto" | "force_live" | "force_vod") – "auto" propagates whatever the info dict says (the usual case). The other two override — handy for testing or when the site mislabels a stream.

Returns:

See class docstring.

Return type:

DirectMediaURL

Raises:

RuntimeError – If yt-dlp could not extract a usable URL (private video, geo-blocked without bypass, removed, …).

Notes

Goes through the same browser-UA / cookies-from-browser / Tor retry chain as youtube_helper.main.video_url_meta_data() (see the README’s “Download resilience” section) — a bot-check or rate-limit on the first attempt is not the end of the story.

youtube_helper.video_comments(url, max_count=100, cookies_from_browser=None, verbose=False)[source]

Fetch comments via yt-dlp’s getcomments.

YouTube increasingly requires browser cookies for comment scraping. If cookies_from_browser is set (e.g. “firefox”, “chrome”, “safari”), yt-dlp will pull cookies from that browser’s profile. On macOS this is the only reliable way to get YouTube comments today.

Returns a list of normalised comment dicts: id, parent, author, text, like_count, timestamp.

Parameters:

  • url (str)

  • max_count (int)

  • cookies_from_browser (str | None)

  • verbose (bool)

Return type:

list[dict[str, Any]]

youtube_helper.video_engagement(url, verbose=False)[source]

Single-video engagement snapshot. Returns the normalised dict shape.

Parameters:
Return type:

dict[str, Any]

youtube_helper.video_subtitles(url, output_dir, langs=('fr', 'en'), auto_only=True, verbose=False)[source]

Download auto-generated (or manual) subtitles into output_dir.

Returns a dict mapping language → absolute path of the saved .vtt file. Languages that aren’t available are silently skipped.

Parameters:
Return type:

dict[str, str]

youtube_helper.video_url_meta_data(url)[source]

Retrieve metadata from a video URL without downloading the video.

Parameters:

url (str) – The URL of the video.

Returns:

A dictionary containing the video’s metadata, including title and description.

Return type:

dict

Notes

The function checks for metadata extraction success, logs the title and a part of the description, and returns metadata.