best_engine_ai_helper.imagine module

imagine — pick and run a local text-to-image model for this machine.

The third kind of model this package chooses. llm and vlm answer with text; embed answers with a vector; image answers with pixels. The job is the same one as everywhere else here — given this hardware, which model should run, and will it be usable? — and so is the shape of the answer: a footprint measured against the accelerator’s real budget, a speed estimate in the unit a human waits in, and a catalogue entry rather than a hardcoded name.

What is different about a diffusion model

A language model is one stack of layers. A diffusion model is three or four separate networks that take turns: one or two text encoders turn the prompt into conditioning, a denoiser (a UNet for the Stable Diffusion 1.x/XL generation, a transformer for SD3 and FLUX) runs the sampling loop, and a VAE decodes the final latent into an image. Two consequences, both load-bearing here:

  1. Memory depends on how they take turns. Held all at once, the footprint is the sum of every component. Handed to diffusers’ enable_model_cpu_offload(), only the component currently computing stays resident and the peak becomes the largest one — the same trade layer streaming makes for an LLM (see best_engine_ai_helper.offload), at a much smaller penalty, because the components swap a handful of times per image rather than once per token.

  2. Time is steps, not tokens. The sampling loop runs a fixed number of denoising steps, each one a full pass through the denoiser, so the honest unit is seconds per image and the biggest lever on it is the step count — which is a property of the model, not a preference: SDXL-Turbo and FLUX.1-schnell are distilled for 1-4 steps where SD 1.5 wants 25-30.

The runtime is diffusers, imported lazily and behind the imagine extra. Nothing above this line needs it: ranking, the footprint arithmetic and the timings are plain Python over a catalogue and answer on an interpreter that has never heard of torch. Nothing in this module reimplements sampling either — it decides which model and whether, and generate() is a thin adapter over the runtime for callers who install it.

Author

Warith Harchaoui <warithmetics@deraison.ai>

best_engine_ai_helper.imagine.COMFORT_SECONDS_PER_IMAGE: float = 90.0

Seconds per image past which generating locally stops being worth it. Ninety seconds is about the point where a person stops waiting and starts doing something else, which changes the job from interactive to batch. Reported as a blocker rather than enforced: a batch job may well accept it.

best_engine_ai_helper.imagine.INSTALL_HINT = "pip install --upgrade 'best-engine-ai-helper[imagine]'"

What to tell a caller whose interpreter cannot import it. Behind an extra, so reaching this ordinarily means the feature was never opted into.

best_engine_ai_helper.imagine.OPENAI_SIZES: dict[str, tuple[int, int]] = {'1024x1024': (1024, 1024), '1024x1536': (1024, 1536), '1024x1792': (1024, 1792), '1536x1024': (1536, 1024), '1792x1024': (1792, 1024), '256x256': (256, 256), '512x512': (512, 512), '768x768': (768, 768)}

Output sizes the OpenAI image API names, mapped to pixel dimensions. auto resolves to the model’s own native resolution, which is the only size a diffusion model is actually good at — asking SD 1.5 for 1024x1024 produces duplicated limbs rather than more detail.

best_engine_ai_helper.imagine.PACKAGE = 'diffusers'

Name of the upstream runtime, spelled once.

best_engine_ai_helper.imagine.available()[source]

Whether the diffusers runtime is importable in this interpreter.

False on a base install: it lives behind the imagine extra. Checked rather than assumed for the same reason offload.available() is — declared and importable are different claims, and a stripped container separates them even when the extra was asked for. A real import rather than find_spec for the same reason too: a package whose __init__ raises has a findable spec and is still unusable, which is how lightweight behaves on a Mac without mlx.

Returns:

True when import diffusers succeeds.

Return type:

bool

Examples

>>> available() in (True, False)
True
best_engine_ai_helper.imagine.bytes_per_param(entry, dtype=None)[source]

Bytes each parameter occupies while the pipeline samples.

Parameters:

  • entry (dict[str, Any]) – Catalog entry; reads dtype (default fp16).

  • dtype (str or None) – Override, for a caller who is going to load the pipeline in something else (nf4 for a 4-bit FLUX, say).

Returns:

Bytes per parameter.

Return type:

float

Examples

>>> bytes_per_param({})
2.0
>>> bytes_per_param({"dtype": "fp32"})
4.0
>>> bytes_per_param({"dtype": "fp16"}, dtype="nf4")
0.5
>>> bytes_per_param({"dtype": "nonsense"})
2.0
best_engine_ai_helper.imagine.footprint_gb(entry, *, dtype=None, offload=False, width=None, height=None)[source]

Resident memory (GB) a pipeline needs to produce one image.

weights x overhead + latents, where which weights depends on how the components take turns:

  • All resident (offload=False, the fast path): every component at once, so the weight term is the whole parameter count.

  • Component offloaded (offload=True): diffusers keeps only the component currently computing on the device, so the peak is the largest one — the denoiser, at _DENOISER_SHARE of the total unless the entry names its components. Slower by the handful of transfers per image that costs, and the difference between running FLUX on a 24 GB card and not running it.

The latent term is what makes resolution expensive: it scales with the output area, not with the model.

Parameters:

  • entry (dict[str, Any]) – Catalog entry; needs size_b.

  • dtype (str or None) – Compute dtype override.

  • offload (bool) – Size for a component-offloaded run rather than an all-resident one.

  • width (int or None) – Output size. None uses the model’s native_size(), which is the size it is good at anyway.

  • height (int or None) – Output size. None uses the model’s native_size(), which is the size it is good at anyway.

Returns:

Resident GB, or inf when the entry carries no parameter count — infinity rather than zero, so an unsized entry can never read as “fits everywhere” and win every comparison.

Return type:

float

Examples

>>> sdxl = {"id": "sdxl", "kind": "image", "size_b": 3.5, "native_size": "1024x1024"}
>>> footprint_gb(sdxl)
9.8
>>> footprint_gb(sdxl, offload=True)
7.2
>>> footprint_gb(sdxl, width=512, height=512)
9.0
>>> footprint_gb({"id": "sd15", "size_b": 1.07, "native_size": "512x512"})
2.9
>>> footprint_gb({"id": "unsized", "kind": "image"})
inf
best_engine_ai_helper.imagine.generate(prompt, *, model, width=1024, height=1024, steps=28, guidance=None, negative_prompt=None, seed=None, dtype=None, offload=False, hf_token=None, loaded=None)[source]

Generate one image and return it as PNG bytes.

Parameters:

  • prompt (str) – What to draw.

  • model (str) – HuggingFace repository id, as in load().

  • width (int) – Output size. Stay near the model’s native_size(): a diffusion model asked for an unfamiliar resolution does not produce more detail, it produces duplicated subjects.

  • height (int) – Output size. Stay near the model’s native_size(): a diffusion model asked for an unfamiliar resolution does not produce more detail, it produces duplicated subjects.

  • steps (int) – Denoising steps. Use the model’s own figure (steps_for()) — a distilled checkpoint at 28 steps is slower and worse.

  • guidance (float or None) – Classifier-free guidance scale. None uses the pipeline’s default, which is the right answer for a distilled model (they want 0.0) and for a normal one (7.5) alike, and those two defaults are why this is not a house constant.

  • negative_prompt (str or None) – What to avoid. Ignored by distilled pipelines that run without guidance; passed through when given.

  • seed (int or None) – Seed for reproducibility. None leaves the generator unseeded.

  • dtype (str | None) – Passed to load() when it has to load the pipeline.

  • offload (bool) – Passed to load() when it has to load the pipeline.

  • hf_token (str | None) – Passed to load() when it has to load the pipeline.

  • loaded (Any or None) – A pipeline from a previous load(), reused as-is. Loading one is seconds to minutes, so a caller making several images should pay it once.

Returns:

The image encoded as PNG.

Return type:

bytes

Raises:

ImportError – When diffusers cannot be imported.

Examples

>>> generate("a red bicycle", model="stabilityai/sdxl-turbo")
best_engine_ai_helper.imagine.hf_id(entry)[source]

The HuggingFace repository diffusers would load.

Image models have no Ollama registry to fall back on, so an entry that names no repository names nothing that can run. hf_id is preferred; vllm_id is accepted because the catalog has always spelled a Hub id that way.

Parameters:

entry (dict[str, Any]) – Catalog entry.

Returns:

The repository id, or None when the entry names none.

Return type:

str or None

Examples

>>> hf_id({"id": "sdxl", "hf_id": "stabilityai/stable-diffusion-xl-base-1.0"})
'stabilityai/stable-diffusion-xl-base-1.0'
>>> hf_id({"id": "sdxl"}) is None
True
best_engine_ai_helper.imagine.is_image_model(entry)[source]

Whether a catalog entry is a text-to-image model.

Parameters:

entry (dict[str, Any]) – Catalog entry.

Returns:

True when its kind is image.

Return type:

bool

Examples

>>> is_image_model({"id": "sdxl", "kind": "image"})
True
>>> is_image_model({"id": "qwen3:8b", "kind": "llm"})
False
best_engine_ai_helper.imagine.load(model, *, dtype=None, offload=False, hf_token=None)[source]

Load a diffusers pipeline.

The adapter, and the only function here that touches the upstream package.

Parameters:

  • model (str) – HuggingFace repository id, or a local path to a saved pipeline.

  • dtype (str or None) – Compute dtype (fp16, bf16, fp32). None lets torch pick float16 on an accelerator and float32 on CPU, which is the only pair that works on both without producing black images.

  • offload (bool) – Hand the pipeline to enable_model_cpu_offload(), so only the component currently computing stays on the device. Slower per image, and what makes a large pipeline fit at all.

  • hf_token (str or None) – HuggingFace token, for a gated repository (FLUX is gated).

Returns:

The diffusers pipeline, untouched.

Return type:

Any

Raises:

ImportError – When diffusers cannot be imported, with the command that fixes it.

Examples

>>> load("stabilityai/sdxl-turbo")
best_engine_ai_helper.imagine.megapixels(width, height)[source]

Output area in megapixels, the unit the latent term is priced in.

Parameters:

  • width (int) – Output dimensions in pixels.

  • height (int) – Output dimensions in pixels.

Returns:

Megapixels, rounded to three decimals.

Return type:

float

Examples

>>> megapixels(1024, 1024)
1.049
>>> megapixels(512, 512)
0.262
best_engine_ai_helper.imagine.native_size(entry)[source]

The resolution a model was trained for.

The one size a diffusion model is actually good at. Asking SD 1.5 for 1024x1024 does not add detail, it duplicates limbs; asking SDXL for 512x512 loses composition. So size: "auto" on the image endpoint resolves to this rather than to a house default.

Parameters:

entry (dict[str, Any]) – Catalog entry; reads native_size ("WxH"), else falls back to 1024x1024.

Returns:

(width, height) in pixels.

Return type:

tuple[int, int]

Examples

>>> native_size({"id": "sd15", "native_size": "512x512"})
(512, 512)
>>> native_size({"id": "sdxl"})
(1024, 1024)
>>> native_size({"id": "broken", "native_size": "nonsense"})
(1024, 1024)
best_engine_ai_helper.imagine.pick(plans)[source]

Choose one image model: the best that runs, else the fastest that fits.

Parameters:

plans (list of dict) – plan() dicts, typically from rank().

Returns:

The chosen plan, or None when nothing fits at all.

Return type:

dict or None

Examples

>>> rows = [
...     {"id": "sd15", "kind": "image", "size_b": 1.07, "steps": 28,
...      "native_size": "512x512", "hf_id": "x/sd15", "benchmarks": {"image": 60}},
...     {"id": "sdxl", "kind": "image", "size_b": 3.5, "steps": 28,
...      "native_size": "1024x1024", "hf_id": "x/sdxl", "benchmarks": {"image": 75}},
... ]
>>> pick(rank(rows, budget_gb=11.0, bandwidth_gbs=1000.0))["catalog_id"]
'sdxl'
>>> pick(rank(rows, budget_gb=4.0, bandwidth_gbs=1000.0))["catalog_id"]
'sd15'
>>> pick([]) is None
True
best_engine_ai_helper.imagine.plan(entry, *, budget_gb=None, bandwidth_gbs=None, dtype=None, width=None, height=None, steps=None)[source]

Everything known about running ONE image model on this machine.

Tries the all-resident footprint first and falls back to the component-offloaded one when that does not fit — the same escalation offload.rank() makes with prefetching, for the same reason: the knob exists, the runtime exposes it, and trying it before rejecting a model is the difference between running FLUX on a 24 GB card and reporting that it cannot be done.

Parameters:

  • entry (dict[str, Any]) – Catalog entry.

  • budget_gb (float or None) – Memory budget from score.effective_budget(). None skips the fit verdict.

  • bandwidth_gbs (float or None) – Memory bandwidth, for the time estimate.

  • dtype (str or None) – Compute dtype override.

  • width (int or None) – Output size; None uses the model’s native one.

  • height (int or None) – Output size; None uses the model’s native one.

  • steps (int or None) – Step override.

Returns:

model, catalog_id, resident_gb, offload (whether the fit needed it), size, steps, seconds_per_image, fits, usable and blockers.

Return type:

dict[str, Any]

Examples

>>> sdxl = {"id": "sdxl", "kind": "image", "size_b": 3.5, "denoiser_size_b": 2.6,
...         "steps": 28, "native_size": "1024x1024",
...         "hf_id": "stabilityai/stable-diffusion-xl-base-1.0"}
>>> p = plan(sdxl, budget_gb=11.0, bandwidth_gbs=1000.0)
>>> p["resident_gb"], p["offload"], p["seconds_per_image"], p["usable"]
(9.8, False, 10.3, True)
>>> # An 8 GB budget only fits it with the components taking turns.
>>> tight = plan(sdxl, budget_gb=8.0, bandwidth_gbs=1000.0)
>>> tight["resident_gb"], tight["offload"], tight["fits"]
(7.5, True, True)
>>> plan(sdxl, budget_gb=2.0)["blockers"][0][:38]
'needs 7.5 GB resident even with the co'
best_engine_ai_helper.imagine.rank(entries, *, budget_gb, bandwidth_gbs=None, dtype=None, width=None, height=None)[source]

Rank the image models this machine could run, best first.

Parameters:

  • entries (list of dict) – Catalog entries; non-image kinds are dropped.

  • budget_gb (float) – Memory budget from score.effective_budget().

  • bandwidth_gbs (float or None) – Memory bandwidth, for the time estimates.

  • dtype (str or None) – Compute dtype override.

  • width (int or None) – Output size; None lets each model use its own native one, which is the fair comparison — a model is not worse for being a 512px model, it is cheaper.

  • height (int or None) – Output size; None lets each model use its own native one, which is the fair comparison — a model is not worse for being a 512px model, it is cheaper.

Returns:

plan() dicts, usable first, then by image-quality score, then by parameter count.

Return type:

list[dict[str, Any]]

Examples

>>> rows = [
...     {"id": "sd15", "kind": "image", "size_b": 1.07, "steps": 28,
...      "native_size": "512x512", "hf_id": "x/sd15",
...      "benchmarks": {"image": 60}},
...     {"id": "sdxl", "kind": "image", "size_b": 3.5, "steps": 28,
...      "native_size": "1024x1024", "hf_id": "x/sdxl",
...      "benchmarks": {"image": 75}},
...     {"id": "chat", "kind": "llm", "size_b": 8},
... ]
>>> [r["catalog_id"] for r in rank(rows, budget_gb=11.0, bandwidth_gbs=1000.0)]
['sdxl', 'sd15']
>>> # On a 4 GB card only the small one runs, and it says why about the other.
>>> tight = rank(rows, budget_gb=4.0, bandwidth_gbs=1000.0)
>>> [(r["catalog_id"], r["usable"]) for r in tight]
[('sd15', True), ('sdxl', False)]
best_engine_ai_helper.imagine.seconds_per_image(entry, bandwidth_gbs, *, steps=None, width=None, height=None)[source]

Estimate how long one image takes on this machine.

steps x denoiser passes, each pass proportional to the denoiser’s size and to the output area. The constant behind it (_STEP_SECONDS_PER_B_PER_MP) is anchored on one measured point and scaled by memory bandwidth, which is the only compute proxy this package has — so treat the result as “seconds or minutes?”, never as a benchmark. It is still the figure that decides usability: a model needing four minutes an image is a different tool from one needing four seconds.

Parameters:

  • entry (dict[str, Any]) – Catalog entry.

  • bandwidth_gbs (float or None) – Memory bandwidth from detect.compute_profile(). None returns None: no estimate rather than a fabricated one.

  • steps (int or None) – Override the catalog’s step count.

  • width (int or None) – Output size; None uses the native one.

  • height (int or None) – Output size; None uses the native one.

Returns:

Seconds per image, rounded to one decimal, or None when bandwidth or model size is unknown.

Return type:

float or None

Examples

>>> sdxl = {"id": "sdxl", "size_b": 3.5, "denoiser_size_b": 2.6,
...         "steps": 28, "native_size": "1024x1024"}
>>> seconds_per_image(sdxl, 1000.0)
10.3
>>> # The same model distilled to 4 steps is the same arithmetic, 7x less of it.
>>> seconds_per_image({**sdxl, "steps": 4}, 1000.0)
1.5
>>> # Half the bandwidth, twice the wait.
>>> seconds_per_image(sdxl, 500.0)
20.6
>>> seconds_per_image(sdxl, None) is None
True
>>> seconds_per_image({"id": "unsized"}, 1000.0) is None
True
best_engine_ai_helper.imagine.steps_for(entry)[source]

Denoising steps the model expects.

A property of the checkpoint, not a taste: a distilled model (Turbo, schnell, Lightning) is trained to converge in 1-4 steps, and running it for 28 is both slower and worse. Getting this from the catalog rather than from a global default is what makes an auto-selected image model usable.

Parameters:

entry (dict[str, Any]) – Catalog entry; reads steps.

Returns:

Step count. A declared 0 is treated as absent rather than honoured: zero denoising steps returns noise, so it can only be a data error.

Return type:

int

Examples

>>> steps_for({"id": "sdxl-turbo", "steps": 4})
4
>>> steps_for({"id": "sd15"})
28
>>> steps_for({"id": "weird", "steps": 0})
28