best_engine_ai_helper.browser module

browser — choose a language model for a machine best-engine never sees: the visitor’s browser.

Everywhere else best-engine probes the machine, then picks ONE model. In a web page it cannot: the machine is the visitor’s, and it reveals itself only at the first question, through what the browser chooses to expose. So the browser target works in two halves.

  1. In advance, here — plan() reads a usage profile (rag-answer…) and the browser catalog (data/browser_models.yaml), and writes an ordered ladder of candidates, every one with its requirements: download size per dtype, shader-f16, a device-memory floor, the exact files to fetch.

  2. At run time, in the page — the page walks the ladder down and takes the first candidate the machine and the line can bear. choose() is that walk, in Python: the reference the JavaScript twin is tested against, and a way to simulate a device (best-engine-ai-helper browser choose).

The criteria are new to best-engine, because a page pays for things a server never does:

  • the download, judged against the bandwidth OBSERVED in the page (a 3 GB model is fine on fibre and absurd on a train), and free once cached;

  • shader-f16, which decides between q4f16 and q4 weights;

  • deviceMemory, a rounded figure, as a proxy for a GPU nobody can query;

  • mobile and data saver, which rule a multi-gigabyte download out;

  • the storage quota left for the browser’s cache.

A signal the browser does not expose (Safari has no deviceMemory) is never held against the machine; an unknown bandwidth is replaced by a deliberately modest assumption, so that silence never buys a 5 GB download.

Author

Warith Harchaoui <warithmetics@deraison.ai>

best_engine_ai_helper.browser.DEFAULT_MIN_SCORE = 0.5

A model measured below this on the usage stays in the catalog, with its score, but leaves the ladder: downloading it would mostly buy answers the page then sets aside. Unmeasured models are kept, on their prior.

best_engine_ai_helper.browser.DEFAULT_POLICY = {'max_download_s': 180, 'probe_bytes': 4000000, 'slow_networks': ['slow-2g', '2g', '3g'], 'storage_margin': 1.1, 'unknown_bandwidth_Bps': 2500000}

Policy written into every plan, read by the page.

best_engine_ai_helper.browser.TRANSFORMERS_JS = '@huggingface/transformers@4.3.2'

the catalog’s ONNX exports and chat templates are checked against it.

Type:

Runtime the plan is written for. The page must import this exact build

best_engine_ai_helper.browser.choose(the_plan, device)[source]

Walk the ladder for one device; the reference of the page’s JavaScript.

device holds what the page measured; every key is optional: webgpu, f16, mobile, save_data, effective_type, device_memory_gb, storage_free_bytes, bandwidth_Bps and cached (the candidates, as "model|dtype", already in the cache).

Returns {"candidate", "reason", "seconds"}; candidate is None when nothing fits, and reason says why: webgpu, mobile, network, device (no rung fits the GPU, memory or storage), download (every fitting rung is too long to fetch on this line). A cached rung costs no download, whatever its size.

Examples

>>> p = plan("rag-answer")
>>> fast = {"webgpu": True, "f16": True, "bandwidth_Bps": 50e6}
>>> c = choose(p, fast)
>>> c["candidate"]["model"].split("/")[1], c["candidate"]["dtype"], c["reason"]
('gemma-4-E4B-it-ONNX', 'q4f16', 'ok')
>>> mid = {**fast, "bandwidth_Bps": 20e6}            # 160 Mbit/s: E4B takes 4 min
>>> choose(p, mid)["candidate"]["model"].split("/")[1]
'gemma-4-E2B-it-ONNX'
>>> choose(p, {**fast, "bandwidth_Bps": 5e6})["reason"]   # 40 Mbit/s: nothing in 3 min
'download'
>>> choose(p, {"webgpu": True, "mobile": True})["reason"]
'mobile'
Parameters:
Return type:

dict[str, Any]

best_engine_ai_helper.browser.load_browser_catalog(seed=None, cache=None)[source]

The browser catalog: the bundled seed, overlaid by the user cache by id.

Examples

>>> cat = load_browser_catalog(cache=Path("/nonexistent"))
>>> cat[0]["id"]
'onnx-community/gemma-4-E4B-it-ONNX'
>>> sorted(cat[1]["variants"])
['q4', 'q4f16']
Parameters:
  • seed (Path | None)

  • cache (Path | None)

Return type:

list[dict[str, Any]]

best_engine_ai_helper.browser.plan(usage='rag-answer', *, languages=None, catalog=None, policy=None, min_score=0.5)[source]

The ordered ladder of browser candidates for usage, ready for a page.

One rung per (model, dtype), best first: models by their rank for the usage (a measured score when the catalog carries one, the curator’s prior otherwise), and within a model the shader-f16 variant first, since it is both smaller and faster where it runs. A model that does not serve every language in languages is left out, and so is a model MEASURED on the usage below min_score; the plan lists both under excluded.

Examples

>>> p = plan("rag-answer", languages=["fr", "en"])
>>> p["usage"], p["runtime"]
('rag-answer', 'transformers.js')
>>> [c["model"].split("/")[1] + " " + c["dtype"] for c in p["candidates"]]
['gemma-4-E4B-it-ONNX q4f16', 'gemma-4-E4B-it-ONNX q4', 'gemma-4-E2B-it-ONNX q4f16', 'gemma-4-E2B-it-ONNX q4']
Parameters:
Return type:

dict[str, Any]

best_engine_ai_helper.browser.refresh(catalog=None, *, fetch=<function _hub_tree>, cache=None)[source]

Re-read each variant’s files and download size from the Hugging Face Hub.

A variant’s files are the ONNX graphs and weight shards whose names end in its dtype (model_q4f16.onnx, embed_tokens_q4f16.onnx_data…); vision and audio encoders are left out, the browser target being text-only. The result is written to the user cache unless cache is False-y.

Examples

>>> tree = [{"path": "onnx/model_q4.onnx", "size": 10},
...         {"path": "onnx/model_q4.onnx_data", "lfs": {"size": 90}},
...         {"path": "onnx/vision_encoder_q4.onnx", "size": 5}]
>>> cat = [{"id": "x/y", "variants": {"q4": {"needs_f16": False}}}]
>>> out = refresh(cat, fetch=lambda repo: tree, cache=None)
>>> out[0]["variants"]["q4"]["bytes"], out[0]["variants"]["q4"]["files"]
(100, ['onnx/model_q4.onnx', 'onnx/model_q4.onnx_data'])
Parameters:
Return type:

list[dict[str, Any]]