best_engine_ai_helper.storage module
storage — what fits on the disk, and what fits in memory at the same time.
Two questions, and the second is the one people actually mean.
“How many models fit on my SSD” is easy and worth answering: disk is finite, model files are large, and a 20 GB pull that fails halfway is a bad way to find out. That part is a subtraction.
The question behind it is usually different. A working setup is rarely one model: it is a text model, a vision model and an embedder, and the thing that decides whether the day goes well is whether those can sit in memory together rather than taking turns. Each one on its own may fit comfortably while the trio does not, and nothing in a per-model view would ever say so. This module answers both, and keeps them clearly apart.
Disk and memory are deliberately not the same budget here. disk_gb is the
file on disk; ram_gb is the peak working set while serving, which includes
the KV cache and runtime buffers. A model costs both, and confusing the two is
how a plan that looked fine runs out of memory.
- best_engine_ai_helper.storage.RESIDENT_KINDS = ('llm', 'vlm', 'embed')
something to write with, something to look at images with, something to embed with.
- Type:
The kinds a working setup usually wants resident at once
- best_engine_ai_helper.storage.report(entries, budget_gb, *, installed=None, backend='ollama', disk=None)[source]
Answer both capacity questions for this machine.
- Parameters:
- Returns:
{"disk", "memory_budget_gb", "already_here", "pullable_now", "too_big_for_disk", "resident_sets", "warnings"}.- Return type:
- best_engine_ai_helper.storage.resident_sets(entries, budget_gb, *, kinds=('llm', 'vlm', 'embed'), backend='ollama', limit=5)[source]
Best combinations, one model per kind, that fit in memory simultaneously.
Takes the strongest candidate of each kind first and works down, so the answer is “the best set you can hold”, not merely “a set that fits”.
Ranked on quality first, memory second. Earlier versions ordered on total memory alone, on the argument that among sets that fit, the one using the most is the most capable. That is a proxy, and a weak one: a 20 GB model scoring 60 is not a better companion than an 18 GB model scoring 85. Benchmark score is the thing actually meant, so it is the thing sorted on; total memory stays as the tiebreak, which keeps the old order exactly when nothing is scored — the normal state for the hundreds of library entries that publish sizes but no leaderboard results.
Searched by branch and bound over kinds sorted best-first, not by enumerating every combination. Two bounds do the work: a set whose remaining kinds cannot fit even at their lightest is skipped, and once
limitresults are in hand, a branch that cannot beat the weakest of them even at its best ends the scan of that kind. Both are exact – the result is identical to an exhaustive product, which tests/test_storage.py pins differentially against this function.The score is a sum across kinds of figures from different leaderboards, so it ranks sets against each other and is not a measurement of anything. It is reported as
total_scoreso the ordering is visible rather than being an invisible key the reader has to take on trust.- Parameters:
entries (list of dict) – Catalog entries, already ranked best-first within each kind if the caller has an opinion about ordering.
budget_gb (float) – Memory available for models together, from
score.effective_budget().kinds (tuple of str) – Which kinds must be present. A kind with no candidate is skipped rather than making the whole answer empty.
backend (str) – Serving backend, so footprints are sized the way they will load.
limit (int) – How many combinations to return.
- Returns:
[{"models": {kind: id}, "total_score": float, "total_ram_gb": float, "headroom_gb": float}], best total score first among those that fit, then largest total memory, then model id so repeated runs agree.- Return type: