best_engine_ai_helper.api module

api — FastAPI HTTP surface for best-engine-ai-helper.

Exposes the two calls the GUI needs:

  • GET /api/system — detected hardware + compute profile + memory budget.

  • GET /api/lightweight — whether layer streaming is available/warranted here, and which too-large models it would unlock (see best_engine_ai_helper.offload).

  • POST /api/recommend — a free-text task -> the same report recommend() produces for the CLI’s report command, as JSON, plus the best PAID model under "cloud" (reference only, on by default; see cli.py’s report --cloud).

It also mounts the OpenAI-compatible surface from best_engine_ai_helper.openai_api on the same app: /v1/models, /v1/chat/completions (text and vision, streaming or not), /v1/completions, /v1/embeddings and /v1/images/generations. Point any OpenAI SDK at http://127.0.0.1:8000/v1 and ask for the model named best — from the outside this package then behaves like an OpenAI endpoint, while inside it is choosing the model this machine should actually run.

A minimal single-page GUI is served at GET /gui (GET / redirects there): it shows the machine’s characteristics and lets you type a task description to get the best local engine(s) for it, alongside the best paid model for comparison. It is bilingual — French by default, English at GET /gui?lang=en — with a header link to switch between the two.

Run the app with any ASGI server:

uvicorn best_engine_ai_helper.api:app --port 8000
# or: best-engine-ai-helper gui

Author

Warith Harchaoui <warithmetics@deraison.ai>

class best_engine_ai_helper.api.PreferencesRequest(*, quality_vs_cost=None, headroom=None, min_tps=None, local_only=None, live_load=None)[source]

Bases: BaseModel

Preferences to persist. Every field optional: a form saves what changed.

Parameters:

  • quality_vs_cost (float | None)

  • headroom (float | None)

  • min_tps (float | None)

  • local_only (bool | None)

  • live_load (bool | None)

headroom: float | None
live_load: bool | None
local_only: bool | None
min_tps: float | None
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

quality_vs_cost: float | None
class best_engine_ai_helper.api.RecommendRequest(*, task=None, headroom=0.5, live=False, cloud=True, quality_vs_cost=0.7, cloud_provider=None)[source]

Bases: BaseModel

Body for POST /api/recommend.

Parameters:
cloud: bool
cloud_provider: str | None
headroom: float
live: bool
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

quality_vs_cost: float
task: str | None
best_engine_ai_helper.api.activity()

Local activity/cost ledger summary (calls, cost, by user/model, errors).

Reads the ledger even when THIS process never called observe.enable() — e.g. a plain uvicorn best_engine_ai_helper.api:app that skips the CLI’s/MCP’s auto-enable — by opening the default-path database read-only in that case, same fallback cli.activity_cmd() uses. Never 404s or errors over “no data”: an unrecorded/empty ledger is a valid, common state.

Return type:

dict[str, Any]

best_engine_ai_helper.api.gui(lang='fr')

Serve the single-page GUI (?lang=en for English, French by default).

Parameters:

lang (str)

Return type:

str

best_engine_ai_helper.api.lightweight(measure=False, kind='llm', answer_tokens=300, batch=1)

Layer streaming: can this machine do it, should it, and what would it run?

The read-only half of the lightweight command group (status plus plan) in one call, so the GUI and any other consumer can show the capability without shelling out.

Parameters:

  • measure (bool) – Measure disk read bandwidth instead of reading the storage-class table. Off by default: it writes and reads back ~256 MB, which is not something a page load should do unasked.

  • kind (str) – llm or vlm.

  • answer_tokens (int) – Answer length every timing is computed for. The per-token rate alone does not tell a caller whether a job finishes before lunch.

  • batch (int) – Sequences answered together. Nearly free in time under streaming (one pass is shared) and one KV cache each in memory.

Returns:

decision (offload.decide()), disk, memory_budget_gb, chosen (offload.pick()), the ranked plans, and comparison (offload.compare()) — the same model streamed against the best this machine runs natively, both priced in seconds for one answer, which is the decision a caller is actually making.

Return type:

dict[str, Any]

best_engine_ai_helper.api.preferences_get()

The saved preferences, with defaults filled in for anything unset.

Return type:

dict[str, Any]

best_engine_ai_helper.api.preferences_set(body)

Save preferences and return the full set as now stored.

Writes ~/.best-engine-ai-helper/preferences.yaml, deliberately not settings.yaml: that file holds API keys, and a browser form should never be in a position to rewrite a file containing secrets. The YAML remains the source of truth — this endpoint writes it and reads it back like any other client, rather than holding state of its own.

Parameters:

body (PreferencesRequest)

Return type:

dict[str, Any]

best_engine_ai_helper.api.providers()

Which cloud providers are usable right now, and for the rest, why not.

The package routes on the keys you actually hold, and this is that rule made visible: a provider with no key is not a candidate, and the row says which variable to set rather than leaving a silently shorter list.

Reports only whether a key resolved and the NAME of the variable it would come from. No key value is ever returned.

Return type:

dict[str, Any]

best_engine_ai_helper.api.recommend(body)

Best local engine(s) for body.task on this machine’s hardware.

Also carries the best PAID model under "cloud" unless body.cloud is false — a reference ranking only (no API key, no network call), the same one the CLI’s report --cloud shows; see best_engine_ai_helper.recommend.cloud_recommend().

Parameters:

body (RecommendRequest)

Return type:

dict[str, Any]

best_engine_ai_helper.api.root()

Redirect the bare root path to the browser GUI.

Return type:

fastapi.responses.RedirectResponse

best_engine_ai_helper.api.system()

Detected hardware, compute profile, and usable memory budget.

Return type:

dict[str, Any]