feat(local-model): judge the fit on the measured peak and rank a tested Mac first

The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-06 22:16:37 +01:00
1 parent 84c1017487
commit 492df776d0
7 files changed
+1241 -502

No files matched your search

+6 -6
View File
@@ -7,12 +7,12 @@
* `budgetGb`, and the size slots of a `LocalModelMessage`). Nothing else may turn bytes into a
* size for display, the browser included: it prints the text it was sent.
*
* GiB and MiB exist only for the budget math (reserves, the wired limit, the prompt cache) and
* never reach a person, with one exception: `memClassGb()`, the memory size a Mac is sold as,
* is how the card names the installed memory ("48 GB memory", what About This Mac shows). It
* is a name of the machine, not a computed size: `formatGb()` of the same bytes reads 51.5,
* which no owner recognises. Every computed size (need, budget, download, free disk) is
* `formatGb()`.
* GiB and MiB exist only for the budget math (reserves, the wired limit, the device total the
* runtime reports) and never reach a person, with one exception: `memClassGb()`, the memory
* size a Mac is sold as, is how the card names the installed memory ("48 GB memory", what
* About This Mac shows). It is a name of the machine, not a computed size: `formatGb()` of the
* same bytes reads 51.5, which no owner recognises. Every computed size (need, budget,
* download, free disk) is `formatGb()`.
*
* Pure: no IO, no state.
*/