mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-16 19:29:33 +01:00
feat(local-model): judge the fit on the measured peak and rank a tested Mac first
The catalogue rows carry the figures measured on a Mac (the MLX build of qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token, about 3 GB of runner overhead, about 5 GB more for a moment while a long prompt is processed). A model's need is now its peak. Rows that were not measured use the same per-token and overhead figures and say `estimated`. qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and OpenCode, with its measured 67.7 tokens per second, so that row may say "Tested on a Mac like this". The ranking changes with it: a pair tested on a Mac like this one comes first, then speed (ok, unknown, slow), then model rank, harness rank and the larger context. The fit grade decides whether a row is listed and what it says, it no longer sorts, so a model is offered at the largest context it fits at. recommend() takes the budget the runtime measured (`budgetSource: measured`), and modelFitsBudget() judges one model against it. Candidate rows now always carry `needSource`, `harnesses`, `installBytes`, `installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt cache tiers are gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
84c1017487
commit
492df776d0
7 files changed
+1241
-502
No files matched your search
@@ -7,12 +7,12 @@
|
||||
* `budgetGb`, and the size slots of a `LocalModelMessage`). Nothing else may turn bytes into a
|
||||
* size for display, the browser included: it prints the text it was sent.
|
||||
*
|
||||
* GiB and MiB exist only for the budget math (reserves, the wired limit, the prompt cache) and
|
||||
* never reach a person, with one exception: `memClassGb()`, the memory size a Mac is sold as,
|
||||
* is how the card names the installed memory ("48 GB memory", what About This Mac shows). It
|
||||
* is a name of the machine, not a computed size: `formatGb()` of the same bytes reads 51.5,
|
||||
* which no owner recognises. Every computed size (need, budget, download, free disk) is
|
||||
* `formatGb()`.
|
||||
* GiB and MiB exist only for the budget math (reserves, the wired limit, the device total the
|
||||
* runtime reports) and never reach a person, with one exception: `memClassGb()`, the memory
|
||||
* size a Mac is sold as, is how the card names the installed memory ("48 GB memory", what
|
||||
* About This Mac shows). It is a name of the machine, not a computed size: `formatGb()` of the
|
||||
* same bytes reads 51.5, which no owner recognises. Every computed size (need, budget,
|
||||
* download, free disk) is `formatGb()`.
|
||||
*
|
||||
* Pure: no IO, no state.
|
||||
*/
|
||||
|
||||
Reference in new issue
Block a user