BitSquare/Codeman
2
0 Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-11-14 02:29:33 +01:00
Code Issues Packages Projects Releases Wiki Activity

feat(local-model): judge the fit on the measured peak and rank a tested Mac first

Browse Source 
The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-06 22:16:37 +01:00
1 parent 84c1017487
commit 492df776d0
7 files changed
+1241 -502
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/config/local-model-catalog.ts
+67 -49
View File
@@ -3,8 +3,9 @@
* knows, as a compiled constant validated by a Zod schema.
*
* Rules:
* - There is NO override file and no runtime download data in this stage: nothing here reads
* the data dir, and a data file can never decide what gets recommended.
* - There is NO override file and no runtime download data here: nothing in this file reads
* the data dir, and a data file can never decide what gets recommended. Which runtime build
* is downloaded is a compiled constant of its own (src/local-model/runtime.ts).
* - A model id is `name:tag` in lowercase with no slash (`LOCAL_MODEL_ID_PATTERN`), so it can
* only ever name a model of the default registry and namespace.
* - A harness is a ROW (`{ cli, minContext, rank, gated?, slowFirstTurn? }`). The `cli` is a CLI
@@ -13,7 +14,7 @@
* src/local-model/harness-facts.ts, never here.
* - The schema checks more than fields. Across rows it requires: unique model ids and ranks,
* unique harness `cli`s and ranks, `contexts` largest first, a `tested` entry naming a harness
* row, `chips` most specific first, and `promptCacheTiers` ascending from 0.
* row, and `chips` most specific first.
* - `getLocalModelCatalog()` parses the constant once, on first use and never at import, and
* throws on a bad row. test/local-model-catalog.test.ts pins that the stock constant parses,
* so a bad row fails the gate and never reaches the route.
@@ -22,26 +23,44 @@
*
* | Figure | Source | Checked |
* |----------------------|-----------------------------------------------------------|--------------------------------------|
* | `id`, `bytes` | registry manifest of the tag, sum of its layer sizes | yes, to the byte, by the facts review |
* | `id` | the registry's tag | yes, by the facts review |
* | `bytes` | `qwen3.6:35b-a3b`: the total of a real pull on a Mac | that row: measured. It is an MLX |
* | | (23,105,798,956). Every other row: the sum of the layer | build on a Mac, 2% above its |
* | | sizes of the tag's registry manifest | manifest. The others: not on a Mac |
* | `heldBytesPerToken`, | measured for `qwen3.6:35b-a3b` on an Apple M5 Pro with | that row: measured. The others carry |
* | `overheadBytes`, | 48 GB: 36,000 to 38,000 bytes held per token of context, | the SAME three figures, not their |
* | `peakExtraBytes` | 2.5 to 3 GB beside the weights, and about 5 GB more for a | own: nobody ran them. `figures` says |
* | | moment while a long prompt is processed | which row is which |
* | `license` | the model card (all Apache-2.0) | read by the research, not re-checked |
* | `paramsB` | 27.8 from the model card; the others are the count in the | no. Only the 12 billion line reads |
* | | model's name | it, and no row is near that line |
* | `kvBytesPerToken` | COMPUTED from each model's config: 2 x full-attention | no. Not measured in a running |
* | | layers x KV heads x head size x 2 bytes | runtime. The review called the |
* | | | gpt-oss figure plausible, no more |
* | `contexts`, `rank` | a judgement of the design | no |
* | `speedPer100Gbps` | one published run on an M1 Max (400 GB/s): generation | the run was read. It stands for less |
* | | only, short prompt. The two q8_0 rows are SCALED by file | than a session: no long prompt, no |
* | | size, not measured | thinking tokens |
* | `tested` | our own run of the model with a harness on a real Mac | one entry, with its measured speed |
* | `minMacosMajor` | the runtime's documentation (macOS 14) | yes, by the facts review |
* | `promptCacheTiers` | a decision of the design, not a measurement | no |
* | harness `minContext` | 65536: the runtime's documentation for both harnesses. | 65536 yes. 32768 no, which is why |
* | | 32768: recalled | that row is gated |
* | `chips` | the bandwidth column of | yes, read at the source for this |
* | | https://github.com/ggml-org/llama.cpp/discussions/4167 | file |
*
* `tested` is empty on every row: nobody has run any of these models with a harness on a Mac
* yet. Until a row has an entry, the recommendation never says "Recommended".
* The memory figures decide a lot and rest on ONE measured model, so for every other row they
* can be wrong in either direction:
* - For the 9B rows they may be too HIGH (the 5 GB for a moment was seen with a prompt of
* 89,000 tokens on the 35B model). That errs on the careful side: a small Mac is told
* "none" before it is told to download something that may not fit.
* - For the two dense `qwen3.8:27b` rows the per-token figure may be too LOW: computed from
* its config alone that model holds 64 KiB per token, more than the 37,000 bytes measured
* on the 35B (whose own computed figure was 20 KiB). Their need at a large context is
* therefore the least certain figure of the catalogue.
* A row gets its own figures when somebody measures it. Until then the check after the
* download (the runtime must report the whole model on the GPU) is what catches a row that
* was judged too kindly.
*
* `tested` has one entry: `qwen3.6:35b-a3b` with OpenCode on an Apple M5 Pro with 48 GB,
* 2026-11-06, 67.7 tokens per second. Only a Mac of exactly that chip and memory size is told
* "Recommended" and shown that speed in place of the estimate.
*/
import { z } from 'zod';
@@ -85,6 +104,7 @@ const TestedOnSchema = z
memGb: positiveInt,
cli: z.string().min(1).max(24),
date: isoDate,
tokensPerSecond: z.number().positive().optional(),
})
.strict();
@@ -95,7 +115,10 @@ const LocalModelEntrySchema = z
license: z.string().min(1).max(40),
paramsB: z.number().positive(),
bytes: positiveInt,
kvBytesPerToken: positiveInt,
heldBytesPerToken: positiveInt,
overheadBytes: z.number().int().nonnegative(),
peakExtraBytes: z.number().int().nonnegative(),
figures: z.enum(['measured', 'estimated']),
contexts: z.array(positiveInt).min(1),
rank: z.number().int(),
speedPer100Gbps: z.number().positive().optional(),
@@ -120,13 +143,6 @@ const ChipEntrySchema = z
})
.strict();
const PromptCacheTierSchema = z
.object({
minMemGb: z.number().int().nonnegative(),
bytes: z.number().int().nonnegative(),
})
.strict();
/** The index of the first value that already occurred earlier in the list, or -1. */
function firstRepeat<T>(values: readonly T[]): number {
const seen = new Set<T>();
@@ -146,7 +162,6 @@ export const LocalModelCatalogSchema: z.ZodType<LocalModelCatalog> = z
schema: z.literal(1),
updated: isoDate,
minMacosMajor: positiveInt,
promptCacheTiers: z.array(PromptCacheTierSchema).min(1),
models: z.array(LocalModelEntrySchema),
harnesses: z.array(LocalHarnessEntrySchema),
chips: z.array(ChipEntrySchema),
@@ -199,23 +214,21 @@ export const LocalModelCatalogSchema: z.ZodType<LocalModelCatalog> = z
break;
}
}
// The tiers are read as "the last one whose minimum is not above this Mac's memory", so they
// must start at 0 (every Mac has a tier) and ascend.
if (catalog.promptCacheTiers[0].minMemGb !== 0) {
issue(['promptCacheTiers', 0, 'minMemGb'], 'the first prompt cache tier must start at 0');
}
for (let i = 1; i < catalog.promptCacheTiers.length; i++) {
if (catalog.promptCacheTiers[i].minMemGb <= catalog.promptCacheTiers[i - 1].minMemGb) {
issue(['promptCacheTiers', i, 'minMemGb'], 'prompt cache tiers must ascend by minMemGb');
break;
}
}
});
const KIB = 1024;
const MIB = 1024 ** 2;
const GIB = 1024 ** 3;
/**
* The memory figures measured for `qwen3.6:35b-a3b` on an Apple M5 Pro with 48 GB (2026-11-06,
* the MLX runner), in decimal bytes. That row carries them as `measured`; every other row
* carries the same three as `estimated`, see the file header.
*/
const MEASURED_MEMORY = {
/** 36,000 to 38,000 bytes stayed held per token of context after a prompt of 89,000 and one of 167,000 tokens. */
heldBytesPerToken: 37_000,
/** The runner held 2.5 to 3 GB beside the weights. */
overheadBytes: 3_000_000_000,
/** While a long prompt was processed the peak stood about 5 GB above what stayed held. */
peakExtraBytes: 5_000_000_000,
} as const;
/**
* The catalogue that ships with Codeman. Never read directly outside this file and its test:
@@ -226,12 +239,6 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
updated: '2026-11-06',
// The oldest macOS the model runtime supports.
minMacosMajor: 14,
// Host memory the runtime's prompt cache may take beside the model: 512 MiB on a Mac below
// 32 GB, 2 GiB from 32 GB. Part of the need of every candidate.
promptCacheTiers: [
{ minMemGb: 0, bytes: 512 * MIB },
{ minMemGb: 32, bytes: 2 * GIB },
],
// Ordered by rank. The higher rank wins among models that fit equally well and are equally
// fast. `flint:40b` is left out on purpose: its plain tag is the slow build on a Mac and it
// has an open bug with tool arguments.
@@ -242,7 +249,8 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
license: 'Apache-2.0',
paramsB: 9,
bytes: 6_550_825_373,
kvBytesPerToken: 32 * KIB,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [65536, 32768],
rank: 1,
// 37.1 tokens per second measured on 400 GB/s.
@@ -255,7 +263,8 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
license: 'Apache-2.0',
paramsB: 9,
bytes: 10_449_217_949,
kvBytesPerToken: 32 * KIB,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [65536, 32768],
rank: 2,
// Not measured: the 9.3 of the row above scaled by file size.
@@ -268,7 +277,8 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
license: 'Apache-2.0',
paramsB: 20,
bytes: 13_793_441_427,
kvBytesPerToken: 24 * KIB,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [131072, 65536],
rank: 3,
// No speed figure: its speed is unknown on every chip.
@@ -279,13 +289,18 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
label: 'Qwen3.6 35B-A3B',
license: 'Apache-2.0',
paramsB: 35,
bytes: 22_621_314_161,
kvBytesPerToken: 20 * KIB,
// What a Mac really downloads: an MLX build, 2% above the 22,621,314,161 of the manifest.
bytes: 23_105_798_956,
...MEASURED_MEMORY,
figures: 'measured',
contexts: [131072, 65536],
rank: 4,
// 54.5 tokens per second measured on 400 GB/s (3 billion parameters are active per token).
// The MLX build ran faster than this figure predicts on the one Mac we tested (67.7
// against 41.8), so on other chips the estimate is on the low side.
speedPer100Gbps: 13.6,
tested: [],
// Run by us with OpenCode on the MacBook Pro (Mac17,13): a four-step task with tool calls.
tested: [{ chip: 'Apple M5 Pro', memGb: 48, cli: 'opencode', date: '2026-11-06', tokensPerSecond: 67.7 }],
},
{
id: 'qwen3.8:27b',
@@ -293,7 +308,8 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
license: 'Apache-2.0',
paramsB: 27.8,
bytes: 17_741_871_939,
kvBytesPerToken: 64 * KIB,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [131072, 65536, 32768],
rank: 5,
// 11.8 tokens per second measured on 400 GB/s.
@@ -306,7 +322,8 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
license: 'Apache-2.0',
paramsB: 27.8,
bytes: 29_978_241_837,
kvBytesPerToken: 64 * KIB,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [262144, 131072],
rank: 6,
// Not measured: the 2.95 of the row above scaled by file size.
@@ -319,7 +336,8 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
license: 'Apache-2.0',
paramsB: 122,
bytes: 81_370_035_877,
kvBytesPerToken: 24 * KIB,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [131072],
rank: 7,
// No speed figure: its speed is unknown on every chip.
src/local-model/format.ts
+6 -6
View File
@@ -7,12 +7,12 @@
* `budgetGb`, and the size slots of a `LocalModelMessage`). Nothing else may turn bytes into a
* size for display, the browser included: it prints the text it was sent.
*
* GiB and MiB exist only for the budget math (reserves, the wired limit, the prompt cache) and
* never reach a person, with one exception: `memClassGb()`, the memory size a Mac is sold as,
* is how the card names the installed memory ("48 GB memory", what About This Mac shows). It
* is a name of the machine, not a computed size: `formatGb()` of the same bytes reads 51.5,
* which no owner recognises. Every computed size (need, budget, download, free disk) is
* `formatGb()`.
* GiB and MiB exist only for the budget math (reserves, the wired limit, the device total the
* runtime reports) and never reach a person, with one exception: `memClassGb()`, the memory
* size a Mac is sold as, is how the card names the installed memory ("48 GB memory", what
* About This Mac shows). It is a name of the machine, not a computed size: `formatGb()` of the
* same bytes reads 51.5, which no owner recognises. Every computed size (need, budget,
* download, free disk) is `formatGb()`.
*
* Pure: no IO, no state.
*/
src/local-model/recommend.ts
+154 -70
View File
@@ -3,21 +3,33 @@
* fits this Mac. Pure: no IO, no clock, no CLI registry, no data dir.
*
* The math, in the order it runs:
* - `deviceBudget()`: L is the wired limit when it is above 0, else 2/3 of memory below 36 GB
* and 3/4 from 36 GB. B = min(L - 1 GiB, memory - reserve), with a reserve of 6 GiB up to
* 18 GB, 8 GiB up to 48 GB and 12 GiB above.
* - `needBytes()`: F = weights + kvBytesPerToken * ctx + 1 GiB + the prompt cache tier of the
* catalogue for this Mac's memory.
* - Fit: F/B up to 0.85 is good, up to 0.98 is tight, above is out.
* - Speed: tokens per second = the model's `speedPer100Gbps` x the chip's bandwidth / 100.
* Below 10 is slow, decided on the unrounded product (9.96 is slow although it prints as
* 10). A chip that is in no chips row, or a model without a speed figure, is UNKNOWN: a
* - `deviceBudget()`. ESTIMATED (all there is before the runtime ran): L is the wired limit
* when it is above 0, else 2/3 of memory below 36 GB and 3/4 from 36 GB, and
* B = min(L - 1 GiB, memory - reserve), with a reserve of 6 GiB up to 18 GB, 8 GiB up to
* 48 GB and 12 GiB above. MEASURED (`RecommendOptions.measuredTotalBytes`, the device total
* the runtime printed when it started): L is that total and B = min(L, memory - reserve),
* without the 1 GiB, because the figure is already what the runtime will really use.
* - `needBytes()`: the PEAK, `bytes + heldBytesPerToken * ctx + overheadBytes +
* peakExtraBytes` of the catalogue row: the weights, what stays held per token of context,
* what the runner holds beside both, and what it takes for a moment while a long prompt is
* processed. Measured for one model on one Mac (see the catalogue header); every other row
* carries the same figures and says `estimated`.
* - Fit: need over budget up to 0.85 is good, up to 0.98 is tight, above is out. The grade
* decides whether a model is offered at all and what its row says. It does NOT sort: the
* need is the worst moment at the full context, so a tight fit is a model that runs, and a
* Mac whose best model is tight should be offered that model, not a smaller one.
* - Speed: a candidate that matches a `tested` entry with a measured speed reports that
* speed. Every other one is estimated: the model's `speedPer100Gbps` x the chip's bandwidth
* / 100. Below 10 is slow, decided on the unrounded figure (9.96 is slow although it prints
* as 10). A chip that is in no chips row, or a model without a speed figure, is UNKNOWN: a
* bandwidth is never guessed.
* - Candidates: (model, context, harness) with `ctx >= harness.minContext`, the harness not
* gated, installed or installable, and accepting the model id.
* - Ranking, lexicographic: good before tight; ok before unknown before slow; the higher model
* rank; the higher harness rank; the larger context. The pick is the first candidate, the
* alternatives are the next four MODELS, each at its own best context and harness.
* - Ranking, lexicographic: tested on a Mac like this before not tested (a measurement beats
* a formula); ok before unknown before slow; the higher model rank; the higher harness
* rank; the larger context. The pick is the first candidate, the alternatives are the next
* four MODELS, each at its own best context and harness, so a model is always offered at
* the largest context it fits at.
* - Disk: a model whose download would leave less than 10 GiB free is not offered. It is not
* dropped in silence: the best-ranked model left out that way is named in `noteDiskDropped`
* with what is missing.
@@ -32,8 +44,8 @@
* installable).
* 6. `memory`: models fit, but none at a context a usable harness needs. Its own sentence
* (`noneMemoryContext`) with the budget and the smallest need at such a context, because
* this Mac can miss by a hair (the 16 GB Mac mini: 10.3 of 10.4 GB) and "too little
* memory" would hide that the answer rests on an estimate.
* "too little memory" would hide that a model does fit at a smaller context and that the
* answer rests on estimates.
* 7. `no-harness`: models fit at such a context, but no usable harness accepts their id.
* 8. `disk`: for every remaining model, free disk minus the download is below 10 GiB (an
* unknown free disk does not block).
@@ -42,16 +54,20 @@
* - Verdict `limited` when NO model with 12 billion parameters or more fits this Mac's memory
* at a context a usable harness works at. It is a statement about the Mac ("This Mac can run
* a small model"), so it is judged on everything that fits, BEFORE the disk filter and
* whatever the pick is: a 36 GB Mac whose pick is a 9B model while a 20B, a 27B and a 35B
* are listed under it is not a small-model Mac, and a 48 GB Mac does not become one because
* its disk is full. A small pick under verdict `ok` carries `noteSmallPick` instead.
* whatever the pick is: a 36 GB Mac whose pick is a 9B model while a 20B is listed under it
* is not a small-model Mac, and a 48 GB Mac does not become one because its disk is full. A
* small pick under verdict `ok` carries `noteSmallPick` instead.
* - The word "Recommended" (`LOCAL_MODEL_LABELS.headlineTested`) only for verdict `ok` with a
* pick whose model has a `tested` entry for exactly this chip, this memory size and this
* harness. Otherwise `headlineOk` (or `headlineLimited`) plus `noteUntested`.
* - Every label in the result comes from `LOCAL_MODEL_LABELS`, every computed size text from
* `formatGb()`. The one size that is not decimal GB is the installed memory in `noneMemory`:
* `HardwareFacts.memClassGb`, the size the Mac is sold as. No number is spliced into a
* sentence here.
* - Every row carries what the chooser needs and nothing the browser must work out: its one
* tag (`tagFor()`), every harness it can run with (`Candidate.harnesses`, the flat harness
* fields first), and the whole download of an install (`installBytes`: the model plus
* `RecommendOptions.runtimeDownloadBytes`).
* - Every label in the result comes from `LOCAL_MODEL_LABELS` or `LOCAL_MODEL_TAG_LABELS`,
* every computed size text from `formatGb()`. The one size that is not decimal GB is the
* installed memory in `noneMemory`: `HardwareFacts.memClassGb`, the size the Mac is sold
* as. No number is spliced into a sentence here.
* - No CLI id is ever compared with a literal: a harness is a catalogue row plus a
* `HarnessFact`.
*/
@@ -59,7 +75,11 @@
import {
LOCAL_MODEL_LABELS,
LOCAL_MODEL_NONE_LABELS,
LOCAL_MODEL_TAG_LABELS,
type Candidate,
type CandidateHarness,
type CandidateTag,
type CandidateTagKind,
type DeviceBudget,
type FitGrade,
type HardwareFacts,
@@ -73,6 +93,7 @@ import {
type Recommendation,
type RecommendOptions,
type SpeedGrade,
type TestedOn,
} from '../types.js';
import { chipBandwidthGbps } from '../config/local-model-catalog.js';
import { formatGb, GIB, MIB } from './format.js';
@@ -81,14 +102,12 @@ import { formatGb, GIB, MIB } from './format.js';
export const FIT_GOOD_MAX_SHARE = 0.85;
/** Need over budget up to this share is a `tight` fit; above it the model is out. */
export const FIT_TIGHT_MAX_SHARE = 0.98;
/** An estimated speed below this many tokens per second is `slow`. */
/** A speed below this many tokens per second is `slow`. */
export const SLOW_BELOW_TOKENS_PER_SECOND = 10;
/** When no model that fits has at least this many billion parameters, the verdict is `limited`. A pick below it is a "small model". */
export const LIMITED_BELOW_PARAMS_B = 12;
/** What must stay free on the disk after the model download. */
export const DISK_HEADROOM_BYTES = 10 * GIB;
/** What the runtime needs beside the weights, the KV cache and the prompt cache (compute buffers). */
export const RUNTIME_OVERHEAD_BYTES = GIB;
/** How many other models are listed under the pick. */
export const MAX_ALTERNATIVES = 4;
@@ -102,30 +121,39 @@ function reserveBytes(memClassGb: number): number {
return 12 * GIB;
}
/** The memory a model may use on this Mac: the device limit L, the budget B and where L came from. */
export function deviceBudget(hw: HardwareFacts): DeviceBudget {
/** A byte count a caller supplied, when it is one: finite and above 0. */
function positiveBytes(value: number | undefined): number | undefined {
return value !== undefined && Number.isFinite(value) && value > 0 ? Math.floor(value) : undefined;
}
/**
* The memory a model may use on this Mac: the device limit L, the budget B and where L came
* from. With `options.measuredTotalBytes` the limit is what the runtime itself reported and
* the 1 GiB the estimate keeps back is not subtracted; the reserve bounds both.
*/
export function deviceBudget(hw: HardwareFacts, options: RecommendOptions = {}): DeviceBudget {
const memBytes = Number.isFinite(hw.memBytes) && hw.memBytes > 0 ? hw.memBytes : 0;
const afterReserve = memBytes - reserveBytes(hw.memClassGb);
const measured = positiveBytes(options.measuredTotalBytes);
if (measured !== undefined) {
return { limitBytes: measured, budgetBytes: Math.max(0, Math.min(measured, afterReserve)), source: 'measured' };
}
// 0 means "not set" (both test Macs report 0): only a value above 0 is a limit.
const wired = hw.wiredLimitMb !== undefined && Number.isFinite(hw.wiredLimitMb) && hw.wiredLimitMb > 0;
const limitBytes = wired
? Math.floor((hw.wiredLimitMb as number) * MIB)
: Math.floor(memBytes * (hw.memClassGb < LARGE_LIMIT_FROM_GB ? 2 / 3 : 3 / 4));
const budgetBytes = Math.max(0, Math.min(limitBytes - GIB, memBytes - reserveBytes(hw.memClassGb)));
const budgetBytes = Math.max(0, Math.min(limitBytes - GIB, afterReserve));
return { limitBytes, budgetBytes, source: wired ? 'wired-limit' : 'estimated' };
}
/** The prompt cache of the last tier whose minimum is not above this Mac's memory size. */
function promptCacheBytes(catalog: LocalModelCatalog, memClassGb: number): number {
let bytes = 0;
for (const tier of catalog.promptCacheTiers) {
if (tier.minMemGb <= memClassGb) bytes = tier.bytes;
}
return bytes;
}
/** F: the bytes `model` needs in memory at a context of `ctx` tokens on this Mac. */
export function needBytes(model: LocalModelEntry, ctx: number, catalog: LocalModelCatalog, hw: HardwareFacts): number {
return model.bytes + model.kvBytesPerToken * ctx + RUNTIME_OVERHEAD_BYTES + promptCacheBytes(catalog, hw.memClassGb);
/**
* The bytes `model` needs in memory at a context of `ctx` tokens, at its PEAK: the weights,
* what stays held for the context, what the runner holds beside them, and what it takes for a
* moment while a long prompt is processed. It does not depend on the Mac.
*/
export function needBytes(model: LocalModelEntry, ctx: number): number {
return model.bytes + model.heldBytesPerToken * ctx + model.overheadBytes + model.peakExtraBytes;
}
function fitGrade(need: number, budgetBytes: number): FitGrade {
@@ -139,12 +167,10 @@ function fitGrade(need: number, budgetBytes: number): FitGrade {
/**
* Whether the catalogue model `modelId` at `context` tokens fits this Mac's budget (a `good`
* or a `tight` fit). The install job asks this right after the runtime started, with the
* budget the runtime measured, BEFORE it downloads the model: a `false` ends the job with
* `does-not-fit`. It is the same judgement `recommend()` makes for a candidate, without the
* ranking, so a model that fits is never ruled out for being listed sixth.
*
* `options` is the stage 2 input, as for `recommend()`; nothing reads it yet, so the budget
* is still the estimate. False for a model the catalogue does not have.
* budget the runtime measured (`options.measuredTotalBytes`), BEFORE it downloads the model: a
* `false` ends the job with `does-not-fit`. It is the same judgement `recommend()` makes for a
* candidate, without the ranking, so a model that fits is never ruled out for being listed
* sixth. False for a model the catalogue does not have.
*/
export function modelFitsBudget(
hw: HardwareFacts,
@@ -153,10 +179,9 @@ export function modelFitsBudget(
context: number,
options: RecommendOptions = {}
): boolean {
void options;
const model = catalog.models.find((entry) => entry.id === modelId);
if (!model) return false;
return fitGrade(needBytes(model, context, catalog, hw), deviceBudget(hw).budgetBytes) !== 'out';
if (!model || !Number.isFinite(context) || context <= 0) return false;
return fitGrade(needBytes(model, context), deviceBudget(hw, options).budgetBytes) !== 'out';
}
/**
@@ -189,13 +214,12 @@ interface UsableHarness {
fact: HarnessFact;
}
const FIT_ORDER: Record<Candidate['fit'], number> = { good: 0, tight: 1 };
const SPEED_ORDER: Record<SpeedGrade, number> = { ok: 0, unknown: 1, slow: 2 };
/** Best first. */
/** Best first. The fit grade is not a key: every candidate here fits, see the file header. */
function compareRanked(a: Ranked, b: Ranked): number {
return (
FIT_ORDER[a.candidate.fit] - FIT_ORDER[b.candidate.fit] ||
Number(b.candidate.tested) - Number(a.candidate.tested) ||
SPEED_ORDER[a.candidate.speed] - SPEED_ORDER[b.candidate.speed] ||
b.modelRank - a.modelRank ||
b.harnessRank - a.harnessRank ||
@@ -212,27 +236,53 @@ function accepts(fact: HarnessFact, modelId: string): boolean {
}
}
/** What the chooser's "Runs with" select needs to know about one harness. */
function candidateHarness(harness: UsableHarness): CandidateHarness {
const { row, fact } = harness;
return {
cli: row.cli,
cliLabel: fact.label,
installed: fact.installed,
// Only a harness that still has to be installed has a command to show.
...(!fact.installed && fact.installCommand ? { installCommand: fact.installCommand } : {}),
...(row.slowFirstTurn ? { slowFirstTurn: true } : {}),
};
}
/**
* The entry that makes this model "tested on a Mac like this": exactly the same chip string,
* the same memory size as sold, the same harness.
*/
function testedEntry(model: LocalModelEntry, hw: HardwareFacts, cli: string): TestedOn | undefined {
return model.tested.find((entry) => entry.chip === hw.chip && entry.memGb === hw.memClassGb && entry.cli === cli);
}
function buildCandidate(
model: LocalModelEntry,
ctx: number,
harness: UsableHarness,
/** Every usable harness that can run this model at this context, `harness` among them. */
runsWith: readonly UsableHarness[],
need: number,
fit: Candidate['fit'],
hw: HardwareFacts,
catalog: LocalModelCatalog,
budgetBytes: number
budgetBytes: number,
runtimeDownloadBytes: number
): Candidate {
const testedOn = testedEntry(model, hw, harness.row.cli);
const gbps = chipBandwidthGbps(catalog.chips, hw.chip);
// A speed measured on a Mac like this one beats the estimate from the chip's bandwidth.
const rawSpeed =
gbps !== undefined && model.speedPer100Gbps !== undefined ? (model.speedPer100Gbps * gbps) / 100 : undefined;
testedOn?.tokensPerSecond ??
(gbps !== undefined && model.speedPer100Gbps !== undefined ? (model.speedPer100Gbps * gbps) / 100 : undefined);
// Graded on the unrounded figure: 9.96 rounds to 10.0 for the payload and is still slow.
const speed: SpeedGrade =
rawSpeed === undefined ? 'unknown' : rawSpeed < SLOW_BELOW_TOKENS_PER_SECOND ? 'slow' : 'ok';
const tokensPerSecond = rawSpeed === undefined ? undefined : round(rawSpeed, 1);
// "A Mac like this" is exact: the same chip string, the same memory size as sold, the same harness.
const tested = model.tested.some(
(entry) => entry.chip === hw.chip && entry.memGb === hw.memClassGb && entry.cli === harness.row.cli
);
// The harness of the flat fields first, then the others, best first.
const others = runsWith.filter((other) => other !== harness).sort((a, b) => b.row.rank - a.row.rank);
const installBytes = model.bytes + runtimeDownloadBytes;
return {
modelId: model.id,
modelLabel: model.label,
@@ -254,10 +304,39 @@ function buildCandidate(
...(speed === 'slow' ? { speedLabel: LOCAL_MODEL_LABELS.speedSlow } : {}),
...(speed === 'unknown' ? { speedLabel: LOCAL_MODEL_LABELS.speedUnknown } : {}),
...(tokensPerSecond !== undefined ? { tokensPerSecond } : {}),
tested,
tested: testedOn !== undefined,
needSource: model.figures,
harnesses: [harness, ...others].map(candidateHarness),
installBytes,
installGb: formatGb(installBytes),
};
}
/**
* The one tag of a row: the first kind that applies, in the order tested, best (the pick),
* tight, slow, small. `undefined` when none does.
*/
function tagFor(candidate: Candidate, isPick: boolean): CandidateTag | undefined {
const kind: CandidateTagKind | undefined = candidate.tested
? 'tested'
: isPick
? 'best'
: candidate.fit === 'tight'
? 'tight'
: candidate.speed === 'slow'
? 'slow'
: candidate.paramsB < LIMITED_BELOW_PARAMS_B
? 'small'
: undefined;
return kind === undefined ? undefined : { kind, label: LOCAL_MODEL_TAG_LABELS[kind] };
}
/** `candidate` with its tag, when it has one. */
function withTag(candidate: Candidate, isPick: boolean): Candidate {
const tag = tagFor(candidate, isPick);
return tag ? { ...candidate, tag } : candidate;
}
/** A `none` answer. `text` replaces the reason's default sentence (the `memory` reason has three). */
function none(
reason: NoneReason,
@@ -281,9 +360,9 @@ function none(
* The recommendation for this Mac. `harnessFacts` holds one fact per catalogue harness the
* caller could resolve; a catalogue harness without a fact is not offered.
*
* `options` is the stage 2 input (`RecommendOptions`: the budget the runtime measured, and
* the runtime download an install still needs). The parameter is in place so callers compile
* against the final signature; nothing reads it yet.
* `options` is what the caller knows beyond the probe: the budget the runtime measured (the
* answer then says `budgetSource: 'measured'`), and the runtime download an install still
* needs (it is added to every row's `installBytes`).
*/
export function recommend(
hw: HardwareFacts,
@@ -291,13 +370,13 @@ export function recommend(
harnessFacts: readonly HarnessFact[],
options: RecommendOptions = {}
): Recommendation {
void options;
// Off macOS and on an Intel Mac there is no budget to compute: a model has no memory to run in.
const noBudget: DeviceBudget = { limitBytes: 0, budgetBytes: 0, source: 'estimated' };
if (hw.platform !== 'darwin') return none('not-macos', noBudget);
if (!isAppleSilicon(hw)) return none('intel', noBudget);
const budget = deviceBudget(hw);
const budget = deviceBudget(hw, options);
const runtimeDownloadBytes = positiveBytes(options.runtimeDownloadBytes) ?? 0;
if (hw.macosMajor !== undefined && hw.macosMajor < catalog.minMacosMajor) {
return none('macos', budget, {
version: hw.macosVersion ?? String(hw.macosMajor),
@@ -309,7 +388,7 @@ export function recommend(
const fitting: { model: LocalModelEntry; ctx: number; need: number; fit: Candidate['fit'] }[] = [];
for (const model of catalog.models) {
for (const ctx of model.contexts) {
const need = needBytes(model, ctx, catalog, hw);
const need = needBytes(model, ctx);
const fit = fitGrade(need, budget.budgetBytes);
if (fit !== 'out') fitting.push({ model, ctx, need, fit });
}
@@ -335,14 +414,14 @@ export function recommend(
usable.filter((harness) => entry.ctx >= harness.row.minContext).map((harness) => ({ ...entry, harness }))
);
if (atHarnessContext.length === 0) {
// Models fit, but only below the context a harness needs. Say both numbers: the smallest
// need at such a context can sit a hair over the line, and the budget is an estimate.
// Models fit, but only below the context a harness needs. Say both numbers: the reader
// sees how far apart they are, and that both are estimates.
const smallestContext = Math.min(...usable.map((harness) => harness.row.minContext));
let smallestNeed: number | undefined;
for (const model of catalog.models) {
for (const ctx of model.contexts) {
if (ctx < smallestContext) continue;
const need = needBytes(model, ctx, catalog, hw);
const need = needBytes(model, ctx);
if (smallestNeed === undefined || need < smallestNeed) smallestNeed = need;
}
}
@@ -379,11 +458,15 @@ export function recommend(
entry.model,
entry.ctx,
entry.harness,
accepted
.filter((other) => other.model === entry.model && other.ctx === entry.ctx)
.map((other) => other.harness),
entry.need,
entry.fit,
hw,
catalog,
budget.budgetBytes
budget.budgetBytes,
runtimeDownloadBytes
),
model: entry.model,
modelRank: entry.model.rank,
@@ -398,7 +481,8 @@ export function recommend(
}
const offered = bestPerModel.filter((entry) => hasRoom(entry.model)).map((entry) => entry.candidate);
const leftOutForDisk = bestPerModel.find((entry) => !hasRoom(entry.model));
const [pick, ...others] = offered;
const pick = withTag(offered[0], true);
const alternatives = offered.slice(1, 1 + MAX_ALTERNATIVES).map((candidate) => withTag(candidate, false));
// About the Mac, not the pick, and judged before the disk filter (see the file header).
const limited = !accepted.some((entry) => entry.model.paramsB >= LIMITED_BELOW_PARAMS_B);
@@ -427,7 +511,7 @@ export function recommend(
headline: { text: headline },
notes,
pick,
alternatives: others.slice(0, MAX_ALTERNATIVES),
alternatives,
budgetBytes: budget.budgetBytes,
budgetGb: formatGb(budget.budgetBytes),
budgetSource: budget.source,
src/types/local-model.ts
+61 -60
View File
@@ -15,10 +15,10 @@
* call time; it holds a function, so it never goes over HTTP).
* - `Recommendation`, `Candidate`, `DeviceBudget`: src/local-model/recommend.ts (pure).
*
* Fields marked "Stage 2" were added for the install. They are OPTIONAL in the type only so
* that the stage 1 catalogue and `recommend()` keep compiling until their owner fills them:
* once `recommend()` and the catalogue rows carry them, they become required, and a consumer
* (the chooser window, the install job) may rely on every one of them.
* A field that is optional here is optional by meaning, and its comment says when it is
* absent (a tested entry without a measured speed, a harness that needs no install command, a
* row without a tag). Everything else is always there, so the chooser window and the install
* job may rely on it.
*
* Three rules every producer and consumer follows:
*
@@ -97,7 +97,11 @@ export interface HardwareFacts {
// ---------- Catalogue ----------
/** One Mac a model and harness pair was really run on by us. A model row with a matching entry may be called "Recommended". */
/**
* One Mac a model and harness pair was really run on by us. A candidate that matches an entry
* (the same chip string, the same memory size as sold, the same harness) is `tested`: it ranks
* before every candidate that is not, and as the pick it may be called "Recommended".
*/
export interface TestedOn {
/** The chip, as `machdep.cpu.brand_string` reports it ('Apple M4'). */
chip: string;
@@ -108,7 +112,7 @@ export interface TestedOn {
/** ISO date of the test ('2026-11-06'). */
date: string;
/**
* Stage 2. The generation speed measured in that test, in tokens per second (67.7 for the
* The generation speed measured in that test, in tokens per second (67.7 for the
* 2026-11-06 run on an Apple M5 Pro with 48 GB). A candidate that matches this entry reports
* this figure as `Candidate.tokensPerSecond` in place of the estimate from the chip's memory
* bandwidth: a measurement on the same Mac beats a formula.
@@ -129,46 +133,41 @@ export interface LocalModelEntry {
/**
* Size of the weights in bytes: both the download and what they take in memory.
*
* Stage 2: the size of the build the runtime really downloads ON A MAC. That is not always
* It is the size of the build the runtime really downloads ON A MAC. That is not always
* the sum of the registry manifest: `qwen3.6:35b-a3b` is an MLX build of 23,105,798,956 bytes
* on Apple silicon, 2% more than the manifest's 22,621,314,161. A row whose Mac size nobody
* measured keeps the manifest sum and says so in `figures`.
*/
bytes: number;
/**
* Stage 1 figure: the bytes of KV cache one token of context costs, COMPUTED from the model's
* config. Measured on a Mac it was too low by almost half (20 KiB against about 37 KB held),
* so the need is computed from `heldBytesPerToken` once that is filled. Kept only until
* `recommend()` stops reading it; it is deleted together with `promptCacheTiers` then.
* Bytes that stay HELD in memory per token of context once the token was processed (measured
* on the MLX runner: 36,000 to 38,000, so about 37,000). A figure computed from a model's
* config alone came out too low by almost half, so a row that was not measured carries the
* measured row's figure and says so in `figures`.
*/
kvBytesPerToken: number;
heldBytesPerToken: number;
/** What the runner holds beside the weights and the context (measured: about 3 GB). */
overheadBytes: number;
/**
* Stage 2. Bytes that stay HELD in memory per token of context once the token was processed
* (measured on the MLX runner: 36,000 to 38,000, so about 37,000).
*/
heldBytesPerToken?: number;
/** Stage 2. What the runner holds beside the weights and the context (measured: about 3 GB). */
overheadBytes?: number;
/**
* Stage 2. The TRANSIENT extra the runner takes while it processes a long prompt, on top of
* The TRANSIENT extra the runner takes while it processes a long prompt, on top of
* what stays held (measured: about 5 GB). The fit is judged on the peak:
* `bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`. For
* `qwen3.6:35b-a3b` at 131072 tokens that is about 36 GB, which is tight on a 48 GB Mac.
*/
peakExtraBytes?: number;
peakExtraBytes: number;
/**
* Stage 2. Where `bytes` and the three memory figures above come from: `measured` on a Mac by
* Where `bytes` and the three memory figures above come from: `measured` on a Mac by
* us, or `estimated` (the manifest size, and the per-token, overhead and peak figures of the
* measured row applied to this one).
*/
figures?: 'measured' | 'estimated';
figures: 'measured' | 'estimated';
/** Context lengths (tokens) the model may be run at, largest first. */
contexts: number[];
/** Preference among models that fit: the HIGHER rank wins. A judgement, not a measurement. */
rank: number;
/** Measured tokens per second per 100 GB/s of memory bandwidth. Absent: the speed of this model is unknown. */
speedPer100Gbps?: number;
/** Macs this model was really run on. Empty today for every row. */
/** Macs this model was really run on. Empty for every row but one. */
tested: TestedOn[];
}
@@ -201,16 +200,6 @@ export interface ChipEntry {
gbps: number;
}
/**
* Host memory the runtime's prompt cache takes, by the size of the Mac. Tiers are ordered by
* `minMemGb` ascending and the last tier with `minMemGb <= HardwareFacts.memClassGb` applies.
*/
export interface PromptCacheTier {
/** Smallest memory size (as sold, binary gigabytes) this tier applies to. The first tier has 0. */
minMemGb: number;
bytes: number;
}
/** The catalogue: a compiled constant, validated by `LocalModelCatalogSchema`. No override file, no download data. */
export interface LocalModelCatalog {
schema: 1;
@@ -218,7 +207,6 @@ export interface LocalModelCatalog {
updated: string;
/** Oldest macOS major version a local model runs on. Below it: verdict `none`, reason `macos`. */
minMacosMajor: number;
promptCacheTiers: PromptCacheTier[];
models: LocalModelEntry[];
harnesses: LocalHarnessEntry[];
chips: ChipEntry[];
@@ -241,7 +229,7 @@ export interface HarnessFact {
/** Not installed, but the registry has an install command for this platform that needs no administrator rights. */
installable: boolean;
/**
* Stage 2. The literal install command the install job would run for this CLI: the STOCK
* The literal install command the install job would run for this CLI: the STOCK
* command of the CLI registry for this platform, never one a `clis.json` override supplies.
* Present exactly when `installable` is true. The consent screen prints it
* (`CandidateHarness.installCommand`).
@@ -253,7 +241,12 @@ export interface HarnessFact {
// ---------- Recommendation ----------
/** Need over budget: up to 0.85 is `good`, up to 0.98 is `tight`, above is `out`. */
/**
* Need over budget: up to 0.85 is `good`, up to 0.98 is `tight`, above is `out`. The need is
* the PEAK, so the grade says how much room is left at the worst moment. It decides whether a
* model is offered at all (`out` never is) and what its row says (`tight`). It does not sort
* the list.
*/
export type FitGrade = 'good' | 'tight' | 'out';
/**
@@ -280,7 +273,7 @@ export type NoneReason = 'not-macos' | 'intel' | 'memory' | 'disk' | 'macos' | '
* Where the device budget came from.
* - `estimated`: a formula over the installed memory. All there is before the runtime ran.
* - `wired-limit`: `iogpu.wired_limit_mb` above 0.
* - `measured` (stage 2): the device total the runtime itself printed when it started
* - `measured`: the device total the runtime itself printed when it started
* (`RecommendOptions.measuredTotalBytes`). The chooser says "estimated" next to the memory
* bar for the first value only.
*/
@@ -304,8 +297,8 @@ export interface DeviceBudget {
}
/**
* Stage 2. What the caller of `recommend()` knows beyond the hardware probe. Both members are
* optional and an empty object is the stage 1 behaviour.
* What the caller of `recommend()` knows beyond the hardware probe. Both members are optional:
* an empty object means the budget is the estimate and an install downloads the model alone.
*/
export interface RecommendOptions {
/**
@@ -346,7 +339,8 @@ export interface RecommendOptions {
* | `noneMacos` | `version`: `HardwareFacts.macosVersion`; `minimum`: the catalogue minimum |
*
* The word "Recommended" exists only in `headlineTested`, which may be used only when the
* pick's `tested` is true. No catalogue row has a tested entry today, so it is never sent.
* pick's `tested` is true: one catalogue row has a tested entry (`qwen3.6:35b-a3b` on an Apple
* M5 Pro with 48 GB), so one kind of Mac is told so.
*/
export const LOCAL_MODEL_LABELS = {
/** Verdict `ok`, pick not tested by us on a Mac like this one. */
@@ -405,7 +399,7 @@ export interface LocalModelMessage {
}
/**
* Stage 2. The one tag a model row of the chooser carries, by what it says.
* The one tag a model row of the chooser carries, by what it says.
* - `tested`: `Candidate.tested` is true. Replaces every other tag.
* - `best`: the pick. The one tag drawn in the accent colour.
* - `tight`: `Candidate.fit` is `tight`.
@@ -431,7 +425,7 @@ export const LOCAL_MODEL_TAG_LABELS = {
/** One of the texts in `LOCAL_MODEL_TAG_LABELS`. */
export type LocalModelTagLabel = (typeof LOCAL_MODEL_TAG_LABELS)[CandidateTagKind];
/** Stage 2. The tag of a model row: what kind it is (for the styling) and the text to print. */
/** The tag of a model row: what kind it is (for the styling) and the text to print. */
export interface CandidateTag {
kind: CandidateTagKind;
/** `LOCAL_MODEL_TAG_LABELS[kind]`. */
@@ -439,7 +433,7 @@ export interface CandidateTag {
}
/**
* Stage 2. One coding agent a candidate's model can be run with on this Mac, for the "Runs
* One coding agent a candidate's model can be run with on this Mac, for the "Runs
* with" select of the chooser. Everything a browser needs is here, so it compares no CLI id.
*/
export interface CandidateHarness {
@@ -477,10 +471,9 @@ export interface Candidate {
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
/**
* What the model needs in memory at this context. Stage 1: F = weights + kvBytesPerToken *
* context + 1 GiB + prompt cache. Stage 2: the PEAK, `bytes + heldBytesPerToken * context +
* overheadBytes + peakExtraBytes` of the catalogue row. The memory bar of the chooser is
* filled to `budgetShare`, which is this over the budget.
* What the model needs in memory at this context: the PEAK, `bytes + heldBytesPerToken *
* context + overheadBytes + peakExtraBytes` of the catalogue row. The memory bar of the
* chooser is filled to `budgetShare`, which is this over the budget.
*/
needBytes: number;
/** `needBytes` as decimal GB text, no unit. */
@@ -501,30 +494,33 @@ export interface Candidate {
/**
* Generation speed in tokens per second, rounded to one decimal. Present exactly when `speed`
* is not `unknown`; `speed` is decided on the unrounded figure. An estimate from the chip's
* memory bandwidth, except (stage 2) for a candidate that is `tested`, which reports the
* memory bandwidth, except for a candidate that is `tested`, which reports the
* speed measured in that test (`TestedOn.tokensPerSecond`).
*/
tokensPerSecond?: number;
/** A `TestedOn` entry of the model matches this Mac (same chip, same `memClassGb`) and this harness. */
/**
* A `TestedOn` entry of the model matches this Mac (same chip, same `memClassGb`) and this
* harness. Such a candidate ranks first, and its `tag` says so in place of any other tag.
*/
tested: boolean;
/** Stage 2. `LocalModelEntry.figures` of the model: whether `needBytes` rests on a measurement of this model. */
needSource?: 'measured' | 'estimated';
/** Stage 2. The row's one tag, see `CandidateTagKind`. Absent when no kind applies. */
/** `LocalModelEntry.figures` of the model: whether `needBytes` rests on a measurement of this model. */
needSource: 'measured' | 'estimated';
/** The row's one tag, see `CandidateTagKind`. Absent when no kind applies. */
tag?: CandidateTag;
/**
* Stage 2. Every coding agent this model can be run with at `context` on this Mac, best
* Every coding agent this model can be run with at `context` on this Mac, best
* first. The first entry is the one the flat fields above name (`cli`, `cliLabel`,
* `cliInstalled`, `slowFirstTurn`). Never empty.
*/
harnesses?: CandidateHarness[];
harnesses: CandidateHarness[];
/**
* Stage 2. The whole download a click on Install starts for this row: `downloadBytes` plus
* The whole download a click on Install starts for this row: `downloadBytes` plus
* `RecommendOptions.runtimeDownloadBytes`. It is the server's own total, and exactly the
* number `POST /api/local-model/install` must be sent as `acknowledgedBytes`.
*/
installBytes?: number;
/** Stage 2. `installBytes` as decimal GB text, no unit. */
installGb?: string;
installBytes: number;
/** `installBytes` as decimal GB text, no unit. */
installGb: string;
}
/**
@@ -547,7 +543,12 @@ export interface Recommendation {
notes: LocalModelMessage[];
/** Present exactly when `verdict` is `ok` or `limited`. */
pick?: Candidate;
/** The next models that fit and have room on the disk (at most four, each at its best context and harness), best first. Empty when `verdict` is `none`. */
/**
* The next models that fit and have room on the disk (at most four), best first: tested
* before not tested, then ok before unknown before slow speed, then the higher model rank.
* Each is at the largest context it fits at, with the best harness that can run it there.
* Empty when `verdict` is `none`.
*/
alternatives: Candidate[];
/** `DeviceBudget.budgetBytes`. 0 when no budget is computed: not macOS, or a Mac without Apple silicon. */
budgetBytes: number;