feat(local-model): list every model that fits, and add Muse Glimmer 30B

The "Run a local model" chooser showed the pick and four more models. It now lists every catalogue model that fits the Mac, and the install accepts every listed row. - MAX_ALTERNATIVES goes from 4 to 11, above what the catalogue holds. It stays a bound: test/local-model-catalog.test.ts fails for a catalogue that outgrows it, so nothing drops off the list unseen. - New row `muse-glimmer:30b-q4_k_m` (Meta, Apache-2.0): SWE-bench Verified 76.0, SWE-bench Pro 51.2, Terminal-Bench 2.1 51.7 on Meta's card. Its K/V cache is the smallest of the 27B class, so it is the first model of that class a 32 GB Mac holds: 21.2 GB at 65,536 tokens, 0.970 of the estimated budget. Figures computed from its config.json and its GGUF header by the llamacpp rule (13 full-attention layers, 39 sliding-window layers of 2,048 tokens, no prediction layer); the sliding-window cache rule was read again in llama.cpp b12111 (src/llama-kv-cache-iswa.cpp). Never run on a Mac: `estimated`. - What changes for a Mac: 32 GB is no longer `limited`. On an M1 Max or M2 Max the pick is Muse Glimmer; on a base or Pro chip the dense model is slow and the pick stays MiMo V2.6 9B, now with the small-pick note. Every other pick is unchanged. - The "set aside" list was read again at its sources on 2026-11-10 and gained ripple-preview:65b (every build requires runtime 0.40.16, one above the pin), Ornith 1.0 (runtime issue 16992), nemotron-3.5-super:120b, qwen3.5:4b, qwen3-coder-next and the MLX and DFlash tags of Muse Glimmer. - The header and the guide now say that runtime issue 16992 has one diagnosed cause in the tool parser every Qwen-family row shares (a tool argument that quotes a whole tool call is cut off; the fix is runtime PR 17021, open). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 02:29:33 +01:00 · 2026-11-10 23:33:59 +01:00
1 parent f62c1f7e2f
commit 9608a17d99
@@ -24,8 +24,9 @@
 *    throws on a bad row. test/local-model-catalog.test.ts pins that the stock constant parses,
 *    so a bad row fails the gate and never reaches the route.
 *
- * Where each figure of the stock rows comes from, and what was checked (2026-11-08). [V] read
- * at the source that day, [M] measured on a Mac, [C] computed from [V] inputs:
+ * Where each figure of the stock rows comes from, and what was checked (2026-11-08; the row
+ * added on 2026-11-10, `muse-glimmer:30b-q4_k_m`, was read that day). [V] read at the source
+ * that day, [M] measured on a Mac, [C] computed from [V] inputs:
 *
 * | Figure               | Source                                                    | Checked                              |
 * |----------------------|-----------------------------------------------------------|--------------------------------------|
@@ -74,8 +75,10 @@
 *    src/local-model/runtime.ts, q8_0: 34 bytes per 32 values), and does not grow after that.
 *    "MTP" below is a GGUF with a multi-token-prediction layer (`nextn_predict_layers` 1 in
 *    its header, read on 2026-11-08 from the first bytes of each model layer: the two `-mtp`
- *    tags and `qwen3.8:27b-q8_0` have it, the two 9B tags and North Mini Code do not). For
- *    such a model the runtime always passes `--spec-type draft-mtp --spec-draft-n-max N`, N
+ *    tags and `qwen3.8:27b-q8_0` have it, the two 9B tags and North Mini Code do not; read
+ *    the same way on 2026-11-10, `muse-glimmer:30b-q4_k_m` does not either). For such a
+ *    model the runtime always passes
+ *    `--spec-type draft-mtp --spec-draft-n-max N`, N
 *    being the tag's own `draft_num_predict` (its params layer: 4 for
 *    `qwen3.8:27b-mtp-q4_k_m`, 2 for `qwen3.6:35b-a3b-mtp-q4_k_m`) or the runtime's default
 *    of 4 (`qwen3.8:27b-q8_0` sets none).
@@ -83,9 +86,11 @@
 *       adds its prediction layer, whose cache llama-server keeps in a draft context of the
 *       same length in the draft's own cache type, f16 by default (2 bytes per value).
 *     - overhead = the fixed caches (the f32 recurrent state of the linear-attention layers,
- *       or the sliding-window cache of `n_swa + 512` cells), held 1 + N times for an MTP
- *       model (llama.cpp keeps one recurrent state per drafted token for rollback,
- *       `need_n_rs_seq()`), + the context checkpoints: `LLAMA_ARG_CTX_CHECKPOINTS` host copies
+ *       or the sliding-window cache of `n_swa + 512` cells: `size_swa` in
+ *       src/llama-kv-cache-iswa.cpp, the cache every model with a sliding window gets), held
+ *       1 + N times for an MTP model (llama.cpp keeps one recurrent state per drafted token
+ *       for rollback, `need_n_rs_seq()`), + the context checkpoints:
+ *       `LLAMA_ARG_CTX_CHECKPOINTS` host copies
 *       (`RUNTIME_CTX_CHECKPOINTS`, 8) of the state that cannot be rolled back (the recurrent
 *       state, or the sliding-window cache), + 1.0 GB for the runner's own buffers (calibrated
 *       on a third-party table of a 27B model on llama.cpp: 8.4 GB at 4,096 tokens with
@@ -125,26 +130,59 @@
 * checked. For an `mlx` row it proves only the weights (issue 19417): the figures above are
 * all that stands between such a row and swap.
 *
- * Models set aside on purpose (a weekly review should look at them again):
+ * Models set aside on purpose (a weekly review should look at them again). Every reason was
+ * read again at its source on 2026-11-10:
+ *  - `ripple-preview:65b` (Ricursive, Apache-2.0, a dense 65B published on 2026-11-09;
+ *    Terminal-Bench 2.1 80.5 and SWE-bench Verified 76.5 on its card; 38.8 GB as a GGUF build,
+ *    45.8 GB as the MLX build a Mac pulls): every build requires runtime 0.40.16, one above
+ *    the pin. Computed from its config.json by the llamacpp rule, its GGUF build would be the
+ *    first row between the 27B and the 122B (47 GB at 131,072 tokens: 64 and 96 GB Macs, slow
+ *    below an Ultra chip). Look at it the day the pin moves.
 *  - `bonsai-2:27b`, `bonsai-2:9b` (ternary builds of Qwen3.8-27B and Qwen3.5-9B; the 27B
- *    would make a 24 GB Mac `ok`): the vendor reports no SWE-bench and no agent loop, and
- *    whether 2-bit weights keep tool calls reliable was never measured.
+ *    would make a 24 GB Mac `ok`): the vendor reports no SWE-bench and no agent loop (its card
+ *    of 2026-11-04 still lists BFCL v3 only), and whether 2-bit weights keep tool calls
+ *    reliable was never measured.
 *  - `ornith-1.5:9b`, `ornith-1.5:35b`, `ornith-1.5:397b` (GGUF only): strong cards, but their
 *    manifests name no renderer and no tool parser, so whether tool calls work through the
- *    runtime is unknown (runtime pull request 17965 is open). `ornith-1.5:122b` has both.
+ *    runtime is unknown (runtime pull request 17965 is still open). `ornith-1.5:122b` has
+ *    both.
+ *  - `ornith:9b`, `ornith:35b` (Ornith 1.0, MIT, with a renderer and a tool parser; SWE-bench
+ *    Verified 69.4 and 75.6 on Ornith's card, against 73.4 for Qwen3.6 35B-A3B): runtime issue
+ *    16992, open, reports the 35B's tool calls arriving as text in long agent chains, and the
+ *    two share the renderer and the parser. The 9B (5.6 GB, no vision projector) is the one
+ *    model that would fit a 16 GB Mac on the estimated budget: 9.3 GB at 65,536 tokens.
 *  - `gpt-oss:20b`: weak in agentic coding (SWE-bench Verified 52.4, Terminal-Bench 2.1 15.2
 *    in NVIDIA's table).
- *  - `flint:40b`: runtime issue 19191 breaks integer tool arguments after the first call.
+ *  - `flint:40b`: runtime issue 19191 (open) breaks integer tool arguments after the first
+ *    call.
 *  - `lfm2.5:24b-a2b`: its licence allows commercial use only below USD 10M of revenue.
 *  - `laguna-xs-2.1`: the runtime's own page names an open problem with it on macOS.
 *    `laguna-s-2.1`: open runtime issues 17602 and 17632.
- *  - `qwen3.8-flash-next:125b`: the Qwen Community License, not an open licence.
+ *  - `qwen3.8-flash-next:125b`: the Qwen Community License, not an open licence, and its
+ *    builds require runtime 0.40.16 since 2026-11-09.
+ *  - `nemotron-3.5-super:120b` (79.6 GB as the MLX build a Mac pulls): Terminal-Bench 2.1
+ *    60.7 and SWE-bench Verified 71.1 in NVIDIA's table, below `ornith-1.5:122b` at a larger
+ *    download, so it adds no choice a 128 GB Mac would take.
+ *  - `qwen3.5:4b-q4_k_m` (3.3 GB, it would fit a 16 GB Mac with room): runtime issue 18916,
+ *    open, has it answer with thinking only in a long conversation at 65,536 tokens.
+ *  - `qwen3-coder-next` (51.7 GB): the generation before Qwen3.6. Its card shows its results
+ *    as pictures only, so there is no figure to rank it by against the rows listed here.
+ *  - `muse-glimmer:30b-mlx` and the `-dflash` tags of that model: open runtime issues 17656
+ *    and 18269 on the MLX build, and no rule here covers the memory of a separate draft
+ *    model. Its GGUF build `muse-glimmer:30b-q4_k_m` is listed.
 *  - The plain tags `qwen3.5:9b`, `mimo-v2.6:9b` and `qwen3.8:27b`: a Mac pulls their MLX
 *    builds, whose 8 GiB of prefix snapshots do not fit the Macs a 9B or a 27B is for. Their
 *    GGUF builds are listed instead.
 *  - `qwen3.5:122b-a10b`: replaced by `ornith-1.5:122b`, a fine-tune of it (SWE-bench
 *    Verified 84.5 against 72.0) whose MLX build carries a renderer and a tool parser.
 *
+ * One open issue touches rows that ARE listed: runtime issue 16992 (above). Its one diagnosed
+ * cause sits in the tool parser itself (it closes a call at the first closing tag, so a tool
+ * argument that quotes a whole tool call is cut off; the fix, runtime pull request 17021, is
+ * open). The names `ornith` and `qwen3.5` map to that one parser in the pinned runtime
+ * (model/parsers/parsers.go), so every Qwen-family row here shares it, the pair we ran
+ * included. The check after a download asks for one plain tool call and cannot see this.
+ *
 * `tested` has one entry: `qwen3.6:35b-a3b` with OpenCode on an Apple M5 Pro with 48 GB,
 * 2026-11-06, 67.7 tokens per second. Only a Mac of exactly that chip and memory size is told
 * "Recommended" and shown that speed in place of the estimate. Every other row was never run
@@ -336,7 +374,7 @@ const MLX_PREFIX_CACHE_BYTES = 8 * 1024 * 1024 * 1024;
 */
 export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
  schema: 1,
-  updated: '2026-11-08',
+  updated: '2026-11-10',
  // The oldest macOS the model runtime supports.
  minMacosMajor: 14,
  // Ordered by rank. The higher rank wins among models that fit and whose speed is graded
@@ -344,7 +382,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
  // that are left out and why.
  models: [
    {
-      // Small Macs (16 to 32 GB, `limited`): the base model of the row below.
+      // Small Macs (18 to 32 GB): the base model of the row below.
      id: 'qwen3.5:9b-q4_k_m',
      label: 'Qwen3.5 9B',
      license: 'Apache-2.0',
@@ -365,7 +403,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      tested: [],
    },
    {
-      // Small Macs (16 to 32 GB, `limited`). MiMo-V2.6-Distill-Qwen-9B, a fine-tune of
+      // Small Macs (18 to 32 GB). MiMo-V2.6-Distill-Qwen-9B, a fine-tune of
      // Qwen3.5-9B for code and agents: SWE-bench Verified 61.1 and SWE-bench Pro 44.6,
      // against 60.0 and 32.0 for its base (Xiaomi's card). Requires exactly the pinned
      // runtime, 0.40.15.
@@ -410,6 +448,33 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      speedPer100Gbps: 13.6,
      tested: [],
    },
+    {
+      // Meta's 30B for local agents, a dense 27.9B language model with a vision encoder:
+      // SWE-bench Verified 76.0, SWE-bench Pro 51.2, Terminal-Bench 2.1 51.7 (Meta's card). Its
+      // K/V cache is the smallest of the 27B class (2 K/V heads of 128, and three layers in
+      // four look at a window of 2,048 tokens), so it is the first model of that class a 32 GB
+      // Mac can hold: 21.2 GB at 65,536 tokens, tight. Dense, so it is slow below a Max chip.
+      // Its own window ends at 131,072 tokens.
+      id: 'muse-glimmer:30b-q4_k_m',
+      label: 'Muse Glimmer 30B',
+      license: 'Apache-2.0',
+      paramsB: 29.6,
+      runner: 'llamacpp',
+      bytes: 18_157_010_252,
+      // 13 full-attention layers x 2 K/V heads x 128 x 2 x 34/32.
+      heldBytesPerToken: 7_072,
+      // 39 sliding-window layers: (2,048 + 512) cells x 2 x 128 x 2 x 34/32 = 54,312,960, live
+      // and in each context checkpoint.
+      overheadBytes: 54_312_960 * (1 + LLAMACPP_CTX_CHECKPOINTS) + LLAMACPP_RUNNER_BYTES + LLAMACPP_PROMPT_CACHE_BYTES,
+      peakExtraBytes: 0,
+      figures: 'estimated',
+      contexts: [131072, 65536],
+      rank: 4,
+      // Not measured: the 2.95 of Qwen3.8 27B, a dense model whose weights are the same size on
+      // llama.cpp (16.8 GB each without the projector).
+      speedPer100Gbps: 2.95,
+      tested: [],
+    },
    {
      // Macs of 36 GB on a slower chip: the model of the tested pair, as a GGUF build whose
      // memory does not grow after it loaded. With its multi-token-prediction layer.
@@ -430,7 +495,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      peakExtraBytes: 0,
      figures: 'estimated',
      contexts: [262144, 131072, 65536],
-      rank: 4,
+      rank: 5,
      // Measured: 54.5 tokens per second on 400 GB/s (llama.cpp, without prediction).
      speedPer100Gbps: 13.6,
      tested: [],
@@ -448,7 +513,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      ...MEASURED_MEMORY,
      figures: 'measured',
      contexts: [131072, 65536],
-      rank: 5,
+      rank: 6,
      // Measured: 67.8 tokens per second on 400 GB/s with MLX. The one Mac we tested ran it at
      // 67.7 on 307 GB/s, so on other chips the estimate is on the low side.
      speedPer100Gbps: 16.9,
@@ -473,7 +538,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      peakExtraBytes: 0,
      figures: 'estimated',
      contexts: [262144, 131072, 65536],
-      rank: 6,
+      rank: 7,
      // Not measured: the 2.95 of the row below scaled by file size.
      speedPer100Gbps: 1.7,
      tested: [],
@@ -497,7 +562,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      peakExtraBytes: 0,
      figures: 'estimated',
      contexts: [262144, 131072, 65536],
-      rank: 7,
+      rank: 8,
      // Measured: 11.8 tokens per second on 400 GB/s (llama.cpp, Q4_K_M, without prediction).
      // This build predicts several tokens at a time, so it is faster than this figure says.
      speedPer100Gbps: 2.95,
@@ -521,7 +586,7 @@ export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
      peakExtraBytes: MEASURED_MEMORY.peakExtraBytes,
      figures: 'estimated',
      contexts: [262144, 131072],
-      rank: 8,
+      rank: 9,
      // Not measured: the 16.9 of Qwen3.6 35B-A3B on MLX x 3/10, its active parameters
      // against these, rounded down.
      speedPer100Gbps: 5,
@@ -28,9 +28,10 @@
 *    gated, installed or installable, and accepting the model id.
 *  - Ranking, lexicographic: tested on a Mac like this before not tested (a measurement beats
 *    a formula); ok before unknown before slow; the higher model rank; the higher harness
- *    rank; the larger context. The pick is the first candidate, the alternatives are the next
- *    four MODELS, each at its own best context and harness, so a model is always offered at
- *    the largest context it fits at.
+ *    rank; the larger context. The pick is the first candidate, the alternatives are EVERY
+ *    other model that fits (`MAX_ALTERNATIVES` is above what the catalogue holds), each at its
+ *    own best context and harness, so a model is always offered at the largest context it
+ *    fits at.
 *  - Disk: a model whose download would leave less than 10 GiB free is not offered. It is not
 *    dropped in silence: the best-ranked model left out that way is named in `noteDiskDropped`
 *    with what is missing.
@@ -109,8 +110,13 @@ export const SLOW_BELOW_TOKENS_PER_SECOND = 10;
 export const LIMITED_BELOW_PARAMS_B = 12;
 /** What must stay free on the disk after the model download. */
 export const DISK_HEADROOM_BYTES = 10 * GIB;
-/** How many other models are listed under the pick. */
-export const MAX_ALTERNATIVES = 4;
+/**
+ * How many other models are listed under the pick, at most. Above what the stock catalogue
+ * holds (9 models: 8 beside the pick), so a Mac is shown every model that fits it and the
+ * install accepts every one of them. Still a bound: test/local-model-catalog.test.ts fails for
+ * a catalogue that outgrows it, so nothing drops out of the list unseen.
+ */
+export const MAX_ALTERNATIVES = 11;

 /** A memory size of at least this many GB (as sold) gets 3/4 of its memory as device limit, below it 2/3. */
 const LARGE_LIMIT_FROM_GB = 36;
@@ -577,10 +577,10 @@ export interface Recommendation {
  /** Present exactly when `verdict` is `ok` or `limited`. */
  pick?: Candidate;
  /**
-   * The next models that fit and have room on the disk (at most four), best first: tested
-   * before not tested, then ok before unknown before slow speed, then the higher model rank.
-   * Each is at the largest context it fits at, with the best harness that can run it there.
-   * Empty when `verdict` is `none`.
+   * Every other model that fits and has room on the disk (`MAX_ALTERNATIVES` is above what the
+   * catalogue holds), best first: tested before not tested, then ok before unknown before
+   * slow speed, then the higher model rank. Each is at the largest context it fits at, with
+   * the best harness that can run it there. Empty when `verdict` is `none`.
   */
  alternatives: Candidate[];
  /** `DeviceBudget.budgetBytes`. 0 when no budget is computed: not macOS, or a Mac without Apple silicon. */