BitSquare/Codeman
2
0 Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-11-14 10:39:32 +01:00
Code Issues Packages Projects Releases Wiki Activity

feat(local-model): Ornith 1.0 9B, the first model a 16 GB Mac can hold

Browse Source 
A 16 GB Mac got no model: the two 9B builds of the catalogue need 10.2 GB at
65,536 tokens, 0.987 of the estimated budget, just past the 0.98 a tight fit
may reach.

New row `ornith:9b-q4_k_m` (Ornith 1.0 9B, MIT), a fine-tune of Qwen3.5-9B for
coding agents whose GGUF build has no vision projector: 5.63 GB against 6.55 GB,
so 9.3 GB at 65,536 tokens, 0.898 of that budget. SWE-bench Verified 69.4,
SWE-bench Pro 42.9, Terminal-Bench 2.1 43.1 on Ornith's card. Its config.json
and its GGUF header are those of Qwen3.5-9B (no prediction layer), so it shares
the figures of the two other 9B rows. Never run on a Mac: `estimated`.

- What changes for a Mac: 16 GB goes from no model to `limited` with this row,
  tight. That is the only pick that changes: the row is ranked under MiMo V2.6
  9B, so every Mac that could already run a 9B keeps its pick and gains an
  alternative. With the budget a 16 GB Mac mini measured (10.7 GB) the pick is
  MiMo as before.
- Why under MiMo although its own card's numbers are higher: the two vendors'
  tables put their common base 7 points apart, and runtime issue 16992 (open)
  was filed on the 35B sibling, which has the same renderer and parser. The
  35B stays set aside.
- The 16 GB rule itself is untouched: same budget estimate, same 0.98, same
  eight context checkpoints.
- Ranks renumbered 1 to 10; the pins of test/local-model-recommend.test.ts and
  the table of docs/local-models.md follow.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-10 23:36:45 +01:00
1 parent 9608a17d99
commit d25cc08606
5 files changed
+151 -69
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/local-models.md
+7 -6
View File
@@ -18,16 +18,16 @@ in [architecture-invariants.md](architecture-invariants.md#local-model-install).
The window offers the best model of the catalogue (`src/config/local-model-catalog.ts`) that
fits this Mac's memory at a context a coding agent can work with, and under it every other
model of the catalogue that fits (nine models today: an 18 GB Mac sees two of them, a 512 GB
Mac all nine). What the catalogue answers today, before the runtime measured anything (the
model of the catalogue that fits (ten models today: a 16 GB Mac sees one of them, a 512 GB
Mac all ten). What the catalogue answers today, before the runtime measured anything (the
budget is then an estimate: 2/3 of the memory below 36 GB, 3/4 from 36 GB, less 1 GiB, and
never more than the memory less 6, 8 or 12 GiB for macOS, Codeman, a browser and the agent):
| Memory | Budget | What it gets |
| ---------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 8 GB | 2.1 GB | Nothing. The smallest model a coding agent can work with needs 10.2 GB at 65,536 tokens |
| 16 GB | 10.4 GB | Nothing. MiMo V2.6 9B at 65,536 tokens needs 10.2 GB, 98.7% of the budget, just past the 98% a tight fit may reach (it is 95% of the 10.7 GB a 16 GB Mac mini measured). On the edge either way: only a run on such a Mac settles it |
| 18 GB | 11.8 GB | A small model (`limited`): MiMo V2.6 9B at 131,072 tokens, 11.4 GB, tight. Its base, Qwen3.5 9B, is the one alternative |
| 8 GB | 2.1 GB | Nothing. The smallest model a coding agent can work with needs 9.3 GB at 65,536 tokens |
| 16 GB | 10.4 GB | A small model (`limited`): Ornith 1.0 9B at 65,536 tokens, 9.3 GB, tight (90%). Its build has no vision projector, which makes it 0.9 GB smaller than the other two 9B builds: MiMo V2.6 9B needs 10.2 GB there, 98.7% of the budget, just past the 98% a tight fit may reach. Once the runtime measures what a 16 GB Mac mini measured (10.7 GB), all three fit and MiMo is the pick. No 9B was run on such a Mac by us: only a run settles it |
| 18 GB | 11.8 GB | A small model (`limited`): MiMo V2.6 9B at 131,072 tokens, 11.4 GB, tight. Ornith 1.0 9B (10.5 GB) and Qwen3.5 9B are the alternatives |
| 24 GB | 16.1 GB | The same, with room. On a base M2 or M3 (100 GB/s) it computes to 9.3 tokens per second: slow |
| 32 GB | 21.8 GB | No longer a small-model Mac: Muse Glimmer 30B fits at 65,536 tokens (21.2 GB, 97%, tight). It is a dense model. On an M1 Max or M2 Max it computes to 11.8 tokens per second and is the pick; on a base or Pro chip it computes to under 6 and is listed as slow, and the pick stays MiMo V2.6 9B at 131,072 tokens with the note that it is a small model. North Mini Code 1.0 needs 23.2 GB at 65,536 tokens; once the runtime measures about 3/4 of the memory, as it did on a 16 and a 48 GB Mac, the Mac gets it at 131,072 tokens, or Qwen3.8 27B at 65,536 tokens on a Max chip |
| 36 GB | 27.9 GB | Qwen3.8 27B at 131,072 tokens on an M4 or M5 Max (27.0 GB, 97%); on a slower chip, where the dense 27B computes to under 10 tokens per second, Qwen3.6 35B-A3B as a GGUF build (27.1 GB, 97%). Both tight. Muse Glimmer 30B at 131,072 tokens (21.6 GB, 78%) is the alternative with room to spare |
@@ -78,6 +78,7 @@ Where every figure comes from:
| Model | Runner | Memory figures | Speed (tokens per second per 100 GB/s) |
| ---------------------------- | --------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------- |
| `qwen3.5:9b-q4_k_m` | llama.cpp | computed from the Qwen3.5-9B config | 9.3: a published run on an M1 Max |
| `ornith:9b-q4_k_m` | llama.cpp | the same (a fine-tune of Qwen3.5-9B; its config and its GGUF header say so) | 9.3: the same architecture and weights size |
| `mimo-v2.6:9b-q4_k_m` | llama.cpp | the same (a fine-tune of Qwen3.5-9B) | 9.3: the same architecture and size |
| `north-mini-code-1.0:q4_k_m` | llama.cpp | computed from its config | 13.6: the 35B-A3B's, the same 3 billion active parameters |
| `muse-glimmer:30b-q4_k_m` | llama.cpp | computed from its config: 13 full-attention and 39 sliding-window layers | 2.95: the dense 27B's, whose weights are the same size |
@@ -88,7 +89,7 @@ Where every figure comes from:
| `ornith-1.5:122b` | MLX | computed from the Qwen3.5-122B-A10B config, scaled by the measured MLX row, + 8 GiB | 5.0: the 35B-A3B's, x 3/10 active parameters |
Sizes are what the registry serves a Mac for each tag (every layer plus the config), read on
2026-11-08 (the Muse Glimmer row on 2026-11-10). Only one row was ever run on
2026-11-08 (the Ornith 1.0 and Muse Glimmer rows on 2026-11-10). Only one row was ever run on
a Mac, so every other answer in the table above is a calculation. For a llama.cpp build the check after the download catches a load that was
judged too kindly, but not what grows after it in host memory (the checkpoints and the prompt
cache, which are counted, not checked); for an MLX build it catches neither (see "The check"