Commit Graph
2 Commits
Author SHA1 Message Date
Codeman maintainer 492df776d0 feat(local-model): judge the fit on the measured peak and rank a tested Mac first
The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:37 +01:00
Codeman maintainer 771072c36b feat(local-model): contract for the install stage (types, stubs, fake runtime)
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 20:05:15 +01:00