Commit Graph
9 Commits
Author SHA1 Message Date
Codeman maintainer 3af94df4f6 feat(local-model): install a model from the server, with a job that can pause
POST /api/local-model/install, cancel, resume and runtime, and DELETE
/api/local-model. The GET gains an `install` object the window polls. All of
it is experimental and admin only in multi-user mode.

One job of five steps (runtime, model, verify, agent, connect) driven by a
pure reducer over explicit state, step and transition tables. One job at a
time (409). The consent is bound to the size shown: `acknowledgedBytes` must
be the server's own total (409 size-changed). A job a server restart
interrupted comes back paused and never continues without a click. Every
failure is a code with one next step.

The model is judged against the budget the runtime measured before it is
downloaded. A missing coding agent is installed with its stock command
through the shared runner, audited. Connect writes one endpoint record under
the reserved id through mutateCustomModelHosts() and switches
customModelEndpointsEnabled on through mutateSettingsFile(), remembering the
flag's value. The model's digest is stored and read again at boot and before
every session start on that endpoint; a changed model refuses the launch.

Remove answers 409 with the session names while a live session uses the
endpoint, undoes the two writes first, restores the flag only when this
feature switched it on and no other endpoint is saved, and deletes only
inside local-model/. The coding agent stays.

The server starts the runtime at boot when a model is installed and stops
it at shutdown beside stopDeepSeekWeb().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 2af413aa9b feat(local-model): the pinned runtime, its supervisor and the runtime client
runtime.ts: the runtime is pinned in a compiled constant (version, URL, size,
sha256). The archive is streamed to a .partial file while it is hashed,
continued with a Range request, retried at most 3 times, and the hash is
compared before anything is unpacked; a mismatch deletes the file. It is
unpacked with /usr/bin/tar into a temporary folder and renamed.

The supervisor runs the runtime as a non-detached child with an environment
built from an allowlist by the pure runtimeEnv(), on a port chosen once by
binding. It restarts the child at most 3 times, stops it with SIGTERM, adopts
or replaces its own orphan on the port and answers port-in-use for any other
listener. parseInferenceComputeTotal() reads the device total the runtime
logs when it starts. Under vitest nothing spawns or fetches.

ollama-client.ts: the pull is parsed incrementally by a pure parser (a 23 GB
pull is about 1.4 million lines), with up to 5 retries on an error line or a
cut stream. verifyModel() loads the model with one tiny chat, checks what the
runtime reports (all on the GPU, the chosen context) and makes one tool
probe; each failure is a code.

developer-tools.ts: only /usr/bin/xcode-select -p, bounded, darwin only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 492df776d0 feat(local-model): judge the fit on the measured peak and rank a tested Mac first
The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:37 +01:00
Codeman maintainer 84c1017487 feat(local-model): the launch refusal hook, and an honest name for a rebuilt log line
- `LocalModelService.launchRefusal(endpointId)`: what a session launch asks
  before it starts on the managed endpoint. It answers the error that
  refuses the launch (the runtime child is not alive, another program holds
  the port, the runtime keeps stopping, the model's digest changed) or
  nothing. New error code `runtime-not-running` with its sentence, for a
  runtime the person stopped: only a refused session start says it.
- The fake runtime's MacBook log line is rebuilt from the two values the
  measurement notes quote, so it is `INFERENCE_COMPUTE_LINE_MACBOOK` now.
  Only the Mac mini line is a verbatim capture and keeps `REAL_`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 20:23:58 +01:00
Codeman maintainer 771072c36b feat(local-model): contract for the install stage (types, stubs, fake runtime)
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 20:05:15 +01:00
Codeman maintainer 8281045cf2 fix(local-model): say what is true of the Mac, and why a model is missing
Review fixes for the read-only recommendation and its card.

Recommendation (pure):
- `limited` is now a statement about the Mac: no model with 12 billion
  parameters or more fits. It is judged on everything that fits, before
  the disk filter and whatever the pick is. A 32 or 36 GB Mac whose pick
  is a 9B model no longer reads "This Mac can run a small model" above a
  list of 20B, 27B and 35B models, and knowing the chip no longer lowers
  the verdict. A small pick under `ok` carries a note that says so.
- A model that fits the memory but has no room on the disk is no longer
  dropped in silence: the best-ranked one is named in a note with the GB
  that are missing. A full disk cannot turn a 48 GB Mac into a
  small-model Mac.
- "Models fit, but not at a context a harness needs" has its own
  sentence with the budget and the need, worded as an estimate. The
  16 GB Mac mini misses by 1.3 points (10.3 of 10.4 GB) and was told
  flatly that its memory is too little, the sentence of an 8 GB Mac.
- The installed memory is named by the size the Mac is sold as (48, not
  51.5): the hardware line and the memory sentence. Every computed size
  stays decimal GB from the one formatter. An unknown memory size is no
  longer reported as "0.0 GB".
- Slow is graded on the unrounded speed: 9.95 to 9.9996 tokens per
  second printed as 10 and read ok.
- The disk sentence names the 10 GiB that must stay free, so its
  numbers add up for the reader.

Card and Settings group:
- The Settings group "Model on this Mac" is hidden on a server that is
  not a Mac, and until the first answer says it is one.
- Only `403 FORBIDDEN` is a refusal. `403 PASSWORD_CHANGE_REQUIRED`
  (an admin who must change the password) is a failed request and is
  asked again, so that admin no longer loses the card until a reload.
- Each fit and speed label carries what it measures as its tooltip and
  accessible name, an alternative with a speed estimate prints it, and
  the count beside "Other models that fit" is gone (it was the number
  of rows, capped at four, not the number of models that fit).
- The line saying nothing can be set up yet sits right under the pick,
  in the colour of every other line (it was 3.9:1), and says whose step
  it is.
- The dismissal key is named for this stage, so hiding the read-only
  card does not hide the install offer later.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 17:37:53 +01:00
Codeman maintainer 34e95ceb62 feat(local-model): GET /api/local-model and the harness facts
The read-only route behind the recommendation, and the one module that reads
the CLI registry for it.

- harness-facts.ts: one fact per catalogue harness row, resolved in the CLI
  registry at call time: the registry's label, `installed` from the existing
  availability probe (asked once per build), `installable` for a stock CLI
  whose install command for this platform needs no administrator rights, and
  `acceptsModelId`, the test a custom model launch applies to the model id.
  A row whose CLI is unknown, disabled or has no custom model mechanism gets
  no fact. No CLI id is compared with a literal.
- routes/local-model-routes.ts: answers { hardware, recommendation,
  catalogUpdated } in the usual envelope and reformats nothing. Admin only in
  multi-user mode (403), checked before anything is probed. A failed probe is
  an answer (200, the hardware's `source` says what was read); a 500 is left
  for a fault in the catalogue, the harness facts or recommend(). It writes no
  file and reads no setting.

Tests: test/local-model-harness-facts.test.ts,
test/routes/local-model-routes.test.ts (app.inject(), no port; the temp HOME
is byte-identical before and after the requests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:27:49 +01:00
Codeman maintainer 92c20fe572 feat(local-model): hardware probe, model catalogue and the pure recommendation
The core of the read-only recommendation: what this Mac is, what the catalogue
knows, and which model fits. Nothing is downloaded, installed or written.

- hardware.ts: one bounded async spawn of /usr/sbin/sysctl with seven keys,
  without -n, parsed as "key: value" lines by the pure parseSysctl(). Free disk
  of the data dir through statfs behind the bounded path probe. Fallbacks from
  node:os, a 60 s cache, never throws, never system_profiler. A wired limit of
  0 means "not set".
- local-model-catalog.ts: seven models, three harness rows (pi gated) and the
  chips table as a compiled constant, validated by a Zod schema on first use.
  No override file. Model ids are name:tag in lowercase without a slash; every
  model carries paramsB and an (empty) tested list. Chip names match on the
  end of the chip string, so 'M4' is not the chip of 'Apple M4 Pro'.
- recommend.ts: device budget (the wired limit when above 0, else 2/3 of
  memory below 36 GB and 3/4 from 36 GB, less the reserve), need = weights +
  KV cache + 1 GiB + prompt cache, fit good up to 0.85 and tight up to 0.98,
  speed ok, unknown or slow (a chip in no chips row is unknown), and the
  ranking. Verdicts ok, limited (best model under 12 billion parameters) and
  none with its reason. "Recommended" only for a pick with a tested entry for
  this Mac, which no row has today.
- types/local-model.ts: comments brought in line with the code (chip matching,
  budgetShare rounding, a budget of 0 on an Intel Mac).
- cli-registry-no-id-branching.test.ts: the guard also catches a comparison of
  a `cli` field with a CLI id (plus one stray blank line Prettier refused).

Tests: test/local-model-format.test.ts, test/local-model-hardware.test.ts
(the captured output of two real Macs), test/local-model-catalog.test.ts,
test/local-model-recommend.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:27:33 +01:00
Codeman maintainer 73ce6a1819 feat(local-model): types and module skeleton for the hardware-aware recommendation
The contract three builders code against in parallel. Nothing is recommended
yet: the route answers 501 and the catalogue is empty.

- src/types/local-model.ts: HardwareFacts, the catalogue rows, HarnessFact,
  Candidate, Recommendation, LocalModelStatus, and LOCAL_MODEL_LABELS, the one
  list of sentences the server may put into the payload (the zh-CN dictionary
  is keyed by them).
- src/local-model/format.ts: formatGb, the one place a byte count becomes a
  size a person reads (decimal GB), and memClassGb.
- src/config/local-model-catalog.ts: the Zod schema (model ids are name:tag,
  lowercase, no slash) and an empty stock catalogue. No override file.
- Skeletons with their final signatures: hardware.ts, recommend.ts,
  harness-facts.ts, routes/local-model-routes.ts (injectable dependencies),
  public/local-model-ui.js with its script tag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 14:56:13 +01:00