docs(local-model): the coding agents of a model, what each was measured with, and the agent install

- docs/local-models.md: the four agents with their first request, whether
  the window installs them and what was seen where (a real model, a
  scripted endpoint, Linux, not the pinned runtime); what differs for Pi
  and for DeepSeek Harness; installing one more agent beside a model.
- docs/api-reference.md: `POST /api/local-model/agent`, `install.harnesses`,
  the `install-agent` action, and `install.cli` as the default agent.
- docs/architecture-invariants.md, CLAUDE.md, src/web/CLAUDE.md: the
  rules: the agent is chosen per session, the agent install is not a
  job, a launcher is never installable, a readiness mark is a measured
  one, and the window's one builder for the agent chips.
- docs/custom-model-endpoints.md: Pi's context window.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-11 10:04:47 +01:00
1 parent 0442b4cdbf
commit 9a34797cd5
6 files changed
+79 -11

No files matched your search

+6 -4
View File
@@ -1716,7 +1716,7 @@ Only replies from `allowedUserIds`, in `guildId`, inside a thread the bridge cre
## Local model
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, where an install of one stands and which models are on the machine. Seven verbs install a model (a first one, or another one beside it), switch to a model that is already there, pause and continue a job, stop and start the model runtime, delete one model, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md).
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, where an install of one stands and which models are on the machine. Eight verbs install a model (a first one, or another one beside it), switch to a model that is already there, pause and continue a job, stop and start the model runtime, delete one model, install one more agent CLI beside an installed model, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md).
**Experimental.** These routes and their payloads are labeled experimental: the fields below, the states, the error codes and the sentences may change or go in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on their shape yet.
@@ -1732,6 +1732,7 @@ Every route is admin only in multi-user mode (`403 FORBIDDEN`, answered before a
| `POST` | `/api/local-model/runtime` | `{ action: "start" \| "stop", newPort? }` | `stop` stops the runtime and keeps it from starting at the next boot; `start` undoes that. `newPort: true` with `start` chooses another port and moves the saved endpoint to it. `409`: `job-running`, `wrong-state` (nothing is installed), `sessions-open` (`stop` or `newPort` while live sessions use the model: their agents keep calling the address they were started with). |
| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the downloaded models (all of them) only with `removeModel: true`. With nothing installed it is what deletes the models an earlier Remove kept. `409`: `job-running`, `sessions-open`. |
| `DELETE` | `/api/local-model/models` | `{ modelId }` | Deletes ONE model from the store, through the runtime. `modelId` is a model name (`name:tag`) of `install.library` whose `actions` list `delete`, never a path. `400 INVALID_INPUT` for a name that is not on this machine. `409`: `job-running`, `wrong-state` (nothing is installed), `model-active` (the model of the installation or of the current job), `runtime-not-running`. `500 OPERATION_FAILED` when the runtime did not delete it. |
| `POST` | `/api/local-model/agent` | `{ cli }` | Installs one more agent CLI beside an installed model and answers at once; the installer runs on in the server and `install.harnesses` says how it stands. `cli` is a row of `install.harnesses` whose `actions` list `install`; the server runs that row's `installCommand`, the release's own stock command, never text from the request. It is not a job: `install.state` stays `installed` whatever the installer does, sessions are not refused, and no live session is looked at. An agent that can already be started answers `200` with the view as it is. `400 INVALID_INPUT` for a CLI that is not a row of `install.harnesses`, or one that one command does not make startable (`setup-needed`). `409`: `job-running` (a job, or another agent install, is running), `wrong-state` (nothing is installed). |
Every verb answers `{ success: true, data: <install> }` with the `install` object (below) as it is after the verb. A refusal the feature names is a `409` whose envelope also carries the conflict:
@@ -1786,16 +1787,17 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or
- `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed. A switch (`POST .../use`) is the two steps `verify` and `connect`.
- `progress` (while a download runs, and as it stood on a paused or failed job): `completedBytes`, `totalBytes`, `completedGb`, `totalGb`, and while it moves `bytesPerSecond`, `mbPerSecond` (whole decimal megabytes per second as text) and `secondsLeft`. During the `model` step both byte figures come from the download's own lines, not from the catalogue.
- `error` (with `failed`; with `installed` only when something blocks a session): `{ code, message: { text, vars? }, next, nextModel?, detail? }`. `next` is the ONE action that helps: `retry` (`POST .../resume`), `other-port` (`POST .../resume` or `.../runtime` with `newPort: true`), `next-model` (choose again; `nextModel` is `{ modelId, modelLabel }` when the server knows one), `start` (`POST .../runtime`) or `remove` (`DELETE`). `detail` is raw text for the curious (an installer's last lines), not translated.
- `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. `port`: the runtime's loopback port, once chosen.
- `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. With `installed`, `cli` is the DEFAULT agent CLI, the one a client that names none starts: the agent of the installation while it can be started, else the first row of `harnesses` that can. `port`: the runtime's loopback port, once chosen.
- `running`: the runtime Codeman started is alive right now. A runtime the person stopped is `running: false` with no `error`.
- `loaded` (with `installed`): what the runtime reported when the job checked the loaded model: `context`, `memoryBytes`, `memoryGb`, `onGpu`.
- `endpointId` (with `installed`): the saved model endpoint a session is started on, with `cli`.
- `endpointId` (with `installed`): the saved model endpoint a session is started on, with `cli` or with any row of `harnesses` that offers `start-session`.
- `developerTools`: `present`, `missing` (the Mac has no Xcode or Command Line Tools, so macOS will offer to install them the first time an agent runs `git` or `python3`) or `unknown`.
- `runtime`: `{ version, installed, downloadBytes, downloadGb, unpackedGb }`. `downloadBytes` is `0` once the runtime is unpacked.
- `folder`: the absolute folder everything of this feature is kept in.
- `library[]`: the models on this machine, the model in use first. Empty with `idle` (no runtime is there to ask). Each row: `id` (the name the runtime lists, `name:tag`), `label`, `context?` (the context it was checked with), `sizeBytes?` and `sizeGb?` (its size on disk as the runtime reports it, else as recorded when it was checked), `active` (it is the model in use), `verified` (Codeman checked it), `status` and `actions[]`. `status` is `ready`, `changed` (checked, and the runtime now reports another digest under the name), `missing` (checked, and the runtime no longer lists it) or `unverified` (the runtime lists it and Codeman never checked it: a model an earlier Remove kept, the model of a job that has not passed its check, or one another program pulled through the runtime's port). `actions` is what the server accepts for THAT row right now, out of `use` and `delete`; `delete` needs the running runtime, and a name that is not a plain `name:tag` is listed with no action.
- `store?`: `{ bytes, gb }`, what the model store takes on disk (the files of its blob folder, so shared layers count once and an interrupted download is included). Present in every state once it was measured; cached for about thirty seconds.
- `actions[]`: exactly what the server accepts right now, out of `install` (with `installed`: add another model), `cancel`, `resume`, `start`, `stop`, `remove` (with `idle` only while `store` holds something), `use` and `delete-model` (only while a row of `library` offers them) and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session).
- `harnesses[]`: the agent CLIs a session on the model in use can be started with, the default one first, then by the catalogue's rank. The agent is chosen per session: a session is started with `mode` set to a row's `cli` and `customModel: { endpointId, modelId }`. Empty in every state but `installed`. Each row: `cli`, `cliLabel`, `default` (true on the row `install.cli` names), `status`, `installCommand?` (with `installable` and `installing`: the literal command `POST .../agent` runs), `slowFirstTurn?` (the first answer takes minutes), `readyMark?` (a literal without white space that the agent's composer draws once it takes input: what a script waits for on `GET /api/sessions/:id/wait-output` before it sends the first prompt; absent for an agent with no measured mark), `error?` (`{ message, detail? }`: the last install of this agent through `POST .../agent` failed; kept in memory only) and `actions[]` (`start-session`, `install`). `status` is `ready` (it can be started, the answer the Run menu gets), `installable` (not on this machine, and `POST .../agent` installs it), `installing` (that install is running; the model stays as it is) or `setup-needed` (not startable, and one command does not make it so: DeepSeek Harness without a terminal profile, or a command that needs an administrator; set it up from the Run menu or a terminal).
- `actions[]`: exactly what the server accepts right now, out of `install` (with `installed`: add another model), `cancel`, `resume`, `start`, `stop`, `remove` (with `idle` only while `store` holds something), `use` and `delete-model` (only while a row of `library` offers them), `install-agent` (only while a row of `harnesses` offers `install`) and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs, nothing blocks a session and, when `harnesses` is known, at least one agent CLI can be started).
Error codes (`install.error.code`), each with one next step:
+3 -1
View File
@@ -714,6 +714,8 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse
**The coding agent** is installed by `runInstallCommand()` of `src/cli-install-runner.ts`, the runner extracted from `cli-registry-routes.ts` (whose route keeps its behaviour and re-exports `installEnv`/`npmGlobalPrefixWritable` for its tests). ⚠️ The job runs only `stockInstallCommand(id)`, read from the COMPILED `STOCK_CLIS`, never the registry merged with `clis.json` (the route's own command still comes from the merged registry, behind its `cliManagementEnabled` gate): this path has no such gate, so an override file must not be able to change what it runs. `HarnessFact.installCommand` is that same string, which is what the consent screen prints. ⚠️ Only for a harness row of the catalogue, only after the consent click, admin only, audited (`cli_install` with `via: 'local-model'`). ⚠️ ONE install slot per CLI id (`tryBeginCliInstall()`), shared by the route and the job. ⚠️ After the installer, successful or not, the resolver caches are forgotten (`forgetResolvedCli()`) and the binary is looked for again: a pre-install miss is negative-cached for up to five minutes, and "Start a session" would otherwise answer "not installed". Still missing is `harness-failed`; found is success whatever the exit code. The runner takes an abort signal that kills the installer's process GROUP.
**The coding agent is chosen per session.** The launch check (`launchRefusalMessage()`) asks about the endpoint and the model, never about the agent, so any CLI with a custom model recipe could always be started on `local-ollama` from the Run menu. What the feature adds is the list: `install.harnesses` (`buildHarnessRows()`, pure) is every catalogue harness that is not gated, has a `HarnessFact`, works at the model's context and accepts its name, each `ready`, `installable`, `installing` or `setup-needed`. ⚠️ `install.cli` stays the DEFAULT agent for a client that names none (the 1.50.0 skill reads only that field): the agent of the installation while it can be started, else the first row that can, so it never names an agent that is gone while another is there. ⚠️ `start-session` is listed only while a row is `ready`, once the rows are known; rows that could not be built (the registry threw) are "unknown", never "none", and a session is still offered. ⚠️ A LAUNCHER (`discovery.launcherProfile`, DeepSeek Harness) is never `installable`: its stock command installs the launcher and leaves it with nothing to launch, and the profile it needs is third-party code behind the privileged grant of `POST /api/deepseek/install-profile`. The rule reads the entry's data, never its id, and such an agent is offered only once the Run menu can start it. ⚠️ `POST /api/local-model/agent` (`installAgent()`) is NOT a job and NOT an event of the reducer: a job would put a working model into `failed` on an `npm` error and refuse every session until it was cleared. It runs `installCodingAgent()`, the function the `agent` step runs (same slot, same compiled command, same audit line), in the background; the state file never hears of it, the outcome lives in memory (`agentInstall`, `agentFailures`), one installer at a time, and `shutdown()` aborts it. A harness row may carry `readyMark` (a literal the composer draws once it takes input, for a script's `wait-output`): only a MEASURED mark goes there, the registry's `composerReadyMark` stands in when the row has none, and Pi has none on purpose (its composer rules are painted before it takes Enter, measured on 1.5.2).
**The connection** (`connect.ts`): one record `{ id: 'local-ollama', label: 'This Mac', baseUrl, authStyle: 'bearer', models: [<id as /v1/models lists it>], defaultModelId, defaultContextLength: <the chosen context> }`, validated by `CustomModelHostSchema` and written through `mutateCustomModelHosts()`, then `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, with the flag's previous value kept in the state file (a second connect must hand the FIRST value back, or Remove could never switch the flag off again). ⚠️ `defaultContextLength` is load-bearing: the runtime's model listing states no context, so this field is what reaches the coding agent as its limit. ⚠️ The id is reserved: `POST /api/model-endpoints` refuses it, and a record with it that this feature did not write is never overwritten (`endpoint-id-taken`). ⚠️ While the state file records the connection, the record is MANAGED (`isManagedLocalEndpoint()`): `PUT`/`DELETE /api/model-endpoints/local-ollama` are refused, discover answers the stored list without a request, and the sweep skips it, so a model somebody pulled through the port never becomes launchable and the URL never leaves the supervised port. A hand-made record with that id (no state) stays an ordinary endpoint. `local-model.json` is not a config-backup file while the endpoint store and the settings are, so a restore can take the record away under an installed model: `boot()` puts the RECORD back when it is gone (`ensureLocalEndpoint()`, never replacing one that is there) and leaves the settings flag as it finds it. ⚠️ The runtime's port has NO authentication, and its CORS allowlist always admits `localhost` origins: any local process or localhost page can pull, create or delete models there. That is the reason for the three rules above and for the digest.
**The digest** `/api/tags` reported after the check is stored. ⚠️ Before a session starts on the managed endpoint (`LocalModelService.launchRefusalMessage()`, asked by BOTH launch paths in `session-routes.ts`: `POST /api/sessions/:id/custom-model` and the quick-start `customModel` branch, before anything is written or created) and at boot it is read again: a different digest, a model that is gone, a runtime Codeman did not start, or a model id other than the installed one refuses the launch with one sentence. The view shows what blocks a session as `error` while the state stays `installed`; it is found live and never stored. Boot recovery of an already running session (`_rebuildCustomModelEnv`) is not guarded: the pane exists and keeps its environment.
@@ -732,7 +734,7 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse
**The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it.
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event, the library rows), `test/local-model-install-service.test.ts` (the three library verbs and the two caches included), `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime: install A, add B, switch back, delete B), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event, the library rows), `test/local-model-install-service.test.ts` (the three library verbs, the agent install and the two caches included), `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime: install A, add B, switch back, delete B), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
### Approvals Inbox
+7
View File
@@ -578,6 +578,13 @@ automatically). Results:
- **Claude, opencode, Pi, Grok, OMP** — verified: a real "hello world" reply
came back through the endpoint.
Pi's model entry carries the endpoint's context length when one is known
(`contextWindow`, the full window, and `maxTokens`, a quarter of it and
8,192 at most). Without it Pi takes an unknown model for a 128,000-token
one (its footer reads `/128k`) and asks for 16,384 reply tokens, so on a
smaller window it never compacts before the server's real window is full.
Measured on pi 1.5.2: with both fields `pi --list-models` shows the window
and the request carries `max_tokens: 8192`.
- **Codex** — the config is structurally correct, and against a llama-swap
server that DOES answer `/v1/responses` (confirmed live: a plain,
no-tool-call chat turn returned a real reply), the picture is more
+61 -4
View File
@@ -210,6 +210,62 @@ This step does not need the `cliManagementEnabled` setting. That setting gates t
CLI management surface; this path can only ever run one vetted command for one catalogue
row.
### Which coding agents, and what each was measured with
The coding agent is chosen **per session**. An installed model lists every agent it can be
run with (`install.harnesses` of the GET), and the window shows them as chips under "Start a
session with": the agent the model was installed with is the default, and the choice is
remembered on the device. The launch check asks about the endpoint and the model, never about
the agent, so a session with one agent and a session with another run on the same model.
| Agent | First request (its fixed prompt) | Installed from the window | What was seen, and where |
| ---------------- | ------------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| OpenCode | about 8,100 tokens, 11 tools (v1.19.5; more with user-level skills) | yes | A tool round trip on a Mac with a real model (the catalogue's tested entry), and on a scripted endpoint for v1.19.5 |
| Claude Code | about 36,000 tokens, so the first answer takes minutes | yes | Shipped in 1.50.0; not measured again for this table |
| Pi | 2,096 tokens, 4 tools (v1.5.2) | yes | A complete tool round trip against a real model: tool call, tool result, final answer, the file on disk |
| DeepSeek Harness | about 7,650 tokens, 25 tools (0.1.1-rc.2) | **no** | A tool round trip on a scripted endpoint only. Against a real model the launch and the model name reached the runtime, and no turn finished |
How to read the last column: the runs of 2026-11-10 were made on **Linux**, through Codeman's
own custom model launch, against Ollama 0.13.0 with a 1.7B model on a CPU ("a real model") or
against a scripted endpoint that replays that runtime's stream of a tool call. A scripted run
proves the agent's side of the protocol, not what a model does with its prompt. None of them
ran on a Mac or on the pinned runtime. Every agent sent the `name:tag` model name unchanged.
Three things follow from how the agents differ:
- **Pi is told the model's window.** Without it Pi takes an unknown model for a
128,000-token one and never compacts before the real window is full. Codeman now writes the
endpoint's context length into the model list it gives Pi (the custom model feature as a
whole, not only this window). Two things remain: a custom model session gives Pi a home
directory of its own, so Pi fetches its two search tools again at every start (it needs the
network for that; `PI_OFFLINE=1` skips it), and Pi asks "Trust project
folder?" in every such session when a folder above the working directory holds agent
skills, because the answer is stored in the home it does not keep.
- **DeepSeek Harness is a launcher.** `dsh` starts nothing by itself: a session needs a
terminal profile, which is third-party code with its own consent in the Run menu. Its
install command installs the launcher only, so this window never installs it. It is listed
as "Needs setup" until the Run menu can start it, and as a normal agent from then on. It is
the one agent of the four besides Claude Code that reports the end of a turn. Its custom
endpoint recipe itself was verified with a real model on a llama.cpp server
([custom-model-endpoints.md](custom-model-endpoints.md)); what is missing is a finished turn
on this feature's runtime.
- **OpenCode and DeepSeek Harness send a second, small request** beside the first prompt (a
title, a side question). The runtime answers one request at a time, so it queues.
### Installing one more agent beside an installed model
An agent that is not on the Mac has "Not installed" on its chip. Selecting it shows the
literal install command, and the main button becomes "Install <agent>". That click is the
consent; `POST /api/local-model/agent` then runs the same stock command under the same rules
as the `agent` step above (catalogue row only, compiled command only, admin only, audited,
one installer at a time).
It is **not a job**. The state stays `installed` whatever the installer does: the model stays
loaded, sessions are not refused, and a session with another agent can be started while it
runs. A failed install shows the sentence and the installer's last lines under the chip, and
leaves everything else as it was. The outcome is kept in memory only, so a restart of the
server forgets a failure (the chip then simply says "Not installed" again).
### The connection
Two writes outside the local model folder, both undone by Remove:
@@ -244,7 +300,7 @@ Switching the setting on also starts that five-minute sweep for every other save
## Starting a session
"Start a session" starts the coding agent on the endpoint `local-ollama` through the
"Start a session" starts the selected coding agent on the endpoint `local-ollama` through the
existing custom model launch. Before a coding agent is STARTED on that endpoint the server
checks, in this order:
@@ -274,7 +330,7 @@ lets an agent in a Codeman session hand work to this model, so Claude in the clo
reviews while the model on this Mac does the bulk: summaries, classification, first drafts, and
files that should not leave the machine. `local_model` reads `GET /api/local-model` and says what
is installed or why nothing can run; `spawn_worker <name> local` starts the runtime when it was
stopped (as Start does), then starts the installed coding agent on `local-ollama` through the same
stopped (as Start does), then starts the default coding agent (`install.cli`) on `local-ollama` through the same
quick-start launch and the same checks as above; `sendmark` gives the worker a task and reads its
answer back from a file it writes in its own folder. The skill never installs a model and never
answers a confirmation, the runtime still answers one request at a time, and while a worker it
@@ -312,8 +368,9 @@ and measures the store at most every thirty, never on every poll.
the runtime is restarted with the context the model was checked with, the model is loaded
and checked again (the tiny chat, the GPU, the tool probe), its digest must still be the
stored one (else the switch fails with `model-changed` before the model is loaded), and
the endpoint record is rewritten. The coding agent is the one the model was checked with
when it is still installed, else the one of the installation.
the endpoint record is rewritten. The default coding agent becomes the one the model was
checked with when it is still installed, else the one of the installation, else any other
agent that is on the Mac and can run the model.
- **Delete** removes one model through the runtime's own delete, after one question in the
row. The request carries the model's name, never a path, and the server deletes only a
name of its own listing. The model in use cannot be deleted: switch first, or Remove. It