mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 10:39:32 +01:00
docs(local-model): the models on this Mac, add, switch and delete
The guide, the API reference, the invariants and the two CLAUDE files describe the library with one model in use: the state file's list, the three verbs and their refusals, the two cached listings of the view, and the window's list with its row actions. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed
1 parent
70dc42b248
commit
2f17b8067a
6 files changed
+114
-25
No files matched your search
+17
-10
@@ -1716,7 +1716,7 @@ Only replies from `allowedUserIds`, in `guildId`, inside a thread the bridge cre
|
||||
|
||||
## Local model
|
||||
|
||||
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, and where an install of one stands. Five verbs install a model, pause and continue the install, stop and start the model runtime, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md).
|
||||
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, where an install of one stands and which models are on the machine. Seven verbs install a model (a first one, or another one beside it), switch to a model that is already there, pause and continue a job, stop and start the model runtime, delete one model, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md).
|
||||
|
||||
**Experimental.** These routes and their payloads are labeled experimental: the fields below, the states, the error codes and the sentences may change or go in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on their shape yet.
|
||||
|
||||
@@ -1725,11 +1725,13 @@ Every route is admin only in multi-user mode (`403 FORBIDDEN`, answered before a
|
||||
| Method | Path | Body | Notes |
|
||||
| -------- | -------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated, install }`. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. |
|
||||
| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed`, `size-changed`. |
|
||||
| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. With a model installed it adds another one: the earlier model stays on disk, and the new one is the model in use when the job ends. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed` (this model at this context is the one in use), `size-changed`, `sessions-open`. |
|
||||
| `POST` | `/api/local-model/use` | `{ modelId }` | Switches to a model of `install.library` whose `actions` list `use`. A job of the two steps `verify` and `connect`; nothing is downloaded. `400 INVALID_INPUT` when the model is not a checked model of the library, or no coding agent for it is installed. `409`: `job-running`, `wrong-state` (nothing is installed), `already-installed`, `sessions-open`. |
|
||||
| `POST` | `/api/local-model/cancel` | none | Interrupts the step in flight; the job is `paused` at that step. `409 wrong-state` when no job is running. |
|
||||
| `POST` | `/api/local-model/resume` | `{ newPort? }` (may be absent) | Continues a `paused` or `failed` job at the same step. `newPort: true` chooses another port first. `409`: `job-running`, `wrong-state`, `sessions-open` (`newPort` while live sessions use the model). |
|
||||
| `POST` | `/api/local-model/runtime` | `{ action: "start" \| "stop", newPort? }` | `stop` stops the runtime and keeps it from starting at the next boot; `start` undoes that. `newPort: true` with `start` chooses another port and moves the saved endpoint to it. `409`: `job-running`, `wrong-state` (nothing is installed), `sessions-open` (`stop` or `newPort` while live sessions use the model: their agents keep calling the address they were started with). |
|
||||
| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the model only with `removeModel: true`. `409`: `job-running`, `sessions-open`. |
|
||||
| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the downloaded models (all of them) only with `removeModel: true`. With nothing installed it is what deletes the models an earlier Remove kept. `409`: `job-running`, `sessions-open`. |
|
||||
| `DELETE` | `/api/local-model/models` | `{ modelId }` | Deletes ONE model from the store, through the runtime. `modelId` is a model name (`name:tag`) of `install.library` whose `actions` list `delete`, never a path. `400 INVALID_INPUT` for a name that is not on this machine. `409`: `job-running`, `wrong-state` (nothing is installed), `model-active` (the model of the installation or of the current job), `runtime-not-running`. `500 OPERATION_FAILED` when the runtime did not delete it. |
|
||||
|
||||
Every verb answers `{ success: true, data: <install> }` with the `install` object (below) as it is after the verb. A refusal the feature names is a `409` whose envelope also carries the conflict:
|
||||
|
||||
@@ -1747,9 +1749,9 @@ Every verb answers `{ success: true, data: <install> }` with the `install` objec
|
||||
}
|
||||
```
|
||||
|
||||
`data.reason` is `job-running`, `already-installed` (remove the installed model first), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model) or `wrong-state`. A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix.
|
||||
`data.reason` is `job-running`, `already-installed` (the model is the one in use already), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model), `wrong-state`, `model-active` (a delete of the model in use: switch first, or remove everything) or `runtime-not-running` (a delete while the runtime is down: start it first). A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix.
|
||||
|
||||
`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. Every `400` the install names, and the `507 OPERATION_FAILED` of a first install on a full disk, carries its sentence as `data.message` (`{ text, vars }`, like a `409`), so a client can print it in its own language.
|
||||
`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. Every `400` the feature names, the `507 OPERATION_FAILED` of a first install on a full disk and the `500 OPERATION_FAILED` of a delete the runtime refused carry their sentence as `data.message` (`{ text, vars }`, like a `409`), so a client can print it in its own language.
|
||||
|
||||
**Consent is bound to the size shown.** `acknowledgedBytes` must be the `installBytes` of the chosen row: the model download plus the runtime download while the runtime is not unpacked yet. The server compares it with its own total before anything starts and answers `409 size-changed` when they differ, so a click confirms exactly the size the person saw.
|
||||
|
||||
@@ -1775,12 +1777,13 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or
|
||||
- `tag?`: `{ kind, label }`, the one tag of the row. `kind` is `tested`, `best`, `tight`, `slow` or `small`;
|
||||
- `fit`: `good` (up to 0.85 of the budget) or `tight` (up to 0.98), with its label `fitLabel`;
|
||||
- `speed`: `ok`, `slow` (under 10 tokens per second) or `unknown`, with `speedLabel` (absent for `ok`) and `tokensPerSecond` (rounded to one decimal, absent for `unknown`; `speed` is decided on the unrounded figure, so `10` can be `slow`). `unknown` means the chip is in no row of the catalogue's chip table, or the catalogue has no speed figure for the model: a speed is never guessed;
|
||||
- `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run.
|
||||
- `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run;
|
||||
- `library?`: the model at this context is a checked model on this machine already (see `install.library`). `active`: it is the model in use. `ready`: `POST /api/local-model/use` switches to it, and nothing is downloaded. Absent otherwise.
|
||||
- `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `estimated` (computed from the installed memory: all there is before the runtime ran), `wired-limit` (taken from `iogpu.wired_limit_mb`) or `measured` (the device total the runtime itself printed when it started; the GET uses it from the first install on).
|
||||
- `catalogUpdated`: ISO date on which the catalogue's rows were last checked.
|
||||
- `install`: where the install stands and what can be done about it.
|
||||
- `state`: `idle` (nothing installed, no job), `installing`, `paused` (Cancel or a server restart stopped the job; it never continues without a click), `failed` or `installed`.
|
||||
- `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed.
|
||||
- `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed. A switch (`POST .../use`) is the two steps `verify` and `connect`.
|
||||
- `progress` (while a download runs, and as it stood on a paused or failed job): `completedBytes`, `totalBytes`, `completedGb`, `totalGb`, and while it moves `bytesPerSecond`, `mbPerSecond` (whole decimal megabytes per second as text) and `secondsLeft`. During the `model` step both byte figures come from the download's own lines, not from the catalogue.
|
||||
- `error` (with `failed`; with `installed` only when something blocks a session): `{ code, message: { text, vars? }, next, nextModel?, detail? }`. `next` is the ONE action that helps: `retry` (`POST .../resume`), `other-port` (`POST .../resume` or `.../runtime` with `newPort: true`), `next-model` (choose again; `nextModel` is `{ modelId, modelLabel }` when the server knows one), `start` (`POST .../runtime`) or `remove` (`DELETE`). `detail` is raw text for the curious (an installer's last lines), not translated.
|
||||
- `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. `port`: the runtime's loopback port, once chosen.
|
||||
@@ -1790,7 +1793,9 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or
|
||||
- `developerTools`: `present`, `missing` (the Mac has no Xcode or Command Line Tools, so macOS will offer to install them the first time an agent runs `git` or `python3`) or `unknown`.
|
||||
- `runtime`: `{ version, installed, downloadBytes, downloadGb, unpackedGb }`. `downloadBytes` is `0` once the runtime is unpacked.
|
||||
- `folder`: the absolute folder everything of this feature is kept in.
|
||||
- `actions[]`: exactly what the server accepts right now, out of `install`, `cancel`, `resume`, `start`, `stop`, `remove` and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session).
|
||||
- `library[]`: the models on this machine, the model in use first. Empty with `idle` (no runtime is there to ask). Each row: `id` (the name the runtime lists, `name:tag`), `label`, `context?` (the context it was checked with), `sizeBytes?` and `sizeGb?` (its size on disk as the runtime reports it, else as recorded when it was checked), `active` (it is the model in use), `verified` (Codeman checked it), `status` and `actions[]`. `status` is `ready`, `changed` (checked, and the runtime now reports another digest under the name), `missing` (checked, and the runtime no longer lists it) or `unverified` (the runtime lists it and Codeman never checked it: a model an earlier Remove kept, the model of a job that has not passed its check, or one another program pulled through the runtime's port). `actions` is what the server accepts for THAT row right now, out of `use` and `delete`; `delete` needs the running runtime, and a name that is not a plain `name:tag` is listed with no action.
|
||||
- `store?`: `{ bytes, gb }`, what the model store takes on disk (the files of its blob folder, so shared layers count once and an interrupted download is included). Present in every state once it was measured; cached for about thirty seconds.
|
||||
- `actions[]`: exactly what the server accepts right now, out of `install` (with `installed`: add another model), `cancel`, `resume`, `start`, `stop`, `remove` (with `idle` only while `store` holds something), `use` and `delete-model` (only while a row of `library` offers them) and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session).
|
||||
|
||||
Error codes (`install.error.code`), each with one next step:
|
||||
|
||||
@@ -1800,9 +1805,11 @@ Error codes (`install.error.code`), each with one next step:
|
||||
| `start` | `runtime-stopped` (the runtime crashed three times in a row and is no longer restarted) |
|
||||
| `other-port` | `port-in-use` |
|
||||
| `next-model` | `model-not-found`, `does-not-fit`, `no-tool-call` |
|
||||
| `remove` | `model-changed` (the model's digest is no longer the one stored at install time; a session on it is refused) |
|
||||
| `remove` | `model-changed` (the model's digest is no longer the one stored when it was checked; a session on it is refused, and so is a switch to it) |
|
||||
|
||||
Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, when the model's digest changed, or for a model other than the installed one.
|
||||
Two codes are only ever the reason of a refused session start, never on `install.error`: `runtime-not-running` (the person stopped the runtime) and `job-unfinished` (a job that changes the model in use is running or stopped: nothing is started until it ended `installed` again).
|
||||
|
||||
Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, while a job that changes the model in use is running or stopped, when the model's digest changed, or for a model other than the one in use.
|
||||
|
||||
Every computed size a person reads travels twice: as bytes, and as decimal GB text without the unit (`"needGb": "29.6"`). Show the text and add the unit; do not divide the bytes yourself, so one request never shows two different roundings. The installed memory is the exception named above: show `memClassGb`.
|
||||
|
||||
|
||||
@@ -720,11 +720,19 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse
|
||||
|
||||
**Remove** stops the child, undoes the connection FIRST (a refused write of the endpoint store or of the settings rejects before any file is deleted), restores the flag only when this feature switched it on AND no other endpoint is saved, then deletes the runtime and the private home and, when asked, the models (`removeLocalModelFiles()`, which refuses any path outside the root). ⚠️ While a live session uses the endpoint the answer is 409 `sessions-open` with the session names (`sessionNamesUsingEndpoint()`, read at the moment of the request). ⚠️ The coding agent stays: what its installer wrote is not ours to remove.
|
||||
|
||||
**The library** (one model in use, several on disk). `LocalModelStateFile.library` lists every model that PASSED the check through Codeman: `{ id, label, context, digest, bytes?, cli?, verifiedAt? }`, no path. The service upserts the entry as a fact when `verify` passes (where it stores the digest); the reducer only carries the list, through every event but `remove`. An `installed` file an earlier build wrote has no list: `libraryOf()` then counts its one model, and an install or a switch from it writes that model into the list before the state's `model` member becomes the new job's. ⚠️ ONE model is in use: the endpoint record still lists exactly one model, the runtime still holds one (`OLLAMA_MAX_LOADED_MODELS=1`), and `launchRefusalMessage()` still starts only the state's model with its stored digest. Never widen the record to "every checked model": the port has no login, and two sessions on two models would make the runtime swap 20 GB on every request.
|
||||
|
||||
⚠️ Three verbs, each a row of the tables and never a special case. **Add**: `install` is also listed for `installed` in `INSTALL_TRANSITIONS`; the job is the same five steps, and the earlier model stays (the `replacedModelId` rule, `replacedModelFor()`, never names a model of the library: only an unchecked or partial download is deleted as a leftover). **Switch**: the event `use` (`installed`, `paused`, `failed`) starts a job of `USE_JOB_STEPS` (`verify`, `connect`) for a library entry, carrying its stored digest in `model.digest`; `stepVerify` compares that digest BEFORE it loads anything and again after the check, and a different or missing one fails with `model-changed`. A switch has no `model` step, so the leftovers of the job it replaced are deleted at the start of `verify` (`dropReplacedModel()`). Its coding agent is the entry's own when that is installed now, else the state's (`recheckCliInstalled()`), else a 400: a job that stopped while it installed another agent leaves the state naming one that is not there. **Delete** (`deleteLibraryModel()`) moves no state, so it is not an event; whether a state allows it is the `delete-model` action of its `INSTALL_STATE_TABLE` row. It takes a model NAME (the request schema and the service both hold it to `LOCAL_MODEL_ID_PATTERN`), requires that name in the server's own listing (the library or the runtime's tags), goes through the runtime's `/api/delete` only (never a file delete, never a path), and trusts only the runtime's answer that the model is gone, never an empty listing. ⚠️ It refuses the model of the installation or of the current job (`model-active`), a running job, and a runtime that is down or does not answer (`runtime-not-running`).
|
||||
|
||||
⚠️ `install` from `installed` and `use` are refused with 409 `sessions-open` while live sessions use the endpoint, like Stop: both change which model the runtime holds and often restart it for another context. ⚠️ While a job that started from `installed` is unfinished (`installing`, `paused`, `failed`) the launch check refuses every start with `job-unfinished` (a code only a refused start says, like `runtime-not-running`), because it wants the state `installed`; the view then offers `use` on the other checked models, which is the way back. A switch is never offered for the model of the stopped job itself: Resume goes on with that one.
|
||||
|
||||
**The view's two listings.** `install.library` (`buildLibraryRows()`, pure) joins the state's entries with what the RUNNING runtime lists (`/api/tags`): `ready`, `changed` (another digest under the name), `missing` (no longer listed) for a checked model, `unverified` for anything else the runtime lists (a model an earlier Remove kept, the model of a job that has not passed its check, one another program pulled through the unauthenticated port), which can be deleted and never switched to. A name that is not a plain `name:tag` is listed with no action. `install.store` is `measureModelStore()`: the sizes of the regular files directly in `models/blobs`, nothing followed, nothing outside the root. ⚠️ Neither is asked for on the one-second poll: `tagsCache` (5 s) and `storeCache` (30 s) are refreshed in the background, only a figure that was never read is waited for (300 ms at most), a runtime that does not answer keeps the last list (or every checked model would read as `missing` for a moment), and every verb and finished step that changes the store throws both away (`forgetStore()`). ⚠️ In `idle` there is no runtime to ask, so the list is empty and only `store` says that an earlier Remove kept models; `remove` is listed there only while it holds something, and `DELETE /api/local-model { removeModel: true }` is what frees it. `Candidate.library` (`active`, `ready`) is added by the ROUTE (`markLibraryCandidates()`), never by `recommend()`, which stays pure and knows nothing of what is installed.
|
||||
|
||||
**Developer tools**: `checkDeveloperTools()` runs ONLY `/usr/bin/xcode-select -p`, bounded, on darwin. ⚠️ Never `git`, `python3` or `xcrun` from the server: on a Mac without the tools each opens a system window. The view carries `developerTools`, and the window prints the one sentence when it is `missing`.
|
||||
|
||||
**The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it.
|
||||
|
||||
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event), `test/local-model-install-service.test.ts`, `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
|
||||
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event, the library rows), `test/local-model-install-service.test.ts` (the three library verbs and the two caches included), `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime: install A, add B, switch back, delete B), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
|
||||
|
||||
### Approvals Inbox
|
||||
|
||||
|
||||
+80
-11
@@ -4,7 +4,9 @@ Codeman can set up a coding model that runs on the Mac the server runs on: no ac
|
||||
no API key, and the session's prompts and files go to a model on this Mac rather than a
|
||||
cloud provider (the coding agent itself may still check for updates). One button on the welcome screen opens a window that
|
||||
shows what fits this Mac, installs the choice and ends on a coding agent session that
|
||||
runs on it.
|
||||
runs on it. The same window then lists the models that are on this Mac with what each takes
|
||||
on disk, adds another one, switches between them and deletes one
|
||||
([The models on this Mac](#the-models-on-this-mac)).
|
||||
|
||||
**Experimental.** The routes and their payloads may change in any release
|
||||
([versioning-policy.md](versioning-policy.md), "Experimental local model routes"). Apple
|
||||
@@ -103,7 +105,9 @@ for a row it does not know. Its answer for each memory size is then pinned in
|
||||
## What an install does, in order
|
||||
|
||||
The person picks a model, presses "Install on this Mac" and confirms the sizes once. The
|
||||
server then runs one job of five steps. The window polls `GET /api/local-model` once a
|
||||
server then runs one job of five steps. The same job adds a second model to an installed one
|
||||
("Add a model"): the runtime is there already, so it downloads the model alone, and the
|
||||
earlier model stays on disk. The window polls `GET /api/local-model` once a
|
||||
second; the job runs in the server, so the window can be closed.
|
||||
|
||||
| Step | What it does |
|
||||
@@ -244,10 +248,13 @@ Switching the setting on also starts that five-minute sweep for every other save
|
||||
existing custom model launch. Before a coding agent is STARTED on that endpoint the server
|
||||
checks, in this order:
|
||||
|
||||
- no job that changes the model in use is running or stopped: until an install or a switch
|
||||
ended, nothing is started ("The model on this Mac is being changed and is not ready");
|
||||
- the runtime child Codeman started is alive (whatever else answers on the port is not
|
||||
ours);
|
||||
- the digest the runtime reports for the installed model is the one stored at install time;
|
||||
- the model asked for is the one that was installed and checked.
|
||||
- the digest the runtime reports for the model in use is the one stored when it was checked;
|
||||
- the model asked for is the model in use. Another model that is on this Mac is not started
|
||||
on the endpoint: switch to it in the window first.
|
||||
|
||||
That check runs wherever a new agent process comes to be on the endpoint: a session created
|
||||
with the model, a model applied to a running session (which restarts its agent), the attach
|
||||
@@ -273,22 +280,75 @@ answer back from a file it writes in its own folder. The skill never installs a
|
||||
answers a confirmation, the runtime still answers one request at a time, and while a worker it
|
||||
opened is live, Stop and Remove are refused until it is closed.
|
||||
|
||||
## The models on this Mac
|
||||
|
||||
The model store can hold several models. **One of them is the model in use**: the one the
|
||||
endpoint record `local-ollama` lists, the runtime is started for and a session can be
|
||||
started on. The runtime holds one model in memory at a time (`OLLAMA_MAX_LOADED_MODELS=1`),
|
||||
so this is what the Mac can really do, and it keeps every rule above: one record, one
|
||||
model, one stored digest per model.
|
||||
|
||||
The installed screen lists what is on this Mac, the model in use first:
|
||||
|
||||
| A row says | It means |
|
||||
| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| In use | The model sessions start on |
|
||||
| Ready | Codeman downloaded and checked it. **Use** switches to it |
|
||||
| Not checked | The runtime lists it and Codeman never checked it: a model an earlier Remove kept, or one another program pulled through the runtime's port. It can only be deleted |
|
||||
| Changed | It was checked, and the runtime now reports another digest under its name. It is never started; delete it and install it again |
|
||||
| Missing | It was checked, and the runtime no longer lists it. **Delete** drops the entry |
|
||||
|
||||
Each row shows the context the model was checked with and its size on disk, as the runtime
|
||||
reports it. Under the list is what all downloads take together: the size of the files in
|
||||
`local-model/models/blobs`, so layers two models share are counted once and an interrupted
|
||||
download is included. The window asks the runtime for the list at most every five seconds
|
||||
and measures the store at most every thirty, never on every poll.
|
||||
|
||||
- **Add a model** opens the chooser. A model that is not on this Mac is installed like the
|
||||
first one, with its sizes confirmed; when the job ends it is the model in use, and the
|
||||
earlier model stays in the list. A row that is on this Mac already says so and is switched
|
||||
to, without a download and without a confirm screen.
|
||||
- **Use** switches to a checked model. It is a job of two steps, `verify` and `connect`:
|
||||
the runtime is restarted with the context the model was checked with, the model is loaded
|
||||
and checked again (the tiny chat, the GPU, the tool probe), its digest must still be the
|
||||
stored one (else the switch fails with `model-changed` before the model is loaded), and
|
||||
the endpoint record is rewritten. The coding agent is the one the model was checked with
|
||||
when it is still installed, else the one of the installation.
|
||||
- **Delete** removes one model through the runtime's own delete, after one question in the
|
||||
row. The request carries the model's name, never a path, and the server deletes only a
|
||||
name of its own listing. The model in use cannot be deleted: switch first, or Remove. It
|
||||
needs the running runtime, so a stopped model shows no Delete until Start.
|
||||
|
||||
Adding a model and switching are refused while a session uses the model ("Sessions are
|
||||
still using this model"), like Stop: their coding agents were started with the other model's
|
||||
name, and the runtime would swap models on every request.
|
||||
|
||||
While a job that started from an installed model runs, or after it stopped (paused or
|
||||
failed), no session is started: the checks above want a finished installation, and the
|
||||
refusal says so (`job-unfinished`). The window
|
||||
then lists the checked models with **Use**, which is the way back to the earlier one. Going
|
||||
back deletes what the stopped job left of a model that never passed its check, the way a new
|
||||
install does; a checked model is never deleted that way.
|
||||
|
||||
## Stop, Start, Remove
|
||||
|
||||
- **Stop** stops the runtime and keeps it from starting at the next boot. **Start** undoes
|
||||
that.
|
||||
- **Remove** stops the runtime, removes the endpoint record, switches the setting off again
|
||||
only when this feature switched it on and no other endpoint is saved, and deletes the
|
||||
runtime and its private home. The downloaded model is deleted only when asked. The coding
|
||||
agent stays. While a live session uses the model, Remove answers with the session names
|
||||
instead.
|
||||
runtime and its private home. The downloaded models (all of them) are deleted only when
|
||||
asked. The coding agent stays. While a live session uses the model, Remove answers with
|
||||
the session names instead.
|
||||
- **Models a Remove kept** are not lost space: with nothing installed the window says how
|
||||
much they still take and offers "Delete them", and after a new install they show in the
|
||||
list as "Not checked", each with its own Delete.
|
||||
|
||||
## Where everything is
|
||||
|
||||
All under the data directory (`~/.codeman`, or the instance's own):
|
||||
|
||||
```
|
||||
local-model.json the state file (mode 0600)
|
||||
local-model.json the state file (mode 0600), with the list of checked models
|
||||
local-model/
|
||||
runtime/ollama-0.40.15.tgz.partial the runtime archive while it downloads
|
||||
runtime/ollama-0.40.15/ the unpacked runtime (about 579 MB)
|
||||
@@ -313,9 +373,12 @@ in App Settings.
|
||||
| ------------ | ------------------------------------------------------------- | ---------------- | -------------------------------------- |
|
||||
| `idle` | stays idle | refused | install |
|
||||
| `installing` | becomes `paused` at the same step | becomes `paused` | cancel |
|
||||
| `paused` | stays paused | refused | resume, install, remove |
|
||||
| `failed` | stays failed, same error | refused | resume, install, remove |
|
||||
| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, remove |
|
||||
| `paused` | stays paused | refused | resume, install, use, delete a model, remove |
|
||||
| `failed` | stays failed, same error | refused | resume, install, use, delete a model, remove |
|
||||
| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, install (add a model), use, delete a model, remove |
|
||||
|
||||
`use` and "delete a model" are listed only while a row of the list offers them, and in
|
||||
`idle` "remove" is offered only while the model store still holds something.
|
||||
|
||||
One job at a time. A job a server restart interrupted comes back paused and never continues
|
||||
without a click. Cancel interrupts the step in flight (the download, the pull, the request,
|
||||
@@ -357,6 +420,12 @@ or `python3`.
|
||||
on a Mac. "Recommended" appears only for the pair we ran, on the Mac we ran it on.
|
||||
- **The model stays in memory for ten minutes** after the last request, then the runtime
|
||||
unloads it.
|
||||
- **One model runs at a time.** Several can be on disk; the runtime holds one, and a switch
|
||||
restarts it and loads the other model (seconds to a minute). Close the sessions on the
|
||||
model first: the window refuses the switch while one is open.
|
||||
- **The disk figures are the runtime's.** A row's size is what the runtime reports for the
|
||||
model, and the total is the size of its blob files. Neither was compared with Finder on a
|
||||
Mac yet.
|
||||
|
||||
## Keeping the catalogue current
|
||||
|
||||
|
||||
@@ -234,6 +234,11 @@ Good to know:
|
||||
but it is changed and removed in the local model window, not here.
|
||||
- Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from
|
||||
starting next time.
|
||||
- More than one model can be on the Mac, and one of them runs at a time. Once a model is
|
||||
installed the window lists the models with what each takes on disk: **Add a model**
|
||||
downloads another one, **Use** switches to a model that is already there (nothing is
|
||||
downloaded; close the sessions on the model first), and **Delete** removes one, after one
|
||||
question. The model in use cannot be deleted: switch first, or use **Remove**.
|
||||
- **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The
|
||||
coding agent stays installed.
|
||||
- If the model on disk is no longer the one that was installed, Codeman refuses to start a
|
||||
|
||||
Reference in new issue
Block a user