mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 10:39:32 +01:00
Merge branch 'feat/local-model-recommendation' into test/desktop-macbook
# Conflicts: # config/test-suites.ts
This commit is contained in:
Codeman maintainer committed
commit
1e097bc368
74 files changed
+25355
-2118
No files matched your search
+68
-11
@@ -1676,15 +1676,44 @@ Only replies from `allowedUserIds`, in `guildId`, inside a thread the bridge cre
|
||||
|
||||
`GET /api/doctor[?category=core|office|other]` returns the `codeman doctor --json` report (`platform`, `summary`, `tools[]` with `status` `ok` \| `missing` \| `outdated` \| `skipped` \| `error`, `version`, `path`, `installHint`). The probe engine is synchronous, so it runs in a child process of the same entry script, never on the server's event loop (30 s timeout). It names install paths and versions, so it is admin only in multi-user mode (`403`). `400` for an unknown category, `500` if the child produces no report.
|
||||
|
||||
## Local model recommendation
|
||||
## Local model
|
||||
|
||||
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, and with which agent CLI. It covers Apple silicon macOS only; anywhere else the answer is still a `200`, with verdict `none`. **Read-only:** the route downloads nothing, installs nothing, writes no file and reads or changes no setting (setting a model up from Codeman is a later step and is not built yet). It names the host's hardware, so it is admin only in multi-user mode (`403`, answered before anything is probed). The models, their sizes and the agent CLIs it considers are a catalogue compiled into the release; there is no override file.
|
||||
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, and where an install of one stands. Five verbs install a model, pause and continue the install, stop and start the model runtime, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md).
|
||||
|
||||
**Experimental.** This endpoint and its payload are labeled experimental until the step that sets a model up ships: that step will add to and may change the fields below, the verdicts and the sentences, in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on its shape yet.
|
||||
**Experimental.** These routes and their payloads are labeled experimental: the fields below, the states, the error codes and the sentences may change or go in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on their shape yet.
|
||||
|
||||
| Method | Path | Body | Notes |
|
||||
| ------ | ------------------ | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated }`. `403 FORBIDDEN` for a non-admin in multi-user mode. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. |
|
||||
Every route is admin only in multi-user mode (`403 FORBIDDEN`, answered before anything is probed, parsed or started): the GET names the host's hardware, and the verbs download and run a program, install an agent CLI and write the server-wide settings. There is no SSE event: the window polls the GET once a second while it is open.
|
||||
|
||||
| Method | Path | Body | Notes |
|
||||
| -------- | -------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated, install }`. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. |
|
||||
| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed`, `size-changed`. |
|
||||
| `POST` | `/api/local-model/cancel` | none | Interrupts the step in flight; the job is `paused` at that step. `409 wrong-state` when no job is running. |
|
||||
| `POST` | `/api/local-model/resume` | `{ newPort? }` (may be absent) | Continues a `paused` or `failed` job at the same step. `newPort: true` chooses another port first. `409`: `job-running`, `wrong-state`, `sessions-open` (`newPort` while live sessions use the model). |
|
||||
| `POST` | `/api/local-model/runtime` | `{ action: "start" \| "stop", newPort? }` | `stop` stops the runtime and keeps it from starting at the next boot; `start` undoes that. `newPort: true` with `start` chooses another port and moves the saved endpoint to it. `409`: `job-running`, `wrong-state` (nothing is installed), `sessions-open` (`stop` or `newPort` while live sessions use the model: their agents keep calling the address they were started with). |
|
||||
| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the model only with `removeModel: true`. `409`: `job-running`, `sessions-open`. |
|
||||
|
||||
Every verb answers `{ success: true, data: <install> }` with the `install` object (below) as it is after the verb. A refusal the feature names is a `409` whose envelope also carries the conflict:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"error": "The download is 23.3 GB now. Check the sizes and confirm again.",
|
||||
"errorCode": "CONFLICT",
|
||||
"data": {
|
||||
"reason": "size-changed",
|
||||
"message": { "text": "The download is {size} GB now. Check the sizes and confirm again.", "vars": { "size": "23.3" } },
|
||||
"expectedBytes": 23291770627,
|
||||
"expectedGb": "23.3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`data.reason` is `job-running`, `already-installed` (remove the installed model first), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model) or `wrong-state`. A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix.
|
||||
|
||||
`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. Every `400` the install names, and the `507 OPERATION_FAILED` of a first install on a full disk, carries its sentence as `data.message` (`{ text, vars }`, like a `409`), so a client can print it in its own language.
|
||||
|
||||
**Consent is bound to the size shown.** `acknowledgedBytes` must be the `installBytes` of the chosen row: the model download plus the runtime download while the runtime is not unpacked yet. The server compares it with its own total before anything starts and answers `409 size-changed` when they differ, so a click confirms exactly the size the person saw.
|
||||
|
||||
A probe that fails is an answer, not an error: a server that is not on macOS, or a Mac whose `sysctl` call failed or timed out, still gets a `200`, and `hardware.source` says what was read.
|
||||
|
||||
@@ -1696,18 +1725,46 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or
|
||||
- `chip`, `model`, `macosVersion`, `macosMajor`, `wiredLimitMb` (`0` means not set), `freeDiskBytes` and `freeDiskGb` (free space on the filesystem of Codeman's data directory). Each is absent when it could not be read.
|
||||
- `source`: `sysctl` (the one bounded `sysctl` call answered), `os-fallback` (macOS, but that call failed: memory, architecture, the chip name and the free disk space come from Node, and nothing only `sysctl` knows, such as `arm64`, `translated`, `model`, `macosVersion`, `macosMajor` or `wiredLimitMb`, is present) or `not-probed` (not macOS, nothing was run).
|
||||
- `recommendation`:
|
||||
- `verdict`: `ok`, `limited` or `none`. `limited` is a statement about the Mac: no model with 12 billion parameters or more fits its memory, so what it can run handles small edits and questions and does not replace a cloud model. It is judged on every model that fits, before the free disk space is looked at and whatever the pick is. With `ok` the pick can still be a small model (a larger one may fit only tightly, have no speed figure or have no room on the disk); `notes` then says so.
|
||||
- `verdict`: `ok`, `limited` or `none`. `limited` is a statement about the Mac: no model with 12 billion parameters or more fits its memory, so what it can run handles small edits and questions and does not replace a cloud model. It is judged on every model that fits, before the free disk space is looked at and whatever the pick is. With `ok` the pick can still be a small model (a larger one may be slow, have no speed figure or have no room on the disk); `notes` then says so.
|
||||
- `reason`, present only with `none`: `not-macos`, `intel`, `memory`, `disk`, `macos` (too old) or `no-harness` (no agent CLI that can be connected to a local model is installed or installable). `memory` has three sentences: nothing fits at all (slot `memory`, the size the Mac is sold as), a model fits but only below the context length an agent CLI needs (slots `budget` and `need`, so a Mac that misses by a little reads as that), and the memory size could not be read (no slot).
|
||||
- `headline` and `notes[]`: `{ text, vars? }`. `text` is an English sentence that may hold `{placeholder}` slots, and `vars` carries the slot values as text. Notes, in order: the pick is a small model although larger ones fit (verdict `ok` only); a model that fits the memory was left out because its download would not leave 10 GiB free on the disk (slots `model` and `missing`: the best-ranked such model and the GB to free up); nobody tested the pick on a Mac like this one.
|
||||
- `pick` (absent with `none`) and `alternatives[]` (at most four, best first; empty with `none`), each one model at one context length with one agent CLI. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing):
|
||||
- `pick` (absent with `none`) and `alternatives[]` (at most four, best first; empty with `none`), each one model at one context length with one agent CLI. The order: a pair tested on a Mac like this one first, then speed (`ok` before `unknown` before `slow`), then the model's rank, the agent CLI's rank and the larger context. The memory fit decides whether a row is listed and what it says (`fit`, `tag`), it does not sort, so a model is offered at the largest context it fits at. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing):
|
||||
- `modelId` (`name:tag`), `modelLabel`, `license`, `paramsB` (parameters in billions), `context` (tokens);
|
||||
- `cli`, `cliLabel`, `cliInstalled` (`false`: the CLI is not on this machine yet but can be installed without administrator rights), `slowFirstTurn?`;
|
||||
- `needBytes` and `needGb` (memory the model needs at that context), `downloadBytes` and `downloadGb`, `budgetShare` (need divided by `budgetBytes`, rounded to three decimals);
|
||||
- `needBytes` and `needGb` (memory the model needs at that context, at its PEAK: the weights, what stays held per token of context, what the runner holds beside both, and what it takes for a moment while a long prompt is processed), `needSource` (`measured`: the figures were measured for this model on a Mac; `estimated`: they are another model's), `downloadBytes` and `downloadGb`, `budgetShare` (need divided by `budgetBytes`, rounded to three decimals);
|
||||
- `installBytes` and `installGb`: the whole download a click on Install starts for this row, `downloadBytes` plus the runtime archive while the runtime is not unpacked. It is the number to send as `acknowledgedBytes`;
|
||||
- `harnesses[]`: every agent CLI the model can be run with at this context, best first: `{ cli, cliLabel, installed, installCommand?, slowFirstTurn? }`. `installCommand` is the literal command the install would run for a CLI that is not installed (the release's own stock command), for the consent screen. The first entry is the one the flat `cli` fields name;
|
||||
- `tag?`: `{ kind, label }`, the one tag of the row. `kind` is `tested`, `best`, `tight`, `slow` or `small`;
|
||||
- `fit`: `good` (up to 0.85 of the budget) or `tight` (up to 0.98), with its label `fitLabel`;
|
||||
- `speed`: `ok`, `slow` (under 10 tokens per second) or `unknown`, with `speedLabel` (absent for `ok`) and `tokensPerSecond` (rounded to one decimal, absent for `unknown`; `speed` is decided on the unrounded figure, so `10` can be `slow`). `unknown` means the chip is in no row of the catalogue's chip table, or the catalogue has no speed figure for the model: a speed is never guessed;
|
||||
- `tested`: we ran this model with this CLI on a Mac like this one. No catalogue row has such an entry yet, so it is `false` today.
|
||||
- `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `wired-limit` (taken from `iogpu.wired_limit_mb`) or `estimated` (computed from the installed memory).
|
||||
- `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run.
|
||||
- `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `estimated` (computed from the installed memory: all there is before the runtime ran), `wired-limit` (taken from `iogpu.wired_limit_mb`) or `measured` (the device total the runtime itself printed when it started; the GET uses it from the first install on).
|
||||
- `catalogUpdated`: ISO date on which the catalogue's rows were last checked.
|
||||
- `install`: where the install stands and what can be done about it.
|
||||
- `state`: `idle` (nothing installed, no job), `installing`, `paused` (Cancel or a server restart stopped the job; it never continues without a click), `failed` or `installed`.
|
||||
- `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed.
|
||||
- `progress` (while a download runs, and as it stood on a paused or failed job): `completedBytes`, `totalBytes`, `completedGb`, `totalGb`, and while it moves `bytesPerSecond`, `mbPerSecond` (whole decimal megabytes per second as text) and `secondsLeft`. During the `model` step both byte figures come from the download's own lines, not from the catalogue.
|
||||
- `error` (with `failed`; with `installed` only when something blocks a session): `{ code, message: { text, vars? }, next, nextModel?, detail? }`. `next` is the ONE action that helps: `retry` (`POST .../resume`), `other-port` (`POST .../resume` or `.../runtime` with `newPort: true`), `next-model` (choose again; `nextModel` is `{ modelId, modelLabel }` when the server knows one), `start` (`POST .../runtime`) or `remove` (`DELETE`). `detail` is raw text for the curious (an installer's last lines), not translated.
|
||||
- `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. `port`: the runtime's loopback port, once chosen.
|
||||
- `running`: the runtime Codeman started is alive right now. A runtime the person stopped is `running: false` with no `error`.
|
||||
- `loaded` (with `installed`): what the runtime reported when the job checked the loaded model: `context`, `memoryBytes`, `memoryGb`, `onGpu`.
|
||||
- `endpointId` (with `installed`): the saved model endpoint a session is started on, with `cli`.
|
||||
- `developerTools`: `present`, `missing` (the Mac has no Xcode or Command Line Tools, so macOS will offer to install them the first time an agent runs `git` or `python3`) or `unknown`.
|
||||
- `runtime`: `{ version, installed, downloadBytes, downloadGb, unpackedGb }`. `downloadBytes` is `0` once the runtime is unpacked.
|
||||
- `folder`: the absolute folder everything of this feature is kept in.
|
||||
- `actions[]`: exactly what the server accepts right now, out of `install`, `cancel`, `resume`, `start`, `stop`, `remove` and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session).
|
||||
|
||||
Error codes (`install.error.code`), each with one next step:
|
||||
|
||||
| Next step | Codes |
|
||||
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `retry` | `runtime-download-failed`, `runtime-damaged`, `runtime-unpack-failed`, `runtime-start-failed`, `disk-full`, `pull-failed`, `context-not-applied`, `load-failed`, `probe-unfinished`, `harness-failed`, `endpoint-id-taken`, `connect-failed`, `internal` |
|
||||
| `start` | `runtime-stopped` (the runtime crashed three times in a row and is no longer restarted) |
|
||||
| `other-port` | `port-in-use` |
|
||||
| `next-model` | `model-not-found`, `does-not-fit`, `no-tool-call` |
|
||||
| `remove` | `model-changed` (the model's digest is no longer the one stored at install time; a session on it is refused) |
|
||||
|
||||
Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, when the model's digest changed, or for a model other than the installed one.
|
||||
|
||||
Every computed size a person reads travels twice: as bytes, and as decimal GB text without the unit (`"needGb": "29.6"`). Show the text and add the unit; do not divide the bytes yourself, so one request never shows two different roundings. The installed memory is the exception named above: show `memClassGb`.
|
||||
|
||||
|
||||
@@ -694,6 +694,38 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse
|
||||
|
||||
⚠️ **The loading banner shows the REAL backend log line, not a guess, and has no countdown or auto-timeout at all.** `getLatestLlamaSwapLogLine()` holds one `GET /api/events` SSE connection open per endpoint (confirmed live to stay open indefinitely — read past 220KB over 8s with no `done`; idle-closed after 30s via `pruneIdleLlamaSwapLogTails`, same 20s sweep as the swap-displacement check above), parsing `logData` frames and keeping only `source: "upstream"` (the real `llama-server` process's own stdout) lines, never `source: "proxy"` (llama-swap's own request-access log). ⚠️ `GET /logs` — the endpoint this feature's own first cut targeted, since the name suggested it — was confirmed live to carry ONLY the proxy log and never a single backend line, even seconds after a real, verified model swap; caught and corrected by a live check before merge, not after. The banner itself dropped its size-scaled expected-time estimate and matching auto-timeout (a guess dressed up as a fact that could kill a genuinely slow load partway through on slower hardware) for a generic hardware/model-size disclaimer plus a user-driven **Cancel** button (`_showCenterStatus`'s `onCancel` option, a real button distinct from the plain "×" close glyph an `'error'`-type banner gets) that ends the wait and closes the session on the user's own call rather than a guessed deadline.
|
||||
|
||||
### Local model install
|
||||
|
||||
**A model that runs on this Mac** (`src/local-model/`, `src/config/local-model-catalog.ts`, routes `src/web/routes/local-model-routes.ts`; EXPERIMENTAL; guide `docs/local-models.md`): the server looks at the Mac it runs on, says which local coding model fits it, and installs one: a pinned runtime it supervises, the model, the checks, the coding agent, and one saved model endpoint. The read side (`hardware.ts`, the catalogue, the pure `recommend()`) is described in the root `CLAUDE.md`; this section is the install.
|
||||
|
||||
**The layout** is one pure function (`localModelPaths()`, `paths.ts`): `<data dir>/local-model/` holds `runtime/ollama-<version>/` (plus its `.tgz.partial` and `.unpack` while it is fetched), `models/` and `home/`; `<data dir>/local-model.json` is the state file. ⚠️ Every path comes from `dataPath()` through that function and NONE is ever read back from the state file (which holds no path), so a state file cannot point a delete anywhere. ⚠️ Nothing is written or deleted outside `local-model/` except the state file and the two writes of the connect step. The runtime's `HOME` is the private `home/` (measured to work), so its key pair lands there and the user's own `~/.ollama` is never created, read or removed; `home/` and a `.ollama/id_ed25519` anywhere are on the sensitive-path list.
|
||||
|
||||
**The pin** (`OLLAMA_RUNTIME_PIN`, `runtime.ts`) is a COMPILED constant: version, URL, size, sha256. ⚠️ No file, setting, env var or request may supply any part of it: a hash that comes from the same writable place as the URL pins nothing. The archive is streamed to `*.partial` while it is hashed, continued with a Range request (3 bounded retries), refused when it grows past the pinned size, and the hash is compared BEFORE `/usr/bin/tar` unpacks anything into a temporary folder that is then renamed.
|
||||
|
||||
**Supervision** (`createRuntimeSupervisor()`): the runtime is a NON-detached child, started at boot once a model is installed and enabled, restarted at most 3 times in a row, SIGTERMed at shutdown beside `stopDeepSeekWeb()`. ⚠️ Its environment comes from the pure `runtimeEnv()` ALLOWLIST (PATH, the private HOME, TMPDIR, LANG, the proxy variables, and exactly `OLLAMA_HOST=127.0.0.1:<port>`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`, `OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`, `OLLAMA_MAX_LOADED_MODELS=1`): never `process.env` whole (it carries `CODEMAN_PASSWORD`, and a user's own `OLLAMA_ORIGINS=*` would open the port to every web page). ⚠️ The port is chosen ONCE by binding (11435 across 20; 11434 belongs to a runtime the user may run) and stored; it is never searched again on a later start, because it is part of the saved endpoint's URL. ⚠️ Pid and port are in the state file: a listener on the port that answers `/api/version` and whose pid runs a binary under OUR runtime folder is our own orphan (a server that was killed leaves one: a child is not ended by its parent's death) and is adopted or replaced; ANY other listener is `port-in-use`, never something to talk to. ⚠️ The supervisor never writes the state file: it reports through `onChange`, and the job stores what it reports. ⚠️ Under vitest nothing spawns and nothing is fetched unless a test injects it.
|
||||
|
||||
**The budget is measured.** The runtime prints the device total in its log line "inference compute" (`total="37.4 GiB"`); `parseInferenceComputeTotal()` reads it, the job stores it and judges the chosen model against it (`modelFitsBudget()`) BEFORE the model is downloaded; from then on the GET recommends with `budgetSource: 'measured'`. One consent covers the whole install: the sizes that were confirmed do not change, so a model that no longer fits is a failure with the next model, not a second question.
|
||||
|
||||
**The job** (`install-job.ts`) is a pure reducer (`reduceInstallJob()`) over three tables in that file (`INSTALL_STATE_TABLE`, `INSTALL_STEP_TABLE`, `INSTALL_TRANSITIONS`), driven by one service per process (`getLocalModelService()`). States: `idle`, `installing`, `paused`, `failed`, `installed`; steps in order: `runtime`, `model`, `verify`, `agent`, `connect` (`agent` is left out when the coding agent is installed). ⚠️ An event the transition table does not list is REFUSED by returning the same object, which the service answers with 409 `wrong-state`; add a transition to the table, never a special case to the service. ⚠️ ONE job at a time: verbs run on one queue, so two installs sent together cannot both find the state free. ⚠️ A job the state file still calls `installing` when the process has none was interrupted by a restart: it becomes `paused` the first time anything looks (`boot()`, a GET or a verb), and NOTHING continues without a click. `shutdown()` therefore writes nothing: it aborts the step and leaves `installing` in the file. ⚠️ An interrupted step is never recorded as failed, whatever its last call answered (an aborted fetch looks like a network error): the job checks its own signal before it writes `failed`. ⚠️ Every step is idempotent, because Resume and "Try again" re-enter the step the job stopped in. ⚠️ Cancel in the `connect` step is honoured only before the first write: once the two writes began they finish, so the pair is never left half done. ⚠️ The state file (`state-store.ts`) has ONE writer, serialized, temp file + rename, mode 0600, and reading is TOTAL (a file that does not parse, another schema, or a state that lacks a member it needs reads as idle): it holds nothing of the user, so it is replaced rather than refused, unlike `settings.json`. ⚠️ Progress is kept in memory and written at most every few seconds and when the job pauses or fails; the window reads the live figure. A 23 GB pull emits about 1.4 million lines, so neither the stream nor one state write per line is affordable.
|
||||
|
||||
**Install is checked before anything is downloaded** (`LocalModelService.install()`): the model, the context and the coding agent must be a row the CURRENT recommendation offers (a model id is looked up, never passed on: without a slash it cannot name another registry host, and the request schema repeats that); the coding agent must be a harness row of the catalogue; `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`: the model plus the runtime archive while the runtime is not unpacked), else 409 `size-changed` with that total, so a consent is bound to the size that was shown; and a saved endpoint that already has the reserved id and is not ours is said now, not after a 23 GB download.
|
||||
|
||||
**The check after the pull** (`verifyModel()`): one tiny chat with `reasoning_effort: "none"`, then `/api/ps` must show `size_vram == size` at the chosen context, then one tool probe (`tools`, `reasoning_effort: "none"`, `max_tokens` 1024) that passes on a tool call with parseable arguments. ⚠️ Thinking is switched off and the cap is generous on purpose: every catalogue model thinks first, and a probe that ends on its token limit proves nothing (`probe-unfinished`, never `no-tool-call`). ⚠️ A failure is a state with a code and ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed". ⚠️ The job fails when the runtime reports no digest for the model: there would be nothing to compare later.
|
||||
|
||||
**The coding agent** is installed by `runInstallCommand()` of `src/cli-install-runner.ts`, the runner extracted from `cli-registry-routes.ts` (whose route keeps its behaviour and re-exports `installEnv`/`npmGlobalPrefixWritable` for its tests). ⚠️ The job runs only `stockInstallCommand(id)`, read from the COMPILED `STOCK_CLIS`, never the registry merged with `clis.json` (the route's own command still comes from the merged registry, behind its `cliManagementEnabled` gate): this path has no such gate, so an override file must not be able to change what it runs. `HarnessFact.installCommand` is that same string, which is what the consent screen prints. ⚠️ Only for a harness row of the catalogue, only after the consent click, admin only, audited (`cli_install` with `via: 'local-model'`). ⚠️ ONE install slot per CLI id (`tryBeginCliInstall()`), shared by the route and the job. ⚠️ After the installer, successful or not, the resolver caches are forgotten (`forgetResolvedCli()`) and the binary is looked for again: a pre-install miss is negative-cached for up to five minutes, and "Start a session" would otherwise answer "not installed". Still missing is `harness-failed`; found is success whatever the exit code. The runner takes an abort signal that kills the installer's process GROUP.
|
||||
|
||||
**The connection** (`connect.ts`): one record `{ id: 'local-ollama', label: 'This Mac', baseUrl, authStyle: 'bearer', models: [<id as /v1/models lists it>], defaultModelId, defaultContextLength: <the chosen context> }`, validated by `CustomModelHostSchema` and written through `mutateCustomModelHosts()`, then `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, with the flag's previous value kept in the state file (a second connect must hand the FIRST value back, or Remove could never switch the flag off again). ⚠️ `defaultContextLength` is load-bearing: the runtime's model listing states no context, so this field is what reaches the coding agent as its limit. ⚠️ The id is reserved: `POST /api/model-endpoints` refuses it, and a record with it that this feature did not write is never overwritten (`endpoint-id-taken`). ⚠️ While the state file records the connection, the record is MANAGED (`isManagedLocalEndpoint()`): `PUT`/`DELETE /api/model-endpoints/local-ollama` are refused, discover answers the stored list without a request, and the sweep skips it, so a model somebody pulled through the port never becomes launchable and the URL never leaves the supervised port. A hand-made record with that id (no state) stays an ordinary endpoint. `local-model.json` is not a config-backup file while the endpoint store and the settings are, so a restore can take the record away under an installed model: `boot()` puts the RECORD back when it is gone (`ensureLocalEndpoint()`, never replacing one that is there) and leaves the settings flag as it finds it. ⚠️ The runtime's port has NO authentication, and its CORS allowlist always admits `localhost` origins: any local process or localhost page can pull, create or delete models there. That is the reason for the three rules above and for the digest.
|
||||
|
||||
**The digest** `/api/tags` reported after the check is stored. ⚠️ Before a session starts on the managed endpoint (`LocalModelService.launchRefusalMessage()`, asked by BOTH launch paths in `session-routes.ts`: `POST /api/sessions/:id/custom-model` and the quick-start `customModel` branch, before anything is written or created) and at boot it is read again: a different digest, a model that is gone, a runtime Codeman did not start, or a model id other than the installed one refuses the launch with one sentence. The view shows what blocks a session as `error` while the state stays `installed`; it is found live and never stored. Boot recovery of an already running session (`_rebuildCustomModelEnv`) is not guarded: the pane exists and keeps its environment.
|
||||
|
||||
**Remove** stops the child, undoes the connection FIRST (a refused write of the endpoint store or of the settings rejects before any file is deleted), restores the flag only when this feature switched it on AND no other endpoint is saved, then deletes the runtime and the private home and, when asked, the models (`removeLocalModelFiles()`, which refuses any path outside the root). ⚠️ While a live session uses the endpoint the answer is 409 `sessions-open` with the session names (`sessionNamesUsingEndpoint()`, read at the moment of the request). ⚠️ The coding agent stays: what its installer wrote is not ours to remove.
|
||||
|
||||
**Developer tools**: `checkDeveloperTools()` runs ONLY `/usr/bin/xcode-select -p`, bounded, on darwin. ⚠️ Never `git`, `python3` or `xcrun` from the server: on a Mac without the tools each opens a system window. The view carries `developerTools`, and the window prints the one sentence when it is `missing`.
|
||||
|
||||
**The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it.
|
||||
|
||||
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event), `test/local-model-install-service.test.ts`, `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
|
||||
|
||||
### Approvals Inbox
|
||||
|
||||
**Approvals Inbox** (cross-session queue of prompts waiting on a human; `approvalsInboxEnabled`, SYNCED, default OFF: every surface is opt-in; only the store and answer endpoints run regardless, so flipping it ON shows anything already pending): `web/approval-inbox.ts` is a `sessionWaits`-style singleton fed by `/api/hook-event`, holding at most ONE item per session (a new prompt supersedes), in-memory; answerable items are claude-mode only, and the DeepSeek status bridge and codex's screen-read dialogs (`web/screen-blocked.ts`) open ALERT-ONLY items (`answerInTerminal`: the answer route refuses them BEFORE its hooks gate, push carries no `approvalId`, the card and overview strip say "answer in the terminal"). Cards are answered via `POST /api/approvals/:id/answer`, which sends a digit / Esc / idle-prompt text through `writeViaMux` (menu answers never carry `\r`).
|
||||
|
||||
@@ -25,7 +25,7 @@ Every run mode Codeman can launch — Claude Code, Terminal/Shell, OpenCode, Cod
|
||||
App Settings → Agents & CLIs → **CLI management** (`cliManagementEnabled`, default OFF; admin-only in multi-user mode) lists every entry with an installed/not-installed badge and:
|
||||
|
||||
- toggles any entry on or off. A `kind: 'shell'` entry cannot be disabled, and the row shows no switch for it. A disabled CLI disappears from the Run menu, the welcome screen and the phone overview, and new session requests for it are rejected.
|
||||
- installs a missing **stock** CLI by running its shipped install command, after a confirm that names the exact command. Only one install per CLI runs at a time, and the command runs without any `CODEMAN_*` variable in its environment. An `npm install -g` command is pointed at `~/.local` (which every resolver searches) when the npm global prefix is not writable by the server user, for example a system node under `/usr`; a writable prefix, a prefix that does not exist yet but could be created, and an explicit `NPM_CONFIG_PREFIX` are left alone. A custom entry's install command is never executed.
|
||||
- installs a missing **stock** CLI by running its shipped install command (the entry as merged with `clis.json`, so an override of a stock entry's install command is what runs here), after a confirm that names the exact command. Only one install per CLI runs at a time, and the command runs without any `CODEMAN_*` variable in its environment. An `npm install -g` command is pointed at `~/.local` (which every resolver searches) when the npm global prefix is not writable by the server user, for example a system node under `/usr`; a writable prefix, a prefix that does not exist yet but could be created, and an explicit `NPM_CONFIG_PREFIX` are left alone. A custom entry's install command is never executed.
|
||||
- adds, edits and deletes **custom** entries (id, label, badge, binaries, launch argv). The server re-validates the whole assembled entry through `CliEntrySchema`, so the form cannot bypass the load-time rules.
|
||||
|
||||
These are the only writes to `clis.json`. They are serialized, and a file that does not parse or has unsafe permissions is refused rather than overwritten; fix it (or `chmod 600` it) and retry. The HTTP routes are listed in `docs/api-reference.md` under *CLI management*.
|
||||
@@ -332,7 +332,7 @@ already have.
|
||||
|
||||
Three rules, and the middle one is why the embed matters:
|
||||
|
||||
1. **The server never executes an entry's `install.command`.** Unchanged, and still enforced by nothing executing it: the field is display text (`CliDiscovery.install.command`).
|
||||
1. **The server executes an entry's `install.command` in exactly two places, both behind a consent click that shows the literal text.** CLI management (above, `cliManagementEnabled`, the merged entry's command) and the local model install (`docs/local-models.md`: only the COMPILED stock command of a catalogue harness row, through `stockInstallCommand()`, so no `clis.json` override reaches it; no setting gates that path, the consent screen does). Both run through the one runner in `src/cli-install-runner.ts` (one install per CLI at a time, no `CODEMAN_*` in the environment, the whole process group killed on a timeout or a cancel) and both are written to the admin audit log in multi-user mode. Everywhere else the field is display text (`CliDiscovery.install.command`).
|
||||
2. **`install.sh` executes only commands embedded in itself.** Those arrive in the same file, over the same TLS fetch, in the same commit as the `curl \| bash` line that fetched the script — identical trust to the hardcoded vendor one-liners it replaces.
|
||||
3. **Nothing fetched at install time is ever executed.** There is no second code path that fetches anything after the script itself has been fetched.
|
||||
|
||||
|
||||
@@ -663,6 +663,45 @@ each result. `scripts/test-local-llm-harnesses.ts` is the standalone script
|
||||
used to check a harness against a real endpoint outside the web UI
|
||||
entirely; see its own `--help` for usage.
|
||||
|
||||
## The endpoint store has one writer
|
||||
|
||||
`~/.codeman/custom-model-hosts.json` is written only through `mutateCustomModelHosts()`
|
||||
(`src/custom-model-hosts-writer.ts`): the create, update, delete and discover routes, the
|
||||
five-minute re-discovery sweep and the local model install all go through it.
|
||||
|
||||
- **One at a time.** Changes run on one queue, and the check a route makes ("this id is
|
||||
free", "this endpoint still exists") runs in the same step as its write. Before, each
|
||||
writer read the file, changed its copy and wrote it back on its own, all through one temp
|
||||
file name: a route and the sweep, or two requests sent together, could lose each other's
|
||||
change or fail outright.
|
||||
- **Atomic.** A temp file with a random name beside the target, then a rename. Mode 0600.
|
||||
- **A file that does not parse is refused, not replaced.** The writing routes answer
|
||||
`409 CONFLICT` with the fix (repair or delete the file, or restore it from a config
|
||||
backup) and leave it byte for byte. Before, such a file read as an empty list, and the next
|
||||
save replaced every stored endpoint and its API key with one record. The message never
|
||||
quotes the file, which holds keys.
|
||||
|
||||
The one direct writer left is config-backup restore, which must work on a file that does
|
||||
not parse.
|
||||
|
||||
## The endpoint of the model on this Mac
|
||||
|
||||
Installing a local model from Codeman ([local-models.md](local-models.md)) saves one endpoint
|
||||
with the reserved id `local-ollama` and the label "This Mac". It shows up in the list and in
|
||||
the Run menu like any other endpoint. Three things are different:
|
||||
|
||||
- A new endpoint cannot be given the id `local-ollama`.
|
||||
- While it is the endpoint the install manages, it cannot be edited or deleted here, and
|
||||
"Discover" answers its stored model list without asking the runtime. Its model list must
|
||||
stay the one model the install checked, and its address the port of the runtime Codeman
|
||||
supervises. Change or remove it in the local model window.
|
||||
- A session is started on it only while Codeman's own runtime is running and the model is
|
||||
still the one that was installed (its digest is compared). Otherwise the launch answers
|
||||
with one sentence.
|
||||
|
||||
An endpoint with that id that somebody made by hand, with no model installed by Codeman, is
|
||||
an ordinary endpoint: it can be edited and deleted as before.
|
||||
|
||||
## Security note
|
||||
|
||||
Every env var this feature can set that redirects a session's traffic
|
||||
|
||||
@@ -0,0 +1,267 @@
|
||||
# A model that runs on this Mac
|
||||
|
||||
Codeman can set up a coding model that runs on the Mac the server runs on: no account, no
|
||||
API key, nothing leaves the machine. One button on the welcome screen opens a window that
|
||||
shows what fits this Mac, installs the choice and ends on a coding agent session that
|
||||
runs on it.
|
||||
|
||||
**Experimental.** The routes and their payloads may change in any release
|
||||
([versioning-policy.md](versioning-policy.md), "Experimental local model routes"). Apple
|
||||
silicon macOS only: anywhere else the window says why nothing is offered.
|
||||
|
||||
This page is the reference for how it works. The HTTP contract is in
|
||||
[api-reference.md](api-reference.md) ("Local model"). The rules that must not be broken are
|
||||
in [architecture-invariants.md](architecture-invariants.md#local-model-install).
|
||||
|
||||
## What an install does, in order
|
||||
|
||||
The person picks a model, presses "Install on this Mac" and confirms the sizes once. The
|
||||
server then runs one job of five steps. The window polls `GET /api/local-model` once a
|
||||
second; the job runs in the server, so the window can be closed.
|
||||
|
||||
| Step | What it does |
|
||||
| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `runtime` | Downloads the pinned runtime, compares its hash, unpacks it, chooses a port once, starts it, reads the measured memory budget and judges the choice again |
|
||||
| `model` | Downloads the model through the runtime |
|
||||
| `verify` | Loads the model and checks it: one tiny chat, the memory and context the runtime reports, one tool call |
|
||||
| `agent` | Installs the coding agent with its stock install command. Left out when the agent is installed |
|
||||
| `connect` | Saves one model endpoint and switches the custom model endpoints setting on |
|
||||
|
||||
Every step can be repeated: Resume and "Try again" enter the step the job stopped in.
|
||||
|
||||
### The runtime
|
||||
|
||||
- **One pinned build.** Ollama 0.40.15 for macOS, a compiled constant
|
||||
(`OLLAMA_RUNTIME_PIN` in `src/local-model/runtime.ts`): the URL, the size (185,971,671
|
||||
bytes) and the sha256. No file, setting, environment variable or request can change which
|
||||
binary is downloaded.
|
||||
- **Download.** Streamed to `*.partial` while it is hashed. A cut stream is continued with a
|
||||
Range request, at most three times. The hash is compared before anything is unpacked, and
|
||||
a mismatch deletes the file. Unpacking is `/usr/bin/tar` into a temporary folder, then one
|
||||
rename.
|
||||
- **No administrator rights.** Everything lives in the data directory.
|
||||
- **Supervised by Codeman.** The runtime is a non-detached child of the server: started at
|
||||
boot once a model is installed and not stopped, restarted at most three times after a
|
||||
crash, sent SIGTERM at shutdown. It is not a login item: it stops with Codeman, and a
|
||||
leftover from a server that was killed is taken back at the next start (see below).
|
||||
- **Its environment is an allowlist.** `PATH`, `TMPDIR`, `LANG` and the proxy variables are
|
||||
copied from the server. `HOME` is a private folder. Exactly these are set:
|
||||
`OLLAMA_HOST=127.0.0.1:<port>`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`,
|
||||
`OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`,
|
||||
`OLLAMA_MAX_LOADED_MODELS=1`. No `CODEMAN_*` variable and none of the user's own
|
||||
`OLLAMA_*` settings reach it.
|
||||
- **The port is chosen once**, by binding, from 11435 across 20 ports, and stored. 11434 is
|
||||
left alone: it is the default of a runtime the user may run themselves, which is never
|
||||
touched.
|
||||
- **An orphan is recognized.** The pid and the port are stored. At boot a listener on the
|
||||
port that answers `/api/version` and whose pid runs a binary under Codeman's runtime folder
|
||||
is Codeman's own leftover from a server that was killed: it is adopted or replaced. Any
|
||||
other listener is "Port 11435 is in use by another program", with "Use another port" as
|
||||
the way out.
|
||||
|
||||
### The memory budget is measured
|
||||
|
||||
Before the runtime exists the budget is an estimate from the installed memory. Right after
|
||||
it starts, the runtime prints the device total in its log line "inference compute"
|
||||
(`total="37.4 GiB"` on a 48 GB MacBook). Codeman reads that line, stores the figure and
|
||||
judges the chosen model against it before a byte of the model is downloaded. A model that no
|
||||
longer fits ends the job there with "does not fit" and the next model to try. From then on
|
||||
the window shows the measured budget and the word "estimated" is gone.
|
||||
|
||||
The need of a model is its peak: the weights, what stays held per token of context, what the
|
||||
runner holds beside both, and what it takes for a moment while a long prompt is processed.
|
||||
|
||||
### The model download
|
||||
|
||||
`POST /api/pull` with `stream: true`. A 23 GB pull sends about 1.4 million progress lines,
|
||||
so the stream is parsed piece by piece and never kept. Cancel aborts the request, which
|
||||
stops the download. Resume pulls again and the runtime continues its partial files. An error
|
||||
line or a cut stream is retried up to five times. Once the pull reported its total, every
|
||||
size shown for the model is that total.
|
||||
|
||||
### The check
|
||||
|
||||
After the pull, in this order:
|
||||
|
||||
1. One tiny chat with `reasoning_effort: "none"`. It loads the model.
|
||||
2. `GET /api/ps` must show the whole model on the GPU (`size_vram` equal to `size`) at the
|
||||
chosen context length.
|
||||
3. One tool probe on `/v1/chat/completions` (a `tools` array, `reasoning_effort: "none"`,
|
||||
`max_tokens` 1024). It passes on a tool call whose arguments parse.
|
||||
|
||||
A failure is a state with a code and one next step, never a bare "failed": `does-not-fit`
|
||||
and `no-tool-call` offer the next model, `probe-unfinished` (the token limit ended the
|
||||
probe, so it proves nothing) and `load-failed` offer "Try again". The digest the runtime
|
||||
reports for the model is stored when the check passed.
|
||||
|
||||
### The coding agent
|
||||
|
||||
When the chosen coding agent is not installed, the job installs it with the CLI registry's
|
||||
own stock install command, through the same runner as App Settings > CLIs
|
||||
(`src/cli-install-runner.ts`). The rules:
|
||||
|
||||
- Only a CLI that is a harness row of the local model catalogue.
|
||||
- Only the **compiled** stock command. A `~/.codeman/clis.json` override cannot change what
|
||||
runs here.
|
||||
- Only after the person's consent click, on a screen that showed the literal command.
|
||||
- Admin only in multi-user mode, and written to the admin audit log (`cli_install`, with
|
||||
`via: "local-model"`).
|
||||
- One installer per CLI at a time, shared with App Settings: while one runs, the other
|
||||
answers that it is busy.
|
||||
- Afterwards the cached lookups of the CLI's binary are forgotten and the binary is looked
|
||||
for again. Still missing is the failure `harness-failed`, with the installer's last lines.
|
||||
|
||||
This step does not need the `cliManagementEnabled` setting. That setting gates the general
|
||||
CLI management surface; this path can only ever run one vetted command for one catalogue
|
||||
row.
|
||||
|
||||
### The connection
|
||||
|
||||
Two writes outside the local model folder, both undone by Remove:
|
||||
|
||||
1. One record in `custom-model-hosts.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "local-ollama",
|
||||
"label": "This Mac",
|
||||
"baseUrl": "http://127.0.0.1:11435",
|
||||
"authStyle": "bearer",
|
||||
"models": ["qwen3.6:35b-a3b"],
|
||||
"defaultModelId": "qwen3.6:35b-a3b",
|
||||
"defaultContextLength": 131072
|
||||
}
|
||||
```
|
||||
|
||||
`defaultContextLength` is the chosen context: the runtime's model listing states none, so
|
||||
this field is what reaches the coding agent as its context limit.
|
||||
|
||||
2. `customModelEndpointsEnabled: true` in `settings.json`. The value it had before is kept
|
||||
in the state file.
|
||||
|
||||
The id `local-ollama` is reserved. A new endpoint cannot take it, and a record with that id
|
||||
that this feature did not write is never overwritten: the install says so before anything is
|
||||
downloaded. While the record is the managed one it cannot be edited or deleted in App
|
||||
Settings and the five-minute discovery sweep leaves it alone, so its model list stays the
|
||||
one model the install checked.
|
||||
|
||||
Switching the setting on also starts that five-minute sweep for every other saved endpoint.
|
||||
|
||||
## Starting a session
|
||||
|
||||
"Start a session" starts the coding agent on the endpoint `local-ollama` through the
|
||||
existing custom model launch. Before a coding agent is STARTED on that endpoint the server
|
||||
checks, in this order:
|
||||
|
||||
- the runtime child Codeman started is alive (whatever else answers on the port is not
|
||||
ours);
|
||||
- the digest the runtime reports for the installed model is the one stored at install time;
|
||||
- the model asked for is the one that was installed and checked.
|
||||
|
||||
That check runs wherever a new agent process comes to be on the endpoint: a session created
|
||||
with the model, a model applied to a running session (which restarts its agent), the attach
|
||||
that starts an agent in a session that has none or whose agent exited, and the rebuild of a
|
||||
session after a reboot (that entry is skipped with "the local model is not ready" and stays
|
||||
on offer). An agent that is already running when Codeman restarts is not started by
|
||||
Codeman, so it is not asked: it keeps running on its port, see below.
|
||||
|
||||
A different digest refuses the launch: "The model on this Mac changed since it was
|
||||
installed. Remove it and install it again." The digest is also read again at boot and when
|
||||
the runtime is started.
|
||||
|
||||
## Stop, Start, Remove
|
||||
|
||||
- **Stop** stops the runtime and keeps it from starting at the next boot. **Start** undoes
|
||||
that.
|
||||
- **Remove** stops the runtime, removes the endpoint record, switches the setting off again
|
||||
only when this feature switched it on and no other endpoint is saved, and deletes the
|
||||
runtime and its private home. The downloaded model is deleted only when asked. The coding
|
||||
agent stays. While a live session uses the model, Remove answers with the session names
|
||||
instead.
|
||||
|
||||
## Where everything is
|
||||
|
||||
All under the data directory (`~/.codeman`, or the instance's own):
|
||||
|
||||
```
|
||||
local-model.json the state file (mode 0600)
|
||||
local-model/
|
||||
runtime/ollama-0.40.15.tgz.partial the runtime archive while it downloads
|
||||
runtime/ollama-0.40.15/ the unpacked runtime (about 579 MB)
|
||||
models/ the model store
|
||||
home/ the runtime's private HOME (its key pair is here)
|
||||
```
|
||||
|
||||
Nothing is written or deleted outside `local-model/` except the state file beside it and the
|
||||
two writes of the connection. The state file holds no path, so it can never point a delete
|
||||
somewhere else. The user's own `~/.ollama` is never created, read or removed.
|
||||
|
||||
`local-model.json` is not part of a config backup. `custom-model-hosts.json` and
|
||||
`settings.json` are, so restoring an older backup can take the endpoint record away while
|
||||
the model stays installed. The record is put back the next time Codeman starts (the setting
|
||||
is left as the backup had it). A restore can also bring a record back after a Remove: it
|
||||
then points at a port nothing listens on, is an ordinary endpoint again, and can be deleted
|
||||
in App Settings.
|
||||
|
||||
## The job's states
|
||||
|
||||
| State | A server restart | Cancel | What can be done |
|
||||
| ------------ | ------------------------------------------------------------- | ---------------- | -------------------------------------- |
|
||||
| `idle` | stays idle | refused | install |
|
||||
| `installing` | becomes `paused` at the same step | becomes `paused` | cancel |
|
||||
| `paused` | stays paused | refused | resume, install, remove |
|
||||
| `failed` | stays failed, same error | refused | resume, install, remove |
|
||||
| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, remove |
|
||||
|
||||
One job at a time. A job a server restart interrupted comes back paused and never continues
|
||||
without a click. Cancel interrupts the step in flight (the download, the pull, the request,
|
||||
the installer's process group); in the last step it is honoured before the first write, and
|
||||
once the two writes began they finish.
|
||||
|
||||
## On a Mac without developer tools
|
||||
|
||||
Coding agents call `git` and `python3`. On a Mac without Xcode or the Command Line Tools
|
||||
those are stubs that open macOS's "install the command line developer tools" window. So the
|
||||
window says one sentence wherever the person commits or finishes, and only when
|
||||
`/usr/bin/xcode-select -p` fails:
|
||||
|
||||
> Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer
|
||||
> to install them the first time an agent needs one.
|
||||
|
||||
Codeman itself only ever runs `/usr/bin/xcode-select -p` for this, bounded, and never `git`
|
||||
or `python3`.
|
||||
|
||||
## What to know before relying on it
|
||||
|
||||
- **The runtime's port has no login.** Any program on this Mac, and any web page served from
|
||||
`localhost`, can use the model server and change its models. That is why the endpoint
|
||||
record lists one model, why the sweep leaves it alone, and why the digest is compared
|
||||
before a session starts.
|
||||
- **A Codeman restart stops the runtime.** A session that is mid-answer on the local model
|
||||
loses that answer. The runtime is started again with the server, and the first request
|
||||
loads the model again (seconds).
|
||||
- **An open session keeps calling the runtime's port.** The coding agent was given the
|
||||
address when it started. While the runtime is stopped, or for the seconds of a Codeman
|
||||
restart, its requests go to a port nothing of ours holds, and any program on this Mac could
|
||||
bind that port and answer them. That is why Stop and "Use another port" are refused while
|
||||
a session uses the model: close those sessions first.
|
||||
- **The runtime does not enforce the context length.** A prompt longer than the context was
|
||||
accepted in testing. Do not rely on the runtime to stop one.
|
||||
- **The model stays in memory for ten minutes** after the last request, then the runtime
|
||||
unloads it.
|
||||
|
||||
## Source
|
||||
|
||||
| What | Where |
|
||||
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| Hardware probe, catalogue, recommendation | `src/local-model/hardware.ts`, `src/config/local-model-catalog.ts`, `src/local-model/recommend.ts` |
|
||||
| Runtime pin, download, environment, supervisor | `src/local-model/runtime.ts` |
|
||||
| Runtime HTTP client, pull parser, checks | `src/local-model/ollama-client.ts` |
|
||||
| Developer tools check | `src/local-model/developer-tools.ts` |
|
||||
| Job (state table, reducer, view, service) | `src/local-model/install-job.ts` |
|
||||
| State file | `src/local-model/state-store.ts` |
|
||||
| Connection and its undoing | `src/local-model/connect.ts` |
|
||||
| Paths | `src/local-model/paths.ts` |
|
||||
| Routes | `src/web/routes/local-model-routes.ts` |
|
||||
| Shared CLI install runner | `src/cli-install-runner.ts` |
|
||||
| Serialized writer of the endpoint store | `src/custom-model-hosts-writer.ts` |
|
||||
@@ -594,6 +594,15 @@ Opt-in and off by default: the server POSTs the Web Push events (permission prom
|
||||
- **Discord bridge** (`src/discord-bridge.ts`, [setup](discord-bridge.md)): the bot token lives in `~/.codeman/discord.json` (0600, tmp+rename) and is never returned by the Discord routes, logged or put in an error, and the attachment, folder and download routes refuse the file. A session whose folder is the home directory can still read it through its own file viewer, which serves that workspace without the blocklist. Inbound replies are typed into a session only from `allowedUserIds`, in the configured guild, inside a thread the bridge created; anything else is dropped unread. A reply becomes one printable line plus Enter, capped at 1000 characters and one per second per tab. Posts send `allowed_mentions` limited to the allowlisted ids, so agent output cannot ping a server. **Files:** an attachment is fetched only from Discord's own HTTPS hosts (no redirects followed), at most 5 per reply and 8 MB each, saved 0600 under `<tab folder>/.codeman-discord/` with the name reduced to `[A-Za-z0-9._-]`; a file a CLI sends must be a regular file whose real path (symlinks followed) lies inside its own tab's folder and is not on the attachment blocklist (`isBlockedAttachmentPath`, since a tab's folder can be the home directory), which stops a sensitive file being uploaded by mistake, and remote sessions send and receive none. A received file is untrusted content to whatever opens it. **What the checks do not stop:** the agent runs as the same OS user, so the path check prevents accidental uploads, not a determined agent that pastes a file's contents into the message. **What leaves the machine:** a permission or question alert sends the rendered screen (whatever the agent printed), and a finished turn or an error sends the CLI's whole last message, read from its transcript (up to 8 posts of 1,900 characters). `idle_prompt` is a warning, so with the default scope every Claude turn left unanswered for a minute is posted, and with **Which events** set to Everything every turn end is; treat the Discord server as holding that output. **Multi-user:** the bridge is one allowlist for the whole server, so an allowlisted Discord account can type into every user's sessions; only an admin can configure it. **Startup hint:** a fixed text (no user input) is added to Claude's system prompt; no other CLI is sent it. The bridge only dials out (REST and the gateway WebSocket), so it needs no inbound exposure. A Discord account on the allowlist can drive agents that run shell commands: it needs 2FA.
|
||||
- **It carries session data to a third party.** Titles and bodies include session names, tool names and error text, all agent- or user-controlled, so Discord gets `allowed_mentions: { parse: [] }` and Slack's `& < >` are escaped: agent output cannot ping a channel. In multi-user mode all three routes are admin-only and the channel is instance-wide: it receives every user's session events, the same reach an admin's own Web Push has, which means non-admins' session details leave the box at the admin's choice.
|
||||
|
||||
## 10d. The model on this Mac (local model install)
|
||||
|
||||
Opt-in by a click, macOS only, admin-only in multi-user mode on every verb. The server downloads one pinned runtime build, runs it as its own child, pulls one model into it, installs the coding agent when it is missing, and saves one endpoint record. Source: `src/local-model/`, routes in `src/web/routes/local-model-routes.ts`. Guide: [`local-models.md`](local-models.md).
|
||||
|
||||
- **Supply chain: a compiled pin.** The runtime's URL, size and sha256 are a constant in `src/local-model/runtime.ts` that no file, setting or request can change; the archive is hashed while it streams and compared BEFORE anything is unpacked, a mismatch deletes it, and only `/usr/bin/tar` unpacks it into a folder under the data dir. The model comes from the runtime's own registry under the name the catalogue pins; the digest the runtime reports after the pull is stored, and a model whose digest changed under that name is refused at the next start.
|
||||
- **A loopback port without a login.** The runtime listens on `127.0.0.1:<port>` (chosen once from 11435) with no authentication, like every model server of its kind: any program of any account on the Mac, and any web page served from `localhost`, can use it and change its models. What limits that: the endpoint record lists ONE model and the discovery sweep leaves it alone, the digest is compared before an agent is started on it (every start path, see the guide), and a listener on the port that is not our own child (judged by the stored pid's command line, never by the port alone) is reported as `port-in-use` and never signalled. An open session keeps calling the port while the runtime is stopped, which is why Stop and a port change are refused while one uses the model.
|
||||
- **A second install path for a coding agent, without `cliManagementEnabled`.** The job runs the COMPILED stock install command of a catalogue harness row (never a `clis.json` override, never a custom entry) through the same runner as CLI management, only after the consent screen showed the literal command, and writes the admin audit entry (`cli_install`, `via: 'local-model'`). The installer runs as the server's user with the same reach as every other spawn.
|
||||
- **The runtime's own files stay inside `~/.codeman/local-model/`**: its private HOME (so no `~/.ollama` is created), its model store and its unpacked folder; its environment is an allowlist (no `CODEMAN_*`). Remove deletes only inside that folder, refuses a folder that is a symbolic link, and ends the runtime (ours by its command line) before any file goes.
|
||||
|
||||
---
|
||||
|
||||
## 11. Quick reference
|
||||
|
||||
+18
-11
@@ -86,18 +86,25 @@ covered like any other stable behavior. Other agent commands and API routes keep
|
||||
their existing guarantees. Promotion to stable requires an explicit documented
|
||||
decision.
|
||||
|
||||
### Experimental local model recommendation
|
||||
### Experimental local model routes
|
||||
|
||||
`GET /api/v1/local-model` (and its `/api` alias) is labeled experimental, so item 3
|
||||
above applies to it: its path, its payload (the hardware facts, the verdicts and
|
||||
reasons, the candidate fields and the English sentences in `headline` and
|
||||
`notes`) and its behavior may change or be removed in a MINOR or PATCH release.
|
||||
It is read-only today: it looks at the Mac the server runs on and says which
|
||||
local model fits. The step that sets a model up from Codeman is not built yet
|
||||
and will change what the route reports, which is why the shape is not promised.
|
||||
It is not behind a setting (it writes nothing and adds no settings key); it is
|
||||
admin only in multi-user mode. Promotion to stable requires an explicit
|
||||
documented decision.
|
||||
`GET /api/v1/local-model`, `POST /api/v1/local-model/install`, `/cancel`, `/resume`
|
||||
and `/runtime`, and `DELETE /api/v1/local-model` (and their `/api` aliases) are
|
||||
labeled experimental, so item 3 above applies to them: their paths, their payloads
|
||||
(the hardware facts, the verdicts and reasons, the candidate fields, the `install`
|
||||
object with its states, steps, error codes and actions, the 409 conflict shape and
|
||||
every English sentence) and their behavior may change or be removed in a MINOR or
|
||||
PATCH release. They look at the Mac the server runs on, say which local model fits,
|
||||
and install, start, stop and remove one ([local-models.md](local-models.md)).
|
||||
|
||||
They are not behind a setting. Reading changes nothing; an install starts only on an
|
||||
explicit request that carries the size the person confirmed. All of them are admin
|
||||
only in multi-user mode. What an install leaves behind is ordinary, stable state: one
|
||||
saved model endpoint with the id `local-ollama` and the `customModelEndpointsEnabled`
|
||||
setting switched on, both of which the stable custom model endpoint routes read like
|
||||
any other endpoint. On-disk state (`local-model.json`, the `local-model/` folder) is
|
||||
internal, as for every other file in the data directory. Promotion to stable requires
|
||||
an explicit documented decision.
|
||||
|
||||
## Deprecation policy
|
||||
|
||||
|
||||
@@ -204,6 +204,45 @@ Which harnesses show up in the Run-menu picker is read live off Codeman's own CL
|
||||
not a fixed list here, so this table can go stale before this page does — a greyed-out or
|
||||
missing entry is the more current answer.
|
||||
|
||||
## A model that runs on this Mac
|
||||
|
||||
If the Codeman server runs on a Mac with Apple silicon, you do not need a server of your
|
||||
own: Codeman can set one model up for you on that Mac. This is experimental.
|
||||
|
||||
1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model
|
||||
on this Mac).
|
||||
2. The window shows what fits this Mac. A bar shows how much of the memory the selected
|
||||
model takes at its full context; a nearly full bar means a tight fit. Pick a model and
|
||||
the coding agent to run it with.
|
||||
3. Press **Install on this Mac**, check the sizes and where they go, and confirm. Nothing is
|
||||
downloaded before this click.
|
||||
4. Codeman downloads a model runtime (Ollama, about 186 MB) and the model into
|
||||
`~/.codeman/local-model`, loads the model, checks that all of it runs on the GPU and that
|
||||
it can call a tool, installs the coding agent if it is missing, and saves an endpoint
|
||||
named **This Mac**. No administrator password is asked for.
|
||||
5. Press **Start a session**.
|
||||
|
||||
Good to know:
|
||||
|
||||
- You can close the window: the download continues while Codeman is running. **Cancel**
|
||||
pauses it and **Continue** picks it up where it stopped. After a Codeman restart an
|
||||
unfinished install waits for your click.
|
||||
- The memory figure is an estimate until the runtime has started once. From then on it is
|
||||
what the runtime itself measured, and the model is judged against that before it is
|
||||
downloaded.
|
||||
- The endpoint "This Mac" belongs to this install. It is in the list and in the Run menu,
|
||||
but it is changed and removed in the local model window, not here.
|
||||
- Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from
|
||||
starting next time.
|
||||
- **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The
|
||||
coding agent stays installed.
|
||||
- If the model on disk is no longer the one that was installed, Codeman refuses to start a
|
||||
session on it and tells you to remove and install it again.
|
||||
- Any program on this Mac, and any web page served from `localhost`, can reach the model
|
||||
runtime: its port has no login. It listens on this Mac only.
|
||||
- On a Mac without Apple's developer tools, macOS will offer to install them the first time
|
||||
a coding agent runs `git` or Python. The window tells you when that applies.
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **No remote or Docker sessions yet.** Both restart their agent differently under the hood
|
||||
|
||||
@@ -193,19 +193,25 @@ attached.
|
||||
section to the Run dropdown, for pointing a harness at your own OpenAI-compatible server
|
||||
instead of its native cloud backend. See [Custom Model Endpoints](Custom-Model-Endpoints).
|
||||
|
||||
**Model on this Mac** is not a setting: it is a read-only answer. When the Codeman server
|
||||
runs on a Mac with Apple silicon, it shows that Mac's chip, memory and free disk space, the
|
||||
local coding model that fits it best with the agent CLI to run it, what the model needs of
|
||||
the memory the Mac can give one, its download size and an estimated speed, and the other
|
||||
models that fit (Good or Tight for the memory, Slow or Speed unknown for the speed). A Mac
|
||||
that can only run a model under 12 billion parameters is told so in plain words, and a Mac
|
||||
with too little memory, too little free disk space or an Intel processor is told why nothing
|
||||
is offered. The numbers are estimates from a catalogue that ships with each release; nothing
|
||||
has been tested on your Mac. **Nothing is downloaded or installed from here yet**: setting
|
||||
the model up from Codeman is the next step and is not built. The same answer appears as a
|
||||
card on the welcome screen, which you can hide per device (it stays here). The group is not
|
||||
shown when the server runs on Linux or Windows, and in multi-user mode only an admin sees
|
||||
it. The "Mac" is always the machine the server runs on, not the device you are reading on.
|
||||
**Model on this Mac** is not a setting: it is the state of a model Codeman can run for you
|
||||
on the Mac the server runs on, with a button that opens the local model window. The same
|
||||
window opens from the **Run a local model** button on the welcome screen. It shows that
|
||||
Mac's chip, memory and free disk space, a bar for how much of the memory the selected model
|
||||
takes, and the models that fit, each with one tag (Best fit, Tight, Slow, Small model, or
|
||||
"Tested on a Mac like this"), a speed and a download size. **Install on this Mac** asks you
|
||||
to confirm the sizes once, then downloads a model runtime and the model into
|
||||
`~/.codeman/local-model`, checks that the model loads and can call a tool, installs the
|
||||
coding agent if it is missing, and ends on **Start a session**. No account, no API key and
|
||||
no administrator password. You can close the window while it downloads, cancel and continue
|
||||
later; after a Codeman restart an unfinished install waits for you to press Continue. Once a
|
||||
model is installed the window offers Start a session, Stop or Start, and Remove. Installing
|
||||
switches **Custom model endpoints** on and saves one endpoint named "This Mac"; Remove takes
|
||||
the endpoint away again and switches the setting back off if nothing else uses it. A Mac
|
||||
that can only run a small model is told so in plain words, and a Mac with too little memory,
|
||||
too little free disk space or an Intel processor is told why nothing is offered. The group
|
||||
is not shown when the server runs on Linux or Windows, and in multi-user mode only an admin
|
||||
sees it. The "Mac" is always the machine the server runs on, not the device you are reading
|
||||
on. Details: [Custom Model Endpoints](Custom-Model-Endpoints#a-model-that-runs-on-this-mac).
|
||||
|
||||
### Agents & CLIs
|
||||
|
||||
|
||||
Reference in new issue
Block a user