BitSquare/Codeman
2
0 Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-11-14 10:39:32 +01:00
Code Issues Packages Projects Releases Wiki Activity

feat(local-model): contract for the install stage (types, stubs, fake runtime)

Browse Source 
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-06 20:05:15 +01:00
1 parent 66ca92f7cf
commit 771072c36b
17 files changed
+3770 -23
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/types/index.ts
+3 -1
View File
@@ -29,7 +29,8 @@
* | plan | PlanItem, PlanTaskStatus, TddPhase | In-memory → `GET /api/sessions/:id/plan/tasks` |
* | orchestrator | OrchestratorState, OrchestratorPlan, OrchestratorConfig, OrchestratorPersistState | `~/.codeman/state.json` → `GET /api/orchestrator/status` |
* | mcp-sync | McpSyncResult, McpSyncTargetResult | Other CLIs' own config files → `GET`/`POST /api/mcp-sync` |
* | local-model | HardwareFacts, LocalModelCatalog, Candidate, Recommendation, LocalModelStatus, LOCAL_MODEL_LABELS | In-memory only (nothing persisted) → `GET /api/local-model` |
* | local-model | HardwareFacts, LocalModelCatalog, Candidate, Recommendation, RecommendOptions, LOCAL_MODEL_LABELS | In-memory only (nothing persisted) → `GET /api/local-model` |
* | local-model-install | LocalModelStatus, LocalModelInstallView, LocalModelStateFile, LocalModelRuntimePin, LOCAL_MODEL_INSTALL_LABELS | `~/.codeman/local-model.json` → `GET /api/local-model` and its install verbs |
*
* ## Cross-domain relationship map
*
@@ -79,3 +80,4 @@ export * from './intent.js';
export * from './foreign-tmux.js';
export * from './mcp-sync.js';
export * from './local-model.js';
export * from './local-model-install.js';
src/types/local-model-install.ts
+582
View File
@@ -0,0 +1,582 @@
/**
* @fileoverview Types of the local model INSTALL: the runtime pin, the job's states, steps and
* error codes, the progress, the `install` object of `GET /api/local-model`, the bodies and
* answers of the install verbs, and the shape of the state file.
*
* The read side (hardware, catalogue, recommendation) is in src/types/local-model.ts. This
* file holds `LocalModelStatus`, the whole `data` of the GET, because it joins the two.
*
* Who produces what:
* - `LocalModelRuntimePin`: a compiled constant in src/local-model/runtime.ts. No file, no
* setting and no request can change which binary is downloaded.
* - `LocalModelStateFile`: src/local-model/state-store.ts, the ONE writer of
* `local-model.json` (mode 0600, temp file + rename).
* - `LocalModelInstallView`: src/local-model/install-job.ts (`buildInstallView()`, pure), from
* the state file plus what is live (the runtime child, the developer tools check).
* - `LocalModelStatus` and the verb answers: src/web/routes/local-model-routes.ts.
*
* The HTTP contract. Every route is admin only in multi-user mode (403 `FORBIDDEN`) and
* EXPERIMENTAL like the GET (docs/versioning-policy.md). Every verb answers
* `{ success: true, data: LocalModelInstallView }` with the view AFTER the verb.
*
* | Route | Body | Refusals |
* |----------------------------------|-----------------------------|-------------------------------------------------------|
* | `GET /api/local-model` | none | none (a failed probe is an answer) |
* | `POST /api/local-model/install` | `LocalModelInstallRequest` | 400 not a row the server offers; 409 `job-running`, |
* | | | `already-installed`, `size-changed` |
* | `POST /api/local-model/cancel` | none | 409 `wrong-state` (no job is running) |
* | `POST /api/local-model/resume` | `LocalModelResumeRequest` | 409 `wrong-state` (not paused or failed) |
* | `POST /api/local-model/runtime` | `LocalModelRuntimeRequest` | 409 `wrong-state` (not installed), `job-running` |
* | `DELETE /api/local-model` | `LocalModelRemoveRequest` | 409 `job-running`, `sessions-open` |
*
* A 409 is the error envelope plus `data: LocalModelConflict`:
* `{ success: false, error, errorCode: 'CONFLICT', data: { reason, message, ... } }`.
*
* The rules of src/types/local-model.ts hold here too:
* 1. A size a person reads travels twice: as bytes and as decimal GB text from `formatGb()`
* (`completedGb`, `totalGb`, `downloadGb`). The browser prints the text and adds or
* divides nothing for display. `mbPerSecond` is the one other unit, decimal MB per second.
* 2. Every sentence the server chooses comes from `LOCAL_MODEL_INSTALL_LABELS`, with its
* numbers and names in `vars`. `{name}` is taken by the translator, so no slot has that name.
* 3. A CLI id is data (`LocalModelInstallView.cli`): the browser passes it on and compares it
* with nothing. What a person reads is `cliLabel`.
*/
import type { HardwareFacts, Recommendation } from './local-model.js';
// ---------- The runtime pin ----------
/**
* The one runtime build Codeman downloads and runs. A compiled constant
* (`OLLAMA_RUNTIME_PIN` in src/local-model/runtime.ts): the hash is compared before anything
* is unpacked, so the pin and the hash must come from the same reviewed source file.
*/
export interface LocalModelRuntimePin {
/** The runtime's family. One value today. */
id: 'ollama';
/** The release ('0.40.15'). `GET /api/version` of the running runtime must answer it. */
version: string;
/** Where the archive is downloaded from. https, a fixed release asset. */
url: string;
/** Size of the archive in bytes. A body that grows past it is refused before the hash is finished. */
bytes: number;
/** Lowercase hex sha256 of the archive. */
sha256: string;
/** What the unpacked folder takes on disk, in bytes (measured: about 579 MB). */
unpackedBytes: number;
/** The oldest macOS major version the runtime supports. */
minMacosMajor: number;
}
// ---------- States, steps, codes ----------
/**
* Where the install stands. The verb of the flow is the same throughout: install, installing,
* installed.
* - `idle`: nothing is installed and no job exists.
* - `installing`: the one job is running; `step` says where.
* - `paused`: the job stopped at `step` because of Cancel or a server restart. It never
* continues without a click.
* - `failed`: the job stopped at `step` with `error`.
* - `installed`: the model is connected. Whether the runtime runs is `running`, not a state.
* The state table (what is on disk, what a restart and Cancel do, the allowed actions) is
* `INSTALL_STATE_TABLE` in src/local-model/install-job.ts.
*/
export type LocalModelInstallState = 'idle' | 'installing' | 'paused' | 'failed' | 'installed';
/**
* The steps of a job, in the order they run.
* - `runtime`: get the pinned runtime (download, compare the hash, unpack), start it, read
* the measured memory budget and judge the chosen model against it.
* - `model`: download the model (the runtime's pull).
* - `verify`: load the model and check it: one tiny chat, the memory and context the runtime
* reports, one tool call.
* - `agent`: install the coding agent. Left out of a job whose coding agent is installed.
* - `connect`: save the connection (the endpoint record and the settings flag).
*/
export type LocalModelInstallStep = 'runtime' | 'model' | 'verify' | 'agent' | 'connect';
/** Every step, in order. A job runs these, without `agent` when the coding agent is installed. */
export const LOCAL_MODEL_INSTALL_STEPS: readonly LocalModelInstallStep[] = [
'runtime',
'model',
'verify',
'agent',
'connect',
];
/**
* Why a job failed, or (in state `installed`) what blocks a session. Each code has one
* sentence (`LOCAL_MODEL_ERROR_LABELS`) and one helpful next step (`LOCAL_MODEL_ERROR_NEXT`).
*
* | Code | Raised when |
* |---------------------------|--------------------------------------------------------------------------|
* | `runtime-download-failed` | the archive download still fails after its retries |
* | `runtime-damaged` | the archive's sha256 is not the pinned one (the file was deleted) |
* | `runtime-unpack-failed` | `/usr/bin/tar` failed |
* | `runtime-start-failed` | the runtime was started and never answered `/api/version` |
* | `runtime-stopped` | installed: the runtime crashed and was restarted 3 times, then given up |
* | `port-in-use` | another program listens on the stored port, or no port of the span is free |
* | `disk-full` | a write failed for lack of space |
* | `pull-failed` | the pull still ends on an error line or a cut stream after its retries |
* | `model-not-found` | the registry no longer has this model under this name |
* | `does-not-fit` | the measured budget rules the model out, or the runtime did not put all of it on the GPU |
* | `context-not-applied` | the runtime loaded the model with another context than the chosen one |
* | `load-failed` | the tiny chat that loads the model failed |
* | `no-tool-call` | the tool probe was answered without a tool call with parseable arguments |
* | `probe-unfinished` | the tool probe ended on its token limit, so it proves nothing |
* | `harness-failed` | the coding agent's install command failed, or its binary is still missing |
* | `endpoint-id-taken` | a saved endpoint with our id exists that this feature did not write |
* | `connect-failed` | the endpoint record or the settings flag could not be written |
* | `model-changed` | installed: the model's digest is not the one stored at install time |
* | `internal` | anything unexpected (the exception text is in `detail`) |
*/
export type LocalModelErrorCode =
| 'runtime-download-failed'
| 'runtime-damaged'
| 'runtime-unpack-failed'
| 'runtime-start-failed'
| 'runtime-stopped'
| 'port-in-use'
| 'disk-full'
| 'pull-failed'
| 'model-not-found'
| 'does-not-fit'
| 'context-not-applied'
| 'load-failed'
| 'no-tool-call'
| 'probe-unfinished'
| 'harness-failed'
| 'endpoint-id-taken'
| 'connect-failed'
| 'model-changed'
| 'internal';
/**
* The ONE action that helps after an error. The window draws one button for it.
* - `retry`: "Try again". `POST /api/local-model/resume`.
* - `other-port`: "Use another port". `POST /api/local-model/resume` with `newPort: true`
* (state `failed`), or `POST /api/local-model/runtime` with `action: 'start'` and
* `newPort: true` (state `installed`).
* - `next-model`: "Try <model>". Back to the chooser with `error.nextModel` preselected; the
* person confirms its sizes and a new `POST /api/local-model/install` replaces the job.
* Without a `nextModel` the window offers the chooser and Remove.
* - `start`: "Start". `POST /api/local-model/runtime` with `action: 'start'`.
* - `remove`: "Remove". `DELETE /api/local-model`.
*/
export type LocalModelNextStep = 'retry' | 'other-port' | 'next-model' | 'start' | 'remove';
/**
* What a client may do in the current state. The view's `actions` lists exactly the ones the
* server would accept right now, so the window shows a button only for those.
* - `install`: `POST /api/local-model/install`.
* - `cancel`: `POST /api/local-model/cancel`.
* - `resume`: `POST /api/local-model/resume`.
* - `start` and `stop`: `POST /api/local-model/runtime`.
* - `remove`: `DELETE /api/local-model`.
* - `start-session`: not a route of this feature: the window starts a session on the endpoint
* `endpointId` with the coding agent `cli`. Offered only while the runtime runs and nothing
* blocks a session.
*/
export type LocalModelAction = 'install' | 'cancel' | 'resume' | 'start' | 'stop' | 'remove' | 'start-session';
/** Whether the Mac has the developer tools (git, Python) a coding agent calls. */
export type LocalModelDeveloperTools =
/** `/usr/bin/xcode-select -p` succeeded. */
| 'present'
/** It failed: macOS will offer to install the tools the first time an agent needs one. */
| 'missing'
/** Not macOS, or the check did not answer in time. Nothing is said then. */
| 'unknown';
// ---------- Sentences ----------
/**
* Every sentence the server may put into the `install` object or a 409, in English (the i18n
* source language). Like `LOCAL_MODEL_LABELS`: the zh-CN dictionary in src/web/public/i18n.js
* has one entry per value, keyed by this exact text, and the i18n test lists them from here.
* A constant of its own, so the sentences of the recommendation and of the install can be
* listed apart.
*
* | Key | Slots (all text) |
* |------------------------|----------------------------------------------------------------------|
* | `errPortInUse` | `port`: the port number |
* | `errModelNotFound`, `errDoesNotFit`, `errLoadFailed`, `errNoToolCall`, `errProbeUnfinished` | `model`: the model's label |
* | `errContextNotApplied` | `model`; `actual`: the context the runtime reports; `wanted`: the chosen one |
* | `errHarnessFailed` | `cli`: the coding agent's label |
* | `errEndpointIdTaken` | `id`: the endpoint id this feature uses |
* | `conflictSizeChanged` | `size`: the server's own total as decimal GB |
* | `conflictSessionsOpen` | `sessions`: the session names, joined with ', ' |
*
* `developerToolsMissing` is not sent: the view carries `developerTools` and the window prints
* this sentence when it is `missing`. It is here so that its wording has one source.
*/
export const LOCAL_MODEL_INSTALL_LABELS = {
errRuntimeDownloadFailed:
'The download of the runtime stopped and could not be continued. Check the network connection, then try again.',
errRuntimeDamaged: 'The runtime download was damaged and has been deleted. Try again.',
errRuntimeUnpackFailed: 'The runtime could not be unpacked on this Mac.',
errRuntimeStartFailed: 'The runtime did not start on this Mac.',
errRuntimeStopped: 'The runtime keeps stopping, so Codeman no longer restarts it.',
errPortInUse: 'Port {port} is in use by another program.',
errDiskFull: 'The disk is full. Free up space, then try again.',
errPullFailed: 'The download keeps stopping. The network may be damaging data. Try again or use another network.',
errModelNotFound: '{model} is no longer offered under this name. Update Codeman or pick another model.',
errDoesNotFit: '{model} does not fit in the memory of this Mac after all.',
errContextNotApplied: '{model} loaded with a context of {actual} tokens instead of {wanted}.',
errLoadFailed: '{model} did not load on this Mac.',
errNoToolCall: '{model} loaded but did not call a tool, so a coding agent cannot work with it.',
errProbeUnfinished: 'The check of {model} did not finish.',
errHarnessFailed: '{cli} could not be installed.',
errEndpointIdTaken:
'A saved model connection named {id} already exists and was not created here. Rename or delete it in App Settings, then try again.',
errConnectFailed: 'Codeman could not save the connection to the model.',
errModelChanged: 'The model on this Mac changed since it was installed. Remove it and install it again.',
errInternal: 'The install stopped on an unexpected error.',
conflictJobRunning: 'An install is already running.',
conflictAlreadyInstalled: 'A model is already installed on this Mac. Remove it first.',
conflictSizeChanged: 'The download is {size} GB now. Check the sizes and confirm again.',
conflictSessionsOpen: 'Sessions are still using this model: {sessions}. Close them first.',
conflictWrongState: 'That is not possible in the current state.',
developerToolsMissing:
'Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer to install them the first time an agent needs one.',
} as const;
/** One of the texts in `LOCAL_MODEL_INSTALL_LABELS`. */
export type LocalModelInstallLabel = (typeof LOCAL_MODEL_INSTALL_LABELS)[keyof typeof LOCAL_MODEL_INSTALL_LABELS];
/** A sentence of the install: a text from `LOCAL_MODEL_INSTALL_LABELS` plus the values of its `{placeholder}` slots. */
export interface LocalModelInstallMessage {
text: LocalModelInstallLabel;
/** Slot values, already formatted as text. Absent when the text has no slot. */
vars?: Record<string, string>;
}
/** The sentence of each error code. */
export const LOCAL_MODEL_ERROR_LABELS: Record<LocalModelErrorCode, LocalModelInstallLabel> = {
'runtime-download-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDownloadFailed,
'runtime-damaged': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDamaged,
'runtime-unpack-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeUnpackFailed,
'runtime-start-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStartFailed,
'runtime-stopped': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStopped,
'port-in-use': LOCAL_MODEL_INSTALL_LABELS.errPortInUse,
'disk-full': LOCAL_MODEL_INSTALL_LABELS.errDiskFull,
'pull-failed': LOCAL_MODEL_INSTALL_LABELS.errPullFailed,
'model-not-found': LOCAL_MODEL_INSTALL_LABELS.errModelNotFound,
'does-not-fit': LOCAL_MODEL_INSTALL_LABELS.errDoesNotFit,
'context-not-applied': LOCAL_MODEL_INSTALL_LABELS.errContextNotApplied,
'load-failed': LOCAL_MODEL_INSTALL_LABELS.errLoadFailed,
'no-tool-call': LOCAL_MODEL_INSTALL_LABELS.errNoToolCall,
'probe-unfinished': LOCAL_MODEL_INSTALL_LABELS.errProbeUnfinished,
'harness-failed': LOCAL_MODEL_INSTALL_LABELS.errHarnessFailed,
'endpoint-id-taken': LOCAL_MODEL_INSTALL_LABELS.errEndpointIdTaken,
'connect-failed': LOCAL_MODEL_INSTALL_LABELS.errConnectFailed,
'model-changed': LOCAL_MODEL_INSTALL_LABELS.errModelChanged,
internal: LOCAL_MODEL_INSTALL_LABELS.errInternal,
};
/** The one helpful next step of each error code. */
export const LOCAL_MODEL_ERROR_NEXT: Record<LocalModelErrorCode, LocalModelNextStep> = {
'runtime-download-failed': 'retry',
'runtime-damaged': 'retry',
'runtime-unpack-failed': 'retry',
'runtime-start-failed': 'retry',
'runtime-stopped': 'start',
'port-in-use': 'other-port',
'disk-full': 'retry',
'pull-failed': 'retry',
'model-not-found': 'next-model',
'does-not-fit': 'next-model',
'context-not-applied': 'retry',
'load-failed': 'retry',
'no-tool-call': 'next-model',
'probe-unfinished': 'retry',
'harness-failed': 'retry',
'endpoint-id-taken': 'retry',
'connect-failed': 'retry',
'model-changed': 'remove',
internal: 'retry',
};
// ---------- The `install` object of the GET ----------
/**
* How far a download is. Present while the `runtime` step downloads the archive and while the
* `model` step pulls, once the total is known; kept on a paused or failed job as it stood.
* During the `model` step both byte figures come from the pull's own lines, never from the
* catalogue.
*/
export interface LocalModelProgress {
completedBytes: number;
totalBytes: number;
/** `completedBytes` as decimal GB text, no unit ('6.2'). */
completedGb: string;
/** `totalBytes` as decimal GB text, no unit ('23.1'). */
totalGb: string;
/** Download speed. Absent until it can be told (the first moments, a paused job). */
bytesPerSecond?: number;
/** `bytesPerSecond` as whole decimal megabytes per second, no unit ('58'). Present exactly when `bytesPerSecond` is. */
mbPerSecond?: string;
/** Seconds left at the current speed, rounded up. Absent when the speed is unknown or 0. */
secondsLeft?: number;
}
/** An error as the window shows it: the sentence, the one action that helps, and the raw detail. */
export interface LocalModelInstallError {
code: LocalModelErrorCode;
/** `LOCAL_MODEL_ERROR_LABELS[code]` with its slots filled. */
message: LocalModelInstallMessage;
/** `LOCAL_MODEL_ERROR_NEXT[code]`. */
next: LocalModelNextStep;
/**
* For `next-model`: the best model that still fits and is not the one that failed. Absent
* when there is none.
*/
nextModel?: { modelId: string; modelLabel: string };
/**
* Raw text for the person who wants to know more: the last lines of an installer, the
* message of a refused write, an exception. Not translated; printed with `textContent`.
*/
detail?: string;
}
/** The model of the job or of the installation. */
export interface LocalModelInstalledModel {
/** `LocalModelEntry.id`. */
id: string;
/** `LocalModelEntry.label`. */
label: string;
}
/** The runtime, as the consent screen needs it. */
export interface LocalModelRuntimeView {
/** `LocalModelRuntimePin.version`. */
version: string;
/** The pinned runtime is unpacked on disk. Then an install downloads only the model. */
installed: boolean;
/** What an install would download for the runtime: the archive size, 0 when `installed`. */
downloadBytes: number;
/** `downloadBytes` as decimal GB text, no unit ('0.2'). */
downloadGb: string;
/** What the unpacked runtime takes on disk, as decimal GB text, no unit ('0.6'). */
unpackedGb: string;
}
/** What the runtime reported about the loaded model when the job checked it. The facts of the Done line. */
export interface LocalModelLoadedFacts {
/** The context length in tokens the runtime loaded the model with. */
context: number;
/** Memory the loaded model took, in bytes (`size` of the runtime's process list). */
memoryBytes: number;
/** `memoryBytes` as decimal GB text, no unit. */
memoryGb: string;
/** All of it is on the GPU (`size_vram` equals `size`). Always true after a job that passed. */
onGpu: boolean;
}
/**
* The `install` object of `GET /api/local-model`: where the install stands and what can be
* done about it. The window polls it once a second while it is open; there is no SSE event.
*
* Which members are present, by state:
*
* | Member | idle | installing | paused | failed | installed |
* |-----------------------------------|------|------------|--------|--------|-----------|
* | `step`, `steps` | | yes | yes | yes | |
* | `progress` | | during a download | as it stood, when known | as it stood, when known | |
* | `error` | | | | yes | only when something blocks a session |
* | `model`, `cli`, `cliLabel`, `context` | | yes | yes | yes | yes |
* | `port` | | once chosen | once chosen | once chosen | yes |
* | `loaded`, `endpointId` | | | | | yes |
*
* `running`, `developerTools`, `runtime`, `folder` and `actions` are always present.
*/
export interface LocalModelInstallView {
state: LocalModelInstallState;
/** The step the job is in, or stopped in. */
step?: LocalModelInstallStep;
/** The steps of THIS job in order, for the step list (`agent` is absent when the coding agent was already installed). */
steps?: LocalModelInstallStep[];
progress?: LocalModelProgress;
error?: LocalModelInstallError;
model?: LocalModelInstalledModel;
/** The coding agent: a CLI registry id, as data. */
cli?: string;
/** The coding agent's display label ('OpenCode'). */
cliLabel?: string;
/** The context length in tokens the runtime is started with. */
context?: number;
/** The loopback port of the runtime. */
port?: number;
/** The runtime child Codeman started is alive right now. */
running: boolean;
developerTools: LocalModelDeveloperTools;
runtime: LocalModelRuntimeView;
/** The absolute folder everything of this feature is kept in, for the "where it goes" line. */
folder: string;
loaded?: LocalModelLoadedFacts;
/** The id of the saved endpoint a session is started on. */
endpointId?: string;
/** What the server accepts right now, see `LocalModelAction`. */
actions: LocalModelAction[];
}
/**
* The `data` of `GET /api/local-model`. Admin only in multi-user mode.
*
* `install` is optional in the type only until the route emits it: from the stage that ships
* the install verbs on it is always present.
*/
export interface LocalModelStatus {
hardware: HardwareFacts;
recommendation: Recommendation;
/** `LocalModelCatalog.updated`. */
catalogUpdated: string;
install?: LocalModelInstallView;
}
// ---------- Request bodies and answers ----------
/**
* Body of `POST /api/local-model/install`. The three choices must name a row the server offers
* right now: a model of `recommendation.pick` or `recommendation.alternatives`, at that
* candidate's `context`, with one of its `harnesses`. Anything else is a 400 before any
* download.
*/
export interface LocalModelInstallRequest {
/** `Candidate.modelId`. */
modelId: string;
/** `CandidateHarness.cli`. */
cli: string;
/** `Candidate.context`. */
context: number;
/**
* The total the person saw and confirmed: `Candidate.installBytes`. The server compares it
* with its own total and answers 409 `size-changed` when they differ, so a consent is bound
* to the size that was shown.
*/
acknowledgedBytes: number;
}
/** Body of `POST /api/local-model/resume`. May be absent or empty. */
export interface LocalModelResumeRequest {
/** True: choose another port first (the answer to `port-in-use`). */
newPort?: boolean;
}
/** Body of `POST /api/local-model/runtime`. */
export interface LocalModelRuntimeRequest {
/** `stop` also keeps the runtime from being started at the next boot; `start` undoes that. */
action: 'start' | 'stop';
/** With `start`: choose another port first (the answer to `port-in-use`), and move the saved endpoint to it. */
newPort?: boolean;
}
/** Body of `DELETE /api/local-model`. May be absent or empty. */
export interface LocalModelRemoveRequest {
/** True: delete the downloaded model too. Default false: the model stays for a later install. */
removeModel?: boolean;
}
/** What every verb answers on success: the view after the verb. */
export type LocalModelVerbAnswer = LocalModelInstallView;
/**
* Why a verb was refused with 409.
* - `job-running`: a job is running (a second install, a resume, a remove or a runtime action).
* - `already-installed`: install while a model is installed. Remove it first.
* - `size-changed`: `acknowledgedBytes` is not the server's total; `expectedBytes` and
* `expectedGb` carry that total, so the window can show it and ask again.
* - `sessions-open`: remove while live sessions use the endpoint; `sessions` names them.
* - `wrong-state`: the verb makes no sense in the current state (see the route table above).
*/
export type LocalModelConflictReason =
| 'job-running'
| 'already-installed'
| 'size-changed'
| 'sessions-open'
| 'wrong-state';
/** The `data` of a 409 of an install verb. */
export interface LocalModelConflict {
reason: LocalModelConflictReason;
/** The sentence for the window (`conflict...` of `LOCAL_MODEL_INSTALL_LABELS`). */
message: LocalModelInstallMessage;
/** `size-changed`: the server's own total in bytes. */
expectedBytes?: number;
/** `size-changed`: `expectedBytes` as decimal GB text, no unit. */
expectedGb?: string;
/** `sessions-open`: the names of the live sessions that use the endpoint. */
sessions?: string[];
}
// ---------- The state file ----------
/**
* `local-model.json` in the data dir: everything that must survive a restart. Mode 0600,
* written through a temp file and a rename, by src/local-model/state-store.ts alone.
*
* Rules:
* - It holds NO path. Every path of this feature is built from `dataPath()` constants
* (`localModelPaths()`), so a state file can never point a delete somewhere else.
* - Reading is total: a missing file, a file that does not parse or one with another `schema`
* reads as `{ schema: 1, state: 'idle' }`.
* - A file read at boot with state `installing` is a job a restart interrupted: it becomes
* `paused` before anything else looks at it.
*/
export interface LocalModelStateFile {
schema: 1;
state: LocalModelInstallState;
/** The step the job is in or stopped in. Absent in `idle` and `installed`. */
step?: LocalModelInstallStep;
/** The steps of the job, in order. Absent in `idle` and `installed`. */
steps?: LocalModelInstallStep[];
/** State `failed`: why. State `installed`: absent (a blocking problem is found live, never stored). */
error?: {
code: LocalModelErrorCode;
/** The slot values of the code's sentence. */
vars?: Record<string, string>;
detail?: string;
/** For a `next-model` code: the model to offer, when the job knew one. */
nextModelId?: string;
};
/** The model of the job or of the installation. */
model?: {
id: string;
label: string;
/** The context length in tokens the person chose. */
context: number;
/** The total the pull reported, once known. */
pullBytes?: number;
/** The digest `/api/tags` reported after the pull. Compared again at boot and before a session starts. */
digest?: string;
};
/** The coding agent of the job or of the installation. */
cli?: { id: string; label: string };
/** The runtime, once its port was chosen. */
runtime?: {
/** The pinned version that was unpacked. */
version: string;
/** The loopback port, chosen once by binding. */
port: number;
/** The pid of the child Codeman last started. Used at boot to tell our own orphan from another program. */
pid?: number;
/** False after Stop: the runtime is not started at boot. */
enabled: boolean;
/** The device total the runtime printed when it started, in bytes. */
measuredTotalBytes?: number;
};
/** How far the last download was when the job paused or failed. A restart may leave it absent. */
progress?: { completedBytes: number; totalBytes: number };
/** What the runtime reported when the job checked the loaded model. */
loaded?: { context: number; sizeBytes: number; sizeVramBytes: number };
/** Present once the `connect` step wrote: what uninstall must undo. */
connection?: {
/** The id of the endpoint record this feature wrote. */
endpointId: string;
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
flagWasEnabled: boolean;
};
/** ISO time of the last write. */
updatedAt?: string;
}
src/types/local-model.ts
+184 -21
View File
@@ -1,9 +1,12 @@
/**
* @fileoverview Types of the hardware-aware local model recommendation: what the server sees of
* this Mac, the catalogue of models and harnesses, and the answer of `GET /api/local-model`.
* this Mac, the catalogue of models and harnesses, and the recommendation part of the answer of
* `GET /api/local-model`.
*
* This stage is read-only: Codeman looks at the Mac and recommends. It downloads nothing,
* installs nothing and writes no file.
* Everything here is the READ side: Codeman looks at the Mac and recommends. The types of the
* install (the job states, the progress, the `install` object of the GET, the request bodies,
* the state file, the runtime pin) and `LocalModelStatus`, the whole `data` of the GET, are in
* src/types/local-model-install.ts. Both files go out through the barrel (`../types.js`).
*
* Who produces what:
* - `HardwareFacts`: src/local-model/hardware.ts (one bounded `sysctl` spawn, never throws).
@@ -11,7 +14,11 @@
* - `HarnessFact`: src/local-model/harness-facts.ts (server only, read from the CLI registry at
* call time; it holds a function, so it never goes over HTTP).
* - `Recommendation`, `Candidate`, `DeviceBudget`: src/local-model/recommend.ts (pure).
* - `LocalModelStatus`: src/web/routes/local-model-routes.ts (the `data` of the GET).
*
* Fields marked "Stage 2" were added for the install. They are OPTIONAL in the type only so
* that the stage 1 catalogue and `recommend()` keep compiling until their owner fills them:
* once `recommend()` and the catalogue rows carry them, they become required, and a consumer
* (the chooser window, the install job) may rely on every one of them.
*
* Three rules every producer and consumer follows:
*
@@ -100,6 +107,13 @@ export interface TestedOn {
cli: string;
/** ISO date of the test ('2026-11-06'). */
date: string;
/**
* Stage 2. The generation speed measured in that test, in tokens per second (67.7 for the
* 2026-11-06 run on an Apple M5 Pro with 48 GB). A candidate that matches this entry reports
* this figure as `Candidate.tokensPerSecond` in place of the estimate from the chip's memory
* bandwidth: a measurement on the same Mac beats a formula.
*/
tokensPerSecond?: number;
}
/** One model of the catalogue. */
@@ -112,10 +126,42 @@ export interface LocalModelEntry {
license: string;
/** Parameter count in billions. When no model that fits has 12 or more, the verdict is `limited`, never `ok`. */
paramsB: number;
/** Size of the weights in bytes: both the download and what they take in memory. */
/**
* Size of the weights in bytes: both the download and what they take in memory.
*
* Stage 2: the size of the build the runtime really downloads ON A MAC. That is not always
* the sum of the registry manifest: `qwen3.6:35b-a3b` is an MLX build of 23,105,798,956 bytes
* on Apple silicon, 2% more than the manifest's 22,621,314,161. A row whose Mac size nobody
* measured keeps the manifest sum and says so in `figures`.
*/
bytes: number;
/** Bytes of KV cache one token of context costs. */
/**
* Stage 1 figure: the bytes of KV cache one token of context costs, COMPUTED from the model's
* config. Measured on a Mac it was too low by almost half (20 KiB against about 37 KB held),
* so the need is computed from `heldBytesPerToken` once that is filled. Kept only until
* `recommend()` stops reading it; it is deleted together with `promptCacheTiers` then.
*/
kvBytesPerToken: number;
/**
* Stage 2. Bytes that stay HELD in memory per token of context once the token was processed
* (measured on the MLX runner: 36,000 to 38,000, so about 37,000).
*/
heldBytesPerToken?: number;
/** Stage 2. What the runner holds beside the weights and the context (measured: about 3 GB). */
overheadBytes?: number;
/**
* Stage 2. The TRANSIENT extra the runner takes while it processes a long prompt, on top of
* what stays held (measured: about 5 GB). The fit is judged on the peak:
* `bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`. For
* `qwen3.6:35b-a3b` at 131072 tokens that is about 36 GB, which is tight on a 48 GB Mac.
*/
peakExtraBytes?: number;
/**
* Stage 2. Where `bytes` and the three memory figures above come from: `measured` on a Mac by
* us, or `estimated` (the manifest size, and the per-token, overhead and peak figures of the
* measured row applied to this one).
*/
figures?: 'measured' | 'estimated';
/** Context lengths (tokens) the model may be run at, largest first. */
contexts: number[];
/** Preference among models that fit: the HIGHER rank wins. A judgement, not a measurement. */
@@ -194,6 +240,13 @@ export interface HarnessFact {
installed: boolean;
/** Not installed, but the registry has an install command for this platform that needs no administrator rights. */
installable: boolean;
/**
* Stage 2. The literal install command the install job would run for this CLI: the STOCK
* command of the CLI registry for this platform, never one a `clis.json` override supplies.
* Present exactly when `installable` is true. The consent screen prints it
* (`CandidateHarness.installCommand`).
*/
installCommand?: string;
/** Whether this CLI can be launched with that model id (its command-line pattern accepts it). */
acceptsModelId: (modelId: string) => boolean;
}
@@ -223,18 +276,54 @@ export type SpeedGrade = 'ok' | 'slow' | 'unknown';
*/
export type NoneReason = 'not-macos' | 'intel' | 'memory' | 'disk' | 'macos' | 'no-harness';
/** Where the device budget came from: a formula (`estimated`) or `iogpu.wired_limit_mb` above 0 (`wired-limit`). */
export type BudgetSource = 'estimated' | 'wired-limit';
/**
* Where the device budget came from.
* - `estimated`: a formula over the installed memory. All there is before the runtime ran.
* - `wired-limit`: `iogpu.wired_limit_mb` above 0.
* - `measured` (stage 2): the device total the runtime itself printed when it started
* (`RecommendOptions.measuredTotalBytes`). The chooser says "estimated" next to the memory
* bar for the first value only.
*/
export type BudgetSource = 'estimated' | 'wired-limit' | 'measured';
/** The memory a model may use on this Mac. */
export interface DeviceBudget {
/** L: the wired limit when above 0, else 2/3 of memory below 36 GB and 3/4 from 36 GB. */
/**
* L: the measured device total when there is one, else the wired limit when above 0, else
* 2/3 of memory below 36 GB and 3/4 from 36 GB.
*/
limitBytes: number;
/** B = min(L - 1 GiB, memory - reserve). What `Candidate.budgetShare` is measured against. */
/**
* What `Candidate.budgetShare` is measured against. Estimated or from the wired limit:
* B = min(L - 1 GiB, memory - reserve). Measured: B = min(L, memory - reserve), without the
* 1 GiB, because the runtime's figure is already what it will really use (37.4 GiB on the
* 48 GB test Mac, which reads '40.2').
*/
budgetBytes: number;
source: BudgetSource;
}
/**
* Stage 2. What the caller of `recommend()` knows beyond the hardware probe. Both members are
* optional and an empty object is the stage 1 behaviour.
*/
export interface RecommendOptions {
/**
* The device total the runtime reported in its log line "inference compute"
* (`total="37.4 GiB"`), in bytes: `parseInferenceComputeTotal()` in
* src/local-model/runtime.ts. Present: the budget is `measured`. The install job passes it
* right after the runtime started, before the model is downloaded, and the GET passes the
* stored figure from then on.
*/
measuredTotalBytes?: number;
/**
* What an install adds to every candidate's download beside the model: the size of the
* pinned runtime archive while the runtime is not installed, 0 once it is. It is the
* difference between `Candidate.installBytes` and `Candidate.downloadBytes`. Default 0.
*/
runtimeDownloadBytes?: number;
}
/**
* Every sentence and label the server may put into the payload, in English (the i18n source
* language). The zh-CN dictionary in src/web/public/i18n.js has one entry per value, keyed by
@@ -315,6 +404,60 @@ export interface LocalModelMessage {
vars?: Record<string, string>;
}
/**
* Stage 2. The one tag a model row of the chooser carries, by what it says.
* - `tested`: `Candidate.tested` is true. Replaces every other tag.
* - `best`: the pick. The one tag drawn in the accent colour.
* - `tight`: `Candidate.fit` is `tight`.
* - `slow`: `Candidate.speed` is `slow`.
* - `small`: the model has fewer than 12 billion parameters.
* A row gets the FIRST kind of this list that applies and no tag when none does, so a pick
* that is tight says "Best fit" (the memory bar already shows how full it is).
*/
export type CandidateTagKind = 'tested' | 'best' | 'tight' | 'slow' | 'small';
/**
* The text of each tag, in English (the i18n source language), like `LOCAL_MODEL_LABELS`. A
* constant of its own so that a consumer can list the tag texts apart from the sentences.
*/
export const LOCAL_MODEL_TAG_LABELS = {
tested: 'Tested on a Mac like this',
best: 'Best fit',
tight: 'Tight',
slow: 'Slow',
small: 'Small model',
} as const satisfies Record<CandidateTagKind, string>;
/** One of the texts in `LOCAL_MODEL_TAG_LABELS`. */
export type LocalModelTagLabel = (typeof LOCAL_MODEL_TAG_LABELS)[CandidateTagKind];
/** Stage 2. The tag of a model row: what kind it is (for the styling) and the text to print. */
export interface CandidateTag {
kind: CandidateTagKind;
/** `LOCAL_MODEL_TAG_LABELS[kind]`. */
label: LocalModelTagLabel;
}
/**
* Stage 2. One coding agent a candidate's model can be run with on this Mac, for the "Runs
* with" select of the chooser. Everything a browser needs is here, so it compares no CLI id.
*/
export interface CandidateHarness {
/** The CLI registry id, as data: what `POST /api/local-model/install` is sent as `cli`. */
cli: string;
/** The registry's display label ('OpenCode'). */
cliLabel: string;
/** False: the install job installs it first (it is installable, or it would not be listed). */
installed: boolean;
/**
* The literal command the install job would run to install it (`HarnessFact.installCommand`).
* Present exactly when `installed` is false. The consent screen prints it as it is.
*/
installCommand?: string;
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
}
/** One model at one context with one harness, measured against this Mac. Plain JSON. */
export interface Candidate {
/** `LocalModelEntry.id`. */
@@ -333,7 +476,12 @@ export interface Candidate {
cliInstalled: boolean;
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
/** F = weights + kvBytesPerToken * context + 1 GiB + prompt cache. */
/**
* What the model needs in memory at this context. Stage 1: F = weights + kvBytesPerToken *
* context + 1 GiB + prompt cache. Stage 2: the PEAK, `bytes + heldBytesPerToken * context +
* overheadBytes + peakExtraBytes` of the catalogue row. The memory bar of the chooser is
* filled to `budgetShare`, which is this over the budget.
*/
needBytes: number;
/** `needBytes` as decimal GB text, no unit. */
needGb: string;
@@ -350,10 +498,33 @@ export interface Candidate {
speed: SpeedGrade;
/** `speedSlow` or `speedUnknown`. Absent when `speed` is `ok`. */
speedLabel?: LocalModelLabel;
/** Estimated generation speed, rounded to one decimal. Present exactly when `speed` is not `unknown`. `speed` is decided on the unrounded figure. */
/**
* Generation speed in tokens per second, rounded to one decimal. Present exactly when `speed`
* is not `unknown`; `speed` is decided on the unrounded figure. An estimate from the chip's
* memory bandwidth, except (stage 2) for a candidate that is `tested`, which reports the
* speed measured in that test (`TestedOn.tokensPerSecond`).
*/
tokensPerSecond?: number;
/** A `TestedOn` entry of the model matches this Mac (same chip, same `memClassGb`) and this harness. */
tested: boolean;
/** Stage 2. `LocalModelEntry.figures` of the model: whether `needBytes` rests on a measurement of this model. */
needSource?: 'measured' | 'estimated';
/** Stage 2. The row's one tag, see `CandidateTagKind`. Absent when no kind applies. */
tag?: CandidateTag;
/**
* Stage 2. Every coding agent this model can be run with at `context` on this Mac, best
* first. The first entry is the one the flat fields above name (`cli`, `cliLabel`,
* `cliInstalled`, `slowFirstTurn`). Never empty.
*/
harnesses?: CandidateHarness[];
/**
* Stage 2. The whole download a click on Install starts for this row: `downloadBytes` plus
* `RecommendOptions.runtimeDownloadBytes`. It is the server's own total, and exactly the
* number `POST /api/local-model/install` must be sent as `acknowledgedBytes`.
*/
installBytes?: number;
/** Stage 2. `installBytes` as decimal GB text, no unit. */
installGb?: string;
}
/**
@@ -385,12 +556,4 @@ export interface Recommendation {
budgetSource: BudgetSource;
}
// ---------- API ----------
/** The `data` of `GET /api/local-model`. Read-only; admin only in multi-user mode. */
export interface LocalModelStatus {
hardware: HardwareFacts;
recommendation: Recommendation;
/** `LocalModelCatalog.updated`. */
catalogUpdated: string;
}
// `LocalModelStatus`, the `data` of `GET /api/local-model`, is in src/types/local-model-install.ts.