mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 10:39:32 +01:00
Merge feat/local-model-recommendation: recommend and install a local coding model on this Mac (experimental)
Conflicts resolved as on test/desktop-macbook: config/test-suites.ts keeps both browser tests (desktop-connect-page and local-model-ui), and the CLAUDE.md state-files line keeps #778's cli-installed entry with the local model's local-model.json and local-model/ entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed
commit
e8a63ede00
80 files changed
+31268
-261
No files matched your search
@@ -29,6 +29,8 @@
|
||||
* | plan | PlanItem, PlanTaskStatus, TddPhase | In-memory → `GET /api/sessions/:id/plan/tasks` |
|
||||
* | orchestrator | OrchestratorState, OrchestratorPlan, OrchestratorConfig, OrchestratorPersistState | `~/.codeman/state.json` → `GET /api/orchestrator/status` |
|
||||
* | mcp-sync | McpSyncResult, McpSyncTargetResult | Other CLIs' own config files → `GET`/`POST /api/mcp-sync` |
|
||||
* | local-model | HardwareFacts, LocalModelCatalog, Candidate, Recommendation, RecommendOptions, LOCAL_MODEL_LABELS | In-memory only (nothing persisted) → `GET /api/local-model` |
|
||||
* | local-model-install | LocalModelStatus, LocalModelInstallView, LocalModelStateFile, LocalModelRuntimePin, LOCAL_MODEL_INSTALL_LABELS | `~/.codeman/local-model.json` → `GET /api/local-model` and its install verbs |
|
||||
*
|
||||
* ## Cross-domain relationship map
|
||||
*
|
||||
@@ -77,3 +79,5 @@ export * from './webview.js';
|
||||
export * from './intent.js';
|
||||
export * from './foreign-tmux.js';
|
||||
export * from './mcp-sync.js';
|
||||
export * from './local-model.js';
|
||||
export * from './local-model-install.js';
|
||||
@@ -0,0 +1,638 @@
|
||||
/**
|
||||
* @fileoverview Types of the local model INSTALL: the runtime pin, the job's states, steps and
|
||||
* error codes, the progress, the `install` object of `GET /api/local-model`, the bodies and
|
||||
* answers of the install verbs, and the shape of the state file.
|
||||
*
|
||||
* The read side (hardware, catalogue, recommendation) is in src/types/local-model.ts. This
|
||||
* file holds `LocalModelStatus`, the whole `data` of the GET, because it joins the two.
|
||||
*
|
||||
* Who produces what:
|
||||
* - `LocalModelRuntimePin`: a compiled constant in src/local-model/runtime.ts. No file, no
|
||||
* setting and no request can change which binary is downloaded.
|
||||
* - `LocalModelStateFile`: src/local-model/state-store.ts, the ONE writer of
|
||||
* `local-model.json` (mode 0600, temp file + rename).
|
||||
* - `LocalModelInstallView`: src/local-model/install-job.ts (`buildInstallView()`, pure), from
|
||||
* the state file plus what is live (the runtime child, the developer tools check).
|
||||
* - `LocalModelStatus` and the verb answers: src/web/routes/local-model-routes.ts.
|
||||
*
|
||||
* The HTTP contract. Every route is admin only in multi-user mode (403 `FORBIDDEN`) and
|
||||
* EXPERIMENTAL like the GET (docs/versioning-policy.md). Every verb answers
|
||||
* `{ success: true, data: LocalModelInstallView }` with the view AFTER the verb.
|
||||
*
|
||||
* | Route | Body | Refusals |
|
||||
* |----------------------------------|-----------------------------|-------------------------------------------------------|
|
||||
* | `GET /api/local-model` | none | none (a failed probe is an answer) |
|
||||
* | `POST /api/local-model/install` | `LocalModelInstallRequest` | 400 not a row the server offers, or a saved endpoint |
|
||||
* | | | already has the reserved id; 409 `job-running`, |
|
||||
* | | | `already-installed`, `size-changed` |
|
||||
* | `POST /api/local-model/cancel` | none | 409 `wrong-state` (no job is running) |
|
||||
* | `POST /api/local-model/resume` | `LocalModelResumeRequest` | 409 `job-running`, `wrong-state` (not paused or |
|
||||
* | | | failed), `sessions-open` (another port while live |
|
||||
* | | | sessions use the model) |
|
||||
* | `POST /api/local-model/runtime` | `LocalModelRuntimeRequest` | 409 `wrong-state` (not installed), `job-running`, |
|
||||
* | | | `sessions-open` (stop, or another port, while live |
|
||||
* | | | sessions use the model) |
|
||||
* | `DELETE /api/local-model` | `LocalModelRemoveRequest` | 409 `job-running`, `sessions-open` |
|
||||
*
|
||||
* A 409 is the error envelope plus `data: LocalModelConflict`:
|
||||
* `{ success: false, error, errorCode: 'CONFLICT', data: { reason, message, ... } }`. A 409
|
||||
* WITHOUT `data` is a write of `settings.json` or of the endpoint store that was refused
|
||||
* because the file does not parse; `error` names the fix. A 400 the service names carries its
|
||||
* sentence the same way: `{ success: false, error, errorCode: 'INVALID_INPUT', data: {
|
||||
* message } }` (`LocalModelRefusal`), so the window can print it in its own language.
|
||||
*
|
||||
* The rules of src/types/local-model.ts hold here too:
|
||||
* 1. A size a person reads travels twice: as bytes and as decimal GB text from `formatGb()`
|
||||
* (`completedGb`, `totalGb`, `downloadGb`). The browser prints the text and adds or
|
||||
* divides nothing for display. `mbPerSecond` is the one other unit, decimal MB per second.
|
||||
* 2. Every sentence the server chooses comes from `LOCAL_MODEL_INSTALL_LABELS`, with its
|
||||
* numbers and names in `vars`. `{name}` is taken by the translator, so no slot has that name.
|
||||
* 3. A CLI id is data (`LocalModelInstallView.cli`): the browser passes it on and compares it
|
||||
* with nothing. What a person reads is `cliLabel`.
|
||||
*/
|
||||
|
||||
import type { HardwareFacts, Recommendation } from './local-model.js';
|
||||
|
||||
// ---------- The runtime pin ----------
|
||||
|
||||
/**
|
||||
* The one runtime build Codeman downloads and runs. A compiled constant
|
||||
* (`OLLAMA_RUNTIME_PIN` in src/local-model/runtime.ts): the hash is compared before anything
|
||||
* is unpacked, so the pin and the hash must come from the same reviewed source file.
|
||||
*/
|
||||
export interface LocalModelRuntimePin {
|
||||
/** The runtime's family. One value today. */
|
||||
id: 'ollama';
|
||||
/** The release ('0.40.15'). `GET /api/version` of the running runtime must answer it. */
|
||||
version: string;
|
||||
/** Where the archive is downloaded from. https, a fixed release asset. */
|
||||
url: string;
|
||||
/** Size of the archive in bytes. A body that grows past it is refused before the hash is finished. */
|
||||
bytes: number;
|
||||
/** Lowercase hex sha256 of the archive. */
|
||||
sha256: string;
|
||||
/** What the unpacked folder takes on disk, in bytes (measured: about 579 MB). */
|
||||
unpackedBytes: number;
|
||||
/** The oldest macOS major version the runtime supports. */
|
||||
minMacosMajor: number;
|
||||
}
|
||||
|
||||
// ---------- States, steps, codes ----------
|
||||
|
||||
/**
|
||||
* Where the install stands. The verb of the flow is the same throughout: install, installing,
|
||||
* installed.
|
||||
* - `idle`: nothing is installed and no job exists.
|
||||
* - `installing`: the one job is running; `step` says where.
|
||||
* - `paused`: the job stopped at `step` because of Cancel or a server restart. It never
|
||||
* continues without a click.
|
||||
* - `failed`: the job stopped at `step` with `error`.
|
||||
* - `installed`: the model is connected. Whether the runtime runs is `running`, not a state.
|
||||
* The state table (what is on disk, what a restart and Cancel do, the allowed actions) is
|
||||
* `INSTALL_STATE_TABLE` in src/local-model/install-job.ts.
|
||||
*/
|
||||
export type LocalModelInstallState = 'idle' | 'installing' | 'paused' | 'failed' | 'installed';
|
||||
|
||||
/**
|
||||
* The steps of a job, in the order they run.
|
||||
* - `runtime`: get the pinned runtime (download, compare the hash, unpack), start it, read
|
||||
* the measured memory budget and judge the chosen model against it.
|
||||
* - `model`: download the model (the runtime's pull).
|
||||
* - `verify`: load the model and check it: one tiny chat, the memory and context the runtime
|
||||
* reports, one tool call.
|
||||
* - `agent`: install the coding agent. Left out of a job whose coding agent is installed.
|
||||
* - `connect`: save the connection (the endpoint record and the settings flag).
|
||||
*/
|
||||
export type LocalModelInstallStep = 'runtime' | 'model' | 'verify' | 'agent' | 'connect';
|
||||
|
||||
/** Every step, in order. A job runs these, without `agent` when the coding agent is installed. */
|
||||
export const LOCAL_MODEL_INSTALL_STEPS: readonly LocalModelInstallStep[] = [
|
||||
'runtime',
|
||||
'model',
|
||||
'verify',
|
||||
'agent',
|
||||
'connect',
|
||||
];
|
||||
|
||||
/**
|
||||
* Why a job failed, or (in state `installed`) what blocks a session. Each code has one
|
||||
* sentence (`LOCAL_MODEL_ERROR_LABELS`) and one helpful next step (`LOCAL_MODEL_ERROR_NEXT`).
|
||||
*
|
||||
* | Code | Raised when |
|
||||
* |---------------------------|--------------------------------------------------------------------------|
|
||||
* | `runtime-download-failed` | the archive download still fails after its retries |
|
||||
* | `runtime-damaged` | the archive's sha256 is not the pinned one (the file was deleted) |
|
||||
* | `runtime-unpack-failed` | `/usr/bin/tar` failed |
|
||||
* | `runtime-start-failed` | the runtime was started and never answered `/api/version`. Installed: |
|
||||
* | | the last start failed (the detail says why), and Start tries again |
|
||||
* | `runtime-stopped` | the runtime crashed and was restarted 3 times, then given up: on an |
|
||||
* | | installed model, or during a job's download |
|
||||
* | `runtime-not-running` | installed: the runtime is not running (the person stopped it). Only a |
|
||||
* | | refused session start says it; the view shows no error for a stopped runtime |
|
||||
* | `port-in-use` | another program listens on the stored port, or no port of the span is free |
|
||||
* | `disk-full` | a write failed for lack of space |
|
||||
* | `pull-failed` | the pull still ends on an error line or a cut stream after its retries |
|
||||
* | `model-not-found` | the registry no longer has this model under this name |
|
||||
* | `does-not-fit` | the measured budget rules the model out, or the runtime did not put all of it on the GPU |
|
||||
* | `context-not-applied` | the runtime loaded the model with another context than the chosen one |
|
||||
* | `load-failed` | the tiny chat that loads the model failed |
|
||||
* | `no-tool-call` | the tool probe was answered without a tool call with parseable arguments |
|
||||
* | `probe-unfinished` | the tool probe ended on its token limit, so it proves nothing |
|
||||
* | `harness-failed` | the coding agent's install command failed, or its binary is still missing |
|
||||
* | `endpoint-id-taken` | a saved endpoint with our id exists that this feature did not write |
|
||||
* | `connect-failed` | the endpoint record or the settings flag could not be written |
|
||||
* | `model-changed` | installed: the model's digest is not the one stored at install time |
|
||||
* | `internal` | anything unexpected (the exception text is in `detail`) |
|
||||
*/
|
||||
export type LocalModelErrorCode =
|
||||
| 'runtime-download-failed'
|
||||
| 'runtime-damaged'
|
||||
| 'runtime-unpack-failed'
|
||||
| 'runtime-start-failed'
|
||||
| 'runtime-stopped'
|
||||
| 'runtime-not-running'
|
||||
| 'port-in-use'
|
||||
| 'disk-full'
|
||||
| 'pull-failed'
|
||||
| 'model-not-found'
|
||||
| 'does-not-fit'
|
||||
| 'context-not-applied'
|
||||
| 'load-failed'
|
||||
| 'no-tool-call'
|
||||
| 'probe-unfinished'
|
||||
| 'harness-failed'
|
||||
| 'endpoint-id-taken'
|
||||
| 'connect-failed'
|
||||
| 'model-changed'
|
||||
| 'internal';
|
||||
|
||||
/**
|
||||
* The ONE action that helps after an error. The window draws one button for it.
|
||||
* - `retry`: "Try again". `POST /api/local-model/resume`.
|
||||
* - `other-port`: "Use another port". `POST /api/local-model/resume` with `newPort: true`
|
||||
* (state `failed`), or `POST /api/local-model/runtime` with `action: 'start'` and
|
||||
* `newPort: true` (state `installed`).
|
||||
* - `next-model`: "Try <model>". Back to the chooser with `error.nextModel` preselected; the
|
||||
* person confirms its sizes and a new `POST /api/local-model/install` replaces the job.
|
||||
* Without a `nextModel` the window offers the chooser and Remove.
|
||||
* - `start`: "Start". `POST /api/local-model/runtime` with `action: 'start'`.
|
||||
* - `remove`: "Remove". `DELETE /api/local-model`.
|
||||
*
|
||||
* `LOCAL_MODEL_ERROR_NEXT` is the default of a code. The view adapts it to the state the
|
||||
* error is shown in, so the button is always one the server accepts there: a code whose
|
||||
* default is `start` says `retry` on a failed job (there is no Start verb for a job), and
|
||||
* `runtime-start-failed` says `start` on an installed model (there is nothing to resume).
|
||||
*/
|
||||
export type LocalModelNextStep = 'retry' | 'other-port' | 'next-model' | 'start' | 'remove';
|
||||
|
||||
/**
|
||||
* What a client may do in the current state. The view's `actions` lists exactly the ones the
|
||||
* server would accept right now, so the window shows a button only for those.
|
||||
* - `install`: `POST /api/local-model/install`.
|
||||
* - `cancel`: `POST /api/local-model/cancel`.
|
||||
* - `resume`: `POST /api/local-model/resume`.
|
||||
* - `start` and `stop`: `POST /api/local-model/runtime`.
|
||||
* - `remove`: `DELETE /api/local-model`.
|
||||
* - `start-session`: not a route of this feature: the window starts a session on the endpoint
|
||||
* `endpointId` with the coding agent `cli`. Offered only while the runtime runs and nothing
|
||||
* blocks a session.
|
||||
*/
|
||||
export type LocalModelAction = 'install' | 'cancel' | 'resume' | 'start' | 'stop' | 'remove' | 'start-session';
|
||||
|
||||
/** Whether the Mac has the developer tools (git, Python) a coding agent calls. */
|
||||
export type LocalModelDeveloperTools =
|
||||
/** `/usr/bin/xcode-select -p` succeeded. */
|
||||
| 'present'
|
||||
/** It failed: macOS will offer to install the tools the first time an agent needs one. */
|
||||
| 'missing'
|
||||
/** Not macOS, or the check did not answer in time. Nothing is said then. */
|
||||
| 'unknown';
|
||||
|
||||
// ---------- Sentences ----------
|
||||
|
||||
/**
|
||||
* Every sentence the server may put into the `install` object or a 409, in English (the i18n
|
||||
* source language). Like `LOCAL_MODEL_LABELS`: the zh-CN dictionary in src/web/public/i18n.js
|
||||
* has one entry per value, keyed by this exact text, and the i18n test lists them from here.
|
||||
* A constant of its own, so the sentences of the recommendation and of the install can be
|
||||
* listed apart.
|
||||
*
|
||||
* | Key | Slots (all text) |
|
||||
* |------------------------|----------------------------------------------------------------------|
|
||||
* | `errPortInUse` | `port`: the port number |
|
||||
* | `errModelNotFound`, `errDoesNotFit`, `errLoadFailed`, `errNoToolCall`, `errProbeUnfinished` | `model`: the model's label |
|
||||
* | `errContextNotApplied` | `model`; `actual`: the context the runtime reports; `wanted`: the chosen one |
|
||||
* | `errHarnessFailed` | `cli`: the coding agent's label |
|
||||
* | `errEndpointIdTaken` | `id`: the endpoint id this feature uses |
|
||||
* | `refusedAgentNotInstallable` | `cli`: the coding agent's label |
|
||||
* | `refusedOtherModel` | `model`: the label of the model that is installed |
|
||||
* | `conflictSizeChanged` | `size`: the server's own total as decimal GB |
|
||||
* | `conflictSessionsOpen` | `sessions`: the session names, joined with ', ' |
|
||||
*
|
||||
* `developerToolsMissing` is not sent: the view carries `developerTools` and the window prints
|
||||
* this sentence when it is `missing`. It is here so that its wording has one source.
|
||||
*
|
||||
* `endpointManaged` and `endpointIdReserved` are what the endpoint routes
|
||||
* (src/web/routes/custom-model-routes.ts) answer for the record this feature manages and for
|
||||
* its reserved id. They have no slot, so the App Settings toast that prints them translates
|
||||
* them as it translates every other whole sentence.
|
||||
*/
|
||||
export const LOCAL_MODEL_INSTALL_LABELS = {
|
||||
errRuntimeDownloadFailed:
|
||||
'The download of the runtime stopped and could not be continued. Check the network connection, then try again.',
|
||||
errRuntimeDamaged: 'The runtime download was damaged and has been deleted. Try again.',
|
||||
errRuntimeUnpackFailed: 'The runtime could not be unpacked on this Mac.',
|
||||
errRuntimeStartFailed: 'The runtime did not start on this Mac.',
|
||||
errRuntimeStopped: 'The runtime keeps stopping, so Codeman no longer restarts it.',
|
||||
errRuntimeNotRunning: 'The model on this Mac is not running. Start it first.',
|
||||
errPortInUse: 'Port {port} is in use by another program.',
|
||||
errDiskFull: 'The disk is full. Free up space, then try again.',
|
||||
errPullFailed: 'The download keeps stopping. The network may be damaging data. Try again or use another network.',
|
||||
errModelNotFound: '{model} is no longer offered under this name. Update Codeman or pick another model.',
|
||||
errDoesNotFit: '{model} does not fit in the memory of this Mac after all.',
|
||||
errContextNotApplied: '{model} loaded with a context of {actual} tokens instead of {wanted}.',
|
||||
errLoadFailed: '{model} did not load on this Mac.',
|
||||
errNoToolCall: '{model} loaded but did not call a tool, so a coding agent cannot work with it.',
|
||||
errProbeUnfinished: 'The check of {model} did not finish.',
|
||||
errHarnessFailed: '{cli} could not be installed.',
|
||||
errEndpointIdTaken:
|
||||
'A saved model connection named {id} already exists and was not created here. Rename or delete it in App Settings, then try again.',
|
||||
errConnectFailed: 'Codeman could not save the connection to the model.',
|
||||
errModelChanged: 'The model on this Mac changed since it was installed. Remove it and install it again.',
|
||||
errInternal: 'The install stopped on an unexpected error.',
|
||||
conflictJobRunning: 'An install is already running.',
|
||||
conflictAlreadyInstalled: 'A model is already installed on this Mac. Remove it first.',
|
||||
conflictSizeChanged: 'The download is {size} GB now. Check the sizes and confirm again.',
|
||||
conflictSessionsOpen: 'Sessions are still using this model: {sessions}. Close them first.',
|
||||
conflictWrongState: 'That is not possible in the current state.',
|
||||
refusedModelNotOffered: 'This model at this context is not offered for this Mac right now.',
|
||||
refusedAgentNotOffered: 'This coding agent is not offered for this model.',
|
||||
refusedAgentNotInstallable: '{cli} is not installed and cannot be installed from here.',
|
||||
refusedOtherModel: 'Only {model} is installed on this Mac. Remove it and install another model first.',
|
||||
endpointManaged:
|
||||
'This endpoint belongs to the model Codeman runs on this Mac. Change or remove it in the local model window.',
|
||||
endpointIdReserved: 'This endpoint id is reserved for the model Codeman runs on this Mac. Pick another id.',
|
||||
developerToolsMissing:
|
||||
'Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer to install them the first time an agent needs one.',
|
||||
} as const;
|
||||
|
||||
/** One of the texts in `LOCAL_MODEL_INSTALL_LABELS`. */
|
||||
export type LocalModelInstallLabel = (typeof LOCAL_MODEL_INSTALL_LABELS)[keyof typeof LOCAL_MODEL_INSTALL_LABELS];
|
||||
|
||||
/** A sentence of the install: a text from `LOCAL_MODEL_INSTALL_LABELS` plus the values of its `{placeholder}` slots. */
|
||||
export interface LocalModelInstallMessage {
|
||||
text: LocalModelInstallLabel;
|
||||
/** Slot values, already formatted as text. Absent when the text has no slot. */
|
||||
vars?: Record<string, string>;
|
||||
}
|
||||
|
||||
/** The sentence of each error code. */
|
||||
export const LOCAL_MODEL_ERROR_LABELS: Record<LocalModelErrorCode, LocalModelInstallLabel> = {
|
||||
'runtime-download-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDownloadFailed,
|
||||
'runtime-damaged': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDamaged,
|
||||
'runtime-unpack-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeUnpackFailed,
|
||||
'runtime-start-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStartFailed,
|
||||
'runtime-stopped': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStopped,
|
||||
'runtime-not-running': LOCAL_MODEL_INSTALL_LABELS.errRuntimeNotRunning,
|
||||
'port-in-use': LOCAL_MODEL_INSTALL_LABELS.errPortInUse,
|
||||
'disk-full': LOCAL_MODEL_INSTALL_LABELS.errDiskFull,
|
||||
'pull-failed': LOCAL_MODEL_INSTALL_LABELS.errPullFailed,
|
||||
'model-not-found': LOCAL_MODEL_INSTALL_LABELS.errModelNotFound,
|
||||
'does-not-fit': LOCAL_MODEL_INSTALL_LABELS.errDoesNotFit,
|
||||
'context-not-applied': LOCAL_MODEL_INSTALL_LABELS.errContextNotApplied,
|
||||
'load-failed': LOCAL_MODEL_INSTALL_LABELS.errLoadFailed,
|
||||
'no-tool-call': LOCAL_MODEL_INSTALL_LABELS.errNoToolCall,
|
||||
'probe-unfinished': LOCAL_MODEL_INSTALL_LABELS.errProbeUnfinished,
|
||||
'harness-failed': LOCAL_MODEL_INSTALL_LABELS.errHarnessFailed,
|
||||
'endpoint-id-taken': LOCAL_MODEL_INSTALL_LABELS.errEndpointIdTaken,
|
||||
'connect-failed': LOCAL_MODEL_INSTALL_LABELS.errConnectFailed,
|
||||
'model-changed': LOCAL_MODEL_INSTALL_LABELS.errModelChanged,
|
||||
internal: LOCAL_MODEL_INSTALL_LABELS.errInternal,
|
||||
};
|
||||
|
||||
/** The one helpful next step of each error code. */
|
||||
export const LOCAL_MODEL_ERROR_NEXT: Record<LocalModelErrorCode, LocalModelNextStep> = {
|
||||
'runtime-download-failed': 'retry',
|
||||
'runtime-damaged': 'retry',
|
||||
'runtime-unpack-failed': 'retry',
|
||||
'runtime-start-failed': 'retry',
|
||||
'runtime-stopped': 'start',
|
||||
'runtime-not-running': 'start',
|
||||
'port-in-use': 'other-port',
|
||||
'disk-full': 'retry',
|
||||
'pull-failed': 'retry',
|
||||
'model-not-found': 'next-model',
|
||||
'does-not-fit': 'next-model',
|
||||
'context-not-applied': 'retry',
|
||||
'load-failed': 'retry',
|
||||
'no-tool-call': 'next-model',
|
||||
'probe-unfinished': 'retry',
|
||||
'harness-failed': 'retry',
|
||||
'endpoint-id-taken': 'retry',
|
||||
'connect-failed': 'retry',
|
||||
'model-changed': 'remove',
|
||||
internal: 'retry',
|
||||
};
|
||||
|
||||
// ---------- The `install` object of the GET ----------
|
||||
|
||||
/**
|
||||
* How far a download is. Present while the `runtime` step downloads the archive and while the
|
||||
* `model` step pulls, once the total is known; kept on a paused or failed job as it stood.
|
||||
* During the `model` step both byte figures come from the pull's own lines, never from the
|
||||
* catalogue.
|
||||
*/
|
||||
export interface LocalModelProgress {
|
||||
completedBytes: number;
|
||||
totalBytes: number;
|
||||
/** `completedBytes` as decimal GB text, no unit ('6.2'). */
|
||||
completedGb: string;
|
||||
/** `totalBytes` as decimal GB text, no unit ('23.1'). */
|
||||
totalGb: string;
|
||||
/** Download speed. Absent until it can be told (the first moments, a paused job). */
|
||||
bytesPerSecond?: number;
|
||||
/** `bytesPerSecond` as whole decimal megabytes per second, no unit ('58'). Present exactly when `bytesPerSecond` is. */
|
||||
mbPerSecond?: string;
|
||||
/** Seconds left at the current speed, rounded up. Absent when the speed is unknown or 0. */
|
||||
secondsLeft?: number;
|
||||
}
|
||||
|
||||
/** An error as the window shows it: the sentence, the one action that helps, and the raw detail. */
|
||||
export interface LocalModelInstallError {
|
||||
code: LocalModelErrorCode;
|
||||
/** `LOCAL_MODEL_ERROR_LABELS[code]` with its slots filled. */
|
||||
message: LocalModelInstallMessage;
|
||||
/** `LOCAL_MODEL_ERROR_NEXT[code]`, adapted to the state the error is shown in (see `LocalModelNextStep`). */
|
||||
next: LocalModelNextStep;
|
||||
/**
|
||||
* For `next-model`: the model to try instead. After `does-not-fit` it is one that needs LESS
|
||||
* memory than the one that failed; after the other codes the best other model that fits.
|
||||
* Absent when there is none.
|
||||
*/
|
||||
nextModel?: { modelId: string; modelLabel: string };
|
||||
/**
|
||||
* Raw text for the person who wants to know more: the last lines of an installer, the
|
||||
* message of a refused write, an exception. Not translated; printed with `textContent`.
|
||||
*/
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
/** The model of the job or of the installation. */
|
||||
export interface LocalModelInstalledModel {
|
||||
/** `LocalModelEntry.id`. */
|
||||
id: string;
|
||||
/** `LocalModelEntry.label`. */
|
||||
label: string;
|
||||
}
|
||||
|
||||
/** The runtime, as the consent screen needs it. */
|
||||
export interface LocalModelRuntimeView {
|
||||
/** `LocalModelRuntimePin.version`. */
|
||||
version: string;
|
||||
/** The pinned runtime is unpacked on disk. Then an install downloads only the model. */
|
||||
installed: boolean;
|
||||
/** What an install would download for the runtime: the archive size, 0 when `installed`. */
|
||||
downloadBytes: number;
|
||||
/** `downloadBytes` as decimal GB text, no unit ('0.2'). */
|
||||
downloadGb: string;
|
||||
/** What the unpacked runtime takes on disk, as decimal GB text, no unit ('0.6'). */
|
||||
unpackedGb: string;
|
||||
}
|
||||
|
||||
/** What the runtime reported about the loaded model when the job checked it. The facts of the Done line. */
|
||||
export interface LocalModelLoadedFacts {
|
||||
/** The context length in tokens the runtime loaded the model with. */
|
||||
context: number;
|
||||
/** Memory the loaded model took, in bytes (`size` of the runtime's process list). */
|
||||
memoryBytes: number;
|
||||
/** `memoryBytes` as decimal GB text, no unit. */
|
||||
memoryGb: string;
|
||||
/** All of it is on the GPU (`size_vram` equals `size`). Always true after a job that passed. */
|
||||
onGpu: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The `install` object of `GET /api/local-model`: where the install stands and what can be
|
||||
* done about it. The window polls it once a second while it is open; there is no SSE event.
|
||||
*
|
||||
* Which members are present, by state:
|
||||
*
|
||||
* | Member | idle | installing | paused | failed | installed |
|
||||
* |-----------------------------------|------|------------|--------|--------|-----------|
|
||||
* | `step`, `steps` | | yes | yes | yes | |
|
||||
* | `progress` | | during a download | as it stood, when known | as it stood, when known | |
|
||||
* | `error` | | | | yes | only when something blocks a session |
|
||||
* | `model`, `cli`, `cliLabel`, `context` | | yes | yes | yes | yes |
|
||||
* | `port` | | once chosen | once chosen | once chosen | yes |
|
||||
* | `loaded`, `endpointId` | | | | | yes |
|
||||
*
|
||||
* `running`, `developerTools`, `runtime`, `folder` and `actions` are always present.
|
||||
*/
|
||||
export interface LocalModelInstallView {
|
||||
state: LocalModelInstallState;
|
||||
/** The step the job is in, or stopped in. */
|
||||
step?: LocalModelInstallStep;
|
||||
/** The steps of THIS job in order, for the step list (`agent` is absent when the coding agent was already installed). */
|
||||
steps?: LocalModelInstallStep[];
|
||||
progress?: LocalModelProgress;
|
||||
error?: LocalModelInstallError;
|
||||
model?: LocalModelInstalledModel;
|
||||
/** The coding agent: a CLI registry id, as data. */
|
||||
cli?: string;
|
||||
/** The coding agent's display label ('OpenCode'). */
|
||||
cliLabel?: string;
|
||||
/** The context length in tokens the runtime is started with. */
|
||||
context?: number;
|
||||
/** The loopback port of the runtime. */
|
||||
port?: number;
|
||||
/** The runtime child Codeman started is alive right now. */
|
||||
running: boolean;
|
||||
developerTools: LocalModelDeveloperTools;
|
||||
runtime: LocalModelRuntimeView;
|
||||
/** The absolute folder everything of this feature is kept in, for the "where it goes" line. */
|
||||
folder: string;
|
||||
loaded?: LocalModelLoadedFacts;
|
||||
/** The id of the saved endpoint a session is started on. */
|
||||
endpointId?: string;
|
||||
/** What the server accepts right now, see `LocalModelAction`. */
|
||||
actions: LocalModelAction[];
|
||||
}
|
||||
|
||||
/** The `data` of `GET /api/local-model`. Admin only in multi-user mode. */
|
||||
export interface LocalModelStatus {
|
||||
hardware: HardwareFacts;
|
||||
recommendation: Recommendation;
|
||||
/** `LocalModelCatalog.updated`. */
|
||||
catalogUpdated: string;
|
||||
/** Where the install stands. Always present. */
|
||||
install: LocalModelInstallView;
|
||||
}
|
||||
|
||||
// ---------- Request bodies and answers ----------
|
||||
|
||||
/**
|
||||
* Body of `POST /api/local-model/install`. The three choices must name a row the server offers
|
||||
* right now: a model of `recommendation.pick` or `recommendation.alternatives`, at that
|
||||
* candidate's `context`, with one of its `harnesses`. Anything else is a 400 before any
|
||||
* download.
|
||||
*/
|
||||
export interface LocalModelInstallRequest {
|
||||
/** `Candidate.modelId`. */
|
||||
modelId: string;
|
||||
/** `CandidateHarness.cli`. */
|
||||
cli: string;
|
||||
/** `Candidate.context`. */
|
||||
context: number;
|
||||
/**
|
||||
* The total the person saw and confirmed: `Candidate.installBytes`. The server compares it
|
||||
* with its own total and answers 409 `size-changed` when they differ, so a consent is bound
|
||||
* to the size that was shown.
|
||||
*/
|
||||
acknowledgedBytes: number;
|
||||
}
|
||||
|
||||
/** Body of `POST /api/local-model/resume`. May be absent or empty. */
|
||||
export interface LocalModelResumeRequest {
|
||||
/** True: choose another port first (the answer to `port-in-use`). */
|
||||
newPort?: boolean;
|
||||
}
|
||||
|
||||
/** Body of `POST /api/local-model/runtime`. */
|
||||
export interface LocalModelRuntimeRequest {
|
||||
/** `stop` also keeps the runtime from being started at the next boot; `start` undoes that. */
|
||||
action: 'start' | 'stop';
|
||||
/** With `start`: choose another port first (the answer to `port-in-use`), and move the saved endpoint to it. */
|
||||
newPort?: boolean;
|
||||
}
|
||||
|
||||
/** Body of `DELETE /api/local-model`. May be absent or empty. */
|
||||
export interface LocalModelRemoveRequest {
|
||||
/** True: delete the downloaded model too. Default false: the model stays for a later install. */
|
||||
removeModel?: boolean;
|
||||
}
|
||||
|
||||
/** The `data` of a 400 the service names: its sentence, for the window to print in its own language. */
|
||||
export interface LocalModelRefusal {
|
||||
message: LocalModelInstallMessage;
|
||||
}
|
||||
|
||||
/** What every verb answers on success: the view after the verb. */
|
||||
export type LocalModelVerbAnswer = LocalModelInstallView;
|
||||
|
||||
/**
|
||||
* Why a verb was refused with 409.
|
||||
* - `job-running`: a job is running (a second install, a resume, a remove or a runtime action).
|
||||
* - `already-installed`: install while a model is installed. Remove it first.
|
||||
* - `size-changed`: `acknowledgedBytes` is not the server's total; `expectedBytes` and
|
||||
* `expectedGb` carry that total, so the window can show it and ask again.
|
||||
* - `sessions-open`: remove, stop or a change of port while live sessions use the endpoint;
|
||||
* `sessions` names them. Their coding agents were given the runtime's address when they
|
||||
* started, so they would go on calling a port nothing of ours holds.
|
||||
* - `wrong-state`: the verb makes no sense in the current state (see the route table above).
|
||||
*/
|
||||
export type LocalModelConflictReason =
|
||||
| 'job-running'
|
||||
| 'already-installed'
|
||||
| 'size-changed'
|
||||
| 'sessions-open'
|
||||
| 'wrong-state';
|
||||
|
||||
/** The `data` of a 409 of an install verb. */
|
||||
export interface LocalModelConflict {
|
||||
reason: LocalModelConflictReason;
|
||||
/** The sentence for the window (`conflict...` of `LOCAL_MODEL_INSTALL_LABELS`). */
|
||||
message: LocalModelInstallMessage;
|
||||
/** `size-changed`: the server's own total in bytes. */
|
||||
expectedBytes?: number;
|
||||
/** `size-changed`: `expectedBytes` as decimal GB text, no unit. */
|
||||
expectedGb?: string;
|
||||
/** `sessions-open`: the names of the live sessions that use the endpoint. */
|
||||
sessions?: string[];
|
||||
}
|
||||
|
||||
// ---------- The state file ----------
|
||||
|
||||
/**
|
||||
* `local-model.json` in the data dir: everything that must survive a restart. Mode 0600,
|
||||
* written through a temp file and a rename, by src/local-model/state-store.ts alone.
|
||||
*
|
||||
* Rules:
|
||||
* - It holds NO path. Every path of this feature is built from `dataPath()` constants
|
||||
* (`localModelPaths()`), so a state file can never point a delete somewhere else.
|
||||
* - Reading is total: a missing file, a file that does not parse or one with another `schema`
|
||||
* reads as `{ schema: 1, state: 'idle' }`.
|
||||
* - A file read at boot with state `installing` is a job a restart interrupted: it becomes
|
||||
* `paused` before anything else looks at it.
|
||||
*/
|
||||
export interface LocalModelStateFile {
|
||||
schema: 1;
|
||||
state: LocalModelInstallState;
|
||||
/** The step the job is in or stopped in. Absent in `idle` and `installed`. */
|
||||
step?: LocalModelInstallStep;
|
||||
/** The steps of the job, in order. Absent in `idle` and `installed`. */
|
||||
steps?: LocalModelInstallStep[];
|
||||
/** State `failed`: why. State `installed`: absent (a blocking problem is found live, never stored). */
|
||||
error?: {
|
||||
code: LocalModelErrorCode;
|
||||
/** The slot values of the code's sentence. */
|
||||
vars?: Record<string, string>;
|
||||
detail?: string;
|
||||
/** For a `next-model` code: the model to offer, when the job knew one. */
|
||||
nextModelId?: string;
|
||||
};
|
||||
/** The model of the job or of the installation. */
|
||||
model?: {
|
||||
id: string;
|
||||
label: string;
|
||||
/** The context length in tokens the person chose. */
|
||||
context: number;
|
||||
/** The total the pull reported, once known. */
|
||||
pullBytes?: number;
|
||||
/** The digest `/api/tags` reported after the pull. Compared again at boot and before a session starts. */
|
||||
digest?: string;
|
||||
};
|
||||
/**
|
||||
* The model of the job this one replaced, when it was another model: what that job left in
|
||||
* the model store is deleted once the runtime runs, before the new model is downloaded.
|
||||
* Absent once that was done, and in `idle` and `installed`.
|
||||
*/
|
||||
replacedModelId?: string;
|
||||
/** The coding agent of the job or of the installation. */
|
||||
cli?: { id: string; label: string };
|
||||
/** The runtime, once its port was chosen. */
|
||||
runtime?: {
|
||||
/**
|
||||
* The runtime version this installation runs. A later build with another pin keeps
|
||||
* starting this version for as long as its folder is unpacked.
|
||||
*/
|
||||
version: string;
|
||||
/** The loopback port, chosen once by binding. */
|
||||
port: number;
|
||||
/**
|
||||
* The pid of the child Codeman last started, stored the moment it is spawned. Used to
|
||||
* tell our own runtime from another program: at boot, and before the folder is deleted.
|
||||
*/
|
||||
pid?: number;
|
||||
/** False after Stop: the runtime is not started at boot. */
|
||||
enabled: boolean;
|
||||
/** The device total the runtime printed when it started, in bytes. */
|
||||
measuredTotalBytes?: number;
|
||||
};
|
||||
/** How far the last download was when the job paused or failed. A restart may leave it absent. */
|
||||
progress?: { completedBytes: number; totalBytes: number };
|
||||
/** What the runtime reported when the job checked the loaded model. */
|
||||
loaded?: { context: number; sizeBytes: number; sizeVramBytes: number };
|
||||
/**
|
||||
* Present once the `connect` step CLAIMED the connection, which it does before its two
|
||||
* writes: what uninstall must undo. A claim with nothing behind it (the step was interrupted
|
||||
* right after it) is harmless: a resumed job writes both, and uninstall removes nothing.
|
||||
*/
|
||||
connection?: {
|
||||
/** The id of the endpoint record this feature wrote. */
|
||||
endpointId: string;
|
||||
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
|
||||
flagWasEnabled: boolean;
|
||||
};
|
||||
/** ISO time of the last write. */
|
||||
updatedAt?: string;
|
||||
}
|
||||
@@ -0,0 +1,569 @@
|
||||
/**
|
||||
* @fileoverview Types of the hardware-aware local model recommendation: what the server sees of
|
||||
* this Mac, the catalogue of models and harnesses, and the recommendation part of the answer of
|
||||
* `GET /api/local-model`.
|
||||
*
|
||||
* Everything here is the READ side: Codeman looks at the Mac and recommends. The types of the
|
||||
* install (the job states, the progress, the `install` object of the GET, the request bodies,
|
||||
* the state file, the runtime pin) and `LocalModelStatus`, the whole `data` of the GET, are in
|
||||
* src/types/local-model-install.ts. Both files go out through the barrel (`../types.js`).
|
||||
*
|
||||
* Who produces what:
|
||||
* - `HardwareFacts`: src/local-model/hardware.ts (one bounded `sysctl` spawn, never throws).
|
||||
* - `LocalModelCatalog` and its rows: src/config/local-model-catalog.ts (a compiled constant).
|
||||
* - `HarnessFact`: src/local-model/harness-facts.ts (server only, read from the CLI registry at
|
||||
* call time; it holds a function, so it never goes over HTTP).
|
||||
* - `Recommendation`, `Candidate`, `DeviceBudget`: src/local-model/recommend.ts (pure).
|
||||
*
|
||||
* A field that is optional here is optional by meaning, and its comment says when it is
|
||||
* absent (a tested entry without a measured speed, a harness that needs no install command, a
|
||||
* row without a tag). Everything else is always there, so the chooser window and the install
|
||||
* job may rely on it.
|
||||
*
|
||||
* Three rules every producer and consumer follows:
|
||||
*
|
||||
* 1. ONE unit for a computed size a person reads: decimal GB, computed from bytes by
|
||||
* `formatGb()` in src/local-model/format.ts. Every such size travels twice, as bytes
|
||||
* (`needBytes`) and as the text to show (`needGb`, a number without the unit, such as
|
||||
* '25.5'). The browser prints the text and never divides bytes itself. GiB appears only
|
||||
* inside the budget math. The one exception is the NAME of the installed memory: the card
|
||||
* says `HardwareFacts.memClassGb` (48 for a Mac sold with 48 GB), see that field.
|
||||
* 2. Every sentence or label the SERVER chooses comes from `LOCAL_MODEL_LABELS` below, so the
|
||||
* browser can translate it (`codemanT(text, vars)`) and its i18n test can list every string.
|
||||
* Numbers are never spliced into a sentence here: they travel in `vars` and fill the
|
||||
* `{placeholder}` slots.
|
||||
* 3. A CLI id is DATA (`LocalHarnessEntry.cli`, `Candidate.cli`). No code compares one with a
|
||||
* literal: what differs between harnesses is a catalogue field or a `HarnessFact`.
|
||||
*/
|
||||
|
||||
// ---------- Hardware ----------
|
||||
|
||||
/**
|
||||
* How `HardwareFacts` was obtained.
|
||||
* - `sysctl`: the spawn answered and `hw.memsize` parsed.
|
||||
* - `os-fallback`: on macOS, but the spawn failed or timed out; memory, architecture and the chip
|
||||
* string come from `node:os` and the free disk from `statfs`. What only sysctl knows (`arm64`,
|
||||
* `translated`, `model`, `macosVersion`, `macosMajor`, `wiredLimitMb`) is absent.
|
||||
* - `not-probed`: not macOS, so `sysctl` was never spawned.
|
||||
*/
|
||||
export type HardwareProbeSource = 'sysctl' | 'os-fallback' | 'not-probed';
|
||||
|
||||
/**
|
||||
* What the server knows about the machine it runs on. Plain JSON (it is part of the GET payload),
|
||||
* and always an answer: a failed probe yields the `os-fallback` or `not-probed` shape, never an
|
||||
* exception.
|
||||
*/
|
||||
export interface HardwareFacts {
|
||||
/** `process.platform` of the server. Anything but `darwin` ends in verdict `none`, reason `not-macos`. */
|
||||
platform: string;
|
||||
/** `os.arch()` of the server process. `x64` on Apple silicon means Node runs under Rosetta. */
|
||||
arch: string;
|
||||
/** `hw.optional.arm64` is 1: the MACHINE is Apple silicon whatever `arch` says. Absent when the key was not read. */
|
||||
arm64?: boolean;
|
||||
/** `sysctl.proc_translated` is 1: this process runs under Rosetta. Absent when the key was not read (Intel Macs lack it). */
|
||||
translated?: boolean;
|
||||
/** Installed memory in bytes (`hw.memsize`, else `os.totalmem()`). The ONLY memory input of the budget math. */
|
||||
memBytes: number;
|
||||
/**
|
||||
* `memBytes` in the unit of every computed size: decimal GB from `formatGb()`, without the unit
|
||||
* ('17.2' for a Mac sold as 16 GB). NOT what the card calls the installed memory: that is
|
||||
* `memClassGb`, see there.
|
||||
*/
|
||||
memGb: string;
|
||||
/**
|
||||
* The memory size the Mac is sold as: whole binary gigabytes (16, 32, 48), 0 when the size is
|
||||
* unknown. Tier rules and `TestedOn.memGb` use it, and it is the ONE size a person reads that
|
||||
* is not decimal GB: the hardware line and the `memory` slot of `noneMemory` print it, because
|
||||
* it is the number on the box and in About This Mac ("48 GB"), and '51.5 GB memory' reads as a
|
||||
* different machine. Every computed size (need, budget, download, free disk) stays decimal GB.
|
||||
*/
|
||||
memClassGb: number;
|
||||
/** `machdep.cpu.brand_string` ('Apple M4 Pro'), else `os.cpus()[0].model`. */
|
||||
chip?: string;
|
||||
/** `hw.model` ('Mac16,10'). */
|
||||
model?: string;
|
||||
/** `kern.osproductversion` ('27.2'). */
|
||||
macosVersion?: string;
|
||||
/** The leading number of `macosVersion` (27). */
|
||||
macosMajor?: number;
|
||||
/** `iogpu.wired_limit_mb` exactly as reported. 0 means "not set" (both test Macs report 0); only a value above 0 is a limit. */
|
||||
wiredLimitMb?: number;
|
||||
/** Free bytes on the filesystem of Codeman's data dir (`statfs`: `bavail * bsize`). */
|
||||
freeDiskBytes?: number;
|
||||
/** `freeDiskBytes` as a person reads it (decimal GB, no unit). Present exactly when `freeDiskBytes` is. */
|
||||
freeDiskGb?: string;
|
||||
source: HardwareProbeSource;
|
||||
}
|
||||
|
||||
// ---------- Catalogue ----------
|
||||
|
||||
/**
|
||||
* One Mac a model and harness pair was really run on by us. A candidate that matches an entry
|
||||
* (the same chip string, the same memory size as sold, the same harness) is `tested`: it ranks
|
||||
* before every candidate that is not, and as the pick it may be called "Recommended".
|
||||
*/
|
||||
export interface TestedOn {
|
||||
/** The chip, as `machdep.cpu.brand_string` reports it ('Apple M4'). */
|
||||
chip: string;
|
||||
/** The Mac's memory as it is sold (`HardwareFacts.memClassGb`: 16, 32, 48). */
|
||||
memGb: number;
|
||||
/** The harness it was tested with: a CLI registry id, as data. */
|
||||
cli: string;
|
||||
/** ISO date of the test ('2026-11-06'). */
|
||||
date: string;
|
||||
/**
|
||||
* The generation speed measured in that test, in tokens per second (67.7 for the
|
||||
* 2026-11-06 run on an Apple M5 Pro with 48 GB). A candidate that matches this entry reports
|
||||
* this figure as `Candidate.tokensPerSecond` in place of the estimate from the chip's memory
|
||||
* bandwidth: a measurement on the same Mac beats a formula.
|
||||
*/
|
||||
tokensPerSecond?: number;
|
||||
}
|
||||
|
||||
/** One model of the catalogue. */
|
||||
export interface LocalModelEntry {
|
||||
/** `name:tag` exactly as the model registry lists it. Lowercase, one colon, no slash (`LOCAL_MODEL_ID_PATTERN`). */
|
||||
id: string;
|
||||
/** The name a person reads ('Qwen3.8 27B'). */
|
||||
label: string;
|
||||
/** SPDX id of the license ('Apache-2.0'). */
|
||||
license: string;
|
||||
/** Parameter count in billions. When no model that fits has 12 or more, the verdict is `limited`, never `ok`. */
|
||||
paramsB: number;
|
||||
/**
|
||||
* Size of the weights in bytes: both the download and what they take in memory.
|
||||
*
|
||||
* It is the size of the build the runtime really downloads ON A MAC. That is not always
|
||||
* the sum of the registry manifest: `qwen3.6:35b-a3b` is an MLX build of 23,105,798,956 bytes
|
||||
* on Apple silicon, 2% more than the manifest's 22,621,314,161. A row whose Mac size nobody
|
||||
* measured keeps the manifest sum and says so in `figures`.
|
||||
*/
|
||||
bytes: number;
|
||||
/**
|
||||
* Bytes that stay HELD in memory per token of context once the token was processed (measured
|
||||
* on the MLX runner: 36,000 to 38,000, so about 37,000). A figure computed from a model's
|
||||
* config alone came out too low by almost half, so a row that was not measured carries the
|
||||
* measured row's figure and says so in `figures`.
|
||||
*/
|
||||
heldBytesPerToken: number;
|
||||
/** What the runner holds beside the weights and the context (measured: about 3 GB). */
|
||||
overheadBytes: number;
|
||||
/**
|
||||
* The TRANSIENT extra the runner takes while it processes a long prompt, on top of
|
||||
* what stays held (measured: about 5 GB). The fit is judged on the peak:
|
||||
* `bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`. For
|
||||
* `qwen3.6:35b-a3b` at 131072 tokens that is about 36 GB, which is tight on a 48 GB Mac.
|
||||
*/
|
||||
peakExtraBytes: number;
|
||||
/**
|
||||
* Where `bytes` and the three memory figures above come from: `measured` on a Mac by
|
||||
* us, or `estimated` (the manifest size, and the per-token, overhead and peak figures of the
|
||||
* measured row applied to this one).
|
||||
*/
|
||||
figures: 'measured' | 'estimated';
|
||||
/** Context lengths (tokens) the model may be run at, largest first. */
|
||||
contexts: number[];
|
||||
/** Preference among models that fit: the HIGHER rank wins. A judgement, not a measurement. */
|
||||
rank: number;
|
||||
/** Measured tokens per second per 100 GB/s of memory bandwidth. Absent: the speed of this model is unknown. */
|
||||
speedPer100Gbps?: number;
|
||||
/** Macs this model was really run on. Empty for every row but one. */
|
||||
tested: TestedOn[];
|
||||
}
|
||||
|
||||
/** One harness (coding agent CLI) that can be connected to a local model. All of it is data. */
|
||||
export interface LocalHarnessEntry {
|
||||
/** A CLI registry id. Data only: never compared with a literal anywhere. */
|
||||
cli: string;
|
||||
/** The smallest context (tokens) at which this harness works. */
|
||||
minContext: number;
|
||||
/** Preference among harnesses: the HIGHER rank wins. */
|
||||
rank: number;
|
||||
/** True: listed in the catalogue but never offered yet (its use with a local model is unverified). */
|
||||
gated?: boolean;
|
||||
/** True: the first answer takes minutes, because the harness sends a very large fixed prompt. */
|
||||
slowFirstTurn?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Memory bandwidth of one chip family. `LocalModelCatalog.chips` is ordered most specific first
|
||||
* ('M4 Max' before 'M4') and the first row that names `HardwareFacts.chip` wins. A row names a
|
||||
* chip when the chip string ENDS with its `match`, starting at the beginning or after a space,
|
||||
* case ignored (`chipNameMatches()` in src/config/local-model-catalog.ts): 'M4' is not the chip
|
||||
* of 'Apple M4 Pro', and a variant the table has never heard of ('Apple M6 Pro') matches no row.
|
||||
* A chip that matches no row has an UNKNOWN speed: a bandwidth is never guessed.
|
||||
*/
|
||||
export interface ChipEntry {
|
||||
/** The chip name the chip string must end with ('M4 Max'). */
|
||||
match: string;
|
||||
/** Memory bandwidth in GB/s. */
|
||||
gbps: number;
|
||||
}
|
||||
|
||||
/** The catalogue: a compiled constant, validated by `LocalModelCatalogSchema`. No override file, no download data. */
|
||||
export interface LocalModelCatalog {
|
||||
schema: 1;
|
||||
/** ISO date the rows were last checked ('2026-11-06'). Reported as `LocalModelStatus.catalogUpdated`. */
|
||||
updated: string;
|
||||
/** Oldest macOS major version a local model runs on. Below it: verdict `none`, reason `macos`. */
|
||||
minMacosMajor: number;
|
||||
models: LocalModelEntry[];
|
||||
harnesses: LocalHarnessEntry[];
|
||||
chips: ChipEntry[];
|
||||
}
|
||||
|
||||
// ---------- Harness facts (server only) ----------
|
||||
|
||||
/**
|
||||
* What the server knows about one catalogue harness on this machine. Built from the CLI registry
|
||||
* at call time, so `recommend()` names no CLI. NOT serializable (it holds a function): it never
|
||||
* leaves the server. `Candidate` carries the parts a browser needs.
|
||||
*/
|
||||
export interface HarnessFact {
|
||||
/** The CLI registry id. */
|
||||
cli: string;
|
||||
/** The registry's display label ('OpenCode'). */
|
||||
label: string;
|
||||
/** The CLI's binary was found on this machine. */
|
||||
installed: boolean;
|
||||
/** Not installed, but the registry has an install command for this platform that needs no administrator rights. */
|
||||
installable: boolean;
|
||||
/**
|
||||
* The literal install command the install job would run for this CLI: the STOCK
|
||||
* command of the CLI registry for this platform, never one a `clis.json` override supplies.
|
||||
* Present exactly when `installable` is true. The consent screen prints it
|
||||
* (`CandidateHarness.installCommand`).
|
||||
*/
|
||||
installCommand?: string;
|
||||
/** Whether this CLI can be launched with that model id (its command-line pattern accepts it). */
|
||||
acceptsModelId: (modelId: string) => boolean;
|
||||
}
|
||||
|
||||
// ---------- Recommendation ----------
|
||||
|
||||
/**
|
||||
* Need over budget: up to 0.85 is `good`, up to 0.98 is `tight`, above is `out`. The need is
|
||||
* the PEAK, so the grade says how much room is left at the worst moment. It decides whether a
|
||||
* model is offered at all (`out` never is) and what its row says (`tight`). It does not sort
|
||||
* the list.
|
||||
*/
|
||||
export type FitGrade = 'good' | 'tight' | 'out';
|
||||
|
||||
/**
|
||||
* Expected generation speed. `unknown` when the chip is in no `ChipEntry` or the model has no
|
||||
* `speedPer100Gbps`. In the ranking `unknown` sits between `ok` and `slow`.
|
||||
*/
|
||||
export type SpeedGrade = 'ok' | 'slow' | 'unknown';
|
||||
|
||||
/**
|
||||
* Why the verdict is `none`.
|
||||
* - `not-macos`: the server does not run on macOS (the card is hidden).
|
||||
* - `intel`: a Mac without Apple silicon.
|
||||
* - `memory`: no model fits the memory budget at a context a usable harness needs. Three
|
||||
* headlines share this reason: `noneMemory` (nothing fits at any context), `noneMemoryContext`
|
||||
* (a model fits, but only below the context a harness needs: the sentence carries the budget
|
||||
* and the need) and `noneMemoryUnknown` (the memory size could not be read).
|
||||
* - `disk`: models fit the memory, but for every one of them free disk minus the download is below 10 GiB.
|
||||
* - `macos`: macOS is older than `LocalModelCatalog.minMacosMajor`.
|
||||
* - `no-harness`: no catalogue harness is installed or installable and accepts a model.
|
||||
*/
|
||||
export type NoneReason = 'not-macos' | 'intel' | 'memory' | 'disk' | 'macos' | 'no-harness';
|
||||
|
||||
/**
|
||||
* Where the device budget came from.
|
||||
* - `estimated`: a formula over the installed memory. All there is before the runtime ran.
|
||||
* - `wired-limit`: `iogpu.wired_limit_mb` above 0.
|
||||
* - `measured`: the device total the runtime itself printed when it started
|
||||
* (`RecommendOptions.measuredTotalBytes`). The chooser says "estimated" next to the memory
|
||||
* bar for the first value only.
|
||||
*/
|
||||
export type BudgetSource = 'estimated' | 'wired-limit' | 'measured';
|
||||
|
||||
/** The memory a model may use on this Mac. */
|
||||
export interface DeviceBudget {
|
||||
/**
|
||||
* L: the measured device total when there is one, else the wired limit when above 0, else
|
||||
* 2/3 of memory below 36 GB and 3/4 from 36 GB.
|
||||
*/
|
||||
limitBytes: number;
|
||||
/**
|
||||
* What `Candidate.budgetShare` is measured against. Estimated or from the wired limit:
|
||||
* B = min(L - 1 GiB, memory - reserve). Measured: B = min(L, memory - reserve), without the
|
||||
* 1 GiB, because the runtime's figure is already what it will really use (37.4 GiB on the
|
||||
* 48 GB test Mac, which reads '40.2').
|
||||
*/
|
||||
budgetBytes: number;
|
||||
source: BudgetSource;
|
||||
}
|
||||
|
||||
/**
|
||||
* What the caller of `recommend()` knows beyond the hardware probe. Both members are optional:
|
||||
* an empty object means the budget is the estimate and an install downloads the model alone.
|
||||
*/
|
||||
export interface RecommendOptions {
|
||||
/**
|
||||
* The device total the runtime reported in its log line "inference compute"
|
||||
* (`total="37.4 GiB"`), in bytes: `parseInferenceComputeTotal()` in
|
||||
* src/local-model/runtime.ts. Present: the budget is `measured`. The install job passes it
|
||||
* right after the runtime started, before the model is downloaded, and the GET passes the
|
||||
* stored figure from then on.
|
||||
*/
|
||||
measuredTotalBytes?: number;
|
||||
/**
|
||||
* What an install adds to every candidate's download beside the model: the size of the
|
||||
* pinned runtime archive while the runtime is not installed, 0 once it is. It is the
|
||||
* difference between `Candidate.installBytes` and `Candidate.downloadBytes`. Default 0.
|
||||
*/
|
||||
runtimeDownloadBytes?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every sentence and label the server may put into the payload, in English (the i18n source
|
||||
* language). The zh-CN dictionary in src/web/public/i18n.js has one entry per value, keyed by
|
||||
* this exact text, and test/local-model-i18n.test.ts imports this constant to list them. Adding
|
||||
* a key is fine; changing a text means changing its zh-CN key too.
|
||||
*
|
||||
* `{placeholder}` slots are filled by the browser from `LocalModelMessage.vars`. `{name}` is
|
||||
* taken (the translator fills it with the display name), so no slot here is called `name`.
|
||||
*
|
||||
* | Key | Slots (all text; sizes are decimal GB without the unit) |
|
||||
* |---------------------|---------------------------------------------------------------------------|
|
||||
* | `noteDiskDropped` | `model`: label of the best model left out for disk space; `missing`: its |
|
||||
* | | download + the disk reserve - free |
|
||||
* | `noneMemory` | `memory`: `HardwareFacts.memClassGb`, the size the Mac is sold as |
|
||||
* | `noneMemoryContext` | `budget`: `Recommendation.budgetGb`; `need`: the smallest need of any |
|
||||
* | | model at a context a usable harness works at |
|
||||
* | `noneDisk` | `download`: smallest download among the models that fit the memory; |
|
||||
* | | `reserve`: what must stay free after it (10 GiB, as decimal GB); `free`: |
|
||||
* | | `HardwareFacts.freeDiskGb`; `missing`: download + reserve - free |
|
||||
* | `noneMacos` | `version`: `HardwareFacts.macosVersion`; `minimum`: the catalogue minimum |
|
||||
*
|
||||
* The word "Recommended" exists only in `headlineTested`, which may be used only when the
|
||||
* pick's `tested` is true: one catalogue row has a tested entry (`qwen3.6:35b-a3b` on an Apple
|
||||
* M5 Pro with 48 GB), so one kind of Mac is told so.
|
||||
*/
|
||||
export const LOCAL_MODEL_LABELS = {
|
||||
/** Verdict `ok`, pick not tested by us on a Mac like this one. */
|
||||
headlineOk: 'Best fit for this Mac',
|
||||
/** Verdict `ok`, pick tested by us on a Mac like this one. */
|
||||
headlineTested: 'Recommended for this Mac',
|
||||
/** Verdict `limited`: no model with 12 billion parameters or more fits this Mac. */
|
||||
headlineLimited:
|
||||
'This Mac can run a small model. It handles small edits and questions. It does not replace a cloud model.',
|
||||
/** A note under an `ok` or `limited` pick whose `tested` is false. */
|
||||
noteUntested: 'Codeman has not run this on a Mac like this yet.',
|
||||
/** A note under an `ok` pick that is itself a small model: larger ones fit this Mac, but rank lower or have no room on the disk. */
|
||||
noteSmallPick:
|
||||
'The best fit is a small model. It handles small edits and questions. It does not replace a cloud model.',
|
||||
/** A note when a model that fits the memory was left out because its download would not leave the disk reserve. */
|
||||
noteDiskDropped:
|
||||
'{model} fits the memory of this Mac too, but its download needs about {missing} GB more free disk space.',
|
||||
fitGood: 'Good',
|
||||
fitTight: 'Tight',
|
||||
speedSlow: 'Slow',
|
||||
speedUnknown: 'Speed unknown',
|
||||
noneNotMacos: 'Codeman recommends a local model only on a Mac with Apple silicon.',
|
||||
noneIntel: 'Local coding models need Apple silicon, and this Mac has an Intel processor.',
|
||||
/** Reason `memory`: nothing fits at any context. */
|
||||
noneMemory: 'With {memory} GB of memory, this Mac cannot run a model that can drive a coding agent.',
|
||||
/** Reason `memory`: a model fits, but only below the context a harness needs. Says both numbers, and ("about") that they are estimates. */
|
||||
noneMemoryContext:
|
||||
'This Mac can give a model about {budget} GB of memory, and the smallest model a coding agent can work with needs about {need} GB.',
|
||||
/** Reason `memory`: the memory size could not be read, so no size is stated. */
|
||||
noneMemoryUnknown: 'Codeman could not read how much memory this Mac has, so it recommends no model here.',
|
||||
/** Reason `disk`. One number: what to do. The other sizes travel in `vars` for a client that wants them. */
|
||||
noneDisk: 'Free up about {missing} GB of disk space to install the smallest model that fits this Mac.',
|
||||
noneMacos: 'Local models need macOS {minimum} or newer, and this Mac runs macOS {version}.',
|
||||
noneNoHarness: 'No coding agent that Codeman can connect to a local model is available on this Mac.',
|
||||
} as const;
|
||||
|
||||
/** One of the texts in `LOCAL_MODEL_LABELS`. The payload's label fields are typed with it, so an off-list string does not compile. */
|
||||
export type LocalModelLabel = (typeof LOCAL_MODEL_LABELS)[keyof typeof LOCAL_MODEL_LABELS];
|
||||
|
||||
/** The headline of each `none` reason. `memory` has two more (`noneMemoryContext`, `noneMemoryUnknown`), see `NoneReason`. */
|
||||
export const LOCAL_MODEL_NONE_LABELS: Record<NoneReason, LocalModelLabel> = {
|
||||
'not-macos': LOCAL_MODEL_LABELS.noneNotMacos,
|
||||
intel: LOCAL_MODEL_LABELS.noneIntel,
|
||||
memory: LOCAL_MODEL_LABELS.noneMemory,
|
||||
disk: LOCAL_MODEL_LABELS.noneDisk,
|
||||
macos: LOCAL_MODEL_LABELS.noneMacos,
|
||||
'no-harness': LOCAL_MODEL_LABELS.noneNoHarness,
|
||||
};
|
||||
|
||||
/** A sentence for the card: a text from `LOCAL_MODEL_LABELS` plus the values of its `{placeholder}` slots. */
|
||||
export interface LocalModelMessage {
|
||||
text: LocalModelLabel;
|
||||
/** Slot values, already formatted as text. Absent when the text has no slot. */
|
||||
vars?: Record<string, string>;
|
||||
}
|
||||
|
||||
/**
|
||||
* The one tag a model row of the chooser carries, by what it says.
|
||||
* - `tested`: `Candidate.tested` is true. Replaces every other tag.
|
||||
* - `best-tight`: the pick, when its `fit` is `tight`. The words must not say less than the
|
||||
* memory bar beside them, which is nearly full for such a row.
|
||||
* - `best`: the pick.
|
||||
* - `tight`: `Candidate.fit` is `tight`.
|
||||
* - `slow`: `Candidate.speed` is `slow`.
|
||||
* - `small`: the model has fewer than 12 billion parameters.
|
||||
* A row gets the FIRST kind of this list that applies and no tag when none does. `tested`,
|
||||
* `best` and `best-tight` are the tags of the row the window opens on, drawn in the accent
|
||||
* colour.
|
||||
*/
|
||||
export type CandidateTagKind = 'tested' | 'best-tight' | 'best' | 'tight' | 'slow' | 'small';
|
||||
|
||||
/**
|
||||
* The text of each tag, in English (the i18n source language), like `LOCAL_MODEL_LABELS`. A
|
||||
* constant of its own so that a consumer can list the tag texts apart from the sentences.
|
||||
*/
|
||||
export const LOCAL_MODEL_TAG_LABELS = {
|
||||
tested: 'Tested on a Mac like this',
|
||||
'best-tight': 'Best fit, tight',
|
||||
best: 'Best fit',
|
||||
tight: 'Tight',
|
||||
slow: 'Slow',
|
||||
small: 'Small model',
|
||||
} as const satisfies Record<CandidateTagKind, string>;
|
||||
|
||||
/** One of the texts in `LOCAL_MODEL_TAG_LABELS`. */
|
||||
export type LocalModelTagLabel = (typeof LOCAL_MODEL_TAG_LABELS)[CandidateTagKind];
|
||||
|
||||
/** The tag of a model row: what kind it is (for the styling) and the text to print. */
|
||||
export interface CandidateTag {
|
||||
kind: CandidateTagKind;
|
||||
/** `LOCAL_MODEL_TAG_LABELS[kind]`. */
|
||||
label: LocalModelTagLabel;
|
||||
}
|
||||
|
||||
/**
|
||||
* One coding agent a candidate's model can be run with on this Mac, for the "Runs
|
||||
* with" select of the chooser. Everything a browser needs is here, so it compares no CLI id.
|
||||
*/
|
||||
export interface CandidateHarness {
|
||||
/** The CLI registry id, as data: what `POST /api/local-model/install` is sent as `cli`. */
|
||||
cli: string;
|
||||
/** The registry's display label ('OpenCode'). */
|
||||
cliLabel: string;
|
||||
/** False: the install job installs it first (it is installable, or it would not be listed). */
|
||||
installed: boolean;
|
||||
/**
|
||||
* The literal command the install job would run to install it (`HarnessFact.installCommand`).
|
||||
* Present exactly when `installed` is false. The consent screen prints it as it is.
|
||||
*/
|
||||
installCommand?: string;
|
||||
/** The harness row's `slowFirstTurn`. */
|
||||
slowFirstTurn?: boolean;
|
||||
}
|
||||
|
||||
/** One model at one context with one harness, measured against this Mac. Plain JSON. */
|
||||
export interface Candidate {
|
||||
/** `LocalModelEntry.id`. */
|
||||
modelId: string;
|
||||
/** `LocalModelEntry.label`. */
|
||||
modelLabel: string;
|
||||
license: string;
|
||||
paramsB: number;
|
||||
/** Context length in tokens. */
|
||||
context: number;
|
||||
/** The harness: a CLI registry id, as data. */
|
||||
cli: string;
|
||||
/** The harness's display label, from the CLI registry. */
|
||||
cliLabel: string;
|
||||
/** False: the harness is not on this machine yet (it is installable, or it would not be a candidate). */
|
||||
cliInstalled: boolean;
|
||||
/** The harness row's `slowFirstTurn`. */
|
||||
slowFirstTurn?: boolean;
|
||||
/**
|
||||
* What the model needs in memory at this context: the PEAK, `bytes + heldBytesPerToken *
|
||||
* context + overheadBytes + peakExtraBytes` of the catalogue row. The memory bar of the
|
||||
* chooser is filled to `budgetShare`, which is this over the budget.
|
||||
*/
|
||||
needBytes: number;
|
||||
/** `needBytes` as decimal GB text, no unit. */
|
||||
needGb: string;
|
||||
/**
|
||||
* What `needBytes` is made of, for the chooser's memory map: the model (`bytes +
|
||||
* overheadBytes`), its context in use (`heldBytesPerToken * context`) and the brief peaks
|
||||
* (`peakExtraBytes`). The three add up to `needBytes`.
|
||||
*/
|
||||
memory?: { modelBytes: number; contextBytes: number; peakBytes: number };
|
||||
/** The model download in bytes (`LocalModelEntry.bytes`). */
|
||||
downloadBytes: number;
|
||||
/** `downloadBytes` as decimal GB text, no unit. */
|
||||
downloadGb: string;
|
||||
/** `needBytes / Recommendation.budgetBytes`, rounded to three decimals (0.83). The fit grade is decided on the unrounded ratio. */
|
||||
budgetShare: number;
|
||||
/** A candidate in a `Recommendation` is never `out`. */
|
||||
fit: Exclude<FitGrade, 'out'>;
|
||||
/** `fitGood` or `fitTight`. */
|
||||
fitLabel: LocalModelLabel;
|
||||
speed: SpeedGrade;
|
||||
/** `speedSlow` or `speedUnknown`. Absent when `speed` is `ok`. */
|
||||
speedLabel?: LocalModelLabel;
|
||||
/**
|
||||
* Generation speed in tokens per second, rounded to one decimal. Present exactly when `speed`
|
||||
* is not `unknown`; `speed` is decided on the unrounded figure. An estimate from the chip's
|
||||
* memory bandwidth, except for a candidate that is `tested`, which reports the
|
||||
* speed measured in that test (`TestedOn.tokensPerSecond`).
|
||||
*/
|
||||
tokensPerSecond?: number;
|
||||
/**
|
||||
* A `TestedOn` entry of the model matches this Mac (same chip, same `memClassGb`) and this
|
||||
* harness. Such a candidate ranks first, and its `tag` says so in place of any other tag.
|
||||
*/
|
||||
tested: boolean;
|
||||
/** `LocalModelEntry.figures` of the model: whether `needBytes` rests on a measurement of this model. */
|
||||
needSource: 'measured' | 'estimated';
|
||||
/** The row's one tag, see `CandidateTagKind`. Absent when no kind applies. */
|
||||
tag?: CandidateTag;
|
||||
/**
|
||||
* Every coding agent this model can be run with at `context` on this Mac, best
|
||||
* first. The first entry is the one the flat fields above name (`cli`, `cliLabel`,
|
||||
* `cliInstalled`, `slowFirstTurn`). Never empty.
|
||||
*/
|
||||
harnesses: CandidateHarness[];
|
||||
/**
|
||||
* The whole download a click on Install starts for this row: `downloadBytes` plus
|
||||
* `RecommendOptions.runtimeDownloadBytes`. It is the server's own total, and exactly the
|
||||
* number `POST /api/local-model/install` must be sent as `acknowledgedBytes`.
|
||||
*/
|
||||
installBytes: number;
|
||||
/** `installBytes` as decimal GB text, no unit. */
|
||||
installGb: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The answer of `recommend()`.
|
||||
* - `ok`: `pick` is the best candidate, and at least one model with 12 billion parameters or
|
||||
* more fits this Mac's memory at a context a usable harness works at. The pick itself can be
|
||||
* smaller (a larger model may fit only tightly, have no speed figure, or have no room on the
|
||||
* disk): `notes` then carries `noteSmallPick`.
|
||||
* - `limited`: `pick` is the best candidate, and NO model with 12 billion parameters or more
|
||||
* fits. The card says so and never says "Recommended".
|
||||
* - `none`: no pick; `reason` says why.
|
||||
*/
|
||||
export interface Recommendation {
|
||||
verdict: 'ok' | 'limited' | 'none';
|
||||
/** Present exactly when `verdict` is `none`. */
|
||||
reason?: NoneReason;
|
||||
/** What the card says first: `headlineOk`, `headlineTested`, `headlineLimited` or a `none` sentence. */
|
||||
headline: LocalModelMessage;
|
||||
/** Further sentences under the pick, in this order: `noteSmallPick`, `noteDiskDropped`, `noteUntested`. May be empty. */
|
||||
notes: LocalModelMessage[];
|
||||
/** Present exactly when `verdict` is `ok` or `limited`. */
|
||||
pick?: Candidate;
|
||||
/**
|
||||
* The next models that fit and have room on the disk (at most four), best first: tested
|
||||
* before not tested, then ok before unknown before slow speed, then the higher model rank.
|
||||
* Each is at the largest context it fits at, with the best harness that can run it there.
|
||||
* Empty when `verdict` is `none`.
|
||||
*/
|
||||
alternatives: Candidate[];
|
||||
/** `DeviceBudget.budgetBytes`. 0 when no budget is computed: not macOS, or a Mac without Apple silicon. */
|
||||
budgetBytes: number;
|
||||
/** `budgetBytes` as decimal GB text, no unit. */
|
||||
budgetGb: string;
|
||||
budgetSource: BudgetSource;
|
||||
}
|
||||
|
||||
// `LocalModelStatus`, the `data` of `GET /api/local-model`, is in src/types/local-model-install.ts.
|
||||
Reference in new issue
Block a user