feat(local-model): contract for the install stage (types, stubs, fake runtime)

The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-06 20:05:15 +01:00
1 parent 66ca92f7cf
commit 771072c36b
17 files changed
+3770 -23

No files matched your search

+110
View File
@@ -0,0 +1,110 @@
/**
* @fileoverview The shared runner of a STOCK CLI's install command: the one place Codeman
* executes an install one-liner, used by `POST /api/clis/:id/install`
* (src/web/routes/cli-registry-routes.ts) and by the local model install job.
*
* CONTRACT STAGE: `CLI_INSTALL_TIMEOUT_MS` is final. Every function below has its final
* signature and a body that throws. The bodies are MOVED here from cli-registry-routes.ts
* next, and that route keeps its behaviour byte for byte (same answers, same audit entry,
* same 409 for a second install of one id, same environment).
*
* What moves, unchanged in behaviour:
* - `installEnv()` and `npmGlobalPrefixWritable()`: the installer's environment is the
* server's minus every `CODEMAN_*` variable, with npm's global prefix pointed at a place
* the server user can write. test/routes/cli-registry-routes.test.ts imports both from the
* route module, which re-exports them from here.
* - `runInstallCommand()`: `shell: true`, its own process group, output capped, the whole
* group killed on timeout. New: an abort signal that kills the group the same way, so the
* install job's Cancel has something to stop.
* - `forgetResolvedCli()`: after an install, successful or not, every cached lookup of the
* CLI's binaries is dropped, or a pre-install miss is replayed for up to five minutes.
* - The set of ids with an install running. ONE set for both callers
* (`tryBeginCliInstall()`), so the route and the job can never run two installers of one
* CLI over the same prefix.
*
* Rules:
* - The command executed is only ever a STOCK entry's (`stockInstallCommand()` reads the
* compiled `STOCK_CLIS`, never the registry merged with `clis.json`, so an override file
* cannot change what runs). A custom entry's install text is never executed.
* - Who may call it is the caller's rule, not this module's: the route keeps its
* `cliManagementEnabled` gate; the install job runs it only for a harness row of the local
* model catalogue, after the person's consent click on a screen that showed the literal
* command, admin only in multi-user mode, and writes the admin audit entry itself.
*/
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** How long an install command may run before its process group is killed. */
export const CLI_INSTALL_TIMEOUT_MS = 300_000;
/** How an install command ended. */
export interface CliInstallResult {
/** The exit code, `null` when the command was killed or could not be started. */
code: number | null;
/** stdout and stderr together, capped. */
output: string;
/** The timeout killed it. */
timedOut: boolean;
/** The abort signal killed it. */
aborted: boolean;
}
export interface RunInstallCommandOptions {
/** Kills the installer's whole process group (SIGTERM, then SIGKILL). */
signal?: AbortSignal;
}
/**
* True when `npm install -g` can write to this environment's npm global prefix, or when that
* cannot be told (then nothing is redirected). Moved from cli-registry-routes.ts unchanged.
*/
export async function npmGlobalPrefixWritable(source: NodeJS.ProcessEnv): Promise<boolean> {
return notImplemented('npmGlobalPrefixWritable', source);
}
/**
* The environment of an install command: `source` minus every `CODEMAN_*` variable, with
* `NPM_CONFIG_PREFIX` pointed at `$HOME/.local` inside the Compose container or when the
* global prefix is not writable. Moved from cli-registry-routes.ts unchanged.
*/
export function installEnv(
source: NodeJS.ProcessEnv = process.env,
prefixWritable: (env: NodeJS.ProcessEnv) => boolean = () => true
): NodeJS.ProcessEnv {
return notImplemented('installEnv', source, prefixWritable);
}
/** Runs one install command to its end. Never throws: a spawn that fails is a result with `code: null`. */
export async function runInstallCommand(
command: string,
options: RunInstallCommandOptions = {}
): Promise<CliInstallResult> {
return notImplemented('runInstallCommand', command, options);
}
/**
* The install command of the STOCK CLI `id` for this platform, read from the compiled
* `STOCK_CLIS`. `undefined` for an id that is not a stock CLI or has no command here.
*/
export function stockInstallCommand(id: string): string | undefined {
return notImplemented('stockInstallCommand', id);
}
/**
* Claims the one install slot of `id`. Returns the function that releases it, or `null` when
* an install of that CLI is already running (the caller answers 409 or waits).
*/
export function tryBeginCliInstall(id: string): (() => void) | null {
return notImplemented('tryBeginCliInstall', id);
}
/**
* Forgets every cached binary lookup of a CLI: the per-id resolver and every per-binary cache,
* hits and misses alike. Called after anything that may have changed what is on disk.
*/
export function forgetResolvedCli(id: string, binaries: readonly string[]): void {
return notImplemented('forgetResolvedCli', id, binaries);
}
+72
View File
@@ -0,0 +1,72 @@
/**
* @fileoverview The serialized writer of `custom-model-hosts.json`, the store of the saved
* custom model endpoints (src/custom-model-hosts.ts).
*
* CONTRACT STAGE: the function below has its final signature and a body that throws; the body
* is written next, and every writer of the file is moved onto it then.
*
* Why it exists: `writeCustomModelHosts()` writes through ONE temp file name per process and
* has no queue, and `readCustomModelHosts()` reads a file that does not parse as an empty
* list. Two overlapping writers (an endpoint route and the 5-minute discovery sweep) can
* leave one body over the tail of a longer one, and the next writer then saves a file that
* holds only its own record: every other endpoint and its API key is gone. The same fault
* `mutateSettingsFile()` ended for `settings.json`.
*
* The shape of `mutateSettingsFile()` (src/web/settings-writer.ts):
* - Serialized: mutations run one at a time on one promise chain. The check a route makes
* (the id is free, the record still exists) runs INSIDE `fn`, in the same step as the write.
* - Atomic: each write goes to a temp file with a random name beside the target and is
* renamed over it. The file is mode 0600 (it holds API keys).
* - Refuses a file it must not trust: only a missing or empty file starts from an empty
* list. A file that cannot be read, does not parse or is not an array is left alone and
* the write rejects with `CustomModelHostsWriteRefusedError`. The message never repeats
* the parser's own text, which can quote an API key.
*
* EVERY writer goes through it: the four endpoint routes of
* src/web/routes/custom-model-routes.ts (create, update, delete, discover-models), the
* discovery sweep `refreshAllCustomModelHosts()`, and the local model's connect step. The one
* direct writer left is config-backup restore, which must work on a file that does not parse.
*/
import type { CustomModelHost } from './custom-model-hosts.js';
import type { ApiResponse } from './types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** A write refused because the existing `custom-model-hosts.json` must not be overwritten. The message is user-facing. */
export class CustomModelHostsWriteRefusedError extends Error {
constructor(message: string) {
super(message);
this.name = 'CustomModelHostsWriteRefusedError';
}
}
/** What a mutation returns: the list to write (absent: write nothing) and what the caller gets back. */
export interface CustomModelHostsMutation<T> {
/** The whole list to store. Absent: the file is left as it is. */
hosts?: CustomModelHost[];
/** Handed back to the caller of `mutateCustomModelHosts()`. */
result: T;
}
/**
* Runs one change to the endpoint store. `fn` receives the current list (a copy it may change)
* and returns what to write and what to hand back. Resolves with `result`. A refused or failed
* mutation rejects its own caller only; the chain keeps going.
*
* `configDir` is the folder of the file, as for `readCustomModelHosts()`. Default: the data dir.
*/
export function mutateCustomModelHosts<T>(
fn: (hosts: CustomModelHost[]) => CustomModelHostsMutation<T>,
configDir?: string
): Promise<T> {
return notImplemented('mutateCustomModelHosts', fn, configDir);
}
/** A refused write answers 409 CONFLICT with the fix; any other failure stays OPERATION_FAILED. Like `settingsWriteErrorResponse()`. */
export function customModelHostsWriteErrorResponse(err: unknown): ApiResponse<never> {
return notImplemented('customModelHostsWriteErrorResponse', err);
}
+124
View File
@@ -0,0 +1,124 @@
/**
* @fileoverview Connecting the installed local model to Codeman, and undoing it: the endpoint
* record, the settings flag, and the checks around them.
*
* CONTRACT STAGE: the two constants are final. Every function below has its final signature
* and a body that throws; the bodies are written next.
*
* These are the ONLY two writes of the local model outside its own folder:
* 1. One record in `custom-model-hosts.json`, through `mutateCustomModelHosts()`:
* `{ id: 'local-ollama', label: 'This Mac', baseUrl: 'http://127.0.0.1:<port>',
* authStyle: 'bearer', models: [<id as /v1/models lists it>], defaultModelId,
* defaultContextLength: <the chosen context> }`, validated by `CustomModelHostSchema`
* before it is written. No `apiKey`. `defaultContextLength` is what reaches the coding
* agent as its context limit, because the runtime's model listing states none.
* 2. `customModelEndpointsEnabled: true` in `settings.json`, through `mutateSettingsFile()`.
* The value it had before is returned and kept in the state file.
*
* Rules:
* - The id is reserved. A record with it that this feature did not write is never
* overwritten: the connect step fails with `endpoint-id-taken`.
* - Both writes are idempotent, so a resumed job writes them again.
* - Undoing removes our record, and switches the flag off again only when this feature
* switched it on AND no other endpoint is saved.
* - The model's digest is compared again before a session starts and at boot
* (`compareModelDigest()`): a model that changed under the same name is not launched.
*/
import type { CustomModelHost } from '../custom-model-hosts.js';
import type { LocalModelErrorCode } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** The id of the endpoint record this feature writes. Reserved: nobody else may use it. */
export const LOCAL_MODEL_ENDPOINT_ID = 'local-ollama';
/** The label of that record, what the Run menu shows in brackets. */
export const LOCAL_MODEL_ENDPOINT_LABEL = 'This Mac';
/** What the endpoint record is built from. */
export interface LocalEndpointInput {
/** The runtime's loopback port. */
port: number;
/** The model id exactly as the runtime's `/v1/models` lists it. */
modelId: string;
/** The chosen context length in tokens. */
context: number;
}
/** The endpoint record of the file header. Pure; the caller validates it with `CustomModelHostSchema`. */
export function buildLocalEndpointRecord(input: LocalEndpointInput): CustomModelHost {
return notImplemented('buildLocalEndpointRecord', input);
}
export interface ConnectLocalModelInput extends LocalEndpointInput {
/**
* True when the state file already records a connection of ours: an existing record with
* our id is then ours and is replaced. False: such a record is somebody else's.
*/
ours: boolean;
/**
* The flag's value before this feature first switched it on, when the state file already
* holds one. It is handed back unchanged, so a second connect cannot overwrite it with `true`.
*/
flagWasEnabled?: boolean;
}
/** How the connect step ended. */
export type ConnectResult =
| {
ok: true;
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
flagWasEnabled: boolean;
}
| { ok: false; code: Extract<LocalModelErrorCode, 'endpoint-id-taken' | 'connect-failed'>; detail?: string };
/** Writes the record and the flag. Idempotent. Never throws: a refused write is `connect-failed` with its message as `detail`. */
export async function connectLocalModel(input: ConnectLocalModelInput): Promise<ConnectResult> {
return notImplemented('connectLocalModel', input);
}
/** Moves our record to another port (the answer to "Use another port" once connected). A missing record is not an error. */
export async function moveLocalEndpoint(port: number): Promise<void> {
return notImplemented('moveLocalEndpoint', port);
}
/**
* Removes our record, and restores the settings flag when this feature switched it on
* (`flagWasEnabled` is false) and no other endpoint is left. Idempotent.
*/
export async function disconnectLocalModel(input: { flagWasEnabled: boolean }): Promise<void> {
return notImplemented('disconnectLocalModel', input);
}
/** Whether `id` is the reserved endpoint id. For the endpoint routes and the discovery sweep, which leave that record alone. */
export function isLocalModelEndpointId(id: string): boolean {
return notImplemented('isLocalModelEndpointId', id);
}
/**
* The names of the live sessions that use the endpoint `endpointId`, in the order given.
* Pure. What `DELETE /api/local-model` answers its 409 with.
*/
export function sessionNamesUsingEndpoint(
sessions: Iterable<{ name: string; customModel?: { endpointId: string } }>,
endpointId: string = LOCAL_MODEL_ENDPOINT_ID
): string[] {
return notImplemented('sessionNamesUsingEndpoint', sessions, endpointId);
}
/**
* Compares the digest stored at install time with the one the runtime reports now. Pure.
* - `same`: launch.
* - `changed`: refuse (`model-changed`): the name now stands for other weights or another template.
* - `unknown`: one of the two is absent (the runtime does not answer, the model is gone).
* The caller refuses as well, with the reason it knows.
*/
export function compareModelDigest(
stored: string | undefined,
current: string | undefined
): 'same' | 'changed' | 'unknown' {
return notImplemented('compareModelDigest', stored, current);
}
+42
View File
@@ -0,0 +1,42 @@
/**
* @fileoverview Whether this Mac has the developer tools a coding agent calls (git, Python).
*
* CONTRACT STAGE: the function below has its final signature and a body that throws; the body
* is written next.
*
* Why it exists: on a Mac without Xcode or the Command Line Tools, `git` and `python3` are
* stubs that open macOS's "install the command line developer tools" window. A coding agent
* runs both, so the install window says one honest sentence when the tools are missing
* (`LOCAL_MODEL_INSTALL_LABELS.developerToolsMissing`).
*
* Rules:
* - The ONLY command ever run is `/usr/bin/xcode-select -p`, bounded by a timeout. Never
* `git`, `python3`, `xcrun` or anything else that would open that window itself.
* - macOS only. Anywhere else, and when the command does not answer in time, the answer is
* `unknown`, and nothing is said.
* - Never throws. The answer is cached for a short while, so a window that polls once a
* second does not spawn once a second.
*/
import type { LocalModelDeveloperTools } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** How long `/usr/bin/xcode-select -p` may take. */
export const DEVELOPER_TOOLS_TIMEOUT_MS = 2_000;
/** What the check calls. Each member defaults to the real thing; a test injects its own. */
export interface DeveloperToolsDeps {
/** Default `process.platform`. */
platform: string;
/** Runs `file` with `args` and resolves with its exit code (`null` when it timed out or could not be started). */
run: (file: string, args: readonly string[], timeoutMs: number) => Promise<number | null>;
}
/** `present` when `/usr/bin/xcode-select -p` exits 0, `missing` when it exits with anything else, `unknown` otherwise. */
export async function checkDeveloperTools(deps: Partial<DeveloperToolsDeps> = {}): Promise<LocalModelDeveloperTools> {
return notImplemented('checkDeveloperTools', deps);
}
+497
View File
@@ -0,0 +1,497 @@
/**
* @fileoverview The local model install job: its state table, the pure reducer over it, the
* pure view of it, and the service that drives it.
*
* CONTRACT STAGE: the three tables (`INSTALL_STATE_TABLE`, `INSTALL_STEP_TABLE`,
* `INSTALL_TRANSITIONS`) are final and are what the reducer's table-driven test iterates.
* Every function below has its final signature and a body that throws; the bodies are written
* next.
*
* ONE job at a time. The job is five steps in a fixed order (`LOCAL_MODEL_INSTALL_STEPS`:
* runtime, model, verify, agent, connect; `agent` is left out when the coding agent is
* installed). Its state is one of five (`LocalModelInstallState`), stored in the state file.
*
* The state table. "On disk" names what lies under the local model folder; the two writes
* outside it belong to the `connect` step.
*
* | State | On disk | A server restart | Cancel | Allowed next actions |
* |--------------|-------------------------------------------|-----------------------------|------------------------|---------------------------------|
* | `idle` | nothing, or leftovers a Remove could not | stays `idle` | refused (409) | install |
* | | delete (never read back) | | | |
* | `installing` | what the finished steps wrote, plus what | becomes `paused` at the | becomes `paused` at | cancel |
* | | the current step has written so far | same step; nothing resumes | the same step | |
* | | (see the step table) | without a click | | |
* | `paused` | as `installing` at that step, nothing in | stays `paused` | refused (409) | resume, install (another |
* | | flight | | | choice), remove |
* | `failed` | as `installing` at that step, nothing in | stays `failed`, same error | refused (409) | resume (try again), install |
* | | flight | | | (another choice), remove |
* | `installed` | the runtime, the private home, the model | stays `installed`; the | refused (409) | start a session, start or stop |
* | | | runtime is started again | | the runtime, remove |
* | | | when it was not stopped | | |
*
* The step table. Every step is IDEMPOTENT, so Resume and "Try again" re-enter the step the
* job stopped in; they first make sure the runtime runs when the step needs it.
*
* | Step | Adds to disk | Cancel | Resume |
* |-----------|---------------------------------------|------------------------------------------|------------------------------------------------|
* | `runtime` | the archive's partial file, then the | aborts the download, the partial file | continues the partial file with a Range |
* | | unpacked runtime, then the private | stays | request, or skips what is already unpacked |
* | | home once the runtime started | | |
* | `model` | the model store: partial blobs, then | aborts the pull request, which stops the | pulls again; the runtime continues its partial |
* | | the model | download; the partial blobs stay | blobs |
* | `verify` | nothing (the model is loaded into | aborts the request in flight | runs the three checks again from the first |
* | | memory) | | |
* | `agent` | whatever the coding agent's own | kills the installer's process group | looks for the binary first and runs the |
* | | installer writes, outside our folder; | | installer only when it is still missing |
* | | never removed by us | | |
* | `connect` | our endpoint record and the settings | honoured before the first write; once | writes both again |
* | | flag: the two writes outside our | the writes began they finish and the job | |
* | | folder | ends `installed` | |
*
* How the memory budget gets measured before a model is downloaded (ONE consent, bound to the
* size shown): the person confirms `Candidate.installBytes` once. The `runtime` step then
* gets and starts the runtime, reads the device total from its log and judges the choice
* again with it (`modelFitsBudget()`; `recommend()` with the same options names the next
* model). When the chosen model at the chosen context no longer fits, the job fails right
* there with `does-not-fit` and the next model that fits, BEFORE the model download starts. When it fits, the job goes on without a second question: the sizes that were
* confirmed did not change. From then on the GET reports `budgetSource: 'measured'`.
*
* Rules:
* - `reduceInstallJob()` is PURE and total: an event the transition table does not list for
* the current state is refused by returning the SAME object, which the service answers
* with 409 `wrong-state`.
* - The reducer moves the state, the step, the error and the progress. The FACTS a step
* learns (the port and pid, the measured total, the pull's total, the digest, what the
* runtime loaded, the previous settings flag) are written by the service through
* `mutateLocalModelState()` and never change the state.
* - A job read as `installing` at boot was interrupted by the restart: the service applies
* the event `restart` before anything else, so it is `paused` and waits for a click.
* - A new install from `paused` or `failed` REPLACES the job. When it names another model,
* what the old job left of its model is deleted from the store first.
* - Install is checked before anything is downloaded: admin only in multi-user mode; the
* model, the context and the coding agent must be a row the current recommendation offers;
* the coding agent must be a harness row of the catalogue; `acknowledgedBytes` must be the
* server's own total.
* - No CLI id is compared with a literal here: the coding agent is a catalogue row plus a
* `HarnessFact`.
*/
import type {
HardwareFacts,
HarnessFact,
LocalModelAction,
LocalModelCatalog,
LocalModelConflict,
LocalModelDeveloperTools,
LocalModelErrorCode,
LocalModelInstallRequest,
LocalModelInstallState,
LocalModelInstallStep,
LocalModelInstallView,
LocalModelRemoveRequest,
LocalModelResumeRequest,
LocalModelRuntimePin,
LocalModelRuntimeRequest,
LocalModelStateFile,
Recommendation,
RecommendOptions,
} from '../types.js';
import type { CliInstallResult, RunInstallCommandOptions } from '../cli-install-runner.js';
import type { ConnectLocalModelInput, ConnectResult } from './connect.js';
import type { OllamaTarget, PullOptions, PullResult, VerifyResult } from './ollama-client.js';
import type { LocalModelPaths } from './paths.js';
import type {
ChooseRuntimePortOptions,
RemoveLocalModelFilesOptions,
RuntimeInstallOptions,
RuntimeInstallResult,
RuntimeSupervisor,
} from './runtime.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
// ---------- The state table ----------
/** One row of the state table. */
export interface InstallStateRow {
/** What lies on disk in this state, in words. */
onDisk: string;
/** The state a server restart turns this one into. */
afterRestart: LocalModelInstallState;
/** What the restart does, in words. */
restart: string;
/** The state Cancel turns this one into, or `null` when Cancel is refused (409 `wrong-state`). */
afterCancel: LocalModelInstallState | null;
/** What Cancel does, in words. */
cancel: string;
/**
* Every action a client may take in this state. The view's `actions` is this list narrowed
* by what is live (`allowedActions()`): `start` only while the runtime is down, `stop` and
* `start-session` only while it runs, `start-session` only when nothing blocks a session.
*/
actions: readonly LocalModelAction[];
}
/** The state table of the file header, as data. */
export const INSTALL_STATE_TABLE: Readonly<Record<LocalModelInstallState, InstallStateRow>> = {
idle: {
onDisk:
'Nothing of ours. The local model folder is absent, or holds leftovers an earlier Remove could not delete; they are never read back.',
afterRestart: 'idle',
restart: 'Nothing happens.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['install'],
},
installing: {
onDisk:
'What the finished steps wrote, plus what the current step has written so far (INSTALL_STEP_TABLE). The state file names the model, the coding agent, the context and the step.',
afterRestart: 'paused',
restart:
'The job comes back paused at the same step. Nothing is downloaded, started or installed until the person presses Resume.',
afterCancel: 'paused',
cancel: 'The step in flight is interrupted the way its step row says, and the job is paused at that step.',
actions: ['cancel'],
},
paused: {
onDisk: 'As installing at that step, with nothing in flight.',
afterRestart: 'paused',
restart: 'Stays paused.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['resume', 'install', 'remove'],
},
failed: {
onDisk: 'As installing at that step, with nothing in flight.',
afterRestart: 'failed',
restart: 'Stays failed, with the same error.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['resume', 'install', 'remove'],
},
installed: {
onDisk:
'The runtime, its private home and the model. The state file holds the port, the pid, the digest, what the runtime loaded and the previous settings flag. Outside the folder: our endpoint record and the settings flag.',
afterRestart: 'installed',
restart:
'Stays installed. The runtime is started again unless the person had stopped it, and the digest is read again: a changed digest or a port taken by another program shows as an error on the view and blocks a session, the state stays installed.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['start-session', 'start', 'stop', 'remove'],
},
};
// ---------- The step table ----------
/** One row of the step table. */
export interface InstallStepRow {
/** What the step does, in words. */
does: string;
/** What it adds to disk (possibly partly, when it is interrupted). */
onDisk: string;
/** How Cancel interrupts it. */
cancel: string;
/** What Resume, and "Try again" after a failure, does: every step is idempotent and is entered again. */
resume: string;
/** Whether the step reports byte progress (`LocalModelProgress`). */
progress: boolean;
/** Whether the runtime must run before the step is entered (the service starts it first, idempotently). */
needsRuntime: boolean;
/** Every error code the step can fail with. `internal` and `disk-full` can come from any step that writes. */
errors: readonly LocalModelErrorCode[];
}
/** The step table of the file header, as data. */
export const INSTALL_STEP_TABLE: Readonly<Record<LocalModelInstallStep, InstallStepRow>> = {
runtime: {
does: 'Downloads the pinned runtime archive unless it is unpacked already, compares its sha256, unpacks it, chooses the port once, starts the runtime, reads the measured memory budget from its log and judges the chosen model against it.',
onDisk:
'runtime/ollama-<version>.tgz.partial while it downloads, runtime/ollama-<version>/ once unpacked, home/ once the runtime started.',
cancel: 'Aborts the download. The partial file stays. A runtime that already runs keeps running.',
resume:
'Continues the partial file with a Range request, or skips the download when the folder is unpacked. Starting the runtime again is a no-op when it runs.',
progress: true,
needsRuntime: false,
errors: [
'runtime-download-failed',
'runtime-damaged',
'runtime-unpack-failed',
'runtime-start-failed',
'port-in-use',
'does-not-fit',
'disk-full',
'internal',
],
},
model: {
does: 'Pulls the model through the runtime, reading the progress stream line by line.',
onDisk: 'models/ with partial blobs, then the finished model.',
cancel: 'Aborts the pull request, which stops the download. The partial blobs stay.',
resume: 'Pulls again. The runtime continues its partial blobs.',
progress: true,
needsRuntime: true,
errors: ['pull-failed', 'model-not-found', 'runtime-start-failed', 'port-in-use', 'disk-full', 'internal'],
},
verify: {
does: 'Loads the model with one tiny chat, checks that the runtime put all of it on the GPU at the chosen context, runs one tool probe, and stores the digest the runtime reports.',
onDisk: 'Nothing new. The model is loaded into memory.',
cancel: 'Aborts the request in flight.',
resume: 'Runs the three checks again from the first.',
progress: false,
needsRuntime: true,
errors: [
'load-failed',
'does-not-fit',
'context-not-applied',
'no-tool-call',
'probe-unfinished',
'runtime-start-failed',
'port-in-use',
'internal',
],
},
agent: {
does: 'Installs the coding agent with its stock install command through the shared CLI install runner, writes the admin audit entry, forgets the cached lookups of its binary and looks for it again.',
onDisk:
'Whatever the coding agent installer writes, outside the local model folder (its own folder in the home directory). Never removed by us.',
cancel: 'Kills the whole process group of the installer.',
resume: 'Looks for the binary first and runs the installer only when it is still missing.',
progress: false,
needsRuntime: false,
errors: ['harness-failed', 'internal'],
},
connect: {
does: 'Writes the endpoint record and switches the custom model endpoints setting on, keeping its previous value in the state file.',
onDisk:
'Our record in custom-model-hosts.json and the flag in settings.json: the two writes outside the local model folder.',
cancel:
'Honoured before the first write. Once the writes began they finish and the job ends installed, so the pair is never left half done.',
resume: 'Writes both again. Both are idempotent.',
progress: false,
needsRuntime: true,
errors: ['endpoint-id-taken', 'connect-failed', 'internal'],
},
};
// ---------- Events and transitions ----------
/** What moves the job. */
export type InstallJobEvent =
/** A new job: `POST /api/local-model/install` passed its checks. From `idle`, `paused` or `failed`. */
| {
type: 'install';
model: { id: string; label: string; context: number };
cli: { id: string; label: string };
/** The steps of this job in order (`jobSteps()`). */
steps: LocalModelInstallStep[];
}
/** A download moved. Changes `progress` only. */
| { type: 'progress'; completedBytes: number; totalBytes: number }
/** The current step finished: the next step of `steps`, or `installed` after the last. Clears `progress`. */
| { type: 'step-done' }
/** The current step failed. */
| {
type: 'fail';
code: LocalModelErrorCode;
vars?: Record<string, string>;
detail?: string;
nextModelId?: string;
}
/** `POST /api/local-model/cancel`. */
| { type: 'cancel' }
/** The server started and found this state in the file. */
| { type: 'restart' }
/** `POST /api/local-model/resume`: the same step again. Clears `error`. */
| { type: 'resume' }
/** `DELETE /api/local-model` finished: back to `idle`, every other member dropped. */
| { type: 'remove' };
export type InstallJobEventType = InstallJobEvent['type'];
/**
* Which state an event leads to, by current state. A pair that is not listed is REFUSED: the
* reducer returns the object it was given. One entry needs a word: `step-done` in
* `installing` leads to `installing` at the next step, and to `installed` when the step that
* finished was the last of the job's `steps`.
*/
export const INSTALL_TRANSITIONS: Readonly<
Record<LocalModelInstallState, Readonly<Partial<Record<InstallJobEventType, LocalModelInstallState>>>>
> = {
idle: { install: 'installing', restart: 'idle', remove: 'idle' },
installing: {
progress: 'installing',
'step-done': 'installing',
fail: 'failed',
cancel: 'paused',
restart: 'paused',
},
paused: { resume: 'installing', install: 'installing', remove: 'idle', restart: 'paused' },
failed: { resume: 'installing', install: 'installing', remove: 'idle', restart: 'failed' },
installed: { remove: 'idle', restart: 'installed' },
};
/** The steps a job runs, in order: all of `LOCAL_MODEL_INSTALL_STEPS`, without `agent` when the coding agent is installed. Pure. */
export function jobSteps(input: { agentInstalled: boolean }): LocalModelInstallStep[] {
return notImplemented('jobSteps', input);
}
/**
* The state file after `event`. Pure: no IO, no clock (`updatedAt` is the state store's
* business). Refuses an event the transition table does not list for the current state by
* returning `current` itself. Keeps `runtime` and `connection` across a new install, because
* what they record is still on disk; `remove` drops everything.
*/
export function reduceInstallJob(current: LocalModelStateFile, event: InstallJobEvent): LocalModelStateFile {
return notImplemented('reduceInstallJob', current, event);
}
// ---------- The view ----------
/** What is live and not in the state file, for the view. Gathered by the service on every GET. */
export interface InstallLiveFacts {
/** The runtime child is alive (`RuntimeStatus.running`). */
running: boolean;
/** The supervisor stopped restarting the runtime (`RuntimeStatus.gaveUp`): the error `runtime-stopped` in state `installed`. */
gaveUp: boolean;
/** Another program holds the stored port: the error `port-in-use` in state `installed`. */
portInUse: boolean;
/** The stored digest against the runtime's current one (`compareModelDigest()`). `changed` is the error `model-changed`. Absent while not installed. */
digest?: 'same' | 'changed' | 'unknown';
developerTools: LocalModelDeveloperTools;
/** The pinned runtime is unpacked on disk. */
runtimeInstalled: boolean;
/** The current download speed. Not stored, so absent after a restart. */
bytesPerSecond?: number;
/** For an error whose next step is `next-model`: the model to offer, from the current recommendation. */
nextModel?: { modelId: string; modelLabel: string };
pin: LocalModelRuntimePin;
/** `LocalModelPaths.root`, for the "where it goes" line. */
folder: string;
}
/** The actions the server accepts right now: the state row's list narrowed by what is live. Pure. */
export function allowedActions(state: LocalModelStateFile, live: InstallLiveFacts): LocalModelAction[] {
return notImplemented('allowedActions', state, live);
}
/** The `install` object of the GET. Pure: every size text comes from `formatGb()`, every sentence from `LOCAL_MODEL_INSTALL_LABELS`. */
export function buildInstallView(state: LocalModelStateFile, live: InstallLiveFacts): LocalModelInstallView {
return notImplemented('buildInstallView', state, live);
}
// ---------- The service ----------
/** Who asked, for the admin audit entry of the coding agent install. */
export interface LocalModelActor {
username: string;
ip?: string;
}
/** How a verb ended: the view after it, a 400 with its message, or a 409 with its conflict. */
export type LocalModelVerbResult =
| { ok: true; view: LocalModelInstallView }
| { ok: false; status: 400; message: string }
| { ok: false; status: 409; conflict: LocalModelConflict };
/**
* Everything the service calls, so a test spawns, fetches and downloads nothing. Each member
* defaults to the real module. The members named like a function of another module have
* exactly that function's signature.
*/
export interface LocalModelServiceDeps {
readHardware: () => Promise<HardwareFacts>;
getCatalog: () => LocalModelCatalog;
buildHarnessFacts: (catalog: LocalModelCatalog) => Promise<HarnessFact[]>;
recommend: (
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[],
options?: RecommendOptions
) => Recommendation;
/** `modelFitsBudget()` of src/local-model/recommend.ts: the judgement of the `runtime` step, with the measured budget. */
modelFitsBudget: (
hw: HardwareFacts,
catalog: LocalModelCatalog,
modelId: string,
context: number,
options?: RecommendOptions
) => boolean;
// src/local-model/runtime.ts
isRuntimeInstalled: (pin?: LocalModelRuntimePin, paths?: LocalModelPaths) => Promise<boolean>;
ensureRuntimeInstalled: (options?: RuntimeInstallOptions) => Promise<RuntimeInstallResult>;
chooseRuntimePort: (options?: ChooseRuntimePortOptions) => Promise<number | undefined>;
removeLocalModelFiles: (options: RemoveLocalModelFilesOptions) => Promise<void>;
supervisor: RuntimeSupervisor;
// src/local-model/ollama-client.ts
pullModel: (target: OllamaTarget, modelId: string, options?: PullOptions) => Promise<PullResult>;
verifyModel: (target: OllamaTarget, modelId: string, context: number, signal?: AbortSignal) => Promise<VerifyResult>;
readModelDigest: (target: OllamaTarget, modelId: string, signal?: AbortSignal) => Promise<string | undefined>;
listOpenAiModels: (target: OllamaTarget, signal?: AbortSignal) => Promise<string[]>;
deleteModel: (target: OllamaTarget, modelId: string, signal?: AbortSignal) => Promise<boolean>;
// src/local-model/developer-tools.ts
checkDeveloperTools: () => Promise<LocalModelDeveloperTools>;
// src/cli-install-runner.ts
stockInstallCommand: (id: string) => string | undefined;
tryBeginCliInstall: (id: string) => (() => void) | null;
runInstallCommand: (command: string, options?: RunInstallCommandOptions) => Promise<CliInstallResult>;
/** Forgets the cached lookups of the CLI's binaries, then says whether its binary is found now. */
recheckCliInstalled: (cli: string) => Promise<boolean>;
/** Writes one admin audit line (`appendAdminAudit()`). */
audit: (entry: { admin: string; action: string; target: string; ip?: string; detail?: unknown }) => Promise<void>;
// src/local-model/connect.ts
connectLocalModel: (input: ConnectLocalModelInput) => Promise<ConnectResult>;
moveLocalEndpoint: (port: number) => Promise<void>;
disconnectLocalModel: (input: { flagWasEnabled: boolean }) => Promise<void>;
// src/local-model/state-store.ts
readState: () => Promise<LocalModelStateFile>;
mutateState: (
fn: (current: LocalModelStateFile) => LocalModelStateFile | undefined
) => Promise<{ before: LocalModelStateFile; after: LocalModelStateFile | undefined }>;
pin: LocalModelRuntimePin;
paths: LocalModelPaths;
}
/** The install as the routes and the server see it. One per process. */
export interface LocalModelService {
/** The `install` object of the GET. Never throws. */
view(): Promise<LocalModelInstallView>;
/** What the GET passes to `recommend()`: the stored measured total and the runtime download that is still needed. */
recommendOptions(): Promise<RecommendOptions>;
/** `POST /api/local-model/install`. Answers once the job is STARTED; the job runs on in the server. */
install(request: LocalModelInstallRequest, actor: LocalModelActor): Promise<LocalModelVerbResult>;
/** `POST /api/local-model/cancel`. Answers once the step in flight was interrupted. */
cancel(): Promise<LocalModelVerbResult>;
/** `POST /api/local-model/resume`. */
resume(request: LocalModelResumeRequest, actor: LocalModelActor): Promise<LocalModelVerbResult>;
/** `POST /api/local-model/runtime`. */
runtime(request: LocalModelRuntimeRequest): Promise<LocalModelVerbResult>;
/**
* `DELETE /api/local-model`. `liveSessionNames` is what `sessionNamesUsingEndpoint()` found:
* not empty answers 409 `sessions-open`. Else: stop the runtime, remove our endpoint record,
* restore the settings flag when this feature switched it on and no other endpoint is left,
* delete the runtime and the private home and (when asked) the models. The coding agent stays.
*/
remove(request: LocalModelRemoveRequest, liveSessionNames: readonly string[]): Promise<LocalModelVerbResult>;
/**
* Called once when the server starts. An interrupted job becomes `paused`. An installed
* runtime that was not stopped is started (an own orphan on the port is adopted or
* replaced, another listener is `port-in-use`) and the digest is read again. Never throws,
* and never keeps the server from starting.
*/
boot(): Promise<void>;
/** Called when the server stops, beside `stopDeepSeekWeb()`: aborts a job in flight (it reads `paused` at the next boot) and stops the runtime. */
shutdown(): Promise<void>;
}
/** A service over `deps`. A test builds its own with fakes. */
export function createLocalModelService(deps: Partial<LocalModelServiceDeps> = {}): LocalModelService {
return notImplemented('createLocalModelService', deps);
}
/** The one service of this process, built on first use with the real modules. What the routes and server.ts call. */
export function getLocalModelService(): LocalModelService {
return notImplemented('getLocalModelService');
}
+275
View File
@@ -0,0 +1,275 @@
/**
* @fileoverview The HTTP client of the model runtime: what Codeman asks its own runtime on
* the loopback port, and the pure functions that read the answers.
*
* CONTRACT STAGE: every function below has its final signature and a body that throws; the
* bodies are written next.
*
* What it asks:
* - `GET /api/version`, `GET /api/tags`, `GET /api/ps`, `GET /v1/models`, `DELETE /api/delete`.
* - `POST /api/pull` with `stream: true`. The answer is one JSON object per line, and a
* 23 GB pull sends about 1.4 million of them, so the stream is parsed INCREMENTALLY
* (`createPullParser()`, pure): only the unfinished last line and the running totals are
* kept, never the body. Aborting the request stops the download; pulling again continues
* it. An error line or a stream that ends without `success` is retried, at most
* `PULL_RETRIES` times.
* - The check after the pull (`verifyModel()`), in this order: one tiny chat with
* `reasoning_effort: "none"` (it loads the model), then `GET /api/ps` must show the whole
* model on the GPU (`size_vram` equal to `size`) at the chosen context
* (`judgeLoaded()`, pure), then one tool probe on `/v1/chat/completions` with a `tools`
* array, `reasoning_effort: "none"` and `max_tokens` of `TOOL_PROBE_MAX_TOKENS`, which
* passes on a tool call whose arguments parse as JSON (`judgeToolProbe()`, pure).
*
* Rules:
* - Nothing here throws for something a runtime can answer: every outcome is a result with
* the job's error code. An abort signal is the result `cancelled`.
* - The runtime does not enforce the context length (a prompt of 167,012 tokens was accepted
* at 131072), so nothing here relies on it to.
* - Every function takes its `fetch` from the target, so a test passes the fake runtime's
* address and spawns nothing.
*/
import type { LocalModelErrorCode } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** How often a pull that ended on an error line or a cut stream is started again. */
export const PULL_RETRIES = 5;
/** `max_tokens` of the tool probe. */
export const TOOL_PROBE_MAX_TOKENS = 1024;
/** Which runtime to ask. */
export interface OllamaTarget {
/** `http://127.0.0.1:<port>`, without a trailing slash. */
baseUrl: string;
/** Default: the global `fetch`. */
fetch?: typeof fetch;
}
// ---------- Listings ----------
/** One model of `GET /api/tags`: what is on disk. */
export interface OllamaTagModel {
/** `name:tag`. */
name: string;
/** Size on disk in bytes. */
size: number;
/** The model's digest, lowercase hex without a prefix. What is stored and compared again before a session starts. */
digest: string;
/** `details.format` ('safetensors' for an MLX build, 'gguf'). */
format?: string;
/** `details.runner` ('mlx', 'ggml'). */
runner?: string;
/** `capabilities` ('completion', 'tools', 'thinking', 'vision'). */
capabilities?: string[];
}
/** One model of `GET /api/ps`: what is loaded. */
export interface OllamaPsModel {
name: string;
/** Memory the loaded model takes, in bytes. */
size: number;
/** The part of `size` that is on the GPU. Equal to `size` when all of it is. */
sizeVram: number;
/** The context length the model was loaded with. */
contextLength: number;
digest?: string;
}
/** The runtime's version ('0.40.15'), or `undefined` when nothing answers there or the answer is not a runtime's. */
export async function getRuntimeVersion(target: OllamaTarget, signal?: AbortSignal): Promise<string | undefined> {
return notImplemented('getRuntimeVersion', target, signal);
}
/** The models on disk. An empty list when the runtime has none or does not answer. */
export async function listTags(target: OllamaTarget, signal?: AbortSignal): Promise<OllamaTagModel[]> {
return notImplemented('listTags', target, signal);
}
/** The models loaded right now. An empty list when none is or the runtime does not answer. */
export async function listRunning(target: OllamaTarget, signal?: AbortSignal): Promise<OllamaPsModel[]> {
return notImplemented('listRunning', target, signal);
}
/** The model ids `GET /v1/models` lists, exactly as listed (the runtime answers `data: null` for none). */
export async function listOpenAiModels(target: OllamaTarget, signal?: AbortSignal): Promise<string[]> {
return notImplemented('listOpenAiModels', target, signal);
}
/** The digest `GET /api/tags` reports for `modelId`, or `undefined` when the model is not on disk. */
export async function readModelDigest(
target: OllamaTarget,
modelId: string,
signal?: AbortSignal
): Promise<string | undefined> {
return notImplemented('readModelDigest', target, modelId, signal);
}
/** Deletes a model from the runtime's store. True when it is gone afterwards (also when it never was there). */
export async function deleteModel(target: OllamaTarget, modelId: string, signal?: AbortSignal): Promise<boolean> {
return notImplemented('deleteModel', target, modelId, signal);
}
// ---------- Pull ----------
/** What the pull's lines add up to so far. */
export interface PullSnapshot {
/** Sum of the last `completed` of every layer seen. */
completedBytes: number;
/** Sum of the `total` of every layer seen. Grows while new layers appear; 0 before the first. */
totalBytes: number;
/** The `status` text of the last line that had one ('pulling manifest', 'writing manifest'). */
status?: string;
/** A line `{"status":"success"}` was seen. */
succeeded: boolean;
/** The text of the first `{"error":"..."}` line. */
error?: string;
/** How many complete lines were parsed. */
lines: number;
}
/**
* Reads a pull stream piece by piece. It keeps the unfinished last line and the totals per
* layer, never the lines themselves, so its memory does not grow with the stream.
*/
export interface PullParser {
/** Feed the next piece of the body. A line may be split anywhere across pieces, a multi-byte character too. */
push(chunk: string | Uint8Array): void;
/** The body ended: reads a last line that had no line break. */
end(): void;
snapshot(): PullSnapshot;
}
/**
* A parser for one pull stream. Pure: no IO, no clock. Lines it understands (both real):
*
* - an MLX build: `{"status":"pulling model","digest":"sha256:model","total":23105798956,"completed":N}`
* - a GGUF build, one layer after the other:
* `{"status":"pulling c5396e06af29","digest":"sha256:c539...","total":397807936,"completed":N}`
* (`completed` is absent until the layer's first bytes arrived)
* - `{"status":"pulling manifest"}`, `{"status":"verifying sha256 digest"}`,
* `{"status":"writing manifest"}`, `{"status":"success"}`
* - `{"error":"max retries exceeded: local error: tls: bad record MAC"}`
*
* A line that is not JSON is skipped and still counted.
*/
export function createPullParser(): PullParser {
return notImplemented('createPullParser');
}
/** How far a pull is, for the caller's progress display. */
export interface PullProgress {
completedBytes: number;
totalBytes: number;
/** Over the last few seconds. Absent until it can be told. */
bytesPerSecond?: number;
}
export interface PullOptions {
/** Aborts the request, which stops the download. The result is `cancelled`. */
signal?: AbortSignal;
/** Called at most a few times a second, and once more when the pull ends. */
onProgress?: (progress: PullProgress) => void;
/** Default `PULL_RETRIES`. */
retries?: number;
/** Waits between retries. A test injects its own. */
sleep?: (ms: number) => Promise<void>;
/** The clock of the speed figure, in milliseconds. A test injects its own. */
now?: () => number;
}
/** How a pull ended. */
export type PullResult =
| {
ok: true;
/** The pull's own total: what every size shown for the model comes from once it is known. */
totalBytes: number;
}
| {
ok: false;
code: Extract<LocalModelErrorCode, 'pull-failed' | 'model-not-found' | 'disk-full'>;
/** The runtime's last error text. */
detail?: string;
}
| { ok: false; code: 'cancelled' };
/** Pulls `modelId` and waits for the end. Resuming is calling it again. Never throws. */
export async function pullModel(target: OllamaTarget, modelId: string, options: PullOptions = {}): Promise<PullResult> {
return notImplemented('pullModel', target, modelId, options);
}
// ---------- The check after the pull ----------
/** The codes the check can end on. */
export type VerifyFailureCode = Extract<
LocalModelErrorCode,
'load-failed' | 'does-not-fit' | 'context-not-applied' | 'no-tool-call' | 'probe-unfinished'
>;
/** What the runtime reported about the loaded model. */
export interface LoadedModel {
context: number;
sizeBytes: number;
sizeVramBytes: number;
}
/** The body of the tiny chat that loads the model: one short user message, `reasoning_effort: "none"`, a small `max_tokens`. Pure. */
export function buildChatProbeRequest(modelId: string): Record<string, unknown> {
return notImplemented('buildChatProbeRequest', modelId);
}
/** The body of the tool probe: a `tools` array, `reasoning_effort: "none"`, `max_tokens: TOOL_PROBE_MAX_TOKENS`. Pure. */
export function buildToolProbeRequest(modelId: string): Record<string, unknown> {
return notImplemented('buildToolProbeRequest', modelId);
}
/**
* Judges the process list after the chat. Pure.
* - the model is not listed: `load-failed`;
* - `sizeVram` is not `size`: `does-not-fit` (part of the model is not on the GPU);
* - `contextLength` is not `context`: `context-not-applied`.
*/
export function judgeLoaded(
models: readonly OllamaPsModel[],
modelId: string,
context: number
): { ok: true; loaded: LoadedModel } | { ok: false; code: VerifyFailureCode; detail?: string } {
return notImplemented('judgeLoaded', models, modelId, context);
}
/**
* Judges the answer of the tool probe. Pure. `status` is the HTTP status, `body` the parsed
* JSON (or `undefined` when it did not parse).
* - a tool call whose `arguments` parse as a JSON object: passed;
* - `finish_reason` is `length`: `probe-unfinished` (the token limit ended it, so it proves nothing);
* - any other answer of status 200: `no-tool-call`;
* - another status: `load-failed`.
*/
export function judgeToolProbe(
status: number,
body: unknown
): { ok: true } | { ok: false; code: VerifyFailureCode; detail?: string } {
return notImplemented('judgeToolProbe', status, body);
}
/** How the check ended. */
export type VerifyResult =
| { ok: true; loaded: LoadedModel }
| { ok: false; code: VerifyFailureCode; detail?: string }
| { ok: false; code: 'cancelled' };
/**
* The whole check, in order: the chat, the process list, the tool probe. Stops at the first
* failure. Never throws.
*/
export async function verifyModel(
target: OllamaTarget,
modelId: string,
context: number,
signal?: AbortSignal
): Promise<VerifyResult> {
return notImplemented('verifyModel', target, modelId, context, signal);
}
+87
View File
@@ -0,0 +1,87 @@
/**
* @fileoverview Where everything of the local model lives on disk. One pure function, so the
* runtime, the install job and uninstall all name the same folders.
*
* Layout, all under the data dir (`dataPath()`, so a second Codeman instance has its own):
*
* ```
* local-model.json the state file (mode 0600)
* local-model/
* runtime/ollama-<version>.tgz.partial the runtime archive while it downloads
* runtime/ollama-<version>.unpack/ where the archive is unpacked, before the rename
* runtime/ollama-<version>/ the unpacked runtime; its binary is `ollama`
* models/ the runtime's model store
* home/ the runtime's private HOME (its key pair lands here)
* ```
*
* Rules:
* - `root` is the ONE folder this feature owns. Nothing is written or deleted outside it,
* except the state file beside it and the two writes of the connect step (the endpoint
* record and the settings flag), which have their own writers.
* - Every path comes from here. None is ever read back from the state file, so a state file
* cannot point a delete anywhere else. `isInsideLocalModelRoot()` is the check a delete
* makes before it runs.
* - Pure apart from `dataPath()`, which resolves (and creates) the data dir.
*/
import { join, resolve, sep } from 'node:path';
import { dataPath } from '../config/instance.js';
/** A runtime version as it may appear in a folder name: digits and dots only ('0.40.15'). */
const VERSION_PATTERN = /^\d+(?:\.\d+){1,3}$/;
export interface LocalModelPaths {
/** `<data dir>/local-model`: the one folder this feature owns. */
root: string;
/** `<root>/runtime`: the archive and every unpacked version. */
runtimeRoot: string;
/** `<root>/runtime/ollama-<version>`: the unpacked runtime. */
runtimeDir: string;
/** The runtime's binary: `<runtimeDir>/ollama`. */
binary: string;
/** `<root>/runtime/ollama-<version>.tgz.partial`: the archive while it downloads. */
archivePartial: string;
/** `<root>/runtime/ollama-<version>.unpack`: the temporary folder the archive is unpacked into. */
unpackDir: string;
/** `<root>/models`: the runtime's model store. */
modelsDir: string;
/** `<root>/home`: the runtime's private HOME. */
homeDir: string;
/** `<data dir>/local-model.json`: the state file. */
stateFile: string;
}
/**
* The paths of the local model for one runtime version. Throws on a version that is not
* digits and dots, so a version string can never add a path segment.
*/
export function localModelPaths(version: string): LocalModelPaths {
if (!VERSION_PATTERN.test(version)) {
throw new Error(`Not a runtime version: ${JSON.stringify(version)}`);
}
const root = dataPath('local-model');
const runtimeRoot = join(root, 'runtime');
const runtimeDir = join(runtimeRoot, `ollama-${version}`);
return {
root,
runtimeRoot,
runtimeDir,
binary: join(runtimeDir, 'ollama'),
archivePartial: join(runtimeRoot, `ollama-${version}.tgz.partial`),
unpackDir: join(runtimeRoot, `ollama-${version}.unpack`),
modelsDir: join(root, 'models'),
homeDir: join(root, 'home'),
stateFile: dataPath('local-model.json'),
};
}
/**
* Whether `path` is the local model folder itself or lies inside it, judged on the resolved
* text (no filesystem access, so a symlink is not followed: a caller that deletes checks the
* real path first). `..` segments are resolved before the comparison.
*/
export function isInsideLocalModelRoot(path: string, root: string = dataPath('local-model')): boolean {
const base = resolve(root);
const target = resolve(path);
return target === base || target.startsWith(base + sep);
}
+31 -1
View File
@@ -71,6 +71,7 @@ import {
type LocalModelMessage,
type NoneReason,
type Recommendation,
type RecommendOptions,
type SpeedGrade,
} from '../types.js';
import { chipBandwidthGbps } from '../config/local-model-catalog.js';
@@ -135,6 +136,29 @@ function fitGrade(need: number, budgetBytes: number): FitGrade {
return 'out';
}
/**
* Whether the catalogue model `modelId` at `context` tokens fits this Mac's budget (a `good`
* or a `tight` fit). The install job asks this right after the runtime started, with the
* budget the runtime measured, BEFORE it downloads the model: a `false` ends the job with
* `does-not-fit`. It is the same judgement `recommend()` makes for a candidate, without the
* ranking, so a model that fits is never ruled out for being listed sixth.
*
* `options` is the stage 2 input, as for `recommend()`; nothing reads it yet, so the budget
* is still the estimate. False for a model the catalogue does not have.
*/
export function modelFitsBudget(
hw: HardwareFacts,
catalog: LocalModelCatalog,
modelId: string,
context: number,
options: RecommendOptions = {}
): boolean {
void options;
const model = catalog.models.find((entry) => entry.id === modelId);
if (!model) return false;
return fitGrade(needBytes(model, context, catalog, hw), deviceBudget(hw).budgetBytes) !== 'out';
}
/**
* Whether the MACHINE is Apple silicon. `hw.optional.arm64` decides when it was read. Without
* it: a process under Rosetta, an arm64 process or an Apple chip name each prove Apple silicon
@@ -256,12 +280,18 @@ function none(
/**
* The recommendation for this Mac. `harnessFacts` holds one fact per catalogue harness the
* caller could resolve; a catalogue harness without a fact is not offered.
*
* `options` is the stage 2 input (`RecommendOptions`: the budget the runtime measured, and
* the runtime download an install still needs). The parameter is in place so callers compile
* against the final signature; nothing reads it yet.
*/
export function recommend(
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[]
harnessFacts: readonly HarnessFact[],
options: RecommendOptions = {}
): Recommendation {
void options;
// Off macOS and on an Intel Mac there is no budget to compute: a model has no memory to run in.
const noBudget: DeviceBudget = { limitBytes: 0, budgetBytes: 0, source: 'estimated' };
if (hw.platform !== 'darwin') return none('not-macos', noBudget);
+308
View File
@@ -0,0 +1,308 @@
/**
* @fileoverview The model runtime: which build Codeman runs, how it gets onto the disk, and
* how it is kept running.
*
* CONTRACT STAGE: `OLLAMA_RUNTIME_PIN` and the constants are final. Every function below has
* its final signature and a body that throws; the bodies are written next.
*
* What this module does:
* - THE PIN. `OLLAMA_RUNTIME_PIN` is a compiled constant. No file, setting, environment
* variable or request can change the URL, the size or the hash.
* - DOWNLOAD (`ensureRuntimeInstalled()`). The archive is streamed to
* `LocalModelPaths.archivePartial` while it is hashed. A cut stream is continued with a
* Range request, at most `RUNTIME_DOWNLOAD_RETRIES` times. A body longer than `pin.bytes`
* is refused. The sha256 is compared BEFORE anything is unpacked, and a mismatch deletes the
* file. Unpacking is `/usr/bin/tar` into `LocalModelPaths.unpackDir`, then one rename to
* `LocalModelPaths.runtimeDir`.
* - ENVIRONMENT (`runtimeEnv()`, pure). The child's environment is built from an ALLOWLIST:
* PATH, TMPDIR, LANG and the proxy variables (`HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`, in
* either case) are copied from the server's; HOME is the private one; and exactly these are
* set: `OLLAMA_HOST=127.0.0.1:<port>`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`,
* `OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`,
* `OLLAMA_MAX_LOADED_MODELS=1`. Nothing else gets through: no `CODEMAN_*` and none of the
* user's own `OLLAMA_*`.
* - PORT (`chooseRuntimePort()`). Chosen ONCE by binding, from `RUNTIME_PORT_FIRST` across
* `RUNTIME_PORT_SPAN`, then stored by the caller. Never searched again on a later start.
* - SUPERVISOR (`createRuntimeSupervisor()`). The runtime is a NON-detached child of the
* server. `start()` first looks at the port: a listener that answers `/api/version` and
* whose pid (`previousPid`) runs a binary under our runtime folder is our own orphan and is
* adopted or replaced; any other listener is the answer `port-in-use`. A child that exits by
* itself is restarted, at most `RUNTIME_MAX_RESTARTS` times in a row; `stop()` sends SIGTERM.
* The supervisor does not write the state file: it reports through `onChange`, and its
* caller stores the pid and the port.
* - BUDGET (`parseInferenceComputeTotal()`, pure). Right after it starts, the runtime prints
* the device total in its log line "inference compute" (`total="37.4 GiB"`). The supervisor
* reads the child's output for it and reports it as `measuredTotalBytes`.
* - REMOVAL (`removeLocalModelFiles()`). Deletes the runtime, the private home and, when
* asked, the models. Only inside `LocalModelPaths.root`.
*
* Under vitest nothing spawns and nothing is fetched unless a test injects it: the default
* `spawn` and `fetch` of the dependency objects refuse there.
*/
import type { ChildProcess, SpawnOptions } from 'node:child_process';
import type { LocalModelErrorCode, LocalModelRuntimePin } from '../types.js';
import type { LocalModelPaths } from './paths.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
// ---------- The pin and the constants ----------
/** The one runtime build Codeman downloads and runs. See the file header. */
export const OLLAMA_RUNTIME_PIN: LocalModelRuntimePin = Object.freeze({
id: 'ollama',
version: '0.40.15',
url: 'https://github.com/ollama/ollama/releases/download/v0.40.15/ollama-darwin.tgz',
bytes: 185_971_671,
sha256: '607c79d5d6315ef9d896db83adde736a4c0057196329843544e79b95afb260a8',
// Measured on a MacBook on 2026-11-06: the unpacked folder takes 579 MB.
unpackedBytes: 579_000_000,
minMacosMajor: 14,
});
/** The first loopback port tried for the runtime. Not 11434: that is the default of a runtime the user may run. */
export const RUNTIME_PORT_FIRST = 11435;
/** How many ports are tried from `RUNTIME_PORT_FIRST`. */
export const RUNTIME_PORT_SPAN = 20;
/** How often in a row a runtime that exited by itself is started again. */
export const RUNTIME_MAX_RESTARTS = 3;
/** How often a cut archive download is continued. */
export const RUNTIME_DOWNLOAD_RETRIES = 3;
/** `OLLAMA_KEEP_ALIVE`: how long the model stays loaded after the last request. */
export const RUNTIME_KEEP_ALIVE = '10m';
// ---------- Pure ----------
/**
* The device total of the runtime's log line "inference compute", in bytes, or `undefined`
* for any other line. The real line (a MacBook, 2026-11-06):
*
* `time=... level=INFO source=types.go:32 msg="inference compute" id=0 ... type=iGPU total="37.4 GiB" available="37.4 GiB"`
*
* reads 40,157,944,217 (37.4 x 1024^3, rounded down). The value is the quoted text after
* ` total=`: a number and a binary unit (`KiB`, `MiB`, `GiB`, `TiB`). `total_vram=` of the
* line "vram-based default context" is not read.
*/
export function parseInferenceComputeTotal(line: string): number | undefined {
return notImplemented('parseInferenceComputeTotal', line);
}
/** What `runtimeEnv()` is built from. */
export interface RuntimeEnvInput {
/** The server's environment. Only the allowlisted names are read from it. */
source: NodeJS.ProcessEnv;
/** The private HOME (`LocalModelPaths.homeDir`). */
home: string;
/** The model store (`LocalModelPaths.modelsDir`). */
modelsDir: string;
/** The loopback port. */
port: number;
/** The context length in tokens. */
contextLength: number;
}
/** The environment of the runtime child: the allowlist of the file header, nothing more. Pure. */
export function runtimeEnv(input: RuntimeEnvInput): NodeJS.ProcessEnv {
return notImplemented('runtimeEnv', input);
}
/**
* Whether `command` (one line of `ps -o command= -p <pid>`) is a process whose binary lies
* under `runtimeRoot` (`LocalModelPaths.runtimeRoot`): our own runtime, so an orphan of an
* earlier server. Pure. `/private` in front of a path is the same path on macOS.
*/
export function isOwnRuntimeCommand(command: string, runtimeRoot: string): boolean {
return notImplemented('isOwnRuntimeCommand', command, runtimeRoot);
}
// ---------- Download and unpack ----------
/** How far the archive download is. */
export interface RuntimeDownloadProgress {
completedBytes: number;
/** `pin.bytes`. */
totalBytes: number;
}
/** What `ensureRuntimeInstalled()` calls. Each member defaults to the real thing; a test injects its own. */
export interface RuntimeInstallDeps {
/** Fetches the archive. Must honour `signal` and a `Range` header. */
fetch: typeof fetch;
/** Unpacks `archive` into the existing empty folder `intoDir` (`/usr/bin/tar -xzf`). Rejects on a non-zero exit. */
untar: (archive: string, intoDir: string, signal?: AbortSignal) => Promise<void>;
/** Waits between retries. */
sleep: (ms: number) => Promise<void>;
}
export interface RuntimeInstallOptions {
/** Default `OLLAMA_RUNTIME_PIN`. A test passes a pin that points at its fake server. */
pin?: LocalModelRuntimePin;
/** Default `localModelPaths(pin.version)`. */
paths?: LocalModelPaths;
/** Aborts the download or the unpack. The partial archive stays for a later Range request. */
signal?: AbortSignal;
/** Called while the archive downloads, at most a few times a second. */
onProgress?: (progress: RuntimeDownloadProgress) => void;
deps?: Partial<RuntimeInstallDeps>;
}
/**
* How getting the runtime ended. A failure carries the job's error code; `cancelled` is the
* abort signal and is not an error.
*/
export type RuntimeInstallResult =
| {
ok: true;
/** `LocalModelPaths.binary`. */
binaryPath: string;
/** True: the folder was already unpacked and nothing was downloaded. */
alreadyInstalled: boolean;
}
| {
ok: false;
code: Extract<
LocalModelErrorCode,
'runtime-download-failed' | 'runtime-damaged' | 'runtime-unpack-failed' | 'disk-full'
>;
detail?: string;
}
| { ok: false; code: 'cancelled' };
/** Whether the pinned runtime is unpacked on disk (its binary exists). Never throws. */
export async function isRuntimeInstalled(pin?: LocalModelRuntimePin, paths?: LocalModelPaths): Promise<boolean> {
return notImplemented('isRuntimeInstalled', pin, paths);
}
/**
* Makes sure the pinned runtime is unpacked: nothing to do when it is, else download (with
* Range resume), compare the hash, unpack, rename. Idempotent, so a resumed job calls it
* again. Never throws: every outcome is a `RuntimeInstallResult`.
*/
export async function ensureRuntimeInstalled(options: RuntimeInstallOptions = {}): Promise<RuntimeInstallResult> {
return notImplemented('ensureRuntimeInstalled', options);
}
/** What `removeLocalModelFiles()` deletes. */
export interface RemoveLocalModelFilesOptions {
/** True: delete the model store too. False: the runtime, its archive and the private home only. */
models: boolean;
/** Default `localModelPaths(OLLAMA_RUNTIME_PIN.version)`. */
paths?: LocalModelPaths;
}
/**
* Deletes what this feature put on disk: every runtime folder and archive, the private home
* and, when asked, the models. Refuses any path that is not inside `LocalModelPaths.root`,
* follows no symlink out of it, and removes `root` itself only when it is empty afterwards.
* The state file and the coding agent are not its business.
*/
export async function removeLocalModelFiles(options: RemoveLocalModelFilesOptions): Promise<void> {
return notImplemented('removeLocalModelFiles', options);
}
// ---------- Port ----------
export interface ChooseRuntimePortOptions {
/** Default `RUNTIME_PORT_FIRST`. */
first?: number;
/** Default `RUNTIME_PORT_SPAN`. */
span?: number;
/** Ports to skip: the one that is in use when the person asked for another. */
avoid?: readonly number[];
/** Whether nothing holds `port` on 127.0.0.1, decided by BINDING it. A test injects its own. */
isFree?: (port: number) => Promise<boolean>;
}
/** The first free loopback port of the span, or `undefined` when none is free. Chosen once, then stored by the caller. */
export async function chooseRuntimePort(options: ChooseRuntimePortOptions = {}): Promise<number | undefined> {
return notImplemented('chooseRuntimePort', options);
}
// ---------- Supervisor ----------
/** What the supervisor knows about the runtime right now. Plain data. */
export interface RuntimeStatus {
/** The child Codeman started (or adopted) is alive and answered `/api/version`. */
running: boolean;
pid?: number;
port?: number;
/** The context length the running child was started with. */
contextLength?: number;
/** The device total parsed from the child's log, in bytes. Absent for an adopted orphan (its log is not ours to read). */
measuredTotalBytes?: number;
/** How often the child was restarted since the last `start()`. */
restarts: number;
/** True: the child exited `RUNTIME_MAX_RESTARTS` times in a row and is no longer restarted (`runtime-stopped`). */
gaveUp: boolean;
}
export interface RuntimeStartOptions {
/** The stored port. */
port: number;
/** The context length in tokens (`OLLAMA_CONTEXT_LENGTH`). A running child with another value is replaced. */
contextLength: number;
/** The pid the state file holds from an earlier server: decides whether a listener on `port` is our own orphan. */
previousPid?: number;
}
/** How a `start()` ended. */
export type RuntimeStartResult =
| {
ok: true;
pid: number;
port: number;
/** What happened to a listener that was already on the port: there was none, or it was ours. */
orphan: 'none' | 'adopted' | 'replaced';
/** The device total, when the log line was read before `start()` returned. */
measuredTotalBytes?: number;
}
| {
ok: false;
/** `port-in-use`: a listener that is not ours. `runtime-start-failed`: no answer in time, or the child exited. */
code: Extract<LocalModelErrorCode, 'port-in-use' | 'runtime-start-failed'>;
detail?: string;
};
/** What the supervisor calls. Each member defaults to the real thing; a test injects its own. */
export interface RuntimeSupervisorDeps {
/** Starts the child. The default refuses under vitest. */
spawn: (command: string, args: readonly string[], options: SpawnOptions) => ChildProcess;
/** Asks the runtime for `/api/version`. */
fetch: typeof fetch;
/** The server's environment, the `source` of `runtimeEnv()`. */
env: NodeJS.ProcessEnv;
/** `ps -o command= -p <pid>`, bounded. `undefined` when the pid is gone. */
commandOfPid: (pid: number) => Promise<string | undefined>;
/** Signals a pid that is not our child (an orphan). */
kill: (pid: number, signal: NodeJS.Signals) => void;
/** Whether nothing holds `port` on 127.0.0.1, decided by binding it. */
isPortFree: (port: number) => Promise<boolean>;
sleep: (ms: number) => Promise<void>;
/** Default `OLLAMA_RUNTIME_PIN`. */
pin: LocalModelRuntimePin;
/** Default `localModelPaths(pin.version)`. */
paths: LocalModelPaths;
/** Called after every change of `status()`: a start, an exit, a restart, the budget line. The caller stores pid and port. */
onChange: (status: RuntimeStatus) => void;
}
/** The one supervisor of the runtime child. */
export interface RuntimeSupervisor {
/**
* Makes sure the runtime runs on `port` with `contextLength`. Idempotent: a child that
* already runs with both is kept. Resolves once `/api/version` answers the pinned version
* and the budget line was read (or a few seconds passed without it). Never throws.
*/
start(options: RuntimeStartOptions): Promise<RuntimeStartResult>;
/** SIGTERM to the child, SIGKILL after a grace period. Resolves once it is gone. Safe to call when nothing runs. */
stop(): Promise<void>;
status(): RuntimeStatus;
}
/** A supervisor. The server keeps ONE (see `getLocalModelService()` in install-job.ts). */
export function createRuntimeSupervisor(deps: Partial<RuntimeSupervisorDeps> = {}): RuntimeSupervisor {
return notImplemented('createRuntimeSupervisor', deps);
}
+57
View File
@@ -0,0 +1,57 @@
/**
* @fileoverview The ONE reader and writer of `local-model.json`, the state file of the local
* model install (`LocalModelStateFile`).
*
* CONTRACT STAGE: `IDLE_LOCAL_MODEL_STATE` is final. Every function below has its final
* signature and a body that throws; the bodies are written next.
*
* Why one module: the install job writes the state, the step and the digest, and the caller
* of the runtime supervisor writes the pid and the port. Two writers with their own temp file
* and rename would lose each other's change, which is the fault `mutateCustomModelHosts()`
* exists to end for the endpoint store. So every write goes through `mutateLocalModelState()`.
*
* The shape of `mutateSettingsFile()` (src/web/settings-writer.ts):
* - Serialized: mutations run one at a time on one promise chain.
* - Atomic: each write goes to a temp file with a random name beside the target and is
* renamed over it. The file is mode 0600.
* - Reading is TOTAL, unlike the settings writer: the file holds no setting of the user, only
* what this feature recorded, so a missing file, one that does not parse or one with
* another `schema` reads as `IDLE_LOCAL_MODEL_STATE` and the next write replaces it.
* - The path is `localModelPaths().stateFile`. The file itself holds no path.
*/
import type { LocalModelStateFile } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** The state when nothing was ever installed. What an unreadable file reads as. */
export const IDLE_LOCAL_MODEL_STATE: LocalModelStateFile = Object.freeze({ schema: 1, state: 'idle' });
/**
* The state a file's text stands for. Pure and total: text that is not a JSON object of
* `schema: 1` with a known `state` is `IDLE_LOCAL_MODEL_STATE`. Members it does not know are
* dropped, and a member of the wrong type is dropped rather than trusted.
*/
export function parseLocalModelState(raw: string): LocalModelStateFile {
return notImplemented('parseLocalModelState', raw);
}
/** The stored state. Never throws: a file that cannot be read is `IDLE_LOCAL_MODEL_STATE`. */
export async function readLocalModelState(): Promise<LocalModelStateFile> {
return notImplemented('readLocalModelState');
}
/**
* Runs one change to the state file. `fn` receives the current state and returns the state to
* write, or `undefined` to write nothing. Resolves with the state before and after the change
* (`after` is `undefined` when nothing was written). `updatedAt` is stamped here. A failed
* write rejects its own caller only; the chain keeps going.
*/
export function mutateLocalModelState(
fn: (current: LocalModelStateFile) => LocalModelStateFile | undefined
): Promise<{ before: LocalModelStateFile; after: LocalModelStateFile | undefined }> {
return notImplemented('mutateLocalModelState', fn);
}
+3 -1
View File
@@ -29,7 +29,8 @@
* | plan | PlanItem, PlanTaskStatus, TddPhase | In-memory → `GET /api/sessions/:id/plan/tasks` |
* | orchestrator | OrchestratorState, OrchestratorPlan, OrchestratorConfig, OrchestratorPersistState | `~/.codeman/state.json` → `GET /api/orchestrator/status` |
* | mcp-sync | McpSyncResult, McpSyncTargetResult | Other CLIs' own config files → `GET`/`POST /api/mcp-sync` |
* | local-model | HardwareFacts, LocalModelCatalog, Candidate, Recommendation, LocalModelStatus, LOCAL_MODEL_LABELS | In-memory only (nothing persisted) → `GET /api/local-model` |
* | local-model | HardwareFacts, LocalModelCatalog, Candidate, Recommendation, RecommendOptions, LOCAL_MODEL_LABELS | In-memory only (nothing persisted) → `GET /api/local-model` |
* | local-model-install | LocalModelStatus, LocalModelInstallView, LocalModelStateFile, LocalModelRuntimePin, LOCAL_MODEL_INSTALL_LABELS | `~/.codeman/local-model.json` → `GET /api/local-model` and its install verbs |
*
* ## Cross-domain relationship map
*
@@ -79,3 +80,4 @@ export * from './intent.js';
export * from './foreign-tmux.js';
export * from './mcp-sync.js';
export * from './local-model.js';
export * from './local-model-install.js';
+582
View File
@@ -0,0 +1,582 @@
/**
* @fileoverview Types of the local model INSTALL: the runtime pin, the job's states, steps and
* error codes, the progress, the `install` object of `GET /api/local-model`, the bodies and
* answers of the install verbs, and the shape of the state file.
*
* The read side (hardware, catalogue, recommendation) is in src/types/local-model.ts. This
* file holds `LocalModelStatus`, the whole `data` of the GET, because it joins the two.
*
* Who produces what:
* - `LocalModelRuntimePin`: a compiled constant in src/local-model/runtime.ts. No file, no
* setting and no request can change which binary is downloaded.
* - `LocalModelStateFile`: src/local-model/state-store.ts, the ONE writer of
* `local-model.json` (mode 0600, temp file + rename).
* - `LocalModelInstallView`: src/local-model/install-job.ts (`buildInstallView()`, pure), from
* the state file plus what is live (the runtime child, the developer tools check).
* - `LocalModelStatus` and the verb answers: src/web/routes/local-model-routes.ts.
*
* The HTTP contract. Every route is admin only in multi-user mode (403 `FORBIDDEN`) and
* EXPERIMENTAL like the GET (docs/versioning-policy.md). Every verb answers
* `{ success: true, data: LocalModelInstallView }` with the view AFTER the verb.
*
* | Route | Body | Refusals |
* |----------------------------------|-----------------------------|-------------------------------------------------------|
* | `GET /api/local-model` | none | none (a failed probe is an answer) |
* | `POST /api/local-model/install` | `LocalModelInstallRequest` | 400 not a row the server offers; 409 `job-running`, |
* | | | `already-installed`, `size-changed` |
* | `POST /api/local-model/cancel` | none | 409 `wrong-state` (no job is running) |
* | `POST /api/local-model/resume` | `LocalModelResumeRequest` | 409 `wrong-state` (not paused or failed) |
* | `POST /api/local-model/runtime` | `LocalModelRuntimeRequest` | 409 `wrong-state` (not installed), `job-running` |
* | `DELETE /api/local-model` | `LocalModelRemoveRequest` | 409 `job-running`, `sessions-open` |
*
* A 409 is the error envelope plus `data: LocalModelConflict`:
* `{ success: false, error, errorCode: 'CONFLICT', data: { reason, message, ... } }`.
*
* The rules of src/types/local-model.ts hold here too:
* 1. A size a person reads travels twice: as bytes and as decimal GB text from `formatGb()`
* (`completedGb`, `totalGb`, `downloadGb`). The browser prints the text and adds or
* divides nothing for display. `mbPerSecond` is the one other unit, decimal MB per second.
* 2. Every sentence the server chooses comes from `LOCAL_MODEL_INSTALL_LABELS`, with its
* numbers and names in `vars`. `{name}` is taken by the translator, so no slot has that name.
* 3. A CLI id is data (`LocalModelInstallView.cli`): the browser passes it on and compares it
* with nothing. What a person reads is `cliLabel`.
*/
import type { HardwareFacts, Recommendation } from './local-model.js';
// ---------- The runtime pin ----------
/**
* The one runtime build Codeman downloads and runs. A compiled constant
* (`OLLAMA_RUNTIME_PIN` in src/local-model/runtime.ts): the hash is compared before anything
* is unpacked, so the pin and the hash must come from the same reviewed source file.
*/
export interface LocalModelRuntimePin {
/** The runtime's family. One value today. */
id: 'ollama';
/** The release ('0.40.15'). `GET /api/version` of the running runtime must answer it. */
version: string;
/** Where the archive is downloaded from. https, a fixed release asset. */
url: string;
/** Size of the archive in bytes. A body that grows past it is refused before the hash is finished. */
bytes: number;
/** Lowercase hex sha256 of the archive. */
sha256: string;
/** What the unpacked folder takes on disk, in bytes (measured: about 579 MB). */
unpackedBytes: number;
/** The oldest macOS major version the runtime supports. */
minMacosMajor: number;
}
// ---------- States, steps, codes ----------
/**
* Where the install stands. The verb of the flow is the same throughout: install, installing,
* installed.
* - `idle`: nothing is installed and no job exists.
* - `installing`: the one job is running; `step` says where.
* - `paused`: the job stopped at `step` because of Cancel or a server restart. It never
* continues without a click.
* - `failed`: the job stopped at `step` with `error`.
* - `installed`: the model is connected. Whether the runtime runs is `running`, not a state.
* The state table (what is on disk, what a restart and Cancel do, the allowed actions) is
* `INSTALL_STATE_TABLE` in src/local-model/install-job.ts.
*/
export type LocalModelInstallState = 'idle' | 'installing' | 'paused' | 'failed' | 'installed';
/**
* The steps of a job, in the order they run.
* - `runtime`: get the pinned runtime (download, compare the hash, unpack), start it, read
* the measured memory budget and judge the chosen model against it.
* - `model`: download the model (the runtime's pull).
* - `verify`: load the model and check it: one tiny chat, the memory and context the runtime
* reports, one tool call.
* - `agent`: install the coding agent. Left out of a job whose coding agent is installed.
* - `connect`: save the connection (the endpoint record and the settings flag).
*/
export type LocalModelInstallStep = 'runtime' | 'model' | 'verify' | 'agent' | 'connect';
/** Every step, in order. A job runs these, without `agent` when the coding agent is installed. */
export const LOCAL_MODEL_INSTALL_STEPS: readonly LocalModelInstallStep[] = [
'runtime',
'model',
'verify',
'agent',
'connect',
];
/**
* Why a job failed, or (in state `installed`) what blocks a session. Each code has one
* sentence (`LOCAL_MODEL_ERROR_LABELS`) and one helpful next step (`LOCAL_MODEL_ERROR_NEXT`).
*
* | Code | Raised when |
* |---------------------------|--------------------------------------------------------------------------|
* | `runtime-download-failed` | the archive download still fails after its retries |
* | `runtime-damaged` | the archive's sha256 is not the pinned one (the file was deleted) |
* | `runtime-unpack-failed` | `/usr/bin/tar` failed |
* | `runtime-start-failed` | the runtime was started and never answered `/api/version` |
* | `runtime-stopped` | installed: the runtime crashed and was restarted 3 times, then given up |
* | `port-in-use` | another program listens on the stored port, or no port of the span is free |
* | `disk-full` | a write failed for lack of space |
* | `pull-failed` | the pull still ends on an error line or a cut stream after its retries |
* | `model-not-found` | the registry no longer has this model under this name |
* | `does-not-fit` | the measured budget rules the model out, or the runtime did not put all of it on the GPU |
* | `context-not-applied` | the runtime loaded the model with another context than the chosen one |
* | `load-failed` | the tiny chat that loads the model failed |
* | `no-tool-call` | the tool probe was answered without a tool call with parseable arguments |
* | `probe-unfinished` | the tool probe ended on its token limit, so it proves nothing |
* | `harness-failed` | the coding agent's install command failed, or its binary is still missing |
* | `endpoint-id-taken` | a saved endpoint with our id exists that this feature did not write |
* | `connect-failed` | the endpoint record or the settings flag could not be written |
* | `model-changed` | installed: the model's digest is not the one stored at install time |
* | `internal` | anything unexpected (the exception text is in `detail`) |
*/
export type LocalModelErrorCode =
| 'runtime-download-failed'
| 'runtime-damaged'
| 'runtime-unpack-failed'
| 'runtime-start-failed'
| 'runtime-stopped'
| 'port-in-use'
| 'disk-full'
| 'pull-failed'
| 'model-not-found'
| 'does-not-fit'
| 'context-not-applied'
| 'load-failed'
| 'no-tool-call'
| 'probe-unfinished'
| 'harness-failed'
| 'endpoint-id-taken'
| 'connect-failed'
| 'model-changed'
| 'internal';
/**
* The ONE action that helps after an error. The window draws one button for it.
* - `retry`: "Try again". `POST /api/local-model/resume`.
* - `other-port`: "Use another port". `POST /api/local-model/resume` with `newPort: true`
* (state `failed`), or `POST /api/local-model/runtime` with `action: 'start'` and
* `newPort: true` (state `installed`).
* - `next-model`: "Try <model>". Back to the chooser with `error.nextModel` preselected; the
* person confirms its sizes and a new `POST /api/local-model/install` replaces the job.
* Without a `nextModel` the window offers the chooser and Remove.
* - `start`: "Start". `POST /api/local-model/runtime` with `action: 'start'`.
* - `remove`: "Remove". `DELETE /api/local-model`.
*/
export type LocalModelNextStep = 'retry' | 'other-port' | 'next-model' | 'start' | 'remove';
/**
* What a client may do in the current state. The view's `actions` lists exactly the ones the
* server would accept right now, so the window shows a button only for those.
* - `install`: `POST /api/local-model/install`.
* - `cancel`: `POST /api/local-model/cancel`.
* - `resume`: `POST /api/local-model/resume`.
* - `start` and `stop`: `POST /api/local-model/runtime`.
* - `remove`: `DELETE /api/local-model`.
* - `start-session`: not a route of this feature: the window starts a session on the endpoint
* `endpointId` with the coding agent `cli`. Offered only while the runtime runs and nothing
* blocks a session.
*/
export type LocalModelAction = 'install' | 'cancel' | 'resume' | 'start' | 'stop' | 'remove' | 'start-session';
/** Whether the Mac has the developer tools (git, Python) a coding agent calls. */
export type LocalModelDeveloperTools =
/** `/usr/bin/xcode-select -p` succeeded. */
| 'present'
/** It failed: macOS will offer to install the tools the first time an agent needs one. */
| 'missing'
/** Not macOS, or the check did not answer in time. Nothing is said then. */
| 'unknown';
// ---------- Sentences ----------
/**
* Every sentence the server may put into the `install` object or a 409, in English (the i18n
* source language). Like `LOCAL_MODEL_LABELS`: the zh-CN dictionary in src/web/public/i18n.js
* has one entry per value, keyed by this exact text, and the i18n test lists them from here.
* A constant of its own, so the sentences of the recommendation and of the install can be
* listed apart.
*
* | Key | Slots (all text) |
* |------------------------|----------------------------------------------------------------------|
* | `errPortInUse` | `port`: the port number |
* | `errModelNotFound`, `errDoesNotFit`, `errLoadFailed`, `errNoToolCall`, `errProbeUnfinished` | `model`: the model's label |
* | `errContextNotApplied` | `model`; `actual`: the context the runtime reports; `wanted`: the chosen one |
* | `errHarnessFailed` | `cli`: the coding agent's label |
* | `errEndpointIdTaken` | `id`: the endpoint id this feature uses |
* | `conflictSizeChanged` | `size`: the server's own total as decimal GB |
* | `conflictSessionsOpen` | `sessions`: the session names, joined with ', ' |
*
* `developerToolsMissing` is not sent: the view carries `developerTools` and the window prints
* this sentence when it is `missing`. It is here so that its wording has one source.
*/
export const LOCAL_MODEL_INSTALL_LABELS = {
errRuntimeDownloadFailed:
'The download of the runtime stopped and could not be continued. Check the network connection, then try again.',
errRuntimeDamaged: 'The runtime download was damaged and has been deleted. Try again.',
errRuntimeUnpackFailed: 'The runtime could not be unpacked on this Mac.',
errRuntimeStartFailed: 'The runtime did not start on this Mac.',
errRuntimeStopped: 'The runtime keeps stopping, so Codeman no longer restarts it.',
errPortInUse: 'Port {port} is in use by another program.',
errDiskFull: 'The disk is full. Free up space, then try again.',
errPullFailed: 'The download keeps stopping. The network may be damaging data. Try again or use another network.',
errModelNotFound: '{model} is no longer offered under this name. Update Codeman or pick another model.',
errDoesNotFit: '{model} does not fit in the memory of this Mac after all.',
errContextNotApplied: '{model} loaded with a context of {actual} tokens instead of {wanted}.',
errLoadFailed: '{model} did not load on this Mac.',
errNoToolCall: '{model} loaded but did not call a tool, so a coding agent cannot work with it.',
errProbeUnfinished: 'The check of {model} did not finish.',
errHarnessFailed: '{cli} could not be installed.',
errEndpointIdTaken:
'A saved model connection named {id} already exists and was not created here. Rename or delete it in App Settings, then try again.',
errConnectFailed: 'Codeman could not save the connection to the model.',
errModelChanged: 'The model on this Mac changed since it was installed. Remove it and install it again.',
errInternal: 'The install stopped on an unexpected error.',
conflictJobRunning: 'An install is already running.',
conflictAlreadyInstalled: 'A model is already installed on this Mac. Remove it first.',
conflictSizeChanged: 'The download is {size} GB now. Check the sizes and confirm again.',
conflictSessionsOpen: 'Sessions are still using this model: {sessions}. Close them first.',
conflictWrongState: 'That is not possible in the current state.',
developerToolsMissing:
'Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer to install them the first time an agent needs one.',
} as const;
/** One of the texts in `LOCAL_MODEL_INSTALL_LABELS`. */
export type LocalModelInstallLabel = (typeof LOCAL_MODEL_INSTALL_LABELS)[keyof typeof LOCAL_MODEL_INSTALL_LABELS];
/** A sentence of the install: a text from `LOCAL_MODEL_INSTALL_LABELS` plus the values of its `{placeholder}` slots. */
export interface LocalModelInstallMessage {
text: LocalModelInstallLabel;
/** Slot values, already formatted as text. Absent when the text has no slot. */
vars?: Record<string, string>;
}
/** The sentence of each error code. */
export const LOCAL_MODEL_ERROR_LABELS: Record<LocalModelErrorCode, LocalModelInstallLabel> = {
'runtime-download-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDownloadFailed,
'runtime-damaged': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDamaged,
'runtime-unpack-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeUnpackFailed,
'runtime-start-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStartFailed,
'runtime-stopped': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStopped,
'port-in-use': LOCAL_MODEL_INSTALL_LABELS.errPortInUse,
'disk-full': LOCAL_MODEL_INSTALL_LABELS.errDiskFull,
'pull-failed': LOCAL_MODEL_INSTALL_LABELS.errPullFailed,
'model-not-found': LOCAL_MODEL_INSTALL_LABELS.errModelNotFound,
'does-not-fit': LOCAL_MODEL_INSTALL_LABELS.errDoesNotFit,
'context-not-applied': LOCAL_MODEL_INSTALL_LABELS.errContextNotApplied,
'load-failed': LOCAL_MODEL_INSTALL_LABELS.errLoadFailed,
'no-tool-call': LOCAL_MODEL_INSTALL_LABELS.errNoToolCall,
'probe-unfinished': LOCAL_MODEL_INSTALL_LABELS.errProbeUnfinished,
'harness-failed': LOCAL_MODEL_INSTALL_LABELS.errHarnessFailed,
'endpoint-id-taken': LOCAL_MODEL_INSTALL_LABELS.errEndpointIdTaken,
'connect-failed': LOCAL_MODEL_INSTALL_LABELS.errConnectFailed,
'model-changed': LOCAL_MODEL_INSTALL_LABELS.errModelChanged,
internal: LOCAL_MODEL_INSTALL_LABELS.errInternal,
};
/** The one helpful next step of each error code. */
export const LOCAL_MODEL_ERROR_NEXT: Record<LocalModelErrorCode, LocalModelNextStep> = {
'runtime-download-failed': 'retry',
'runtime-damaged': 'retry',
'runtime-unpack-failed': 'retry',
'runtime-start-failed': 'retry',
'runtime-stopped': 'start',
'port-in-use': 'other-port',
'disk-full': 'retry',
'pull-failed': 'retry',
'model-not-found': 'next-model',
'does-not-fit': 'next-model',
'context-not-applied': 'retry',
'load-failed': 'retry',
'no-tool-call': 'next-model',
'probe-unfinished': 'retry',
'harness-failed': 'retry',
'endpoint-id-taken': 'retry',
'connect-failed': 'retry',
'model-changed': 'remove',
internal: 'retry',
};
// ---------- The `install` object of the GET ----------
/**
* How far a download is. Present while the `runtime` step downloads the archive and while the
* `model` step pulls, once the total is known; kept on a paused or failed job as it stood.
* During the `model` step both byte figures come from the pull's own lines, never from the
* catalogue.
*/
export interface LocalModelProgress {
completedBytes: number;
totalBytes: number;
/** `completedBytes` as decimal GB text, no unit ('6.2'). */
completedGb: string;
/** `totalBytes` as decimal GB text, no unit ('23.1'). */
totalGb: string;
/** Download speed. Absent until it can be told (the first moments, a paused job). */
bytesPerSecond?: number;
/** `bytesPerSecond` as whole decimal megabytes per second, no unit ('58'). Present exactly when `bytesPerSecond` is. */
mbPerSecond?: string;
/** Seconds left at the current speed, rounded up. Absent when the speed is unknown or 0. */
secondsLeft?: number;
}
/** An error as the window shows it: the sentence, the one action that helps, and the raw detail. */
export interface LocalModelInstallError {
code: LocalModelErrorCode;
/** `LOCAL_MODEL_ERROR_LABELS[code]` with its slots filled. */
message: LocalModelInstallMessage;
/** `LOCAL_MODEL_ERROR_NEXT[code]`. */
next: LocalModelNextStep;
/**
* For `next-model`: the best model that still fits and is not the one that failed. Absent
* when there is none.
*/
nextModel?: { modelId: string; modelLabel: string };
/**
* Raw text for the person who wants to know more: the last lines of an installer, the
* message of a refused write, an exception. Not translated; printed with `textContent`.
*/
detail?: string;
}
/** The model of the job or of the installation. */
export interface LocalModelInstalledModel {
/** `LocalModelEntry.id`. */
id: string;
/** `LocalModelEntry.label`. */
label: string;
}
/** The runtime, as the consent screen needs it. */
export interface LocalModelRuntimeView {
/** `LocalModelRuntimePin.version`. */
version: string;
/** The pinned runtime is unpacked on disk. Then an install downloads only the model. */
installed: boolean;
/** What an install would download for the runtime: the archive size, 0 when `installed`. */
downloadBytes: number;
/** `downloadBytes` as decimal GB text, no unit ('0.2'). */
downloadGb: string;
/** What the unpacked runtime takes on disk, as decimal GB text, no unit ('0.6'). */
unpackedGb: string;
}
/** What the runtime reported about the loaded model when the job checked it. The facts of the Done line. */
export interface LocalModelLoadedFacts {
/** The context length in tokens the runtime loaded the model with. */
context: number;
/** Memory the loaded model took, in bytes (`size` of the runtime's process list). */
memoryBytes: number;
/** `memoryBytes` as decimal GB text, no unit. */
memoryGb: string;
/** All of it is on the GPU (`size_vram` equals `size`). Always true after a job that passed. */
onGpu: boolean;
}
/**
* The `install` object of `GET /api/local-model`: where the install stands and what can be
* done about it. The window polls it once a second while it is open; there is no SSE event.
*
* Which members are present, by state:
*
* | Member | idle | installing | paused | failed | installed |
* |-----------------------------------|------|------------|--------|--------|-----------|
* | `step`, `steps` | | yes | yes | yes | |
* | `progress` | | during a download | as it stood, when known | as it stood, when known | |
* | `error` | | | | yes | only when something blocks a session |
* | `model`, `cli`, `cliLabel`, `context` | | yes | yes | yes | yes |
* | `port` | | once chosen | once chosen | once chosen | yes |
* | `loaded`, `endpointId` | | | | | yes |
*
* `running`, `developerTools`, `runtime`, `folder` and `actions` are always present.
*/
export interface LocalModelInstallView {
state: LocalModelInstallState;
/** The step the job is in, or stopped in. */
step?: LocalModelInstallStep;
/** The steps of THIS job in order, for the step list (`agent` is absent when the coding agent was already installed). */
steps?: LocalModelInstallStep[];
progress?: LocalModelProgress;
error?: LocalModelInstallError;
model?: LocalModelInstalledModel;
/** The coding agent: a CLI registry id, as data. */
cli?: string;
/** The coding agent's display label ('OpenCode'). */
cliLabel?: string;
/** The context length in tokens the runtime is started with. */
context?: number;
/** The loopback port of the runtime. */
port?: number;
/** The runtime child Codeman started is alive right now. */
running: boolean;
developerTools: LocalModelDeveloperTools;
runtime: LocalModelRuntimeView;
/** The absolute folder everything of this feature is kept in, for the "where it goes" line. */
folder: string;
loaded?: LocalModelLoadedFacts;
/** The id of the saved endpoint a session is started on. */
endpointId?: string;
/** What the server accepts right now, see `LocalModelAction`. */
actions: LocalModelAction[];
}
/**
* The `data` of `GET /api/local-model`. Admin only in multi-user mode.
*
* `install` is optional in the type only until the route emits it: from the stage that ships
* the install verbs on it is always present.
*/
export interface LocalModelStatus {
hardware: HardwareFacts;
recommendation: Recommendation;
/** `LocalModelCatalog.updated`. */
catalogUpdated: string;
install?: LocalModelInstallView;
}
// ---------- Request bodies and answers ----------
/**
* Body of `POST /api/local-model/install`. The three choices must name a row the server offers
* right now: a model of `recommendation.pick` or `recommendation.alternatives`, at that
* candidate's `context`, with one of its `harnesses`. Anything else is a 400 before any
* download.
*/
export interface LocalModelInstallRequest {
/** `Candidate.modelId`. */
modelId: string;
/** `CandidateHarness.cli`. */
cli: string;
/** `Candidate.context`. */
context: number;
/**
* The total the person saw and confirmed: `Candidate.installBytes`. The server compares it
* with its own total and answers 409 `size-changed` when they differ, so a consent is bound
* to the size that was shown.
*/
acknowledgedBytes: number;
}
/** Body of `POST /api/local-model/resume`. May be absent or empty. */
export interface LocalModelResumeRequest {
/** True: choose another port first (the answer to `port-in-use`). */
newPort?: boolean;
}
/** Body of `POST /api/local-model/runtime`. */
export interface LocalModelRuntimeRequest {
/** `stop` also keeps the runtime from being started at the next boot; `start` undoes that. */
action: 'start' | 'stop';
/** With `start`: choose another port first (the answer to `port-in-use`), and move the saved endpoint to it. */
newPort?: boolean;
}
/** Body of `DELETE /api/local-model`. May be absent or empty. */
export interface LocalModelRemoveRequest {
/** True: delete the downloaded model too. Default false: the model stays for a later install. */
removeModel?: boolean;
}
/** What every verb answers on success: the view after the verb. */
export type LocalModelVerbAnswer = LocalModelInstallView;
/**
* Why a verb was refused with 409.
* - `job-running`: a job is running (a second install, a resume, a remove or a runtime action).
* - `already-installed`: install while a model is installed. Remove it first.
* - `size-changed`: `acknowledgedBytes` is not the server's total; `expectedBytes` and
* `expectedGb` carry that total, so the window can show it and ask again.
* - `sessions-open`: remove while live sessions use the endpoint; `sessions` names them.
* - `wrong-state`: the verb makes no sense in the current state (see the route table above).
*/
export type LocalModelConflictReason =
| 'job-running'
| 'already-installed'
| 'size-changed'
| 'sessions-open'
| 'wrong-state';
/** The `data` of a 409 of an install verb. */
export interface LocalModelConflict {
reason: LocalModelConflictReason;
/** The sentence for the window (`conflict...` of `LOCAL_MODEL_INSTALL_LABELS`). */
message: LocalModelInstallMessage;
/** `size-changed`: the server's own total in bytes. */
expectedBytes?: number;
/** `size-changed`: `expectedBytes` as decimal GB text, no unit. */
expectedGb?: string;
/** `sessions-open`: the names of the live sessions that use the endpoint. */
sessions?: string[];
}
// ---------- The state file ----------
/**
* `local-model.json` in the data dir: everything that must survive a restart. Mode 0600,
* written through a temp file and a rename, by src/local-model/state-store.ts alone.
*
* Rules:
* - It holds NO path. Every path of this feature is built from `dataPath()` constants
* (`localModelPaths()`), so a state file can never point a delete somewhere else.
* - Reading is total: a missing file, a file that does not parse or one with another `schema`
* reads as `{ schema: 1, state: 'idle' }`.
* - A file read at boot with state `installing` is a job a restart interrupted: it becomes
* `paused` before anything else looks at it.
*/
export interface LocalModelStateFile {
schema: 1;
state: LocalModelInstallState;
/** The step the job is in or stopped in. Absent in `idle` and `installed`. */
step?: LocalModelInstallStep;
/** The steps of the job, in order. Absent in `idle` and `installed`. */
steps?: LocalModelInstallStep[];
/** State `failed`: why. State `installed`: absent (a blocking problem is found live, never stored). */
error?: {
code: LocalModelErrorCode;
/** The slot values of the code's sentence. */
vars?: Record<string, string>;
detail?: string;
/** For a `next-model` code: the model to offer, when the job knew one. */
nextModelId?: string;
};
/** The model of the job or of the installation. */
model?: {
id: string;
label: string;
/** The context length in tokens the person chose. */
context: number;
/** The total the pull reported, once known. */
pullBytes?: number;
/** The digest `/api/tags` reported after the pull. Compared again at boot and before a session starts. */
digest?: string;
};
/** The coding agent of the job or of the installation. */
cli?: { id: string; label: string };
/** The runtime, once its port was chosen. */
runtime?: {
/** The pinned version that was unpacked. */
version: string;
/** The loopback port, chosen once by binding. */
port: number;
/** The pid of the child Codeman last started. Used at boot to tell our own orphan from another program. */
pid?: number;
/** False after Stop: the runtime is not started at boot. */
enabled: boolean;
/** The device total the runtime printed when it started, in bytes. */
measuredTotalBytes?: number;
};
/** How far the last download was when the job paused or failed. A restart may leave it absent. */
progress?: { completedBytes: number; totalBytes: number };
/** What the runtime reported when the job checked the loaded model. */
loaded?: { context: number; sizeBytes: number; sizeVramBytes: number };
/** Present once the `connect` step wrote: what uninstall must undo. */
connection?: {
/** The id of the endpoint record this feature wrote. */
endpointId: string;
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
flagWasEnabled: boolean;
};
/** ISO time of the last write. */
updatedAt?: string;
}
+184 -21
View File
@@ -1,9 +1,12 @@
/**
* @fileoverview Types of the hardware-aware local model recommendation: what the server sees of
* this Mac, the catalogue of models and harnesses, and the answer of `GET /api/local-model`.
* this Mac, the catalogue of models and harnesses, and the recommendation part of the answer of
* `GET /api/local-model`.
*
* This stage is read-only: Codeman looks at the Mac and recommends. It downloads nothing,
* installs nothing and writes no file.
* Everything here is the READ side: Codeman looks at the Mac and recommends. The types of the
* install (the job states, the progress, the `install` object of the GET, the request bodies,
* the state file, the runtime pin) and `LocalModelStatus`, the whole `data` of the GET, are in
* src/types/local-model-install.ts. Both files go out through the barrel (`../types.js`).
*
* Who produces what:
* - `HardwareFacts`: src/local-model/hardware.ts (one bounded `sysctl` spawn, never throws).
@@ -11,7 +14,11 @@
* - `HarnessFact`: src/local-model/harness-facts.ts (server only, read from the CLI registry at
* call time; it holds a function, so it never goes over HTTP).
* - `Recommendation`, `Candidate`, `DeviceBudget`: src/local-model/recommend.ts (pure).
* - `LocalModelStatus`: src/web/routes/local-model-routes.ts (the `data` of the GET).
*
* Fields marked "Stage 2" were added for the install. They are OPTIONAL in the type only so
* that the stage 1 catalogue and `recommend()` keep compiling until their owner fills them:
* once `recommend()` and the catalogue rows carry them, they become required, and a consumer
* (the chooser window, the install job) may rely on every one of them.
*
* Three rules every producer and consumer follows:
*
@@ -100,6 +107,13 @@ export interface TestedOn {
cli: string;
/** ISO date of the test ('2026-11-06'). */
date: string;
/**
* Stage 2. The generation speed measured in that test, in tokens per second (67.7 for the
* 2026-11-06 run on an Apple M5 Pro with 48 GB). A candidate that matches this entry reports
* this figure as `Candidate.tokensPerSecond` in place of the estimate from the chip's memory
* bandwidth: a measurement on the same Mac beats a formula.
*/
tokensPerSecond?: number;
}
/** One model of the catalogue. */
@@ -112,10 +126,42 @@ export interface LocalModelEntry {
license: string;
/** Parameter count in billions. When no model that fits has 12 or more, the verdict is `limited`, never `ok`. */
paramsB: number;
/** Size of the weights in bytes: both the download and what they take in memory. */
/**
* Size of the weights in bytes: both the download and what they take in memory.
*
* Stage 2: the size of the build the runtime really downloads ON A MAC. That is not always
* the sum of the registry manifest: `qwen3.6:35b-a3b` is an MLX build of 23,105,798,956 bytes
* on Apple silicon, 2% more than the manifest's 22,621,314,161. A row whose Mac size nobody
* measured keeps the manifest sum and says so in `figures`.
*/
bytes: number;
/** Bytes of KV cache one token of context costs. */
/**
* Stage 1 figure: the bytes of KV cache one token of context costs, COMPUTED from the model's
* config. Measured on a Mac it was too low by almost half (20 KiB against about 37 KB held),
* so the need is computed from `heldBytesPerToken` once that is filled. Kept only until
* `recommend()` stops reading it; it is deleted together with `promptCacheTiers` then.
*/
kvBytesPerToken: number;
/**
* Stage 2. Bytes that stay HELD in memory per token of context once the token was processed
* (measured on the MLX runner: 36,000 to 38,000, so about 37,000).
*/
heldBytesPerToken?: number;
/** Stage 2. What the runner holds beside the weights and the context (measured: about 3 GB). */
overheadBytes?: number;
/**
* Stage 2. The TRANSIENT extra the runner takes while it processes a long prompt, on top of
* what stays held (measured: about 5 GB). The fit is judged on the peak:
* `bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`. For
* `qwen3.6:35b-a3b` at 131072 tokens that is about 36 GB, which is tight on a 48 GB Mac.
*/
peakExtraBytes?: number;
/**
* Stage 2. Where `bytes` and the three memory figures above come from: `measured` on a Mac by
* us, or `estimated` (the manifest size, and the per-token, overhead and peak figures of the
* measured row applied to this one).
*/
figures?: 'measured' | 'estimated';
/** Context lengths (tokens) the model may be run at, largest first. */
contexts: number[];
/** Preference among models that fit: the HIGHER rank wins. A judgement, not a measurement. */
@@ -194,6 +240,13 @@ export interface HarnessFact {
installed: boolean;
/** Not installed, but the registry has an install command for this platform that needs no administrator rights. */
installable: boolean;
/**
* Stage 2. The literal install command the install job would run for this CLI: the STOCK
* command of the CLI registry for this platform, never one a `clis.json` override supplies.
* Present exactly when `installable` is true. The consent screen prints it
* (`CandidateHarness.installCommand`).
*/
installCommand?: string;
/** Whether this CLI can be launched with that model id (its command-line pattern accepts it). */
acceptsModelId: (modelId: string) => boolean;
}
@@ -223,18 +276,54 @@ export type SpeedGrade = 'ok' | 'slow' | 'unknown';
*/
export type NoneReason = 'not-macos' | 'intel' | 'memory' | 'disk' | 'macos' | 'no-harness';
/** Where the device budget came from: a formula (`estimated`) or `iogpu.wired_limit_mb` above 0 (`wired-limit`). */
export type BudgetSource = 'estimated' | 'wired-limit';
/**
* Where the device budget came from.
* - `estimated`: a formula over the installed memory. All there is before the runtime ran.
* - `wired-limit`: `iogpu.wired_limit_mb` above 0.
* - `measured` (stage 2): the device total the runtime itself printed when it started
* (`RecommendOptions.measuredTotalBytes`). The chooser says "estimated" next to the memory
* bar for the first value only.
*/
export type BudgetSource = 'estimated' | 'wired-limit' | 'measured';
/** The memory a model may use on this Mac. */
export interface DeviceBudget {
/** L: the wired limit when above 0, else 2/3 of memory below 36 GB and 3/4 from 36 GB. */
/**
* L: the measured device total when there is one, else the wired limit when above 0, else
* 2/3 of memory below 36 GB and 3/4 from 36 GB.
*/
limitBytes: number;
/** B = min(L - 1 GiB, memory - reserve). What `Candidate.budgetShare` is measured against. */
/**
* What `Candidate.budgetShare` is measured against. Estimated or from the wired limit:
* B = min(L - 1 GiB, memory - reserve). Measured: B = min(L, memory - reserve), without the
* 1 GiB, because the runtime's figure is already what it will really use (37.4 GiB on the
* 48 GB test Mac, which reads '40.2').
*/
budgetBytes: number;
source: BudgetSource;
}
/**
* Stage 2. What the caller of `recommend()` knows beyond the hardware probe. Both members are
* optional and an empty object is the stage 1 behaviour.
*/
export interface RecommendOptions {
/**
* The device total the runtime reported in its log line "inference compute"
* (`total="37.4 GiB"`), in bytes: `parseInferenceComputeTotal()` in
* src/local-model/runtime.ts. Present: the budget is `measured`. The install job passes it
* right after the runtime started, before the model is downloaded, and the GET passes the
* stored figure from then on.
*/
measuredTotalBytes?: number;
/**
* What an install adds to every candidate's download beside the model: the size of the
* pinned runtime archive while the runtime is not installed, 0 once it is. It is the
* difference between `Candidate.installBytes` and `Candidate.downloadBytes`. Default 0.
*/
runtimeDownloadBytes?: number;
}
/**
* Every sentence and label the server may put into the payload, in English (the i18n source
* language). The zh-CN dictionary in src/web/public/i18n.js has one entry per value, keyed by
@@ -315,6 +404,60 @@ export interface LocalModelMessage {
vars?: Record<string, string>;
}
/**
* Stage 2. The one tag a model row of the chooser carries, by what it says.
* - `tested`: `Candidate.tested` is true. Replaces every other tag.
* - `best`: the pick. The one tag drawn in the accent colour.
* - `tight`: `Candidate.fit` is `tight`.
* - `slow`: `Candidate.speed` is `slow`.
* - `small`: the model has fewer than 12 billion parameters.
* A row gets the FIRST kind of this list that applies and no tag when none does, so a pick
* that is tight says "Best fit" (the memory bar already shows how full it is).
*/
export type CandidateTagKind = 'tested' | 'best' | 'tight' | 'slow' | 'small';
/**
* The text of each tag, in English (the i18n source language), like `LOCAL_MODEL_LABELS`. A
* constant of its own so that a consumer can list the tag texts apart from the sentences.
*/
export const LOCAL_MODEL_TAG_LABELS = {
tested: 'Tested on a Mac like this',
best: 'Best fit',
tight: 'Tight',
slow: 'Slow',
small: 'Small model',
} as const satisfies Record<CandidateTagKind, string>;
/** One of the texts in `LOCAL_MODEL_TAG_LABELS`. */
export type LocalModelTagLabel = (typeof LOCAL_MODEL_TAG_LABELS)[CandidateTagKind];
/** Stage 2. The tag of a model row: what kind it is (for the styling) and the text to print. */
export interface CandidateTag {
kind: CandidateTagKind;
/** `LOCAL_MODEL_TAG_LABELS[kind]`. */
label: LocalModelTagLabel;
}
/**
* Stage 2. One coding agent a candidate's model can be run with on this Mac, for the "Runs
* with" select of the chooser. Everything a browser needs is here, so it compares no CLI id.
*/
export interface CandidateHarness {
/** The CLI registry id, as data: what `POST /api/local-model/install` is sent as `cli`. */
cli: string;
/** The registry's display label ('OpenCode'). */
cliLabel: string;
/** False: the install job installs it first (it is installable, or it would not be listed). */
installed: boolean;
/**
* The literal command the install job would run to install it (`HarnessFact.installCommand`).
* Present exactly when `installed` is false. The consent screen prints it as it is.
*/
installCommand?: string;
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
}
/** One model at one context with one harness, measured against this Mac. Plain JSON. */
export interface Candidate {
/** `LocalModelEntry.id`. */
@@ -333,7 +476,12 @@ export interface Candidate {
cliInstalled: boolean;
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
/** F = weights + kvBytesPerToken * context + 1 GiB + prompt cache. */
/**
* What the model needs in memory at this context. Stage 1: F = weights + kvBytesPerToken *
* context + 1 GiB + prompt cache. Stage 2: the PEAK, `bytes + heldBytesPerToken * context +
* overheadBytes + peakExtraBytes` of the catalogue row. The memory bar of the chooser is
* filled to `budgetShare`, which is this over the budget.
*/
needBytes: number;
/** `needBytes` as decimal GB text, no unit. */
needGb: string;
@@ -350,10 +498,33 @@ export interface Candidate {
speed: SpeedGrade;
/** `speedSlow` or `speedUnknown`. Absent when `speed` is `ok`. */
speedLabel?: LocalModelLabel;
/** Estimated generation speed, rounded to one decimal. Present exactly when `speed` is not `unknown`. `speed` is decided on the unrounded figure. */
/**
* Generation speed in tokens per second, rounded to one decimal. Present exactly when `speed`
* is not `unknown`; `speed` is decided on the unrounded figure. An estimate from the chip's
* memory bandwidth, except (stage 2) for a candidate that is `tested`, which reports the
* speed measured in that test (`TestedOn.tokensPerSecond`).
*/
tokensPerSecond?: number;
/** A `TestedOn` entry of the model matches this Mac (same chip, same `memClassGb`) and this harness. */
tested: boolean;
/** Stage 2. `LocalModelEntry.figures` of the model: whether `needBytes` rests on a measurement of this model. */
needSource?: 'measured' | 'estimated';
/** Stage 2. The row's one tag, see `CandidateTagKind`. Absent when no kind applies. */
tag?: CandidateTag;
/**
* Stage 2. Every coding agent this model can be run with at `context` on this Mac, best
* first. The first entry is the one the flat fields above name (`cli`, `cliLabel`,
* `cliInstalled`, `slowFirstTurn`). Never empty.
*/
harnesses?: CandidateHarness[];
/**
* Stage 2. The whole download a click on Install starts for this row: `downloadBytes` plus
* `RecommendOptions.runtimeDownloadBytes`. It is the server's own total, and exactly the
* number `POST /api/local-model/install` must be sent as `acknowledgedBytes`.
*/
installBytes?: number;
/** Stage 2. `installBytes` as decimal GB text, no unit. */
installGb?: string;
}
/**
@@ -385,12 +556,4 @@ export interface Recommendation {
budgetSource: BudgetSource;
}
// ---------- API ----------
/** The `data` of `GET /api/local-model`. Read-only; admin only in multi-user mode. */
export interface LocalModelStatus {
hardware: HardwareFacts;
recommendation: Recommendation;
/** `LocalModelCatalog.updated`. */
catalogUpdated: string;
}
// `LocalModelStatus`, the `data` of `GET /api/local-model`, is in src/types/local-model-install.ts.