BitSquare/Codeman
2
0 Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-11-15 02:49:33 +01:00
Code Issues Packages Projects Releases Wiki Activity

feat(local-model): contract for the install stage (types, stubs, fake runtime)

Browse Source 
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-06 20:05:15 +01:00
1 parent 66ca92f7cf
commit 771072c36b
17 files changed
+3770 -23
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/local-model/connect.ts
+124
View File
@@ -0,0 +1,124 @@
/**
* @fileoverview Connecting the installed local model to Codeman, and undoing it: the endpoint
* record, the settings flag, and the checks around them.
*
* CONTRACT STAGE: the two constants are final. Every function below has its final signature
* and a body that throws; the bodies are written next.
*
* These are the ONLY two writes of the local model outside its own folder:
* 1. One record in `custom-model-hosts.json`, through `mutateCustomModelHosts()`:
* `{ id: 'local-ollama', label: 'This Mac', baseUrl: 'http://127.0.0.1:<port>',
* authStyle: 'bearer', models: [<id as /v1/models lists it>], defaultModelId,
* defaultContextLength: <the chosen context> }`, validated by `CustomModelHostSchema`
* before it is written. No `apiKey`. `defaultContextLength` is what reaches the coding
* agent as its context limit, because the runtime's model listing states none.
* 2. `customModelEndpointsEnabled: true` in `settings.json`, through `mutateSettingsFile()`.
* The value it had before is returned and kept in the state file.
*
* Rules:
* - The id is reserved. A record with it that this feature did not write is never
* overwritten: the connect step fails with `endpoint-id-taken`.
* - Both writes are idempotent, so a resumed job writes them again.
* - Undoing removes our record, and switches the flag off again only when this feature
* switched it on AND no other endpoint is saved.
* - The model's digest is compared again before a session starts and at boot
* (`compareModelDigest()`): a model that changed under the same name is not launched.
*/
import type { CustomModelHost } from '../custom-model-hosts.js';
import type { LocalModelErrorCode } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** The id of the endpoint record this feature writes. Reserved: nobody else may use it. */
export const LOCAL_MODEL_ENDPOINT_ID = 'local-ollama';
/** The label of that record, what the Run menu shows in brackets. */
export const LOCAL_MODEL_ENDPOINT_LABEL = 'This Mac';
/** What the endpoint record is built from. */
export interface LocalEndpointInput {
/** The runtime's loopback port. */
port: number;
/** The model id exactly as the runtime's `/v1/models` lists it. */
modelId: string;
/** The chosen context length in tokens. */
context: number;
}
/** The endpoint record of the file header. Pure; the caller validates it with `CustomModelHostSchema`. */
export function buildLocalEndpointRecord(input: LocalEndpointInput): CustomModelHost {
return notImplemented('buildLocalEndpointRecord', input);
}
export interface ConnectLocalModelInput extends LocalEndpointInput {
/**
* True when the state file already records a connection of ours: an existing record with
* our id is then ours and is replaced. False: such a record is somebody else's.
*/
ours: boolean;
/**
* The flag's value before this feature first switched it on, when the state file already
* holds one. It is handed back unchanged, so a second connect cannot overwrite it with `true`.
*/
flagWasEnabled?: boolean;
}
/** How the connect step ended. */
export type ConnectResult =
| {
ok: true;
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
flagWasEnabled: boolean;
}
| { ok: false; code: Extract<LocalModelErrorCode, 'endpoint-id-taken' | 'connect-failed'>; detail?: string };
/** Writes the record and the flag. Idempotent. Never throws: a refused write is `connect-failed` with its message as `detail`. */
export async function connectLocalModel(input: ConnectLocalModelInput): Promise<ConnectResult> {
return notImplemented('connectLocalModel', input);
}
/** Moves our record to another port (the answer to "Use another port" once connected). A missing record is not an error. */
export async function moveLocalEndpoint(port: number): Promise<void> {
return notImplemented('moveLocalEndpoint', port);
}
/**
* Removes our record, and restores the settings flag when this feature switched it on
* (`flagWasEnabled` is false) and no other endpoint is left. Idempotent.
*/
export async function disconnectLocalModel(input: { flagWasEnabled: boolean }): Promise<void> {
return notImplemented('disconnectLocalModel', input);
}
/** Whether `id` is the reserved endpoint id. For the endpoint routes and the discovery sweep, which leave that record alone. */
export function isLocalModelEndpointId(id: string): boolean {
return notImplemented('isLocalModelEndpointId', id);
}
/**
* The names of the live sessions that use the endpoint `endpointId`, in the order given.
* Pure. What `DELETE /api/local-model` answers its 409 with.
*/
export function sessionNamesUsingEndpoint(
sessions: Iterable<{ name: string; customModel?: { endpointId: string } }>,
endpointId: string = LOCAL_MODEL_ENDPOINT_ID
): string[] {
return notImplemented('sessionNamesUsingEndpoint', sessions, endpointId);
}
/**
* Compares the digest stored at install time with the one the runtime reports now. Pure.
* - `same`: launch.
* - `changed`: refuse (`model-changed`): the name now stands for other weights or another template.
* - `unknown`: one of the two is absent (the runtime does not answer, the model is gone).
* The caller refuses as well, with the reason it knows.
*/
export function compareModelDigest(
stored: string | undefined,
current: string | undefined
): 'same' | 'changed' | 'unknown' {
return notImplemented('compareModelDigest', stored, current);
}
src/local-model/developer-tools.ts
+42
View File
@@ -0,0 +1,42 @@
/**
* @fileoverview Whether this Mac has the developer tools a coding agent calls (git, Python).
*
* CONTRACT STAGE: the function below has its final signature and a body that throws; the body
* is written next.
*
* Why it exists: on a Mac without Xcode or the Command Line Tools, `git` and `python3` are
* stubs that open macOS's "install the command line developer tools" window. A coding agent
* runs both, so the install window says one honest sentence when the tools are missing
* (`LOCAL_MODEL_INSTALL_LABELS.developerToolsMissing`).
*
* Rules:
* - The ONLY command ever run is `/usr/bin/xcode-select -p`, bounded by a timeout. Never
* `git`, `python3`, `xcrun` or anything else that would open that window itself.
* - macOS only. Anywhere else, and when the command does not answer in time, the answer is
* `unknown`, and nothing is said.
* - Never throws. The answer is cached for a short while, so a window that polls once a
* second does not spawn once a second.
*/
import type { LocalModelDeveloperTools } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** How long `/usr/bin/xcode-select -p` may take. */
export const DEVELOPER_TOOLS_TIMEOUT_MS = 2_000;
/** What the check calls. Each member defaults to the real thing; a test injects its own. */
export interface DeveloperToolsDeps {
/** Default `process.platform`. */
platform: string;
/** Runs `file` with `args` and resolves with its exit code (`null` when it timed out or could not be started). */
run: (file: string, args: readonly string[], timeoutMs: number) => Promise<number | null>;
}
/** `present` when `/usr/bin/xcode-select -p` exits 0, `missing` when it exits with anything else, `unknown` otherwise. */
export async function checkDeveloperTools(deps: Partial<DeveloperToolsDeps> = {}): Promise<LocalModelDeveloperTools> {
return notImplemented('checkDeveloperTools', deps);
}
src/local-model/install-job.ts
+497
View File
@@ -0,0 +1,497 @@
/**
* @fileoverview The local model install job: its state table, the pure reducer over it, the
* pure view of it, and the service that drives it.
*
* CONTRACT STAGE: the three tables (`INSTALL_STATE_TABLE`, `INSTALL_STEP_TABLE`,
* `INSTALL_TRANSITIONS`) are final and are what the reducer's table-driven test iterates.
* Every function below has its final signature and a body that throws; the bodies are written
* next.
*
* ONE job at a time. The job is five steps in a fixed order (`LOCAL_MODEL_INSTALL_STEPS`:
* runtime, model, verify, agent, connect; `agent` is left out when the coding agent is
* installed). Its state is one of five (`LocalModelInstallState`), stored in the state file.
*
* The state table. "On disk" names what lies under the local model folder; the two writes
* outside it belong to the `connect` step.
*
* | State | On disk | A server restart | Cancel | Allowed next actions |
* |--------------|-------------------------------------------|-----------------------------|------------------------|---------------------------------|
* | `idle` | nothing, or leftovers a Remove could not | stays `idle` | refused (409) | install |
* | | delete (never read back) | | | |
* | `installing` | what the finished steps wrote, plus what | becomes `paused` at the | becomes `paused` at | cancel |
* | | the current step has written so far | same step; nothing resumes | the same step | |
* | | (see the step table) | without a click | | |
* | `paused` | as `installing` at that step, nothing in | stays `paused` | refused (409) | resume, install (another |
* | | flight | | | choice), remove |
* | `failed` | as `installing` at that step, nothing in | stays `failed`, same error | refused (409) | resume (try again), install |
* | | flight | | | (another choice), remove |
* | `installed` | the runtime, the private home, the model | stays `installed`; the | refused (409) | start a session, start or stop |
* | | | runtime is started again | | the runtime, remove |
* | | | when it was not stopped | | |
*
* The step table. Every step is IDEMPOTENT, so Resume and "Try again" re-enter the step the
* job stopped in; they first make sure the runtime runs when the step needs it.
*
* | Step | Adds to disk | Cancel | Resume |
* |-----------|---------------------------------------|------------------------------------------|------------------------------------------------|
* | `runtime` | the archive's partial file, then the | aborts the download, the partial file | continues the partial file with a Range |
* | | unpacked runtime, then the private | stays | request, or skips what is already unpacked |
* | | home once the runtime started | | |
* | `model` | the model store: partial blobs, then | aborts the pull request, which stops the | pulls again; the runtime continues its partial |
* | | the model | download; the partial blobs stay | blobs |
* | `verify` | nothing (the model is loaded into | aborts the request in flight | runs the three checks again from the first |
* | | memory) | | |
* | `agent` | whatever the coding agent's own | kills the installer's process group | looks for the binary first and runs the |
* | | installer writes, outside our folder; | | installer only when it is still missing |
* | | never removed by us | | |
* | `connect` | our endpoint record and the settings | honoured before the first write; once | writes both again |
* | | flag: the two writes outside our | the writes began they finish and the job | |
* | | folder | ends `installed` | |
*
* How the memory budget gets measured before a model is downloaded (ONE consent, bound to the
* size shown): the person confirms `Candidate.installBytes` once. The `runtime` step then
* gets and starts the runtime, reads the device total from its log and judges the choice
* again with it (`modelFitsBudget()`; `recommend()` with the same options names the next
* model). When the chosen model at the chosen context no longer fits, the job fails right
* there with `does-not-fit` and the next model that fits, BEFORE the model download starts. When it fits, the job goes on without a second question: the sizes that were
* confirmed did not change. From then on the GET reports `budgetSource: 'measured'`.
*
* Rules:
* - `reduceInstallJob()` is PURE and total: an event the transition table does not list for
* the current state is refused by returning the SAME object, which the service answers
* with 409 `wrong-state`.
* - The reducer moves the state, the step, the error and the progress. The FACTS a step
* learns (the port and pid, the measured total, the pull's total, the digest, what the
* runtime loaded, the previous settings flag) are written by the service through
* `mutateLocalModelState()` and never change the state.
* - A job read as `installing` at boot was interrupted by the restart: the service applies
* the event `restart` before anything else, so it is `paused` and waits for a click.
* - A new install from `paused` or `failed` REPLACES the job. When it names another model,
* what the old job left of its model is deleted from the store first.
* - Install is checked before anything is downloaded: admin only in multi-user mode; the
* model, the context and the coding agent must be a row the current recommendation offers;
* the coding agent must be a harness row of the catalogue; `acknowledgedBytes` must be the
* server's own total.
* - No CLI id is compared with a literal here: the coding agent is a catalogue row plus a
* `HarnessFact`.
*/
import type {
HardwareFacts,
HarnessFact,
LocalModelAction,
LocalModelCatalog,
LocalModelConflict,
LocalModelDeveloperTools,
LocalModelErrorCode,
LocalModelInstallRequest,
LocalModelInstallState,
LocalModelInstallStep,
LocalModelInstallView,
LocalModelRemoveRequest,
LocalModelResumeRequest,
LocalModelRuntimePin,
LocalModelRuntimeRequest,
LocalModelStateFile,
Recommendation,
RecommendOptions,
} from '../types.js';
import type { CliInstallResult, RunInstallCommandOptions } from '../cli-install-runner.js';
import type { ConnectLocalModelInput, ConnectResult } from './connect.js';
import type { OllamaTarget, PullOptions, PullResult, VerifyResult } from './ollama-client.js';
import type { LocalModelPaths } from './paths.js';
import type {
ChooseRuntimePortOptions,
RemoveLocalModelFilesOptions,
RuntimeInstallOptions,
RuntimeInstallResult,
RuntimeSupervisor,
} from './runtime.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
// ---------- The state table ----------
/** One row of the state table. */
export interface InstallStateRow {
/** What lies on disk in this state, in words. */
onDisk: string;
/** The state a server restart turns this one into. */
afterRestart: LocalModelInstallState;
/** What the restart does, in words. */
restart: string;
/** The state Cancel turns this one into, or `null` when Cancel is refused (409 `wrong-state`). */
afterCancel: LocalModelInstallState | null;
/** What Cancel does, in words. */
cancel: string;
/**
* Every action a client may take in this state. The view's `actions` is this list narrowed
* by what is live (`allowedActions()`): `start` only while the runtime is down, `stop` and
* `start-session` only while it runs, `start-session` only when nothing blocks a session.
*/
actions: readonly LocalModelAction[];
}
/** The state table of the file header, as data. */
export const INSTALL_STATE_TABLE: Readonly<Record<LocalModelInstallState, InstallStateRow>> = {
idle: {
onDisk:
'Nothing of ours. The local model folder is absent, or holds leftovers an earlier Remove could not delete; they are never read back.',
afterRestart: 'idle',
restart: 'Nothing happens.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['install'],
},
installing: {
onDisk:
'What the finished steps wrote, plus what the current step has written so far (INSTALL_STEP_TABLE). The state file names the model, the coding agent, the context and the step.',
afterRestart: 'paused',
restart:
'The job comes back paused at the same step. Nothing is downloaded, started or installed until the person presses Resume.',
afterCancel: 'paused',
cancel: 'The step in flight is interrupted the way its step row says, and the job is paused at that step.',
actions: ['cancel'],
},
paused: {
onDisk: 'As installing at that step, with nothing in flight.',
afterRestart: 'paused',
restart: 'Stays paused.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['resume', 'install', 'remove'],
},
failed: {
onDisk: 'As installing at that step, with nothing in flight.',
afterRestart: 'failed',
restart: 'Stays failed, with the same error.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['resume', 'install', 'remove'],
},
installed: {
onDisk:
'The runtime, its private home and the model. The state file holds the port, the pid, the digest, what the runtime loaded and the previous settings flag. Outside the folder: our endpoint record and the settings flag.',
afterRestart: 'installed',
restart:
'Stays installed. The runtime is started again unless the person had stopped it, and the digest is read again: a changed digest or a port taken by another program shows as an error on the view and blocks a session, the state stays installed.',
afterCancel: null,
cancel: 'Refused: no job is running.',
actions: ['start-session', 'start', 'stop', 'remove'],
},
};
// ---------- The step table ----------
/** One row of the step table. */
export interface InstallStepRow {
/** What the step does, in words. */
does: string;
/** What it adds to disk (possibly partly, when it is interrupted). */
onDisk: string;
/** How Cancel interrupts it. */
cancel: string;
/** What Resume, and "Try again" after a failure, does: every step is idempotent and is entered again. */
resume: string;
/** Whether the step reports byte progress (`LocalModelProgress`). */
progress: boolean;
/** Whether the runtime must run before the step is entered (the service starts it first, idempotently). */
needsRuntime: boolean;
/** Every error code the step can fail with. `internal` and `disk-full` can come from any step that writes. */
errors: readonly LocalModelErrorCode[];
}
/** The step table of the file header, as data. */
export const INSTALL_STEP_TABLE: Readonly<Record<LocalModelInstallStep, InstallStepRow>> = {
runtime: {
does: 'Downloads the pinned runtime archive unless it is unpacked already, compares its sha256, unpacks it, chooses the port once, starts the runtime, reads the measured memory budget from its log and judges the chosen model against it.',
onDisk:
'runtime/ollama-<version>.tgz.partial while it downloads, runtime/ollama-<version>/ once unpacked, home/ once the runtime started.',
cancel: 'Aborts the download. The partial file stays. A runtime that already runs keeps running.',
resume:
'Continues the partial file with a Range request, or skips the download when the folder is unpacked. Starting the runtime again is a no-op when it runs.',
progress: true,
needsRuntime: false,
errors: [
'runtime-download-failed',
'runtime-damaged',
'runtime-unpack-failed',
'runtime-start-failed',
'port-in-use',
'does-not-fit',
'disk-full',
'internal',
],
},
model: {
does: 'Pulls the model through the runtime, reading the progress stream line by line.',
onDisk: 'models/ with partial blobs, then the finished model.',
cancel: 'Aborts the pull request, which stops the download. The partial blobs stay.',
resume: 'Pulls again. The runtime continues its partial blobs.',
progress: true,
needsRuntime: true,
errors: ['pull-failed', 'model-not-found', 'runtime-start-failed', 'port-in-use', 'disk-full', 'internal'],
},
verify: {
does: 'Loads the model with one tiny chat, checks that the runtime put all of it on the GPU at the chosen context, runs one tool probe, and stores the digest the runtime reports.',
onDisk: 'Nothing new. The model is loaded into memory.',
cancel: 'Aborts the request in flight.',
resume: 'Runs the three checks again from the first.',
progress: false,
needsRuntime: true,
errors: [
'load-failed',
'does-not-fit',
'context-not-applied',
'no-tool-call',
'probe-unfinished',
'runtime-start-failed',
'port-in-use',
'internal',
],
},
agent: {
does: 'Installs the coding agent with its stock install command through the shared CLI install runner, writes the admin audit entry, forgets the cached lookups of its binary and looks for it again.',
onDisk:
'Whatever the coding agent installer writes, outside the local model folder (its own folder in the home directory). Never removed by us.',
cancel: 'Kills the whole process group of the installer.',
resume: 'Looks for the binary first and runs the installer only when it is still missing.',
progress: false,
needsRuntime: false,
errors: ['harness-failed', 'internal'],
},
connect: {
does: 'Writes the endpoint record and switches the custom model endpoints setting on, keeping its previous value in the state file.',
onDisk:
'Our record in custom-model-hosts.json and the flag in settings.json: the two writes outside the local model folder.',
cancel:
'Honoured before the first write. Once the writes began they finish and the job ends installed, so the pair is never left half done.',
resume: 'Writes both again. Both are idempotent.',
progress: false,
needsRuntime: true,
errors: ['endpoint-id-taken', 'connect-failed', 'internal'],
},
};
// ---------- Events and transitions ----------
/** What moves the job. */
export type InstallJobEvent =
/** A new job: `POST /api/local-model/install` passed its checks. From `idle`, `paused` or `failed`. */
| {
type: 'install';
model: { id: string; label: string; context: number };
cli: { id: string; label: string };
/** The steps of this job in order (`jobSteps()`). */
steps: LocalModelInstallStep[];
}
/** A download moved. Changes `progress` only. */
| { type: 'progress'; completedBytes: number; totalBytes: number }
/** The current step finished: the next step of `steps`, or `installed` after the last. Clears `progress`. */
| { type: 'step-done' }
/** The current step failed. */
| {
type: 'fail';
code: LocalModelErrorCode;
vars?: Record<string, string>;
detail?: string;
nextModelId?: string;
}
/** `POST /api/local-model/cancel`. */
| { type: 'cancel' }
/** The server started and found this state in the file. */
| { type: 'restart' }
/** `POST /api/local-model/resume`: the same step again. Clears `error`. */
| { type: 'resume' }
/** `DELETE /api/local-model` finished: back to `idle`, every other member dropped. */
| { type: 'remove' };
export type InstallJobEventType = InstallJobEvent['type'];
/**
* Which state an event leads to, by current state. A pair that is not listed is REFUSED: the
* reducer returns the object it was given. One entry needs a word: `step-done` in
* `installing` leads to `installing` at the next step, and to `installed` when the step that
* finished was the last of the job's `steps`.
*/
export const INSTALL_TRANSITIONS: Readonly<
Record<LocalModelInstallState, Readonly<Partial<Record<InstallJobEventType, LocalModelInstallState>>>>
> = {
idle: { install: 'installing', restart: 'idle', remove: 'idle' },
installing: {
progress: 'installing',
'step-done': 'installing',
fail: 'failed',
cancel: 'paused',
restart: 'paused',
},
paused: { resume: 'installing', install: 'installing', remove: 'idle', restart: 'paused' },
failed: { resume: 'installing', install: 'installing', remove: 'idle', restart: 'failed' },
installed: { remove: 'idle', restart: 'installed' },
};
/** The steps a job runs, in order: all of `LOCAL_MODEL_INSTALL_STEPS`, without `agent` when the coding agent is installed. Pure. */
export function jobSteps(input: { agentInstalled: boolean }): LocalModelInstallStep[] {
return notImplemented('jobSteps', input);
}
/**
* The state file after `event`. Pure: no IO, no clock (`updatedAt` is the state store's
* business). Refuses an event the transition table does not list for the current state by
* returning `current` itself. Keeps `runtime` and `connection` across a new install, because
* what they record is still on disk; `remove` drops everything.
*/
export function reduceInstallJob(current: LocalModelStateFile, event: InstallJobEvent): LocalModelStateFile {
return notImplemented('reduceInstallJob', current, event);
}
// ---------- The view ----------
/** What is live and not in the state file, for the view. Gathered by the service on every GET. */
export interface InstallLiveFacts {
/** The runtime child is alive (`RuntimeStatus.running`). */
running: boolean;
/** The supervisor stopped restarting the runtime (`RuntimeStatus.gaveUp`): the error `runtime-stopped` in state `installed`. */
gaveUp: boolean;
/** Another program holds the stored port: the error `port-in-use` in state `installed`. */
portInUse: boolean;
/** The stored digest against the runtime's current one (`compareModelDigest()`). `changed` is the error `model-changed`. Absent while not installed. */
digest?: 'same' | 'changed' | 'unknown';
developerTools: LocalModelDeveloperTools;
/** The pinned runtime is unpacked on disk. */
runtimeInstalled: boolean;
/** The current download speed. Not stored, so absent after a restart. */
bytesPerSecond?: number;
/** For an error whose next step is `next-model`: the model to offer, from the current recommendation. */
nextModel?: { modelId: string; modelLabel: string };
pin: LocalModelRuntimePin;
/** `LocalModelPaths.root`, for the "where it goes" line. */
folder: string;
}
/** The actions the server accepts right now: the state row's list narrowed by what is live. Pure. */
export function allowedActions(state: LocalModelStateFile, live: InstallLiveFacts): LocalModelAction[] {
return notImplemented('allowedActions', state, live);
}
/** The `install` object of the GET. Pure: every size text comes from `formatGb()`, every sentence from `LOCAL_MODEL_INSTALL_LABELS`. */
export function buildInstallView(state: LocalModelStateFile, live: InstallLiveFacts): LocalModelInstallView {
return notImplemented('buildInstallView', state, live);
}
// ---------- The service ----------
/** Who asked, for the admin audit entry of the coding agent install. */
export interface LocalModelActor {
username: string;
ip?: string;
}
/** How a verb ended: the view after it, a 400 with its message, or a 409 with its conflict. */
export type LocalModelVerbResult =
| { ok: true; view: LocalModelInstallView }
| { ok: false; status: 400; message: string }
| { ok: false; status: 409; conflict: LocalModelConflict };
/**
* Everything the service calls, so a test spawns, fetches and downloads nothing. Each member
* defaults to the real module. The members named like a function of another module have
* exactly that function's signature.
*/
export interface LocalModelServiceDeps {
readHardware: () => Promise<HardwareFacts>;
getCatalog: () => LocalModelCatalog;
buildHarnessFacts: (catalog: LocalModelCatalog) => Promise<HarnessFact[]>;
recommend: (
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[],
options?: RecommendOptions
) => Recommendation;
/** `modelFitsBudget()` of src/local-model/recommend.ts: the judgement of the `runtime` step, with the measured budget. */
modelFitsBudget: (
hw: HardwareFacts,
catalog: LocalModelCatalog,
modelId: string,
context: number,
options?: RecommendOptions
) => boolean;
// src/local-model/runtime.ts
isRuntimeInstalled: (pin?: LocalModelRuntimePin, paths?: LocalModelPaths) => Promise<boolean>;
ensureRuntimeInstalled: (options?: RuntimeInstallOptions) => Promise<RuntimeInstallResult>;
chooseRuntimePort: (options?: ChooseRuntimePortOptions) => Promise<number | undefined>;
removeLocalModelFiles: (options: RemoveLocalModelFilesOptions) => Promise<void>;
supervisor: RuntimeSupervisor;
// src/local-model/ollama-client.ts
pullModel: (target: OllamaTarget, modelId: string, options?: PullOptions) => Promise<PullResult>;
verifyModel: (target: OllamaTarget, modelId: string, context: number, signal?: AbortSignal) => Promise<VerifyResult>;
readModelDigest: (target: OllamaTarget, modelId: string, signal?: AbortSignal) => Promise<string | undefined>;
listOpenAiModels: (target: OllamaTarget, signal?: AbortSignal) => Promise<string[]>;
deleteModel: (target: OllamaTarget, modelId: string, signal?: AbortSignal) => Promise<boolean>;
// src/local-model/developer-tools.ts
checkDeveloperTools: () => Promise<LocalModelDeveloperTools>;
// src/cli-install-runner.ts
stockInstallCommand: (id: string) => string | undefined;
tryBeginCliInstall: (id: string) => (() => void) | null;
runInstallCommand: (command: string, options?: RunInstallCommandOptions) => Promise<CliInstallResult>;
/** Forgets the cached lookups of the CLI's binaries, then says whether its binary is found now. */
recheckCliInstalled: (cli: string) => Promise<boolean>;
/** Writes one admin audit line (`appendAdminAudit()`). */
audit: (entry: { admin: string; action: string; target: string; ip?: string; detail?: unknown }) => Promise<void>;
// src/local-model/connect.ts
connectLocalModel: (input: ConnectLocalModelInput) => Promise<ConnectResult>;
moveLocalEndpoint: (port: number) => Promise<void>;
disconnectLocalModel: (input: { flagWasEnabled: boolean }) => Promise<void>;
// src/local-model/state-store.ts
readState: () => Promise<LocalModelStateFile>;
mutateState: (
fn: (current: LocalModelStateFile) => LocalModelStateFile | undefined
) => Promise<{ before: LocalModelStateFile; after: LocalModelStateFile | undefined }>;
pin: LocalModelRuntimePin;
paths: LocalModelPaths;
}
/** The install as the routes and the server see it. One per process. */
export interface LocalModelService {
/** The `install` object of the GET. Never throws. */
view(): Promise<LocalModelInstallView>;
/** What the GET passes to `recommend()`: the stored measured total and the runtime download that is still needed. */
recommendOptions(): Promise<RecommendOptions>;
/** `POST /api/local-model/install`. Answers once the job is STARTED; the job runs on in the server. */
install(request: LocalModelInstallRequest, actor: LocalModelActor): Promise<LocalModelVerbResult>;
/** `POST /api/local-model/cancel`. Answers once the step in flight was interrupted. */
cancel(): Promise<LocalModelVerbResult>;
/** `POST /api/local-model/resume`. */
resume(request: LocalModelResumeRequest, actor: LocalModelActor): Promise<LocalModelVerbResult>;
/** `POST /api/local-model/runtime`. */
runtime(request: LocalModelRuntimeRequest): Promise<LocalModelVerbResult>;
/**
* `DELETE /api/local-model`. `liveSessionNames` is what `sessionNamesUsingEndpoint()` found:
* not empty answers 409 `sessions-open`. Else: stop the runtime, remove our endpoint record,
* restore the settings flag when this feature switched it on and no other endpoint is left,
* delete the runtime and the private home and (when asked) the models. The coding agent stays.
*/
remove(request: LocalModelRemoveRequest, liveSessionNames: readonly string[]): Promise<LocalModelVerbResult>;
/**
* Called once when the server starts. An interrupted job becomes `paused`. An installed
* runtime that was not stopped is started (an own orphan on the port is adopted or
* replaced, another listener is `port-in-use`) and the digest is read again. Never throws,
* and never keeps the server from starting.
*/
boot(): Promise<void>;
/** Called when the server stops, beside `stopDeepSeekWeb()`: aborts a job in flight (it reads `paused` at the next boot) and stops the runtime. */
shutdown(): Promise<void>;
}
/** A service over `deps`. A test builds its own with fakes. */
export function createLocalModelService(deps: Partial<LocalModelServiceDeps> = {}): LocalModelService {
return notImplemented('createLocalModelService', deps);
}
/** The one service of this process, built on first use with the real modules. What the routes and server.ts call. */
export function getLocalModelService(): LocalModelService {
return notImplemented('getLocalModelService');
}
src/local-model/ollama-client.ts
+275
View File
@@ -0,0 +1,275 @@
/**
* @fileoverview The HTTP client of the model runtime: what Codeman asks its own runtime on
* the loopback port, and the pure functions that read the answers.
*
* CONTRACT STAGE: every function below has its final signature and a body that throws; the
* bodies are written next.
*
* What it asks:
* - `GET /api/version`, `GET /api/tags`, `GET /api/ps`, `GET /v1/models`, `DELETE /api/delete`.
* - `POST /api/pull` with `stream: true`. The answer is one JSON object per line, and a
* 23 GB pull sends about 1.4 million of them, so the stream is parsed INCREMENTALLY
* (`createPullParser()`, pure): only the unfinished last line and the running totals are
* kept, never the body. Aborting the request stops the download; pulling again continues
* it. An error line or a stream that ends without `success` is retried, at most
* `PULL_RETRIES` times.
* - The check after the pull (`verifyModel()`), in this order: one tiny chat with
* `reasoning_effort: "none"` (it loads the model), then `GET /api/ps` must show the whole
* model on the GPU (`size_vram` equal to `size`) at the chosen context
* (`judgeLoaded()`, pure), then one tool probe on `/v1/chat/completions` with a `tools`
* array, `reasoning_effort: "none"` and `max_tokens` of `TOOL_PROBE_MAX_TOKENS`, which
* passes on a tool call whose arguments parse as JSON (`judgeToolProbe()`, pure).
*
* Rules:
* - Nothing here throws for something a runtime can answer: every outcome is a result with
* the job's error code. An abort signal is the result `cancelled`.
* - The runtime does not enforce the context length (a prompt of 167,012 tokens was accepted
* at 131072), so nothing here relies on it to.
* - Every function takes its `fetch` from the target, so a test passes the fake runtime's
* address and spawns nothing.
*/
import type { LocalModelErrorCode } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** How often a pull that ended on an error line or a cut stream is started again. */
export const PULL_RETRIES = 5;
/** `max_tokens` of the tool probe. */
export const TOOL_PROBE_MAX_TOKENS = 1024;
/** Which runtime to ask. */
export interface OllamaTarget {
/** `http://127.0.0.1:<port>`, without a trailing slash. */
baseUrl: string;
/** Default: the global `fetch`. */
fetch?: typeof fetch;
}
// ---------- Listings ----------
/** One model of `GET /api/tags`: what is on disk. */
export interface OllamaTagModel {
/** `name:tag`. */
name: string;
/** Size on disk in bytes. */
size: number;
/** The model's digest, lowercase hex without a prefix. What is stored and compared again before a session starts. */
digest: string;
/** `details.format` ('safetensors' for an MLX build, 'gguf'). */
format?: string;
/** `details.runner` ('mlx', 'ggml'). */
runner?: string;
/** `capabilities` ('completion', 'tools', 'thinking', 'vision'). */
capabilities?: string[];
}
/** One model of `GET /api/ps`: what is loaded. */
export interface OllamaPsModel {
name: string;
/** Memory the loaded model takes, in bytes. */
size: number;
/** The part of `size` that is on the GPU. Equal to `size` when all of it is. */
sizeVram: number;
/** The context length the model was loaded with. */
contextLength: number;
digest?: string;
}
/** The runtime's version ('0.40.15'), or `undefined` when nothing answers there or the answer is not a runtime's. */
export async function getRuntimeVersion(target: OllamaTarget, signal?: AbortSignal): Promise<string | undefined> {
return notImplemented('getRuntimeVersion', target, signal);
}
/** The models on disk. An empty list when the runtime has none or does not answer. */
export async function listTags(target: OllamaTarget, signal?: AbortSignal): Promise<OllamaTagModel[]> {
return notImplemented('listTags', target, signal);
}
/** The models loaded right now. An empty list when none is or the runtime does not answer. */
export async function listRunning(target: OllamaTarget, signal?: AbortSignal): Promise<OllamaPsModel[]> {
return notImplemented('listRunning', target, signal);
}
/** The model ids `GET /v1/models` lists, exactly as listed (the runtime answers `data: null` for none). */
export async function listOpenAiModels(target: OllamaTarget, signal?: AbortSignal): Promise<string[]> {
return notImplemented('listOpenAiModels', target, signal);
}
/** The digest `GET /api/tags` reports for `modelId`, or `undefined` when the model is not on disk. */
export async function readModelDigest(
target: OllamaTarget,
modelId: string,
signal?: AbortSignal
): Promise<string | undefined> {
return notImplemented('readModelDigest', target, modelId, signal);
}
/** Deletes a model from the runtime's store. True when it is gone afterwards (also when it never was there). */
export async function deleteModel(target: OllamaTarget, modelId: string, signal?: AbortSignal): Promise<boolean> {
return notImplemented('deleteModel', target, modelId, signal);
}
// ---------- Pull ----------
/** What the pull's lines add up to so far. */
export interface PullSnapshot {
/** Sum of the last `completed` of every layer seen. */
completedBytes: number;
/** Sum of the `total` of every layer seen. Grows while new layers appear; 0 before the first. */
totalBytes: number;
/** The `status` text of the last line that had one ('pulling manifest', 'writing manifest'). */
status?: string;
/** A line `{"status":"success"}` was seen. */
succeeded: boolean;
/** The text of the first `{"error":"..."}` line. */
error?: string;
/** How many complete lines were parsed. */
lines: number;
}
/**
* Reads a pull stream piece by piece. It keeps the unfinished last line and the totals per
* layer, never the lines themselves, so its memory does not grow with the stream.
*/
export interface PullParser {
/** Feed the next piece of the body. A line may be split anywhere across pieces, a multi-byte character too. */
push(chunk: string | Uint8Array): void;
/** The body ended: reads a last line that had no line break. */
end(): void;
snapshot(): PullSnapshot;
}
/**
* A parser for one pull stream. Pure: no IO, no clock. Lines it understands (both real):
*
* - an MLX build: `{"status":"pulling model","digest":"sha256:model","total":23105798956,"completed":N}`
* - a GGUF build, one layer after the other:
* `{"status":"pulling c5396e06af29","digest":"sha256:c539...","total":397807936,"completed":N}`
* (`completed` is absent until the layer's first bytes arrived)
* - `{"status":"pulling manifest"}`, `{"status":"verifying sha256 digest"}`,
* `{"status":"writing manifest"}`, `{"status":"success"}`
* - `{"error":"max retries exceeded: local error: tls: bad record MAC"}`
*
* A line that is not JSON is skipped and still counted.
*/
export function createPullParser(): PullParser {
return notImplemented('createPullParser');
}
/** How far a pull is, for the caller's progress display. */
export interface PullProgress {
completedBytes: number;
totalBytes: number;
/** Over the last few seconds. Absent until it can be told. */
bytesPerSecond?: number;
}
export interface PullOptions {
/** Aborts the request, which stops the download. The result is `cancelled`. */
signal?: AbortSignal;
/** Called at most a few times a second, and once more when the pull ends. */
onProgress?: (progress: PullProgress) => void;
/** Default `PULL_RETRIES`. */
retries?: number;
/** Waits between retries. A test injects its own. */
sleep?: (ms: number) => Promise<void>;
/** The clock of the speed figure, in milliseconds. A test injects its own. */
now?: () => number;
}
/** How a pull ended. */
export type PullResult =
| {
ok: true;
/** The pull's own total: what every size shown for the model comes from once it is known. */
totalBytes: number;
}
| {
ok: false;
code: Extract<LocalModelErrorCode, 'pull-failed' | 'model-not-found' | 'disk-full'>;
/** The runtime's last error text. */
detail?: string;
}
| { ok: false; code: 'cancelled' };
/** Pulls `modelId` and waits for the end. Resuming is calling it again. Never throws. */
export async function pullModel(target: OllamaTarget, modelId: string, options: PullOptions = {}): Promise<PullResult> {
return notImplemented('pullModel', target, modelId, options);
}
// ---------- The check after the pull ----------
/** The codes the check can end on. */
export type VerifyFailureCode = Extract<
LocalModelErrorCode,
'load-failed' | 'does-not-fit' | 'context-not-applied' | 'no-tool-call' | 'probe-unfinished'
>;
/** What the runtime reported about the loaded model. */
export interface LoadedModel {
context: number;
sizeBytes: number;
sizeVramBytes: number;
}
/** The body of the tiny chat that loads the model: one short user message, `reasoning_effort: "none"`, a small `max_tokens`. Pure. */
export function buildChatProbeRequest(modelId: string): Record<string, unknown> {
return notImplemented('buildChatProbeRequest', modelId);
}
/** The body of the tool probe: a `tools` array, `reasoning_effort: "none"`, `max_tokens: TOOL_PROBE_MAX_TOKENS`. Pure. */
export function buildToolProbeRequest(modelId: string): Record<string, unknown> {
return notImplemented('buildToolProbeRequest', modelId);
}
/**
* Judges the process list after the chat. Pure.
* - the model is not listed: `load-failed`;
* - `sizeVram` is not `size`: `does-not-fit` (part of the model is not on the GPU);
* - `contextLength` is not `context`: `context-not-applied`.
*/
export function judgeLoaded(
models: readonly OllamaPsModel[],
modelId: string,
context: number
): { ok: true; loaded: LoadedModel } | { ok: false; code: VerifyFailureCode; detail?: string } {
return notImplemented('judgeLoaded', models, modelId, context);
}
/**
* Judges the answer of the tool probe. Pure. `status` is the HTTP status, `body` the parsed
* JSON (or `undefined` when it did not parse).
* - a tool call whose `arguments` parse as a JSON object: passed;
* - `finish_reason` is `length`: `probe-unfinished` (the token limit ended it, so it proves nothing);
* - any other answer of status 200: `no-tool-call`;
* - another status: `load-failed`.
*/
export function judgeToolProbe(
status: number,
body: unknown
): { ok: true } | { ok: false; code: VerifyFailureCode; detail?: string } {
return notImplemented('judgeToolProbe', status, body);
}
/** How the check ended. */
export type VerifyResult =
| { ok: true; loaded: LoadedModel }
| { ok: false; code: VerifyFailureCode; detail?: string }
| { ok: false; code: 'cancelled' };
/**
* The whole check, in order: the chat, the process list, the tool probe. Stops at the first
* failure. Never throws.
*/
export async function verifyModel(
target: OllamaTarget,
modelId: string,
context: number,
signal?: AbortSignal
): Promise<VerifyResult> {
return notImplemented('verifyModel', target, modelId, context, signal);
}
src/local-model/paths.ts
+87
View File
@@ -0,0 +1,87 @@
/**
* @fileoverview Where everything of the local model lives on disk. One pure function, so the
* runtime, the install job and uninstall all name the same folders.
*
* Layout, all under the data dir (`dataPath()`, so a second Codeman instance has its own):
*
* ```
* local-model.json the state file (mode 0600)
* local-model/
* runtime/ollama-<version>.tgz.partial the runtime archive while it downloads
* runtime/ollama-<version>.unpack/ where the archive is unpacked, before the rename
* runtime/ollama-<version>/ the unpacked runtime; its binary is `ollama`
* models/ the runtime's model store
* home/ the runtime's private HOME (its key pair lands here)
* ```
*
* Rules:
* - `root` is the ONE folder this feature owns. Nothing is written or deleted outside it,
* except the state file beside it and the two writes of the connect step (the endpoint
* record and the settings flag), which have their own writers.
* - Every path comes from here. None is ever read back from the state file, so a state file
* cannot point a delete anywhere else. `isInsideLocalModelRoot()` is the check a delete
* makes before it runs.
* - Pure apart from `dataPath()`, which resolves (and creates) the data dir.
*/
import { join, resolve, sep } from 'node:path';
import { dataPath } from '../config/instance.js';
/** A runtime version as it may appear in a folder name: digits and dots only ('0.40.15'). */
const VERSION_PATTERN = /^\d+(?:\.\d+){1,3}$/;
export interface LocalModelPaths {
/** `<data dir>/local-model`: the one folder this feature owns. */
root: string;
/** `<root>/runtime`: the archive and every unpacked version. */
runtimeRoot: string;
/** `<root>/runtime/ollama-<version>`: the unpacked runtime. */
runtimeDir: string;
/** The runtime's binary: `<runtimeDir>/ollama`. */
binary: string;
/** `<root>/runtime/ollama-<version>.tgz.partial`: the archive while it downloads. */
archivePartial: string;
/** `<root>/runtime/ollama-<version>.unpack`: the temporary folder the archive is unpacked into. */
unpackDir: string;
/** `<root>/models`: the runtime's model store. */
modelsDir: string;
/** `<root>/home`: the runtime's private HOME. */
homeDir: string;
/** `<data dir>/local-model.json`: the state file. */
stateFile: string;
}
/**
* The paths of the local model for one runtime version. Throws on a version that is not
* digits and dots, so a version string can never add a path segment.
*/
export function localModelPaths(version: string): LocalModelPaths {
if (!VERSION_PATTERN.test(version)) {
throw new Error(`Not a runtime version: ${JSON.stringify(version)}`);
}
const root = dataPath('local-model');
const runtimeRoot = join(root, 'runtime');
const runtimeDir = join(runtimeRoot, `ollama-${version}`);
return {
root,
runtimeRoot,
runtimeDir,
binary: join(runtimeDir, 'ollama'),
archivePartial: join(runtimeRoot, `ollama-${version}.tgz.partial`),
unpackDir: join(runtimeRoot, `ollama-${version}.unpack`),
modelsDir: join(root, 'models'),
homeDir: join(root, 'home'),
stateFile: dataPath('local-model.json'),
};
}
/**
* Whether `path` is the local model folder itself or lies inside it, judged on the resolved
* text (no filesystem access, so a symlink is not followed: a caller that deletes checks the
* real path first). `..` segments are resolved before the comparison.
*/
export function isInsideLocalModelRoot(path: string, root: string = dataPath('local-model')): boolean {
const base = resolve(root);
const target = resolve(path);
return target === base || target.startsWith(base + sep);
}
src/local-model/recommend.ts
+31 -1
View File
@@ -71,6 +71,7 @@ import {
type LocalModelMessage,
type NoneReason,
type Recommendation,
type RecommendOptions,
type SpeedGrade,
} from '../types.js';
import { chipBandwidthGbps } from '../config/local-model-catalog.js';
@@ -135,6 +136,29 @@ function fitGrade(need: number, budgetBytes: number): FitGrade {
return 'out';
}
/**
* Whether the catalogue model `modelId` at `context` tokens fits this Mac's budget (a `good`
* or a `tight` fit). The install job asks this right after the runtime started, with the
* budget the runtime measured, BEFORE it downloads the model: a `false` ends the job with
* `does-not-fit`. It is the same judgement `recommend()` makes for a candidate, without the
* ranking, so a model that fits is never ruled out for being listed sixth.
*
* `options` is the stage 2 input, as for `recommend()`; nothing reads it yet, so the budget
* is still the estimate. False for a model the catalogue does not have.
*/
export function modelFitsBudget(
hw: HardwareFacts,
catalog: LocalModelCatalog,
modelId: string,
context: number,
options: RecommendOptions = {}
): boolean {
void options;
const model = catalog.models.find((entry) => entry.id === modelId);
if (!model) return false;
return fitGrade(needBytes(model, context, catalog, hw), deviceBudget(hw).budgetBytes) !== 'out';
}
/**
* Whether the MACHINE is Apple silicon. `hw.optional.arm64` decides when it was read. Without
* it: a process under Rosetta, an arm64 process or an Apple chip name each prove Apple silicon
@@ -256,12 +280,18 @@ function none(
/**
* The recommendation for this Mac. `harnessFacts` holds one fact per catalogue harness the
* caller could resolve; a catalogue harness without a fact is not offered.
*
* `options` is the stage 2 input (`RecommendOptions`: the budget the runtime measured, and
* the runtime download an install still needs). The parameter is in place so callers compile
* against the final signature; nothing reads it yet.
*/
export function recommend(
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[]
harnessFacts: readonly HarnessFact[],
options: RecommendOptions = {}
): Recommendation {
void options;
// Off macOS and on an Intel Mac there is no budget to compute: a model has no memory to run in.
const noBudget: DeviceBudget = { limitBytes: 0, budgetBytes: 0, source: 'estimated' };
if (hw.platform !== 'darwin') return none('not-macos', noBudget);
src/local-model/runtime.ts
+308
View File
@@ -0,0 +1,308 @@
/**
* @fileoverview The model runtime: which build Codeman runs, how it gets onto the disk, and
* how it is kept running.
*
* CONTRACT STAGE: `OLLAMA_RUNTIME_PIN` and the constants are final. Every function below has
* its final signature and a body that throws; the bodies are written next.
*
* What this module does:
* - THE PIN. `OLLAMA_RUNTIME_PIN` is a compiled constant. No file, setting, environment
* variable or request can change the URL, the size or the hash.
* - DOWNLOAD (`ensureRuntimeInstalled()`). The archive is streamed to
* `LocalModelPaths.archivePartial` while it is hashed. A cut stream is continued with a
* Range request, at most `RUNTIME_DOWNLOAD_RETRIES` times. A body longer than `pin.bytes`
* is refused. The sha256 is compared BEFORE anything is unpacked, and a mismatch deletes the
* file. Unpacking is `/usr/bin/tar` into `LocalModelPaths.unpackDir`, then one rename to
* `LocalModelPaths.runtimeDir`.
* - ENVIRONMENT (`runtimeEnv()`, pure). The child's environment is built from an ALLOWLIST:
* PATH, TMPDIR, LANG and the proxy variables (`HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`, in
* either case) are copied from the server's; HOME is the private one; and exactly these are
* set: `OLLAMA_HOST=127.0.0.1:<port>`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`,
* `OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`,
* `OLLAMA_MAX_LOADED_MODELS=1`. Nothing else gets through: no `CODEMAN_*` and none of the
* user's own `OLLAMA_*`.
* - PORT (`chooseRuntimePort()`). Chosen ONCE by binding, from `RUNTIME_PORT_FIRST` across
* `RUNTIME_PORT_SPAN`, then stored by the caller. Never searched again on a later start.
* - SUPERVISOR (`createRuntimeSupervisor()`). The runtime is a NON-detached child of the
* server. `start()` first looks at the port: a listener that answers `/api/version` and
* whose pid (`previousPid`) runs a binary under our runtime folder is our own orphan and is
* adopted or replaced; any other listener is the answer `port-in-use`. A child that exits by
* itself is restarted, at most `RUNTIME_MAX_RESTARTS` times in a row; `stop()` sends SIGTERM.
* The supervisor does not write the state file: it reports through `onChange`, and its
* caller stores the pid and the port.
* - BUDGET (`parseInferenceComputeTotal()`, pure). Right after it starts, the runtime prints
* the device total in its log line "inference compute" (`total="37.4 GiB"`). The supervisor
* reads the child's output for it and reports it as `measuredTotalBytes`.
* - REMOVAL (`removeLocalModelFiles()`). Deletes the runtime, the private home and, when
* asked, the models. Only inside `LocalModelPaths.root`.
*
* Under vitest nothing spawns and nothing is fetched unless a test injects it: the default
* `spawn` and `fetch` of the dependency objects refuse there.
*/
import type { ChildProcess, SpawnOptions } from 'node:child_process';
import type { LocalModelErrorCode, LocalModelRuntimePin } from '../types.js';
import type { LocalModelPaths } from './paths.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
// ---------- The pin and the constants ----------
/** The one runtime build Codeman downloads and runs. See the file header. */
export const OLLAMA_RUNTIME_PIN: LocalModelRuntimePin = Object.freeze({
id: 'ollama',
version: '0.40.15',
url: 'https://github.com/ollama/ollama/releases/download/v0.40.15/ollama-darwin.tgz',
bytes: 185_971_671,
sha256: '607c79d5d6315ef9d896db83adde736a4c0057196329843544e79b95afb260a8',
// Measured on a MacBook on 2026-11-06: the unpacked folder takes 579 MB.
unpackedBytes: 579_000_000,
minMacosMajor: 14,
});
/** The first loopback port tried for the runtime. Not 11434: that is the default of a runtime the user may run. */
export const RUNTIME_PORT_FIRST = 11435;
/** How many ports are tried from `RUNTIME_PORT_FIRST`. */
export const RUNTIME_PORT_SPAN = 20;
/** How often in a row a runtime that exited by itself is started again. */
export const RUNTIME_MAX_RESTARTS = 3;
/** How often a cut archive download is continued. */
export const RUNTIME_DOWNLOAD_RETRIES = 3;
/** `OLLAMA_KEEP_ALIVE`: how long the model stays loaded after the last request. */
export const RUNTIME_KEEP_ALIVE = '10m';
// ---------- Pure ----------
/**
* The device total of the runtime's log line "inference compute", in bytes, or `undefined`
* for any other line. The real line (a MacBook, 2026-11-06):
*
* `time=... level=INFO source=types.go:32 msg="inference compute" id=0 ... type=iGPU total="37.4 GiB" available="37.4 GiB"`
*
* reads 40,157,944,217 (37.4 x 1024^3, rounded down). The value is the quoted text after
* ` total=`: a number and a binary unit (`KiB`, `MiB`, `GiB`, `TiB`). `total_vram=` of the
* line "vram-based default context" is not read.
*/
export function parseInferenceComputeTotal(line: string): number | undefined {
return notImplemented('parseInferenceComputeTotal', line);
}
/** What `runtimeEnv()` is built from. */
export interface RuntimeEnvInput {
/** The server's environment. Only the allowlisted names are read from it. */
source: NodeJS.ProcessEnv;
/** The private HOME (`LocalModelPaths.homeDir`). */
home: string;
/** The model store (`LocalModelPaths.modelsDir`). */
modelsDir: string;
/** The loopback port. */
port: number;
/** The context length in tokens. */
contextLength: number;
}
/** The environment of the runtime child: the allowlist of the file header, nothing more. Pure. */
export function runtimeEnv(input: RuntimeEnvInput): NodeJS.ProcessEnv {
return notImplemented('runtimeEnv', input);
}
/**
* Whether `command` (one line of `ps -o command= -p <pid>`) is a process whose binary lies
* under `runtimeRoot` (`LocalModelPaths.runtimeRoot`): our own runtime, so an orphan of an
* earlier server. Pure. `/private` in front of a path is the same path on macOS.
*/
export function isOwnRuntimeCommand(command: string, runtimeRoot: string): boolean {
return notImplemented('isOwnRuntimeCommand', command, runtimeRoot);
}
// ---------- Download and unpack ----------
/** How far the archive download is. */
export interface RuntimeDownloadProgress {
completedBytes: number;
/** `pin.bytes`. */
totalBytes: number;
}
/** What `ensureRuntimeInstalled()` calls. Each member defaults to the real thing; a test injects its own. */
export interface RuntimeInstallDeps {
/** Fetches the archive. Must honour `signal` and a `Range` header. */
fetch: typeof fetch;
/** Unpacks `archive` into the existing empty folder `intoDir` (`/usr/bin/tar -xzf`). Rejects on a non-zero exit. */
untar: (archive: string, intoDir: string, signal?: AbortSignal) => Promise<void>;
/** Waits between retries. */
sleep: (ms: number) => Promise<void>;
}
export interface RuntimeInstallOptions {
/** Default `OLLAMA_RUNTIME_PIN`. A test passes a pin that points at its fake server. */
pin?: LocalModelRuntimePin;
/** Default `localModelPaths(pin.version)`. */
paths?: LocalModelPaths;
/** Aborts the download or the unpack. The partial archive stays for a later Range request. */
signal?: AbortSignal;
/** Called while the archive downloads, at most a few times a second. */
onProgress?: (progress: RuntimeDownloadProgress) => void;
deps?: Partial<RuntimeInstallDeps>;
}
/**
* How getting the runtime ended. A failure carries the job's error code; `cancelled` is the
* abort signal and is not an error.
*/
export type RuntimeInstallResult =
| {
ok: true;
/** `LocalModelPaths.binary`. */
binaryPath: string;
/** True: the folder was already unpacked and nothing was downloaded. */
alreadyInstalled: boolean;
}
| {
ok: false;
code: Extract<
LocalModelErrorCode,
'runtime-download-failed' | 'runtime-damaged' | 'runtime-unpack-failed' | 'disk-full'
>;
detail?: string;
}
| { ok: false; code: 'cancelled' };
/** Whether the pinned runtime is unpacked on disk (its binary exists). Never throws. */
export async function isRuntimeInstalled(pin?: LocalModelRuntimePin, paths?: LocalModelPaths): Promise<boolean> {
return notImplemented('isRuntimeInstalled', pin, paths);
}
/**
* Makes sure the pinned runtime is unpacked: nothing to do when it is, else download (with
* Range resume), compare the hash, unpack, rename. Idempotent, so a resumed job calls it
* again. Never throws: every outcome is a `RuntimeInstallResult`.
*/
export async function ensureRuntimeInstalled(options: RuntimeInstallOptions = {}): Promise<RuntimeInstallResult> {
return notImplemented('ensureRuntimeInstalled', options);
}
/** What `removeLocalModelFiles()` deletes. */
export interface RemoveLocalModelFilesOptions {
/** True: delete the model store too. False: the runtime, its archive and the private home only. */
models: boolean;
/** Default `localModelPaths(OLLAMA_RUNTIME_PIN.version)`. */
paths?: LocalModelPaths;
}
/**
* Deletes what this feature put on disk: every runtime folder and archive, the private home
* and, when asked, the models. Refuses any path that is not inside `LocalModelPaths.root`,
* follows no symlink out of it, and removes `root` itself only when it is empty afterwards.
* The state file and the coding agent are not its business.
*/
export async function removeLocalModelFiles(options: RemoveLocalModelFilesOptions): Promise<void> {
return notImplemented('removeLocalModelFiles', options);
}
// ---------- Port ----------
export interface ChooseRuntimePortOptions {
/** Default `RUNTIME_PORT_FIRST`. */
first?: number;
/** Default `RUNTIME_PORT_SPAN`. */
span?: number;
/** Ports to skip: the one that is in use when the person asked for another. */
avoid?: readonly number[];
/** Whether nothing holds `port` on 127.0.0.1, decided by BINDING it. A test injects its own. */
isFree?: (port: number) => Promise<boolean>;
}
/** The first free loopback port of the span, or `undefined` when none is free. Chosen once, then stored by the caller. */
export async function chooseRuntimePort(options: ChooseRuntimePortOptions = {}): Promise<number | undefined> {
return notImplemented('chooseRuntimePort', options);
}
// ---------- Supervisor ----------
/** What the supervisor knows about the runtime right now. Plain data. */
export interface RuntimeStatus {
/** The child Codeman started (or adopted) is alive and answered `/api/version`. */
running: boolean;
pid?: number;
port?: number;
/** The context length the running child was started with. */
contextLength?: number;
/** The device total parsed from the child's log, in bytes. Absent for an adopted orphan (its log is not ours to read). */
measuredTotalBytes?: number;
/** How often the child was restarted since the last `start()`. */
restarts: number;
/** True: the child exited `RUNTIME_MAX_RESTARTS` times in a row and is no longer restarted (`runtime-stopped`). */
gaveUp: boolean;
}
export interface RuntimeStartOptions {
/** The stored port. */
port: number;
/** The context length in tokens (`OLLAMA_CONTEXT_LENGTH`). A running child with another value is replaced. */
contextLength: number;
/** The pid the state file holds from an earlier server: decides whether a listener on `port` is our own orphan. */
previousPid?: number;
}
/** How a `start()` ended. */
export type RuntimeStartResult =
| {
ok: true;
pid: number;
port: number;
/** What happened to a listener that was already on the port: there was none, or it was ours. */
orphan: 'none' | 'adopted' | 'replaced';
/** The device total, when the log line was read before `start()` returned. */
measuredTotalBytes?: number;
}
| {
ok: false;
/** `port-in-use`: a listener that is not ours. `runtime-start-failed`: no answer in time, or the child exited. */
code: Extract<LocalModelErrorCode, 'port-in-use' | 'runtime-start-failed'>;
detail?: string;
};
/** What the supervisor calls. Each member defaults to the real thing; a test injects its own. */
export interface RuntimeSupervisorDeps {
/** Starts the child. The default refuses under vitest. */
spawn: (command: string, args: readonly string[], options: SpawnOptions) => ChildProcess;
/** Asks the runtime for `/api/version`. */
fetch: typeof fetch;
/** The server's environment, the `source` of `runtimeEnv()`. */
env: NodeJS.ProcessEnv;
/** `ps -o command= -p <pid>`, bounded. `undefined` when the pid is gone. */
commandOfPid: (pid: number) => Promise<string | undefined>;
/** Signals a pid that is not our child (an orphan). */
kill: (pid: number, signal: NodeJS.Signals) => void;
/** Whether nothing holds `port` on 127.0.0.1, decided by binding it. */
isPortFree: (port: number) => Promise<boolean>;
sleep: (ms: number) => Promise<void>;
/** Default `OLLAMA_RUNTIME_PIN`. */
pin: LocalModelRuntimePin;
/** Default `localModelPaths(pin.version)`. */
paths: LocalModelPaths;
/** Called after every change of `status()`: a start, an exit, a restart, the budget line. The caller stores pid and port. */
onChange: (status: RuntimeStatus) => void;
}
/** The one supervisor of the runtime child. */
export interface RuntimeSupervisor {
/**
* Makes sure the runtime runs on `port` with `contextLength`. Idempotent: a child that
* already runs with both is kept. Resolves once `/api/version` answers the pinned version
* and the budget line was read (or a few seconds passed without it). Never throws.
*/
start(options: RuntimeStartOptions): Promise<RuntimeStartResult>;
/** SIGTERM to the child, SIGKILL after a grace period. Resolves once it is gone. Safe to call when nothing runs. */
stop(): Promise<void>;
status(): RuntimeStatus;
}
/** A supervisor. The server keeps ONE (see `getLocalModelService()` in install-job.ts). */
export function createRuntimeSupervisor(deps: Partial<RuntimeSupervisorDeps> = {}): RuntimeSupervisor {
return notImplemented('createRuntimeSupervisor', deps);
}
src/local-model/state-store.ts
+57
View File
@@ -0,0 +1,57 @@
/**
* @fileoverview The ONE reader and writer of `local-model.json`, the state file of the local
* model install (`LocalModelStateFile`).
*
* CONTRACT STAGE: `IDLE_LOCAL_MODEL_STATE` is final. Every function below has its final
* signature and a body that throws; the bodies are written next.
*
* Why one module: the install job writes the state, the step and the digest, and the caller
* of the runtime supervisor writes the pid and the port. Two writers with their own temp file
* and rename would lose each other's change, which is the fault `mutateCustomModelHosts()`
* exists to end for the endpoint store. So every write goes through `mutateLocalModelState()`.
*
* The shape of `mutateSettingsFile()` (src/web/settings-writer.ts):
* - Serialized: mutations run one at a time on one promise chain.
* - Atomic: each write goes to a temp file with a random name beside the target and is
* renamed over it. The file is mode 0600.
* - Reading is TOTAL, unlike the settings writer: the file holds no setting of the user, only
* what this feature recorded, so a missing file, one that does not parse or one with
* another `schema` reads as `IDLE_LOCAL_MODEL_STATE` and the next write replaces it.
* - The path is `localModelPaths().stateFile`. The file itself holds no path.
*/
import type { LocalModelStateFile } from '../types.js';
/** Marks a body the contract stage left unwritten. */
function notImplemented(what: string, ..._args: unknown[]): never {
throw new Error(`${what} is not implemented yet`);
}
/** The state when nothing was ever installed. What an unreadable file reads as. */
export const IDLE_LOCAL_MODEL_STATE: LocalModelStateFile = Object.freeze({ schema: 1, state: 'idle' });
/**
* The state a file's text stands for. Pure and total: text that is not a JSON object of
* `schema: 1` with a known `state` is `IDLE_LOCAL_MODEL_STATE`. Members it does not know are
* dropped, and a member of the wrong type is dropped rather than trusted.
*/
export function parseLocalModelState(raw: string): LocalModelStateFile {
return notImplemented('parseLocalModelState', raw);
}
/** The stored state. Never throws: a file that cannot be read is `IDLE_LOCAL_MODEL_STATE`. */
export async function readLocalModelState(): Promise<LocalModelStateFile> {
return notImplemented('readLocalModelState');
}
/**
* Runs one change to the state file. `fn` receives the current state and returns the state to
* write, or `undefined` to write nothing. Resolves with the state before and after the change
* (`after` is `undefined` when nothing was written). `updatedAt` is stamped here. A failed
* write rejects its own caller only; the chain keeps going.
*/
export function mutateLocalModelState(
fn: (current: LocalModelStateFile) => LocalModelStateFile | undefined
): Promise<{ before: LocalModelStateFile; after: LocalModelStateFile | undefined }> {
return notImplemented('mutateLocalModelState', fn);
}