The catalogue had seven Qwen-family rows that all carried the memory
figures measured on one model (Qwen3.6 35B-A3B on the MLX runner), so a
9B or a dense 27B was judged with figures that do not describe it, two
rows had the wrong Mac download size, and every Mac below 32 GB got
"none".
Now eight rows, each with the runner a Mac really uses for its tag (a new
`runner` field: `mlx` or `llamacpp`) and memory figures of its own:
- the 35B-A3B MLX row keeps its measured figures and the tested pair;
- every llama.cpp row is computed from its model's config.json: a q8_0
K/V cache for the whole context (what the runtime now sets), its
recurrent or sliding-window state, 1.0 GB of runner and the 1 GiB host
prompt cache;
- the one other MLX row (Ornith 1.5 122B) is computed the same way,
scaled by what the measured row showed, plus the runner's 8 GiB of
prefix snapshots.
Sizes are the registry's Mac pull totals (2026-11-08, lowercase ids
verified there). What each memory size gets: 8 GB none; 16 to 32 GB a 9B
(limited); 36 GB Qwen3.8 27B or the 35B-A3B as GGUF; 48 GB the tested
pair (or the 27B on a Max chip); 64 and 96 GB the 27B or the 35B-A3B;
128 to 512 GB Ornith 1.5 122B. Nothing but the one row was measured.
The catalogue test recomputes every computed figure from a table of
architectures and lets two rows share figures only when they share the
architecture and the runner. The rule tests of recommend() now run on a
frozen copy of the 2026-11-06 catalogue, so a weekly catalogue update
changes only the per-memory table and the real-Mac tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/check-local-model-catalog.mjs reads every catalogue tag the way
Ollama 0.40.15 on Apple silicon reads it: it asks for the manifest list,
takes the mlx build when the tag offers one (else llamacpp, else ggml),
and compares that build's pull total (layers plus config) with the row's
`bytes`. It also reports a tag that is gone, a build whose `requires` is
newer than the pinned runtime, a pinned digest that moved, a newer Ollama
release, and a pinned ollama-darwin.tgz that is gone or changed. It never
requests a layer, bounds every request, retries nothing and validates
everything that reaches the report. Exit 0 clean, 1 drift, 2 failed.
The plain Docker manifest of a list tag is the llamacpp build, not what a
Mac pulls. Read the right way, the first run finds two rows sized from the
wrong build: qwen3.5:9b (a Mac downloads 7,639,229,431 bytes, +16.6%) and
qwen3.8:27b (18,174,721,847, +2.4%).
.github/workflows/local-model-catalog.yml runs it Mondays at 06:00 UTC
and on dispatch. The check job installs with --ignore-scripts and holds a
read-only token; a separate job with issues: write and no checkout opens
or updates the one "Local model catalogue: weekly check" issue on drift,
comments on it on a clean week when it is open, and is skipped when the
check failed. Actions are pinned by commit.
docs/local-models.md gains "Keeping the catalogue current".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `LocalModelService.launchRefusal(endpointId)`: what a session launch asks
before it starts on the managed endpoint. It answers the error that
refuses the launch (the runtime child is not alive, another program holds
the port, the runtime keeps stopping, the model's digest changed) or
nothing. New error code `runtime-not-running` with its sentence, for a
runtime the person stopped: only a refused session start says it.
- The fake runtime's MacBook log line is rebuilt from the two values the
measurement notes quote, so it is `INFERENCE_COMPUTE_LINE_MACBOOK` now.
Only the Mac mini line is a verbatim capture and keeps `REAL_`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.
Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
states, steps and error codes with one sentence and one next step each,
the progress, the `install` object of GET /api/local-model, the bodies and
409 answers of the install verbs, the state file. `LocalModelStatus` moves
here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
measured speed of a tested entry, the `measured` budget source,
`RecommendOptions`, and the row tag, the coding agents and the install
total of a candidate. All optional until the catalogue and `recommend()`
carry them, so the read-only stage keeps compiling and its tests stay
green.
Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.
Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.
Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A forked or spawned thread's rollout carries its own session_meta followed
by its parent's and a copy of the parent's history. A later session_meta
for a declared ancestor now opens an inherited prefix that is dropped:
by ordinal below subagent_history_start_ordinal in paginated history, and
up to the thread_settings_applied event Codex appends after the copy in
legacy history, checked against the parent's own messages when the copy
holds more than one candidate. An unrelated id still refuses the file.
Store attribution now has its own budget (--metadata-bytes) separate from
the transcript budget (--scan-bytes), reads each head in 32 KiB steps, and
--max-entries is a flag, so id lookups work on stores of thousands of
rollouts. An unattributable store file or unreadable directory is counted
instead of aborting the lookup, and a rollout that a running Codex appends
to is read as a snapshot up to its last complete line.
User records carry no phase, date bounds are UTC calendar dates, the
injected-context rule is the shared one, a symlinked sessions root is
followed as the Past Sessions scanner follows it, and --json with
--format text is refused.
feat(codex): raise codex approval, question and trust dialogs as BLOCKED
Conflict resolved against #634 in src/session.ts _probePaneWorking():
#624's assignment (a dialog on screen is never work) is kept, and #634's
trust-dialog verdict block (log the hold once, remember the reading)
follows it, before _readWatching().
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On codex-cli 0.162.0, with 0.163.0 released, every launch first shows the
"Update available" prompt, and choosing Skip leads to a variant of the
workspace trust prompt: "2. Back to Agent Command Center" over the key-hint
row "enter continue · esc back". The pattern only accepted cancel,
interrupt, quit, answer or all as the closing word, so a fresh git repo
launched from Codeman sat on that trust prompt reading idle, with no red
alert and no blocked signal (measured live on a throwaway instance).
Adding "back" is safe because the hint row must still START with one of
the anchored hints ("enter continue" here): the popups the user opens at
the keyboard (/model, /permissions, /review, Implement this plan?) end in
"esc back" too, but their row starts with "enter select", so they stay
unmatched, as every existing negative fixture confirms. The new fixture is
the live capture with the scratch path replaced.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Codex is the one CLI Codeman launches with its approvals on by default, and
it declares `hooks: 'none'`, so nothing tells Codeman it stopped to ask. A
codex session parked on "Would you like to run the following command?" read
as plainly idle, and one parked on a question read as WORKING, because the
question's own key hints end in `esc to interrupt`, codex's working line.
This adds a generic, optional registry field,
`capabilities.workDetect.blockedLine` (plus `blockedLines`, the window), and
the session state that reads it:
- `isBlockedScreen()` (session-activity.ts, pure) joins the last
`blockedLines` non-blank rows of a capture and tests the pattern once, so a
pattern can demand several markers and anchor the dialog's footer on the
end of the window.
- `Session` reads it off the same capture the idle confirmation already
takes, emits `blockedChanged`, and exposes `screenBlocked`. It raises on
one reading, and while a dialog is up re-reads the screen every 2 s,
because answering resumes the turn and a busy pane would not reach the
settle path again until the turn ended. It clears only after two readable
captures in a row without the dialog, so a capture taken mid-repaint cannot
clear and re-raise it; an unreadable capture changes nothing. A blocked
screen wins over the working line in the same capture.
- The pattern goes through `compileVersionRegex()` in both the schema refine
and at use (lazily cached), like every other config regex.
Codex's pattern was measured on a live codex-cli 0.162.0 pane on 2026-10-14
(`-a on-request -s read-only`, widths 200 down to 36): it needs a `› N.`
option row and then the dialog's key-hint row as the last row, allowing for
that row to wrap below about 60 columns. Command, edit and network-escalation
approvals, single and multi-question dialogs and the startup trust prompt
match; the idle and working views, menus the user opened (`/model`,
`/permissions`, `/review`), the composer holding the dialogs' words and a
transcript quoting them verbatim do not. `Implement this plan?` is left out
on purpose: its footer is the same as `/model`'s and the turn has ended
there. The captures are kept as fixtures (trailing whitespace trimmed, the
scratch path replaced).
Nothing consumes `blockedChanged` yet; the next commit raises the alert.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The XLSX preview hands an admitted copy of the workbook to ExcelJS 4.4.0,
which links each worksheet to its comments, comment notes (VML), tables and
drawing by looking the relationship Target up verbatim in maps keyed by its
own spelling, then dereferences the result unchecked. openpyxl, the library
most agents write .xlsx files with, uses absolute Targets
(/xl/tables/table1.xml), its own comment part name (xl/comments/comment1.xml,
which ExcelJS never loads) and a drawing XML ExcelJS parses to nothing. One
comment, table or chart was enough for "Spreadsheet preview failed".
repairExcelJsModel() (spreadsheet-xlsx-core.js) runs on ExcelJS's parsed
model right before its reconcile: it re-points a link to the key ExcelJS
stored for the same part (so an openpyxl table is kept) and drops a link,
tablePart, drawing or hyperlink reference that would dereference nothing.
It adds nothing and resolves keys as own properties only. The worker wraps
workbook.xlsx.reconcile per instance to call it, and throws if a future
ExcelJS renames that method. SPREADSHEET_ASSET_VERSION is refreshed.
Tests: unit tests on model objects, and three real openpyxl 3.1.5
workbooks in test/fixtures/xlsx that fail the worker tests without the hook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Custom Model Endpoint Profiles (#393) let a session point its CLI at a
custom OpenAI-compatible endpoint by injecting env vars or a config file
and restarting the CLI in place. Review of the apply path found four
things, two of them destructive. This lands all four plus the smaller
items from the same review.
1. Clearing a selection did not clear it. The injected vars reach the CLI
via `tmux setenv`, which persists at the tmux-session level and is
inherited by `respawn-pane` (measured: `setenv FOO bar` survived two
successive `respawn-pane -k`), so deleting the keys from the session's
envOverrides relaunched the CLI still pointed at the old endpoint, and
for the configDir kinds at a HOME/CODEX_HOME/GROK_HOME that had just
been deleted. `Session.setCustomModel()` now reports the removed keys,
queues them (`_pendingEnvUnsets`), and `RespawnPaneOptions.unsetEnvKeys`
carries them into `applyEnvOverrides()`, which `setenv -u`s them before
re-applying the live overrides, on the same path that already unsets
the legacy CLAUDE_CODE_EFFORT_LEVEL. Verified on a private tmux socket
that `setenv -u HOME` hands the next respawn the global HOME back.
2. Applying a model to a local claude session killed the pane. The
relaunch was `claude --session-id <id>` and Claude refuses an id that
already has a transcript, and unlike the dead-pane respawn this one
kills a working pane first. `restartCli()` now pins the live
conversation id as the resume id for that respawn when the CLI's launch
declares a `fallback` chain, which renders the same
`--resume <id> || --session-id <id>` shape the docker and remote pane
commands use. Gated on the registry shape, not the CLI id: an entry
whose resume id is minted by the CLI itself never declares that chain.
3. pi, omp and grok wrote their config file and then launched without the
`--model` that selects it, so the file was ignored. The registry entry
now declares `customModelInjection.launchModel` (`custom/{modelId}` for
pi and omp, grok's `[model.codeman-custom]` block name), the builder
renders it, and `_withCustomModelLaunchModel()` applies it onto the
respawn options through `legacyConfigField`, leaving the stored
<Mode>Config untouched so a clear falls back to the user's own model.
A model id the CLI's `model` token pattern cannot carry is refused
with a 400 rather than silently dropped by the argv engine.
4. Remote (SSH) and Docker sessions reported `restarted: true` and changed
nothing: their `restartCli()` reattaches the durable tmux rather than
relaunching the agent, and the env lands on the local pane. Both are
refused with a 400 until those paths are plumbed.
Smaller items from the same review:
- The selection survives a Codeman restart as the disk-only `__customModel`
bookkeeping (endpoint, model, injected key NAMES, config dir, launch
model; never the values, which carry the API key). Recovery re-derives
the values from the endpoint store through the same apply path the route
uses and keeps the bookkeeping even when the endpoint is gone, so a
later clear still has keys to unset.
- Discovery goes through `webviewFetch()`, so the RESOLVED address is
judged by the same egress guard the web-tab proxy uses, and `baseUrl`
reuses `webviewUrlSchema` (http(s) only, no embedded credentials,
link-local and cloud-metadata addresses refused). undici's `fetch failed`
wrapper is unwrapped so the user sees the ECONNREFUSED underneath.
- `custom-model-hosts.json` is written 0600 via tmp+rename, the per-session
config dir 0700/0600 (pi and omp embed the key literally), and that dir
is removed with the session.
- `PR.md` is gone from the repo root and the design doc moved to
`docs/custom-model-endpoints-plan.md` with the LAN address and the
personal name scrubbed; every reference follows. The guide's `authStyle`
text matches the shipped schema (`bearer | api-key`, default `bearer`)
and says that `customModelEndpointsEnabled` is read by nothing until
the picker lands.
- `config/tsconfig.scripts.json` typechecks `scripts/test-local-llm-harnesses.ts`
(four real type errors fixed). It is not yet wired into `npm run typecheck`
because that line differs on master; adding `&& tsc -p config/tsconfig.scripts.json`
there is the one-line follow-up.
Tests: `test/session-custom-model-restart.test.ts` drives a real Session and
fails on the unfixed code for items 1 to 3; the route suite covers item 4
and the pattern refusal; `test/tmux-manager.test.ts` pins that the unsets
run before the overrides and that a shell-metachar key never reaches tmux.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Point any Codeman-supported harness (Claude, opencode, Codex, Gemini, Pi,
Grok, DeepSeek, OMP) at a custom OpenAI-compatible endpoint instead of its
native cloud backend, for a given session. Covers local hardware (llama.cpp,
Ollama, vLLM, DGX Spark, Strix Halo) and cloud (Azure AI Foundry, OpenRouter).
Off by default (customModelEndpointsEnabled, synced, default OFF).
- Registry: capabilities.customModelInjection per CLI entry (env /
configContentEnv / configDir / unsupported kinds)
- Pure injection builder (custom-model-injection.ts) turning an endpoint +
model id into the real env vars / config content per CLI
- Endpoint store + CRUD routes (custom-model-hosts.ts,
custom-model-routes.ts), discovery via GET /v1/models, SSRF-guarded
- Session integration: Session.setCustomModel()/restartCli()
(POST /api/sessions/:id/custom-model), reusing the existing
respawn-pane -k primitive to restart the CLI process with new env
- Multi-user hardening: every new redirect-capable env var added to its
CLI's privilegedEnvKeys, closing a pre-existing gap where several were
already reachable via the generic envOverrides field's prefix allowlist
- Standalone scripts/test-local-llm-harnesses.mjs: spawns real CLI binaries
against a real endpoint outside the web UI, independent of tmux/sessions
- Mock-server contract tests (test/fixtures/mock-openai-server.ts) replaying
every CLI's injected values through a real HTTP shape
Real end-to-end validation against a live llama-swap server (inside a
codeman/agent:llm-test Docker image with all 9 CLI binaries) found and
fixed three real bugs before they shipped:
- Codex's config.toml schema was wrong ([model].default table instead of
a top-level model string + [model_providers.custom]); fixing it then
surfaced a genuine, documented protocol incompatibility (Codex only
speaks the Responses API since Feb 2026, which llama.cpp/llama-swap
don't implement)
- Claude Code's async session-title-generation call validates
ANTHROPIC_DEFAULT_HAIKU_MODEL against its own internal model list and
hangs the whole -p invocation on an unrecognized name; documented for
chunk 6, worked around in the standalone script only (--bare is NOT
safe for a real interactive session, which needs hooks)
- The discovery route's authStyle: 'both' option (send both Authorization
and api-key headers) reliably hung a real server; removed the option
entirely rather than just changing the default
Status: draft. Chunk 6 (frontend toolbar/settings UI) not yet built — see
PR.md and deployment_plan.md for the full chunk breakdown and confidence
table.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HqNWfmtBU2KN29SvSVWB3
Codeman maintainer