Consolidates the four contributor changesets, and rewords the em-dashes this release
added in comments and docs. The five in test/settings-save-failure-ui.test.ts stay:
they assert existing toast text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- share one context-length resolution (discovered figure, endpoint default fallback) across the routes and boot recovery, with a recovery test
- record listed per-slot context figures with a slot count of 1 and force the slot-count fallback once per rediscovery
- open up the codex model launch token for qwen3-style ids (model-plain-colon), pinned by a spawn-golden case
- pass the full discovered window to opencode as limit.context
- translate the picker search strings and document the discovery precedence
Squash of PR #658. Discovery records each llama.cpp model's parallel slot
count (from --parallel/-np, or /props total_slots when the launch command
states none), and a Claude launch warns when the context split across
those slots leaves less than 40K tokens per request. An explicit
--parallel beside a /props context records a slot count of 1, so a later
round overwrites an earlier count, and a non-2xx /props is remembered per
launch command while a network error is retried.
Merge-time fix by the maintainer: the architecture-invariants sentence on
the explicit --parallel case now says it records a slot count of 1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflict resolution against the five PRs that landed while this was in review, plus
the items left for merge on the thread.
The real one was `session-ui.js`. #454 refactored all eight non-Claude `run*()`
functions to funnel through one `_launchQuickStartInstances()` helper that does the
POST itself, while this PR replaced that same POST in each of them with
`_quickStartWithCustomModelConfirm()`. Resolved in the helper rather than seven times
over: the helper now goes through the confirm path, and each body builder carries the
`customModel` spread. `runAntigravity` deliberately does NOT, since antigravity's
`customModelInjection` is `unsupported`; parity with this PR's own per-mode choices is
asserted rather than assumed.
That merge creates a question neither feature had alone: the confirm dialog now runs
inside a loop that can launch up to 20 instances. Both questions it can ask (context
window too small, and loading this will unload the model another session is using) are
decisions about the ENDPOINT, and every instance in a batch targets the same one, so
the answer is taken once and carried to the rest. Without that a 20-instance launch
asks the same question 20 times.
Also: `sse-events.ts` is 161 constants (master added two for remote wake, this adds
one, verified by counting rather than by arithmetic), `server.ts` keeps both new SSE
prefixes, the two comments pointing at code that no longer exists are corrected, and
CLAUDE.md's SSE and route counts move to 161 / ~236 / custom-model (6).
`pumpLlamaSwapLogTail`'s unparsed remainder is now capped at 64 KiB. It only shrank at
a `\n\n` frame boundary, so a backend that streams without one would grow it for the
life of a deliberately indefinite connection.
NOT changed, deliberately: the context warning and the swap-conflict warning still
share one `confirmed` flag with the context check first, so confirming "launch anyway"
on a too-small context also skips the "this unloads it for another session" ask. That
is the author's documented choice and the reviewer's own note calls it minor. Both
fixes are worse to make here than to defer: separate flags are new wire surface landed
unreviewed during a release, and reordering the checks adds a network round trip to a
path that currently short-circuits. Raised as a follow-up instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Discovery now also parses a GB figure out of an auto-discovered model's own
description (llama-swap writes "Auto-discovered 16.35 GB - parameters
auto-fitted by llama.cpp"), stored per model as modelSizesGB - unlike
context length this needs no /props probe (the figure is right there in
/v1/models) so it is populated for every model regardless of loaded state.
A hand-configured profile's own description has no such figure and
correctly gets no entry.
The loading banner (_watchLlamaSwapLoading) now looks this up and, when
known, shows it plus a rough estimate from a small size->time matrix
(_estimateModelLoad/_MODEL_LOAD_TIME_MATRIX, session-ui.js) -
"Loading qwen3.8-27b-ud-q4_k_xl (16.4 GB, typically ~1-3 min) on
llama-swap... this can take a while" - and uses that same estimate's own
bracket to scale the banner's default give-up timeout for a very large
model, instead of a flat 5 minutes for everything. Explicitly labelled as
an UNMEASURED, typical-hardware estimate in every relevant comment - this
is not benchmarked against any real endpoint's actual storage/GPU, just a
reasonable expectation-setter. A model with no discoverable size (a
hand-configured profile) gets no size/estimate shown at all, matching the
"never a guess" convention modelContextLengths already established.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
Addresses two live-validation findings on the Run-menu custom-model picker:
1. Both claude.ai and ANTHROPIC_API_KEY set warning. Claude Code still
coexists an OAuth login with an injected ANTHROPIC_API_KEY in the same
config directory and warns about it (confirmed cosmetic - the API key
wins for actual requests, verified via a real session's own API Usage
Billing line). A custom-model claude session now gets an isolated
CLAUDE_CONFIG_DIR (registry-declared via a new configDirVar field, empty,
no files written into it) so there is nothing to conflict with. projects
is symlinked (junction on Windows) back into the real config dir so the
response viewer, subagent windows and Read My Mind keep working for that
session, best-effort.
2. Context-window overflow. Claude Code assumes a large default context
window for a model id it doesn't recognise and never compacts, so a
custom endpoint's real, much smaller context (verified live: a 400
exceeding a 16384-token llama-swap model with a stock ~33.7K-token system
prompt) silently overflows. Discovery now also learns each model's real
context length from llama.cpp/llama-swap's GET /props?model=<id> (n_ctx),
but ONLY for a model llama-swap's own /v1/models response already marks
status.value === 'loaded' - never an unloaded one, since llama-swap
treats ?model= as a routing hint and probing an unloaded model risks
triggering an actual, slow, GPU-swapping load as a side effect of
read-only discovery. A server with no status field at all gets no
enrichment rather than a guess; a model not probed this round keeps its
previously-learned value until it disappears from the list entirely.
Stored per model (CustomModelHost.modelContextLengths) and applied via a
new contextLengthVar registry field, set to
CLAUDE_CODE_MAX_CONTEXT_TOKENS for claude.
Both new fields live on the existing env-kind customModelInjection
capability shape, declared only on claude's registry entry - every other
CLI's injection is unaffected (pinned by test).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
Follow-up to #393, picking up the work Ark0N invited in his merge comment:
"generate those entries from the saved profiles rather than a fixed
duplicate per harness, and put it in a follow-up PR so this one stays the
backend... The Run-menu picker is yours if you want it."
Adds the frontend surface the backend has been waiting on:
- Run menu: a "Custom Endpoints" section lists one entry per (harness that
supports customModelInjection, saved endpoint) pair, e.g.
"Claude Code (llama.cpp)". The harness list comes from
window.__codemanCustomModelClis, injected at page render straight off the
CLI registry's own capabilities (never a hardcoded id list in the
frontend), so a CLI whose injection recipe lands later appears with no
frontend change. Picking an entry runs that harness's own existing run*()
function unmodified (case creation, env overrides, everything, forced to
a single instance) and then applies the endpoint's default model to the
session it creates via the existing POST /api/sessions/:id/custom-model
route. Entries are hidden for a remote/docker active case, since that
route already refuses both.
- Settings: App Settings -> Models gets a "Custom model endpoints" group
wiring up the customModelEndpointsEnabled toggle (declared since #393,
read by nothing until now) plus CRUD against the existing
/api/model-endpoints routes: list, add/edit (inline form), delete,
discover models.
- Backend: CustomModelHost gains an optional defaultModelId, the model the
picker applies with no further choice per endpoint (one generated menu
entry per CLI+endpoint pair, not per CLI+endpoint+model). The route
refuses a value that isn't one of the endpoint's own discovered models,
and a fresh discovery drops a default that no longer appears rather than
carrying an invalid one forward.
Docs: docs/custom-model-endpoints.md describes the new picker and settings
panel; CLAUDE.md's Custom Model Endpoint Profiles entry drops the
"backend-only" status note and documents the picker's generation mechanism.
Tests: four new route tests cover defaultModelId validation, acceptance,
and the drop/keep behaviour across a re-discovery; a new render-index-html
test pins the __codemanCustomModelClis injection (present, agent CLIs
supporting the capability, antigravity and shell excluded) and its
solo-window skip. No browser test was added for the Run-menu picker itself
or the settings CRUD panel (this box has no tmux, so the live server used
by test:browser/test:mobile could not be exercised here) -- worth a
Playwright pass before merge, same as any other frontend PR.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
Custom Model Endpoint Profiles (#393) let a session point its CLI at a
custom OpenAI-compatible endpoint by injecting env vars or a config file
and restarting the CLI in place. Review of the apply path found four
things, two of them destructive. This lands all four plus the smaller
items from the same review.
1. Clearing a selection did not clear it. The injected vars reach the CLI
via `tmux setenv`, which persists at the tmux-session level and is
inherited by `respawn-pane` (measured: `setenv FOO bar` survived two
successive `respawn-pane -k`), so deleting the keys from the session's
envOverrides relaunched the CLI still pointed at the old endpoint, and
for the configDir kinds at a HOME/CODEX_HOME/GROK_HOME that had just
been deleted. `Session.setCustomModel()` now reports the removed keys,
queues them (`_pendingEnvUnsets`), and `RespawnPaneOptions.unsetEnvKeys`
carries them into `applyEnvOverrides()`, which `setenv -u`s them before
re-applying the live overrides, on the same path that already unsets
the legacy CLAUDE_CODE_EFFORT_LEVEL. Verified on a private tmux socket
that `setenv -u HOME` hands the next respawn the global HOME back.
2. Applying a model to a local claude session killed the pane. The
relaunch was `claude --session-id <id>` and Claude refuses an id that
already has a transcript, and unlike the dead-pane respawn this one
kills a working pane first. `restartCli()` now pins the live
conversation id as the resume id for that respawn when the CLI's launch
declares a `fallback` chain, which renders the same
`--resume <id> || --session-id <id>` shape the docker and remote pane
commands use. Gated on the registry shape, not the CLI id: an entry
whose resume id is minted by the CLI itself never declares that chain.
3. pi, omp and grok wrote their config file and then launched without the
`--model` that selects it, so the file was ignored. The registry entry
now declares `customModelInjection.launchModel` (`custom/{modelId}` for
pi and omp, grok's `[model.codeman-custom]` block name), the builder
renders it, and `_withCustomModelLaunchModel()` applies it onto the
respawn options through `legacyConfigField`, leaving the stored
<Mode>Config untouched so a clear falls back to the user's own model.
A model id the CLI's `model` token pattern cannot carry is refused
with a 400 rather than silently dropped by the argv engine.
4. Remote (SSH) and Docker sessions reported `restarted: true` and changed
nothing: their `restartCli()` reattaches the durable tmux rather than
relaunching the agent, and the env lands on the local pane. Both are
refused with a 400 until those paths are plumbed.
Smaller items from the same review:
- The selection survives a Codeman restart as the disk-only `__customModel`
bookkeeping (endpoint, model, injected key NAMES, config dir, launch
model; never the values, which carry the API key). Recovery re-derives
the values from the endpoint store through the same apply path the route
uses and keeps the bookkeeping even when the endpoint is gone, so a
later clear still has keys to unset.
- Discovery goes through `webviewFetch()`, so the RESOLVED address is
judged by the same egress guard the web-tab proxy uses, and `baseUrl`
reuses `webviewUrlSchema` (http(s) only, no embedded credentials,
link-local and cloud-metadata addresses refused). undici's `fetch failed`
wrapper is unwrapped so the user sees the ECONNREFUSED underneath.
- `custom-model-hosts.json` is written 0600 via tmp+rename, the per-session
config dir 0700/0600 (pi and omp embed the key literally), and that dir
is removed with the session.
- `PR.md` is gone from the repo root and the design doc moved to
`docs/custom-model-endpoints-plan.md` with the LAN address and the
personal name scrubbed; every reference follows. The guide's `authStyle`
text matches the shipped schema (`bearer | api-key`, default `bearer`)
and says that `customModelEndpointsEnabled` is read by nothing until
the picker lands.
- `config/tsconfig.scripts.json` typechecks `scripts/test-local-llm-harnesses.ts`
(four real type errors fixed). It is not yet wired into `npm run typecheck`
because that line differs on master; adding `&& tsc -p config/tsconfig.scripts.json`
there is the one-line follow-up.
Tests: `test/session-custom-model-restart.test.ts` drives a real Session and
fails on the unfixed code for items 1 to 3; the route suite covers item 4
and the pattern refusal; `test/tmux-manager.test.ts` pins that the unsets
run before the overrides and that a shell-metachar key never reaches tmux.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Point any Codeman-supported harness (Claude, opencode, Codex, Gemini, Pi,
Grok, DeepSeek, OMP) at a custom OpenAI-compatible endpoint instead of its
native cloud backend, for a given session. Covers local hardware (llama.cpp,
Ollama, vLLM, DGX Spark, Strix Halo) and cloud (Azure AI Foundry, OpenRouter).
Off by default (customModelEndpointsEnabled, synced, default OFF).
- Registry: capabilities.customModelInjection per CLI entry (env /
configContentEnv / configDir / unsupported kinds)
- Pure injection builder (custom-model-injection.ts) turning an endpoint +
model id into the real env vars / config content per CLI
- Endpoint store + CRUD routes (custom-model-hosts.ts,
custom-model-routes.ts), discovery via GET /v1/models, SSRF-guarded
- Session integration: Session.setCustomModel()/restartCli()
(POST /api/sessions/:id/custom-model), reusing the existing
respawn-pane -k primitive to restart the CLI process with new env
- Multi-user hardening: every new redirect-capable env var added to its
CLI's privilegedEnvKeys, closing a pre-existing gap where several were
already reachable via the generic envOverrides field's prefix allowlist
- Standalone scripts/test-local-llm-harnesses.mjs: spawns real CLI binaries
against a real endpoint outside the web UI, independent of tmux/sessions
- Mock-server contract tests (test/fixtures/mock-openai-server.ts) replaying
every CLI's injected values through a real HTTP shape
Real end-to-end validation against a live llama-swap server (inside a
codeman/agent:llm-test Docker image with all 9 CLI binaries) found and
fixed three real bugs before they shipped:
- Codex's config.toml schema was wrong ([model].default table instead of
a top-level model string + [model_providers.custom]); fixing it then
surfaced a genuine, documented protocol incompatibility (Codex only
speaks the Responses API since Feb 2026, which llama.cpp/llama-swap
don't implement)
- Claude Code's async session-title-generation call validates
ANTHROPIC_DEFAULT_HAIKU_MODEL against its own internal model list and
hangs the whole -p invocation on an unrecognized name; documented for
chunk 6, worked around in the standalone script only (--bare is NOT
safe for a real interactive session, which needs hooks)
- The discovery route's authStyle: 'both' option (send both Authorization
and api-key headers) reliably hung a real server; removed the option
entirely rather than just changing the default
Status: draft. Chunk 6 (frontend toolbar/settings UI) not yet built — see
PR.md and deployment_plan.md for the full chunk breakdown and confidence
table.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017HqNWfmtBU2KN29SvSVWB3
Codeman maintainer