Merge pull request #752: fix(custom-model): context limits, Codex model override, context discovery and picker filter

This commit is contained in:
Codeman maintainer committed 2026-11-04 16:51:56 +01:00
commit 3bcb55ce96
29 files changed
+1125 -102

No files matched your search

@@ -0,0 +1,5 @@
---
'aicodeman': patch
---
Custom endpoints now hand their context window to the CLIs that ask for it. OpenCode and Codex receive the discovered context length on injection (#749), and Codex launches on the selected custom model instead of its default (#750). Endpoint discovery reads each model's context size from `/v1/models` (with a sanity ceiling), with a per-endpoint "Default context length" field for gateways that answer nothing, and boot recovery keeps the saved figure (#754). The "Choose a model" picker also grew a search box that filters the rows as you type.
+30 -7
View File
@@ -1205,24 +1205,47 @@ data}` envelope on the wire — unwrap it the same way). Answers `[]` for a
boolean` reports whether one is stored, so a client can render "unchanged boolean` reports whether one is stored, so a client can render "unchanged
if left blank" without ever holding the real value. if left blank" without ever holding the real value.
- `POST /api/v1/model-endpoints` with `{ id, label, baseUrl, apiKey?, - `POST /api/v1/model-endpoints` with `{ id, label, baseUrl, apiKey?,
authStyle?, defaultModelId? }` creates one. `id` must match authStyle?, defaultModelId?, defaultContextLength? }` creates one. `id`
must match
`^[a-zA-Z0-9_-]+$`; `authStyle` is `bearer` (default) or `api-key`, never `^[a-zA-Z0-9_-]+$`; `authStyle` is `bearer` (default) or `api-key`, never
both (a real server hung indefinitely when sent both headers on one both (a real server hung indefinitely when sent both headers on one
request); `baseUrl` must be `http(s)`, carry no embedded credentials, and request); `defaultContextLength` is a user-set fallback used when
assigning a model that discovery has no context length for (discovery
itself never writes it); `baseUrl` must be `http(s)`, carry no embedded
credentials, and
is refused if it points at (or resolves to) a link-local or is refused if it points at (or resolves to) a link-local or
cloud-metadata address. `409 ALREADY_EXISTS` on a duplicate id. cloud-metadata address. `409 ALREADY_EXISTS` on a duplicate id.
- `PUT /api/v1/model-endpoints/:id` updates one. An **absent** `apiKey` - `PUT /api/v1/model-endpoints/:id` updates one, also accepting
`defaultContextLength`. An **absent** `apiKey`
keeps the stored one rather than clearing it — the client never receives keeps the stored one rather than clearing it — the client never receives
the real value to resend deliberately unchanged, so omission is the only the real value to resend deliberately unchanged, so omission is the only
way to say "leave it alone"; there is no way to clear a key back to unset way to say "leave it alone"; there is no way to clear a key back to unset
this way. `defaultModelId`, when set, must be one of that endpoint's own this way. `defaultContextLength` behaves the opposite way: a PUT without
it clears the stored value, so send the field back to keep it.
`defaultModelId`, when set, must be one of that endpoint's own
`models` (`400 INVALID_INPUT` otherwise). `models` (`400 INVALID_INPUT` otherwise).
- `DELETE /api/v1/model-endpoints/:id` removes one. - `DELETE /api/v1/model-endpoints/:id` removes one.
- `POST /api/v1/model-endpoints/:id/discover-models` fetches the endpoint's - `POST /api/v1/model-endpoints/:id/discover-models` fetches the endpoint's
own `GET /v1/models` and stores the result as `models`, updating own `GET /v1/models` and stores the result as `models`, updating
`lastDiscoveredAt`, plus (best-effort, only for a model llama-swap's own `lastDiscoveredAt`, plus (best-effort) `modelContextLengths`,
response already reports loaded) `modelContextLengths` and `modelSizesGB`. `modelSlotCounts` and `modelSizesGB`: every listing entry is read for a
A `defaultModelId` that no longer appears in the fresh list is dropped stated context figure (`context_length`, then `context_window`) and a
size, BEFORE any loaded check; only the launch-command/`/props`
enrichment needs a model llama-swap's own response reports loaded. A
loaded model's context figure comes from llama-swap's own launch command
(`--fit-ctx` / `-c` / `--ctx-size`), falling back to `/props` (`n_ctx`,
`total_slots`) for whatever the command did not answer, and OVERWRITES the
listed figure for that model; the model's own listing
(`context_length`, then `context_window`) only ever fills the gap when
neither probe does, and is recorded as a single-slot figure —
llama-swap lists the PER-SLOT window, so whenever the listed figure is
the one taken the stored slot count is 1. A server with no `status`
field on any entry (plain llama.cpp, a non-llama-swap gateway) gets no
launch-command/`/props` enrichment at all, only whatever its listings
state. `modelContextLengths` falls back to the
endpoint's `defaultContextLength` when a session is assigned a model
discovery never learned a window for. A `defaultModelId` that no longer
appears in the fresh list is dropped
rather than carried forward invalid. Failures answer `422 OPERATION_FAILED` rather than carried forward invalid. Failures answer `422 OPERATION_FAILED`
with the underlying connection error, or a named egress refusal if the with the underlying connection error, or a named egress refusal if the
resolved address turned out to be blocked. The same refresh also runs resolved address turned out to be blocked. The same refresh also runs
+2 -2
View File
@@ -630,9 +630,9 @@ Adding the header changed the preamble, so `CODEMAN_PREAMBLE` was bumped (1.22.0
**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`, design doc `docs/custom-model-endpoints-plan.md`; full stack — settings-panel CRUD + the Run-menu picker, on top of the backend below): points a session at a user-configured custom OpenAI-compatible endpoint — local (llama.cpp, DGX Spark, Strix Halo) or cloud (Azure AI Foundry, OpenRouter) — instead of its harness's native cloud backend. Endpoints are a read/write-array store (`custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`) discovered via `GET <baseUrl>/v1/models`; `CustomModelHost.authStyle` is `'bearer'` (default, `Authorization: Bearer`) or `'api-key'` (Azure's convention) — **never both**, live-tested against a real server: sending both headers on one request reliably hangs it indefinitely, reproduced 3×. **Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`, design doc `docs/custom-model-endpoints-plan.md`; full stack — settings-panel CRUD + the Run-menu picker, on top of the backend below): points a session at a user-configured custom OpenAI-compatible endpoint — local (llama.cpp, DGX Spark, Strix Halo) or cloud (Azure AI Foundry, OpenRouter) — instead of its harness's native cloud backend. Endpoints are a read/write-array store (`custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`) discovered via `GET <baseUrl>/v1/models`; `CustomModelHost.authStyle` is `'bearer'` (default, `Authorization: Bearer`) or `'api-key'` (Azure's convention) — **never both**, live-tested against a real server: sending both headers on one request reliably hangs it indefinitely, reproduced 3×.
⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified. ⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/copilot/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified.
⚠️ **Two launch paths, chosen by mechanism, not preference**: opencode/codex/gemini/pi/grok/deepseek/omp apply the selection ONE-SHOT, before the session/process ever exists, with no restart at all; claude alone still applies a selection by **restarting the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` relaunches a CLI in an existing pane, so a CLI whose launch declares a `fallback` chain (claude) gets a conversation pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. The dead-pane respawn shares this pin, see [Dead-pane respawn: the resume pin](#dead-pane-respawn-the-resume-pin). ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600. ⚠️ **Two launch paths, chosen by mechanism, not preference**: opencode/codex/gemini/pi/grok/deepseek/omp apply the selection ONE-SHOT, before the session/process ever exists, with no restart at all; claude alone still applies a selection by **restarting the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` relaunches a CLI in an existing pane, so a CLI whose launch declares a `fallback` chain (claude) gets a conversation pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. The dead-pane respawn shares this pin, see [Dead-pane respawn: the resume pin](#dead-pane-respawn-the-resume-pin). ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name), and codex gets one too (`--model <bare modelId>`, its pattern widened to `model-plain-colon` for qwen3-style ids): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600.
⚠️ **Security**: every env var this feature can redirect (`ANTHROPIC_BASE_URL`, `GOOGLE_GEMINI_BASE_URL`, `CODEX_HOME`, `GROK_HOME`, `HOME` for pi/omp, `OPENCODE_CONFIG_CONTENT`, etc.) is in that CLI's `privilegedEnvKeys` — several of these were reachable via the generic `envOverrides` field's prefix allowlist BEFORE this feature existed (the env allowlist is global and prefix-based, not per-CLI-scoped), so building this surfaced and closed a pre-existing gap rather than opening a new one. `ANTHROPIC_*` is deliberately NOT in claude's `allowedPrefixes` at all — Anthropic-traffic redirection can only happen through this feature's own admin-configured, SSRF-guarded route, never a plain client-supplied `envOverrides`. ⚠️ **Security**: every env var this feature can redirect (`ANTHROPIC_BASE_URL`, `GOOGLE_GEMINI_BASE_URL`, `CODEX_HOME`, `GROK_HOME`, `HOME` for pi/omp, `OPENCODE_CONFIG_CONTENT`, etc.) is in that CLI's `privilegedEnvKeys` — several of these were reachable via the generic `envOverrides` field's prefix allowlist BEFORE this feature existed (the env allowlist is global and prefix-based, not per-CLI-scoped), so building this surfaced and closed a pre-existing gap rather than opening a new one. `ANTHROPIC_*` is deliberately NOT in claude's `allowedPrefixes` at all — Anthropic-traffic redirection can only happen through this feature's own admin-configured, SSRF-guarded route, never a plain client-supplied `envOverrides`.
+43 -17
View File
@@ -67,15 +67,23 @@ configured, `PUT`/`DELETE /api/model-endpoints/:id` update or remove one.
Endpoint management is admin-only in multi-user mode, same as remote/docker Endpoint management is admin-only in multi-user mode, same as remote/docker
hosts — these are machine-level infra, not per-user settings. hosts — these are machine-level infra, not per-user settings.
**Context length is discovered too, opportunistically and safely.** The plain **Context length is discovered too, opportunistically and safely.** Older
`GET /v1/models` response has no context-window field. Discovery only ever servers state no context-window field in the plain `GET /v1/models`
looks for one for a model llama-swap's own response already reports response at all; current llama-swap states one per model (both
`context_length` and `context_window`) and gateways like Bifrost pass
`context_window` through — that listed figure is the *lowest*-precedence
source, used only when neither probe below answers, and because llama-swap
documents it as the PER-SLOT window, a model whose figure comes from the
listing is recorded with a slot count of 1 so the slot check never divides a
per-slot figure a second time. The probes are only ever looked for on a model
llama-swap's own response already reports
`status.value === "loaded"` for — never for an unloaded one, because `status.value === "loaded"` for — never for an unloaded one, because
llama-swap treats `?model=` as a routing hint and asking about a model that llama-swap treats `?model=` as a routing hint and asking about a model that
isn't loaded risks triggering an actual (slow, GPU-swapping) load as a side isn't loaded risks triggering an actual (slow, GPU-swapping) load as a side
effect of what should be read-only discovery. A server with no `status` field effect of what should be read-only discovery. A server with no `status` field
on any entry at all (not llama-swap) gets no context-length enrichment, on any entry at all (not llama-swap) gets no launch-command/`/props`
rather than guessing. A model's previously-learned context length survives a enrichment, only whatever its listings state — rather than guessing. A
model's previously-learned context length survives a
later cycle where it wasn't the loaded one; it's dropped only once the model later cycle where it wasn't the loaded one; it's dropped only once the model
disappears from the endpoint's list entirely. Stored per model in disappears from the endpoint's list entirely. Stored per model in
`modelContextLengths` and applied automatically (see "Applying a model to a `modelContextLengths` and applied automatically (see "Applying a model to a
@@ -286,6 +294,15 @@ hot-swap. A Claude session is relaunched with `--resume <conversation> ||
grok are relaunched with the `--model` value that selects the injected grok are relaunched with the `--model` value that selects the injected
provider (`custom/<modelId>` for pi and omp, `codeman-custom` for grok), provider (`custom/<modelId>` for pi and omp, `codeman-custom` for grok),
since for those three the config file alone does not switch the model. since for those three the config file alone does not switch the model.
**Codex is relaunched with `--model <modelId>` too**, the bare selected id
(not a provider selector): the launch param forces the session onto the
model its generated `config.toml` already selected, so a launch-time model
carried over from App Settings can never override the injected selection.
That launch arg must satisfy the codex entry's `model-plain-colon` token
pattern (a leading letter or digit, then letters, digits and `._-:/`, per
`stock.ts`), so an id outside that shape, e.g. LM Studio's
`org/model@q4_k_m`, is refused with a 400 at apply time now, where a
`config.toml` alone used to carry it.
**Remote (SSH) and Docker sessions are refused** (400) for now: their restart **Remote (SSH) and Docker sessions are refused** (400) for now: their restart
reattaches the durable remote/in-container tmux rather than relaunching the reattaches the durable remote/in-container tmux rather than relaunching the
agent, so the selection would report success and change nothing. agent, so the selection would report success and change nothing.
@@ -293,14 +310,16 @@ agent, so the selection would report success and change nothing.
**Claude gets two more env vars when known/applicable, both declared on its **Claude gets two more env vars when known/applicable, both declared on its
registry entry (`contextLengthVar`/`configDirVar`), not hardcoded here:** registry entry (`contextLengthVar`/`configDirVar`), not hardcoded here:**
- `CLAUDE_CODE_MAX_CONTEXT_TOKENS` is set to `modelId`'s discovered context - `CLAUDE_CODE_MAX_CONTEXT_TOKENS` is set to `modelId`'s context length —
length (see the discovery section above) whenever one is known. Without the discovered figure (see the discovery section above), falling back to
the endpoint's user-set `defaultContextLength` — whenever one is known.
Without
it, Claude Code assumes a large (200k) window for any unrecognized custom it, Claude Code assumes a large (200k) window for any unrecognized custom
model id and never compacts, which reliably overflows a much smaller real model id and never compacts, which reliably overflows a much smaller real
local context — confirmed live: a stock ~33.7K-token system prompt against local context — confirmed live: a stock ~33.7K-token system prompt against
a 16384-token llama-swap model failed with `exceeds the available context a 16384-token llama-swap model failed with `exceeds the available context
size`. No entry for the model in `modelContextLengths` means the var is size`. Neither a `modelContextLengths` entry nor a `defaultContextLength`
simply omitted, never a guess. ⚠️ **This var only affects when Claude for the model means the var is simply omitted, never a guess. ⚠️ **This var only affects when Claude
Code compacts conversation _history_ — it cannot fix a model whose real Code compacts conversation _history_ — it cannot fix a model whose real
context is smaller than Claude Code's own fixed per-turn overhead** context is smaller than Claude Code's own fixed per-turn overhead**
(system prompt + tool schemas, empirically ~36.4K tokens, confirmed live (system prompt + tool schemas, empirically ~36.4K tokens, confirmed live
@@ -441,15 +460,17 @@ Both apply routes (the restart route and the one-shot `POST
/api/quick-start`) now check for this **before** launching or restarting /api/quick-start`) now check for this **before** launching or restarting
anything, gated on the CLI's registry entry declaring a `contextLengthVar` anything, gated on the CLI's registry entry declaring a `contextLengthVar`
(currently only claude — the check is a no-op for every other CLI by (currently only claude — the check is a no-op for every other CLI by
construction, never a hardcoded mode check). If the model's discovered construction, never a hardcoded mode check). If the model's context (the
context (`modelContextLengths`, from discovery above) is below discovered figure from `modelContextLengths`, or the endpoint's
`CLAUDE_MIN_SAFE_CONTEXT_TOKENS` (40000, comfortably above the measured `defaultContextLength` when discovery never learned one for that model) is
below `CLAUDE_MIN_SAFE_CONTEXT_TOKENS` (40000, comfortably above the measured
~36.4K overhead), the response is `{requiresContextWarning: true, modelId, ~36.4K overhead), the response is `{requiresContextWarning: true, modelId,
contextLength, minSafeContextTokens}` (plus `slotCount` when the model runs more than one parallel slot) instead of applying — nothing is contextLength, minSafeContextTokens}` (plus `slotCount` when the model runs more than one parallel slot) instead of applying — nothing is
restarted or created yet. A context length that was never discovered at restarted or created yet. A model with neither a discovered figure nor a
all skips the check entirely (nothing to compare, so it fails open rather `defaultContextLength` skips the check entirely (nothing to compare, so it
than warning on every model an endpoint hasn't reported a size for). fails open rather than warning on every model an endpoint hasn't reported a
The same warning fires when the context divided across the model's parallel slots (`contextLength / slotCount`) is below the floor, because Claude Code sends several 40K-token requests at once and the slots share one pool. Retrying with `confirmedContext: true` launches anyway (the legacy `confirmed: true` still means both questions). size for).
The same warning fires when the context divided across the model's parallel slots (`contextLength / slotCount`) is below the floor, because Claude Code sends several 40K-token requests at once and the slots share one pool. ⚠️ `defaultContextLength` is the model's whole window, never a per-slot figure, so a default small enough to split under the slot count trips that slot division exactly like a discovered figure does. Retrying with `confirmedContext: true` launches anyway (the legacy `confirmed: true` still means both questions).
The Run-menu picker shows this as an in-app modal The Run-menu picker shows this as an in-app modal
(`#customModelContextWarningModal`, matching the llama-swap conflict (`#customModelContextWarningModal`, matching the llama-swap conflict
@@ -526,7 +547,12 @@ for '<id>' not found. Defaulting to fallback metadata...` on launch —
and fabricating a fake catalog entry would mean copying the _shape_ of and fabricating a fake catalog entry would mean copying the _shape_ of
OpenAI's own proprietary schema (their per-model system-prompt content OpenAI's own proprietary schema (their per-model system-prompt content
included) for a warning that doesn't otherwise affect behavior — not included) for a warning that doesn't otherwise affect behavior — not
something to build into discovery. something to build into discovery. ⚠️ The fallback-metadata warning is
NOT about the context window going un-overridden. Codex does now get a
real context-window override: when a figure is known the generated
`config.toml` sets the top-level `model_context_window` to it and
`model_auto_compact_token_limit` to 75% of it (floored), so Codex
compacts at that point instead of trusting its fallback guess.
- **Gemini** — fails with `Invalid auth method selected`, traced to an - **Gemini** — fails with `Invalid auth method selected`, traced to an
undocumented `GATEWAY` auth path gemini-cli selects once undocumented `GATEWAY` auth path gemini-cli selects once
`GOOGLE_GEMINI_BASE_URL` is set. Unresolved after real investigation `GOOGLE_GEMINI_BASE_URL` is set. Unresolved after real investigation
+17 -9
View File
@@ -27,14 +27,20 @@ hosts — these are machine-level infra, not a per-user setting.
shows up without another manual click of **Discover**. One endpoint being unreachable on a shows up without another manual click of **Discover**. One endpoint being unreachable on a
given cycle (powered off, wrong network) never blocks the others from refreshing. given cycle (powered off, wrong network) never blocks the others from refreshing.
**Context length is picked up automatically where it can be, safely.** Against a **Context length is picked up automatically where it can be, safely.** Every model's own
llama.cpp/llama-swap server, discovery also learns each _currently loaded_ model's real listing entry is read for a stated context figure (`context_length`, then `context_window`,
context window and applies it to the launched session (Claude Code today — see below), so which current llama.cpp, llama-swap and gateway-style servers state to varying degrees);
the harness stops assuming a large default window for a model name it doesn't recognise and against a llama-swap server the stronger figures from its launch command and from llama.cpp's
overflowing a much smaller real one. It's deliberately never probed for a model that isn't `/props` probe win where they exist, and those probes only ever run for a model the server
already loaded, since asking a llama-swap server about an unloaded model can trigger an already reports loaded, since asking about an unloaded model can trigger an actual, slow model
actual, slow model swap as a side effect — a model just not currently loaded keeps whatever swap as a side effect. A model that states no figure keeps whatever an earlier cycle learned:
context length an earlier cycle already learned for it instead. the stored figure always beats the endpoint's **Default context length** field
(`modelContextLengths[id] ?? defaultContextLength`), which only applies where no figure was
ever learned (that number is the model's whole window, never a per-slot figure). A model
neither covers simply gets no override, never a guess. The window is applied to the launched session by the harnesses
that can take it: Claude Code today (see below), OpenCode as the model's context limit, and
Codex as a `model_context_window` entry in its generated config file, which also sets Codex's
own auto-compact point via `model_auto_compact_token_limit` at 75% of the window.
## Running a session against one ## Running a session against one
@@ -48,7 +54,9 @@ pointed at.
**Which model it uses depends on how many the endpoint has discovered.** With exactly one, **Which model it uses depends on how many the endpoint has discovered.** With exactly one,
the session launches straight away on that model — nothing to choose. With two or more, a the session launches straight away on that model — nothing to choose. With two or more, a
small dialog asks which one to use for this launch before starting the session; the small dialog asks which one to use for this launch before starting the session; a search box
at the top narrows a long list with a case-insensitive substring match, so a server hosting
hundreds of models stays usable. The
endpoint's default model, if set, is marked but not auto-picked, so a launch can deliberately endpoint's default model, if set, is marked but not auto-picked, so a launch can deliberately
use a different one without changing the saved default. The list is not raw discovery order use a different one without changing the saved default. The list is not raw discovery order
either: the model llama-swap reports loaded and ready is moved to the top and tagged either: the model llama-swap reports loaded and ready is moved to the top and tagged
+5
View File
@@ -21,6 +21,7 @@ export type TokenPattern =
| 'model' | 'model'
| 'model-claude' | 'model-claude'
| 'model-plain' | 'model-plain'
| 'model-plain-colon'
| 'model-pi' | 'model-pi'
| 'id' | 'id'
| 'id-dotted' | 'id-dotted'
@@ -47,6 +48,10 @@ const PATTERNS: Record<TokenPattern, RegExp> = {
// copilot --model. Same characters as `model`, but a leading alphanumeric: the value follows a // copilot --model. Same characters as `model`, but a leading alphanumeric: the value follows a
// flag, so `--yolo` must not pass as a model id and become its own flag (the allowAll clamp bypass). // flag, so `--yolo` must not pass as a model id and become its own flag (the allowAll clamp bypass).
'model-plain': /^[a-zA-Z0-9][a-zA-Z0-9._\-/]*$/, 'model-plain': /^[a-zA-Z0-9][a-zA-Z0-9._\-/]*$/,
// codex --model, and the value a custom-model `launchModel` is forced onto it. Same
// shape as `model-plain` (leading alphanumeric, so a flag cannot ride the flag) plus
// `:` — Ollama-style ids like `qwen3:32b` must survive onto the command line.
'model-plain-colon': /^[a-zA-Z0-9][a-zA-Z0-9._\-/:]*$/,
// buildPiCommand — `:` for a thinking suffix (`sonnet:high`), `/` for `provider/id` // buildPiCommand — `:` for a thinking suffix (`sonnet:high`), `/` for `provider/id`
'model-pi': /^[a-zA-Z0-9._\-/:]+$/, 'model-pi': /^[a-zA-Z0-9._\-/:]+$/,
// opencode --session, codex resume // opencode --session, codex resume
+7 -3
View File
@@ -628,7 +628,10 @@ const CODEX: CliEntry = {
params: { params: {
bypassApprovals: { type: 'bool' }, bypassApprovals: { type: 'bool' },
animations: { type: 'bool' }, animations: { type: 'bool' },
model: { type: 'token', pattern: 'model' }, // `model-plain-colon`, not `model`: the custom-model feature forces the selected id
// onto this param (`launchModel: '{modelId}'` below), and Ollama-style ids carry a
// `:` (`qwen3:32b`). A leading alphanumeric still keeps `--yolo` from riding a flag.
model: { type: 'token', pattern: 'model-plain-colon' },
reasoningEffort: { type: 'enum', values: [...CODEX_REASONING_EFFORTS] }, reasoningEffort: { type: 'enum', values: [...CODEX_REASONING_EFFORTS] },
resumeId: { type: 'token', pattern: 'id' }, resumeId: { type: 'token', pattern: 'id' },
}, },
@@ -841,6 +844,7 @@ const CODEX: CliEntry = {
dirEnvVar: 'CODEX_HOME', dirEnvVar: 'CODEX_HOME',
fileName: 'config.toml', fileName: 'config.toml',
template: 'codex-toml', template: 'codex-toml',
launchModel: '{modelId}',
}, },
// CODEX_HOME already matches the CODEX_ allowedPrefix above, so it was ALREADY // CODEX_HOME already matches the CODEX_ allowedPrefix above, so it was ALREADY
// reachable via plain envOverrides before this feature existed. It is arguably // reachable via plain envOverrides before this feature existed. It is arguably
@@ -1921,8 +1925,8 @@ const COPILOT: CliEntry = {
// custom provider and drops the GitHub sign-in requirement; COPILOT_PROVIDER_TYPE defaults to // custom provider and drops the GitHub sign-in requirement; COPILOT_PROVIDER_TYPE defaults to
// "openai", which is what a Codeman custom endpoint is (llama.cpp, vLLM, Ollama). // "openai", which is what a Codeman custom endpoint is (llama.cpp, vLLM, Ollama).
// COPILOT_MODEL names the model and COPILOT_PROVIDER_MODEL_ID / _WIRE_MODEL default to it. // COPILOT_MODEL names the model and COPILOT_PROVIDER_MODEL_ID / _WIRE_MODEL default to it.
// The OpenAI-style route is `<base>/v1/chat/completions`, so the base gets the `/v1` suffix // The OpenAI-style route is `<base>/v1/chat/completions`, so the base gets the `/v1`
// the same way deepseek's does. All four already match the COPILOT_ prefix allowlist, so // suffix via appendV1Suffix below. All four already match the COPILOT_ prefix allowlist, so
// they are privileged env keys below: a non-granted multi-user owner must not be able to // they are privileged env keys below: a non-granted multi-user owner must not be able to
// redirect the session's endpoint or credentials through plain envOverrides. // redirect the session's endpoint or credentials through plain envOverrides.
customModelInjection: { customModelInjection: {
+12 -16
View File
@@ -812,9 +812,11 @@ export interface CliCapabilities {
* for pi and omp (`--model custom/<id>`, or the CLI stays on its own default * for pi and omp (`--model custom/<id>`, or the CLI stays on its own default
* provider and reports "No API key found for the selected model") or for * provider and reports "No API key found for the selected model") or for
* grok (`--model codeman-custom`, the `[model.<name>]` block the config * grok (`--model codeman-custom`, the `[model.<name>]` block the config
* declares). Absent = the config alone selects the model (claude's env vars, * declares), and codex carries its bare selected id (`{modelId}`, an id the
* opencode's blob, codex's top-level `model` key). Applied by the session's * entry's `model` token pattern must be able to carry). Absent = the config
* respawn options through the entry's `legacyConfigField`, never by id. * alone selects the model (claude's env vars, opencode's blob). Applied by
* the session's respawn options through the entry's `legacyConfigField`,
* never by id.
* *
* `contextLengthVar` (env kind only): the env var a discovered per-model context-window * `contextLengthVar` (env kind only): the env var a discovered per-model context-window
* size is written to when known (claude's `CLAUDE_CODE_MAX_CONTEXT_TOKENS`) — without it, * size is written to when known (claude's `CLAUDE_CODE_MAX_CONTEXT_TOKENS`) — without it,
@@ -856,19 +858,13 @@ export interface CliCapabilities {
* reuses that file. * reuses that file.
* *
* `appendV1Suffix` (env kind only): the raw `endpoint.baseUrl` gets `withV1Suffix()` * `appendV1Suffix` (env kind only): the raw `endpoint.baseUrl` gets `withV1Suffix()`
* applied before being written to `baseUrlVar`, instead of being used verbatim. * applied before being written to `baseUrlVar`, instead of being used verbatim. Only
* DeepSeek needs this and claude/gemini must NOT get it — a per-CLI asymmetry confirmed * deepseek and copilot declare it — both speak to `<base>/v1/chat/completions` and a
* by reading each SDK's own request-building source, not assumed: DeepSeek Harness's * gateway saves the bare root. claude/gemini must NOT get it: Claude Code's own SDK was
* bundled `@deepseek-ai/dsh-llm-deepseek` concatenates `${connection.baseURL}/chat/ * confirmed working end-to-end against the RAW `baseUrl`, so appending a suffix there
* completions` with no `/v1` insertion of its own (its real public API base, * would be wrong, not just redundant. The per-session config-file recipes do not need
* `https://api.deepseek.com`, expects the caller's base URL to already carry any * this flag at all — `renderConfigFile()` applies the suffix inside every template, so
* needed prefix), while llama-swap/llama.cpp only ever serves the OpenAI-conventional * no configDir entry carries a raw base URL.
* `/v1/chat/completions` — confirmed live: a bare `POST <baseUrl>/chat/completions`
* 404s, `POST <baseUrl>/v1/chat/completions` succeeds, and the harness's own error
* message template (`DeepSeek API error (HTTP ${status})`) reproduces the exact
* `HTTP_404` this feature originally shipped with unexplained. Claude Code's own SDK,
* by contrast, was already confirmed working end-to-end against the RAW `baseUrl` with
* no suffix — appending one there would be wrong, not just redundant.
*/ */
customModelInjection: customModelInjection:
| { | {
+35 -8
View File
@@ -52,18 +52,22 @@ export interface CustomModelHost {
defaultModelId?: string; defaultModelId?: string;
/** /**
* Discovered context-window size (tokens) per model id, keyed by the same strings as * Discovered context-window size (tokens) per model id, keyed by the same strings as
* `models`. Populated opportunistically during discovery (`custom-model-routes.ts`) from * `models`. Populated only by discovery (`custom-model-routes.ts`), from the strongest
* llama.cpp/llama-swap's `GET /props?model=<id>` — the plain OpenAI-shaped `/v1/models` * source the server offers: the llama-swap launch command reported by `GET /running`
* response has no such field. Only ever probed for a model the server already reports as * (the source of truth for a loaded model), then llama.cpp's `GET /props?model=<id>`
* loaded (llama-swap's `status.value === 'loaded'`); an unloaded one is deliberately never * `n_ctx` (only ever probed for a model the server already reports as loaded, since
* probed, since llama-swap treats `/props?model=` as a routing hint that can trigger an * llama-swap treats `/props?model=` as a routing hint that can trigger an actual
* actual (slow, GPU-swapping) model load as a side effect of merely asking. A model this * (slow, GPU-swapping) model load as a side effect of merely asking), then the model's
* has no entry for simply gets no context-length env override applied — never a guess. * own `GET /v1/models` entry (`context_length`, else `context_window`, which current
* llama-swap, Bifrost and OpenRouter-style servers each state to varying degrees). A
* model with no entry here falls back to the endpoint's `defaultContextLength`; with
* neither, it simply gets no context-length env override applied, never a guess.
*/ */
modelContextLengths?: Record<string, number>; modelContextLengths?: Record<string, number>;
/** /**
* Discovered number of parallel server slots per model id, keyed like `modelContextLengths` * Discovered number of parallel server slots per model id, keyed like `modelContextLengths`
* and populated on the same loaded-models-only basis. Read from the launch command's * but populated only for models the server reports loaded (a listing never states a slot
* count, unlike `contextLengths`, which every entry can supply). Read from the launch command's
* `--parallel`/`-np` when it states one, otherwise from llama.cpp's own `/props` * `--parallel`/`-np` when it states one, otherwise from llama.cpp's own `/props`
* `total_slots` (a llama-server started without `--parallel` picks the count itself, so * `total_slots` (a llama-server started without `--parallel` picks the count itself, so
* the command cannot say). llama-server splits its context across these slots (by default * the command cannot say). llama-server splits its context across these slots (by default
@@ -88,6 +92,29 @@ export interface CustomModelHost {
* server-side check relies on. * server-side check relies on.
*/ */
modelSizesGB?: Record<string, number>; modelSizesGB?: Record<string, number>;
/**
* User-set fallback context length (tokens), applied via
* `resolveCustomModelContextLength()` at every read site (the swap route, the
* quick-start route, boot recovery) only when `modelContextLengths` has no entry for
* the selected model (a gateway that states no per-model figure, a model never loaded
* yet). It is the model's WHOLE window, never a per-slot figure: the Claude floor
* check divides it by any stored slot count exactly like a discovered figure. Never
* written by discovery, so a value set in the endpoint editor survives every
* re-discovery sweep; unset keeps the long-standing behaviour of letting each CLI
* assume its own default.
*/
defaultContextLength?: number;
}
/**
* The context window to apply for `modelId` on `host`: the discovered per-model figure when
* discovery has one, else the endpoint's user-set `defaultContextLength`, else undefined
* (no override applied, never a guess). ONE helper for every read site — the swap route,
* the quickstart route and boot recovery each re-write the CLI's config files, and a site
* that forgot the fallback silently loses the endpoint default on the next respawn.
*/
export function resolveCustomModelContextLength(host: CustomModelHost, modelId: string): number | undefined {
return host.modelContextLengths?.[modelId] ?? host.defaultContextLength;
} }
export function customModelHostsPath(configDir: string): string { export function customModelHostsPath(configDir: string): string {
+37 -6
View File
@@ -136,12 +136,12 @@ export function buildCustomModelInjection(
} }
case 'configContentEnv': { case 'configContentEnv': {
const content = renderConfigContent(cap.template, endpoint, modelId, apiKey); const content = renderConfigContent(cap.template, endpoint, modelId, apiKey, contextLength);
return withLaunchModel({ kind: 'env', envOverrides: { [cap.envVar]: content } }, cap.launchModel, modelId); return withLaunchModel({ kind: 'env', envOverrides: { [cap.envVar]: content } }, cap.launchModel, modelId);
} }
case 'configDir': { case 'configDir': {
const { content, extraEnv } = renderConfigFile(cap.template, endpoint, modelId, apiKey); const { content, extraEnv } = renderConfigFile(cap.template, endpoint, modelId, apiKey, contextLength);
return withLaunchModel( return withLaunchModel(
{ kind: 'configDir', dirEnvVar: cap.dirEnvVar, files: [{ relPath: cap.fileName, content }], extraEnv }, { kind: 'configDir', dirEnvVar: cap.dirEnvVar, files: [{ relPath: cap.fileName, content }], extraEnv },
cap.launchModel, cap.launchModel,
@@ -164,24 +164,48 @@ function withLaunchModel<T extends EnvInjection | ConfigDirInjection>(
return { ...result, launchModel: template.split('{modelId}').join(modelId) }; return { ...result, launchModel: template.split('{modelId}').join(modelId) };
} }
/**
* One quarter of the window is held back for the reply, capped at 8192 tokens (a small local
* model's window must not lose a disproportionate slice to the reservation). Returns undefined
* for an unknown or non-finite size — opencode then keeps its own default guess, which is the
* pre-fix behaviour; never invent a limit here.
*/
function opencodeModelLimit(contextLength: number | undefined): { context: number; output: number } | undefined {
if (contextLength === undefined || !Number.isFinite(contextLength)) return undefined;
const total = Math.trunc(contextLength);
if (total <= 0) return undefined;
const output = Math.max(1, Math.min(8192, Math.floor(total / 4)));
// `limit.context` is the FULL window as far as opencode is concerned — it reserves for
// the reply itself — so the discovered total is passed through, never a shrunk figure.
return { context: total, output };
}
function renderConfigContent( function renderConfigContent(
template: 'opencode-json', template: 'opencode-json',
endpoint: CustomModelEndpoint, endpoint: CustomModelEndpoint,
modelId: string, modelId: string,
apiKey: string apiKey: string,
contextLength?: number
): string { ): string {
switch (template) { switch (template) {
case 'opencode-json': case 'opencode-json': {
// `limit` carries the discovered context window (issue #749): without it opencode assumes
// a large default window for an unrecognized model id and eventually overflows a much
// smaller local server. `context` is the FULL window and opencode reserves for the reply
// inside it itself, so `opencodeModelLimit` passes the real total through and only
// `output` gets the capped reservation.
const limit = opencodeModelLimit(contextLength);
return JSON.stringify({ return JSON.stringify({
$schema: 'https://opencode.ai/config.json', $schema: 'https://opencode.ai/config.json',
provider: { provider: {
custom: { custom: {
options: { baseURL: withV1Suffix(endpoint.baseUrl), apiKey }, options: { baseURL: withV1Suffix(endpoint.baseUrl), apiKey },
models: { [modelId]: {} }, models: { [modelId]: limit ? { limit } : {} },
}, },
}, },
model: `custom/${modelId}`, model: `custom/${modelId}`,
}); });
}
} }
} }
@@ -195,7 +219,8 @@ function renderConfigFile(
template: 'codex-toml' | 'pi-models-json' | 'omp-models-yml' | 'grok-toml', template: 'codex-toml' | 'pi-models-json' | 'omp-models-yml' | 'grok-toml',
endpoint: CustomModelEndpoint, endpoint: CustomModelEndpoint,
modelId: string, modelId: string,
apiKey: string apiKey: string,
contextLength?: number
): { content: string; extraEnv?: Record<string, string> } { ): { content: string; extraEnv?: Record<string, string> } {
const baseUrl = withV1Suffix(endpoint.baseUrl); const baseUrl = withV1Suffix(endpoint.baseUrl);
switch (template) { switch (template) {
@@ -217,6 +242,12 @@ function renderConfigFile(
const content = [ const content = [
`model = ${quoted(modelId)}`, `model = ${quoted(modelId)}`,
`model_provider = "custom"`, `model_provider = "custom"`,
...(contextLength !== undefined && Number.isFinite(contextLength) && contextLength > 0
? [
`model_context_window = ${Math.trunc(contextLength)}`,
`model_auto_compact_token_limit = ${Math.max(1, Math.floor(contextLength * 0.75))}`,
]
: []),
'', '',
'[model_providers.custom]', '[model_providers.custom]',
`name = "Custom Endpoint"`, `name = "Custom Endpoint"`,
+2 -1
View File
@@ -2724,7 +2724,8 @@ export class Session extends EventEmitter {
/** /**
* Force the custom-model selection's `launchModel` (pi/omp `custom/<id>`, grok's * Force the custom-model selection's `launchModel` (pi/omp `custom/<id>`, grok's
* `[model.<name>]` block name) onto the CLI's `model` launch param. Where that param * `[model.<name>]` block name, codex's bare selected id) onto the CLI's `model`
* launch param. Where that param
* lives is registry DATA — the entry's `legacyConfigField` (`piConfig`, `grokConfig`, * lives is registry DATA — the entry's `legacyConfigField` (`piConfig`, `grokConfig`,
* ...) or the top-level `model` for an entry that declares none — so this stays a * ...) or the top-level `model` for an entry that declares none — so this stays a
* generic reader rather than a branch per CLI. Applied on the OPTIONS only: the stored * generic reader rather than a branch per CLI. Applied on the OPTIONS only: the stored
+6
View File
@@ -615,6 +615,9 @@
'API key': 'API 密钥', 'API key': 'API 密钥',
'Optional. Left blank on edit keeps the existing key.': '可选。编辑时留空将保留现有密钥。', 'Optional. Left blank on edit keeps the existing key.': '可选。编辑时留空将保留现有密钥。',
'Auth header': '认证请求头', 'Auth header': '认证请求头',
'Default context length (tokens)': '默认上下文长度(token 数)',
"Used for models whose size cannot be discovered. The model's whole window; the Claude floor check divides it by any stored slot count. Leave empty to let each CLI assume its own default.":
'用于无法自动发现上下文大小的模型。这是模型的完整窗口;Claude 下限检查会将其除以已存储的槽位数。留空则让各个 CLI 使用其自身的默认值。',
'Never send both — some servers hang indefinitely.': '切勿同时发送两者 — 部分服务器会因此无限期挂起。', 'Never send both — some servers hang indefinitely.': '切勿同时发送两者 — 部分服务器会因此无限期挂起。',
'Authorization: Bearer (default)': 'Authorization: Bearer(默认)', 'Authorization: Bearer (default)': 'Authorization: Bearer(默认)',
'api-key header (Azure)': 'api-key 请求头(Azure)', 'api-key header (Azure)': 'api-key 请求头(Azure)',
@@ -627,6 +630,9 @@
'Last used': '上次使用', 'Last used': '上次使用',
'That endpoint no longer exists': '该端点已不存在', 'That endpoint no longer exists': '该端点已不存在',
'No models discovered for this endpoint yet': '此端点尚未发现任何模型', 'No models discovered for this endpoint yet': '此端点尚未发现任何模型',
// Model picker search box (index.html placeholder/aria-label and the no-matches line)
'Search models': '搜索模型',
'No models match': '没有匹配的模型',
'Subagent Options': '子智能体选项', 'Subagent Options': '子智能体选项',
'Enable Tracking': '启用跟踪', 'Enable Tracking': '启用跟踪',
'Active Tab Only': '仅活动标签页', 'Active Tab Only': '仅活动标签页',
+12
View File
@@ -1033,6 +1033,14 @@
</div> </div>
<div class="modal-body"> <div class="modal-body">
<p class="form-hint" id="customModelPickHint"></p> <p class="form-hint" id="customModelPickHint"></p>
<!-- Case-insensitive filter over the rendered rows (app.filterCustomModelPick):
a display layer only, the rows themselves are never re-rendered. Reset and
focused on every open, same in-memory rule as the rail's search box. -->
<input type="search" id="customModelPickSearch" class="session-sidebar-filter-input custom-model-pick-search"
placeholder="Search models" aria-label="Search models"
autocomplete="off" autocapitalize="off" spellcheck="false" enterkeyhint="search"
oninput="app.filterCustomModelPick(this.value)">
<p class="form-hint custom-model-pick-no-matches" id="customModelPickNoMatches" role="status" hidden>No models match</p>
<div id="customModelPickList" class="run-mode-custom-models"></div> <div id="customModelPickList" class="run-mode-custom-models"></div>
</div> </div>
</div> </div>
@@ -2791,6 +2799,10 @@
<div class="set-row-text"><span class="set-row-label">Base URL</span></div> <div class="set-row-text"><span class="set-row-label">Base URL</span></div>
<input type="text" id="customModelHostBaseUrl" class="set-input" placeholder="http://192.168.1.50:8080"> <input type="text" id="customModelHostBaseUrl" class="set-input" placeholder="http://192.168.1.50:8080">
</div> </div>
<div class="set-row has-field">
<div class="set-row-text"><span class="set-row-label">Default context length (tokens)</span><span class="set-row-desc">Used for models whose size cannot be discovered. The model's whole window; the Claude floor check divides it by any stored slot count. Leave empty to let each CLI assume its own default.</span></div>
<input type="number" id="customModelHostDefaultContextLength" class="set-input" min="1" max="100000000" step="1" placeholder="131072">
</div>
<div class="set-row has-field"> <div class="set-row has-field">
<div class="set-row-text"><span class="set-row-label">API key</span><span class="set-row-desc">Optional. Left blank on edit keeps the existing key.</span></div> <div class="set-row-text"><span class="set-row-label">API key</span><span class="set-row-desc">Optional. Left blank on edit keeps the existing key.</span></div>
<input type="password" id="customModelHostApiKey" class="set-input" autocomplete="new-password"> <input type="password" id="customModelHostApiKey" class="set-input" autocomplete="new-password">
+46 -1
View File
@@ -1088,13 +1088,58 @@ Object.assign(CodemanApp.prototype, {
const promotion = m === currentlyLoaded ? 'Currently loaded' : m === lastUsed ? 'Last used' : null; const promotion = m === currentlyLoaded ? 'Currently loaded' : m === lastUsed ? 'Last used' : null;
const tags = [promotion, m === host.defaultModelId ? 'Default' : null].filter(Boolean); const tags = [promotion, m === host.defaultModelId ? 'Default' : null].filter(Boolean);
const arg = escapeHtml(JSON.stringify(m)); const arg = escapeHtml(JSON.stringify(m));
// data-model-id is the filter's haystack (escapeHtml covers `"` as well as
// `<>&`, so a hostile model id cannot break out of the attribute); the
// filter reads it back through dataset, which decodes it.
return ` return `
<button class="run-mode-option" onclick="app.chooseCustomModelAndRun(${arg})"> <button class="run-mode-option" data-model-id="${escapeHtml(m)}" onclick="app.chooseCustomModelAndRun(${arg})">
<span class="run-mode-dot ${escapeHtml(mode)}"></span>${escapeHtml(m)}${tags.map((t) => ` <span class="set-scope">${escapeHtml(t)}</span>`).join('')} <span class="run-mode-dot ${escapeHtml(mode)}"></span>${escapeHtml(m)}${tags.map((t) => ` <span class="set-scope">${escapeHtml(t)}</span>`).join('')}
</button>`; </button>`;
}) })
.join(''); .join('');
modal.classList.add('active'); modal.classList.add('active');
// A fresh open always starts unfiltered with the caret in the search box — the
// list was just rebuilt, so clearing the input is the whole reset; focus is the
// user's next keystroke away. Inside the generation guard above, so a
// superseded probe can neither focus nor clear a modal that now shows another
// endpoint's models.
const search = document.getElementById('customModelPickSearch');
if (search) {
search.value = '';
// readmymind-ui.js's rule: on touch devices never focus an input unprompted, or
// the OS keyboard pops over the very list the picker just opened to show. Desktop
// keeps the caret in the box, one keystroke from filtering.
if (!(typeof MobileDetection !== 'undefined' && MobileDetection.isTouchDevice())) {
search.focus();
}
}
this.filterCustomModelPick('');
},
/**
* View filter for the open "Choose a model" picker: case-insensitive substring
* over each row's model id, driven from the search input's own `oninput`.
* Deliberately toggles `hidden` on the already-rendered buttons rather than
* re-rendering from a second copy of the model list: the loaded/last-used
* promotion, the Default pill, the escaped inline click argument and the async
* generation guard all then stay exactly where they are, and clearing the
* query restores the list byte for byte. ⚠️ `.run-mode-option` carries its own
* `display: flex` (styles.css, restated in mobile.css), which outranks the UA
* `[hidden]` rule, so styles.css re-asserts `display: none !important` for
* these rows — without it the filter would silently do nothing.
*/
filterCustomModelPick(value) {
const list = document.getElementById('customModelPickList');
if (!list) return;
const query = String(value || '').trim().toLowerCase();
let visible = 0;
for (const btn of list.querySelectorAll('button[data-model-id]')) {
const match = !query || btn.dataset.modelId.toLowerCase().includes(query);
btn.hidden = !match;
if (match) visible += 1;
}
const noMatches = document.getElementById('customModelPickNoMatches');
if (noMatches) noMatches.hidden = visible > 0;
}, },
closeCustomModelPickModal() { closeCustomModelPickModal() {
+16
View File
@@ -3875,6 +3875,9 @@ Object.assign(CodemanApp.prototype, {
document.getElementById('customModelHostApiKey').value = ''; // the server never returns the real value (apiKeySet is a bool) document.getElementById('customModelHostApiKey').value = ''; // the server never returns the real value (apiKeySet is a bool)
document.getElementById('customModelHostApiKey').placeholder = host?.apiKeySet ? '•••••••• (unchanged if left blank)' : ''; document.getElementById('customModelHostApiKey').placeholder = host?.apiKeySet ? '•••••••• (unchanged if left blank)' : '';
document.getElementById('customModelHostAuthStyle').value = host?.authStyle || 'bearer'; document.getElementById('customModelHostAuthStyle').value = host?.authStyle || 'bearer';
// User-set fallback, round-tripped verbatim (discovery never touches it, so the
// cached copy is authoritative); blank means "no default", cleared on save.
document.getElementById('customModelHostDefaultContextLength').value = host?.defaultContextLength ?? '';
this._populateCustomModelDefaultSelect(host); this._populateCustomModelDefaultSelect(host);
document.getElementById('customModelHostEditor').style.display = ''; document.getElementById('customModelHostEditor').style.display = '';
}, },
@@ -3900,6 +3903,18 @@ Object.assign(CodemanApp.prototype, {
const baseUrl = document.getElementById('customModelHostBaseUrl').value.trim(); const baseUrl = document.getElementById('customModelHostBaseUrl').value.trim();
const apiKeyInput = document.getElementById('customModelHostApiKey').value; const apiKeyInput = document.getElementById('customModelHostApiKey').value;
const authStyle = document.getElementById('customModelHostAuthStyle').value; const authStyle = document.getElementById('customModelHostAuthStyle').value;
// Blank means "no default" (key omitted, so PUT clears it); a value must satisfy
// the same bound the server schema enforces, caught here for a friendlier message.
const defaultContextLengthInput = document.getElementById('customModelHostDefaultContextLength').value.trim();
let defaultContextLength;
if (defaultContextLengthInput !== '') {
const parsed = Number(defaultContextLengthInput);
if (!Number.isInteger(parsed) || parsed < 1 || parsed > 100000000) {
this.showToast('Default context length must be a whole number of tokens between 1 and 100,000,000', 'warning');
return;
}
defaultContextLength = parsed;
}
const defaultModelId = document.getElementById('customModelHostDefaultModel').value || undefined; const defaultModelId = document.getElementById('customModelHostDefaultModel').value || undefined;
if (!id || !label || !baseUrl) { if (!id || !label || !baseUrl) {
this.showToast('Id, label and base URL are all required', 'warning'); this.showToast('Id, label and base URL are all required', 'warning');
@@ -3920,6 +3935,7 @@ Object.assign(CodemanApp.prototype, {
baseUrl, baseUrl,
authStyle, authStyle,
defaultModelId, defaultModelId,
defaultContextLength,
apiKey: apiKeyInput || undefined, apiKey: apiKeyInput || undefined,
models: existing?.models, models: existing?.models,
lastDiscoveredAt: existing?.lastDiscoveredAt, lastDiscoveredAt: existing?.lastDiscoveredAt,
+35
View File
@@ -7814,6 +7814,41 @@ html[data-skin="og"] .welcome-primary:hover {
opacity: 0.8; opacity: 0.8;
} }
/* The picker's search box reuses .session-sidebar-filter-input's theme-token
styling (same reasoning as the .set-scope pill above) — only its own spacing
and the no-matches line are added here. 0.35rem is the standard gap the
modal's neighbouring form fields already use (`.form-hint`'s own margin-top):
the global `* { margin: 0 }` reset left the box sitting flush under the CLI
hint line above it, and the old 0.5rem read as too big a gap above the list
(compounded by the always-visible no-matches line the rule below fixes). */
#customModelPickModal .custom-model-pick-search {
margin-top: 0.35rem;
margin-bottom: 0.35rem;
}
#customModelPickModal .custom-model-pick-no-matches {
text-align: center;
padding: 1rem 0;
}
/* Same class of bug as the row rule below: the no-matches line carries
`.form-hint`, whose `display: block` outranks the UA `[hidden]` rule, so
`filterCustomModelPick()` toggling `hidden` changed nothing on screen and
"No models match" showed while every model was listed with an empty search
box. The `[hidden]` re-assertion makes the attribute hide the line again. */
#customModelPickModal .custom-model-pick-no-matches[hidden] {
display: none !important;
}
/* A filtered-out row's `hidden` must actually hide it: `.run-mode-option` sets
`display: flex` (and mobile.css restates the class in a media query), which
outranks the UA `[hidden] { display: none }` rule, so the filter would
silently do nothing without this re-assertion. Scoped to this modal's rows;
the Run menu's own .run-mode-option entries never take `hidden`. */
#customModelPickModal .run-mode-option[hidden] {
display: none !important;
}
/* Custom Model Endpoint Profiles: llama-swap model-swap confirmation — replaces a native /* Custom Model Endpoint Profiles: llama-swap model-swap confirmation — replaces a native
confirm() popup (docs/custom-model-endpoints-plan.md) so it looks and feels like the confirm() popup (docs/custom-model-endpoints-plan.md) so it looks and feels like the
rest of the app instead of a browser chrome dialog. Shares the context-window-too-small rest of the app instead of a browser chrome dialog. Shares the context-window-too-small
+64 -9
View File
@@ -21,7 +21,7 @@ import { isMultiUserMode } from '../../config/multiuser.js';
import { getDataDir } from '../../config/instance.js'; import { getDataDir } from '../../config/instance.js';
import { isBlockedWebviewUrl } from '../webview-egress-policy.js'; import { isBlockedWebviewUrl } from '../webview-egress-policy.js';
import { egressBlockedReason, webviewFetch } from '../webview-egress.js'; import { egressBlockedReason, webviewFetch } from '../webview-egress.js';
import { CustomModelHostSchema } from '../schemas.js'; import { CustomModelHostSchema, MAX_DISCOVERED_CONTEXT_TOKENS } from '../schemas.js';
import { readCustomModelHosts, writeCustomModelHosts, type CustomModelHost } from '../../custom-model-hosts.js'; import { readCustomModelHosts, writeCustomModelHosts, type CustomModelHost } from '../../custom-model-hosts.js';
import type { CliEntry } from '../../config/cli-registry/types.js'; import type { CliEntry } from '../../config/cli-registry/types.js';
@@ -158,9 +158,14 @@ function authHeaders(host: Pick<CustomModelHost, 'apiKey' | 'authStyle'>): Recor
export interface DiscoveryResult { export interface DiscoveryResult {
models: string[]; models: string[];
/** See `CustomModelHost.modelContextLengths` — only ever populated for models already loaded. */ /**
* See `CustomModelHost.modelContextLengths`: populated from a model's own `/v1/models`
* entry (`context_length`, else `context_window`) whenever the server states one, and
* from the launch command or `/props` for loaded models, which overwrite (and so win
* over) any listed figure for the same model.
*/
contextLengths: Record<string, number>; contextLengths: Record<string, number>;
/** See `CustomModelHost.modelSlotCounts`; populated on the same loaded-models-only basis as `contextLengths`. */ /** See `CustomModelHost.modelSlotCounts`; populated only for models the server reports loaded (launch command or `/props`). No listing states a slot count, unlike `contextLengths`. */
slotCounts: Record<string, number>; slotCounts: Record<string, number>;
/** See `CustomModelHost.modelSizesGB` — populated for every model whose own listing states one. */ /** See `CustomModelHost.modelSizesGB` — populated for every model whose own listing states one. */
sizesGB: Record<string, number>; sizesGB: Record<string, number>;
@@ -182,6 +187,20 @@ function parseSizeGB(description: unknown): number | undefined {
return Number.isFinite(size) && size > 0 ? size : undefined; return Number.isFinite(size) && size > 0 ? size : undefined;
} }
/**
* Best-effort: reads a context length (tokens) stated by a model's OWN entry in
* `GET /v1/models`. OpenRouter-style servers call the field `context_length`; current
* llama-swap sends both `context_length` and `context_window` per model, and current
* Bifrost (a gateway in front of llama-swap) passes `context_window` through. Older
* servers send neither, and a server that states nothing gets no entry, never a guess.
* Non-numeric, zero, negative, fractional or over-the-ceiling values are ignored, the
* same rule the stored field's schema enforces: a wrong figure here is worse than none.
*/
function parseListedContextLength(value: unknown): number | undefined {
if (typeof value !== 'number' || !Number.isInteger(value) || value <= 0) return undefined;
return value <= MAX_DISCOVERED_CONTEXT_TOKENS ? value : undefined;
}
/** /**
* Best-effort: fetches `GET /props?model=<id>` (llama.cpp-native, llama-swap-proxied) for * Best-effort: fetches `GET /props?model=<id>` (llama.cpp-native, llama-swap-proxied) for
* ONE already-loaded model and pulls its `n_ctx` and `total_slots` out. Never called for a model that * ONE already-loaded model and pulls its `n_ctx` and `total_slots` out. Never called for a model that
@@ -312,25 +331,42 @@ async function discoverModels(
}); });
if (!res.ok) throw new Error(`HTTP ${res.status}`); if (!res.ok) throw new Error(`HTTP ${res.status}`);
const body = (await res.json()) as { const body = (await res.json()) as {
data?: Array<{ id?: unknown; status?: { value?: unknown }; description?: unknown }>; data?: Array<{
id?: unknown;
status?: { value?: unknown };
description?: unknown;
context_length?: unknown;
context_window?: unknown;
}>;
}; };
const entries = body.data ?? []; const entries = body.data ?? [];
const models = entries.map((m) => m.id).filter((id): id is string => typeof id === 'string' && id.length > 0); const models = entries.map((m) => m.id).filter((id): id is string => typeof id === 'string' && id.length > 0);
const sizesGB: Record<string, number> = {}; const sizesGB: Record<string, number> = {};
// The lowest-precedence context source, and the only one a gateway without /running or
// /props of its own (e.g. Bifrost) can offer: the figure a model's own listing states
// (`context_length`, else `context_window`). The loaded-model pass below OVERWRITES
// these for whatever it reads from the launch command or /props, so precedence per
// model stays: `/running` cmd, then `/props`, then this listed figure.
const contextLengths: Record<string, number> = {};
for (const entry of entries) { for (const entry of entries) {
if (typeof entry.id !== 'string' || !entry.id) continue; if (typeof entry.id !== 'string' || !entry.id) continue;
const size = parseSizeGB(entry.description); const size = parseSizeGB(entry.description);
if (size !== undefined) sizesGB[entry.id] = size; if (size !== undefined) sizesGB[entry.id] = size;
const listed = parseListedContextLength(entry.context_length) ?? parseListedContextLength(entry.context_window);
if (listed !== undefined) contextLengths[entry.id] = listed;
} }
// llama-swap-specific, feature-detected: a server that never mentions `status` on ANY // llama-swap-specific, feature-detected: a server that never mentions `status` on ANY
// entry gets no context-length enrichment at all, rather than treating "no status field" // entry gets no launch-command/props context enrichment at all, rather than treating
// as "assume unloaded" — either reading is a guess, and skipping is the safe one, since // "no status field" as "assume unloaded"; either reading is a guess, and skipping is
// fetchContextLength must only ever run against a model this server itself calls loaded. // the safe one, since fetchContextLength must only ever run against a model this server
// itself calls loaded. Listed figures above are unaffected: reading them triggers nothing.
const hasStatusField = entries.some((m) => m && typeof m === 'object' && 'status' in m); const hasStatusField = entries.some((m) => m && typeof m === 'object' && 'status' in m);
const contextLengths: Record<string, number> = {};
const slotCounts: Record<string, number> = {}; const slotCounts: Record<string, number> = {};
// Ids whose figure THIS round's loaded-model probe answered — the listed figure remains
// the source only for the ids NOT in this set (see the per-slot correction below).
const ctxFromProbe = new Set<string>();
if (hasStatusField) { if (hasStatusField) {
const loadedIds = entries const loadedIds = entries
.filter((m) => m.status && typeof m.status === 'object' && (m.status as { value?: unknown }).value === 'loaded') .filter((m) => m.status && typeof m.status === 'object' && (m.status as { value?: unknown }).value === 'loaded')
@@ -364,11 +400,30 @@ async function discoverModels(
// merges onto what was learned before). // merges onto what was learned before).
if (cmdSlots !== undefined && ctxFromProps) slots = 1; if (cmdSlots !== undefined && ctxFromProps) slots = 1;
} }
if (ctx !== undefined) contextLengths[id] = ctx; // Assigning only when a probe answered keeps the listed figure (if any) when
// neither the launch command nor /props stated one, and replaces it with the
// stronger source whenever they did: cmd > /props > listed, never the reverse.
if (ctx !== undefined) {
contextLengths[id] = ctx;
ctxFromProbe.add(id);
}
if (slots !== undefined) slotCounts[id] = slots; if (slots !== undefined) slotCounts[id] = slots;
} }
} }
} }
// llama-swap documents the LISTED `context_length` as the PER-SLOT window, so whenever
// the listed figure is the one taken this round — the model was not loaded, or the
// probe answered nothing — its slot count is 1 by definition: the floor check must NOT
// divide a per-slot figure a second time. Without recording 1 here, the count a loaded
// round stored survives the merge (applyDiscoveredModels keeps what a later round does
// not overwrite): 262144 @ 4 slots stored, model unloaded, listed 65536 → 65536 with 4
// slots → a false 16384-per-slot warning. Even a count this round's launch command
// parsed must yield — pairing a per-slot listed figure with the cmd's --parallel is the
// same double division. Same rule the /props branch above applies. Ids whose figure the
// probe DID answer keep their probe-derived count untouched.
for (const id of Object.keys(contextLengths)) {
if (!ctxFromProbe.has(id)) slotCounts[id] = 1;
}
return { models, contextLengths, slotCounts, sizesGB }; return { models, contextLengths, slotCounts, sizesGB };
} }
+28 -15
View File
@@ -70,8 +70,9 @@ import {
SessionWaitOutputQuerySchema, SessionWaitOutputQuerySchema,
CustomModelSelectionSchema, CustomModelSelectionSchema,
} from '../schemas.js'; } from '../schemas.js';
import { readCustomModelHosts } from '../../custom-model-hosts.js'; import { readCustomModelHosts, resolveCustomModelContextLength } from '../../custom-model-hosts.js';
import { applyCustomModelInjection, removeConfigDir } from '../../custom-model-injection-apply.js'; import { applyCustomModelInjection, removeConfigDir } from '../../custom-model-injection-apply.js';
import { buildCustomModelInjection } from '../../custom-model-injection.js';
import { import {
getLlamaSwapStatus, getLlamaSwapStatus,
triggerLlamaSwapLoad, triggerLlamaSwapLoad,
@@ -1541,7 +1542,10 @@ export function registerSessionRoutes(
if (!endpoint) { if (!endpoint) {
return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found'); return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
} }
const contextLength = endpoint.modelContextLengths?.[body.modelId]; // Discovery's per-model figure wins; the endpoint's user-set default covers models
// whose size discovery could not determine (a gateway that states none, say). With
// neither, the existing "no override, CLI assumes its own default" behaviour stands.
const contextLength = resolveCustomModelContextLength(endpoint, body.modelId);
const slotCount = endpoint.modelSlotCounts?.[body.modelId]; const slotCount = endpoint.modelSlotCounts?.[body.modelId];
// Some CLIs (today: only claude) carry enough of their own fixed system-prompt/tool- // Some CLIs (today: only claude) carry enough of their own fixed system-prompt/tool-
@@ -1608,10 +1612,28 @@ export function registerSessionRoutes(
} }
// A CLI whose config alone cannot select the model also gets its `model` launch param // A CLI whose config alone cannot select the model also gets its `model` launch param
// forced (pi/omp `custom/<id>`, grok's block name). The argv engine DROPS a token that // forced (pi/omp `custom/<id>`, grok's block name, and codex now carries its bare
// selected id too). The argv engine DROPS a token that
// fails its pattern rather than quoting it, which would silently launch the CLI on its // fails its pattern rather than quoting it, which would silently launch the CLI on its
// own default provider again, so refuse an id the pattern cannot carry up front. // own default provider again, so refuse an id the pattern cannot carry up front.
// The check runs on the PURE build result, before applyCustomModelInjection below:
// the forced launch id only comes out of that build, and writing first would mean a
// refused id already replaced the config dir a live codex session uses as CODEX_HOME.
const modelSpec = entry.launch.params.model; const modelSpec = entry.launch.params.model;
const injection = buildCustomModelInjection(entry, endpoint, body.modelId, contextLength);
if (injection.kind === 'unsupported') {
return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `${session.mode} has no known custom-model mechanism`);
}
if (
injection.launchModel !== undefined &&
modelSpec?.type === 'token' &&
!matchesPattern(modelSpec.pattern, injection.launchModel)
) {
return createErrorResponse(
ApiErrorCode.INVALID_INPUT,
`Model id ${JSON.stringify(body.modelId)} cannot be passed to ${session.mode} on its command line`
);
}
const applied = applyCustomModelInjection( const applied = applyCustomModelInjection(
entry, entry,
endpoint, endpoint,
@@ -1623,17 +1645,6 @@ export function registerSessionRoutes(
if (!applied) { if (!applied) {
return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `${session.mode} has no known custom-model mechanism`); return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `${session.mode} has no known custom-model mechanism`);
} }
if (
applied.launchModel !== undefined &&
modelSpec?.type === 'token' &&
!matchesPattern(modelSpec.pattern, applied.launchModel)
) {
removeConfigDir(applied.configDir);
return createErrorResponse(
ApiErrorCode.INVALID_INPUT,
`Model id ${JSON.stringify(body.modelId)} cannot be passed to ${session.mode} on its command line`
);
}
const { previousConfigDir } = session.setCustomModel( const { previousConfigDir } = session.setCustomModel(
{ {
@@ -4447,7 +4458,9 @@ export function registerSessionRoutes(
const cmHosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR); const cmHosts = await readCustomModelHosts(CODEMAN_CONFIG_DIR);
const cmEndpoint = cmHosts.find((h) => h.id === customModel.endpointId); const cmEndpoint = cmHosts.find((h) => h.id === customModel.endpointId);
if (!cmEndpoint) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found'); if (!cmEndpoint) return createErrorResponse(ApiErrorCode.NOT_FOUND, 'Model endpoint not found');
const cmContextLength = cmEndpoint.modelContextLengths?.[customModel.modelId]; // Per-model discovery figure first, user-set endpoint default second, nothing third
// (see the dedicated route's own comment above).
const cmContextLength = resolveCustomModelContextLength(cmEndpoint, customModel.modelId);
const cmSlotCount = cmEndpoint.modelSlotCounts?.[customModel.modelId]; const cmSlotCount = cmEndpoint.modelSlotCounts?.[customModel.modelId];
// See the dedicated route's own comment for the full reasoning: some CLIs' own fixed // See the dedicated route's own comment for the full reasoning: some CLIs' own fixed
+14 -1
View File
@@ -2387,6 +2387,14 @@ export const WebviewUpdateSchema = WebviewBaseSchema.partial();
/** POST /api/webviews/probe: reachability + framing check for the editor's Test button. */ /** POST /api/webviews/probe: reachability + framing check for the editor's Test button. */
export const WebviewProbeSchema = z.object({ url: webviewUrlSchema }); export const WebviewProbeSchema = z.object({ url: webviewUrlSchema });
/**
* Ceiling on a stated context length (tokens), shared between the bounds below and
* discovery (`custom-model-routes.ts`): anything above it reads as nonsense and is
* ignored, because a bogus figure here overflows the smaller server, which is the exact
* failure the discovery field exists to prevent.
*/
export const MAX_DISCOVERED_CONTEXT_TOKENS = 100_000_000;
// Custom Model Endpoint Profiles (docs/custom-model-endpoints-plan.md) — a // Custom Model Endpoint Profiles (docs/custom-model-endpoints-plan.md) — a
// user-configured custom OpenAI-compatible endpoint, local (llama.cpp) or cloud // user-configured custom OpenAI-compatible endpoint, local (llama.cpp) or cloud
// (Azure AI Foundry, etc.). Lives below `webviewUrlSchema` because `baseUrl` IS that // (Azure AI Foundry, etc.). Lives below `webviewUrlSchema` because `baseUrl` IS that
@@ -2410,11 +2418,16 @@ export const CustomModelHostSchema = z.object({
defaultModelId: z.string().max(200).optional(), defaultModelId: z.string().max(200).optional(),
// Server-populated by discovery (custom-model-routes.ts); accepted here only so a client // Server-populated by discovery (custom-model-routes.ts); accepted here only so a client
// round-tripping the GET response back through PUT (edit-save) doesn't drop it. // round-tripping the GET response back through PUT (edit-save) doesn't drop it.
modelContextLengths: z.record(z.string().max(200), z.number().int().positive().max(100_000_000)).optional(), modelContextLengths: z
.record(z.string().max(200), z.number().int().positive().max(MAX_DISCOVERED_CONTEXT_TOKENS))
.optional(),
// Same reasoning as modelContextLengths above. // Same reasoning as modelContextLengths above.
modelSlotCounts: z.record(z.string().max(200), z.number().int().positive().max(4096)).optional(), modelSlotCounts: z.record(z.string().max(200), z.number().int().positive().max(4096)).optional(),
// Same reasoning as modelContextLengths above. // Same reasoning as modelContextLengths above.
modelSizesGB: z.record(z.string().max(200), z.number().positive().max(100_000)).optional(), modelSizesGB: z.record(z.string().max(200), z.number().positive().max(100_000)).optional(),
// User-set fallback for models discovery has no context length for (session-routes.ts);
// discovery never writes it. Same bound as the modelContextLengths values above.
defaultContextLength: z.number().int().positive().max(MAX_DISCOVERED_CONTEXT_TOKENS).optional(),
}); });
/** /**
+2 -2
View File
@@ -80,7 +80,7 @@ import { attachmentRegistry, buildFileThumbnailRoute, registerExternalAttachment
import { folderGrants } from '../folder-listing.js'; import { folderGrants } from '../folder-listing.js';
import { getCli, enabledClis, listClis } from '../config/cli-registry/registry.js'; import { getCli, enabledClis, listClis } from '../config/cli-registry/registry.js';
import { isCliEntryInstalled, probeStockCliAvailability } from '../utils/cli-installed-probes.js'; import { isCliEntryInstalled, probeStockCliAvailability } from '../utils/cli-installed-probes.js';
import { readCustomModelHosts } from '../custom-model-hosts.js'; import { readCustomModelHosts, resolveCustomModelContextLength } from '../custom-model-hosts.js';
import { applyCustomModelInjection, customModelConfigDir, removeConfigDir } from '../custom-model-injection-apply.js'; import { applyCustomModelInjection, customModelConfigDir, removeConfigDir } from '../custom-model-injection-apply.js';
import type { CustomModelBookkeeping } from '../types/session.js'; import type { CustomModelBookkeeping } from '../types/session.js';
import { registerGeneratedArtifactAttachment } from '../generated-artifact-attachments.js'; import { registerGeneratedArtifactAttachment } from '../generated-artifact-attachments.js';
@@ -1470,7 +1470,7 @@ export class WebServer extends EventEmitter {
endpoint, endpoint,
saved.modelId, saved.modelId,
session.id, session.id,
endpoint.modelContextLengths?.[saved.modelId] resolveCustomModelContextLength(endpoint, saved.modelId)
)?.envOverrides; )?.envOverrides;
} catch (err) { } catch (err) {
console.warn('[WebServer] Failed to rebuild custom-model env on recovery:', err); console.warn('[WebServer] Failed to rebuild custom-model env on recovery:', err);
+9
View File
@@ -171,6 +171,15 @@ describe('codex', () => {
expect(cx({ model: 'gpt-5', resumeSessionId: 'roll_42' })).toBe('codex --model gpt-5 resume roll_42'); expect(cx({ model: 'gpt-5', resumeSessionId: 'roll_42' })).toBe('codex --model gpt-5 resume roll_42');
}); });
it('carries an Ollama-style `qwen3:32b` model id, and still drops a leading-flag shape', () => {
// `model-plain-colon`: the custom-model feature forces the selected id onto this param,
// and ids like `qwen3:32b` carry a colon. The leading-alphanumeric rule is what keeps a
// flag from riding the flag, so `:x` and `-x` must still drop out.
expect(cx({ model: 'qwen3:32b' })).toBe('codex --model qwen3:32b');
expect(cx({ model: ':leading' })).toBe('codex');
expect(cx({ model: '-yolo' })).toBe('codex');
});
it('sends reasoning effort as one model_reasoning_effort config value, for every level', () => { it('sends reasoning effort as one model_reasoning_effort config value, for every level', () => {
for (const level of CODEX_REASONING_EFFORTS) { for (const level of CODEX_REASONING_EFFORTS) {
expect(cx({ reasoningEffort: level })).toBe(`codex --config model_reasoning_effort=${level}`); expect(cx({ reasoningEffort: level })).toBe(`codex --config model_reasoning_effort=${level}`);
@@ -481,6 +481,52 @@ describe('refreshAllCustomModelHosts: parallel-slot enrichment (llama.cpp splits
const [updated] = await readCustomModelHosts(dir); const [updated] = await readCustomModelHosts(dir);
expect(updated.modelSlotCounts).toEqual({ a: 4 }); expect(updated.modelSlotCounts).toEqual({ a: 4 });
}); });
it('a round where only the listed figure answers resets the slot count a loaded round learned', async () => {
// The false-warning scenario, measured on a real llama-swap: round 1 has the model
// loaded under `--ctx-size 262144 --parallel 4` — a TOTAL window shared across slots,
// correctly stored with its count. Round 2 finds it unloaded and takes the LISTED
// `context_length`, which llama-swap documents as the PER-SLOT window. If the stored 4
// survived, the floor check would divide the per-slot figure a second time
// (65536/4 = 16384) and warn about a model that serves 65536 to every request.
const dir = getDataDir();
await writeCustomModelHosts(dir, [host({ id: 'ep', baseUrl: 'http://localhost:8080' })]);
fetchMock.mockImplementation(async (url: URL) => {
if (url.pathname === '/v1/models') {
return new Response(JSON.stringify({ data: [{ id: 'm', status: { value: 'loaded' } }] }), { status: 200 });
}
if (url.pathname === '/running') {
return new Response(
JSON.stringify({
running: [{ model: 'm', state: 'ready', cmd: 'llama-server -m /models/m.gguf --ctx-size 262144 --parallel 4' }],
}),
{ status: 200 }
);
}
throw new Error(`unexpected request: ${url.href}`); // the command answers both questions
});
await refreshAllCustomModelHosts();
let [round1] = await readCustomModelHosts(dir);
expect(round1.modelContextLengths).toEqual({ m: 262144 });
expect(round1.modelSlotCounts).toEqual({ m: 4 });
// Round 2: unloaded, no /running or /props probe in order, listing its per-slot figure.
fetchMock.mockResolvedValue(
new Response(JSON.stringify({ data: [{ id: 'm', status: { value: 'unloaded' }, context_length: 65536 }] }), {
status: 200,
})
);
await refreshAllCustomModelHosts();
const [round2] = await readCustomModelHosts(dir);
expect(round2.modelContextLengths).toEqual({ m: 65536 });
// 1 slot, not the stale 4: 65536 per slot with 1 recorded slot is the honest pair, so
// the floor check reads 65536/request — exactly what the server gives a single request.
expect(round2.modelSlotCounts).toEqual({ m: 1 });
});
}); });
describe('refreshAllCustomModelHosts: model-size enrichment (parsed from /v1/models description)', () => { describe('refreshAllCustomModelHosts: model-size enrichment (parsed from /v1/models description)', () => {
+53 -1
View File
@@ -8,6 +8,7 @@
*/ */
import { describe, it, expect } from 'vitest'; import { describe, it, expect } from 'vitest';
import { parse as parseToml } from 'smol-toml';
import { getCli } from '../src/config/cli-registry/index.js'; import { getCli } from '../src/config/cli-registry/index.js';
import { import {
buildCustomModelInjection, buildCustomModelInjection,
@@ -112,6 +113,31 @@ describe('buildCustomModelInjection', () => {
expect(parsed.provider.custom.models.qwen3).toEqual({}); expect(parsed.provider.custom.models.qwen3).toEqual({});
}); });
it('opencode: threads a discovered context length into the model limit as the full window (issue #749)', () => {
const result = buildCustomModelInjection(entryOrThrow('opencode'), endpoint, 'qwen3', 16384);
if (result.kind !== 'env') throw new Error('unreachable');
const parsed = JSON.parse(result.envOverrides.OPENCODE_CONFIG_CONTENT);
// `limit.context` IS the full window to opencode — it reserves for the reply itself —
// so the discovered total passes through; `output` only caps the reply at a quarter.
expect(parsed.provider.custom.models.qwen3).toEqual({ limit: { context: 16384, output: 4096 } });
});
it('opencode: caps the response headroom reservation at 8192 for big windows', () => {
const result = buildCustomModelInjection(entryOrThrow('opencode'), endpoint, 'qwen3', 262144);
if (result.kind !== 'env') throw new Error('unreachable');
const parsed = JSON.parse(result.envOverrides.OPENCODE_CONFIG_CONTENT);
expect(parsed.provider.custom.models.qwen3).toEqual({ limit: { context: 262144, output: 8192 } });
});
it('opencode: never invents a limit for an unknown or non-finite context length (issue #749 regression)', () => {
for (const contextLength of [undefined, NaN]) {
const result = buildCustomModelInjection(entryOrThrow('opencode'), endpoint, 'qwen3', contextLength);
if (result.kind !== 'env') throw new Error('unreachable');
const parsed = JSON.parse(result.envOverrides.OPENCODE_CONFIG_CONTENT);
expect(parsed.provider.custom.models.qwen3).toEqual({});
}
});
it('codex: configDir writes an isolated config.toml with model/base_url, and the key rides as extraEnv (never a literal TOML field)', () => { it('codex: configDir writes an isolated config.toml with model/base_url, and the key rides as extraEnv (never a literal TOML field)', () => {
const result = buildCustomModelInjection(entryOrThrow('codex'), endpoint, 'qwen3'); const result = buildCustomModelInjection(entryOrThrow('codex'), endpoint, 'qwen3');
expect(result.kind).toBe('configDir'); expect(result.kind).toBe('configDir');
@@ -133,6 +159,30 @@ describe('buildCustomModelInjection', () => {
expect(result.files[0].content).toContain('model = "weird\\"model"'); expect(result.files[0].content).toContain('model = "weird\\"model"');
}); });
it('codex: writes the discovered window as TOP-LEVEL model_context_window with a 75% compact limit (issue #750)', () => {
const result = buildCustomModelInjection(entryOrThrow('codex'), endpoint, 'qwen3', 16384);
if (result.kind !== 'configDir') throw new Error('unreachable');
const doc = parseToml(result.files[0].content) as Record<string, unknown>;
// Top-level is the shape codex reads: parked under [model_providers.custom] (its
// provider table) the same keys would be silently ignored by codex itself.
expect(doc.model_context_window).toBe(16384);
expect(doc.model_auto_compact_token_limit).toBe(12288); // 75% of the window, floored
const provider = (doc.model_providers as Record<string, Record<string, unknown>>).custom;
expect(provider.model_context_window).toBeUndefined();
expect(provider.model_auto_compact_token_limit).toBeUndefined();
});
it('codex: writes neither context line for an unknown, non-finite or zero context length', () => {
for (const contextLength of [undefined, NaN, 0]) {
const result = buildCustomModelInjection(entryOrThrow('codex'), endpoint, 'qwen3', contextLength);
if (result.kind !== 'configDir') throw new Error('unreachable');
const doc = parseToml(result.files[0].content) as Record<string, unknown>;
expect(doc.model_context_window, String(contextLength)).toBeUndefined();
expect(doc.model_auto_compact_token_limit, String(contextLength)).toBeUndefined();
expect(doc.model).toBe('qwen3'); // the rest of the file is still written
}
});
it('pi: configDir writes .pi/agent/models.json, redirected via HOME (verified live — PI_CONFIG_DIR does nothing for pi)', () => { it('pi: configDir writes .pi/agent/models.json, redirected via HOME (verified live — PI_CONFIG_DIR does nothing for pi)', () => {
const result = buildCustomModelInjection(entryOrThrow('pi'), endpoint, 'qwen3'); const result = buildCustomModelInjection(entryOrThrow('pi'), endpoint, 'qwen3');
if (result.kind !== 'configDir') throw new Error('unreachable'); if (result.kind !== 'configDir') throw new Error('unreachable');
@@ -232,7 +282,9 @@ describe('launchModel (the model launch param that selects the injected provider
}); });
it('CLIs whose config selects the model on its own declare none', () => { it('CLIs whose config selects the model on its own declare none', () => {
for (const id of ['claude', 'opencode', 'codex', 'gemini', 'deepseek']) { // codex now declares launchModel (selected-model launch override) and is covered by
// its own restart coverage, not here.
for (const id of ['claude', 'opencode', 'gemini', 'deepseek']) {
const result = buildCustomModelInjection(entryOrThrow(id), endpoint, 'qwen3'); const result = buildCustomModelInjection(entryOrThrow(id), endpoint, 'qwen3');
if (result.kind === 'unsupported') throw new Error('unreachable'); if (result.kind === 'unsupported') throw new Error('unreachable');
expect(result.launchModel, id).toBeUndefined(); expect(result.launchModel, id).toBeUndefined();
+91
View File
@@ -0,0 +1,91 @@
/**
* @fileoverview Boot recovery rebuilds a saved custom-model selection through the SAME
* context-length resolution the routes use: the model's discovered figure, falling back to
* the endpoint's `defaultContextLength` (`resolveCustomModelContextLength`). Before the
* shared helper, `_rebuildCustomModelEnv` read `modelContextLengths` alone — so a session
* on a model that only the endpoint default covers came back from a Codeman restart with
* `CLAUDE_CODE_MAX_CONTEXT_TOKENS` (or a config file's context figure) silently gone,
* re-arming exactly the window overflow the var exists to prevent.
*
* `_rebuildCustomModelEnv` reads no `this` state (it re-derives everything from the endpoint
* store and the Session it is handed), so it is driven straight off the prototype with an
* empty receiver — `restoreMuxSessions()` itself cannot be reached under vitest, the same
* constraint recorded in test/session-model-recovery.test.ts.
*/
import { afterEach, describe, expect, it } from 'vitest';
import { getDataDir } from '../src/config/instance.js';
import { writeCustomModelHosts, type CustomModelHost } from '../src/custom-model-hosts.js';
import { customModelConfigDir, removeConfigDir } from '../src/custom-model-injection-apply.js';
import { Session } from '../src/session.js';
import { WebServer } from '../src/web/server.js';
import type { CustomModelBookkeeping } from '../src/types/session.js';
type RebuildFn = (session: Session, saved: CustomModelBookkeeping) => Promise<Record<string, string> | undefined>;
/** The real method, with an empty receiver — see the file header for why that is sound. */
function rebuildCustomModelEnv(session: Session, saved: CustomModelBookkeeping) {
const fn = (WebServer.prototype as unknown as { _rebuildCustomModelEnv: RebuildFn })._rebuildCustomModelEnv;
expect(typeof fn, '_rebuildCustomModelEnv vanished from WebServer.prototype').toBe('function');
return fn.call({}, session, saved);
}
function endpoint(overrides: Partial<CustomModelHost> = {}): CustomModelHost {
return {
id: 'ep',
label: 'Ep',
baseUrl: 'http://localhost:8080',
apiKey: 'recovery-probe-key',
models: ['qwen3'],
...overrides,
};
}
describe('_rebuildCustomModelEnv: context-length fallback on boot recovery', () => {
const configs: string[] = [];
afterEach(() => {
for (const id of configs.splice(0)) removeConfigDir(customModelConfigDir(id));
});
it('falls back to the endpoint defaultContextLength when discovery recorded no per-model figure', async () => {
await writeCustomModelHosts(getDataDir(), [endpoint({ defaultContextLength: 32768 })]);
const session = new Session({ workingDir: '/tmp', mode: 'claude' });
configs.push(session.id);
const env = await rebuildCustomModelEnv(session, { endpointId: 'ep', modelId: 'qwen3', envKeys: [] });
expect(env?.CLAUDE_CODE_MAX_CONTEXT_TOKENS).toBe('32768');
});
it('a discovered per-model figure still wins over the endpoint default', async () => {
await writeCustomModelHosts(getDataDir(), [
endpoint({ modelContextLengths: { qwen3: 131072 }, defaultContextLength: 32768 }),
]);
const session = new Session({ workingDir: '/tmp', mode: 'claude' });
configs.push(session.id);
const env = await rebuildCustomModelEnv(session, { endpointId: 'ep', modelId: 'qwen3', envKeys: [] });
expect(env?.CLAUDE_CODE_MAX_CONTEXT_TOKENS).toBe('131072');
});
it('states nothing rather than guessing when neither source knows a window', async () => {
await writeCustomModelHosts(getDataDir(), [endpoint()]);
const session = new Session({ workingDir: '/tmp', mode: 'claude' });
configs.push(session.id);
const env = await rebuildCustomModelEnv(session, { endpointId: 'ep', modelId: 'qwen3', envKeys: [] });
expect(env?.CLAUDE_CODE_MAX_CONTEXT_TOKENS).toBeUndefined();
});
it('drops the selection to undefined (kept for clearing) when the endpoint no longer exists', async () => {
await writeCustomModelHosts(getDataDir(), []);
const session = new Session({ workingDir: '/tmp', mode: 'claude' });
configs.push(session.id);
await expect(rebuildCustomModelEnv(session, { endpointId: 'gone', modelId: 'qwen3', envKeys: [] })).resolves.toBe(
undefined
);
});
});
+173
View File
@@ -20,6 +20,10 @@ import { describe, expect, it, vi } from 'vitest';
const CONSTANTS_JS = readFileSync(new URL('../src/web/public/constants.js', import.meta.url), 'utf-8'); const CONSTANTS_JS = readFileSync(new URL('../src/web/public/constants.js', import.meta.url), 'utf-8');
const SESSION_UI_JS = readFileSync(new URL('../src/web/public/session-ui.js', import.meta.url), 'utf-8'); const SESSION_UI_JS = readFileSync(new URL('../src/web/public/session-ui.js', import.meta.url), 'utf-8');
// Only for the [hidden] CSS guard below — the filter toggles the `hidden` property, and
// `.run-mode-option`'s own `display: flex` would silently defeat it without the
// `!important` re-assertion in styles.css.
const STYLES_CSS = readFileSync(new URL('../src/web/public/styles.css', import.meta.url), 'utf-8');
// Only for the real _showCenterStatus DOM tests below (`bootAppWithRealCenterStatus`) — // Only for the real _showCenterStatus DOM tests below (`bootAppWithRealCenterStatus`) —
// every other test in this file stubs _showCenterStatus itself and has no need of it. // every other test in this file stubs _showCenterStatus itself and has no need of it.
const PANELS_UI_JS = readFileSync(new URL('../src/web/public/panels-ui.js', import.meta.url), 'utf-8'); const PANELS_UI_JS = readFileSync(new URL('../src/web/public/panels-ui.js', import.meta.url), 'utf-8');
@@ -55,6 +59,9 @@ function bootApp(
<div class="modal" id="customModelPickModal"> <div class="modal" id="customModelPickModal">
<h3 id="customModelPickTitle"></h3> <h3 id="customModelPickTitle"></h3>
<p id="customModelPickHint"></p> <p id="customModelPickHint"></p>
<input type="search" id="customModelPickSearch" aria-label="Search models"
oninput="app.filterCustomModelPick(this.value)">
<p id="customModelPickNoMatches" role="status" hidden>No models match</p>
<div id="customModelPickList"></div> <div id="customModelPickList"></div>
</div> </div>
<div class="modal" id="customModelSwapConfirmModal"> <div class="modal" id="customModelSwapConfirmModal">
@@ -532,6 +539,172 @@ describe('Custom Model Endpoint Profiles: the "which model" picker', () => {
await app.selectCustomModelEntry('claude', 'llama-box'); await app.selectCustomModelEntry('claude', 'llama-box');
expect(toastMessage).toMatch(/no models discovered/i); expect(toastMessage).toMatch(/no models discovered/i);
}); });
it('resets and focuses the search box every time the picker opens', async () => {
const { win, app } = bootApp({
hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: ['qwen3', 'llama3'] }],
});
win.app = app;
await app.selectCustomModelEntry('claude', 'llama-box');
const search = win.document.getElementById('customModelPickSearch') as HTMLInputElement;
// The caret lands in the box, so the first keystroke filters with no click.
expect(win.document.activeElement).toBe(search);
// A stale query from a previous open must not survive into the next one.
search.value = 'zzz';
search.dispatchEvent(new win.Event('input'));
app.closeCustomModelPickModal();
await app.selectCustomModelEntry('claude', 'llama-box');
expect(search.value).toBe('');
expect(win.document.activeElement).toBe(search);
expect(win.document.getElementById('customModelPickNoMatches')!.hasAttribute('hidden')).toBe(true);
const visible = [...win.document.getElementById('customModelPickList')!.querySelectorAll('button')].filter(
(b) => !b.hasAttribute('hidden')
);
expect(visible).toHaveLength(2);
});
it('leaves the search box unfocused on touch devices so the OS keyboard stays off the list', async () => {
const { win, app } = bootApp({
hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: ['qwen3', 'llama3'] }],
});
win.app = app;
// The guard reads the global the browser defines (mobile-detection.js).
(win as unknown as { MobileDetection: { isTouchDevice: () => boolean } }).MobileDetection = {
isTouchDevice: () => true,
};
await app.selectCustomModelEntry('claude', 'llama-box');
const search = win.document.getElementById('customModelPickSearch') as HTMLInputElement;
expect(win.document.activeElement).not.toBe(search);
// Only the focus is skipped: the rows still render, unfiltered and functional.
const visible = [...win.document.getElementById('customModelPickList')!.querySelectorAll('button')].filter(
(b) => !b.hasAttribute('hidden')
);
expect(visible).toHaveLength(2);
});
it('filters rows case-insensitively on the model id, keeping the surviving row fully functional', async () => {
const { win, app } = bootApp({
hosts: [
{
id: 'llama-box',
label: 'llama.cpp',
baseUrl: 'http://x',
models: ['qwen3', 'llama3'],
defaultModelId: 'qwen3',
},
],
});
let launched: unknown[] | null = null;
app.runCustomModelEntry = async (...args: unknown[]) => {
launched = args;
};
win.app = app;
await app.selectCustomModelEntry('claude', 'llama-box');
const search = win.document.getElementById('customModelPickSearch') as HTMLInputElement;
search.value = 'QWEN';
search.dispatchEvent(new win.Event('input'));
const buttons = [...win.document.getElementById('customModelPickList')!.querySelectorAll('button')];
expect(buttons.filter((b) => b.hasAttribute('hidden')).map((b) => b.textContent!.trim())).toEqual(['llama3']);
const [visible] = buttons.filter((b) => !b.hasAttribute('hidden'));
// The surviving row keeps its Default badge and its real, callable click handler.
expect(visible.textContent).toContain('qwen3');
expect(visible.textContent).toContain('Default');
(visible as unknown as HTMLButtonElement & { onclick: (e: unknown) => void }).onclick(
new (win as any).Event('click')
);
expect(launched).toEqual(['claude', 'llama-box', 'qwen3']);
});
it('shows the no-matches line only while a query matches nothing, and clears it with the query', async () => {
const { win, app } = bootApp({
hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: ['qwen3', 'llama3'] }],
});
win.app = app;
await app.selectCustomModelEntry('claude', 'llama-box');
const search = win.document.getElementById('customModelPickSearch') as HTMLInputElement;
const noMatches = win.document.getElementById('customModelPickNoMatches')!;
const isHidden = () =>
win.document.getElementById('customModelPickList')!.querySelectorAll('button[hidden]').length;
// Whitespace is not a query: everything stays visible, no message.
search.value = ' ';
search.dispatchEvent(new win.Event('input'));
expect(noMatches.hasAttribute('hidden')).toBe(true);
expect(isHidden()).toBe(0);
search.value = 'ghost';
search.dispatchEvent(new win.Event('input'));
expect(noMatches.hasAttribute('hidden')).toBe(false);
expect(isHidden()).toBe(2);
search.value = '';
search.dispatchEvent(new win.Event('input'));
expect(noMatches.hasAttribute('hidden')).toBe(true);
expect(isHidden()).toBe(0);
});
it('keeps an HTML-significant model id inert in the picker rows too, and filterable by its raw text', async () => {
// Same vector as the Run-menu row above, now via data-model-id: escapeHtml covers
// the double quote, and the filter reads the decoded raw id back off dataset.
const dangerousModel = '"><img src=x onerror=alert(1)>';
const { win, app } = bootApp({
hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: ['qwen3', dangerousModel] }],
});
win.app = app;
await app.selectCustomModelEntry('claude', 'llama-box');
const list = win.document.getElementById('customModelPickList')!;
expect(list.querySelector('img')).toBeNull();
const search = win.document.getElementById('customModelPickSearch') as HTMLInputElement;
search.value = 'src=x';
search.dispatchEvent(new win.Event('input'));
const visible = list.querySelectorAll('button:not([hidden])');
expect(visible).toHaveLength(1);
expect(visible[0].getAttribute('data-model-id')).toBe(dangerousModel);
});
it('re-asserts display:none !important for hidden picker rows so the filter cannot be silently defeated', () => {
// `.run-mode-option` carries its own `display: flex` (styles.css, restated in
// mobile.css), which outranks the UA `[hidden]` rule — without this scoped
// !important re-assertion, toggling `hidden` would change nothing on screen.
const rule = STYLES_CSS.match(/#customModelPickModal \.run-mode-option\[hidden\]\s*\{[^}]*\}/);
expect(rule, 'picker rows need a [hidden] rule scoped to #customModelPickModal').toBeTruthy();
expect(rule![0]).toMatch(/display:\s*none\s*!important/);
});
it('pins "No models match" hidden with an empty query and shown only when a query hides every row, in CSS too', async () => {
// The visible symptom: the line carries `.form-hint`, whose `display: block`
// outranks the UA `[hidden]` rule, so `hidden` alone showed "No models match"
// while every model was listed with an empty search box. The attribute half
// was always right (session-ui.js's `filterCustomModelPick`); the CSS half
// needed the same `[hidden] { display: none !important }` re-assertion the
// rows above carry. This test pins both states AND the CSS that makes them
// actually render.
const rule = STYLES_CSS.match(/#customModelPickModal \.custom-model-pick-no-matches\[hidden\]\s*\{[^}]*\}/);
expect(rule, 'the no-matches line needs a [hidden] rule scoped to #customModelPickModal').toBeTruthy();
expect(rule![0]).toMatch(/display:\s*none\s*!important/);
const { win, app } = bootApp({
hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: ['qwen3', 'llama3'] }],
});
win.app = app;
await app.selectCustomModelEntry('claude', 'llama-box');
const noMatches = win.document.getElementById('customModelPickNoMatches')!;
// Empty query, models listed: the attribute must be set, and the CSS rule
// above is what makes that attribute actually hide the line.
expect(noMatches.hasAttribute('hidden')).toBe(true);
// A non-empty query that hides every row: shown again.
const search = win.document.getElementById('customModelPickSearch') as HTMLInputElement;
search.value = 'ghost';
search.dispatchEvent(new win.Event('input'));
expect(noMatches.hasAttribute('hidden')).toBe(false);
});
}); });
describe('Custom Model Endpoint Profiles: _getCustomModelCurrentlyLoaded is client-side bounded', () => { describe('Custom Model Endpoint Profiles: _getCustomModelCurrentlyLoaded is client-side bounded', () => {
+164 -1
View File
@@ -11,7 +11,7 @@
import { describe, it, expect, vi, afterEach } from 'vitest'; import { describe, it, expect, vi, afterEach } from 'vitest';
import { registerCustomModelRoutes } from '../../src/web/routes/custom-model-routes.js'; import { registerCustomModelRoutes } from '../../src/web/routes/custom-model-routes.js';
import { webviewFetch } from '../../src/web/webview-egress.js'; import { webviewFetch } from '../../src/web/webview-egress.js';
import { createRouteTestHarness } from './_route-test-utils.js'; import { createRouteTestHarness, type RouteTestHarness } from './_route-test-utils.js';
vi.mock('../../src/web/webview-egress.js', async () => { vi.mock('../../src/web/webview-egress.js', async () => {
const actual = await vi.importActual<typeof import('../../src/web/webview-egress.js')>( const actual = await vi.importActual<typeof import('../../src/web/webview-egress.js')>(
@@ -417,3 +417,166 @@ describe('apiKey is never handed back to the browser', () => {
expect(fetchMock).toHaveBeenCalledTimes(1); expect(fetchMock).toHaveBeenCalledTimes(1);
}); });
}); });
describe('context-length discovery from the /v1/models listing', () => {
// The gateway path (a Bifrost in front of llama-swap, an OpenRouter-style server):
// no /running, no /props, just a per-model `context_length` or `context_window` in the
// model listing. Distinct ports per test keep the file's shared data dir unambiguous.
afterEach(() => {
fetchMock.mockReset();
});
type StoredHost = { id: string; modelContextLengths?: Record<string, number>; defaultContextLength?: number };
type HarnessApp = RouteTestHarness['app'];
async function endpointWith(id: string, port: number, extra: Record<string, unknown> = {}): Promise<HarnessApp> {
const { app } = await setup();
const res = await app.inject({
method: 'POST',
url: '/api/model-endpoints',
payload: { id, label: 'A', baseUrl: `http://localhost:${port}`, ...extra },
});
expect(res.statusCode).toBe(200);
return app;
}
async function storedHost(app: HarnessApp, id: string): Promise<StoredHost | undefined> {
const list = await app.inject({ method: 'GET', url: '/api/model-endpoints' });
return (list.json() as StoredHost[]).find((h) => h.id === id);
}
it('reads context_length (preferred) then context_window from each entry into modelContextLengths', async () => {
const app = await endpointWith('ep-ctx-listed', 8101);
fetchMock.mockResolvedValue(
new Response(
JSON.stringify({
data: [
{ id: 'both', context_length: 131072, context_window: 32768 },
{ id: 'window-only', context_window: 8192 },
{ id: 'no-figure' },
],
}),
{ status: 200 }
)
);
const discover = await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-ctx-listed/discover-models' });
expect(discover.json().success).toBe(true);
const stored = await storedHost(app, 'ep-ctx-listed');
expect(stored?.modelContextLengths).toEqual({ both: 131072, 'window-only': 8192 });
});
it('ignores non-numeric, zero, negative and absurd listed values; the schema bound itself is still accepted', async () => {
const app = await endpointWith('ep-ctx-bad', 8102);
fetchMock.mockResolvedValue(
new Response(
JSON.stringify({
data: [
{ id: 'string', context_length: '131072' },
{ id: 'zero', context_window: 0 },
{ id: 'negative', context_length: -10 },
{ id: 'over-bound', context_length: 100_000_001 },
{ id: 'fractional', context_window: 16384.5 },
{ id: 'at-bound', context_length: 100_000_000 },
],
}),
{ status: 200 }
)
);
await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-ctx-bad/discover-models' });
const stored = await storedHost(app, 'ep-ctx-bad');
expect(stored?.modelContextLengths).toEqual({ 'at-bound': 100_000_000 });
});
it('prefers the /running launch command over the listed figure for a loaded model', async () => {
const app = await endpointWith('ep-ctx-cmd', 8103);
fetchMock.mockImplementation(async (url: URL) => {
if (url.pathname === '/v1/models') {
return new Response(
JSON.stringify({ data: [{ id: 'cmd-wins', status: { value: 'loaded' }, context_length: 131072 }] }),
{ status: 200 }
);
}
if (url.pathname === '/running') {
return new Response(
JSON.stringify({ running: [{ model: 'cmd-wins', state: 'ready', cmd: 'llama-server --ctx-size 16384' }] }),
{ status: 200 }
);
}
if (url.pathname === '/props') return new Response('nope', { status: 404 });
throw new Error(`unexpected request in this test: ${url.href}`);
});
await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-ctx-cmd/discover-models' });
const stored = await storedHost(app, 'ep-ctx-cmd');
expect(stored?.modelContextLengths).toEqual({ 'cmd-wins': 16384 });
});
it('keeps a previously learned value the fresh listing says nothing about, and discovery never touches defaultContextLength', async () => {
const app = await endpointWith('ep-ctx-keep', 8104, {
models: ['re-listed', 'untouched'],
modelContextLengths: { 're-listed': 16384, untouched: 4096 },
defaultContextLength: 65536,
});
fetchMock.mockResolvedValue(
new Response(
JSON.stringify({ data: [{ id: 're-listed', context_length: 32768 }, { id: 'untouched' }] }),
{ status: 200 }
)
);
await app.inject({ method: 'POST', url: '/api/model-endpoints/ep-ctx-keep/discover-models' });
const stored = await storedHost(app, 'ep-ctx-keep');
expect(stored?.modelContextLengths).toEqual({ 're-listed': 32768, untouched: 4096 });
expect(stored?.defaultContextLength).toBe(65536);
});
});
describe('defaultContextLength (the user-set fallback discovery never overwrites)', () => {
afterEach(() => {
fetchMock.mockReset();
});
it('stores it on create, updates it on PUT, and a PUT without it clears it', async () => {
const { app } = await setup();
const create = await app.inject({
method: 'POST',
url: '/api/model-endpoints',
payload: { id: 'ep-defctx', label: 'A', baseUrl: 'http://localhost:8105', defaultContextLength: 131072 },
});
expect(create.json().data.host.defaultContextLength).toBe(131072);
const update = await app.inject({
method: 'PUT',
url: '/api/model-endpoints/ep-defctx',
payload: { label: 'A', baseUrl: 'http://localhost:8105', defaultContextLength: 32768 },
});
expect(update.json().data.host.defaultContextLength).toBe(32768);
const cleared = await app.inject({
method: 'PUT',
url: '/api/model-endpoints/ep-defctx',
payload: { label: 'A', baseUrl: 'http://localhost:8105' },
});
expect(cleared.json().data.host.defaultContextLength).toBeUndefined();
});
it('rejects zero, negative, fractional, over-the-bound and non-numeric values', async () => {
const { app } = await setup();
for (const defaultContextLength of [0, -1, 1.5, 100_000_001, '131072']) {
const res = await app.inject({
method: 'POST',
url: '/api/model-endpoints',
payload: { id: 'ep-defctx-bad', label: 'A', baseUrl: 'http://localhost:8106', defaultContextLength },
});
expect(res.json().success, String(defaultContextLength)).toBe(false);
expect(res.json().errorCode, String(defaultContextLength)).toBe('INVALID_INPUT');
}
});
});
@@ -122,6 +122,9 @@ describe('POST /api/quick-start: customModel (one-shot custom-model launch)', ()
const { sessionId } = res.json(); const { sessionId } = res.json();
const session = ctx.sessions.get(sessionId) as unknown as Session; const session = ctx.sessions.get(sessionId) as unknown as Session;
expect(session.customModel?.endpointId).toBe('ep1'); expect(session.customModel?.endpointId).toBe('ep1');
// #750, pinned the same way as the pi/grok/omp one-shot paths below: the FIRST
// launch already carries the selected id, not just the file on disk.
expect(session.getCustomModelForPersist()?.launchModel).toBe('qwen3');
expect(restartSpy).not.toHaveBeenCalled(); expect(restartSpy).not.toHaveBeenCalled();
const configDir = customModelConfigDir(sessionId); const configDir = customModelConfigDir(sessionId);
@@ -367,6 +370,50 @@ describe('POST /api/quick-start: customModel (one-shot custom-model launch)', ()
expect(res.statusCode).toBe(200); expect(res.statusCode).toBe(200);
expect(res.json().requiresContextWarning).toBeUndefined(); expect(res.json().requiresContextWarning).toBeUndefined();
}); });
it('warns using the endpoint defaultContextLength when discovery recorded no per-model figure', async () => {
const endpoint: CustomModelHost = {
id: 'ep-default',
label: 'silent gateway',
baseUrl: 'http://192.168.1.53:8080',
apiKey: 'k',
defaultContextLength: 16384,
};
await writeCustomModelHosts(getDataDir(), [ENDPOINT, endpoint]);
const res = await quickStart({
caseName: 'cm-default-ctx',
mode: 'claude',
customModel: { endpointId: 'ep-default', modelId: 'some-unknown-model' },
});
const body = res.json();
expect(body.requiresContextWarning).toBe(true);
expect(body.contextLength).toBe(16384);
expect(ctx.sessions.size).toBe(1); // nothing was created
});
it('a discovered per-model figure wins over the endpoint defaultContextLength', async () => {
const endpoint: CustomModelHost = {
id: 'ep-precedence',
label: 'mixed box',
baseUrl: 'http://192.168.1.54:8080',
apiKey: 'k',
modelContextLengths: { m: 131072 },
defaultContextLength: 16384,
};
await writeCustomModelHosts(getDataDir(), [ENDPOINT, endpoint]);
const res = await quickStart({
caseName: 'cm-precedence',
mode: 'claude',
customModel: { endpointId: 'ep-precedence', modelId: 'm' },
});
expect(res.statusCode).toBe(200);
expect(res.json().requiresContextWarning).toBeUndefined();
expect(ctx.sessions.size).toBe(2);
});
}); });
describe('triggering the actual llama-swap load (not just watching for it)', () => { describe('triggering the actual llama-swap load (not just watching for it)', () => {
+96 -3
View File
@@ -9,7 +9,7 @@ import { createRouteTestHarness } from './_route-test-utils.js';
import { createMockSession } from '../mocks/index.js'; import { createMockSession } from '../mocks/index.js';
import { getDataDir } from '../../src/config/instance.js'; import { getDataDir } from '../../src/config/instance.js';
import { writeCustomModelHosts, type CustomModelHost } from '../../src/custom-model-hosts.js'; import { writeCustomModelHosts, type CustomModelHost } from '../../src/custom-model-hosts.js';
import { existsSync, readFileSync, statSync } from 'node:fs'; import { existsSync, readFileSync, rmSync, statSync } from 'node:fs';
import { join } from 'node:path'; import { join } from 'node:path';
import { webviewFetch } from '../../src/web/webview-egress.js'; import { webviewFetch } from '../../src/web/webview-egress.js';
@@ -203,6 +203,10 @@ describe('POST /api/sessions/:id/custom-model', () => {
const { app, ctx } = await setup(); const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!; const session = ctx.sessions.get('test-session-1')!;
session.mode = 'pi'; session.mode = 'pi';
// Start from no dir at all (a sibling test's successful pi apply may have left one),
// so the existsSync below says something about THIS request.
const dir = join(getDataDir(), 'custom-model-configs', 'test-session-1');
rmSync(dir, { recursive: true, force: true });
const res = await app.inject({ const res = await app.inject({
method: 'POST', method: 'POST',
@@ -214,8 +218,45 @@ describe('POST /api/sessions/:id/custom-model', () => {
expect(res.json().errorCode).toBe('INVALID_INPUT'); expect(res.json().errorCode).toBe('INVALID_INPUT');
expect(session.setCustomModel).not.toHaveBeenCalled(); expect(session.setCustomModel).not.toHaveBeenCalled();
expect(session.restartCli).not.toHaveBeenCalled(); expect(session.restartCli).not.toHaveBeenCalled();
// The config dir written before the check is cleaned up again. // The pattern check runs on the pure build BEFORE any write, so a refused id leaves
expect(existsSync(join(getDataDir(), 'custom-model-configs', 'test-session-1'))).toBe(false); // no directory behind at all, nothing to clean up.
expect(existsSync(dir)).toBe(false);
});
it('codex: a refused forced launch id never touches the live config dir the session runs on as CODEX_HOME', async () => {
const { app, ctx } = await setup();
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'codex';
// First apply succeeds: the session now runs codex on qwen3:32b, whose config.toml
// lives in the per-session dir the running CLI itself uses as CODEX_HOME.
const ok = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep1', modelId: 'qwen3:32b' },
});
expect(ok.statusCode).toBe(200);
const dir = join(getDataDir(), 'custom-model-configs', 'test-session-1');
const written = join(dir, 'config.toml');
expect(existsSync(written)).toBe(true);
// `@` is outside codex's `model-plain-colon` pattern. The pattern check now runs on
// the pure build BEFORE any write, so the refusal must leave the dir — and the file
// the live CLI reads — exactly as they were, rather than replacing and deleting the
// CODEX_HOME of a session that stays on qwen3:32b.
const before = readFileSync(written, 'utf8');
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep1', modelId: 'org/model@q4_k_m' },
});
expect(res.json().success).toBe(false);
expect(res.json().errorCode).toBe('INVALID_INPUT');
expect(session.setCustomModel).toHaveBeenCalledTimes(1);
expect(session.restartCli).toHaveBeenCalledTimes(1);
expect(existsSync(dir)).toBe(true);
expect(readFileSync(written, 'utf8')).toBe(before);
}); });
it('clear removes the previous config dir the session reports', async () => { it('clear removes the previous config dir the session reports', async () => {
@@ -449,6 +490,58 @@ describe('POST /api/sessions/:id/custom-model', () => {
expect(session.restartCli).not.toHaveBeenCalled(); expect(session.restartCli).not.toHaveBeenCalled();
}); });
it('the endpoint defaultContextLength stands in when discovery recorded no per-model figure', async () => {
const { app, ctx } = await setup();
// A gateway that states nothing per model: only the user's default is known.
const endpoint: CustomModelHost = {
id: 'ep-default',
label: 'silent gateway',
baseUrl: 'http://192.168.1.53:8080',
apiKey: 'k',
defaultContextLength: 16384,
};
await writeCustomModelHosts(getDataDir(), [CLAUDE_ENDPOINT, endpoint]);
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'claude';
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep-default', modelId: 'some-unknown-model' },
});
const body = res.json();
expect(body.requiresContextWarning).toBe(true);
expect(body.contextLength).toBe(16384);
expect(session.setCustomModel).not.toHaveBeenCalled();
});
it('a discovered per-model figure wins over the endpoint defaultContextLength', async () => {
const { app, ctx } = await setup();
// The default is small enough to warn on; the discovered figure for THIS model is
// not, so the fallback must not leak into the per-model path.
const endpoint: CustomModelHost = {
id: 'ep-precedence',
label: 'mixed box',
baseUrl: 'http://192.168.1.54:8080',
apiKey: 'k',
modelContextLengths: { 'qwen3.8-27b-ud-q4_k_xl': 131072 },
defaultContextLength: 16384,
};
await writeCustomModelHosts(getDataDir(), [CLAUDE_ENDPOINT, endpoint]);
const session = ctx.sessions.get('test-session-1')!;
session.mode = 'claude';
const res = await app.inject({
method: 'POST',
url: '/api/sessions/test-session-1/custom-model',
payload: { endpointId: 'ep-precedence', modelId: 'qwen3.8-27b-ud-q4_k_xl' },
});
expect(res.json().requiresContextWarning).toBeUndefined();
expect(session.setCustomModel).toHaveBeenCalledTimes(1);
});
// llama-server splits its context across --parallel slots, and Claude Code sends several // llama-server splits its context across --parallel slots, and Claude Code sends several
// requests at once, so a context that clears the floor as a whole can still fail per slot. // requests at once, so a context that clears the floor as a whole can still fail per slot.
describe('parallel slots share the context', () => { describe('parallel slots share the context', () => {
+28
View File
@@ -27,6 +27,8 @@ import { join } from 'node:path';
import { afterEach, describe, expect, it, vi } from 'vitest'; import { afterEach, describe, expect, it, vi } from 'vitest';
import { Session } from '../src/session.js'; import { Session } from '../src/session.js';
import { getCli } from '../src/config/cli-registry/index.js';
import { buildCustomModelInjection } from '../src/custom-model-injection.js';
import { TmuxManager } from '../src/tmux-manager.js'; import { TmuxManager } from '../src/tmux-manager.js';
import type { MuxSession, SessionMode } from '../src/types.js'; import type { MuxSession, SessionMode } from '../src/types.js';
@@ -225,3 +227,29 @@ describe('launchModel reaches the CLI through its own launch param', () => {
expect(respawn.mock.calls[0][0].model).toBeUndefined(); expect(respawn.mock.calls[0][0].model).toBeUndefined();
}); });
}); });
describe('Codex custom endpoint model overrides', () => {
it('selects the local model over a native launch default and restores the default on clear', async () => {
const entry = getCli('codex');
if (!entry) throw new Error('Missing Codex entry');
const injection = buildCustomModelInjection(
entry,
{ id: 'ep', label: 'Local', baseUrl: 'http://box:8080' },
'qwen3'
);
if (injection.kind !== 'configDir') throw new Error('Expected isolated Codex config');
const { session, respawn } = liveSession('codex', {
codexConfig: { model: 'native-model', reasoningEffort: 'high' },
});
session.setCustomModel(
{ endpointId: 'ep', modelId: 'qwen3', envKeys: ['CODEX_HOME'], launchModel: injection.launchModel },
{ CODEX_HOME: '/isolated/codex' }
);
await session.restartCli();
expect(respawn.mock.calls[0][0].codexConfig).toEqual({ model: 'qwen3', reasoningEffort: 'high' });
session.setCustomModel(undefined);
respawn.mockClear();
await session.restartCli();
expect(respawn.mock.calls[0][0].codexConfig).toEqual({ model: 'native-model', reasoningEffort: 'high' });
});
});