Merge pull request #752: fix(custom-model): context limits, Codex model override, context discovery and picker filter

docs/api-reference.md

@@ -1205,24 +1205,47 @@ data}` envelope on the wire — unwrap it the same way). Answers `[]` for a
 boolean` reports whether one is stored, so a client can render "unchanged
   if left blank" without ever holding the real value.
 - `POST /api/v1/model-endpoints` with `{ id, label, baseUrl, apiKey?,
-authStyle?, defaultModelId? }` creates one. `id` must match
+authStyle?, defaultModelId?, defaultContextLength? }` creates one. `id`
+  must match
   `^[a-zA-Z0-9_-]+$`; `authStyle` is `bearer` (default) or `api-key`, never
   both (a real server hung indefinitely when sent both headers on one
-  request); `baseUrl` must be `http(s)`, carry no embedded credentials, and
+  request); `defaultContextLength` is a user-set fallback used when
+  assigning a model that discovery has no context length for (discovery
+  itself never writes it); `baseUrl` must be `http(s)`, carry no embedded
+  credentials, and
   is refused if it points at (or resolves to) a link-local or
   cloud-metadata address. `409 ALREADY_EXISTS` on a duplicate id.
-- `PUT /api/v1/model-endpoints/:id` updates one. An **absent** `apiKey`
+- `PUT /api/v1/model-endpoints/:id` updates one, also accepting
+  `defaultContextLength`. An **absent** `apiKey`
   keeps the stored one rather than clearing it — the client never receives
   the real value to resend deliberately unchanged, so omission is the only
   way to say "leave it alone"; there is no way to clear a key back to unset
-  this way. `defaultModelId`, when set, must be one of that endpoint's own
+  this way. `defaultContextLength` behaves the opposite way: a PUT without
+  it clears the stored value, so send the field back to keep it.
+  `defaultModelId`, when set, must be one of that endpoint's own
   `models` (`400 INVALID_INPUT` otherwise).
 - `DELETE /api/v1/model-endpoints/:id` removes one.
 - `POST /api/v1/model-endpoints/:id/discover-models` fetches the endpoint's
   own `GET /v1/models` and stores the result as `models`, updating
-  `lastDiscoveredAt`, plus (best-effort, only for a model llama-swap's own
-  response already reports loaded) `modelContextLengths` and `modelSizesGB`.
-  A `defaultModelId` that no longer appears in the fresh list is dropped
+  `lastDiscoveredAt`, plus (best-effort) `modelContextLengths`,
+  `modelSlotCounts` and `modelSizesGB`: every listing entry is read for a
+  stated context figure (`context_length`, then `context_window`) and a
+  size, BEFORE any loaded check; only the launch-command/`/props`
+  enrichment needs a model llama-swap's own response reports loaded. A
+  loaded model's context figure comes from llama-swap's own launch command
+  (`--fit-ctx` / `-c` / `--ctx-size`), falling back to `/props` (`n_ctx`,
+  `total_slots`) for whatever the command did not answer, and OVERWRITES the
+  listed figure for that model; the model's own listing
+  (`context_length`, then `context_window`) only ever fills the gap when
+  neither probe does, and is recorded as a single-slot figure —
+  llama-swap lists the PER-SLOT window, so whenever the listed figure is
+  the one taken the stored slot count is 1. A server with no `status`
+  field on any entry (plain llama.cpp, a non-llama-swap gateway) gets no
+  launch-command/`/props` enrichment at all, only whatever its listings
+  state. `modelContextLengths` falls back to the
+  endpoint's `defaultContextLength` when a session is assigned a model
+  discovery never learned a window for. A `defaultModelId` that no longer
+  appears in the fresh list is dropped
   rather than carried forward invalid. Failures answer `422 OPERATION_FAILED`
   with the underlying connection error, or a named egress refusal if the
   resolved address turned out to be blocked. The same refresh also runs

docs/architecture-invariants.md

@@ -630,9 +630,9 @@ Adding the header changed the preamble, so `CODEMAN_PREAMBLE` was bumped (1.22.0
 
 **Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`, design doc `docs/custom-model-endpoints-plan.md`; full stack — settings-panel CRUD + the Run-menu picker, on top of the backend below): points a session at a user-configured custom OpenAI-compatible endpoint — local (llama.cpp, DGX Spark, Strix Halo) or cloud (Azure AI Foundry, OpenRouter) — instead of its harness's native cloud backend. Endpoints are a read/write-array store (`custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`) discovered via `GET <baseUrl>/v1/models`; `CustomModelHost.authStyle` is `'bearer'` (default, `Authorization: Bearer`) or `'api-key'` (Azure's convention) — **never both**, live-tested against a real server: sending both headers on one request reliably hangs it indefinitely, reproduced 3×.
 
-⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified.
+⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/copilot/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified.
 
-⚠️ **Two launch paths, chosen by mechanism, not preference**: opencode/codex/gemini/pi/grok/deepseek/omp apply the selection ONE-SHOT, before the session/process ever exists, with no restart at all; claude alone still applies a selection by **restarting the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` relaunches a CLI in an existing pane, so a CLI whose launch declares a `fallback` chain (claude) gets a conversation pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. The dead-pane respawn shares this pin, see [Dead-pane respawn: the resume pin](#dead-pane-respawn-the-resume-pin). ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600.
+⚠️ **Two launch paths, chosen by mechanism, not preference**: opencode/codex/gemini/pi/grok/deepseek/omp apply the selection ONE-SHOT, before the session/process ever exists, with no restart at all; claude alone still applies a selection by **restarting the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` relaunches a CLI in an existing pane, so a CLI whose launch declares a `fallback` chain (claude) gets a conversation pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. The dead-pane respawn shares this pin, see [Dead-pane respawn: the resume pin](#dead-pane-respawn-the-resume-pin). ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name), and codex gets one too (`--model <bare modelId>`, its pattern widened to `model-plain-colon` for qwen3-style ids): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600.
 
 ⚠️ **Security**: every env var this feature can redirect (`ANTHROPIC_BASE_URL`, `GOOGLE_GEMINI_BASE_URL`, `CODEX_HOME`, `GROK_HOME`, `HOME` for pi/omp, `OPENCODE_CONFIG_CONTENT`, etc.) is in that CLI's `privilegedEnvKeys` — several of these were reachable via the generic `envOverrides` field's prefix allowlist BEFORE this feature existed (the env allowlist is global and prefix-based, not per-CLI-scoped), so building this surfaced and closed a pre-existing gap rather than opening a new one. `ANTHROPIC_*` is deliberately NOT in claude's `allowedPrefixes` at all — Anthropic-traffic redirection can only happen through this feature's own admin-configured, SSRF-guarded route, never a plain client-supplied `envOverrides`.
 

docs/custom-model-endpoints.md

@@ -67,15 +67,23 @@ configured, `PUT`/`DELETE /api/model-endpoints/:id` update or remove one.
 Endpoint management is admin-only in multi-user mode, same as remote/docker
 hosts — these are machine-level infra, not per-user settings.
 
-**Context length is discovered too, opportunistically and safely.** The plain
-`GET /v1/models` response has no context-window field. Discovery only ever
-looks for one for a model llama-swap's own response already reports
+**Context length is discovered too, opportunistically and safely.** Older
+servers state no context-window field in the plain `GET /v1/models`
+response at all; current llama-swap states one per model (both
+`context_length` and `context_window`) and gateways like Bifrost pass
+`context_window` through — that listed figure is the *lowest*-precedence
+source, used only when neither probe below answers, and because llama-swap
+documents it as the PER-SLOT window, a model whose figure comes from the
+listing is recorded with a slot count of 1 so the slot check never divides a
+per-slot figure a second time. The probes are only ever looked for on a model
+llama-swap's own response already reports
 `status.value === "loaded"` for — never for an unloaded one, because
 llama-swap treats `?model=` as a routing hint and asking about a model that
 isn't loaded risks triggering an actual (slow, GPU-swapping) load as a side
 effect of what should be read-only discovery. A server with no `status` field
-on any entry at all (not llama-swap) gets no context-length enrichment,
-rather than guessing. A model's previously-learned context length survives a
+on any entry at all (not llama-swap) gets no launch-command/`/props`
+enrichment, only whatever its listings state — rather than guessing. A
+model's previously-learned context length survives a
 later cycle where it wasn't the loaded one; it's dropped only once the model
 disappears from the endpoint's list entirely. Stored per model in
 `modelContextLengths` and applied automatically (see "Applying a model to a
@@ -286,6 +294,15 @@ hot-swap. A Claude session is relaunched with `--resume <conversation> ||
 grok are relaunched with the `--model` value that selects the injected
 provider (`custom/<modelId>` for pi and omp, `codeman-custom` for grok),
 since for those three the config file alone does not switch the model.
+**Codex is relaunched with `--model <modelId>` too**, the bare selected id
+(not a provider selector): the launch param forces the session onto the
+model its generated `config.toml` already selected, so a launch-time model
+carried over from App Settings can never override the injected selection.
+That launch arg must satisfy the codex entry's `model-plain-colon` token
+pattern (a leading letter or digit, then letters, digits and `._-:/`, per
+`stock.ts`), so an id outside that shape, e.g. LM Studio's
+`org/model@q4_k_m`, is refused with a 400 at apply time now, where a
+`config.toml` alone used to carry it.
 **Remote (SSH) and Docker sessions are refused** (400) for now: their restart
 reattaches the durable remote/in-container tmux rather than relaunching the
 agent, so the selection would report success and change nothing.
@@ -293,14 +310,16 @@ agent, so the selection would report success and change nothing.
 **Claude gets two more env vars when known/applicable, both declared on its
 registry entry (`contextLengthVar`/`configDirVar`), not hardcoded here:**
 
-- `CLAUDE_CODE_MAX_CONTEXT_TOKENS` is set to `modelId`'s discovered context
-  length (see the discovery section above) whenever one is known. Without
+- `CLAUDE_CODE_MAX_CONTEXT_TOKENS` is set to `modelId`'s context length —
+  the discovered figure (see the discovery section above), falling back to
+  the endpoint's user-set `defaultContextLength` — whenever one is known.
+  Without
   it, Claude Code assumes a large (200k) window for any unrecognized custom
   model id and never compacts, which reliably overflows a much smaller real
   local context — confirmed live: a stock ~33.7K-token system prompt against
   a 16384-token llama-swap model failed with `exceeds the available context
-size`. No entry for the model in `modelContextLengths` means the var is
-  simply omitted, never a guess. ⚠️ **This var only affects when Claude
+size`. Neither a `modelContextLengths` entry nor a `defaultContextLength`
+  for the model means the var is simply omitted, never a guess. ⚠️ **This var only affects when Claude
   Code compacts conversation _history_ — it cannot fix a model whose real
   context is smaller than Claude Code's own fixed per-turn overhead**
   (system prompt + tool schemas, empirically ~36.4K tokens, confirmed live
@@ -441,15 +460,17 @@ Both apply routes (the restart route and the one-shot `POST
 /api/quick-start`) now check for this **before** launching or restarting
 anything, gated on the CLI's registry entry declaring a `contextLengthVar`
 (currently only claude — the check is a no-op for every other CLI by
-construction, never a hardcoded mode check). If the model's discovered
-context (`modelContextLengths`, from discovery above) is below
-`CLAUDE_MIN_SAFE_CONTEXT_TOKENS` (40000, comfortably above the measured
+construction, never a hardcoded mode check). If the model's context (the
+discovered figure from `modelContextLengths`, or the endpoint's
+`defaultContextLength` when discovery never learned one for that model) is
+below `CLAUDE_MIN_SAFE_CONTEXT_TOKENS` (40000, comfortably above the measured
 ~36.4K overhead), the response is `{requiresContextWarning: true, modelId,
 contextLength, minSafeContextTokens}` (plus `slotCount` when the model runs more than one parallel slot) instead of applying — nothing is
-restarted or created yet. A context length that was never discovered at
-all skips the check entirely (nothing to compare, so it fails open rather
-than warning on every model an endpoint hasn't reported a size for).
-The same warning fires when the context divided across the model's parallel slots (`contextLength / slotCount`) is below the floor, because Claude Code sends several 40K-token requests at once and the slots share one pool. Retrying with `confirmedContext: true` launches anyway (the legacy `confirmed: true` still means both questions).
+restarted or created yet. A model with neither a discovered figure nor a
+`defaultContextLength` skips the check entirely (nothing to compare, so it
+fails open rather than warning on every model an endpoint hasn't reported a
+size for).
+The same warning fires when the context divided across the model's parallel slots (`contextLength / slotCount`) is below the floor, because Claude Code sends several 40K-token requests at once and the slots share one pool. ⚠️ `defaultContextLength` is the model's whole window, never a per-slot figure, so a default small enough to split under the slot count trips that slot division exactly like a discovered figure does. Retrying with `confirmedContext: true` launches anyway (the legacy `confirmed: true` still means both questions).
 
 The Run-menu picker shows this as an in-app modal
 (`#customModelContextWarningModal`, matching the llama-swap conflict
@@ -526,7 +547,12 @@ for '<id>' not found. Defaulting to fallback metadata...` on launch —
   and fabricating a fake catalog entry would mean copying the _shape_ of
   OpenAI's own proprietary schema (their per-model system-prompt content
   included) for a warning that doesn't otherwise affect behavior — not
-  something to build into discovery.
+  something to build into discovery. ⚠️ The fallback-metadata warning is
+  NOT about the context window going un-overridden. Codex does now get a
+  real context-window override: when a figure is known the generated
+  `config.toml` sets the top-level `model_context_window` to it and
+  `model_auto_compact_token_limit` to 75% of it (floored), so Codex
+  compacts at that point instead of trusting its fallback guess.
 - **Gemini** — fails with `Invalid auth method selected`, traced to an
   undocumented `GATEWAY` auth path gemini-cli selects once
   `GOOGLE_GEMINI_BASE_URL` is set. Unresolved after real investigation

docs/wiki/Custom-Model-Endpoints.md

@@ -27,14 +27,20 @@ hosts — these are machine-level infra, not a per-user setting.
 shows up without another manual click of **Discover**. One endpoint being unreachable on a
 given cycle (powered off, wrong network) never blocks the others from refreshing.
 
-**Context length is picked up automatically where it can be, safely.** Against a
-llama.cpp/llama-swap server, discovery also learns each _currently loaded_ model's real
-context window and applies it to the launched session (Claude Code today — see below), so
-the harness stops assuming a large default window for a model name it doesn't recognise and
-overflowing a much smaller real one. It's deliberately never probed for a model that isn't
-already loaded, since asking a llama-swap server about an unloaded model can trigger an
-actual, slow model swap as a side effect — a model just not currently loaded keeps whatever
-context length an earlier cycle already learned for it instead.
+**Context length is picked up automatically where it can be, safely.** Every model's own
+listing entry is read for a stated context figure (`context_length`, then `context_window`,
+which current llama.cpp, llama-swap and gateway-style servers state to varying degrees);
+against a llama-swap server the stronger figures from its launch command and from llama.cpp's
+`/props` probe win where they exist, and those probes only ever run for a model the server
+already reports loaded, since asking about an unloaded model can trigger an actual, slow model
+swap as a side effect. A model that states no figure keeps whatever an earlier cycle learned:
+the stored figure always beats the endpoint's **Default context length** field
+(`modelContextLengths[id] ?? defaultContextLength`), which only applies where no figure was
+ever learned (that number is the model's whole window, never a per-slot figure). A model
+neither covers simply gets no override, never a guess. The window is applied to the launched session by the harnesses
+that can take it: Claude Code today (see below), OpenCode as the model's context limit, and
+Codex as a `model_context_window` entry in its generated config file, which also sets Codex's
+own auto-compact point via `model_auto_compact_token_limit` at 75% of the window.
 
 ## Running a session against one
 
@@ -48,7 +54,9 @@ pointed at.
 
 **Which model it uses depends on how many the endpoint has discovered.** With exactly one,
 the session launches straight away on that model — nothing to choose. With two or more, a
-small dialog asks which one to use for this launch before starting the session; the
+small dialog asks which one to use for this launch before starting the session; a search box
+at the top narrows a long list with a case-insensitive substring match, so a server hosting
+hundreds of models stays usable. The
 endpoint's default model, if set, is marked but not auto-picked, so a launch can deliberately
 use a different one without changing the saved default. The list is not raw discovery order
 either: the model llama-swap reports loaded and ready is moved to the top and tagged