mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-15 02:49:33 +01:00
docs(custom-model): align the docs and comments with the core behaviour
This commit is contained in:
1 parent
d980d70173
commit
82d251d52b
11 files changed
+73
-38
No files matched your search
+11
-7
@@ -1225,18 +1225,22 @@ authStyle?, defaultModelId?, defaultContextLength? }` creates one. `id`
|
||||
- `DELETE /api/v1/model-endpoints/:id` removes one.
|
||||
- `POST /api/v1/model-endpoints/:id/discover-models` fetches the endpoint's
|
||||
own `GET /v1/models` and stores the result as `models`, updating
|
||||
`lastDiscoveredAt`, plus (best-effort, only for a model llama-swap's own
|
||||
response already reports loaded) `modelContextLengths`,
|
||||
`modelSlotCounts` and `modelSizesGB`. A loaded model's context figure
|
||||
comes from llama-swap's own launch command (`--fit-ctx` / `-c` /
|
||||
`--ctx-size`), falling back to `/props` (`n_ctx`, `total_slots`) for
|
||||
whatever the command did not answer; the model's own listing
|
||||
`lastDiscoveredAt`, plus (best-effort) `modelContextLengths`,
|
||||
`modelSlotCounts` and `modelSizesGB`: every listing entry is read for a
|
||||
stated context figure (`context_length`, then `context_window`) and a
|
||||
size, BEFORE any loaded check; only the launch-command/`/props`
|
||||
enrichment needs a model llama-swap's own response reports loaded. A
|
||||
loaded model's context figure comes from llama-swap's own launch command
|
||||
(`--fit-ctx` / `-c` / `--ctx-size`), falling back to `/props` (`n_ctx`,
|
||||
`total_slots`) for whatever the command did not answer, and OVERWRITES the
|
||||
listed figure for that model; the model's own listing
|
||||
(`context_length`, then `context_window`) only ever fills the gap when
|
||||
neither probe does, and is recorded as a single-slot figure —
|
||||
llama-swap lists the PER-SLOT window, so whenever the listed figure is
|
||||
the one taken the stored slot count is 1. A server with no `status`
|
||||
field on any entry (plain llama.cpp, a non-llama-swap gateway) gets no
|
||||
context/size enrichment at all. `modelContextLengths` falls back to the
|
||||
launch-command/`/props` enrichment at all, only whatever its listings
|
||||
state. `modelContextLengths` falls back to the
|
||||
endpoint's `defaultContextLength` when a session is assigned a model
|
||||
discovery never learned a window for. A `defaultModelId` that no longer
|
||||
appears in the fresh list is dropped
|
||||
|
||||
@@ -630,7 +630,7 @@ Adding the header changed the preamble, so `CODEMAN_PREAMBLE` was bumped (1.22.0
|
||||
|
||||
⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/copilot/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified.
|
||||
|
||||
⚠️ **Two launch paths, chosen by mechanism, not preference**: opencode/codex/gemini/pi/grok/deepseek/omp apply the selection ONE-SHOT, before the session/process ever exists, with no restart at all; claude alone still applies a selection by **restarting the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` relaunches a CLI in an existing pane, so a CLI whose launch declares a `fallback` chain (claude) gets a conversation pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. The dead-pane respawn shares this pin, see [Dead-pane respawn: the resume pin](#dead-pane-respawn-the-resume-pin). ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600.
|
||||
⚠️ **Two launch paths, chosen by mechanism, not preference**: opencode/codex/gemini/pi/grok/deepseek/omp apply the selection ONE-SHOT, before the session/process ever exists, with no restart at all; claude alone still applies a selection by **restarting the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` relaunches a CLI in an existing pane, so a CLI whose launch declares a `fallback` chain (claude) gets a conversation pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. The dead-pane respawn shares this pin, see [Dead-pane respawn: the resume pin](#dead-pane-respawn-the-resume-pin). ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name), and codex gets one too (`--model <bare modelId>`, its pattern widened to `model-plain-colon` for qwen3-style ids): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600.
|
||||
|
||||
⚠️ **Security**: every env var this feature can redirect (`ANTHROPIC_BASE_URL`, `GOOGLE_GEMINI_BASE_URL`, `CODEX_HOME`, `GROK_HOME`, `HOME` for pi/omp, `OPENCODE_CONFIG_CONTENT`, etc.) is in that CLI's `privilegedEnvKeys` — several of these were reachable via the generic `envOverrides` field's prefix allowlist BEFORE this feature existed (the env allowlist is global and prefix-based, not per-CLI-scoped), so building this surfaced and closed a pre-existing gap rather than opening a new one. `ANTHROPIC_*` is deliberately NOT in claude's `allowedPrefixes` at all — Anthropic-traffic redirection can only happen through this feature's own admin-configured, SSRF-guarded route, never a plain client-supplied `envOverrides`.
|
||||
|
||||
|
||||
@@ -294,6 +294,15 @@ hot-swap. A Claude session is relaunched with `--resume <conversation> ||
|
||||
grok are relaunched with the `--model` value that selects the injected
|
||||
provider (`custom/<modelId>` for pi and omp, `codeman-custom` for grok),
|
||||
since for those three the config file alone does not switch the model.
|
||||
**Codex is relaunched with `--model <modelId>` too**, the bare selected id
|
||||
(not a provider selector): the launch param forces the session onto the
|
||||
model its generated `config.toml` already selected, so a launch-time model
|
||||
carried over from App Settings can never override the injected selection.
|
||||
That launch arg must satisfy the codex entry's `model-plain-colon` token
|
||||
pattern (a leading letter or digit, then letters, digits and `._-:/`, per
|
||||
`stock.ts`), so an id outside that shape, e.g. LM Studio's
|
||||
`org/model@q4_k_m`, is refused with a 400 at apply time now, where a
|
||||
`config.toml` alone used to carry it.
|
||||
**Remote (SSH) and Docker sessions are refused** (400) for now: their restart
|
||||
reattaches the durable remote/in-container tmux rather than relaunching the
|
||||
agent, so the selection would report success and change nothing.
|
||||
@@ -451,15 +460,17 @@ Both apply routes (the restart route and the one-shot `POST
|
||||
/api/quick-start`) now check for this **before** launching or restarting
|
||||
anything, gated on the CLI's registry entry declaring a `contextLengthVar`
|
||||
(currently only claude — the check is a no-op for every other CLI by
|
||||
construction, never a hardcoded mode check). If the model's discovered
|
||||
context (`modelContextLengths`, from discovery above) is below
|
||||
`CLAUDE_MIN_SAFE_CONTEXT_TOKENS` (40000, comfortably above the measured
|
||||
construction, never a hardcoded mode check). If the model's context (the
|
||||
discovered figure from `modelContextLengths`, or the endpoint's
|
||||
`defaultContextLength` when discovery never learned one for that model) is
|
||||
below `CLAUDE_MIN_SAFE_CONTEXT_TOKENS` (40000, comfortably above the measured
|
||||
~36.4K overhead), the response is `{requiresContextWarning: true, modelId,
|
||||
contextLength, minSafeContextTokens}` (plus `slotCount` when the model runs more than one parallel slot) instead of applying — nothing is
|
||||
restarted or created yet. A context length that was never discovered at
|
||||
all skips the check entirely (nothing to compare, so it fails open rather
|
||||
than warning on every model an endpoint hasn't reported a size for).
|
||||
The same warning fires when the context divided across the model's parallel slots (`contextLength / slotCount`) is below the floor, because Claude Code sends several 40K-token requests at once and the slots share one pool. Retrying with `confirmedContext: true` launches anyway (the legacy `confirmed: true` still means both questions).
|
||||
restarted or created yet. A model with neither a discovered figure nor a
|
||||
`defaultContextLength` skips the check entirely (nothing to compare, so it
|
||||
fails open rather than warning on every model an endpoint hasn't reported a
|
||||
size for).
|
||||
The same warning fires when the context divided across the model's parallel slots (`contextLength / slotCount`) is below the floor, because Claude Code sends several 40K-token requests at once and the slots share one pool. ⚠️ `defaultContextLength` is the model's whole window, never a per-slot figure, so a default small enough to split under the slot count trips that slot division exactly like a discovered figure does. Retrying with `confirmedContext: true` launches anyway (the legacy `confirmed: true` still means both questions).
|
||||
|
||||
The Run-menu picker shows this as an in-app modal
|
||||
(`#customModelContextWarningModal`, matching the llama-swap conflict
|
||||
@@ -536,7 +547,12 @@ for '<id>' not found. Defaulting to fallback metadata...` on launch —
|
||||
and fabricating a fake catalog entry would mean copying the _shape_ of
|
||||
OpenAI's own proprietary schema (their per-model system-prompt content
|
||||
included) for a warning that doesn't otherwise affect behavior — not
|
||||
something to build into discovery.
|
||||
something to build into discovery. ⚠️ The fallback-metadata warning is
|
||||
NOT about the context window going un-overridden. Codex does now get a
|
||||
real context-window override: when a figure is known the generated
|
||||
`config.toml` sets the top-level `model_context_window` to it and
|
||||
`model_auto_compact_token_limit` to 75% of it (floored), so Codex
|
||||
compacts at that point instead of trusting its fallback guess.
|
||||
- **Gemini** — fails with `Invalid auth method selected`, traced to an
|
||||
undocumented `GATEWAY` auth path gemini-cli selects once
|
||||
`GOOGLE_GEMINI_BASE_URL` is set. Unresolved after real investigation
|
||||
|
||||
@@ -27,14 +27,19 @@ hosts — these are machine-level infra, not a per-user setting.
|
||||
shows up without another manual click of **Discover**. One endpoint being unreachable on a
|
||||
given cycle (powered off, wrong network) never blocks the others from refreshing.
|
||||
|
||||
**Context length is picked up automatically where it can be, safely.** Against a
|
||||
llama.cpp/llama-swap server, discovery also learns each _currently loaded_ model's real
|
||||
context window and applies it to the launched session (Claude Code today — see below), so
|
||||
the harness stops assuming a large default window for a model name it doesn't recognise and
|
||||
overflowing a much smaller real one. It's deliberately never probed for a model that isn't
|
||||
already loaded, since asking a llama-swap server about an unloaded model can trigger an
|
||||
actual, slow model swap as a side effect — a model just not currently loaded keeps whatever
|
||||
context length an earlier cycle already learned for it instead.
|
||||
**Context length is picked up automatically where it can be, safely.** Every model's own
|
||||
listing entry is read for a stated context figure (`context_length`, then `context_window`,
|
||||
which current llama.cpp, llama-swap and gateway-style servers state to varying degrees);
|
||||
against a llama-swap server the stronger figures from its launch command and from llama.cpp's
|
||||
`/props` probe win where they exist, and those probes only ever run for a model the server
|
||||
already reports loaded, since asking about an unloaded model can trigger an actual, slow model
|
||||
swap as a side effect. A model that states no figure and was never loaded falls back to the
|
||||
endpoint's **Default context length** field (that number is the model's whole window, never a
|
||||
per-slot figure), and a model neither covers keeps whatever an earlier cycle learned or simply
|
||||
gets no override, never a guess. The window is applied to the launched session by the harnesses
|
||||
that can take it: Claude Code today (see below), OpenCode as the model's context limit, and
|
||||
Codex as a `model_context_window` entry in its generated config file, which also sets Codex's
|
||||
own auto-compact point via `model_auto_compact_token_limit` at 75% of the window.
|
||||
|
||||
## Running a session against one
|
||||
|
||||
@@ -48,7 +53,9 @@ pointed at.
|
||||
|
||||
**Which model it uses depends on how many the endpoint has discovered.** With exactly one,
|
||||
the session launches straight away on that model — nothing to choose. With two or more, a
|
||||
small dialog asks which one to use for this launch before starting the session; the
|
||||
small dialog asks which one to use for this launch before starting the session; a search box
|
||||
at the top narrows a long list with a case-insensitive substring match, so a server hosting
|
||||
hundreds of models stays usable. The
|
||||
endpoint's default model, if set, is marked but not auto-picked, so a launch can deliberately
|
||||
use a different one without changing the saved default. The list is not raw discovery order
|
||||
either: the model llama-swap reports loaded and ready is moved to the top and tagged
|
||||
|
||||
@@ -812,9 +812,11 @@ export interface CliCapabilities {
|
||||
* for pi and omp (`--model custom/<id>`, or the CLI stays on its own default
|
||||
* provider and reports "No API key found for the selected model") or for
|
||||
* grok (`--model codeman-custom`, the `[model.<name>]` block the config
|
||||
* declares). Absent = the config alone selects the model (claude's env vars,
|
||||
* opencode's blob, codex's top-level `model` key). Applied by the session's
|
||||
* respawn options through the entry's `legacyConfigField`, never by id.
|
||||
* declares), and codex carries its bare selected id (`{modelId}`, an id the
|
||||
* entry's `model` token pattern must be able to carry). Absent = the config
|
||||
* alone selects the model (claude's env vars, opencode's blob). Applied by
|
||||
* the session's respawn options through the entry's `legacyConfigField`,
|
||||
* never by id.
|
||||
*
|
||||
* `contextLengthVar` (env kind only): the env var a discovered per-model context-window
|
||||
* size is written to when known (claude's `CLAUDE_CODE_MAX_CONTEXT_TOKENS`) — without it,
|
||||
|
||||
@@ -66,7 +66,8 @@ export interface CustomModelHost {
|
||||
modelContextLengths?: Record<string, number>;
|
||||
/**
|
||||
* Discovered number of parallel server slots per model id, keyed like `modelContextLengths`
|
||||
* and populated on the same loaded-models-only basis. Read from the launch command's
|
||||
* but populated only for models the server reports loaded (a listing never states a slot
|
||||
* count, unlike `contextLengths`, which every entry can supply). Read from the launch command's
|
||||
* `--parallel`/`-np` when it states one, otherwise from llama.cpp's own `/props`
|
||||
* `total_slots` (a llama-server started without `--parallel` picks the count itself, so
|
||||
* the command cannot say). llama-server splits its context across these slots (by default
|
||||
@@ -92,9 +93,12 @@ export interface CustomModelHost {
|
||||
*/
|
||||
modelSizesGB?: Record<string, number>;
|
||||
/**
|
||||
* User-set fallback context length (tokens), applied by the read sites in
|
||||
* `session-routes.ts` only when `modelContextLengths` has no entry for the selected
|
||||
* model (a gateway that states no per-model figure, a model never loaded yet). Never
|
||||
* User-set fallback context length (tokens), applied via
|
||||
* `resolveCustomModelContextLength()` at every read site (the swap route, the
|
||||
* quick-start route, boot recovery) only when `modelContextLengths` has no entry for
|
||||
* the selected model (a gateway that states no per-model figure, a model never loaded
|
||||
* yet). It is the model's WHOLE window, never a per-slot figure: the Claude floor
|
||||
* check divides it by any stored slot count exactly like a discovered figure. Never
|
||||
* written by discovery, so a value set in the endpoint editor survives every
|
||||
* re-discovery sweep; unset keeps the long-standing behaviour of letting each CLI
|
||||
* assume its own default.
|
||||
|
||||
+2
-1
@@ -2724,7 +2724,8 @@ export class Session extends EventEmitter {
|
||||
|
||||
/**
|
||||
* Force the custom-model selection's `launchModel` (pi/omp `custom/<id>`, grok's
|
||||
* `[model.<name>]` block name) onto the CLI's `model` launch param. Where that param
|
||||
* `[model.<name>]` block name, codex's bare selected id) onto the CLI's `model`
|
||||
* launch param. Where that param
|
||||
* lives is registry DATA — the entry's `legacyConfigField` (`piConfig`, `grokConfig`,
|
||||
* ...) or the top-level `model` for an entry that declares none — so this stays a
|
||||
* generic reader rather than a branch per CLI. Applied on the OPTIONS only: the stored
|
||||
|
||||
@@ -616,8 +616,8 @@
|
||||
'Optional. Left blank on edit keeps the existing key.': '可选。编辑时留空将保留现有密钥。',
|
||||
'Auth header': '认证请求头',
|
||||
'Default context length (tokens)': '默认上下文长度(token 数)',
|
||||
'Used for models whose size cannot be discovered. Leave empty to let each CLI assume its own default.':
|
||||
'用于无法自动发现上下文大小的模型。留空则让各个 CLI 使用其自身的默认值。',
|
||||
"Used for models whose size cannot be discovered. The model's whole window; the Claude floor check divides it by any stored slot count. Leave empty to let each CLI assume its own default.":
|
||||
'用于无法自动发现上下文大小的模型。这是模型的完整窗口;Claude 下限检查会将其除以已存储的槽位数。留空则让各个 CLI 使用其自身的默认值。',
|
||||
'Never send both — some servers hang indefinitely.': '切勿同时发送两者 — 部分服务器会因此无限期挂起。',
|
||||
'Authorization: Bearer (default)': 'Authorization: Bearer(默认)',
|
||||
'api-key header (Azure)': 'api-key 请求头(Azure)',
|
||||
|
||||
@@ -2800,7 +2800,7 @@
|
||||
<input type="text" id="customModelHostBaseUrl" class="set-input" placeholder="http://192.168.1.50:8080">
|
||||
</div>
|
||||
<div class="set-row has-field">
|
||||
<div class="set-row-text"><span class="set-row-label">Default context length (tokens)</span><span class="set-row-desc">Used for models whose size cannot be discovered. Leave empty to let each CLI assume its own default.</span></div>
|
||||
<div class="set-row-text"><span class="set-row-label">Default context length (tokens)</span><span class="set-row-desc">Used for models whose size cannot be discovered. The model's whole window; the Claude floor check divides it by any stored slot count. Leave empty to let each CLI assume its own default.</span></div>
|
||||
<input type="number" id="customModelHostDefaultContextLength" class="set-input" min="1" max="100000000" step="1" placeholder="131072">
|
||||
</div>
|
||||
<div class="set-row has-field">
|
||||
|
||||
@@ -165,7 +165,7 @@ export interface DiscoveryResult {
|
||||
* over) any listed figure for the same model.
|
||||
*/
|
||||
contextLengths: Record<string, number>;
|
||||
/** See `CustomModelHost.modelSlotCounts`; populated on the same loaded-models-only basis as `contextLengths`. */
|
||||
/** See `CustomModelHost.modelSlotCounts`; populated only for models the server reports loaded (launch command or `/props`). No listing states a slot count, unlike `contextLengths`. */
|
||||
slotCounts: Record<string, number>;
|
||||
/** See `CustomModelHost.modelSizesGB` — populated for every model whose own listing states one. */
|
||||
sizesGB: Record<string, number>;
|
||||
|
||||
@@ -1611,7 +1611,8 @@ export function registerSessionRoutes(
|
||||
}
|
||||
|
||||
// A CLI whose config alone cannot select the model also gets its `model` launch param
|
||||
// forced (pi/omp `custom/<id>`, grok's block name). The argv engine DROPS a token that
|
||||
// forced (pi/omp `custom/<id>`, grok's block name, and codex now carries its bare
|
||||
// selected id too). The argv engine DROPS a token that
|
||||
// fails its pattern rather than quoting it, which would silently launch the CLI on its
|
||||
// own default provider again, so refuse an id the pattern cannot carry up front.
|
||||
const modelSpec = entry.launch.params.model;
|
||||
|
||||
Reference in new issue
Block a user