diff --git a/.changeset/custom-model-picker-in-use.md b/.changeset/custom-model-picker-in-use.md index fa344495..d8b27bba 100644 --- a/.changeset/custom-model-picker-in-use.md +++ b/.changeset/custom-model-picker-in-use.md @@ -2,4 +2,4 @@ 'aicodeman': patch --- -The "Choose a model" picker now says which model is actually in use on an endpoint that cannot report it. Only llama-swap answers that question, so behind a gateway such as Bifrost (and behind a bare llama.cpp, Ollama or vLLM) nothing was marked and the list fell back to plain discovery order, apart from the one CLI that happened to have launched there before. A live session whose own injected endpoint is that host now promotes its model as **"In use by a session"**, under its own tag and behind llama-swap's own "Currently loaded" answer, reading only what Codeman injected rather than text scraped off a terminal, and never a stopped session. When genuinely nothing is known the picker says so instead of leaving an unmarked wall, and it stays quiet for an endpoint that merely failed to answer. Pinned by test/custom-model-run-menu-ui.test.ts. +The "Choose a model" picker now says which model is actually in use on an endpoint that cannot report it. Only llama-swap answers that question, so behind a gateway such as Bifrost (and behind a bare llama.cpp, Ollama or vLLM) nothing was marked and the list fell back to plain discovery order, apart from the one CLI that happened to have launched there before. A live session whose own injected endpoint is that host now promotes its model as **"In use by a session"**, under its own tag and behind llama-swap's own "Currently loaded" answer, reading only what Codeman injected rather than text scraped off a terminal, and never a stopped or exited session. When nothing is known the picker says so instead of leaving an unmarked wall; its wording claims only that nothing was reported, which stays true for an endpoint that is down, since the route answers every probe failure as `isLlamaSwap: false`. Pinned by test/custom-model-run-menu-ui.test.ts. diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index 699a664a..cd0f07e8 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -638,7 +638,7 @@ Adding the header changed the preamble, so `CODEMAN_PREAMBLE` was bumped (1.22.0 ⚠️ **The Run-menu picker generates entries from `window.__codemanCustomModelClis`** (`server.ts`, injected at page render from `enabledClis().filter(kind==='agent' && customModelInjection.kind!=='unsupported')`, JSON-escaped against a literal `` via the exported `escapeScriptJson()` since `label` is a user-`clis.json`-settable string unlike the neighbouring booleans-only `__codemanCliAvailable`), never a hardcoded per-CLI id list in the frontend — the same "no branching on CLI id outside stock.ts" discipline the registry itself enforces. One entry per (capable, INSTALLED CLI, saved endpoint) pair, e.g. "Claude Code (llama.cpp)", filtered through `isCliAvailable()` like the stock entries. Clicking one calls `selectCustomModelEntry(mode, endpointId)` (`session-ui.js`), which re-fetches the endpoint (never trusts anything cached from the dropdown's render — the 5-minute sweep below or a settings edit may have changed it since) and decides the model: exactly one discovered model launches straight away, two or more open `#customModelPickModal` to ask, with `defaultModelId` marked but never auto-chosen (asking exists so ONE launch can deliberately differ from the saved default). -⚠️ **The picker promotes exactly one row to the top**: whichever model llama-swap reports `ready` right now (tagged "Currently loaded", queried via `GET /api/model-endpoints/:id/running-status`, client-side bounded to ~800ms via `Promise.race` so a sleeping/firewalled endpoint cannot leave the modal invisible for the route's own 5s server-side timeout) beats a merely remembered choice, and — only when nothing is currently loaded — the model actually launched last for this exact (harness, endpoint) pair (tagged "Last used", read from the per-device `codeman:customModelLastUsed::` localStorage key). Neither tag reorders past the top, and the "Default" pill is a SEPARATE span rather than a third value of the same slot, so a promoted row that is also the endpoint's `defaultModelId` shows both (on a single-purpose GPU box that is the common case; an exclusive slot silently dropped the Default marking for exactly that row). "Last used" is written by `_runCustomModelEntryViaRestart` (claude) and `_quickStartWithCustomModelConfirm` (every one-shot launch; `runCustomModelEntry` itself only dispatches between the two) only once the model is actually applied — never on the mere click — because a context-window-warning decline means this exact model cannot work with this CLI at all, and promoting a model that cannot launch would be actively wrong, not just premature. +⚠️ **The picker promotes exactly one row to the top**: whichever model llama-swap reports `ready` right now (tagged "Currently loaded", queried via `GET /api/model-endpoints/:id/running-status`, client-side bounded to ~800ms via `Promise.race` so a sleeping/firewalled endpoint cannot leave the modal invisible for the route's own 5s server-side timeout) beats a live session's pin (tagged "In use by a session": `_getCustomModelInUseBySession` walks `this.sessions` for one of your own sessions whose `customModel.endpointId` is that host, newest `createdAt` wins), which applies only when nothing is currently loaded and beats a merely remembered choice, the model actually launched last for this exact (harness, endpoint) pair (tagged "Last used", read from the per-device `codeman:customModelLastUsed::` localStorage key). ⚠️ The pin reads only `customModel`, never `displayModel` (screen-scraped display text: a terminal that merely printed a matching string is not evidence), and never counts a session with `status: 'stopped'`, a null `pid` (its PTY exited) or a recorded `paneExit`, the same live-pane test keep-warm applies. ⚠️ When nothing was promoted and the probe answered `isLlamaSwap: false` (the shape `getLlamaSwapStatus` gives for EVERY failure it catches, a down or refused endpoint included, so the line claims only that nothing was reported), the static `#customModelPickUnknown` line says so out loud; its `[hidden]` rule must keep `display: none !important` scoped to `#customModelPickModal`, because `.form-hint`'s `display: block` outranks the UA `[hidden]` rule. Neither tag reorders past the top, and the "Default" pill is a SEPARATE span rather than a third value of the same slot, so a promoted row that is also the endpoint's `defaultModelId` shows both (on a single-purpose GPU box that is the common case; an exclusive slot silently dropped the Default marking for exactly that row). "Last used" is written by `_runCustomModelEntryViaRestart` (claude) and `_quickStartWithCustomModelConfirm` (every one-shot launch; `runCustomModelEntry` itself only dispatches between the two) only once the model is actually applied — never on the mere click — because a context-window-warning decline means this exact model cannot work with this CLI at all, and promoting a model that cannot launch would be actively wrong, not just premature. Either way the actual launch (`runCustomModelEntry`) routes through `run()` itself via a temporary `_runMode` swap — never `setRunMode()`, which would persist it as the user's new default — rather than a parallel dispatch table, which is what gives a custom-model launch the same `_runInFlight` lock every other Run click gets and means a CLI whose injection recipe lands later needs no update here. It then GETs `/api/sessions/:id/wait?until=idle&timeout=20000` on that session BEFORE applying — measured live, a freshly launched CLI reports itself `busy` for its own startup (boot spinner, workspace-trust check) well before the apply call would otherwise reach it, and the apply route's `isBusy()` guard correctly can't tell that apart from a real turn in progress, so every fresh launch failed with `SESSION_BUSY` until this wait was added. A timeout there is a normal 200 per the wait endpoint's own contract, never an error, so a session still busy after 20s just reaches the apply call anyway and gets that route's own honest error. It then calls `POST /api/sessions/:id/custom-model` on the session `run()` produced, guarded by snapshotting `activeSessionId` before the call and requiring it to have actually changed after — every `run*()` handles its own failure internally and returns normally rather than throwing, so a declined/failed launch must not silently re-point and restart whatever session was already open. diff --git a/docs/custom-model-endpoints.md b/docs/custom-model-endpoints.md index c621bab9..81bf7add 100644 --- a/docs/custom-model-endpoints.md +++ b/docs/custom-model-endpoints.md @@ -213,16 +213,17 @@ evidence first, and the first one that answers wins: cannot report its own loaded model, which is every server except llama-swap: a bare llama.cpp, Ollama, vLLM, and a gateway such as Bifrost fronting a whole fleet all answer `isLlamaSwap: false`. - `_getCustomModelInUseBySession` walks `this.sessions` for a session of - this very instance whose `customModel.endpointId` is this host, newest + `_getCustomModelInUseBySession` walks `this.sessions` for one of your own + sessions whose `customModel.endpointId` is this host, newest `createdAt` wins, and promotes its model. A live session is first-hand evidence of a model in use on the endpoint, because Codeman put it there, so it earns the promotion and the tag says where the knowledge came from. Two fences keep it honest: it reads only `customModel`, never `displayModel` (display text scraped off a CLI footer, so a terminal that merely printed a similar string must not mark a row), - and a session with `status: 'stopped'` or a recorded `paneExit` does - not count, since nothing is on the endpoint any more. Sessions on other + and a session with `status: 'stopped'`, a null `pid` (its PTY exited, + the same live-pane test keep-warm applies) or a recorded `paneExit` + does not count, since nothing is on the endpoint any more. Sessions on other endpoints, and models the host no longer discovers, are skipped. - **"Last used"** appears only when neither of the above answers: the model actually launched last for this exact (harness, endpoint) pair, @@ -248,11 +249,14 @@ with neither promotion nor default shows no tag at all. When nothing is known about any row, `#customModelPickUnknown` says so out loud instead of leaving an alphabetical wall that reads as the app -ignoring the question. It asserts only what the probe actually answered: -`isLlamaSwap: false` is a real "I cannot tell you", whereas a probe that -timed out or failed (`_getCustomModelRunningStatus` resolving `null`) is -not, so an unreachable endpoint stays silent rather than claiming -something it does not know. Like its sibling no-matches line it needs an +ignoring the question. It shows when the probe answered +`isLlamaSwap: false`, which the route gives for every failure it +catches, so one answer covers a server that cannot report and an +endpoint that is down or refused alike; the line's wording therefore +claims only that nothing was reported. A probe that never got any +answer (`_getCustomModelRunningStatus` resolving `null`, Codeman's own +request failed or the client-side bound won) keeps the line hidden. +Like its sibling no-matches line it needs an explicit `[hidden]` rule scoped to `#customModelPickModal`, because `.form-hint`'s `display: block` outranks the UA `[hidden]` rule. diff --git a/src/web/public/i18n.js b/src/web/public/i18n.js index 3dc01d6d..2399324d 100644 --- a/src/web/public/i18n.js +++ b/src/web/public/i18n.js @@ -629,8 +629,8 @@ 'Currently loaded': '当前已加载', 'In use by a session': '会话正在使用', 'Last used': '上次使用', - 'Nothing is marked: this endpoint cannot report which model it has loaded, and no running session uses one. Pick the model to launch, or start a session on this endpoint first.': - '未标记任何模型：此端点无法报告它当前加载了哪个模型，并且没有运行中的会话正在使用它。请选择要启动的模型，或先在该端点上启动一个会话。', + 'Nothing is marked: this endpoint did not report which model it has loaded, and none of your running sessions uses one. Pick the model to launch.': + '未标记任何模型：此端点没有报告它已加载了哪个模型，你运行中的会话也没有在使用它。请选择要启动的模型。', 'That endpoint no longer exists': '该端点已不存在', 'No models discovered for this endpoint yet': '此端点尚未发现任何模型', // Model picker search box (index.html placeholder/aria-label and the no-matches line) diff --git a/src/web/public/index.html b/src/web/public/index.html index 189e19eb..ce6242e1 100644 --- a/src/web/public/index.html +++ b/src/web/public/index.html @@ -1045,7 +1045,7 @@ answered that it is not llama-swap (see _openCustomModelPickModal): it says out loud WHY nothing is marked, in the user's language, rather than leaving an alphabetical wall that looks like the question was ignored. --> - +
diff --git a/src/web/public/session-ui.js b/src/web/public/session-ui.js index f87da923..2226008c 100644 --- a/src/web/public/session-ui.js +++ b/src/web/public/session-ui.js @@ -1011,10 +1011,13 @@ Object.assign(CodemanApp.prototype, { * `_watchLlamaSwapLoading`'s own check. * * Resolves the raw route payload rather than a model id so the caller can tell the - * three answers apart: llama-swap naming a model, an endpoint that positively cannot - * say (safe to explain in the UI), and no answer at all (timeout/unreachable, which - * resolves `null` and must NEVER be described as "cannot report", since the endpoint - * may well have been asleep rather than mute). Never throws: a failed probe should + * three answers apart: llama-swap naming a model; `isLlamaSwap: false`, which is what + * the route answers for EVERY failure it catches (a server without a `/running` + * route, a refused or unresolvable endpoint, a 401 from a wrong key) — safe to + * explain in the UI as long as the wording also fits an endpoint that is down; and + * `null`, only when Codeman's own request to the route failed or the client-side + * bound below won the race, which stays silent, since the endpoint may merely have + * been asleep. Never throws: a failed probe should * just skip promotion, not break the picker. * * ⚠️ Client-side bounded to ~800ms via Promise.race, on top of (never instead of) the @@ -1061,10 +1064,12 @@ Object.assign(CodemanApp.prototype, { * is display text scraped off a CLI's footer or statusline for a session header (pane * derived, `source: 'screen'`), and promoting a row because a terminal printed a * similar string is worse than showing nothing. - * ⚠️ Requires a live pane: `status: 'stopped'` or a recorded `paneExit` means nothing - * is on the endpoint any more, and a dead session's stale pin is exactly what an "in - * use" claim must not resurrect. `session:updated` keeps it current, so a session - * cleared back to the CLI's own backend or moved to another endpoint stops counting. + * ⚠️ Requires a live pane: `status: 'stopped'`, a null `pid` (the PTY exited; the + * same live-pane test keep-warm applies, `src/prompt-cache-keepwarm.ts`) or a recorded + * `paneExit` means nothing is on the endpoint any more, and a dead session's stale pin + * is exactly what an "in use" claim must not resurrect. `session:updated` keeps it + * current, so a session cleared back to the CLI's own backend or moved to another + * endpoint stops counting. * ⚠️ A since-deleted endpoint simply matches no host, so no extra liveness check is * needed here. */ @@ -1073,7 +1078,7 @@ Object.assign(CodemanApp.prototype, { if (models.length === 0) return null; let newest = null; for (const [, session] of this.sessions || []) { - if (!session || session.status === 'stopped' || session.paneExit) continue; + if (!session || session.status === 'stopped' || session.pid === null || session.paneExit) continue; const pinned = session.customModel; if (!pinned || pinned.endpointId !== host.id) continue; if (!models.includes(pinned.modelId)) continue; // undiscovered since: no row to promote @@ -1129,10 +1134,13 @@ Object.assign(CodemanApp.prototype, { `${cliLabel} → ${host.label} — ${(host.models || []).length} models discovered.`; // A plain static line (its markup lives in index.html), shown when NOTHING is known // about any row: 40 alphabetical rows with no mark on any of them otherwise read as - // the app ignoring the question. It only asserts what the probe actually answered: - // `isLlamaSwap: false` is a real "I cannot tell you", where a probe that timed out or - // failed (`runningStatus === null`) is not, the box may merely be asleep, so an - // unreachable endpoint stays silent instead of explaining something it does not know. + // the app ignoring the question. `isLlamaSwap: false` is what the route answers for + // every failure it catches: a server that cannot report AND a down or refused one + // alike (`getLlamaSwapStatus`, custom-model-routes.ts), so the line's wording + // promises only that nothing was reported, which stays true either way. A probe + // that never got an answer at all (`runningStatus === null`: Codeman's own request + // failed or the 800ms bound won) keeps the line hidden, the box may merely be + // asleep. // A remembered id that has since vanished from host.models is no promotion: count // only a promoted id that is actually a row, or a stale "last used" would both skip // tagging any row and hide this line, leaving 40 unmarked rows with no explanation. diff --git a/src/web/public/styles.css b/src/web/public/styles.css index 3431fd8e..e9145138 100644 --- a/src/web/public/styles.css +++ b/src/web/public/styles.css @@ -7824,9 +7824,9 @@ html[data-skin="og"] .welcome-primary:hover { padding: 1rem 0; } -/* "This endpoint cannot report which model it has loaded": the same `.form-hint` - `display: block` beats `[hidden]` trap the two rules below fix for the no-matches - line and the filtered rows, so the attribute needs the same re-assertion. */ +/* "Nothing is marked: this endpoint did not report which model it has loaded": the same + `.form-hint` `display: block` beats `[hidden]` trap the two rules below fix for the + no-matches line and the filtered rows, so the attribute needs the same re-assertion. */ #customModelPickModal .custom-model-pick-unknown { padding-bottom: 0.15rem; } diff --git a/test/custom-model-run-menu-ui.test.ts b/test/custom-model-run-menu-ui.test.ts index 579ce8ea..65e50be2 100644 --- a/test/custom-model-run-menu-ui.test.ts +++ b/test/custom-model-run-menu-ui.test.ts @@ -64,7 +64,7 @@ function bootApp( - +