fix(custom-model): show which model a session is using when an endpoint cannot report it

Only llama-swap answers what is loaded, so behind a gateway such as Bifrost
nothing was marked and the picker fell back to discovery order apart from the
one CLI that had launched there before. A live session whose own injected
endpoint is that host now promotes its model as "In use by a session", under
its own tag and below llama-swap's "Currently loaded" answer, reading only
what Codeman injected rather than terminal text, and never a stopped session.
When genuinely nothing is known the picker says so instead of leaving an
unmarked wall, and stays quiet for an endpoint that merely failed to answer.
This commit is contained in:
Devvyn committed 2026-11-02 19:05:22 +08:00
1 parent 884e67d07e
commit 1f0823c582
7 files changed
+336 -40

No files matched your search

+3
View File
@@ -627,7 +627,10 @@
'Custom Endpoints': '自定义端点',
'Choose a model': '选择模型',
'Currently loaded': '当前已加载',
'In use by a session': '会话正在使用',
'Last used': '上次使用',
'Nothing is marked: this endpoint cannot report which model it has loaded, and no running session uses one. Pick the model to launch, or start a session on this endpoint first.':
'未标记任何模型:此端点无法报告它当前加载了哪个模型,并且没有运行中的会话正在使用它。请选择要启动的模型,或先在该端点上启动一个会话。',
'That endpoint no longer exists': '该端点已不存在',
'No models discovered for this endpoint yet': '此端点尚未发现任何模型',
// Model picker search box (index.html placeholder/aria-label and the no-matches line)
+5
View File
@@ -1041,6 +1041,11 @@
autocomplete="off" autocapitalize="off" spellcheck="false" enterkeyhint="search"
oninput="app.filterCustomModelPick(this.value)">
<p class="form-hint custom-model-pick-no-matches" id="customModelPickNoMatches" role="status" hidden>No models match</p>
<!-- Shown only when no row carries a loaded/in-use/last-used mark AND the endpoint
answered that it is not llama-swap (see _openCustomModelPickModal): it says out
loud WHY nothing is marked, in the user's language, rather than leaving an
alphabetical wall that looks like the question was ignored. -->
<p class="form-hint custom-model-pick-unknown" id="customModelPickUnknown" role="status" hidden>Nothing is marked: this endpoint cannot report which model it has loaded, and no running session uses one. Pick the model to launch, or start a session on this endpoint first.</p>
<div id="customModelPickList" class="run-mode-custom-models"></div>
</div>
</div>
+88 -22
View File
@@ -1002,12 +1002,20 @@ Object.assign(CodemanApp.prototype, {
},
/**
* Best-effort lookup of the model llama-swap currently has loaded and ready on this
* endpoint, so the picker can offer it first instead of making the user remember what
* they picked last time it mattered. Mirrors `_watchLlamaSwapLoading`'s own
* `state === 'ready'` check. Returns null for a plain (non-llama-swap) server, an
* unreachable endpoint, or a loaded model this host no longer lists as discovered —
* never throws, since a failed probe should just skip promotion, not break the picker.
* Best-effort lookup of what the endpoint itself says it has loaded and ready, so the
* picker can offer it first instead of making the user remember what they picked last
* time it mattered. Only llama-swap answers this question (the route reads its
* `/running`); every other OpenAI-compatible server (a bare llama.cpp, Ollama, vLLM,
* or a gateway such as Bifrost fronting a whole fleet) replies `isLlamaSwap: false`,
* which is why `_getCustomModelInUseBySession` exists. `state === 'ready'` mirrors
* `_watchLlamaSwapLoading`'s own check.
*
* Resolves the raw route payload rather than a model id so the caller can tell the
* three answers apart: llama-swap naming a model, an endpoint that positively cannot
* say (safe to explain in the UI), and no answer at all (timeout/unreachable, which
* resolves `null` and must NEVER be described as "cannot report", since the endpoint
* may well have been asleep rather than mute). Never throws: a failed probe should
* just skip promotion, not break the picker.
*
* ⚠️ Client-side bounded to ~800ms via Promise.race, on top of (never instead of) the
* route's own 5s server-side timeout (`RUNNING_TIMEOUT_MS`, custom-model-routes.ts) —
@@ -1024,17 +1032,56 @@ Object.assign(CodemanApp.prototype, {
* `vi.useFakeTimers()` patches, so a param is the only way to test the timeout without
* actually waiting on it. Real callers never pass it.
*/
async _getCustomModelCurrentlyLoaded(host, timeoutMs = 800) {
async _getCustomModelRunningStatus(host, timeoutMs = 800) {
const probe = this._apiJson(`/api/model-endpoints/${encodeURIComponent(host.id)}/running-status`).catch(
() => null
);
const timeout = new Promise((resolve) => setTimeout(() => resolve(null), timeoutMs));
const status = await Promise.race([probe, timeout]);
return Promise.race([probe, timeout]);
},
/**
* The model llama-swap reports `ready` on this endpoint AND that this host still lists
* as discovered (an undropped-but-undiscovered model has no row to promote), else null.
*/
_pickReadyRunningModel(status, host) {
if (!status?.isLlamaSwap) return null;
const ready = (status.running || []).find((r) => r.state === 'ready' && (host.models || []).includes(r.model));
return ready?.model || null;
},
/**
* The model a LIVE session of this instance is pointed at on `host`, or null. It is the
* fallback for every endpoint that cannot answer "what is loaded?" about itself.
* A session already running on the endpoint is first-hand evidence of a model in use
* there: Codeman itself injected that endpoint and model into it, so it earns the same
* promotion llama-swap's own answer gets, under its own honest tag.
*
* ⚠️ Reads ONLY `customModel` (what Codeman injected), never `displayModel`: that field
* is display text scraped off a CLI's footer or statusline for a session header (pane
* derived, `source: 'screen'`), and promoting a row because a terminal printed a
* similar string is worse than showing nothing.
* ⚠️ Requires a live pane: `status: 'stopped'` or a recorded `paneExit` means nothing
* is on the endpoint any more, and a dead session's stale pin is exactly what an "in
* use" claim must not resurrect. `session:updated` keeps it current, so a session
* cleared back to the CLI's own backend or moved to another endpoint stops counting.
* ⚠️ A since-deleted endpoint simply matches no host, so no extra liveness check is
* needed here.
*/
_getCustomModelInUseBySession(host) {
const models = host?.models || [];
if (models.length === 0) return null;
let newest = null;
for (const [, session] of this.sessions || []) {
if (!session || session.status === 'stopped' || session.paneExit) continue;
const pinned = session.customModel;
if (!pinned || pinned.endpointId !== host.id) continue;
if (!models.includes(pinned.modelId)) continue; // undiscovered since: no row to promote
if (!newest || (newest.createdAt || 0) <= (session.createdAt || 0)) newest = session;
}
return newest?.customModel?.modelId || null;
},
/**
* Renders the "which model" picker for a (harness, endpoint) pair with more than one
* discovered model. Async since it now awaits the currently-loaded-model probe below,
@@ -1055,15 +1102,17 @@ Object.assign(CodemanApp.prototype, {
const generation = (this._customModelPickGeneration = (this._customModelPickGeneration || 0) + 1);
const isCurrent = () => this._customModelPickGeneration === generation;
// Whichever model llama-swap actually has loaded right now beats a merely
// remembered choice — it's what a launch would attach to with zero wait, while
// "last used" might have been swapped out by another session since. Neither
// reorders past the top: exactly one model is promoted, everything else keeps
// its discovery order.
const currentlyLoaded = await this._getCustomModelCurrentlyLoaded(host);
// Three kinds of knowledge, best first: what llama-swap says it has loaded right
// now (zero wait to attach to), what a live session of this instance is pointed at
// on that endpoint (the endpoint itself cannot say, but Codeman put it there), and
// only then this device's memory of the last launch. One wins, and it never reorders
// anything else: every other row keeps its discovery order.
const runningStatus = await this._getCustomModelRunningStatus(host);
if (!isCurrent()) return; // a newer pick opened (and possibly already rendered) while this probe was in flight
const lastUsed = currentlyLoaded ? null : this._getCustomModelLastUsed(mode, host.id);
const promoted = currentlyLoaded || lastUsed;
const currentlyLoaded = this._pickReadyRunningModel(runningStatus, host);
const inUse = currentlyLoaded ? null : this._getCustomModelInUseBySession(host);
const lastUsed = currentlyLoaded || inUse ? null : this._getCustomModelLastUsed(mode, host.id);
const promoted = currentlyLoaded || inUse || lastUsed;
const models = [...(host.models || [])];
if (promoted && models.includes(promoted)) {
models.splice(models.indexOf(promoted), 1);
@@ -1078,14 +1127,31 @@ Object.assign(CodemanApp.prototype, {
document.getElementById('customModelPickTitle').textContent = 'Choose a model';
document.getElementById('customModelPickHint').textContent =
`${cliLabel} → ${host.label} — ${(host.models || []).length} models discovered.`;
// A plain static line (its markup lives in index.html), shown when NOTHING is known
// about any row: 40 alphabetical rows with no mark on any of them otherwise read as
// the app ignoring the question. It only asserts what the probe actually answered:
// `isLlamaSwap: false` is a real "I cannot tell you", where a probe that timed out or
// failed (`runningStatus === null`) is not, the box may merely be asleep, so an
// unreachable endpoint stays silent instead of explaining something it does not know.
const nothingKnown = !promoted && runningStatus?.isLlamaSwap === false;
const unknownLine = document.getElementById('customModelPickUnknown');
if (unknownLine) unknownLine.hidden = !nothingKnown;
list.innerHTML = models
.map((m) => {
// Two independent tags, never one exclusive slot: the promotion tag says what
// llama-swap (or this device's history) knows about the model, the Default pill
// says what the saved endpoint says about it, and on a single-purpose GPU box the
// promoted model IS the default more often than not. One slot holding whichever
// applied first silently dropped the Default marking for exactly that row.
const promotion = m === currentlyLoaded ? 'Currently loaded' : m === lastUsed ? 'Last used' : null;
// Two independent tags, never one exclusive slot: the promotion tag says what the
// endpoint, a live session, or this device's history knows about the model, the
// Default pill says what the saved endpoint says about it, and on a
// single-purpose GPU box the promoted model IS the default more often than not.
// One slot holding whichever applied first silently dropped the Default marking
// for exactly that row.
const promotion =
m === currentlyLoaded
? 'Currently loaded'
: m === inUse
? 'In use by a session'
: m === lastUsed
? 'Last used'
: null;
const tags = [promotion, m === host.defaultModelId ? 'Default' : null].filter(Boolean);
const arg = escapeHtml(JSON.stringify(m));
// data-model-id is the filter's haystack (escapeHtml covers `"` as well as
+11
View File
@@ -7824,6 +7824,17 @@ html[data-skin="og"] .welcome-primary:hover {
padding: 1rem 0;
}
/* "This endpoint cannot report which model it has loaded": the same `.form-hint`
`display: block` beats `[hidden]` trap the two rules below fix for the no-matches
line and the filtered rows, so the attribute needs the same re-assertion. */
#customModelPickModal .custom-model-pick-unknown {
padding-bottom: 0.15rem;
}
#customModelPickModal .custom-model-pick-unknown[hidden] {
display: none !important;
}
/* Same class of bug as the row rule below: the no-matches line carries
`.form-hint`, whose `display: block` outranks the UA `[hidden]` rule, so
`filterCustomModelPick()` toggling `hidden` changed nothing on screen and