feat(custom-model): ask which model on launch when an endpoint has more than one, and re-discover models every 5 minutes

Two enhancements requested after live-validating PR #430 against a real llama.cpp server: 1. Model picker dialog. Picking a Run-menu Custom Endpoints entry used to apply the endpoint's defaultModelId (or the first discovered model) silently. Now, via the new selectCustomModelEntry() (session-ui.js): - exactly one discovered model launches straight away, same as before - two or more open a new #customModelPickModal listing every discovered model; defaultModelId (if set) is marked but never auto-chosen, since the point of asking is letting ONE launch deliberately differ from the saved default, not just confirming it The endpoint is re-fetched at click time rather than trusting anything cached from the dropdown's own render, since the model list can have changed (the sweep below, or a settings-panel edit) since it opened. runCustomModelEntry() itself — the actual launch, routed through run() for the in-flight lock, snapshot-guarded against applying to the wrong session — is unchanged; it now just always receives an explicit model id from one of these two paths instead of computing one itself. 2. Periodic re-discovery. Every saved endpoint's models now refresh automatically every 5 minutes in the background (CUSTOM_MODEL_REDISCOVER_INTERVAL_MS, server.ts, registered the same way as the Codex plan-usage poll it sits beside — this.cleanup.setInterval, off under testMode), so a model the server starts or stops serving shows up without another manual "Discover" click. The manual POST .../discover-models route and the new refreshAllCustomModelHosts() sweep (custom-model-routes.ts) now share one pure merge step (applyDiscoveredModels: stamps lastDiscoveredAt, drops a defaultModelId that no longer appears) rather than two copies that could drift. The sweep is best-effort per host — one endpoint being unreachable on a cycle never blocks the others — and re-reads the store before each host's write, keyed by id, so a concurrent edit or delete from the settings panel always wins over a sweep that started before it. Tests: test/custom-model-endpoint-rediscovery.test.ts is a new, dedicated file for the sweep (kept separate from custom-model-routes.test.ts because that file's data dir is shared across every test in it — one temp HOME per FILE, not per test — which would make a sweep-touches-every-host assertion meaningless there). test/custom-model-run-menu-ui.test.ts gained a new describe block driving the real picker modal through JSDOM: single-model bypass, multi-model dialog with the default marked-not-chosen, picking a row closes the modal and launches with that exact model, the endpoint re-fetch, and the two "vanished by click time" toast paths. Docs: docs/custom-model-endpoints.md, docs/wiki/Custom-Model-Endpoints.md, docs/api-reference.md and CLAUDE.md's dense feature paragraph all updated — the last of these also caught up two sentences that had gone stale after the draft-review fixes landed (the picker routes through run() now, not a raw run*() call). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqZeHrRS6DYcGcGX2p9EwG
2026-10-20 09:09:35 +02:00 · 2026-09-16 08:50:18 +08:00
1 parent 60e1bd52f7
commit 5a9ff07f57
@@ -227,7 +227,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph

 **DeepSeek web UI** (`POST`/`GET`/`DELETE /api/deepseek/web`, `deepseek-web-server.ts`): the Run menu's "DeepSeek web UI..." entry supervises ONE background `dsh web` child process, deliberately **NOT a shell session**. The session version worked and was still wrong in use: it put a terminal tab on screen next to the web tab the user actually asked for, every single time, and nothing about a long-lived HTTP server needs to be a tab. ⚠️ What a session gave for free now has to be paid for explicitly, and every piece is load-bearing: **exactly one** server (a second click REUSES it rather than racing it for a port, which two sessions structurally could not do), **restarted when the browser authority changes** (`--trusted-host` fences dsh's `/api` against the browser authority, and a Codeman reachable at both loopback and a tailnet name has two, so whoever asks last wins: the asker is by definition the origin about to load the page), **killed on shutdown** (`stopDeepSeekWeb()` in the server teardown, because the child is detached so its whole plugin tree can be signalled at once, which also means it would OUTLIVE Codeman and hold its port against the next start), and **failures returned to the caller**, since with no tab there is nowhere for a stack trace to land. ⚠️ The port search starts at dsh's own default 3080 and walks 40, never fixed: that default is precisely the port most likely to be taken already by the user's own `dsh web`, and hardcoding it killed this feature with EADDRINUSE once. Free-port detection BINDS rather than connects (a connect probe cannot tell "free" from "listening but not answering yet"), so it is racy by nature and the caller still waits for the server to really answer before reporting success. ⚠️ Both `POST` and `DELETE` sit at the **same privilege bar as the profile installer** (`canUsernameRunPrivilegedCommands`) even though the action reads as "open a page": booting a dsh profile executes the plugin code in it, and the server is a single shared instance, so stopping it in multi-user mode takes it out from under other users' tabs.

-**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`, design doc `docs/custom-model-endpoints-plan.md`; full stack — settings-panel CRUD + the Run-menu picker, on top of the backend below): points a session at a user-configured custom OpenAI-compatible endpoint — local (llama.cpp, DGX Spark, Strix Halo) or cloud (Azure AI Foundry, OpenRouter) — instead of its harness's native cloud backend. Endpoints are a read/write-array store (`custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`) discovered via `GET <baseUrl>/v1/models`; `CustomModelHost.authStyle` is `'bearer'` (default, `Authorization: Bearer`) or `'api-key'` (Azure's convention) — **never both**, live-tested against a real server: sending both headers on one request reliably hangs it indefinitely, reproduced 3×. ⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified. ⚠️ Applying a selection **restarts the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` kills a WORKING pane, so a CLI whose launch declares a `fallback` chain (claude) gets the live conversation id pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600. ⚠️ **Security**: every env var this feature can redirect (`ANTHROPIC_BASE_URL`, `GOOGLE_GEMINI_BASE_URL`, `CODEX_HOME`, `GROK_HOME`, `HOME` for pi/omp, `OPENCODE_CONFIG_CONTENT`, etc.) is in that CLI's `privilegedEnvKeys` — several of these were reachable via the generic `envOverrides` field's prefix allowlist BEFORE this feature existed (the env allowlist is global and prefix-based, not per-CLI-scoped), so building this surfaced and closed a pre-existing gap ratherLine truncated
+**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`, design doc `docs/custom-model-endpoints-plan.md`; full stack — settings-panel CRUD + the Run-menu picker, on top of the backend below): points a session at a user-configured custom OpenAI-compatible endpoint — local (llama.cpp, DGX Spark, Strix Halo) or cloud (Azure AI Foundry, OpenRouter) — instead of its harness's native cloud backend. Endpoints are a read/write-array store (`custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`) discovered via `GET <baseUrl>/v1/models`; `CustomModelHost.authStyle` is `'bearer'` (default, `Authorization: Bearer`) or `'api-key'` (Azure's convention) — **never both**, live-tested against a real server: sending both headers on one request reliably hangs it indefinitely, reproduced 3×. ⚠️ The actual per-CLI redirect is `capabilities.customModelInjection` on the CLI registry (four kinds: `env` for claude/gemini/deepseek, `configContentEnv` reusing opencode's existing `OPENCODE_CONFIG_CONTENT`, `configDir` for codex/pi/grok/omp — writes an isolated per-session config file, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config — and `unsupported` for antigravity, which has no known mechanism), computed by the pure `custom-model-injection.ts` (mirrors `session-cli-builder.ts`'s no-IO discipline). ⚠️ `PI_CONFIG_DIR` does NOTHING for pi or omp (grepped pi's entire bundled JS source — the string appears nowhere); both hardcode `~/.pi/agent/models.json` / `~/.omp/agent/models.yml` with no dedicated override, so the real redirect for both is the child process's own **`HOME`**, and both need `models` as an ARRAY of `{id}` objects (an object keyed by id silently loads zero models). Grok's real mechanism turned out to be a `config.toml` `[model.<name>]` block redirected via `GROK_HOME` — its original env-var-based recipe was flat-out wrong (produced "Not signed in" against a real binary), not just unverified. ⚠️ Applying a selection **restarts the session's CLI process in place** via `Session.restartCli()` — a de-restricted `reattachRemote()` reusing the same `respawn-pane -k` primitive local/remote respawns already share — because every one of these harnesses reads its endpoint config at process start, never per-turn, so there is no live hot-swap; `Session.setCustomModel()` undoes the PREVIOUS selection's env keys (and deletes its old `configDir`) before merging the new ones in, so switching endpoints or clearing back to native cloud never leaves a stale key behind. ⚠️ Deleting a key from `_envOverrides` is NOT enough on its own: `tmux setenv` persists at the tmux-session level and is inherited by `respawn-pane` (measured: `setenv FOO bar` survived two successive `respawn-pane -k`), so the retired keys are queued (`_pendingEnvUnsets`) and ride `RespawnPaneOptions.unsetEnvKeys` into `applyEnvOverrides()`, which `setenv -u`s them BEFORE re-applying the live overrides. ⚠️ `restartCli()` kills a WORKING pane, so a CLI whose launch declares a `fallback` chain (claude) gets the live conversation id pinned as `resumeSessionId` for that one respawn: `--session-id <id>` refuses an id that already has a transcript (`Session ID ... is already in use`), and without the `--resume <id> || --session-id <id>` shape the docker/remote pane commands already use, applying a model killed the pane and lost the session. ⚠️ pi, omp and grok need the config file AND a `model` launch param (`custom/<id>` for pi/omp, grok's `[model.codeman-custom]` block name): that is the registry's `customModelInjection.launchModel` template, applied onto the respawn options through `legacyConfigField` by `_withCustomModelLaunchModel()`, never by id, and a model id the CLI's `model` token pattern cannot carry is refused with a 400 rather than silently dropped by the argv engine. ⚠️ Remote (SSH) and Docker sessions are REFUSED (400): their `restartCli()` reattaches a durable tmux rather than restarting the agent and the env lands on the local pane, so they used to report `restarted:true` and change nothing. The selection survives a Codeman restart as the disk-only `__customModel` (bookkeeping: env KEYS, config dir, launch model; never the values, which carry the API key and are re-derived from the endpoint store on recovery), the config dir is removed with the session, and every secret-bearing file (`custom-model-hosts.json`, the per-session config dir) is written 0600. ⚠️ **Security**: every env var this feature can redirect (`ANTHROPIC_BASE_URL`, `GOOGLE_GEMINI_BASE_URL`, `CODEX_HOME`, `GROK_HOME`, `HOME` for pi/omp, `OPENCODE_CONFIG_CONTENT`, etc.) is in that CLI's `privilegedEnvKeys` — several of these were reachable via the generic `envOverrides` field's prefix allowlist BEFORE this feature existed (the env allowlist is global and prefix-based, not per-CLI-scoped), so building this surfaced and closed a pre-existing gap ratherLine truncated

 **Run launch synchronization**: the Run entrypoint holds an in-flight lock and disables `#runBtn` for the whole launch (≥500ms), so a double click cannot create duplicate sessions with the same `w<n>-<case>` name. `_ensureCreatedSessionVisible()` runs before `selectSession()`, and `_onSessionCreated()` stays an idempotent upsert, so POST-first and SSE-first ordering both produce exactly one rendered tab. ⚠️ **Closing has the mirror-image race and one owner**: `closeSession()` reads `wasActive` BEFORE its `await` and announces the delete via `_closingSessions`, while `_onSessionDeleted` skips the active-session handoff for an id in that set. Both used to read `activeSessionId` after the fact, so the `session_deleted` broadcast for your own delete could null it first and closing the tab you were on landed on the welcome screen instead of the next session, on the same build, depending on timing. The fallback also picks the first order entry that is still in `sessions` (a dead id can linger in `sessionOrder`, same reason Alt+N indexes a live-filtered list). A delete from ANOTHER client still shows the welcome screen, which is the honest answer when what you were looking at was taken away. Tests: `test/session-close-fallback.test.ts`. → [architecture-invariants#run-launch-synchronization](docs/architecture-invariants.md#run-launch-synchronization)

@@ -551,7 +551,11 @@ user guide: [`custom-model-endpoints.md`](custom-model-endpoints.md).
  `lastDiscoveredAt`. A `defaultModelId` that no longer appears in the fresh
  list is dropped rather than carried forward invalid. Failures answer
  `502 OPERATION_FAILED` with the underlying connection error, or a named
-  egress refusal if the resolved address turned out to be blocked.
+  egress refusal if the resolved address turned out to be blocked. The same
+  refresh also runs automatically for every saved endpoint every 5 minutes
+  in the background (`refreshAllCustomModelHosts()`, `custom-model-routes.ts`,
+  started from `server.ts`), so there is no route for triggering "refresh
+  all" — one endpoint being unreachable on a cycle never blocks the others.
 - `POST /api/v1/sessions/:id/custom-model` with `{ endpointId, modelId } |
  { clear: true }` applies (or clears) the session's selection and
  **restarts the session's CLI process in place** — every supported harness
@@ -66,18 +66,26 @@ configured, `PUT`/`DELETE /api/model-endpoints/:id` update or remove one.
 Endpoint management is admin-only in multi-user mode, same as remote/docker
 hosts — these are machine-level infra, not per-user settings.

-`defaultModelId` names which discovered model the Run-menu picker applies
-for that endpoint with no further choice — the settings panel's Edit form
-exposes it as a select populated from the endpoint's own discovered
-`models`, and the route refuses a value that isn't one of them. Leaving it
-unset falls back to the first discovered model; re-discovering drops a
-default that no longer appears in the fresh list rather than carrying an
-invalid one forward.
+`defaultModelId` names which discovered model the picker pre-marks for that
+endpoint — the settings panel's Edit form exposes it as a select populated
+from the endpoint's own discovered `models`, and the route refuses a value
+that isn't one of them. It is applied automatically only when the endpoint
+has exactly one discovered model (nothing to choose); with two or more it
+is a pre-selection in the model-picker dialog below, never a silent default.
+Re-discovering drops a default that no longer appears in the fresh list
+rather than carrying an invalid one forward.
+
+**Model lists refresh themselves.** A background sweep (`server.ts`,
+`CUSTOM_MODEL_REDISCOVER_INTERVAL_MS`, every 5 minutes) re-discovers every
+saved endpoint the same way the manual `POST .../discover-models` route
+does, best-effort per endpoint — one being unreachable on a given cycle
+never blocks the others. Off under `npm test`, same reasoning as the Codex
+plan-usage poll it sits beside: no real network to hit, no server instance
+to keep the timer alive for.

 ## The Run-menu picker

-With the setting on and at least one endpoint carrying a usable default
-model (either an explicit `defaultModelId` or just one discovered model),
+With the setting on and at least one endpoint carrying a discovered model,
 the toolbar's Run dropdown grows a **Custom Endpoints** section: one entry
 per (harness that can redirect to a custom endpoint, saved endpoint) pair,
 e.g. "Claude Code (llama.cpp)". The harness list is read off the CLI
@@ -86,13 +94,22 @@ registry's own `capabilities.customModelInjection` at page render
 in the frontend — so a CLI whose injection recipe lands later shows up with
 no frontend change, and Antigravity (`unsupported`) never does.

-Picking an entry runs a single session on that harness exactly the way its
+Picking an entry re-fetches the endpoint (`selectCustomModelEntry()`,
+`session-ui.js`) rather than trusting anything cached from the dropdown's
+own render — the model list can have changed via the 5-minute sweep above
+or a settings-panel edit since the menu opened. With exactly one discovered
+model it runs straight away; with two or more, a small modal
+(`#customModelPickModal`) lists them and asks which one to use for this
+launch, with the endpoint's `defaultModelId` marked but not auto-chosen —
+the point of asking is letting one launch deliberately differ from the
+saved default, not just confirming it. Whichever way the model was decided,
+the launch itself runs a single session on that harness exactly the way its
 own Run-menu entry would (same case creation, env overrides, everything),
-then immediately applies the endpoint's default model to it via the route
-below. It is a one-off "try this endpoint" action, not a sticky mode: the
-plain Run button still means "this harness, native cloud" afterward.
-Entries are hidden entirely for a remote or Docker active case, since the
-apply route refuses both (see the next section).
+then immediately applies the endpoint and model to it via the route below.
+It is a one-off "try this endpoint" action, not a sticky mode: the plain
+Run button still means "this harness, native cloud" afterward. Entries are
+hidden entirely for a remote or Docker active case, since the apply route
+refuses both (see the next section).

 ## Applying a model to a session

@@ -16,20 +16,32 @@ Still in App Settings → Models → Custom model endpoints:
   say). An API key is optional; most local servers don't check one.
 2. **Discover** — fetches the endpoint's own model list over `GET /v1/models` and stores it.
 3. Pick a **default model** from what was discovered. This is the model the Run-menu entry
-   below applies with no further choice, so set it once you know which one you want.
+   applies directly when only one model is discovered; with two or more, it's just the one
+   pre-marked in the picker dialog described below, not a silent default.

 Endpoint management is admin-only in multi-user mode, the same as remote hosts and Docker
 hosts — these are machine-level infra, not a per-user setting.

+**Model lists refresh themselves.** Every saved endpoint is re-discovered automatically every
+5 minutes in the background, so a model the server starts serving later — or stops serving —
+shows up without another manual click of **Discover**. One endpoint being unreachable on a
+given cycle (powered off, wrong network) never blocks the others from refreshing.
+
 ## Running a session against one

-With the setting on and at least one endpoint carrying a usable default model, the **Run**
+With the setting on and at least one endpoint carrying a discovered model, the **Run**
 dropdown grows a **Custom Endpoints** section: one entry per harness that can redirect to a
 custom endpoint, per saved endpoint, e.g. "Claude Code (llama.cpp)". Picking one starts a
-session on that harness exactly the way its own entry would, then points it at the
-endpoint's default model. It is a one-off "try this endpoint" action, not a sticky mode — the
-plain **Run** button still means "this harness, native cloud" afterward, and a fresh session
-never inherits whatever the last one was pointed at.
+session on that harness exactly the way its own entry would. It is a one-off "try this
+endpoint" action, not a sticky mode — the plain **Run** button still means "this harness,
+native cloud" afterward, and a fresh session never inherits whatever the last one was
+pointed at.
+
+**Which model it uses depends on how many the endpoint has discovered.** With exactly one,
+the session launches straight away on that model — nothing to choose. With two or more, a
+small dialog asks which one to use for this launch before starting the session; the
+endpoint's default model, if set, is marked but not auto-picked, so a launch can deliberately
+use a different one without changing the saved default.

 Applying a selection **restarts the harness's process in place** — same tab, same
 conversation where the harness supports resuming one, fresh environment. That restart is
@@ -311,6 +311,9 @@
    'What the Run-menu picker applies for this endpoint. Discover models first.':
      '运行菜单选择器会为此端点应用该模型。请先发现可用模型。',
    'Custom Endpoints': '自定义端点',
+    'Choose a model': '选择模型',
+    'That endpoint no longer exists': '该端点已不存在',
+    'No models discovered for this endpoint yet': '此端点尚未发现任何模型',
    'Subagent Options': '子智能体选项',
    'Enable Tracking': '启用跟踪',
    'Active Tab Only': '仅活动标签页',
@@ -910,6 +910,23 @@
      </div>
    </div>

+    <!-- Custom Model Endpoint Profiles: "which model" picker (docs/custom-model-endpoints-plan.md).
+         Shown only when the chosen endpoint has more than one discovered model — see
+         selectCustomModelEntry() in session-ui.js, which skips straight to launch otherwise. -->
+    <div class="modal" id="customModelPickModal">
+      <div class="modal-backdrop" onclick="app.closeCustomModelPickModal()"></div>
+      <div class="modal-content" style="max-width: 380px;">
+        <div class="modal-header">
+          <h3 id="customModelPickTitle">Choose a model</h3>
+          <button class="modal-close" onclick="app.closeCustomModelPickModal()" aria-label="Close model picker">&times;</button>
+        </div>
+        <div class="modal-body">
+          <p class="form-hint" id="customModelPickHint"></p>
+          <div id="customModelPickList" class="run-mode-custom-models"></div>
+        </div>
+      </div>
+    </div>
+
    <!-- Cron Jobs Modal -->
    <div class="modal" id="cronModal">
      <div class="modal-backdrop" onclick="app.closeCron()"></div>
@@ -579,8 +579,9 @@ Object.assign(CodemanApp.prototype, {

    const rows = [];
    for (const host of hosts) {
-      const modelId = host.defaultModelId || (host.models || [])[0];
-      if (!modelId) continue; // nothing discovered yet — the settings panel explains why
+      const models = host.models || [];
+      if (models.length === 0) continue; // nothing discovered yet — the settings panel explains why
+      const modelId = host.defaultModelId || models[0];
      for (const cli of capableClis) {
        // escapeHtml(JSON.stringify(...)) on EVERY arg, not just the untrusted
        // one: JSON.stringify's own double quotes would otherwise terminate this
@@ -589,11 +590,11 @@ Object.assign(CodemanApp.prototype, {
        // modelId (server-controlled, from the endpoint's own /v1/models reply,
        // not this box's) into markup instead of inert data. Same idiom as
        // deleteCase's onclick a few hundred lines down.
-        const args = [cli.id, host.id, modelId].map((v) => escapeHtml(JSON.stringify(v))).join(', ');
+        const args = [cli.id, host.id].map((v) => escapeHtml(JSON.stringify(v))).join(', ');
        rows.push(`
          <button class="run-mode-option" data-mode="${escapeHtml(cli.id)}" data-endpoint="${escapeHtml(host.id)}"
-                  onclick="app.runCustomModelEntry(${args})"
-                  title="${escapeHtml(cli.label)} → ${escapeHtml(host.baseUrl)} (${escapeHtml(modelId)})">
+                  onclick="app.selectCustomModelEntry(${args})"
+                  title="${escapeHtml(cli.label)} → ${escapeHtml(host.baseUrl)} (${escapeHtml(modelId)}${models.length > 1 ? `, +${models.length - 1} more` : ''})">
            <span class="run-mode-dot ${escapeHtml(cli.id)}"></span>${escapeHtml(cli.label)} (${escapeHtml(host.label)})
          </button>`);
      }
@@ -604,6 +605,76 @@ Object.assign(CodemanApp.prototype, {
    container.innerHTML = rows.join('');
  },

+  /**
+   * Decides whether picking a Run-menu Custom Endpoint entry can launch
+   * straight away or needs to ask which model first. Re-fetches the endpoint
+   * rather than trusting anything cached from the menu render: the models
+   * list (or the default) could have changed — a re-discovery cycle running
+   * every 5 minutes in the background, or an edit in the settings panel —
+   * between opening the dropdown and clicking a row.
+   */
+  async selectCustomModelEntry(mode, endpointId) {
+    document.getElementById('runModeMenu')?.classList.remove('active');
+    const hosts = await this._apiJson('/api/model-endpoints');
+    const host = (hosts || []).find((h) => h.id === endpointId);
+    if (!host) {
+      this.showToast('That endpoint no longer exists', 'error');
+      return;
+    }
+    const models = host.models || [];
+    if (models.length === 0) {
+      this.showToast('No models discovered for this endpoint yet', 'warning');
+      return;
+    }
+    // Exactly one model: nothing to choose, so asking would just be an extra
+    // click for the same answer every time. Two or more: always ask, even
+    // with a defaultModelId set — the point of asking is letting THIS launch
+    // differ from the default, not just confirming it.
+    if (models.length === 1) {
+      return this.runCustomModelEntry(mode, endpointId, models[0]);
+    }
+    this._openCustomModelPickModal(mode, host);
+  },
+
+  /** Renders the "which model" picker for a (harness, endpoint) pair with more than one discovered model. */
+  _openCustomModelPickModal(mode, host) {
+    const modal = document.getElementById('customModelPickModal');
+    const list = document.getElementById('customModelPickList');
+    if (!modal || !list) return;
+    this._pendingCustomModelPick = { mode, endpointId: host.id };
+    const cliLabel = (window.__codemanCustomModelClis || []).find((c) => c.id === mode)?.label || mode;
+    // A static title (translatable by i18n.js's exact-string walker) plus a
+    // dynamic hint carrying the specifics — same split webviewModalTitle uses,
+    // since the walker cannot i18n a string a variable is already spliced into.
+    document.getElementById('customModelPickTitle').textContent = 'Choose a model';
+    document.getElementById('customModelPickHint').textContent =
+      `${cliLabel} → ${host.label} — ${(host.models || []).length} models discovered.`;
+    list.innerHTML = (host.models || [])
+      .map((m) => {
+        const isDefault = m === host.defaultModelId;
+        const arg = escapeHtml(JSON.stringify(m));
+        return `
+          <button class="run-mode-option" onclick="app.chooseCustomModelAndRun(${arg})">
+            <span class="run-mode-dot ${escapeHtml(mode)}"></span>${escapeHtml(m)}${isDefault ? ' <span class="set-scope">Default</span>' : ''}
+          </button>`;
+      })
+      .join('');
+    modal.classList.add('active');
+  },
+
+  closeCustomModelPickModal() {
+    document.getElementById('customModelPickModal')?.classList.remove('active');
+    this._pendingCustomModelPick = null;
+  },
+
+  /** A model row in the picker modal was clicked: close it and launch with that choice. */
+  chooseCustomModelAndRun(modelId) {
+    const pending = this._pendingCustomModelPick;
+    this.closeCustomModelPickModal();
+    if (!pending) return; // modal reopened/closed from elsewhere between render and click
+    void this.runCustomModelEntry(pending.mode, pending.endpointId, modelId);
+  },
+
  /**
   * Runs a session on `mode` and immediately applies `endpointId`/`modelId` to it
   * via POST /api/sessions/:id/custom-model (see session-routes.ts) — the same
@@ -107,6 +107,51 @@ function describeFetchError(err: unknown): string {

 type RedactedHost = ReturnType<typeof redactApiKey>;

+/**
+ * Merges a fresh `GET /v1/models` result into a host record: stamps
+ * `lastDiscoveredAt`, and drops `defaultModelId` if it no longer appears in
+ * the fresh list (it would otherwise leave the Run-menu picker applying a
+ * model id the endpoint just told us it doesn't serve). Pure — no IO, so the
+ * manual route (which reports a fetch failure's *reason* to the caller) and
+ * the periodic sweep below (which only cares whether it can move on) can
+ * each do their own `discoverModels()` + error handling around one shared
+ * "how to apply a successful result" step.
+ */
+function applyDiscoveredModels(host: CustomModelHost, models: string[]): CustomModelHost {
+  const defaultModelId = host.defaultModelId && models.includes(host.defaultModelId) ? host.defaultModelId : undefined;
+  return { ...host, models, defaultModelId, lastDiscoveredAt: new Date().toISOString() };
+}
+
+/**
+ * Re-discovers every saved endpoint's models, best-effort. One endpoint being
+ * unreachable (powered off, wrong network) must not stop the others from
+ * refreshing, and a read-modify-write per host (rather than one batch write
+ * at the end) means a crash or restart mid-sweep loses at most the endpoints
+ * not yet reached, never a write already applied. Exported so both the
+ * periodic timer (server.ts) and a test can drive it directly.
+ */
+export async function refreshAllCustomModelHosts(): Promise<void> {
+  const dataDir = getDataDir();
+  const hosts = await readCustomModelHosts(dataDir);
+  for (const host of hosts) {
+    if (isBlockedWebviewUrl(host.baseUrl)) continue;
+    let models: string[];
+    try {
+      models = await discoverModels(host);
+    } catch {
+      continue; // unreachable this cycle — try again next tick, not fatal to the sweep
+    }
+    // Re-read + splice by id rather than reusing the array captured above: an
+    // admin editing or deleting an endpoint via the API mid-sweep must win,
+    // not be silently overwritten by a refresh that started before their change.
+    const current = await readCustomModelHosts(dataDir);
+    const index = current.findIndex((item) => item.id === host.id);
+    if (index === -1) continue; // deleted mid-sweep
+    current[index] = applyDiscoveredModels(current[index], models);
+    await writeCustomModelHosts(dataDir, current);
+  }
+}
+
 export function registerCustomModelRoutes(app: FastifyInstance): void {
  app.get('/api/model-endpoints', async (req): Promise<RedactedHost[]> => {
    if (isMultiUserMode() && !isAdmin(req)) return [];
@@ -179,12 +224,7 @@ export function registerCustomModelRoutes(app: FastifyInstance): void {
      try {
        const models = await discoverModels(host);
        const next = [...hosts];
-        // A default that no longer appears in the fresh list would leave the Run-menu
-        // picker applying a model id the endpoint just told us it doesn't serve; drop
-        // it rather than carry it forward silently invalid.
-        const defaultModelId =
-          host.defaultModelId && models.includes(host.defaultModelId) ? host.defaultModelId : undefined;
-        next[index] = { ...host, models, defaultModelId, lastDiscoveredAt: new Date().toISOString() };
+        next[index] = applyDiscoveredModels(host, models);
        await writeCustomModelHosts(CODEMAN_CONFIG_DIR, next);
        return { success: true, data: { models } };
      } catch (err) {
@@ -27,4 +27,4 @@ export { registerWsRoutes } from './ws-routes.js';
 export { registerVoiceRoutes } from './voice-routes.js';
 export { registerWebviewRoutes, tryWebviewRefererFallback } from './webview-routes.js';
 export { registerTabLayoutRoutes } from './tab-layout-routes.js';
-export { registerCustomModelRoutes } from './custom-model-routes.js';
+export { registerCustomModelRoutes, refreshAllCustomModelHosts } from './custom-model-routes.js';
@@ -190,6 +190,7 @@ import {
  registerWebviewRoutes,
  registerTabLayoutRoutes,
  registerCustomModelRoutes,
+  refreshAllCustomModelHosts,
  tryWebviewRefererFallback,
 } from './routes/index.js';
 import { isLostWebviewFrameNavigation } from './webview-proxy.js';
@@ -202,6 +203,7 @@ const __dirname = dirname(fileURLToPath(import.meta.url));
 // while capping growth of `sseClientsById` and blocking pathological inputs.
 const SSE_CLIENT_ID_RE = /^[A-Za-z0-9_-]{8,64}$/;
 const CODEX_USAGE_POLL_INTERVAL_MS = 5 * 60_000;
+const CUSTOM_MODEL_REDISCOVER_INTERVAL_MS = 5 * 60_000;

 function escapeHtmlText(value: string): string {
  return value.replaceAll('&', '&amp;').replaceAll('<', '&lt;').replaceAll('>', '&gt;');
@@ -2734,6 +2736,25 @@ export class WebServer extends EventEmitter {
      });
    }

+    // Custom Model Endpoint Profiles (docs/custom-model-endpoints-plan.md): keeps
+    // each saved endpoint's discovered model list current with no manual
+    // "Discover" click, so a model added on the server side (or one that drops
+    // off) shows up in the Run-menu picker within one cycle. Best-effort per
+    // endpoint (refreshAllCustomModelHosts skips one that's unreachable rather
+    // than failing the sweep) and off in tests for the same reason the Codex
+    // poll above is — no real network to hit, no server instance to keep alive.
+    if (!this.testMode) {
+      this.cleanup.setInterval(
+        () => {
+          refreshAllCustomModelHosts().catch((err) => {
+            console.error('[custom-model] periodic re-discovery failed:', getErrorMessage(err));
+          });
+        },
+        CUSTOM_MODEL_REDISCOVER_INTERVAL_MS,
+        { description: 'custom model endpoint re-discovery' }
+      );
+    }
+
    // Start scheduled runs cleanup timer
    this.cleanup.setInterval(
      () => {
@@ -0,0 +1,126 @@
+/**
+ * @fileoverview Tests for `refreshAllCustomModelHosts()`, the periodic
+ * background sweep behind server.ts's "custom model endpoint re-discovery"
+ * timer (docs/custom-model-endpoints-plan.md). Kept in its own file rather
+ * than folded into test/routes/custom-model-routes.test.ts: that file's data
+ * dir is shared across every test in it (one temp HOME per FILE, not per
+ * test — test/setup.ts), and a sweep that walks every saved host would pick
+ * up every host any other test in that file happened to create, making an
+ * exact call-count or exact-host assertion meaningless. A dedicated file
+ * gets its own clean temp HOME.
+ *
+ * Port: N/A (no server; drives readCustomModelHosts/writeCustomModelHosts
+ * directly plus the mocked webviewFetch dispatcher).
+ */
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+import { getDataDir } from '../src/config/instance.js';
+import { readCustomModelHosts, writeCustomModelHosts, type CustomModelHost } from '../src/custom-model-hosts.js';
+import { refreshAllCustomModelHosts } from '../src/web/routes/custom-model-routes.js';
+import { webviewFetch } from '../src/web/webview-egress.js';
+
+vi.mock('../src/web/webview-egress.js', async () => {
+  const actual = await vi.importActual<typeof import('../src/web/webview-egress.js')>('../src/web/webview-egress.js');
+  return { ...actual, webviewFetch: vi.fn() };
+});
+
+const fetchMock = vi.mocked(webviewFetch);
+
+function host(overrides: Partial<CustomModelHost> & Pick<CustomModelHost, 'id' | 'baseUrl'>): CustomModelHost {
+  return { label: overrides.id, ...overrides };
+}
+
+beforeEach(() => {
+  fetchMock.mockReset();
+});
+
+describe('refreshAllCustomModelHosts (the periodic re-discovery sweep)', () => {
+  it('refreshes every saved endpoint, best-effort — one unreachable host does not stop the others', async () => {
+    const dir = getDataDir();
+    await writeCustomModelHosts(dir, [
+      host({ id: 'ok', baseUrl: 'http://localhost:8080' }),
+      host({ id: 'down', baseUrl: 'http://localhost:8081' }),
+    ]);
+    fetchMock.mockImplementation(async (url: URL) => {
+      if (url.href.includes('8081')) throw new TypeError('fetch failed', { cause: new Error('ECONNREFUSED') });
+      return new Response(JSON.stringify({ data: [{ id: 'qwen3' }] }), { status: 200 });
+    });
+
+    await refreshAllCustomModelHosts();
+
+    const hosts = await readCustomModelHosts(dir);
+    const ok = hosts.find((h) => h.id === 'ok');
+    const down = hosts.find((h) => h.id === 'down');
+    expect(ok?.models).toEqual(['qwen3']);
+    expect(ok?.lastDiscoveredAt).toBeTruthy();
+    expect(down?.models ?? []).toEqual([]);
+    expect(down?.lastDiscoveredAt).toBeFalsy();
+  });
+
+  it('skips a host whose baseUrl is blocked, without making a request', async () => {
+    const dir = getDataDir();
+    // Written directly rather than through the POST route, which already
+    // refuses this at save time — this simulates a record that pre-dates the
+    // guard, or was hand-edited on disk. The sweep must not trust it either.
+    await writeCustomModelHosts(dir, [host({ id: 'meta', baseUrl: 'http://169.254.169.254/' })]);
+
+    fetchMock.mockResolvedValue(new Response(JSON.stringify({ data: [{ id: 'x' }] }), { status: 200 }));
+    await refreshAllCustomModelHosts();
+    expect(fetchMock).not.toHaveBeenCalled();
+  });
+
+  it('drops a stale default and preserves lastDiscoveredAt semantics, same as manual discovery', async () => {
+    const dir = getDataDir();
+    await writeCustomModelHosts(dir, [
+      host({ id: 'ep', baseUrl: 'http://localhost:8080', models: ['qwen3'], defaultModelId: 'qwen3' }),
+    ]);
+    fetchMock.mockResolvedValue(new Response(JSON.stringify({ data: [{ id: 'llama3' }] }), { status: 200 }));
+
+    await refreshAllCustomModelHosts();
+
+    const [updated] = await readCustomModelHosts(dir);
+    expect(updated.models).toEqual(['llama3']);
+    expect(updated.defaultModelId).toBeUndefined();
+    expect(updated.lastDiscoveredAt).toBeTruthy();
+  });
+
+  it('keeps a default that is still present after the sweep', async () => {
+    const dir = getDataDir();
+    await writeCustomModelHosts(dir, [
+      host({ id: 'ep', baseUrl: 'http://localhost:8080', models: ['qwen3'], defaultModelId: 'qwen3' }),
+    ]);
+    fetchMock.mockResolvedValue(
+      new Response(JSON.stringify({ data: [{ id: 'qwen3' }, { id: 'llama3' }] }), { status: 200 })
+    );
+
+    await refreshAllCustomModelHosts();
+
+    const [updated] = await readCustomModelHosts(dir);
+    expect(updated.defaultModelId).toBe('qwen3');
+  });
+
+  it('does not resurrect an endpoint deleted while the sweep was in flight', async () => {
+    const dir = getDataDir();
+    await writeCustomModelHosts(dir, [host({ id: 'deleted', baseUrl: 'http://localhost:8080' })]);
+
+    fetchMock.mockImplementation(async () => {
+      // Simulate an admin deleting the endpoint between the sweep's fetch and
+      // its read-modify-write — the delete must win, not be overwritten by a
+      // refresh that started before it.
+      const current = await readCustomModelHosts(dir);
+      await writeCustomModelHosts(
+        dir,
+        current.filter((h) => h.id !== 'deleted')
+      );
+      return new Response(JSON.stringify({ data: [{ id: 'qwen3' }] }), { status: 200 });
+    });
+
+    await expect(refreshAllCustomModelHosts()).resolves.toBeUndefined();
+    const hosts = await readCustomModelHosts(dir);
+    expect(hosts.find((h) => h.id === 'deleted')).toBeUndefined();
+  });
+
+  it('leaves the store untouched when there are no saved endpoints at all', async () => {
+    await expect(refreshAllCustomModelHosts()).resolves.toBeUndefined();
+    expect(fetchMock).not.toHaveBeenCalled();
+  });
+});
@@ -49,6 +49,11 @@ function bootApp(
        <div id="runModeCustomModelHeader" style="display:none"></div>
        <div id="runModeCustomModels"></div>
      </div>
+      <div class="modal" id="customModelPickModal">
+        <h3 id="customModelPickTitle"></h3>
+        <p id="customModelPickHint"></p>
+        <div id="customModelPickList"></div>
+      </div>
    </body>`,
    { url: 'http://localhost/', runScripts: 'dangerously' }
  );
@@ -68,6 +73,10 @@ function bootApp(
  app.loadAppSettingsFromStorage = () => ({ customModelEndpointsEnabled: options.settingsEnabled ?? true });
  app.isCliAvailable = options.cliAvailable ?? (() => true);
  app.showToast = () => {};
+  // Default no-op so a button's onclick (selectCustomModelEntry -> possibly
+  // straight to runCustomModelEntry for a single-model host) never rejects
+  // with "this.run is not a function"; tests of the launch itself override it.
+  app.run = async () => {};
  // _apiJson unwraps the {success,data} envelope for real against a live
  // server; here it stands in for that, driven from a fixed `hosts` fixture
  // so these tests exercise the picker's OWN code, not the envelope helper.
@@ -172,6 +181,122 @@ describe('Custom Model Endpoint Profiles: Run-menu picker generation', () => {
  });
 });

+describe('Custom Model Endpoint Profiles: the "which model" picker', () => {
+  it('launches straight away for a host with exactly one discovered model, no dialog', async () => {
+    const { win, app } = bootApp({
+      hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://localhost:8080', models: ['qwen3'] }],
+    });
+    let launched: unknown[] | null = null;
+    app.runCustomModelEntry = async (...args: unknown[]) => {
+      launched = args;
+    };
+
+    await app.selectCustomModelEntry('claude', 'llama-box');
+
+    expect(launched).toEqual(['claude', 'llama-box', 'qwen3']);
+    expect(win.document.getElementById('customModelPickModal')!.classList.contains('active')).toBe(false);
+  });
+
+  it('opens the picker for a host with more than one discovered model, rather than launching directly', async () => {
+    const { win, app } = bootApp({
+      hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://localhost:8080', models: ['qwen3', 'llama3'] }],
+    });
+    let launched = false;
+    app.runCustomModelEntry = async () => {
+      launched = true;
+    };
+
+    await app.selectCustomModelEntry('claude', 'llama-box');
+
+    expect(launched).toBe(false);
+    const modal = win.document.getElementById('customModelPickModal')!;
+    expect(modal.classList.contains('active')).toBe(true);
+    const list = win.document.getElementById('customModelPickList')!;
+    expect(list.querySelectorAll('button').length).toBe(2);
+    expect(list.textContent).toContain('qwen3');
+    expect(list.textContent).toContain('llama3');
+  });
+
+  it('always asks with 2+ models, even when a defaultModelId is set — the point is letting this launch differ', async () => {
+    const { win, app } = bootApp({
+      hosts: [
+        {
+          id: 'llama-box',
+          label: 'llama.cpp',
+          baseUrl: 'http://localhost:8080',
+          models: ['qwen3', 'llama3'],
+          defaultModelId: 'qwen3',
+        },
+      ],
+    });
+    await app.selectCustomModelEntry('claude', 'llama-box');
+    const modal = win.document.getElementById('customModelPickModal')!;
+    expect(modal.classList.contains('active')).toBe(true);
+    // The default is marked, not auto-chosen.
+    expect(win.document.getElementById('customModelPickList')!.textContent).toContain('Default');
+  });
+
+  it('picking a row in the modal closes it and launches with that exact model', async () => {
+    const { win, app } = bootApp({
+      hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://localhost:8080', models: ['qwen3', 'llama3'] }],
+    });
+    let launched: unknown[] | null = null;
+    app.runCustomModelEntry = async (...args: unknown[]) => {
+      launched = args;
+    };
+    win.app = app;
+
+    await app.selectCustomModelEntry('claude', 'llama-box');
+    const buttons = win.document.getElementById('customModelPickList')!.querySelectorAll('button');
+    const llama3Btn = [...buttons].find((b) => b.textContent?.includes('llama3')) as unknown as HTMLButtonElement & {
+      onclick: (e: unknown) => void;
+    };
+    expect(typeof llama3Btn.onclick).toBe('function');
+    llama3Btn.onclick(new (win as any).Event('click'));
+
+    expect(launched).toEqual(['claude', 'llama-box', 'llama3']);
+    expect(win.document.getElementById('customModelPickModal')!.classList.contains('active')).toBe(false);
+  });
+
+  it('re-fetches the endpoint at click time rather than trusting anything cached from the menu render', async () => {
+    // The background re-discovery sweep (server-side, every 5 minutes) or a
+    // settings-panel edit can change the model list between opening the
+    // dropdown and clicking a row — the picker must reflect what is current.
+    let fetchCount = 0;
+    const { win, app } = bootApp({});
+    app._apiJson = async (path: string) => {
+      if (path !== '/api/model-endpoints') return null;
+      fetchCount += 1;
+      return [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: ['qwen3', 'llama3', 'phi4'] }];
+    };
+    await app.selectCustomModelEntry('claude', 'llama-box');
+    expect(fetchCount).toBe(1);
+    expect(win.document.getElementById('customModelPickList')!.querySelectorAll('button').length).toBe(3);
+  });
+
+  it('toasts and does nothing when the endpoint has vanished by click time', async () => {
+    const { app } = bootApp({ hosts: [] });
+    let toastMessage: string | null = null;
+    app.showToast = (msg: string) => {
+      toastMessage = msg;
+    };
+    await app.selectCustomModelEntry('claude', 'ghost-endpoint');
+    expect(toastMessage).toMatch(/no longer exists/i);
+  });
+
+  it('toasts and does nothing when the endpoint has zero discovered models by click time', async () => {
+    const { app } = bootApp({
+      hosts: [{ id: 'llama-box', label: 'llama.cpp', baseUrl: 'http://x', models: [] }],
+    });
+    let toastMessage: string | null = null;
+    app.showToast = (msg: string) => {
+      toastMessage = msg;
+    };
+    await app.selectCustomModelEntry('claude', 'llama-box');
+    expect(toastMessage).toMatch(/no models discovered/i);
+  });
+});
+
 describe('Custom Model Endpoint Profiles: applying a picked entry', () => {
  it('does not apply the endpoint to a session that was already open when the launch fails', async () => {
    const { app } = bootApp({});