fix(custom-model): core review fixes for context-length discovery and injection

- share one context-length resolution (discovered figure, endpoint default fallback) across the routes and boot recovery, with a recovery test
- record listed per-slot context figures with a slot count of 1 and force the slot-count fallback once per rediscovery
- open up the codex model launch token for qwen3-style ids (model-plain-colon), pinned by a spawn-golden case
- pass the full discovered window to opencode as limit.context
- translate the picker search strings and document the discovery precedence
This commit is contained in:
Devvyn committed 2026-11-01 21:26:54 +08:00
1 parent dbd1b6e587
commit 1b8ac40baf
16 files changed
+256 -47

No files matched your search

+22 -5
View File
@@ -1205,13 +1205,18 @@ data}` envelope on the wire — unwrap it the same way). Answers `[]` for a
boolean` reports whether one is stored, so a client can render "unchanged
if left blank" without ever holding the real value.
- `POST /api/v1/model-endpoints` with `{ id, label, baseUrl, apiKey?,
authStyle?, defaultModelId? }` creates one. `id` must match
authStyle?, defaultModelId?, defaultContextLength? }` creates one. `id`
must match
`^[a-zA-Z0-9_-]+$`; `authStyle` is `bearer` (default) or `api-key`, never
both (a real server hung indefinitely when sent both headers on one
request); `baseUrl` must be `http(s)`, carry no embedded credentials, and
request); `defaultContextLength` is a user-set fallback used when
assigning a model that discovery has no context length for (discovery
itself never writes it); `baseUrl` must be `http(s)`, carry no embedded
credentials, and
is refused if it points at (or resolves to) a link-local or
cloud-metadata address. `409 ALREADY_EXISTS` on a duplicate id.
- `PUT /api/v1/model-endpoints/:id` updates one. An **absent** `apiKey`
- `PUT /api/v1/model-endpoints/:id` updates one, also accepting
`defaultContextLength`. An **absent** `apiKey`
keeps the stored one rather than clearing it — the client never receives
the real value to resend deliberately unchanged, so omission is the only
way to say "leave it alone"; there is no way to clear a key back to unset
@@ -1221,8 +1226,20 @@ authStyle?, defaultModelId? }` creates one. `id` must match
- `POST /api/v1/model-endpoints/:id/discover-models` fetches the endpoint's
own `GET /v1/models` and stores the result as `models`, updating
`lastDiscoveredAt`, plus (best-effort, only for a model llama-swap's own
response already reports loaded) `modelContextLengths` and `modelSizesGB`.
A `defaultModelId` that no longer appears in the fresh list is dropped
response already reports loaded) `modelContextLengths`,
`modelSlotCounts` and `modelSizesGB`. A loaded model's context figure
comes from llama-swap's own launch command (`--fit-ctx` / `-c` /
`--ctx-size`), falling back to `/props` (`n_ctx`, `total_slots`) for
whatever the command did not answer; the model's own listing
(`context_length`, then `context_window`) only ever fills the gap when
neither probe does, and is recorded as a single-slot figure —
llama-swap lists the PER-SLOT window, so whenever the listed figure is
the one taken the stored slot count is 1. A server with no `status`
field on any entry (plain llama.cpp, a non-llama-swap gateway) gets no
context/size enrichment at all. `modelContextLengths` falls back to the
endpoint's `defaultContextLength` when a session is assigned a model
discovery never learned a window for. A `defaultModelId` that no longer
appears in the fresh list is dropped
rather than carried forward invalid. Failures answer `422 OPERATION_FAILED`
with the underlying connection error, or a named egress refusal if the
resolved address turned out to be blocked. The same refresh also runs