diff --git a/docs/api-reference.md b/docs/api-reference.md index 7e15f1bf..c59bdbe2 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1220,7 +1220,9 @@ authStyle?, defaultModelId?, defaultContextLength? }` creates one. `id` keeps the stored one rather than clearing it — the client never receives the real value to resend deliberately unchanged, so omission is the only way to say "leave it alone"; there is no way to clear a key back to unset - this way. `defaultModelId`, when set, must be one of that endpoint's own + this way. `defaultContextLength` behaves the opposite way: a PUT without + it clears the stored value, so send the field back to keep it. + `defaultModelId`, when set, must be one of that endpoint's own `models` (`400 INVALID_INPUT` otherwise). - `DELETE /api/v1/model-endpoints/:id` removes one. - `POST /api/v1/model-endpoints/:id/discover-models` fetches the endpoint's diff --git a/docs/wiki/Custom-Model-Endpoints.md b/docs/wiki/Custom-Model-Endpoints.md index c1f6896d..6ec5bac9 100644 --- a/docs/wiki/Custom-Model-Endpoints.md +++ b/docs/wiki/Custom-Model-Endpoints.md @@ -33,10 +33,11 @@ which current llama.cpp, llama-swap and gateway-style servers state to varying d against a llama-swap server the stronger figures from its launch command and from llama.cpp's `/props` probe win where they exist, and those probes only ever run for a model the server already reports loaded, since asking about an unloaded model can trigger an actual, slow model -swap as a side effect. A model that states no figure and was never loaded falls back to the -endpoint's **Default context length** field (that number is the model's whole window, never a -per-slot figure), and a model neither covers keeps whatever an earlier cycle learned or simply -gets no override, never a guess. The window is applied to the launched session by the harnesses +swap as a side effect. A model that states no figure keeps whatever an earlier cycle learned: +the stored figure always beats the endpoint's **Default context length** field +(`modelContextLengths[id] ?? defaultContextLength`), which only applies where no figure was +ever learned (that number is the model's whole window, never a per-slot figure). A model +neither covers simply gets no override, never a guess. The window is applied to the launched session by the harnesses that can take it: Claude Code today (see below), OpenCode as the model's context limit, and Codex as a `model_context_window` entry in its generated config file, which also sets Codex's own auto-compact point via `model_auto_compact_token_limit` at 75% of the window.