mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 02:29:33 +01:00
fix(custom-model): warn when parallel llama.cpp slots leave too little context per request (#658)
Squash of PR #658. Discovery records each llama.cpp model's parallel slot count (from --parallel/-np, or /props total_slots when the launch command states none), and a Claude launch warns when the context split across those slots leaves less than 40K tokens per request. An explicit --parallel beside a /props context records a slot count of 1, so a later round overwrites an earlier count, and a non-2xx /props is remembered per launch command while a network error is retried. Merge-time fix by the maintainer: the architecture-invariants sentence on the explicit --parallel case now says it records a slot count of 1. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
34cf798b10
commit
33d697afe3
17 files changed
+710
-34
No files matched your search
@@ -996,7 +996,10 @@ confirmed? } | { clear: true }` applies (or clears) the session's
|
||||
just because a swap is needed at all — only when it would disrupt
|
||||
someone else.
|
||||
- `{requiresContextWarning: true, modelId, contextLength,
|
||||
minSafeContextTokens}` — Claude Code's own fixed per-turn overhead
|
||||
minSafeContextTokens, slotCount?}` — `slotCount` is present whenever the model runs more than
|
||||
one parallel llama.cpp slot, and the warning also fires when the context
|
||||
divided across those slots is below the floor even though the whole context
|
||||
is not. Claude Code's own fixed per-turn overhead
|
||||
(system prompt + tool schemas) can exceed a small model's entire
|
||||
discovered context on its own, before any conversation history exists
|
||||
to compact, guaranteeing the very first message fails regardless of
|
||||
|
||||
Reference in new issue
Block a user