mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 18:39:33 +01:00
fix(custom-model): warn when parallel llama.cpp slots leave too little context per request (#658)
Squash of PR #658. Discovery records each llama.cpp model's parallel slot count (from --parallel/-np, or /props total_slots when the launch command states none), and a Claude launch warns when the context split across those slots leaves less than 40K tokens per request. An explicit --parallel beside a /props context records a slot count of 1, so a later round overwrites an earlier count, and a non-2xx /props is remembered per launch command while a network error is retried. Merge-time fix by the maintainer: the architecture-invariants sentence on the explicit --parallel case now says it records a slot count of 1. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
34cf798b10
commit
33d697afe3
17 files changed
+710
-34
No files matched your search
@@ -0,0 +1,5 @@
|
||||
---
|
||||
'aicodeman': patch
|
||||
---
|
||||
|
||||
A custom llama.cpp model whose parallel slots share a context pool too small for concurrent Claude Code requests now gets an in-app warning. Four simultaneous 40K-token requests against a 4-slot ~131K pool all failed with "Context size has been exceeded" while the old total-context check stayed silent. Discovery records each model's slot count (from `--parallel`/`-np`, or `/props` `total_slots` when the launch command states none) and the warning names the fix: `--parallel 1`, or a context large enough for every slot.
|
||||
Reference in new issue
Block a user