fix(custom-model): warn when parallel llama.cpp slots leave too little context per request (#658)

Squash of PR #658. Discovery records each llama.cpp model's parallel slot
count (from --parallel/-np, or /props total_slots when the launch command
states none), and a Claude launch warns when the context split across
those slots leaves less than 40K tokens per request. An explicit
--parallel beside a /props context records a slot count of 1, so a later
round overwrites an earlier count, and a non-2xx /props is remembered per
launch command while a network error is retried.

Merge-time fix by the maintainer: the architecture-invariants sentence on
the explicit --parallel case now says it records a slot count of 1.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
DevvynandClaude Opus 5.5 authored and Codeman maintainer committed 2026-10-22 19:32:24 +02:00
1 parent 34cf798b10
commit 33d697afe3
17 files changed
+710 -34

No files matched your search

+5
View File
@@ -0,0 +1,5 @@
---
'aicodeman': patch
---
A custom llama.cpp model whose parallel slots share a context pool too small for concurrent Claude Code requests now gets an in-app warning. Four simultaneous 40K-token requests against a 4-slot ~131K pool all failed with "Context size has been exceeded" while the old total-context check stayed silent. Discovery records each model's slot count (from `--parallel`/`-np`, or `/props` `total_slots` when the launch command states none) and the warning names the fix: `--parallel 1`, or a context large enough for every slot.