Merge branch 'feat/local-model-harness-choice' into feat/local-model-skill-agents

This commit is contained in:
Codeman maintainer committed 2026-11-11 10:16:48 +01:00
commit cda1572b77
2 files changed
+16 -7

No files matched your search

+6 -2
View File
@@ -223,7 +223,7 @@ the agent, so a session with one agent and a session with another run on the sam
| OpenCode | about 8,100 tokens, 11 tools (v1.19.5; more with user-level skills) | yes | A tool round trip on a Mac with a real model (the catalogue's tested entry), and on a scripted endpoint for v1.19.5 |
| Claude Code | about 36,000 tokens, so the first answer takes minutes | yes | Shipped in 1.50.0; not measured again for this table |
| Pi | 2,096 tokens, 4 tools (v1.5.2) | yes | A complete tool round trip against a real model: tool call, tool result, final answer, the file on disk |
| DeepSeek Harness | about 7,650 tokens, 25 tools (0.1.1-rc.2) | **no** | A tool round trip on a scripted endpoint only. Against a real model the launch and the model name reached the runtime, and no turn finished |
| DeepSeek Harness | about 7,650 tokens, 25 tools (0.1.1-rc.2); 6,983, 27 tools (0.2.2-rc.1) | **no** | A tool round trip on a scripted endpoint only, on both versions. Against a real model the launch and the model name reached the runtime, and no turn finished |
How to read the last column: the runs of 2026-11-10 were made on **Linux**, through Codeman's
own custom model launch, against Ollama 0.13.0 with a 1.7B model on a CPU ("a real model") or
@@ -248,7 +248,11 @@ Three things follow from how the agents differ:
the one agent of the four besides Claude Code that reports the end of a turn. Its custom
endpoint recipe itself was verified with a real model on a llama.cpp server
([custom-model-endpoints.md](custom-model-endpoints.md)); what is missing is a finished turn
on this feature's runtime.
on this feature's runtime. On today's version (0.2.2-rc.1 with dsh-TUI 0.19.0) two more
things were seen: a setup guide opens in every session until a person has finished it once
(a person in a tab presses through it; a script must not, see the codeman skill), and its
transcript file was renamed, so Codeman's response viewer and `last-response` are empty for
a dsh session on that version until the reader learns the new name.
- **OpenCode and DeepSeek Harness send a second, small request** beside the first prompt (a
title, a side question). The runtime answers one request at a time, so it queues.
+10 -5
View File
@@ -83,10 +83,11 @@
* | | | | prompts sent at the rule stayed |
* | | | | unsent), and it draws nothing that |
* | | | | is a contract once it does |
* | `deepseek` | about 7,650 tokens, 25 tools | SCRIPTED only (0.1.1-rc.2 with dsh-TUI | none here: the CLI registry's `❯` |
* | | (0.1.1-rc.2) | 0.10.0-beta.1, not today's versions). | stands; a prompt sent at it was |
* | | | REAL: the launch and the model name | submitted, 2 of 2 launches |
* | | | reached the runtime, no turn finished | |
* | `deepseek` | about 7,650 tokens, 25 tools | SCRIPTED only, on both versions | none here: the CLI registry's `❯` |
* | | (0.1.1-rc.2); 6,983 tokens, 27 | (0.1.1-rc.2 with dsh-TUI 0.10.0-beta.1 | stands; a prompt sent at it was |
* | | tools (0.2.2-rc.1) | and 0.2.2-rc.1 with dsh-TUI 0.19.0). | submitted, 2 of 2 launches. On |
* | | | REAL: the launch and the model name | dsh-TUI 0.19.0 a setup guide draws |
* | | | reached the runtime, no turn finished | `❯` too until it was finished once |
*
* Every one of them sent the `name:tag` model id unchanged. `pi` was gated until 2026-11-10 for
* two open questions, both answered by its run: the id (above), and the window, which Codeman
@@ -94,7 +95,11 @@
* 128,000-token one). `deepseek` is a LAUNCHER: its install command leaves it with nothing to
* launch, so src/local-model/harness-facts.ts never reports it installable and a Mac is offered
* it only where it was already set up. It is offered on a scripted run and one unfinished real
* one: gate the row again (`gated: true`) should a run on a Mac not finish a turn.
* one: gate the row again (`gated: true`) should a run on a Mac not finish a turn. Two things
* about dsh-TUI 0.19.0 that a client of its `readyMark` must know: until a person has finished
* its setup guide once, every session opens on that guide and the guide's cursor is the mark
* (the codeman skill reads the pane after the mark for that reason), and its transcript file
* was renamed, so Codeman's `last-response` is empty for it for now.
*
* MEMORY. The need is `bytes + heldBytesPerToken x context + overheadBytes + peakExtraBytes`
* (src/local-model/recommend.ts). The two runners spend memory differently, so a row's three