mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 02:29:33 +01:00
Merge branch 'feat/local-model-harness-choice' into feat/local-model-skill-agents
This commit is contained in:
commit
cda1572b77
2 files changed
+16
-7
No files matched your search
@@ -223,7 +223,7 @@ the agent, so a session with one agent and a session with another run on the sam
|
||||
| OpenCode | about 8,100 tokens, 11 tools (v1.19.5; more with user-level skills) | yes | A tool round trip on a Mac with a real model (the catalogue's tested entry), and on a scripted endpoint for v1.19.5 |
|
||||
| Claude Code | about 36,000 tokens, so the first answer takes minutes | yes | Shipped in 1.50.0; not measured again for this table |
|
||||
| Pi | 2,096 tokens, 4 tools (v1.5.2) | yes | A complete tool round trip against a real model: tool call, tool result, final answer, the file on disk |
|
||||
| DeepSeek Harness | about 7,650 tokens, 25 tools (0.1.1-rc.2) | **no** | A tool round trip on a scripted endpoint only. Against a real model the launch and the model name reached the runtime, and no turn finished |
|
||||
| DeepSeek Harness | about 7,650 tokens, 25 tools (0.1.1-rc.2); 6,983, 27 tools (0.2.2-rc.1) | **no** | A tool round trip on a scripted endpoint only, on both versions. Against a real model the launch and the model name reached the runtime, and no turn finished |
|
||||
|
||||
How to read the last column: the runs of 2026-11-10 were made on **Linux**, through Codeman's
|
||||
own custom model launch, against Ollama 0.13.0 with a 1.7B model on a CPU ("a real model") or
|
||||
@@ -248,7 +248,11 @@ Three things follow from how the agents differ:
|
||||
the one agent of the four besides Claude Code that reports the end of a turn. Its custom
|
||||
endpoint recipe itself was verified with a real model on a llama.cpp server
|
||||
([custom-model-endpoints.md](custom-model-endpoints.md)); what is missing is a finished turn
|
||||
on this feature's runtime.
|
||||
on this feature's runtime. On today's version (0.2.2-rc.1 with dsh-TUI 0.19.0) two more
|
||||
things were seen: a setup guide opens in every session until a person has finished it once
|
||||
(a person in a tab presses through it; a script must not, see the codeman skill), and its
|
||||
transcript file was renamed, so Codeman's response viewer and `last-response` are empty for
|
||||
a dsh session on that version until the reader learns the new name.
|
||||
- **OpenCode and DeepSeek Harness send a second, small request** beside the first prompt (a
|
||||
title, a side question). The runtime answers one request at a time, so it queues.
|
||||
|
||||
|
||||
@@ -83,10 +83,11 @@
|
||||
* | | | | prompts sent at the rule stayed |
|
||||
* | | | | unsent), and it draws nothing that |
|
||||
* | | | | is a contract once it does |
|
||||
* | `deepseek` | about 7,650 tokens, 25 tools | SCRIPTED only (0.1.1-rc.2 with dsh-TUI | none here: the CLI registry's `❯` |
|
||||
* | | (0.1.1-rc.2) | 0.10.0-beta.1, not today's versions). | stands; a prompt sent at it was |
|
||||
* | | | REAL: the launch and the model name | submitted, 2 of 2 launches |
|
||||
* | | | reached the runtime, no turn finished | |
|
||||
* | `deepseek` | about 7,650 tokens, 25 tools | SCRIPTED only, on both versions | none here: the CLI registry's `❯` |
|
||||
* | | (0.1.1-rc.2); 6,983 tokens, 27 | (0.1.1-rc.2 with dsh-TUI 0.10.0-beta.1 | stands; a prompt sent at it was |
|
||||
* | | tools (0.2.2-rc.1) | and 0.2.2-rc.1 with dsh-TUI 0.19.0). | submitted, 2 of 2 launches. On |
|
||||
* | | | REAL: the launch and the model name | dsh-TUI 0.19.0 a setup guide draws |
|
||||
* | | | reached the runtime, no turn finished | `❯` too until it was finished once |
|
||||
*
|
||||
* Every one of them sent the `name:tag` model id unchanged. `pi` was gated until 2026-11-10 for
|
||||
* two open questions, both answered by its run: the id (above), and the window, which Codeman
|
||||
@@ -94,7 +95,11 @@
|
||||
* 128,000-token one). `deepseek` is a LAUNCHER: its install command leaves it with nothing to
|
||||
* launch, so src/local-model/harness-facts.ts never reports it installable and a Mac is offered
|
||||
* it only where it was already set up. It is offered on a scripted run and one unfinished real
|
||||
* one: gate the row again (`gated: true`) should a run on a Mac not finish a turn.
|
||||
* one: gate the row again (`gated: true`) should a run on a Mac not finish a turn. Two things
|
||||
* about dsh-TUI 0.19.0 that a client of its `readyMark` must know: until a person has finished
|
||||
* its setup guide once, every session opens on that guide and the guide's cursor is the mark
|
||||
* (the codeman skill reads the pane after the mark for that reason), and its transcript file
|
||||
* was renamed, so Codeman's `last-response` is empty for it for now.
|
||||
*
|
||||
* MEMORY. The need is `bytes + heldBytesPerToken x context + overheadBytes + peakExtraBytes`
|
||||
* (src/local-model/recommend.ts). The two runners spend memory differently, so a row's three
|
||||
|
||||
Reference in new issue
Block a user