BitSquare/Codeman
2
0 Fork 0
mirror of https://github.com/Ark0N/Codeman.git synced 2026-11-15 10:59:32 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,920 Commits 119 Branches 211 Tags
afc5d7ab2a15c3352ccb649c30ad7a2aa63efd99
Commit Graph
12 Commits
Author SHA1 Message Date
Codeman maintainer d22bfaaaaf feat(local-model): the coding agent is chosen per session, with Pi and DeepSeek Harness 
An installed model knew one coding agent: the one it was installed with.
The launch check never asked about the agent, only about the endpoint
and the model, so nothing stopped a second one. What was missing is the
list, and a way to get an agent that is not on the Mac yet.

The install view gains `harnesses`: every catalogue harness the model in
use can be run with, each with how it stands (`ready`, `installable`,
`installing`, `setup-needed`), the literal install command, a readiness
mark for scripts, and the actions the server accepts for it. `cli` stays
the default agent, and names one that can be started whenever one can,
so a client that reads only `cli` never starts an agent that is gone.

`POST /api/local-model/agent { cli }` installs one more agent beside an
installed model. It is not a job and not an event of the reducer: the
state stays `installed` whatever the installer does, so a failed npm
install never blocks a session on a model that works. Same install slot
and same compiled stock command as the `agent` step, one at a time, the
outcome in memory only.

The catalogue:
- `pi` is no longer gated. Its two open questions were answered by a run
  against a real model (the `name:tag` id reaches the runtime unchanged;
  its first request is 2,096 tokens), and Codeman now passes it the
  window. Its minimum is 65536 like the others: nothing below was run,
  and no model row offers less, so no Mac's pick changes.
- `deepseek` is new. It is a launcher: its install command leaves it
  with nothing to launch, so it is never reported installable (read off
  `discovery.launcherProfile`, not the id) and a Mac is offered it only
  where it was already set up. Its tool round trip was seen on a
  scripted endpoint only; the header says how to gate it again.
- a row may carry `readyMark`; `opencode` has the measured one.

A switch no longer refuses for the agent a model was checked with when
another agent that can run it is on the Mac.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
 2026-11-11 09:53:54 +01:00
Codeman maintainer 70dc42b248 feat(local-model): a library of models, with one in use 
The local model window could hold one model. A second install was refused
(409 already-installed), and a model a Remove kept could never be freed
from the window again. The store can now hold several models, and exactly
one of them is the model in use: the one the endpoint record lists, the
runtime is started for and a session may be launched on.

Server
- LocalModelStateFile.library lists every model that passed the check
  (id, label, context, digest, size, coding agent). The service writes
  the entry when verify passes; the reducer carries it and remove drops
  it. An installed file of an earlier build counts its one model.
- POST /api/local-model/install is allowed from installed: it adds a
  model, and the earlier one stays. A replaced job no longer deletes a
  model of the library, only an unchecked or partial download.
- POST /api/local-model/use switches to a checked model: a job of the
  two steps verify and connect. The stored digest is compared before
  anything is loaded (model-changed otherwise). Nothing is downloaded.
- DELETE /api/local-model/models deletes one model by its name, through
  the runtime's own delete, never by a path, never the model in use.
- Adding and switching answer 409 sessions-open while a live session
  uses the endpoint. Until such a job ends installed, a session start is
  refused with the new code job-unfinished.
- The view carries library (the state's entries joined with what the
  running runtime lists: ready, changed, missing, unverified) and store
  (the size of the blob folder). Both are cached and refreshed in the
  background, never read on the one-second poll.
- With nothing installed, remove is offered while the store still holds
  models an earlier Remove kept.

Window
- The installed screen lists the models on this Mac with the context
  each was checked with, its state and its size on disk, with Use and
  Delete per row (Delete asks once, in the row) and Add a model.
- In the chooser a model that is on this Mac already says so and is
  switched to without a download or a confirm screen.
- A paused or failed job that has a checked model beside it shows the
  way back to that model.

Every route is admin only in multi-user mode, like the existing ones.

Not run on a Mac: the tests use fakes and the fake runtime, and the
listing, store size and delete were checked against a real Ollama 0.13.0
on Linux, not the pinned 0.40.15 build for macOS.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
 2026-11-10 23:52:14 +01:00
Codeman maintainer 2cd3cbd6a0 fix(local-model): count what llama-server keeps beside the K/V cache, and bound its checkpoints 
The llama.cpp memory rule of the catalogue left out two allocations that the llama.cpp the
runtime bundles (b12111, the runtime's LLAMA_CPP_VERSION) makes for every GGUF row:

- Context checkpoints. For a model whose state cannot be rolled back (the recurrent state of
  the Qwen-family rows, the sliding window of North Mini Code), llama-server keeps host copies
  of that state, two or three per request, up to 32 per slot by default
  (tools/server/server-context.cpp create_checkpoint). For the 27B that is 32 x 157 MB = 5 GB,
  none of which was counted. The runtime now sets LLAMA_ARG_CTX_CHECKPOINTS=8 (read by
  llama-server from its environment like LLAMA_ARG_CACHE_RAM; 8 was llama.cpp's own default
  before it went to 32), and every llamacpp row counts 8 copies of its state.
- Multi-token prediction. The two -mtp tags and qwen3.8:27b-q8_0 carry a prediction layer
  (nextn_predict_layers 1, read from the first bytes of each GGUF; the 9B tags and North Mini
  Code have none). The runtime then always passes --spec-type draft-mtp --spec-draft-n-max 4,
  so llama.cpp keeps 4 rollback copies of the recurrent state (need_n_rs_seq(),
  llama-memory-recurrent.cpp) and the prediction layer's cache in the draft context's cache
  type, f16 by default. The rows counted one state and that layer at q8_0.

What moves (test/local-model-recommend.test.ts, "every memory size", re-pinned):

- 16 GB: limited -> none. MiMo 9B at 65,536 tokens needs 10.2 GB, 0.987 of the 10.4 GB
  estimate. With the budget the Mac mini measured (10.7 GB after the reserve) it is offered,
  0.954, tight; that test is kept. With 4 checkpoints instead of 8 it would be 0.966 on the
  estimate: this size sits on the edge, and only a run on such a Mac settles it.
- 18 GB: the same pick, 0.964 instead of 0.928.
- 36 GB: the same picks, 0.966 (27B on M4/M5 Max) and 0.976 (35B-A3B GGUF on M3).
- 48 GB on a Max chip: the 27B at 262,144 tokens is tight (0.853) instead of good.
- 32 GB with a measured budget of 0.74: North Mini Code at 131,072 tokens (was 262,144), the
  27B at 65,536 on a Max chip (was 131,072).
- Every MLX row, and everything from 64 GB up, is unchanged.

test/local-model-catalog.test.ts recomputes the new rule from the architecture table and holds
the catalogue to LLAMA_ARG_CTX_CHECKPOINTS. The catalogue header, runtime.ts, the types,
docs/local-models.md, CLAUDE.md and architecture-invariants say the same. Read in the source,
not measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-08 10:26:57 +01:00
Codeman maintainer 989fcd778a feat(local-model): a catalogue for every Mac from 8 to 512 GB, each row with its own figures 
The catalogue had seven Qwen-family rows that all carried the memory
figures measured on one model (Qwen3.6 35B-A3B on the MLX runner), so a
9B or a dense 27B was judged with figures that do not describe it, two
rows had the wrong Mac download size, and every Mac below 32 GB got
"none".

Now eight rows, each with the runner a Mac really uses for its tag (a new
`runner` field: `mlx` or `llamacpp`) and memory figures of its own:

- the 35B-A3B MLX row keeps its measured figures and the tested pair;
- every llama.cpp row is computed from its model's config.json: a q8_0
  K/V cache for the whole context (what the runtime now sets), its
  recurrent or sliding-window state, 1.0 GB of runner and the 1 GiB host
  prompt cache;
- the one other MLX row (Ornith 1.5 122B) is computed the same way,
  scaled by what the measured row showed, plus the runner's 8 GiB of
  prefix snapshots.

Sizes are the registry's Mac pull totals (2026-11-08, lowercase ids
verified there). What each memory size gets: 8 GB none; 16 to 32 GB a 9B
(limited); 36 GB Qwen3.8 27B or the 35B-A3B as GGUF; 48 GB the tested
pair (or the 27B on a Max chip); 64 and 96 GB the 27B or the 35B-A3B;
128 to 512 GB Ornith 1.5 122B. Nothing but the one row was measured.

The catalogue test recomputes every computed figure from a table of
architectures and lets two rows share figures only when they share the
architecture and the runner. The rule tests of recommend() now run on a
frozen copy of the 2026-11-06 catalogue, so a weekly catalogue update
changes only the per-memory table and the real-Mac tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-08 09:33:48 +01:00
Codeman maintainer eb91279a3b feat(local-model): a memory map and real colour in the chooser, the button first 
The owner's verdict on the chooser: nice, but not nice enough, too much just
blue. He also wants "Run a local model" on top of the start screen, above
Run Claude Code.

The memory bar is now a map: the model, its context and the brief peaks side
by side on the track of what this Mac can give a model, each with its size in
a legend, and the verdict on the fit beside the figure. recommend() sends what
the need is made of as `Candidate.memory`.

Colour now carries meaning: green for good and tested, amber for tight, orange
for slow, teal for the context. Violet is the feature's own colour, on its
start screen button and its main action. Every row has a size tile and a speed
gauge scaled to the fastest row in the list. The light skins get deeper values
of the same colours, and the window is 40 px taller to hold it without
scrolling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-07 03:21:35 +01:00
Codeman maintainer 77d29d4621 feat(local-model): one frame for the window, honest figures, a table for the consent 
The window is one frame for the whole flow: a fixed height on desktop, App
Settings' boxed close button, the buttons in a foot of their own outside
the part that scrolls, so a click never moves the next button away and
nothing is cut off; the short screens are centred in it on desktop and sit
above the buttons on a phone.

The chooser lists the pick first and then the other models by download
size, largest first, a column the reader can see. A pick that only just
fits says "Best fit, tight". The memory figure is an estimate and says so
once, in whole GB; its legend names the peak, and the installed line uses
the same words beside what the model takes now, so the two figures are not
read as a contradiction. The legend, the column headers and the foot's
promise are set in --text-dim, the bar's track in --text-muted so its free
part can be seen, and only the download figure keeps tabular figures.

The consent screen is the download as a table: the chosen row, the runtime,
the coding agent with its installer's command as the row's second line, the
total in the strong weight, then where it goes in one line. The download bar
means the model download alone and is gone once the model is in; the screen
that ends an install shows "Start a session" alone; an error on an installed
model is the headline with a grey dot; an error's first sentence is the
headline; Remove is quiet everywhere but red in its own question. The
no-recommendation reasons are one sentence each, and the untested note is
in Codeman's voice. A refusal the server names is printed from its sentence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 23:55:02 +01:00
Codeman maintainer 492df776d0 feat(local-model): judge the fit on the measured peak and rank a tested Mac first 
The catalogue rows carry the figures measured on a Mac (the MLX build of
qwen3.6:35b-a3b is 23,105,798,956 bytes, about 37 KB held per context token,
about 3 GB of runner overhead, about 5 GB more for a moment while a long
prompt is processed). A model's need is now its peak. Rows that were not
measured use the same per-token and overhead figures and say `estimated`.

qwen3.6:35b-a3b gets the tested entry for an Apple M5 Pro with 48 GB and
OpenCode, with its measured 67.7 tokens per second, so that row may say
"Tested on a Mac like this".

The ranking changes with it: a pair tested on a Mac like this one comes
first, then speed (ok, unknown, slow), then model rank, harness rank and the
larger context. The fit grade decides whether a row is listed and what it
says, it no longer sorts, so a model is offered at the largest context it
fits at. recommend() takes the budget the runtime measured
(`budgetSource: measured`), and modelFitsBudget() judges one model against it.

Candidate rows now always carry `needSource`, `harnesses`, `installBytes`,
`installGb` and, where one applies, `tag`. `kvBytesPerToken` and the prompt
cache tiers are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 22:16:37 +01:00
Codeman maintainer 771072c36b feat(local-model): contract for the install stage (types, stubs, fake runtime) 
The types, the module skeletons and the test fake that let the runtime, the
install job and the window be built at the same time.

Types:
- src/types/local-model-install.ts (new): the runtime pin, the install
  states, steps and error codes with one sentence and one next step each,
  the progress, the `install` object of GET /api/local-model, the bodies and
  409 answers of the install verbs, the state file. `LocalModelStatus` moves
  here; `install` is optional until the route emits it.
- src/types/local-model.ts: the catalogue row's Mac memory figures, the
  measured speed of a tested entry, the `measured` budget source,
  `RecommendOptions`, and the row tag, the coding agents and the install
  total of a candidate. All optional until the catalogue and `recommend()`
  carry them, so the read-only stage keeps compiling and its tests stay
  green.

Modules with final signatures and bodies that throw:
runtime.ts (the compiled pin is final), ollama-client.ts, developer-tools.ts,
state-store.ts, install-job.ts (the state, step and transition tables are
final), connect.ts, custom-model-hosts-writer.ts, cli-install-runner.ts.
Nothing imports them yet, so the server is unchanged.

Implemented: paths.ts (every folder of the feature, from dataPath()), and
`modelFitsBudget()` in recommend.ts, the judgement the job makes with the
measured budget before it downloads a model. `recommend()` takes the options
parameter and does not read it yet.

Test fake: test/fixtures/fake-ollama.ts, a runtime on a port the system
picks, in the shapes measured on a Mac: version, tags, ps, the pull stream
(MLX and GGUF lines, an error line, a cut stream, a resumed pull), chat with
and without a tool call, the model list, a runtime archive with Range
support, and a fake child process that logs the device total.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 20:05:15 +01:00
Codeman maintainer 8281045cf2 fix(local-model): say what is true of the Mac, and why a model is missing 
Review fixes for the read-only recommendation and its card.

Recommendation (pure):
- `limited` is now a statement about the Mac: no model with 12 billion
  parameters or more fits. It is judged on everything that fits, before
  the disk filter and whatever the pick is. A 32 or 36 GB Mac whose pick
  is a 9B model no longer reads "This Mac can run a small model" above a
  list of 20B, 27B and 35B models, and knowing the chip no longer lowers
  the verdict. A small pick under `ok` carries a note that says so.
- A model that fits the memory but has no room on the disk is no longer
  dropped in silence: the best-ranked one is named in a note with the GB
  that are missing. A full disk cannot turn a 48 GB Mac into a
  small-model Mac.
- "Models fit, but not at a context a harness needs" has its own
  sentence with the budget and the need, worded as an estimate. The
  16 GB Mac mini misses by 1.3 points (10.3 of 10.4 GB) and was told
  flatly that its memory is too little, the sentence of an 8 GB Mac.
- The installed memory is named by the size the Mac is sold as (48, not
  51.5): the hardware line and the memory sentence. Every computed size
  stays decimal GB from the one formatter. An unknown memory size is no
  longer reported as "0.0 GB".
- Slow is graded on the unrounded speed: 9.95 to 9.9996 tokens per
  second printed as 10 and read ok.
- The disk sentence names the 10 GiB that must stay free, so its
  numbers add up for the reader.

Card and Settings group:
- The Settings group "Model on this Mac" is hidden on a server that is
  not a Mac, and until the first answer says it is one.
- Only `403 FORBIDDEN` is a refusal. `403 PASSWORD_CHANGE_REQUIRED`
  (an admin who must change the password) is a failed request and is
  asked again, so that admin no longer loses the card until a reload.
- Each fit and speed label carries what it measures as its tooltip and
  accessible name, an alternative with a speed estimate prints it, and
  the count beside "Other models that fit" is gone (it was the number
  of rows, capped at four, not the number of models that fit).
- The line saying nothing can be set up yet sits right under the pick,
  in the colour of every other line (it was 3.9:1), and says whose step
  it is.
- The dismissal key is named for this stage, so hiding the read-only
  card does not hide the install offer later.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 17:37:53 +01:00
Codeman maintainer 4739542397 chore(local-model): drop build-time wording from two comments 
The label list was "frozen for the parallel build" and the card's test said
the route answers 501: both were true while the pieces were written side by
side and are not any more. Comments only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 16:30:59 +01:00
Codeman maintainer 92c20fe572 feat(local-model): hardware probe, model catalogue and the pure recommendation 
The core of the read-only recommendation: what this Mac is, what the catalogue
knows, and which model fits. Nothing is downloaded, installed or written.

- hardware.ts: one bounded async spawn of /usr/sbin/sysctl with seven keys,
  without -n, parsed as "key: value" lines by the pure parseSysctl(). Free disk
  of the data dir through statfs behind the bounded path probe. Fallbacks from
  node:os, a 60 s cache, never throws, never system_profiler. A wired limit of
  0 means "not set".
- local-model-catalog.ts: seven models, three harness rows (pi gated) and the
  chips table as a compiled constant, validated by a Zod schema on first use.
  No override file. Model ids are name:tag in lowercase without a slash; every
  model carries paramsB and an (empty) tested list. Chip names match on the
  end of the chip string, so 'M4' is not the chip of 'Apple M4 Pro'.
- recommend.ts: device budget (the wired limit when above 0, else 2/3 of
  memory below 36 GB and 3/4 from 36 GB, less the reserve), need = weights +
  KV cache + 1 GiB + prompt cache, fit good up to 0.85 and tight up to 0.98,
  speed ok, unknown or slow (a chip in no chips row is unknown), and the
  ranking. Verdicts ok, limited (best model under 12 billion parameters) and
  none with its reason. "Recommended" only for a pick with a tested entry for
  this Mac, which no row has today.
- types/local-model.ts: comments brought in line with the code (chip matching,
  budgetShare rounding, a budget of 0 on an Intel Mac).
- cli-registry-no-id-branching.test.ts: the guard also catches a comparison of
  a `cli` field with a CLI id (plus one stray blank line Prettier refused).

Tests: test/local-model-format.test.ts, test/local-model-hardware.test.ts
(the captured output of two real Macs), test/local-model-catalog.test.ts,
test/local-model-recommend.test.ts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 16:27:33 +01:00
Codeman maintainer 73ce6a1819 feat(local-model): types and module skeleton for the hardware-aware recommendation 
The contract three builders code against in parallel. Nothing is recommended
yet: the route answers 501 and the catalogue is empty.

- src/types/local-model.ts: HardwareFacts, the catalogue rows, HarnessFact,
  Candidate, Recommendation, LocalModelStatus, and LOCAL_MODEL_LABELS, the one
  list of sentences the server may put into the payload (the zh-CN dictionary
  is keyed by them).
- src/local-model/format.ts: formatGb, the one place a byte count becomes a
  size a person reads (decimal GB), and memClassGb.
- src/config/local-model-catalog.ts: the Zod schema (model ids are name:tag,
  lowercase, no slash) and an empty stock catalogue. No override file.
- Skeletons with their final signatures: hardware.ts, recommend.ts,
  harness-facts.ts, routes/local-model-routes.ts (injectable dependencies),
  public/local-model-ui.js with its script tag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-11-06 14:56:13 +01:00