mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-14 10:39:32 +01:00
Merge branch 'feat/local-model-recommendation' into test/desktop-macbook
# Conflicts: # config/test-suites.ts
This commit is contained in:
commit
1e097bc368
74 files changed
+25355
-2118
No files matched your search
@@ -204,6 +204,45 @@ Which harnesses show up in the Run-menu picker is read live off Codeman's own CL
|
||||
not a fixed list here, so this table can go stale before this page does — a greyed-out or
|
||||
missing entry is the more current answer.
|
||||
|
||||
## A model that runs on this Mac
|
||||
|
||||
If the Codeman server runs on a Mac with Apple silicon, you do not need a server of your
|
||||
own: Codeman can set one model up for you on that Mac. This is experimental.
|
||||
|
||||
1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model
|
||||
on this Mac).
|
||||
2. The window shows what fits this Mac. A bar shows how much of the memory the selected
|
||||
model takes at its full context; a nearly full bar means a tight fit. Pick a model and
|
||||
the coding agent to run it with.
|
||||
3. Press **Install on this Mac**, check the sizes and where they go, and confirm. Nothing is
|
||||
downloaded before this click.
|
||||
4. Codeman downloads a model runtime (Ollama, about 186 MB) and the model into
|
||||
`~/.codeman/local-model`, loads the model, checks that all of it runs on the GPU and that
|
||||
it can call a tool, installs the coding agent if it is missing, and saves an endpoint
|
||||
named **This Mac**. No administrator password is asked for.
|
||||
5. Press **Start a session**.
|
||||
|
||||
Good to know:
|
||||
|
||||
- You can close the window: the download continues while Codeman is running. **Cancel**
|
||||
pauses it and **Continue** picks it up where it stopped. After a Codeman restart an
|
||||
unfinished install waits for your click.
|
||||
- The memory figure is an estimate until the runtime has started once. From then on it is
|
||||
what the runtime itself measured, and the model is judged against that before it is
|
||||
downloaded.
|
||||
- The endpoint "This Mac" belongs to this install. It is in the list and in the Run menu,
|
||||
but it is changed and removed in the local model window, not here.
|
||||
- Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from
|
||||
starting next time.
|
||||
- **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The
|
||||
coding agent stays installed.
|
||||
- If the model on disk is no longer the one that was installed, Codeman refuses to start a
|
||||
session on it and tells you to remove and install it again.
|
||||
- Any program on this Mac, and any web page served from `localhost`, can reach the model
|
||||
runtime: its port has no login. It listens on this Mac only.
|
||||
- On a Mac without Apple's developer tools, macOS will offer to install them the first time
|
||||
a coding agent runs `git` or Python. The window tells you when that applies.
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **No remote or Docker sessions yet.** Both restart their agent differently under the hood
|
||||
|
||||
@@ -193,19 +193,25 @@ attached.
|
||||
section to the Run dropdown, for pointing a harness at your own OpenAI-compatible server
|
||||
instead of its native cloud backend. See [Custom Model Endpoints](Custom-Model-Endpoints).
|
||||
|
||||
**Model on this Mac** is not a setting: it is a read-only answer. When the Codeman server
|
||||
runs on a Mac with Apple silicon, it shows that Mac's chip, memory and free disk space, the
|
||||
local coding model that fits it best with the agent CLI to run it, what the model needs of
|
||||
the memory the Mac can give one, its download size and an estimated speed, and the other
|
||||
models that fit (Good or Tight for the memory, Slow or Speed unknown for the speed). A Mac
|
||||
that can only run a model under 12 billion parameters is told so in plain words, and a Mac
|
||||
with too little memory, too little free disk space or an Intel processor is told why nothing
|
||||
is offered. The numbers are estimates from a catalogue that ships with each release; nothing
|
||||
has been tested on your Mac. **Nothing is downloaded or installed from here yet**: setting
|
||||
the model up from Codeman is the next step and is not built. The same answer appears as a
|
||||
card on the welcome screen, which you can hide per device (it stays here). The group is not
|
||||
shown when the server runs on Linux or Windows, and in multi-user mode only an admin sees
|
||||
it. The "Mac" is always the machine the server runs on, not the device you are reading on.
|
||||
**Model on this Mac** is not a setting: it is the state of a model Codeman can run for you
|
||||
on the Mac the server runs on, with a button that opens the local model window. The same
|
||||
window opens from the **Run a local model** button on the welcome screen. It shows that
|
||||
Mac's chip, memory and free disk space, a bar for how much of the memory the selected model
|
||||
takes, and the models that fit, each with one tag (Best fit, Tight, Slow, Small model, or
|
||||
"Tested on a Mac like this"), a speed and a download size. **Install on this Mac** asks you
|
||||
to confirm the sizes once, then downloads a model runtime and the model into
|
||||
`~/.codeman/local-model`, checks that the model loads and can call a tool, installs the
|
||||
coding agent if it is missing, and ends on **Start a session**. No account, no API key and
|
||||
no administrator password. You can close the window while it downloads, cancel and continue
|
||||
later; after a Codeman restart an unfinished install waits for you to press Continue. Once a
|
||||
model is installed the window offers Start a session, Stop or Start, and Remove. Installing
|
||||
switches **Custom model endpoints** on and saves one endpoint named "This Mac"; Remove takes
|
||||
the endpoint away again and switches the setting back off if nothing else uses it. A Mac
|
||||
that can only run a small model is told so in plain words, and a Mac with too little memory,
|
||||
too little free disk space or an Intel processor is told why nothing is offered. The group
|
||||
is not shown when the server runs on Linux or Windows, and in multi-user mode only an admin
|
||||
sees it. The "Mac" is always the machine the server runs on, not the device you are reading
|
||||
on. Details: [Custom Model Endpoints](Custom-Model-Endpoints#a-model-that-runs-on-this-mac).
|
||||
|
||||
### Agents & CLIs
|
||||
|
||||
|
||||
Reference in new issue
Block a user