From 452f7c41dda2d2f50c1c820c6d439a88d2c0a67d Mon Sep 17 00:00:00 2001 From: Codeman maintainer Date: Wed, 11 Nov 2026 10:13:57 +0100 Subject: [PATCH] docs(wiki): Local Models, the walkthrough from install to cloud plus local A page of its own in the user manual: install a model, pick the coding agent per session (and install one that is missing), keep more than one model on the Mac, and hand work to the local model from a cloud session with the codeman skill, with the prompts to say and the limits to know. Linked from the sidebar and the Home index, and from the two pages that already mention the local model (Custom Model Endpoints, Codeman Skill). Co-Authored-By: Claude Fable 5.2 --- docs/wiki/Codeman-Skill.md | 2 + docs/wiki/Custom-Model-Endpoints.md | 8 +- docs/wiki/Home.md | 1 + docs/wiki/Local-Models.md | 114 ++++++++++++++++++++++++++++ docs/wiki/_Sidebar.md | 1 + 5 files changed, 125 insertions(+), 1 deletion(-) create mode 100644 docs/wiki/Local-Models.md diff --git a/docs/wiki/Codeman-Skill.md b/docs/wiki/Codeman-Skill.md index 864d141b..1df61055 100644 --- a/docs/wiki/Codeman-Skill.md +++ b/docs/wiki/Codeman-Skill.md @@ -138,6 +138,8 @@ from your side. See [Agent CLIs](Agent-CLIs) for the run modes. Claude Code lead can hand work to it: Claude in the cloud plans and reviews, the local model takes summaries, first drafts and files that should stay on the machine. It answers one request at a time and is slower than a cloud model, and the skill never installs one for you. +You can name the coding agent the worker runs with ("a local worker with Pi"), as long as +that agent is installed; the walkthrough is in [Local Models](Local-Models#cloud-and-local-together). With Claude Code workers, where the installed Claude Code supports it, the lead can also message a worker directly instead of typing into its terminal. diff --git a/docs/wiki/Custom-Model-Endpoints.md b/docs/wiki/Custom-Model-Endpoints.md index fee21861..5ed206ae 100644 --- a/docs/wiki/Custom-Model-Endpoints.md +++ b/docs/wiki/Custom-Model-Endpoints.md @@ -207,7 +207,9 @@ missing entry is the more current answer. ## A model that runs on this Mac If the Codeman server runs on a Mac with Apple silicon, you do not need a server of your -own: Codeman can set one model up for you on that Mac. This is experimental. +own: Codeman can set one model up for you on that Mac. This is experimental. The walkthrough, +with the coding agents and the cloud plus local setup, is on its own page: +[Local Models](Local-Models). 1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model on this Mac). @@ -239,6 +241,10 @@ Good to know: downloads another one, **Use** switches to a model that is already there (nothing is downloaded; close the sessions on the model first), and **Delete** removes one, after one question. The model in use cannot be deleted: switch first, or use **Remove**. +- The coding agent is chosen per session. The window lists every agent the model can be run + with (OpenCode, Claude Code, Pi, DeepSeek Harness) under **Start a session with**, and + installs one that is missing after showing you the command. DeepSeek Harness is set up + from the Run menu first. - **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The coding agent stays installed. - If the model on disk is no longer the one that was installed, Codeman refuses to start a diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 0efad3c4..f525a137 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -66,6 +66,7 @@ codeman web # then open http://localhost:3000 | [Tile Grid](Tile-Grid) | How do I watch and drive several sessions side by side? | | [Agent CLIs](Agent-CLIs) | Which agent should this session run, and how do I set it up? | | [Custom Model Endpoints](Custom-Model-Endpoints) | How do I point a session at my own OpenAI-compatible endpoint? | +| [Local Models](Local-Models) | How do I run a coding model on my own Mac, and use it next to a cloud model? | | [Working With Files](Working-With-Files) | How do I read, edit, and attach files? | | [Git Worktrees](Git-Worktrees) | How do several agents work on one repository without sharing a folder? | | [Input And Voice](Input-And-Voice) | How do I talk to an agent, including by voice? | diff --git a/docs/wiki/Local-Models.md b/docs/wiki/Local-Models.md new file mode 100644 index 00000000..56cfa946 --- /dev/null +++ b/docs/wiki/Local-Models.md @@ -0,0 +1,114 @@ +# Local Models + +Run a coding model on your own Mac, next to the cloud models you already use. Codeman +installs the model, keeps it running, and lets you start sessions on it or hand it work from +a cloud session. No account and no API key: the model runs only on that Mac. + +This is **experimental**, and it needs a Mac with Apple silicon as the machine the Codeman +server runs on. On a machine that is not a Mac the button stays hidden. + +## Install a model + +1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model + on this Mac). +2. The window shows which models fit this Mac. The bar at the top is the memory the Mac can + give a model, filled to what the selected model needs at its peak. A nearly full bar is a + tight fit. +3. Pick a model, and under **Runs with** the coding agent to start it with. +4. Press **Install on this Mac**. The next screen lists every download with its size and + where it goes. Nothing is downloaded before you confirm there. +5. Codeman downloads a model runtime and the model into `~/.codeman/local-model`, loads the + model, checks that all of it runs on the GPU and that it can call a tool, and installs + the coding agent if it is missing. No administrator password is asked for. +6. Press **Start a session**. + +You can close the window while it downloads. **Cancel** pauses the install and **Continue** +picks it up where it stopped. After a Codeman restart an unfinished install waits for your +click. + +## Pick the coding agent + +A model is driven by a coding agent: the program that reads your prompt, calls tools and +edits files. You choose it **per session**. Open the window again and look at **Start a +session with**: one chip per agent the model can be run with. + +| Agent | Pick it when | +| ---------------- | ---------------------------------------------------------------------------------------------- | +| OpenCode | You have no reason to choose. It is the pair Codeman was tested with on a Mac | +| Pi | You want the first answer soon. Its fixed prompt is the smallest by far, which also leaves the most room for your work | +| Claude Code | You want Claude Code's own tools and habits. Its first answer on a local model takes minutes | +| DeepSeek Harness | You already use it. It reports when a turn has ended, which helps when another session drives it | + +Select a chip and press **Start a session**. The choice is remembered on that device. + +**A chip that says "Not installed"** is an agent Codeman can install for you. Select it: the +window shows the exact command it would run, and the main button becomes **Install** with +the agent's name. The model stays ready while the installer runs, and you can start a +session with another agent meanwhile. If the install fails, the window says so under the +chip and nothing else changes. + +**A chip that says "Needs setup"** is an agent that needs more than one command. DeepSeek +Harness is the one today: set it up from the **Run** menu first, then it can be started +here. + +## More than one model + +Once a model is installed, the window lists **Models on this Mac** with what each takes on +disk, and the total under the list. + +- **Add a model** opens the list of models again and downloads another one. When it is + ready it becomes the model in use; the earlier one stays on the Mac. +- **Use** switches to a model that is already there. Nothing is downloaded: Codeman loads it + and checks it again. +- **Delete** removes one model, after one question that says how much space it frees. + +One model runs at a time. Adding and switching are refused while sessions still use the +current model: close those sessions first. The model in use cannot be deleted: switch first, +or use **Remove**, which takes the runtime and the connection away (and the downloaded +models too, if you tick that). + +## Cloud and local together + +The reason to keep a local model next to a cloud one: the cloud model plans and reviews, the +Mac does the bulk. From a Claude Code session in Codeman, with the +[Codeman skill](Codeman-Skill) (on by default), you can just say it: + +| You say | What happens | +| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | +| "Is there a local model on this Mac?" | It reads what is installed, which agents can run it and which models are on the Mac. Read-only | +| "Use the local model to summarize every file in `docs/`, then check its summaries." | A worker on the local model does the reading; the cloud session reads its answer back and reviews it | +| "Spawn a local worker with Pi to draft tests for `src/parser.ts`." | The worker runs the local model through Pi | +| "Start two cloud workers for the API and the UI, and a local one for the changelog." | One mixed fleet: cloud workers side by side, the Mac as a queue | + +The worker sessions appear as tabs in the dashboard and are closed when the work is done. + +Good to know before you rely on it: + +- **One request at a time.** The local model answers one request at a time, so two local + workers take turns. Fan out to the cloud, and give the Mac a queue. +- **Slower than a cloud model**, and the first request after a start loads the model first. +- **Review what comes back.** A local model is good at bulk work and weaker at judgment. + Treat its answer as a first draft. +- **The skill never installs anything.** A model or an agent that is missing is yours to + install in the window; the skill tells you what is missing and stops. +- **While a local worker is open**, the window refuses Stop, Remove and a switch of models, + and names the sessions that are in the way. + +## Good to know + +- **What was tested.** One pair was run on a real Mac: OpenCode with Qwen3.6 35B-A3B on an + M5 Pro with 48 GB. The window marks that pair on a Mac like it. Everything else in the + list is calculated from each model's own figures, and the install checks the model on your + Mac before it lets you start a session. +- **Memory first.** A model that does not fit in memory is not offered. An estimate becomes + a measurement the first time the runtime starts, and the model is judged against that + before it is downloaded. +- **The developer tools.** Coding agents use git and Python. On a Mac without them, macOS + offers to install them the first time an agent needs one. +- **Stop** keeps the runtime from starting with Codeman next time. **Start** undoes that. +- **In multi-user mode** only an admin sees and uses this. + +More detail: [A model that runs on this Mac](Custom-Model-Endpoints#a-model-that-runs-on-this-mac) +for how it relates to your own model servers, and +[`docs/local-models.md`](https://github.com/Ark0N/Codeman/blob/master/docs/local-models.md) +for how it works inside. diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 36a04909..dc1ebeb4 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -14,6 +14,7 @@ - [Tile Grid](Tile-Grid) - [Agent CLIs](Agent-CLIs) - [Custom Model Endpoints](Custom-Model-Endpoints) +- [Local Models](Local-Models) - [Working With Files](Working-With-Files) - [Git Worktrees](Git-Worktrees) - [Input And Voice](Input-And-Voice)