diff --git a/docs/wiki/Codeman-Skill.md b/docs/wiki/Codeman-Skill.md index 864d141b..1df61055 100644 --- a/docs/wiki/Codeman-Skill.md +++ b/docs/wiki/Codeman-Skill.md @@ -138,6 +138,8 @@ from your side. See [Agent CLIs](Agent-CLIs) for the run modes. Claude Code lead can hand work to it: Claude in the cloud plans and reviews, the local model takes summaries, first drafts and files that should stay on the machine. It answers one request at a time and is slower than a cloud model, and the skill never installs one for you. +You can name the coding agent the worker runs with ("a local worker with Pi"), as long as +that agent is installed; the walkthrough is in [Local Models](Local-Models#cloud-and-local-together). With Claude Code workers, where the installed Claude Code supports it, the lead can also message a worker directly instead of typing into its terminal. diff --git a/docs/wiki/Custom-Model-Endpoints.md b/docs/wiki/Custom-Model-Endpoints.md index fee21861..5ed206ae 100644 --- a/docs/wiki/Custom-Model-Endpoints.md +++ b/docs/wiki/Custom-Model-Endpoints.md @@ -207,7 +207,9 @@ missing entry is the more current answer. ## A model that runs on this Mac If the Codeman server runs on a Mac with Apple silicon, you do not need a server of your -own: Codeman can set one model up for you on that Mac. This is experimental. +own: Codeman can set one model up for you on that Mac. This is experimental. The walkthrough, +with the coding agents and the cloud plus local setup, is on its own page: +[Local Models](Local-Models). 1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model on this Mac). @@ -239,6 +241,10 @@ Good to know: downloads another one, **Use** switches to a model that is already there (nothing is downloaded; close the sessions on the model first), and **Delete** removes one, after one question. The model in use cannot be deleted: switch first, or use **Remove**. +- The coding agent is chosen per session. The window lists every agent the model can be run + with (OpenCode, Claude Code, Pi, DeepSeek Harness) under **Start a session with**, and + installs one that is missing after showing you the command. DeepSeek Harness is set up + from the Run menu first. - **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The coding agent stays installed. - If the model on disk is no longer the one that was installed, Codeman refuses to start a diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 0efad3c4..f525a137 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -66,6 +66,7 @@ codeman web # then open http://localhost:3000 | [Tile Grid](Tile-Grid) | How do I watch and drive several sessions side by side? | | [Agent CLIs](Agent-CLIs) | Which agent should this session run, and how do I set it up? | | [Custom Model Endpoints](Custom-Model-Endpoints) | How do I point a session at my own OpenAI-compatible endpoint? | +| [Local Models](Local-Models) | How do I run a coding model on my own Mac, and use it next to a cloud model? | | [Working With Files](Working-With-Files) | How do I read, edit, and attach files? | | [Git Worktrees](Git-Worktrees) | How do several agents work on one repository without sharing a folder? | | [Input And Voice](Input-And-Voice) | How do I talk to an agent, including by voice? | diff --git a/docs/wiki/Local-Models.md b/docs/wiki/Local-Models.md new file mode 100644 index 00000000..56cfa946 --- /dev/null +++ b/docs/wiki/Local-Models.md @@ -0,0 +1,114 @@ +# Local Models + +Run a coding model on your own Mac, next to the cloud models you already use. Codeman +installs the model, keeps it running, and lets you start sessions on it or hand it work from +a cloud session. No account and no API key: the model runs only on that Mac. + +This is **experimental**, and it needs a Mac with Apple silicon as the machine the Codeman +server runs on. On a machine that is not a Mac the button stays hidden. + +## Install a model + +1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model + on this Mac). +2. The window shows which models fit this Mac. The bar at the top is the memory the Mac can + give a model, filled to what the selected model needs at its peak. A nearly full bar is a + tight fit. +3. Pick a model, and under **Runs with** the coding agent to start it with. +4. Press **Install on this Mac**. The next screen lists every download with its size and + where it goes. Nothing is downloaded before you confirm there. +5. Codeman downloads a model runtime and the model into `~/.codeman/local-model`, loads the + model, checks that all of it runs on the GPU and that it can call a tool, and installs + the coding agent if it is missing. No administrator password is asked for. +6. Press **Start a session**. + +You can close the window while it downloads. **Cancel** pauses the install and **Continue** +picks it up where it stopped. After a Codeman restart an unfinished install waits for your +click. + +## Pick the coding agent + +A model is driven by a coding agent: the program that reads your prompt, calls tools and +edits files. You choose it **per session**. Open the window again and look at **Start a +session with**: one chip per agent the model can be run with. + +| Agent | Pick it when | +| ---------------- | ---------------------------------------------------------------------------------------------- | +| OpenCode | You have no reason to choose. It is the pair Codeman was tested with on a Mac | +| Pi | You want the first answer soon. Its fixed prompt is the smallest by far, which also leaves the most room for your work | +| Claude Code | You want Claude Code's own tools and habits. Its first answer on a local model takes minutes | +| DeepSeek Harness | You already use it. It reports when a turn has ended, which helps when another session drives it | + +Select a chip and press **Start a session**. The choice is remembered on that device. + +**A chip that says "Not installed"** is an agent Codeman can install for you. Select it: the +window shows the exact command it would run, and the main button becomes **Install** with +the agent's name. The model stays ready while the installer runs, and you can start a +session with another agent meanwhile. If the install fails, the window says so under the +chip and nothing else changes. + +**A chip that says "Needs setup"** is an agent that needs more than one command. DeepSeek +Harness is the one today: set it up from the **Run** menu first, then it can be started +here. + +## More than one model + +Once a model is installed, the window lists **Models on this Mac** with what each takes on +disk, and the total under the list. + +- **Add a model** opens the list of models again and downloads another one. When it is + ready it becomes the model in use; the earlier one stays on the Mac. +- **Use** switches to a model that is already there. Nothing is downloaded: Codeman loads it + and checks it again. +- **Delete** removes one model, after one question that says how much space it frees. + +One model runs at a time. Adding and switching are refused while sessions still use the +current model: close those sessions first. The model in use cannot be deleted: switch first, +or use **Remove**, which takes the runtime and the connection away (and the downloaded +models too, if you tick that). + +## Cloud and local together + +The reason to keep a local model next to a cloud one: the cloud model plans and reviews, the +Mac does the bulk. From a Claude Code session in Codeman, with the +[Codeman skill](Codeman-Skill) (on by default), you can just say it: + +| You say | What happens | +| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | +| "Is there a local model on this Mac?" | It reads what is installed, which agents can run it and which models are on the Mac. Read-only | +| "Use the local model to summarize every file in `docs/`, then check its summaries." | A worker on the local model does the reading; the cloud session reads its answer back and reviews it | +| "Spawn a local worker with Pi to draft tests for `src/parser.ts`." | The worker runs the local model through Pi | +| "Start two cloud workers for the API and the UI, and a local one for the changelog." | One mixed fleet: cloud workers side by side, the Mac as a queue | + +The worker sessions appear as tabs in the dashboard and are closed when the work is done. + +Good to know before you rely on it: + +- **One request at a time.** The local model answers one request at a time, so two local + workers take turns. Fan out to the cloud, and give the Mac a queue. +- **Slower than a cloud model**, and the first request after a start loads the model first. +- **Review what comes back.** A local model is good at bulk work and weaker at judgment. + Treat its answer as a first draft. +- **The skill never installs anything.** A model or an agent that is missing is yours to + install in the window; the skill tells you what is missing and stops. +- **While a local worker is open**, the window refuses Stop, Remove and a switch of models, + and names the sessions that are in the way. + +## Good to know + +- **What was tested.** One pair was run on a real Mac: OpenCode with Qwen3.6 35B-A3B on an + M5 Pro with 48 GB. The window marks that pair on a Mac like it. Everything else in the + list is calculated from each model's own figures, and the install checks the model on your + Mac before it lets you start a session. +- **Memory first.** A model that does not fit in memory is not offered. An estimate becomes + a measurement the first time the runtime starts, and the model is judged against that + before it is downloaded. +- **The developer tools.** Coding agents use git and Python. On a Mac without them, macOS + offers to install them the first time an agent needs one. +- **Stop** keeps the runtime from starting with Codeman next time. **Start** undoes that. +- **In multi-user mode** only an admin sees and uses this. + +More detail: [A model that runs on this Mac](Custom-Model-Endpoints#a-model-that-runs-on-this-mac) +for how it relates to your own model servers, and +[`docs/local-models.md`](https://github.com/Ark0N/Codeman/blob/master/docs/local-models.md) +for how it works inside. diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 36a04909..dc1ebeb4 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -14,6 +14,7 @@ - [Tile Grid](Tile-Grid) - [Agent CLIs](Agent-CLIs) - [Custom Model Endpoints](Custom-Model-Endpoints) +- [Local Models](Local-Models) - [Working With Files](Working-With-Files) - [Git Worktrees](Git-Worktrees) - [Input And Voice](Input-And-Voice)