diff --git a/CLAUDE.md b/CLAUDE.md index 496d72b8..5ad992e3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -187,11 +187,13 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Case rename** (`POST /api/cases/:name/rename`, `src/case-rename.ts`; `docs/wiki/Core-Concepts.md#case`): a case under the cases dir is its folder name and always moves; a linked case renames its alias and moves its real folder only with `renameFolder`; remote and Docker cases are refused. ⚠️ A folder that sessions have open is the hazard: live sessions at or under it make the route answer `409 requiresStop` with their names, and only `stopSessions: true` ends them (via `ctx.cleanupSession`: closed like a tab close, uploads included, nothing is restarted); the free-target pre-check runs before that and refuses a dangling symlink too (`lstat`, as `moveCaseFolder` does), so no session is ended for a move that cannot happen. `withCaseLock` holds the old and new paths for the whole operation (`isCasePathLocked`; the `Session` constructor throws for a working directory inside a held path, so nothing is constructed mid-move). ⚠️ A create awaits the owner's layout lock and a settings read between `new Session` and the server's map, where the rename looks for open sessions, so the constructor also records the session (`noteCaseSessionStarting`) until `registerSessionWithLayout()` or `stop()` settles it (30 s TTL for one that never registers), and a folder move refuses `409` with `renameCode: "BUSY"` while one is recorded at or under the folder (`isCaseSessionStarting`, checked in the same tick that takes the lock); an alias-only rename is not refused. A create whose path was validated before the move but whose `Session` is constructed after it is not covered (the plain check-then-use race of any create path), `moveCaseFolder` refuses an existing target itself (POSIX `rename` would silently replace an EMPTY directory) and retries `EBUSY`/`EPERM`/`EACCES` before answering `423 LOCKED`. If the registries (`linked-cases.json`, `caseOrder`, `lastUsedCase`) fail after the move, `settings.json` and the folder are put back (a failed move-back says the folder is now at the new path), and an error after `stopSessions` carries `stoppedSessions`. Claude Code keys history by path (`claudeProjectDirName`: every non-alphanumeric becomes `-`), so `/projects/` is renamed too, or merged entry by entry, never overwriting, into one already at the new key (case delete never removes history, so an earlier case of the new name leaves one); other CLIs' history, Docker bind mounts, `git worktree` links, cron jobs' `workingDir`, Claude history of subfolder sessions and Claude Code's per-project entry in `~/.claude.json` (local-scope `mcpServers`, `allowedTools`, trust; writing another CLI's config is the opt-in MCP sync's job) are NOT migrated. The history scanner prefers the decoded project key over a transcript's `cwd` that no longer exists, so Resume follows a rename. Only an admin's rename touches the server-wide `settings.json` (`caseOrder`, `lastUsedCase`); the owner's default session group is carried across with `TabLayoutService`. `settings.json` is written before `linked-cases.json`, so a failed or refused registry write puts `settings.json` back and leaves the registry as it was. ⚠️ Every write to `linked-cases.json` (link, unlink, rename, and a create or clone into a custom folder) goes through `mutateLinkedCases()` (`src/web/linked-cases-writer.ts`: serialized, temp file + rename, and a file that does not parse is refused, never read as `{}`; the one direct writer is config-backup restore, `restoreConfigBackup()`, which must work on such a file), and a check that must still hold when the entry is written (the name is free; for a rename, the old entry still names the folder) runs INSIDE its callback: two clones that claimed one name used to both pass a re-read made outside it and both answer success. Pinned by `test/linked-cases-writer.test.ts`, and for a rename by `test/routes/case-rename-routes.test.ts`. -**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`): points a session at a user-configured OpenAI-compatible endpoint (store `custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`, 0600). `authStyle` is `bearer` or `api-key`, never both headers (hangs the server). The per-CLI redirect is registry data, `capabilities.customModelInjection` (`env` / `configContentEnv` / `configDir` / `unsupported`), computed by the pure `custom-model-injection.ts`; ⚠️ `configDir` writes an isolated per-session config, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config; the one exception is a recipe's `linkFromRealHome` (deepseek's `profiles`, `sessions`, `settings.yaml`), which symlinks those named entries back out of the real home so they survive the redirect. ⚠️ Claude applies via `Session.restartCli()` (a relaunch in the existing pane, so the conversation is pinned as `resumeSessionId`), the rest one-shot at launch; retired env keys must also be `setenv -u`'d (`_pendingEnvUnsets`), since tmux env survives `respawn-pane`. ⚠️ Remote/Docker sessions are refused (400). ⚠️ Persist only KEYS (`__customModel`), never values (they carry the API key); the one exception is `ownEnv`'s `homeEnvVar` entry, the session's OWN home path (absolute only: a `clis.json` override can name any variable there), which `setCustomModel()` puts back on clear and recovery clamps, since the selection overwrote the tmux `setenv` that held it. ⚠️ Every redirectable var must be in that CLI's `privilegedEnvKeys`, and `ANTHROPIC_*` stays out of claude's `allowedPrefixes`. ⚠️ The Run-menu picker builds entries from `window.__codemanCustomModelClis` (escaped via `escapeScriptJson()`), never a hardcoded CLI id list, and launches through `run()` via a temporary `_runMode` swap, never `setRunMode()`. → [architecture-invariants#custom-model-endpoint-profiles](docs/architecture-invariants.md#custom-model-endpoint-profiles) +**Custom Model Endpoint Profiles** (opt-in, `customModelEndpointsEnabled`, SYNCED, default OFF; `docs/custom-model-endpoints.md`): points a session at a user-configured OpenAI-compatible endpoint (store `custom-model-hosts.ts`, `~/.codeman/custom-model-hosts.json`, 0600). `authStyle` is `bearer` or `api-key`, never both headers (hangs the server). The per-CLI redirect is registry data, `capabilities.customModelInjection` (`env` / `configContentEnv` / `configDir` / `unsupported`), computed by the pure `custom-model-injection.ts`; ⚠️ `configDir` writes an isolated per-session config, NEVER the user's real `~/.codex`/`~/.pi`/`~/.omp`/grok config; the one exception is a recipe's `linkFromRealHome` (deepseek's `profiles`, `sessions`, `settings.yaml`), which symlinks those named entries back out of the real home so they survive the redirect. ⚠️ Claude applies via `Session.restartCli()` (a relaunch in the existing pane, so the conversation is pinned as `resumeSessionId`), the rest one-shot at launch; retired env keys must also be `setenv -u`'d (`_pendingEnvUnsets`), since tmux env survives `respawn-pane`. ⚠️ Remote/Docker sessions are refused (400). ⚠️ Persist only KEYS (`__customModel`), never values (they carry the API key); the one exception is `ownEnv`'s `homeEnvVar` entry, the session's OWN home path (absolute only: a `clis.json` override can name any variable there), which `setCustomModel()` puts back on clear and recovery clamps, since the selection overwrote the tmux `setenv` that held it. ⚠️ Every redirectable var must be in that CLI's `privilegedEnvKeys`, and `ANTHROPIC_*` stays out of claude's `allowedPrefixes`. ⚠️ The Run-menu picker builds entries from `window.__codemanCustomModelClis` (escaped via `escapeScriptJson()`), never a hardcoded CLI id list, and launches through `run()` via a temporary `_runMode` swap, never `setRunMode()`. ⚠️ Every write of `custom-model-hosts.json` goes through `mutateCustomModelHosts()` (`src/custom-model-hosts-writer.ts`: one at a time, temp file + rename, the route's check inside the callback, and a file that does not parse is refused with 409, never read as `[]` and replaced); the one direct writer is config-backup restore. The endpoint id `local-ollama` is reserved for the local model install (see Local model install). Pinned by `test/custom-model-hosts-writer.test.ts`. → [architecture-invariants#custom-model-endpoint-profiles](docs/architecture-invariants.md#custom-model-endpoint-profiles) ⚠️ **llama-swap endpoints** (one model at a time): the apply routes check `GET /running` and return `requiresConfirmation` before evicting a model another live session uses; `confirmedSwap` and `confirmedContext` are SEPARATE flags and must stay so. Claude alone gets a context floor (`CLAUDE_MIN_SAFE_CONTEXT_TOKENS`); context is parsed from `/running`'s `cmd`, never trusted from `/props`. The floor also divides that context by the model's parallel slot count (`modelSlotCounts`: `--parallel`/`-np` from `cmd`, else `/props` `total_slots`; `perSlotContext()`), and `/props` only ever RAISES that warning: it supplies the context only when `cmd` states none and the count only when `cmd` has no `--parallel`, never a larger pool. `/props` is probed once per launch command (it refreshes llama-swap's last-use time, so a `ttl` would never expire), and a `/props` context beside an explicit `--parallel N` is already per slot, so it is not divided again. Backend log lines come from llama-swap's `/api/events` `upstream` source, never `/logs`. → [architecture-invariants#custom-model-endpoint-profiles](docs/architecture-invariants.md#custom-model-endpoint-profiles) -**Local model recommendation** (`GET /api/local-model`, `src/local-model/`, `src/config/local-model-catalog.ts`, `src/types/local-model.ts`; EXPERIMENTAL, read-only stage): the server looks at the Mac it runs on and says which local coding model fits it and with which agent CLI. ⚠️ **Read-only**: it downloads nothing, installs nothing, writes no file and adds no settings key; the install flow is a later stage and nothing here may grow a write before it. Apple silicon macOS only: anywhere else the answer is a `200` with verdict `none` (reasons `not-macos`, `intel`, `memory`, `disk`, `macos`, `no-harness`), and a failed probe is an answer, never a 500. The probe (`hardware.ts`) is ONE bounded async spawn of `/usr/sbin/sysctl` WITHOUT `-n`, parsed as `key: value` lines by the pure `parseSysctl()`, plus `statfs` of the data dir (probed bounded first, #516) and `node:os` fallbacks, cached 60 s; ⚠️ never `system_profiler`, never a second spawn, and `iogpu.wired_limit_mb` of 0 means not set. The catalogue is a compiled constant validated by a Zod schema: ⚠️ no override file and no runtime data, and a model id is `name:tag` in lowercase with no slash (`LOCAL_MODEL_ID_PATTERN`). ⚠️ A harness is a catalogue ROW (`{ cli, minContext, rank, gated?, slowFirstTurn? }`) plus a `HarnessFact` built from the CLI registry at call time (`harness-facts.ts`); no code compares a CLI id (both id-branching guards scan these files). `recommend()` (`recommend.ts`) is pure. ⚠️ `limited` is a statement about the MAC (no model with 12 billion parameters or more fits), judged before the disk filter and whatever the pick is; a small pick under `ok` carries `noteSmallPick`, and a model left out for disk space is named in `noteDiskDropped`, never dropped in silence. ⚠️ The word "Recommended" exists only in `headlineTested` and is sent only when the pick's model has a `tested` entry for exactly this chip, memory size and harness; no row has one today. ⚠️ Speed is `ok`/`slow`/`unknown`, graded on the unrounded figure, and a chip in no chips row is `unknown`: never guess a bandwidth. ⚠️ ONE unit for every computed size a person reads: decimal GB text from `formatGb()` (`format.ts`), sent beside the bytes, and the browser never divides bytes; the one exception is the installed memory, shown as `memClassGb` (the size the Mac is sold as, 48 and not 51.5). ⚠️ Every sentence the server chooses is a `LOCAL_MODEL_LABELS` text with `{placeholder}` slots filled from `vars` (no apostrophe in a text: the zh-CN key is single-quoted), and each needs a zh-CN entry. Admin only in multi-user mode (403 before anything is probed). Tests: `test/local-model-*.test.ts`, `test/routes/local-model-routes.test.ts`. Card rules: `src/web/CLAUDE.md` (Model on this Mac). → `docs/api-reference.md` (Local model recommendation), `docs/versioning-policy.md` +**Local model recommendation** (`GET /api/local-model`, `src/local-model/`, `src/config/local-model-catalog.ts`, `src/types/local-model.ts`; EXPERIMENTAL): the server looks at the Mac it runs on and says which local coding model fits it and with which agent CLI. ⚠️ The READ side stays read-only: the probe, the catalogue and `recommend()` download nothing, install nothing and write nothing; everything that changes the machine is the install below, behind an explicit request. Apple silicon macOS only: anywhere else the answer is a `200` with verdict `none` (reasons `not-macos`, `intel`, `memory`, `disk`, `macos`, `no-harness`), and a failed probe is an answer, never a 500. The probe (`hardware.ts`) is ONE bounded async spawn of `/usr/sbin/sysctl` WITHOUT `-n`, parsed as `key: value` lines by the pure `parseSysctl()`, plus `statfs` of the data dir (probed bounded first, #516) and `node:os` fallbacks, cached 60 s; ⚠️ never `system_profiler`, never a second spawn, and `iogpu.wired_limit_mb` of 0 means not set. The catalogue is a compiled constant validated by a Zod schema: ⚠️ no override file and no runtime data, and a model id is `name:tag` in lowercase with no slash (`LOCAL_MODEL_ID_PATTERN`, so an id can never name another registry host). ⚠️ A harness is a catalogue ROW (`{ cli, minContext, rank, gated?, slowFirstTurn? }`) plus a `HarnessFact` built from the CLI registry at call time (`harness-facts.ts`); no code compares a CLI id (both id-branching guards scan these files). `recommend()` (`recommend.ts`) is pure. ⚠️ A model's need is its PEAK (`bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`, measured on a Mac for one row and marked `estimated` on the others), judged against a budget that is `estimated` until the runtime ran and `measured` from then on (`RecommendOptions.measuredTotalBytes`). ⚠️ Ranking, lexicographic: tested on a Mac like this first, then speed `ok` before `unknown` before `slow`, then model rank, harness rank, larger context; the fit grade decides whether a row is listed and what it says, it does NOT sort, so a model is offered at the largest context it fits at. ⚠️ `limited` is a statement about the MAC (no model with 12 billion parameters or more fits), judged before the disk filter and whatever the pick is; a small pick under `ok` carries `noteSmallPick`, and a model left out for disk space is named in `noteDiskDropped`, never dropped in silence. ⚠️ The word "Recommended" exists only in `headlineTested`, and the tag "Tested on a Mac like this" only on a row whose model has a `tested` entry for exactly this chip, memory size and harness. ⚠️ Speed is `ok`/`slow`/`unknown`, graded on the unrounded figure, and a chip in no chips row is `unknown`: never guess a bandwidth. ⚠️ ONE unit for every computed size a person reads: decimal GB text from `formatGb()` (`format.ts`), sent beside the bytes, and the browser never divides bytes; the one exception is the installed memory, shown as `memClassGb` (the size the Mac is sold as, 48 and not 51.5). ⚠️ Every sentence the server chooses is a `LOCAL_MODEL_LABELS`, `LOCAL_MODEL_TAG_LABELS` or `LOCAL_MODEL_INSTALL_LABELS` text with `{placeholder}` slots filled from `vars` (no apostrophe in a text: the zh-CN key is single-quoted), and each needs a zh-CN entry. Admin only in multi-user mode (403 before anything is probed). Tests: `test/local-model-*.test.ts`, `test/routes/local-model-routes.test.ts`. Window rules: `src/web/CLAUDE.md` (Run a model on this Mac). → `docs/api-reference.md` (Local model), `docs/versioning-policy.md` + +**Local model install** (`POST /api/local-model/install|cancel|resume|runtime`, `DELETE /api/local-model`; `src/local-model/{install-job,runtime,ollama-client,connect,state-store,paths,developer-tools}.ts`; EXPERIMENTAL, admin only in multi-user mode on EVERY verb; guide `docs/local-models.md`): installs the chosen model on the Mac the server runs on and ends on a session of the coding agent running on it. ONE job of five idempotent steps (`runtime`, `model`, `verify`, `agent`, `connect`; `agent` only when the coding agent is missing), a pure reducer (`reduceInstallJob()`) over three tables in `install-job.ts` driven by one service per process (`getLocalModelService()`); the window polls the GET once a second, no SSE event. ⚠️ Layout only through `localModelPaths()` (all `dataPath()`): `local-model/runtime/ollama-/`, `local-model/models/`, `local-model/home/` (the runtime's PRIVATE HOME, so the user's `~/.ollama` is never touched) and the state file `local-model.json` (0600, ONE serialized writer, total reader, holds NO path); nothing is written or deleted outside `local-model/` except that file and the two writes of the connect step. ⚠️ The runtime pin (`OLLAMA_RUNTIME_PIN`: version, URL, size, sha256) is a COMPILED constant no file, setting or request may supply; the hash is compared BEFORE anything is unpacked. ⚠️ The runtime is a NON-detached child whose environment comes from the `runtimeEnv()` ALLOWLIST, never `process.env` (no `CODEMAN_*`, none of the user's `OLLAMA_*`); the port is chosen ONCE by binding (from 11435, never 11434) and stored; a listener on it is adopted or replaced only when its stored pid runs a binary under OUR runtime folder, anything else is `port-in-use`; started at boot when installed and enabled, max 3 restarts, SIGTERM at shutdown beside `stopDeepSeekWeb()`; under vitest nothing spawns or fetches. ⚠️ Consent is bound to the size: `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`) or the answer is 409 `size-changed`; the model, context and CLI must be a row the current recommendation offers (400). ⚠️ ONE job at a time (409 `job-running`); a job a restart interrupted comes back `paused` and NEVER continues without a click (so `shutdown()` writes nothing), and an interrupted step is never recorded as failed. ⚠️ An event the transition table does not list is refused (same object back, 409 `wrong-state`): add a row to the table, never a special case. ⚠️ The budget is MEASURED from the runtime's "inference compute" log line and the model judged against it BEFORE it is downloaded. ⚠️ The pull stream is parsed incrementally (about 1.4 million lines for 23 GB): never buffer it, never write state per line. ⚠️ A failure is a code with ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed"; a probe that ended on its token limit is `probe-unfinished`, not `no-tool-call`. ⚠️ The coding agent is installed through the shared runner `src/cli-install-runner.ts` (extracted from `cli-registry-routes.ts`, ONE install slot per CLI for both callers) with `stockInstallCommand()`, the COMPILED stock command, never the `clis.json`-merged one (this path has no `cliManagementEnabled` gate); only for a catalogue harness row, after the consent click that showed the literal command, audited (`cli_install`, `via: 'local-model'`); afterwards `forgetResolvedCli()` and a fresh lookup, and still missing is `harness-failed`. ⚠️ Connect writes ONE endpoint record under the RESERVED id `local-ollama` through `mutateCustomModelHosts()` and `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, remembering the flag's first value; a record with that id that is not ours is never overwritten, and while it is ours it is MANAGED: the endpoint routes refuse to edit or delete it and the discovery sweep skips it (the runtime's port has no login, so its `models` must stay the one model that was checked). ⚠️ The model digest is stored and read again at boot and before EVERY session start on that endpoint (`launchRefusalMessage()`, asked by both custom-model launch paths in `session-routes.ts`): a changed or missing model, a runtime Codeman did not start, or another model id refuses the launch with one sentence. ⚠️ Remove (409 `sessions-open` with the names while a live session uses the endpoint) undoes the connection FIRST, restores the flag only when this feature switched it on AND no other endpoint is saved, deletes only inside `local-model/` (the models only when asked), and leaves the coding agent. ⚠️ Developer tools: only `/usr/bin/xcode-select -p`, bounded; never `git`/`python3`/`xcrun` from the server (each opens a system window on such a Mac). Tests: `test/local-model-install-job.test.ts` (every state with every event), `-install-service`, `-install-e2e` (real modules over `test/fixtures/fake-ollama.ts`), `-state-store`, `-connect`, `test/cli-install-runner.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`. → [architecture-invariants#local-model-install](docs/architecture-invariants.md#local-model-install), `docs/local-models.md`, `docs/api-reference.md` (Local model) **MCP server sync** (opt-in, `mcpSyncEnabled`, SYNCED, default OFF; `src/mcp-sync.ts`, `GET`/`POST /api/mcp-sync`): copies each installed, enabled CLI's user-level MCP servers into the others. It is the ONE subsystem that writes another CLI's REAL user config (`~/.claude.json`, `~/.codex/config.toml`, `~/.gemini/*`, opencode's), which is why it is opt-in and admin-only in multi-user mode (both verbs 403 for a non-admin, and the Settings group is hidden for them). Where each CLI keeps the file is registry data, `capabilities.mcpConfig` (`{ path, format, relocation? }`), never a branch on the id. ⚠️ Tools that sync but are not run modes (GitHub Copilot CLI, the first) are plain data in `MCP_SYNC_ONLY_TOOLS` (`src/mcp-sync-targets.ts`), take part only when installed or already configured, and their companion `settings.json` (`disabledMcpServers`) is read, never written; a test must clear their relocation vars too. ⚠️ ADDITIVE only: a name already defined, in any shape, is never edited or removed (a different same-name definition is a reported conflict), and a server switched off in its own CLI is never copied. ⚠️ Never write a file that did not parse; re-parse the NEW text and require every added server to read back before the tmp+rename (written through a symlink, previous file kept as `.codeman-bak`, one apply at a time, else 409). ⚠️ A file that receives copied `env`/`headers` (secrets) is left `0600`, and so is the backup. ⚠️ Responses carry server NAMES only, never env values, headers or file text: a parse failure is reported by line and column (`describeMcpSyncError`), never the parser's own message (smol-toml and V8 both quote source). ⚠️ `mcpConfig.relocation` names the env var the CLI reads to move its file (`CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `XDG_CONFIG_HOME`, `GEMINI_CLI_HOME`, and `COPILOT_HOME` for Copilot), resolved from the SERVER env at call time; a relative value reports the target `skipped`, never a guessed write, and a per-session `envOverrides` relocation is not followed. Tests must pass `home` (which drops the `process.env` default) or clear those vars first. → `docs/cli-registry.md` (MCP server sync), `docs/api-reference.md`, `docs/wiki/Settings-Reference.md` @@ -325,7 +327,7 @@ One module per domain in `src/web/routes/` (plus a barrel; `ls src/web/routes/` ## State Files -All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all three installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). +All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `local-model.json` (local model install: state, step, model, port, pid, digest, the previous `customModelEndpointsEnabled` value; mode 0600, holds no path, not part of a config backup) + `local-model/` (the pinned runtime, the model store and the runtime's private HOME, removed by `DELETE /api/local-model`), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all three installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). **Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored. diff --git a/docs/api-reference.md b/docs/api-reference.md index b0464ac8..e2d759a5 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1676,15 +1676,44 @@ Only replies from `allowedUserIds`, in `guildId`, inside a thread the bridge cre `GET /api/doctor[?category=core|office|other]` returns the `codeman doctor --json` report (`platform`, `summary`, `tools[]` with `status` `ok` \| `missing` \| `outdated` \| `skipped` \| `error`, `version`, `path`, `installHint`). The probe engine is synchronous, so it runs in a child process of the same entry script, never on the server's event loop (30 s timeout). It names install paths and versions, so it is admin only in multi-user mode (`403`). `400` for an unknown category, `500` if the child produces no report. -## Local model recommendation +## Local model -`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, and with which agent CLI. It covers Apple silicon macOS only; anywhere else the answer is still a `200`, with verdict `none`. **Read-only:** the route downloads nothing, installs nothing, writes no file and reads or changes no setting (setting a model up from Codeman is a later step and is not built yet). It names the host's hardware, so it is admin only in multi-user mode (`403`, answered before anything is probed). The models, their sizes and the agent CLIs it considers are a catalogue compiled into the release; there is no override file. +`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, and where an install of one stands. Five verbs install a model, pause and continue the install, stop and start the model runtime, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md). -**Experimental.** This endpoint and its payload are labeled experimental until the step that sets a model up ships: that step will add to and may change the fields below, the verdicts and the sentences, in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on its shape yet. +**Experimental.** These routes and their payloads are labeled experimental: the fields below, the states, the error codes and the sentences may change or go in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on their shape yet. -| Method | Path | Body | Notes | -| ------ | ------------------ | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated }`. `403 FORBIDDEN` for a non-admin in multi-user mode. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. | +Every route is admin only in multi-user mode (`403 FORBIDDEN`, answered before anything is probed, parsed or started): the GET names the host's hardware, and the verbs download and run a program, install an agent CLI and write the server-wide settings. There is no SSE event: the window polls the GET once a second while it is open. + +| Method | Path | Body | Notes | +| -------- | -------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated, install }`. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. | +| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed`, `size-changed`. | +| `POST` | `/api/local-model/cancel` | none | Interrupts the step in flight; the job is `paused` at that step. `409 wrong-state` when no job is running. | +| `POST` | `/api/local-model/resume` | `{ newPort? }` (may be absent) | Continues a `paused` or `failed` job at the same step. `newPort: true` chooses another port first. `409`: `job-running`, `wrong-state`. | +| `POST` | `/api/local-model/runtime` | `{ action: "start" \| "stop", newPort? }` | `stop` stops the runtime and keeps it from starting at the next boot; `start` undoes that. `newPort: true` with `start` chooses another port and moves the saved endpoint to it. `409`: `job-running`, `wrong-state` (nothing is installed). | +| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the model only with `removeModel: true`. `409`: `job-running`, `sessions-open`. | + +Every verb answers `{ success: true, data: }` with the `install` object (below) as it is after the verb. A refusal the feature names is a `409` whose envelope also carries the conflict: + +```json +{ + "success": false, + "error": "The download is 23.3 GB now. Check the sizes and confirm again.", + "errorCode": "CONFLICT", + "data": { + "reason": "size-changed", + "message": { "text": "The download is {size} GB now. Check the sizes and confirm again.", "vars": { "size": "23.3" } }, + "expectedBytes": 23291770627, + "expectedGb": "23.3" + } +} +``` + +`data.reason` is `job-running`, `already-installed` (remove the installed model first), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model) or `wrong-state`. A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix. + +`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. + +**Consent is bound to the size shown.** `acknowledgedBytes` must be the `installBytes` of the chosen row: the model download plus the runtime download while the runtime is not unpacked yet. The server compares it with its own total before anything starts and answers `409 size-changed` when they differ, so a click confirms exactly the size the person saw. A probe that fails is an answer, not an error: a server that is not on macOS, or a Mac whose `sysctl` call failed or timed out, still gets a `200`, and `hardware.source` says what was read. @@ -1696,18 +1725,46 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or - `chip`, `model`, `macosVersion`, `macosMajor`, `wiredLimitMb` (`0` means not set), `freeDiskBytes` and `freeDiskGb` (free space on the filesystem of Codeman's data directory). Each is absent when it could not be read. - `source`: `sysctl` (the one bounded `sysctl` call answered), `os-fallback` (macOS, but that call failed: memory, architecture, the chip name and the free disk space come from Node, and nothing only `sysctl` knows, such as `arm64`, `translated`, `model`, `macosVersion`, `macosMajor` or `wiredLimitMb`, is present) or `not-probed` (not macOS, nothing was run). - `recommendation`: - - `verdict`: `ok`, `limited` or `none`. `limited` is a statement about the Mac: no model with 12 billion parameters or more fits its memory, so what it can run handles small edits and questions and does not replace a cloud model. It is judged on every model that fits, before the free disk space is looked at and whatever the pick is. With `ok` the pick can still be a small model (a larger one may fit only tightly, have no speed figure or have no room on the disk); `notes` then says so. + - `verdict`: `ok`, `limited` or `none`. `limited` is a statement about the Mac: no model with 12 billion parameters or more fits its memory, so what it can run handles small edits and questions and does not replace a cloud model. It is judged on every model that fits, before the free disk space is looked at and whatever the pick is. With `ok` the pick can still be a small model (a larger one may be slow, have no speed figure or have no room on the disk); `notes` then says so. - `reason`, present only with `none`: `not-macos`, `intel`, `memory`, `disk`, `macos` (too old) or `no-harness` (no agent CLI that can be connected to a local model is installed or installable). `memory` has three sentences: nothing fits at all (slot `memory`, the size the Mac is sold as), a model fits but only below the context length an agent CLI needs (slots `budget` and `need`, so a Mac that misses by a little reads as that), and the memory size could not be read (no slot). - `headline` and `notes[]`: `{ text, vars? }`. `text` is an English sentence that may hold `{placeholder}` slots, and `vars` carries the slot values as text. Notes, in order: the pick is a small model although larger ones fit (verdict `ok` only); a model that fits the memory was left out because its download would not leave 10 GiB free on the disk (slots `model` and `missing`: the best-ranked such model and the GB to free up); nobody tested the pick on a Mac like this one. - - `pick` (absent with `none`) and `alternatives[]` (at most four, best first; empty with `none`), each one model at one context length with one agent CLI. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing): + - `pick` (absent with `none`) and `alternatives[]` (at most four, best first; empty with `none`), each one model at one context length with one agent CLI. The order: a pair tested on a Mac like this one first, then speed (`ok` before `unknown` before `slow`), then the model's rank, the agent CLI's rank and the larger context. The memory fit decides whether a row is listed and what it says (`fit`, `tag`), it does not sort, so a model is offered at the largest context it fits at. A model whose download would leave less than 10 GiB free on the disk is not listed (an unknown free disk space excludes nothing): - `modelId` (`name:tag`), `modelLabel`, `license`, `paramsB` (parameters in billions), `context` (tokens); - `cli`, `cliLabel`, `cliInstalled` (`false`: the CLI is not on this machine yet but can be installed without administrator rights), `slowFirstTurn?`; - - `needBytes` and `needGb` (memory the model needs at that context), `downloadBytes` and `downloadGb`, `budgetShare` (need divided by `budgetBytes`, rounded to three decimals); + - `needBytes` and `needGb` (memory the model needs at that context, at its PEAK: the weights, what stays held per token of context, what the runner holds beside both, and what it takes for a moment while a long prompt is processed), `needSource` (`measured`: the figures were measured for this model on a Mac; `estimated`: they are another model's), `downloadBytes` and `downloadGb`, `budgetShare` (need divided by `budgetBytes`, rounded to three decimals); + - `installBytes` and `installGb`: the whole download a click on Install starts for this row, `downloadBytes` plus the runtime archive while the runtime is not unpacked. It is the number to send as `acknowledgedBytes`; + - `harnesses[]`: every agent CLI the model can be run with at this context, best first: `{ cli, cliLabel, installed, installCommand?, slowFirstTurn? }`. `installCommand` is the literal command the install would run for a CLI that is not installed (the release's own stock command), for the consent screen. The first entry is the one the flat `cli` fields name; + - `tag?`: `{ kind, label }`, the one tag of the row. `kind` is `tested`, `best`, `tight`, `slow` or `small`; - `fit`: `good` (up to 0.85 of the budget) or `tight` (up to 0.98), with its label `fitLabel`; - `speed`: `ok`, `slow` (under 10 tokens per second) or `unknown`, with `speedLabel` (absent for `ok`) and `tokensPerSecond` (rounded to one decimal, absent for `unknown`; `speed` is decided on the unrounded figure, so `10` can be `slow`). `unknown` means the chip is in no row of the catalogue's chip table, or the catalogue has no speed figure for the model: a speed is never guessed; - - `tested`: we ran this model with this CLI on a Mac like this one. No catalogue row has such an entry yet, so it is `false` today. - - `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `wired-limit` (taken from `iogpu.wired_limit_mb`) or `estimated` (computed from the installed memory). + - `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run. + - `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `estimated` (computed from the installed memory: all there is before the runtime ran), `wired-limit` (taken from `iogpu.wired_limit_mb`) or `measured` (the device total the runtime itself printed when it started; the GET uses it from the first install on). - `catalogUpdated`: ISO date on which the catalogue's rows were last checked. +- `install`: where the install stands and what can be done about it. + - `state`: `idle` (nothing installed, no job), `installing`, `paused` (Cancel or a server restart stopped the job; it never continues without a click), `failed` or `installed`. + - `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed. + - `progress` (while a download runs, and as it stood on a paused or failed job): `completedBytes`, `totalBytes`, `completedGb`, `totalGb`, and while it moves `bytesPerSecond`, `mbPerSecond` (whole decimal megabytes per second as text) and `secondsLeft`. During the `model` step both byte figures come from the download's own lines, not from the catalogue. + - `error` (with `failed`; with `installed` only when something blocks a session): `{ code, message: { text, vars? }, next, nextModel?, detail? }`. `next` is the ONE action that helps: `retry` (`POST .../resume`), `other-port` (`POST .../resume` or `.../runtime` with `newPort: true`), `next-model` (choose again; `nextModel` is `{ modelId, modelLabel }` when the server knows one), `start` (`POST .../runtime`) or `remove` (`DELETE`). `detail` is raw text for the curious (an installer's last lines), not translated. + - `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. `port`: the runtime's loopback port, once chosen. + - `running`: the runtime Codeman started is alive right now. A runtime the person stopped is `running: false` with no `error`. + - `loaded` (with `installed`): what the runtime reported when the job checked the loaded model: `context`, `memoryBytes`, `memoryGb`, `onGpu`. + - `endpointId` (with `installed`): the saved model endpoint a session is started on, with `cli`. + - `developerTools`: `present`, `missing` (the Mac has no Xcode or Command Line Tools, so macOS will offer to install them the first time an agent runs `git` or `python3`) or `unknown`. + - `runtime`: `{ version, installed, downloadBytes, downloadGb, unpackedGb }`. `downloadBytes` is `0` once the runtime is unpacked. + - `folder`: the absolute folder everything of this feature is kept in. + - `actions[]`: exactly what the server accepts right now, out of `install`, `cancel`, `resume`, `start`, `stop`, `remove` and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session). + +Error codes (`install.error.code`), each with one next step: + +| Next step | Codes | +| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `retry` | `runtime-download-failed`, `runtime-damaged`, `runtime-unpack-failed`, `runtime-start-failed`, `disk-full`, `pull-failed`, `context-not-applied`, `load-failed`, `probe-unfinished`, `harness-failed`, `endpoint-id-taken`, `connect-failed`, `internal` | +| `start` | `runtime-stopped` (the runtime crashed three times in a row and is no longer restarted) | +| `other-port` | `port-in-use` | +| `next-model` | `model-not-found`, `does-not-fit`, `no-tool-call` | +| `remove` | `model-changed` (the model's digest is no longer the one stored at install time; a session on it is refused) | + +Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, when the model's digest changed, or for a model other than the installed one. Every computed size a person reads travels twice: as bytes, and as decimal GB text without the unit (`"needGb": "29.6"`). Show the text and add the unit; do not divide the bytes yourself, so one request never shows two different roundings. The installed memory is the exception named above: show `memClassGb`. diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index 5d67547d..53b53b9c 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -694,6 +694,38 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse ⚠️ **The loading banner shows the REAL backend log line, not a guess, and has no countdown or auto-timeout at all.** `getLatestLlamaSwapLogLine()` holds one `GET /api/events` SSE connection open per endpoint (confirmed live to stay open indefinitely — read past 220KB over 8s with no `done`; idle-closed after 30s via `pruneIdleLlamaSwapLogTails`, same 20s sweep as the swap-displacement check above), parsing `logData` frames and keeping only `source: "upstream"` (the real `llama-server` process's own stdout) lines, never `source: "proxy"` (llama-swap's own request-access log). ⚠️ `GET /logs` — the endpoint this feature's own first cut targeted, since the name suggested it — was confirmed live to carry ONLY the proxy log and never a single backend line, even seconds after a real, verified model swap; caught and corrected by a live check before merge, not after. The banner itself dropped its size-scaled expected-time estimate and matching auto-timeout (a guess dressed up as a fact that could kill a genuinely slow load partway through on slower hardware) for a generic hardware/model-size disclaimer plus a user-driven **Cancel** button (`_showCenterStatus`'s `onCancel` option, a real button distinct from the plain "×" close glyph an `'error'`-type banner gets) that ends the wait and closes the session on the user's own call rather than a guessed deadline. +### Local model install + +**A model that runs on this Mac** (`src/local-model/`, `src/config/local-model-catalog.ts`, routes `src/web/routes/local-model-routes.ts`; EXPERIMENTAL; guide `docs/local-models.md`): the server looks at the Mac it runs on, says which local coding model fits it, and installs one: a pinned runtime it supervises, the model, the checks, the coding agent, and one saved model endpoint. The read side (`hardware.ts`, the catalogue, the pure `recommend()`) is described in the root `CLAUDE.md`; this section is the install. + +**The layout** is one pure function (`localModelPaths()`, `paths.ts`): `/local-model/` holds `runtime/ollama-/` (plus its `.tgz.partial` and `.unpack` while it is fetched), `models/` and `home/`; `/local-model.json` is the state file. ⚠️ Every path comes from `dataPath()` through that function and NONE is ever read back from the state file (which holds no path), so a state file cannot point a delete anywhere. ⚠️ Nothing is written or deleted outside `local-model/` except the state file and the two writes of the connect step. The runtime's `HOME` is the private `home/` (measured to work), so its key pair lands there and the user's own `~/.ollama` is never created, read or removed; `home/` and a `.ollama/id_ed25519` anywhere are on the sensitive-path list. + +**The pin** (`OLLAMA_RUNTIME_PIN`, `runtime.ts`) is a COMPILED constant: version, URL, size, sha256. ⚠️ No file, setting, env var or request may supply any part of it: a hash that comes from the same writable place as the URL pins nothing. The archive is streamed to `*.partial` while it is hashed, continued with a Range request (3 bounded retries), refused when it grows past the pinned size, and the hash is compared BEFORE `/usr/bin/tar` unpacks anything into a temporary folder that is then renamed. + +**Supervision** (`createRuntimeSupervisor()`): the runtime is a NON-detached child, started at boot once a model is installed and enabled, restarted at most 3 times in a row, SIGTERMed at shutdown beside `stopDeepSeekWeb()`. ⚠️ Its environment comes from the pure `runtimeEnv()` ALLOWLIST (PATH, the private HOME, TMPDIR, LANG, the proxy variables, and exactly `OLLAMA_HOST=127.0.0.1:`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`, `OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`, `OLLAMA_MAX_LOADED_MODELS=1`): never `process.env` whole (it carries `CODEMAN_PASSWORD`, and a user's own `OLLAMA_ORIGINS=*` would open the port to every web page). ⚠️ The port is chosen ONCE by binding (11435 across 20; 11434 belongs to a runtime the user may run) and stored; it is never searched again on a later start, because it is part of the saved endpoint's URL. ⚠️ Pid and port are in the state file: a listener on the port that answers `/api/version` and whose pid runs a binary under OUR runtime folder is our own orphan (a server that was killed leaves one: a child is not ended by its parent's death) and is adopted or replaced; ANY other listener is `port-in-use`, never something to talk to. ⚠️ The supervisor never writes the state file: it reports through `onChange`, and the job stores what it reports. ⚠️ Under vitest nothing spawns and nothing is fetched unless a test injects it. + +**The budget is measured.** The runtime prints the device total in its log line "inference compute" (`total="37.4 GiB"`); `parseInferenceComputeTotal()` reads it, the job stores it and judges the chosen model against it (`modelFitsBudget()`) BEFORE the model is downloaded; from then on the GET recommends with `budgetSource: 'measured'`. One consent covers the whole install: the sizes that were confirmed do not change, so a model that no longer fits is a failure with the next model, not a second question. + +**The job** (`install-job.ts`) is a pure reducer (`reduceInstallJob()`) over three tables in that file (`INSTALL_STATE_TABLE`, `INSTALL_STEP_TABLE`, `INSTALL_TRANSITIONS`), driven by one service per process (`getLocalModelService()`). States: `idle`, `installing`, `paused`, `failed`, `installed`; steps in order: `runtime`, `model`, `verify`, `agent`, `connect` (`agent` is left out when the coding agent is installed). ⚠️ An event the transition table does not list is REFUSED by returning the same object, which the service answers with 409 `wrong-state`; add a transition to the table, never a special case to the service. ⚠️ ONE job at a time: verbs run on one queue, so two installs sent together cannot both find the state free. ⚠️ A job the state file still calls `installing` when the process has none was interrupted by a restart: it becomes `paused` the first time anything looks (`boot()`, a GET or a verb), and NOTHING continues without a click. `shutdown()` therefore writes nothing: it aborts the step and leaves `installing` in the file. ⚠️ An interrupted step is never recorded as failed, whatever its last call answered (an aborted fetch looks like a network error): the job checks its own signal before it writes `failed`. ⚠️ Every step is idempotent, because Resume and "Try again" re-enter the step the job stopped in. ⚠️ Cancel in the `connect` step is honoured only before the first write: once the two writes began they finish, so the pair is never left half done. ⚠️ The state file (`state-store.ts`) has ONE writer, serialized, temp file + rename, mode 0600, and reading is TOTAL (a file that does not parse, another schema, or a state that lacks a member it needs reads as idle): it holds nothing of the user, so it is replaced rather than refused, unlike `settings.json`. ⚠️ Progress is kept in memory and written at most every few seconds and when the job pauses or fails; the window reads the live figure. A 23 GB pull emits about 1.4 million lines, so neither the stream nor one state write per line is affordable. + +**Install is checked before anything is downloaded** (`LocalModelService.install()`): the model, the context and the coding agent must be a row the CURRENT recommendation offers (a model id is looked up, never passed on: without a slash it cannot name another registry host, and the request schema repeats that); the coding agent must be a harness row of the catalogue; `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`: the model plus the runtime archive while the runtime is not unpacked), else 409 `size-changed` with that total, so a consent is bound to the size that was shown; and a saved endpoint that already has the reserved id and is not ours is said now, not after a 23 GB download. + +**The check after the pull** (`verifyModel()`): one tiny chat with `reasoning_effort: "none"`, then `/api/ps` must show `size_vram == size` at the chosen context, then one tool probe (`tools`, `reasoning_effort: "none"`, `max_tokens` 1024) that passes on a tool call with parseable arguments. ⚠️ Thinking is switched off and the cap is generous on purpose: every catalogue model thinks first, and a probe that ends on its token limit proves nothing (`probe-unfinished`, never `no-tool-call`). ⚠️ A failure is a state with a code and ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed". ⚠️ The job fails when the runtime reports no digest for the model: there would be nothing to compare later. + +**The coding agent** is installed by `runInstallCommand()` of `src/cli-install-runner.ts`, the runner extracted from `cli-registry-routes.ts` (whose route keeps its behaviour and re-exports `installEnv`/`npmGlobalPrefixWritable` for its tests). ⚠️ The job runs only `stockInstallCommand(id)`, read from the COMPILED `STOCK_CLIS`, never the registry merged with `clis.json` (the route's own command still comes from the merged registry, behind its `cliManagementEnabled` gate): this path has no such gate, so an override file must not be able to change what it runs. `HarnessFact.installCommand` is that same string, which is what the consent screen prints. ⚠️ Only for a harness row of the catalogue, only after the consent click, admin only, audited (`cli_install` with `via: 'local-model'`). ⚠️ ONE install slot per CLI id (`tryBeginCliInstall()`), shared by the route and the job. ⚠️ After the installer, successful or not, the resolver caches are forgotten (`forgetResolvedCli()`) and the binary is looked for again: a pre-install miss is negative-cached for up to five minutes, and "Start a session" would otherwise answer "not installed". Still missing is `harness-failed`; found is success whatever the exit code. The runner takes an abort signal that kills the installer's process GROUP. + +**The connection** (`connect.ts`): one record `{ id: 'local-ollama', label: 'This Mac', baseUrl, authStyle: 'bearer', models: [], defaultModelId, defaultContextLength: }`, validated by `CustomModelHostSchema` and written through `mutateCustomModelHosts()`, then `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, with the flag's previous value kept in the state file (a second connect must hand the FIRST value back, or Remove could never switch the flag off again). ⚠️ `defaultContextLength` is load-bearing: the runtime's model listing states no context, so this field is what reaches the coding agent as its limit. ⚠️ The id is reserved: `POST /api/model-endpoints` refuses it, and a record with it that this feature did not write is never overwritten (`endpoint-id-taken`). ⚠️ While the state file records the connection, the record is MANAGED (`isManagedLocalEndpoint()`): `PUT`/`DELETE /api/model-endpoints/local-ollama` are refused, discover answers the stored list without a request, and the sweep skips it, so a model somebody pulled through the port never becomes launchable and the URL never leaves the supervised port. A hand-made record with that id (no state) stays an ordinary endpoint. `local-model.json` is not a config-backup file while the endpoint store and the settings are, so a restore can take the record away under an installed model: `boot()` puts the RECORD back when it is gone (`ensureLocalEndpoint()`, never replacing one that is there) and leaves the settings flag as it finds it. ⚠️ The runtime's port has NO authentication, and its CORS allowlist always admits `localhost` origins: any local process or localhost page can pull, create or delete models there. That is the reason for the three rules above and for the digest. + +**The digest** `/api/tags` reported after the check is stored. ⚠️ Before a session starts on the managed endpoint (`LocalModelService.launchRefusalMessage()`, asked by BOTH launch paths in `session-routes.ts`: `POST /api/sessions/:id/custom-model` and the quick-start `customModel` branch, before anything is written or created) and at boot it is read again: a different digest, a model that is gone, a runtime Codeman did not start, or a model id other than the installed one refuses the launch with one sentence. The view shows what blocks a session as `error` while the state stays `installed`; it is found live and never stored. Boot recovery of an already running session (`_rebuildCustomModelEnv`) is not guarded: the pane exists and keeps its environment. + +**Remove** stops the child, undoes the connection FIRST (a refused write of the endpoint store or of the settings rejects before any file is deleted), restores the flag only when this feature switched it on AND no other endpoint is saved, then deletes the runtime and the private home and, when asked, the models (`removeLocalModelFiles()`, which refuses any path outside the root). ⚠️ While a live session uses the endpoint the answer is 409 `sessions-open` with the session names (`sessionNamesUsingEndpoint()`, read at the moment of the request). ⚠️ The coding agent stays: what its installer wrote is not ours to remove. + +**Developer tools**: `checkDeveloperTools()` runs ONLY `/usr/bin/xcode-select -p`, bounded, on darwin. ⚠️ Never `git`, `python3` or `xcrun` from the server: on a Mac without the tools each opens a system window. The view carries `developerTools`, and the window prints the one sentence when it is `missing`. + +**The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it. + +Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event), `test/local-model-install-service.test.ts`, `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`. + ### Approvals Inbox **Approvals Inbox** (cross-session queue of prompts waiting on a human; `approvalsInboxEnabled`, SYNCED, default OFF: every surface is opt-in; only the store and answer endpoints run regardless, so flipping it ON shows anything already pending): `web/approval-inbox.ts` is a `sessionWaits`-style singleton fed by `/api/hook-event`, holding at most ONE item per session (a new prompt supersedes), in-memory; answerable items are claude-mode only, and the DeepSeek status bridge and codex's screen-read dialogs (`web/screen-blocked.ts`) open ALERT-ONLY items (`answerInTerminal`: the answer route refuses them BEFORE its hooks gate, push carries no `approvalId`, the card and overview strip say "answer in the terminal"). Cards are answered via `POST /api/approvals/:id/answer`, which sends a digit / Esc / idle-prompt text through `writeViaMux` (menu answers never carry `\r`). diff --git a/docs/custom-model-endpoints.md b/docs/custom-model-endpoints.md index 78001ed4..31f5de00 100644 --- a/docs/custom-model-endpoints.md +++ b/docs/custom-model-endpoints.md @@ -663,6 +663,45 @@ each result. `scripts/test-local-llm-harnesses.ts` is the standalone script used to check a harness against a real endpoint outside the web UI entirely; see its own `--help` for usage. +## The endpoint store has one writer + +`~/.codeman/custom-model-hosts.json` is written only through `mutateCustomModelHosts()` +(`src/custom-model-hosts-writer.ts`): the create, update, delete and discover routes, the +five-minute re-discovery sweep and the local model install all go through it. + +- **One at a time.** Changes run on one queue, and the check a route makes ("this id is + free", "this endpoint still exists") runs in the same step as its write. Before, each + writer read the file, changed its copy and wrote it back on its own, all through one temp + file name: a route and the sweep, or two requests sent together, could lose each other's + change or fail outright. +- **Atomic.** A temp file with a random name beside the target, then a rename. Mode 0600. +- **A file that does not parse is refused, not replaced.** The writing routes answer + `409 CONFLICT` with the fix (repair or delete the file, or restore it from a config + backup) and leave it byte for byte. Before, such a file read as an empty list, and the next + save replaced every stored endpoint and its API key with one record. The message never + quotes the file, which holds keys. + +The one direct writer left is config-backup restore, which must work on a file that does +not parse. + +## The endpoint of the model on this Mac + +Installing a local model from Codeman ([local-models.md](local-models.md)) saves one endpoint +with the reserved id `local-ollama` and the label "This Mac". It shows up in the list and in +the Run menu like any other endpoint. Three things are different: + +- A new endpoint cannot be given the id `local-ollama`. +- While it is the endpoint the install manages, it cannot be edited or deleted here, and + "Discover" answers its stored model list without asking the runtime. Its model list must + stay the one model the install checked, and its address the port of the runtime Codeman + supervises. Change or remove it in the local model window. +- A session is started on it only while Codeman's own runtime is running and the model is + still the one that was installed (its digest is compared). Otherwise the launch answers + with one sentence. + +An endpoint with that id that somebody made by hand, with no model installed by Codeman, is +an ordinary endpoint: it can be edited and deleted as before. + ## Security note Every env var this feature can set that redirects a session's traffic diff --git a/docs/local-models.md b/docs/local-models.md new file mode 100644 index 00000000..b8c33d0b --- /dev/null +++ b/docs/local-models.md @@ -0,0 +1,255 @@ +# A model that runs on this Mac + +Codeman can set up a coding model that runs on the Mac the server runs on: no account, no +API key, nothing leaves the machine. One button on the welcome screen opens a window that +shows what fits this Mac, installs the choice and ends on a coding agent session that +runs on it. + +**Experimental.** The routes and their payloads may change in any release +([versioning-policy.md](versioning-policy.md), "Experimental local model routes"). Apple +silicon macOS only: anywhere else the window says why nothing is offered. + +This page is the reference for how it works. The HTTP contract is in +[api-reference.md](api-reference.md) ("Local model"). The rules that must not be broken are +in [architecture-invariants.md](architecture-invariants.md#local-model-install). + +## What an install does, in order + +The person picks a model, presses "Install on this Mac" and confirms the sizes once. The +server then runs one job of five steps. The window polls `GET /api/local-model` once a +second; the job runs in the server, so the window can be closed. + +| Step | What it does | +| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `runtime` | Downloads the pinned runtime, compares its hash, unpacks it, chooses a port once, starts it, reads the measured memory budget and judges the choice again | +| `model` | Downloads the model through the runtime | +| `verify` | Loads the model and checks it: one tiny chat, the memory and context the runtime reports, one tool call | +| `agent` | Installs the coding agent with its stock install command. Left out when the agent is installed | +| `connect` | Saves one model endpoint and switches the custom model endpoints setting on | + +Every step can be repeated: Resume and "Try again" enter the step the job stopped in. + +### The runtime + +- **One pinned build.** Ollama 0.40.15 for macOS, a compiled constant + (`OLLAMA_RUNTIME_PIN` in `src/local-model/runtime.ts`): the URL, the size (185,971,671 + bytes) and the sha256. No file, setting, environment variable or request can change which + binary is downloaded. +- **Download.** Streamed to `*.partial` while it is hashed. A cut stream is continued with a + Range request, at most three times. The hash is compared before anything is unpacked, and + a mismatch deletes the file. Unpacking is `/usr/bin/tar` into a temporary folder, then one + rename. +- **No administrator rights.** Everything lives in the data directory. +- **Supervised by Codeman.** The runtime is a non-detached child of the server: started at + boot once a model is installed and not stopped, restarted at most three times after a + crash, sent SIGTERM at shutdown. It is not a login item: it stops with Codeman, and a + leftover from a server that was killed is taken back at the next start (see below). +- **Its environment is an allowlist.** `PATH`, `TMPDIR`, `LANG` and the proxy variables are + copied from the server. `HOME` is a private folder. Exactly these are set: + `OLLAMA_HOST=127.0.0.1:`, `OLLAMA_MODELS`, `OLLAMA_CONTEXT_LENGTH`, + `OLLAMA_NO_CLOUD=1`, `OLLAMA_KEEP_ALIVE=10m`, `OLLAMA_NUM_PARALLEL=1`, + `OLLAMA_MAX_LOADED_MODELS=1`. No `CODEMAN_*` variable and none of the user's own + `OLLAMA_*` settings reach it. +- **The port is chosen once**, by binding, from 11435 across 20 ports, and stored. 11434 is + left alone: it is the default of a runtime the user may run themselves, which is never + touched. +- **An orphan is recognized.** The pid and the port are stored. At boot a listener on the + port that answers `/api/version` and whose pid runs a binary under Codeman's runtime folder + is Codeman's own leftover from a server that was killed: it is adopted or replaced. Any + other listener is "Port 11435 is in use by another program", with "Use another port" as + the way out. + +### The memory budget is measured + +Before the runtime exists the budget is an estimate from the installed memory. Right after +it starts, the runtime prints the device total in its log line "inference compute" +(`total="37.4 GiB"` on a 48 GB MacBook). Codeman reads that line, stores the figure and +judges the chosen model against it before a byte of the model is downloaded. A model that no +longer fits ends the job there with "does not fit" and the next model to try. From then on +the window shows the measured budget and the word "estimated" is gone. + +The need of a model is its peak: the weights, what stays held per token of context, what the +runner holds beside both, and what it takes for a moment while a long prompt is processed. + +### The model download + +`POST /api/pull` with `stream: true`. A 23 GB pull sends about 1.4 million progress lines, +so the stream is parsed piece by piece and never kept. Cancel aborts the request, which +stops the download. Resume pulls again and the runtime continues its partial files. An error +line or a cut stream is retried up to five times. Once the pull reported its total, every +size shown for the model is that total. + +### The check + +After the pull, in this order: + +1. One tiny chat with `reasoning_effort: "none"`. It loads the model. +2. `GET /api/ps` must show the whole model on the GPU (`size_vram` equal to `size`) at the + chosen context length. +3. One tool probe on `/v1/chat/completions` (a `tools` array, `reasoning_effort: "none"`, + `max_tokens` 1024). It passes on a tool call whose arguments parse. + +A failure is a state with a code and one next step, never a bare "failed": `does-not-fit` +and `no-tool-call` offer the next model, `probe-unfinished` (the token limit ended the +probe, so it proves nothing) and `load-failed` offer "Try again". The digest the runtime +reports for the model is stored when the check passed. + +### The coding agent + +When the chosen coding agent is not installed, the job installs it with the CLI registry's +own stock install command, through the same runner as App Settings > CLIs +(`src/cli-install-runner.ts`). The rules: + +- Only a CLI that is a harness row of the local model catalogue. +- Only the **compiled** stock command. A `~/.codeman/clis.json` override cannot change what + runs here. +- Only after the person's consent click, on a screen that showed the literal command. +- Admin only in multi-user mode, and written to the admin audit log (`cli_install`, with + `via: "local-model"`). +- One installer per CLI at a time, shared with App Settings: while one runs, the other + answers that it is busy. +- Afterwards the cached lookups of the CLI's binary are forgotten and the binary is looked + for again. Still missing is the failure `harness-failed`, with the installer's last lines. + +This step does not need the `cliManagementEnabled` setting. That setting gates the general +CLI management surface; this path can only ever run one vetted command for one catalogue +row. + +### The connection + +Two writes outside the local model folder, both undone by Remove: + +1. One record in `custom-model-hosts.json`: + + ```json + { + "id": "local-ollama", + "label": "This Mac", + "baseUrl": "http://127.0.0.1:11435", + "authStyle": "bearer", + "models": ["qwen3.6:35b-a3b"], + "defaultModelId": "qwen3.6:35b-a3b", + "defaultContextLength": 131072 + } + ``` + + `defaultContextLength` is the chosen context: the runtime's model listing states none, so + this field is what reaches the coding agent as its context limit. + +2. `customModelEndpointsEnabled: true` in `settings.json`. The value it had before is kept + in the state file. + +The id `local-ollama` is reserved. A new endpoint cannot take it, and a record with that id +that this feature did not write is never overwritten: the install says so before anything is +downloaded. While the record is the managed one it cannot be edited or deleted in App +Settings and the five-minute discovery sweep leaves it alone, so its model list stays the +one model the install checked. + +Switching the setting on also starts that five-minute sweep for every other saved endpoint. + +## Starting a session + +"Start a session" starts the coding agent on the endpoint `local-ollama` through the +existing custom model launch. Before any session is started on that endpoint the server +checks, in this order: + +- the runtime child Codeman started is alive (whatever else answers on the port is not + ours); +- the digest the runtime reports for the installed model is the one stored at install time; +- the model asked for is the one that was installed and checked. + +A different digest refuses the launch: "The model on this Mac changed since it was +installed. Remove it and install it again." The digest is also read again at boot and when +the runtime is started. + +## Stop, Start, Remove + +- **Stop** stops the runtime and keeps it from starting at the next boot. **Start** undoes + that. +- **Remove** stops the runtime, removes the endpoint record, switches the setting off again + only when this feature switched it on and no other endpoint is saved, and deletes the + runtime and its private home. The downloaded model is deleted only when asked. The coding + agent stays. While a live session uses the model, Remove answers with the session names + instead. + +## Where everything is + +All under the data directory (`~/.codeman`, or the instance's own): + +``` +local-model.json the state file (mode 0600) +local-model/ + runtime/ollama-0.40.15.tgz.partial the runtime archive while it downloads + runtime/ollama-0.40.15/ the unpacked runtime (about 579 MB) + models/ the model store + home/ the runtime's private HOME (its key pair is here) +``` + +Nothing is written or deleted outside `local-model/` except the state file beside it and the +two writes of the connection. The state file holds no path, so it can never point a delete +somewhere else. The user's own `~/.ollama` is never created, read or removed. + +`local-model.json` is not part of a config backup. `custom-model-hosts.json` and +`settings.json` are, so restoring an older backup can take the endpoint record away while +the model stays installed. The record is put back the next time Codeman starts (the setting +is left as the backup had it). A restore can also bring a record back after a Remove: it +then points at a port nothing listens on, is an ordinary endpoint again, and can be deleted +in App Settings. + +## The job's states + +| State | A server restart | Cancel | What can be done | +| ------------ | ------------------------------------------------------------- | ---------------- | -------------------------------------- | +| `idle` | stays idle | refused | install | +| `installing` | becomes `paused` at the same step | becomes `paused` | cancel | +| `paused` | stays paused | refused | resume, install, remove | +| `failed` | stays failed, same error | refused | resume, install, remove | +| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, remove | + +One job at a time. A job a server restart interrupted comes back paused and never continues +without a click. Cancel interrupts the step in flight (the download, the pull, the request, +the installer's process group); in the last step it is honoured before the first write, and +once the two writes began they finish. + +## On a Mac without developer tools + +Coding agents call `git` and `python3`. On a Mac without Xcode or the Command Line Tools +those are stubs that open macOS's "install the command line developer tools" window. So the +window says one sentence wherever the person commits or finishes, and only when +`/usr/bin/xcode-select -p` fails: + +> Coding agents use git and Python. This Mac has no developer tools yet, so macOS will offer +> to install them the first time an agent needs one. + +Codeman itself only ever runs `/usr/bin/xcode-select -p` for this, bounded, and never `git` +or `python3`. + +## What to know before relying on it + +- **The runtime's port has no login.** Any program on this Mac, and any web page served from + `localhost`, can use the model server and change its models. That is why the endpoint + record lists one model, why the sweep leaves it alone, and why the digest is compared + before a session starts. +- **A Codeman restart stops the runtime.** A session that is mid-answer on the local model + loses that answer. The runtime is started again with the server, and the first request + loads the model again (seconds). +- **The runtime does not enforce the context length.** A prompt longer than the context was + accepted in testing. Do not rely on the runtime to stop one. +- **The model stays in memory for ten minutes** after the last request, then the runtime + unloads it. + +## Source + +| What | Where | +| ---------------------------------------------- | -------------------------------------------------------------------------------------------------- | +| Hardware probe, catalogue, recommendation | `src/local-model/hardware.ts`, `src/config/local-model-catalog.ts`, `src/local-model/recommend.ts` | +| Runtime pin, download, environment, supervisor | `src/local-model/runtime.ts` | +| Runtime HTTP client, pull parser, checks | `src/local-model/ollama-client.ts` | +| Developer tools check | `src/local-model/developer-tools.ts` | +| Job (state table, reducer, view, service) | `src/local-model/install-job.ts` | +| State file | `src/local-model/state-store.ts` | +| Connection and its undoing | `src/local-model/connect.ts` | +| Paths | `src/local-model/paths.ts` | +| Routes | `src/web/routes/local-model-routes.ts` | +| Shared CLI install runner | `src/cli-install-runner.ts` | +| Serialized writer of the endpoint store | `src/custom-model-hosts-writer.ts` | diff --git a/docs/versioning-policy.md b/docs/versioning-policy.md index ed6dd809..ffd7b0cf 100644 --- a/docs/versioning-policy.md +++ b/docs/versioning-policy.md @@ -86,18 +86,25 @@ covered like any other stable behavior. Other agent commands and API routes keep their existing guarantees. Promotion to stable requires an explicit documented decision. -### Experimental local model recommendation +### Experimental local model routes -`GET /api/v1/local-model` (and its `/api` alias) is labeled experimental, so item 3 -above applies to it: its path, its payload (the hardware facts, the verdicts and -reasons, the candidate fields and the English sentences in `headline` and -`notes`) and its behavior may change or be removed in a MINOR or PATCH release. -It is read-only today: it looks at the Mac the server runs on and says which -local model fits. The step that sets a model up from Codeman is not built yet -and will change what the route reports, which is why the shape is not promised. -It is not behind a setting (it writes nothing and adds no settings key); it is -admin only in multi-user mode. Promotion to stable requires an explicit -documented decision. +`GET /api/v1/local-model`, `POST /api/v1/local-model/install`, `/cancel`, `/resume` +and `/runtime`, and `DELETE /api/v1/local-model` (and their `/api` aliases) are +labeled experimental, so item 3 above applies to them: their paths, their payloads +(the hardware facts, the verdicts and reasons, the candidate fields, the `install` +object with its states, steps, error codes and actions, the 409 conflict shape and +every English sentence) and their behavior may change or be removed in a MINOR or +PATCH release. They look at the Mac the server runs on, say which local model fits, +and install, start, stop and remove one ([local-models.md](local-models.md)). + +They are not behind a setting. Reading changes nothing; an install starts only on an +explicit request that carries the size the person confirmed. All of them are admin +only in multi-user mode. What an install leaves behind is ordinary, stable state: one +saved model endpoint with the id `local-ollama` and the `customModelEndpointsEnabled` +setting switched on, both of which the stable custom model endpoint routes read like +any other endpoint. On-disk state (`local-model.json`, the `local-model/` folder) is +internal, as for every other file in the data directory. Promotion to stable requires +an explicit documented decision. ## Deprecation policy diff --git a/docs/wiki/Custom-Model-Endpoints.md b/docs/wiki/Custom-Model-Endpoints.md index a7e4f0d6..4ee5aa59 100644 --- a/docs/wiki/Custom-Model-Endpoints.md +++ b/docs/wiki/Custom-Model-Endpoints.md @@ -204,6 +204,45 @@ Which harnesses show up in the Run-menu picker is read live off Codeman's own CL not a fixed list here, so this table can go stale before this page does — a greyed-out or missing entry is the more current answer. +## A model that runs on this Mac + +If the Codeman server runs on a Mac with Apple silicon, you do not need a server of your +own: Codeman can set one model up for you on that Mac. This is experimental. + +1. Press **Run a local model** on the welcome screen (or open App Settings > Models > Model + on this Mac). +2. The window shows what fits this Mac. A bar shows how much of the memory the selected + model takes at its full context; a nearly full bar means a tight fit. Pick a model and + the coding agent to run it with. +3. Press **Install on this Mac**, check the sizes and where they go, and confirm. Nothing is + downloaded before this click. +4. Codeman downloads a model runtime (Ollama, about 186 MB) and the model into + `~/.codeman/local-model`, loads the model, checks that all of it runs on the GPU and that + it can call a tool, installs the coding agent if it is missing, and saves an endpoint + named **This Mac**. No administrator password is asked for. +5. Press **Start a session**. + +Good to know: + +- You can close the window: the download continues while Codeman is running. **Cancel** + pauses it and **Continue** picks it up where it stopped. After a Codeman restart an + unfinished install waits for your click. +- The memory figure is an estimate until the runtime has started once. From then on it is + what the runtime itself measured, and the model is judged against that before it is + downloaded. +- The endpoint "This Mac" belongs to this install. It is in the list and in the Run menu, + but it is changed and removed in the local model window, not here. +- Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from + starting next time. +- **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The + coding agent stays installed. +- If the model on disk is no longer the one that was installed, Codeman refuses to start a + session on it and tells you to remove and install it again. +- Any program on this Mac, and any web page served from `localhost`, can reach the model + runtime: its port has no login. It listens on this Mac only. +- On a Mac without Apple's developer tools, macOS will offer to install them the first time + a coding agent runs `git` or Python. The window tells you when that applies. + ## What it does not do - **No remote or Docker sessions yet.** Both restart their agent differently under the hood diff --git a/docs/wiki/Settings-Reference.md b/docs/wiki/Settings-Reference.md index 0ffcdf52..a88b55ad 100644 --- a/docs/wiki/Settings-Reference.md +++ b/docs/wiki/Settings-Reference.md @@ -193,19 +193,25 @@ attached. section to the Run dropdown, for pointing a harness at your own OpenAI-compatible server instead of its native cloud backend. See [Custom Model Endpoints](Custom-Model-Endpoints). -**Model on this Mac** is not a setting: it is a read-only answer. When the Codeman server -runs on a Mac with Apple silicon, it shows that Mac's chip, memory and free disk space, the -local coding model that fits it best with the agent CLI to run it, what the model needs of -the memory the Mac can give one, its download size and an estimated speed, and the other -models that fit (Good or Tight for the memory, Slow or Speed unknown for the speed). A Mac -that can only run a model under 12 billion parameters is told so in plain words, and a Mac -with too little memory, too little free disk space or an Intel processor is told why nothing -is offered. The numbers are estimates from a catalogue that ships with each release; nothing -has been tested on your Mac. **Nothing is downloaded or installed from here yet**: setting -the model up from Codeman is the next step and is not built. The same answer appears as a -card on the welcome screen, which you can hide per device (it stays here). The group is not -shown when the server runs on Linux or Windows, and in multi-user mode only an admin sees -it. The "Mac" is always the machine the server runs on, not the device you are reading on. +**Model on this Mac** is not a setting: it is the state of a model Codeman can run for you +on the Mac the server runs on, with a button that opens the local model window. The same +window opens from the **Run a local model** button on the welcome screen. It shows that +Mac's chip, memory and free disk space, a bar for how much of the memory the selected model +takes, and the models that fit, each with one tag (Best fit, Tight, Slow, Small model, or +"Tested on a Mac like this"), a speed and a download size. **Install on this Mac** asks you +to confirm the sizes once, then downloads a model runtime and the model into +`~/.codeman/local-model`, checks that the model loads and can call a tool, installs the +coding agent if it is missing, and ends on **Start a session**. No account, no API key and +no administrator password. You can close the window while it downloads, cancel and continue +later; after a Codeman restart an unfinished install waits for you to press Continue. Once a +model is installed the window offers Start a session, Stop or Start, and Remove. Installing +switches **Custom model endpoints** on and saves one endpoint named "This Mac"; Remove takes +the endpoint away again and switches the setting back off if nothing else uses it. A Mac +that can only run a small model is told so in plain words, and a Mac with too little memory, +too little free disk space or an Intel processor is told why nothing is offered. The group +is not shown when the server runs on Linux or Windows, and in multi-user mode only an admin +sees it. The "Mac" is always the machine the server runs on, not the device you are reading +on. Details: [Custom Model Endpoints](Custom-Model-Endpoints#a-model-that-runs-on-this-mac). ### Agents & CLIs