diff --git a/CLAUDE.md b/CLAUDE.md index 245016ee..7019f4a0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -194,7 +194,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Local model recommendation** (`GET /api/local-model`, `src/local-model/`, `src/config/local-model-catalog.ts`, `src/types/local-model.ts`; EXPERIMENTAL): the server looks at the Mac it runs on and says which local coding model fits it and with which agent CLI. ⚠️ The READ side stays read-only: the probe, the catalogue and `recommend()` download nothing, install nothing and write nothing; everything that changes the machine is the install below, behind an explicit request. Apple silicon macOS only: anywhere else the answer is a `200` with verdict `none` (reasons `not-macos`, `intel`, `memory`, `disk`, `macos`, `no-harness`), and a failed probe is an answer, never a 500. The probe (`hardware.ts`) is ONE bounded async spawn of `/usr/sbin/sysctl` WITHOUT `-n`, parsed as `key: value` lines by the pure `parseSysctl()`, plus `statfs` of the data dir (probed bounded first, #516) and `node:os` fallbacks, cached 60 s; ⚠️ never `system_profiler`, never a second spawn, and `iogpu.wired_limit_mb` of 0 means not set. The catalogue is a compiled constant validated by a Zod schema: ⚠️ no override file and no runtime data, and a model id is `name:tag` in lowercase with no slash (`LOCAL_MODEL_ID_PATTERN`, so an id can never name another registry host). ⚠️ A harness is a catalogue ROW (`{ cli, minContext, rank, gated?, slowFirstTurn? }`) plus a `HarnessFact` built from the CLI registry at call time (`harness-facts.ts`); no code compares a CLI id (both id-branching guards scan these files). `recommend()` (`recommend.ts`) is pure. ⚠️ A model's need is its PEAK (`bytes + heldBytesPerToken * context + overheadBytes + peakExtraBytes`, measured on a Mac for one row and marked `estimated` on the others), and every row names its `runner` (`mlx` or `llamacpp`, the build a Mac really pulls) and carries figures of its OWN: computed from its model's config.json by that runner's rule (catalogue header, MEMORY), never borrowed from another row; `test/local-model-catalog.test.ts` recomputes them, and the `llamacpp` rows assume the runtime's `OLLAMA_KV_CACHE_TYPE=q8_0`, `LLAMA_ARG_CACHE_RAM=1024` and `LLAMA_ARG_CTX_CHECKPOINTS=8` (`runtimeEnv()`) and count what llama-server keeps beside the cache (8 context checkpoints of the recurrent or sliding-window state, and for a GGUF with a prediction layer one rollback copy of the recurrent state per drafted token and that layer's cache in f16), so change the two together. The need is judged against a budget that is `estimated` until the runtime ran and `measured` from then on (`RecommendOptions.measuredTotalBytes`). ⚠️ Ranking, lexicographic: tested on a Mac like this first, then speed `ok` before `unknown` before `slow`, then model rank, harness rank, larger context; the fit grade decides whether a row is listed and what it says, it does NOT sort, so a model is offered at the largest context it fits at. ⚠️ `limited` is a statement about the MAC (no model with 12 billion parameters or more fits), judged before the disk filter and whatever the pick is; a small pick under `ok` carries `noteSmallPick`, and a model left out for disk space is named in `noteDiskDropped`, never dropped in silence. ⚠️ The word "Recommended" exists only in `headlineTested`, and the tag "Tested on a Mac like this" only on a row whose model has a `tested` entry for exactly this chip, memory size and harness. ⚠️ Speed is `ok`/`slow`/`unknown`, graded on the unrounded figure, and a chip in no chips row is `unknown`: never guess a bandwidth. ⚠️ ONE unit for every computed size a person reads: decimal GB text from `formatGb()` (`format.ts`), sent beside the bytes, and the browser never divides bytes; the one exception is the installed memory, shown as `memClassGb` (the size the Mac is sold as, 48 and not 51.5). ⚠️ Every sentence the server chooses is a `LOCAL_MODEL_LABELS`, `LOCAL_MODEL_TAG_LABELS` or `LOCAL_MODEL_INSTALL_LABELS` text with `{placeholder}` slots filled from `vars` (no apostrophe in a text: the zh-CN key is single-quoted), and each needs a zh-CN entry. Admin only in multi-user mode (403 before anything is probed). Tests: `test/local-model-*.test.ts`, `test/routes/local-model-routes.test.ts`. Window rules: `src/web/CLAUDE.md` (Run a model on this Mac). → `docs/api-reference.md` (Local model), `docs/versioning-policy.md` -**Local model install** (`POST /api/local-model/install|cancel|resume|runtime`, `DELETE /api/local-model`; `src/local-model/{install-job,runtime,ollama-client,connect,state-store,paths,developer-tools}.ts`; EXPERIMENTAL, admin only in multi-user mode on EVERY verb; guide `docs/local-models.md`): installs the chosen model on the Mac the server runs on and ends on a session of the coding agent running on it. ONE job of five idempotent steps (`runtime`, `model`, `verify`, `agent`, `connect`; `agent` only when the coding agent is missing), a pure reducer (`reduceInstallJob()`) over three tables in `install-job.ts` driven by one service per process (`getLocalModelService()`); the window polls the GET once a second, no SSE event. ⚠️ Layout only through `localModelPaths()` (all `dataPath()`): `local-model/runtime/ollama-/`, `local-model/models/`, `local-model/home/` (the runtime's PRIVATE HOME, so the user's `~/.ollama` is never touched) and the state file `local-model.json` (0600, ONE serialized writer, total reader, holds NO path); nothing is written or deleted outside `local-model/` except that file and the two writes of the connect step. ⚠️ The runtime pin (`OLLAMA_RUNTIME_PIN`: version, URL, size, sha256) is a COMPILED constant no file, setting or request may supply; the hash is compared BEFORE anything is unpacked. ⚠️ The runtime is a NON-detached child whose environment comes from the `runtimeEnv()` ALLOWLIST, never `process.env` (no `CODEMAN_*`, none of the user's `OLLAMA_*`); the port is chosen ONCE by binding (from 11435, never 11434) and stored; a listener on it is adopted or replaced only when its stored pid runs a binary under OUR runtime folder, anything else is `port-in-use`; started at boot when installed and enabled, max 3 restarts, SIGTERM at shutdown beside `stopDeepSeekWeb()`; under vitest nothing spawns or fetches. ⚠️ Consent is bound to the size: `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`) or the answer is 409 `size-changed`; the model, context and CLI must be a row the current recommendation offers (400). ⚠️ ONE job at a time (409 `job-running`); a job a restart interrupted comes back `paused` and NEVER continues without a click (so `shutdown()` writes nothing), and an interrupted step is never recorded as failed. ⚠️ An event the transition table does not list is refused (same object back, 409 `wrong-state`): add a row to the table, never a special case. ⚠️ The budget is MEASURED from the runtime's "inference compute" log line and the model judged against it BEFORE it is downloaded. ⚠️ The pull stream is parsed incrementally (about 1.4 million lines for 23 GB): never buffer it, never write state per line. ⚠️ A failure is a code with ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed"; a probe that ended on its token limit is `probe-unfinished`, not `no-tool-call`. ⚠️ The coding agent is installed through the shared runner `src/cli-install-runner.ts` (extracted from `cli-registry-routes.ts`, ONE install slot per CLI for both callers) with `stockInstallCommand()`, the COMPILED stock command, never the `clis.json`-merged one (this path has no `cliManagementEnabled` gate); only for a catalogue harness row, after the consent click that showed the literal command, audited (`cli_install`, `via: 'local-model'`); afterwards `forgetResolvedCli()` and a fresh lookup, and still missing is `harness-failed`. ⚠️ Connect writes ONE endpoint record under the RESERVED id `local-ollama` through `mutateCustomModelHosts()` and `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, remembering the flag's first value; a record with that id that is not ours is never overwritten, and while it is ours it is MANAGED: the endpoint routes refuse to edit or delete it and the discovery sweep skips it (the runtime's port has no login, so its `models` must stay the one model that was checked). ⚠️ The model digest is stored and read again at boot and before every START of a coding agent on that endpoint (`launchRefusalMessage()`): the two custom-model launch routes in `session-routes.ts` ask it before anything is written, the reboot-restore rebuild asks it before it builds (reason `local-model-not-ready`, left on offer), and `Session`'s launch guard (`setLaunchGuard()`, set in `setupSessionListeners()`) asks it right before a NEW agent process starts (a fresh pane, the respawn of a dead one, `restartCli()`), never for an attach to an agent that runs; a refusal is a `LaunchRefusedError` that never falls back to a direct PTY. A changed or missing model, a runtime Codeman did not start or that did not start, or another model id refuses the launch with one sentence. ⚠️ The connection is CLAIMED in the state file (`connection`, with the flag's value read by `inspectLocalEndpoint()`) BEFORE the endpoint record and the settings flag are written, so a state write that fails or a kill between the two never leaves a record the state file does not know; a record under the reserved id that says exactly what ours would say is ours (`isSameLocalEndpoint()`). ⚠️ A job whose end could not be written (a full disk) is kept in memory (`unrecorded`): the view shows it, the next verb writes it, and `installing` in the file while this process runs no job always reads as `paused` or that failure, never as a running job. ⚠️ The runtime's pid is stored the moment it is spawned, and Stop, Remove and the boot of anything that is not a running installation end a runtime an earlier server left behind (`supervisor.stop({ previousPid })`, ours by its command line) before any file is deleted; an installed model keeps the runtime VERSION it was installed with while that folder is unpacked, and a start that fails shows as `runtime-start-failed` with Start as the next step. Stop and "Use another port" answer 409 `sessions-open` while a live session uses the model. ⚠️ Remove (409 `sessions-open` with the names while a live session uses the endpoint) undoes the connection FIRST, restores the flag only when this feature switched it on AND no other endpoint is saved, deletes only inside `local-model/` (the models only when asked), and leaves the coding agent. ⚠️ Developer tools: only `/usr/bin/xcode-select -p`, bounded; never `git`/`python3`/`xcrun` from the server (each opens a system window on such a Mac). Tests: `test/local-model-install-job.test.ts` (every state with every event), `-install-service`, `-install-e2e` (real modules over `test/fixtures/fake-ollama.ts`), `-state-store`, `-connect`, `test/cli-install-runner.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`. → [architecture-invariants#local-model-install](docs/architecture-invariants.md#local-model-install), `docs/local-models.md`, `docs/api-reference.md` (Local model) +**Local model install** (`POST /api/local-model/install|use|cancel|resume|runtime`, `DELETE /api/local-model`, `DELETE /api/local-model/models`; `src/local-model/{install-job,runtime,ollama-client,connect,state-store,paths,developer-tools}.ts`; EXPERIMENTAL, admin only in multi-user mode on EVERY verb; guide `docs/local-models.md`): installs the chosen model on the Mac the server runs on and ends on a session of the coding agent running on it. ONE job of five idempotent steps (`runtime`, `model`, `verify`, `agent`, `connect`; `agent` only when the coding agent is missing), a pure reducer (`reduceInstallJob()`) over three tables in `install-job.ts` driven by one service per process (`getLocalModelService()`); the window polls the GET once a second, no SSE event. ⚠️ A LIBRARY with ONE model in use: the store may hold several models and `LocalModelStateFile.library` lists the ones that passed the check (written by the service when `verify` passes, carried by the reducer, dropped by `remove`; an `installed` file without it counts its one model, `libraryOf()`), but the endpoint record lists ONE model, the runtime holds one and only the state's model is ever launched: never widen the record to every checked model. `install` from `installed` adds a model (the earlier one stays: `replacedModelFor()` never names a library model), `use` switches to a checked one (a job of `USE_JOB_STEPS`, `verify` then `connect`; the stored digest is compared before anything is loaded, else `model-changed`), and both answer 409 `sessions-open` while a live session uses the endpoint; until such a job ends `installed`, no session starts (`launchRefusal()` answers `job-unfinished`). `DELETE /api/local-model/models` deletes ONE model by its NAME (`LOCAL_MODEL_ID_PATTERN`, never a path), only a name of the server's own listing, only through the runtime's own delete, never the model of the installation or of the current job (409 `model-active`), and it is no reducer event: its row is the `delete-model` action of `INSTALL_STATE_TABLE`. ⚠️ The view's `library` (state entries joined with the runtime's `/api/tags`: `ready`, `changed`, `missing`, `unverified`) and `store` (the blob folder's size, `measureModelStore()`) are cached (5 s, 30 s), refreshed in the background and thrown away by every verb and step that changes them (`forgetStore()`): never ask the runtime or the disk on the one-second poll, and a runtime that does not answer keeps the last list. ⚠️ Layout only through `localModelPaths()` (all `dataPath()`): `local-model/runtime/ollama-/`, `local-model/models/`, `local-model/home/` (the runtime's PRIVATE HOME, so the user's `~/.ollama` is never touched) and the state file `local-model.json` (0600, ONE serialized writer, total reader, holds NO path); nothing is written or deleted outside `local-model/` except that file and the two writes of the connect step. ⚠️ The runtime pin (`OLLAMA_RUNTIME_PIN`: version, URL, size, sha256) is a COMPILED constant no file, setting or request may supply; the hash is compared BEFORE anything is unpacked. ⚠️ The runtime is a NON-detached child whose environment comes from the `runtimeEnv()` ALLOWLIST, never `process.env` (no `CODEMAN_*`, none of the user's `OLLAMA_*`); the port is chosen ONCE by binding (from 11435, never 11434) and stored; a listener on it is adopted or replaced only when its stored pid runs a binary under OUR runtime folder, anything else is `port-in-use`; started at boot when installed and enabled, max 3 restarts, SIGTERM at shutdown beside `stopDeepSeekWeb()`; under vitest nothing spawns or fetches. ⚠️ Consent is bound to the size: `acknowledgedBytes` must equal the server's own total (`Candidate.installBytes`) or the answer is 409 `size-changed`; the model, context and CLI must be a row the current recommendation offers (400). ⚠️ ONE job at a time (409 `job-running`); a job a restart interrupted comes back `paused` and NEVER continues without a click (so `shutdown()` writes nothing), and an interrupted step is never recorded as failed. ⚠️ An event the transition table does not list is refused (same object back, 409 `wrong-state`): add a row to the table, never a special case. ⚠️ The budget is MEASURED from the runtime's "inference compute" log line and the model judged against it BEFORE it is downloaded. ⚠️ The pull stream is parsed incrementally (about 1.4 million lines for 23 GB): never buffer it, never write state per line. ⚠️ A failure is a code with ONE next step (`LOCAL_MODEL_ERROR_NEXT`), never a bare "failed"; a probe that ended on its token limit is `probe-unfinished`, not `no-tool-call`. ⚠️ The coding agent is installed through the shared runner `src/cli-install-runner.ts` (extracted from `cli-registry-routes.ts`, ONE install slot per CLI for both callers) with `stockInstallCommand()`, the COMPILED stock command, never the `clis.json`-merged one (this path has no `cliManagementEnabled` gate); only for a catalogue harness row, after the consent click that showed the literal command, audited (`cli_install`, `via: 'local-model'`); afterwards `forgetResolvedCli()` and a fresh lookup, and still missing is `harness-failed`. ⚠️ Connect writes ONE endpoint record under the RESERVED id `local-ollama` through `mutateCustomModelHosts()` and `customModelEndpointsEnabled: true` through `mutateSettingsFile()`, remembering the flag's first value; a record with that id that is not ours is never overwritten, and while it is ours it is MANAGED: the endpoint routes refuse to edit or delete it and the discovery sweep skips it (the runtime's port has no login, so its `models` must stay the one model that was checked). ⚠️ The model digest is stored and read again at boot and before every START of a coding agent on that endpoint (`launchRefusalMessage()`): the two custom-model launch routes in `session-routes.ts` ask it before anything is written, the reboot-restore rebuild asks it before it builds (reason `local-model-not-ready`, left on offer), and `Session`'s launch guard (`setLaunchGuard()`, set in `setupSessionListeners()`) asks it right before a NEW agent process starts (a fresh pane, the respawn of a dead one, `restartCli()`), never for an attach to an agent that runs; a refusal is a `LaunchRefusedError` that never falls back to a direct PTY. A changed or missing model, a runtime Codeman did not start or that did not start, or another model id refuses the launch with one sentence. ⚠️ The connection is CLAIMED in the state file (`connection`, with the flag's value read by `inspectLocalEndpoint()`) BEFORE the endpoint record and the settings flag are written, so a state write that fails or a kill between the two never leaves a record the state file does not know; a record under the reserved id that says exactly what ours would say is ours (`isSameLocalEndpoint()`). ⚠️ A job whose end could not be written (a full disk) is kept in memory (`unrecorded`): the view shows it, the next verb writes it, and `installing` in the file while this process runs no job always reads as `paused` or that failure, never as a running job. ⚠️ The runtime's pid is stored the moment it is spawned, and Stop, Remove and the boot of anything that is not a running installation end a runtime an earlier server left behind (`supervisor.stop({ previousPid })`, ours by its command line) before any file is deleted; an installed model keeps the runtime VERSION it was installed with while that folder is unpacked, and a start that fails shows as `runtime-start-failed` with Start as the next step. Stop and "Use another port" answer 409 `sessions-open` while a live session uses the model. ⚠️ Remove (409 `sessions-open` with the names while a live session uses the endpoint) undoes the connection FIRST, restores the flag only when this feature switched it on AND no other endpoint is saved, deletes only inside `local-model/` (the models only when asked), and leaves the coding agent. ⚠️ Developer tools: only `/usr/bin/xcode-select -p`, bounded; never `git`/`python3`/`xcrun` from the server (each opens a system window on such a Mac). Tests: `test/local-model-install-job.test.ts` (every state with every event, the library rows), `-install-service`, `-install-e2e` (real modules over `test/fixtures/fake-ollama.ts`, a second model included), `-state-store`, `-connect`, `test/cli-install-runner.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`. → [architecture-invariants#local-model-install](docs/architecture-invariants.md#local-model-install), `docs/local-models.md`, `docs/api-reference.md` (Local model) **MCP server sync** (opt-in, `mcpSyncEnabled`, SYNCED, default OFF; `src/mcp-sync.ts`, `GET`/`POST /api/mcp-sync`): copies each installed, enabled CLI's user-level MCP servers into the others. It is the ONE subsystem that writes another CLI's REAL user config (`~/.claude.json`, `~/.codex/config.toml`, `~/.gemini/*`, opencode's), which is why it is opt-in and admin-only in multi-user mode (both verbs 403 for a non-admin, and the Settings group is hidden for them). Where each CLI keeps the file is registry data, `capabilities.mcpConfig` (`{ path, format, relocation? }`), never a branch on the id. ⚠️ Tools that sync but are not run modes (GitHub Copilot CLI, the first) are plain data in `MCP_SYNC_ONLY_TOOLS` (`src/mcp-sync-targets.ts`), take part only when installed or already configured, and their companion `settings.json` (`disabledMcpServers`) is read, never written; a test must clear their relocation vars too. ⚠️ ADDITIVE only: a name already defined, in any shape, is never edited or removed (a different same-name definition is a reported conflict), and a server switched off in its own CLI is never copied. ⚠️ Never write a file that did not parse; re-parse the NEW text and require every added server to read back before the tmp+rename (written through a symlink, previous file kept as `.codeman-bak`, one apply at a time, else 409). ⚠️ A file that receives copied `env`/`headers` (secrets) is left `0600`, and so is the backup. ⚠️ Responses carry server NAMES only, never env values, headers or file text: a parse failure is reported by line and column (`describeMcpSyncError`), never the parser's own message (smol-toml and V8 both quote source). ⚠️ `mcpConfig.relocation` names the env var the CLI reads to move its file (`CLAUDE_CONFIG_DIR`, `CODEX_HOME`, `XDG_CONFIG_HOME`, `GEMINI_CLI_HOME`, and `COPILOT_HOME` for Copilot), resolved from the SERVER env at call time; a relative value reports the target `skipped`, never a guessed write, and a per-session `envOverrides` relocation is not followed. Tests must pass `home` (which drops the `process.env` default) or clear those vars first. → `docs/cli-registry.md` (MCP server sync), `docs/api-reference.md`, `docs/wiki/Settings-Reference.md` @@ -328,7 +328,7 @@ One module per domain in `src/web/routes/` (plus a barrel; `ls src/web/routes/` ## State Files -All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `local-model.json` (local model install: state, step, model, port, pid, digest, the previous `customModelEndpointsEnabled` value; mode 0600, holds no path, not part of a config backup) + `local-model/` (the pinned runtime, the model store and the runtime's private HOME, removed by `DELETE /api/local-model`), `agent-skill-default-migrated.json` (written once by the 1.50.0 boot migration of `agentSkillEnabled`; while it exists the migration never runs again, so deleting it would switch a stored off on at the next start; never tracked or restored), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back), `cli-installed` (empty, touched by `install.sh` when it installs a CLI: a running server reads only its mtime, and a change ends that CLI's negative-cache backoff) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all four installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). +All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `local-model.json` (local model install: state, step, model, port, pid, digest, the checked models of the library, the previous `customModelEndpointsEnabled` value; mode 0600, holds no path, not part of a config backup) + `local-model/` (the pinned runtime, the model store and the runtime's private HOME, removed by `DELETE /api/local-model`), `agent-skill-default-migrated.json` (written once by the 1.50.0 boot migration of `agentSkillEnabled`; while it exists the migration never runs again, so deleting it would switch a stored off on at the next start; never tracked or restored), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back), `cli-installed` (empty, touched by `install.sh` when it installs a CLI: a running server reads only its mtime, and a change ends that CLI's negative-cache backoff) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all four installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). **Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored. diff --git a/docs/api-reference.md b/docs/api-reference.md index e57e944f..8dc218ed 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1716,7 +1716,7 @@ Only replies from `allowedUserIds`, in `guildId`, inside a thread the bridge cre ## Local model -`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, and where an install of one stands. Five verbs install a model, pause and continue the install, stop and start the model runtime, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md). +`GET /api/local-model` looks at the machine the server runs on and says which local coding model fits it, with which agent CLI, where an install of one stands and which models are on the machine. Seven verbs install a model (a first one, or another one beside it), switch to a model that is already there, pause and continue a job, stop and start the model runtime, delete one model, and remove everything again. It covers Apple silicon macOS only; anywhere else the GET is still a `200`, with verdict `none`, and nothing can be installed. How it works: [local-models.md](local-models.md). **Experimental.** These routes and their payloads are labeled experimental: the fields below, the states, the error codes and the sentences may change or go in a MINOR or PATCH release (see [versioning-policy.md](versioning-policy.md), "What SemVer does NOT cover", item 3). Do not build on their shape yet. @@ -1725,11 +1725,13 @@ Every route is admin only in multi-user mode (`403 FORBIDDEN`, answered before a | Method | Path | Body | Notes | | -------- | -------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `GET` | `/api/local-model` | none | `200` with `{ hardware, recommendation, catalogUpdated, install }`. `500 INTERNAL_ERROR` only when the recommendation itself fails, never for a failed probe. | -| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed`, `size-changed`. | +| `POST` | `/api/local-model/install` | `{ modelId, cli, context, acknowledgedBytes }` | Starts the one install job and answers at once; the job runs on in the server. With a model installed it adds another one: the earlier model stays on disk, and the new one is the model in use when the job ends. `400 INVALID_INPUT` when the three choices are not a row the server offers right now. `409`: `job-running`, `already-installed` (this model at this context is the one in use), `size-changed`, `sessions-open`. | +| `POST` | `/api/local-model/use` | `{ modelId }` | Switches to a model of `install.library` whose `actions` list `use`. A job of the two steps `verify` and `connect`; nothing is downloaded. `400 INVALID_INPUT` when the model is not a checked model of the library, or no coding agent for it is installed. `409`: `job-running`, `wrong-state` (nothing is installed), `already-installed`, `sessions-open`. | | `POST` | `/api/local-model/cancel` | none | Interrupts the step in flight; the job is `paused` at that step. `409 wrong-state` when no job is running. | | `POST` | `/api/local-model/resume` | `{ newPort? }` (may be absent) | Continues a `paused` or `failed` job at the same step. `newPort: true` chooses another port first. `409`: `job-running`, `wrong-state`, `sessions-open` (`newPort` while live sessions use the model). | | `POST` | `/api/local-model/runtime` | `{ action: "start" \| "stop", newPort? }` | `stop` stops the runtime and keeps it from starting at the next boot; `start` undoes that. `newPort: true` with `start` chooses another port and moves the saved endpoint to it. `409`: `job-running`, `wrong-state` (nothing is installed), `sessions-open` (`stop` or `newPort` while live sessions use the model: their agents keep calling the address they were started with). | -| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the model only with `removeModel: true`. `409`: `job-running`, `sessions-open`. | +| `DELETE` | `/api/local-model` | `{ removeModel? }` (may be absent) | Stops the runtime, removes the saved endpoint, restores the settings flag when this feature switched it on and no other endpoint is saved, deletes the runtime; the downloaded models (all of them) only with `removeModel: true`. With nothing installed it is what deletes the models an earlier Remove kept. `409`: `job-running`, `sessions-open`. | +| `DELETE` | `/api/local-model/models` | `{ modelId }` | Deletes ONE model from the store, through the runtime. `modelId` is a model name (`name:tag`) of `install.library` whose `actions` list `delete`, never a path. `400 INVALID_INPUT` for a name that is not on this machine. `409`: `job-running`, `wrong-state` (nothing is installed), `model-active` (the model of the installation or of the current job), `runtime-not-running`. `500 OPERATION_FAILED` when the runtime did not delete it. | Every verb answers `{ success: true, data: }` with the `install` object (below) as it is after the verb. A refusal the feature names is a `409` whose envelope also carries the conflict: @@ -1747,9 +1749,9 @@ Every verb answers `{ success: true, data: }` with the `install` objec } ``` -`data.reason` is `job-running`, `already-installed` (remove the installed model first), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model) or `wrong-state`. A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix. +`data.reason` is `job-running`, `already-installed` (the model is the one in use already), `size-changed` (with `expectedBytes` and `expectedGb`), `sessions-open` (with `sessions`, the names of the live sessions that use the model), `wrong-state`, `model-active` (a delete of the model in use: switch first, or remove everything) or `runtime-not-running` (a delete while the runtime is down: start it first). A `409 CONFLICT` without `data` means a config file that must not be overwritten (`settings.json` or `custom-model-hosts.json` does not parse); `error` names the fix. -`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. Every `400` the install names, and the `507 OPERATION_FAILED` of a first install on a full disk, carries its sentence as `data.message` (`{ text, vars }`, like a `409`), so a client can print it in its own language. +`POST /api/local-model/install` also answers `400` when a saved model endpoint already uses the id `local-ollama` and was not created by this feature: rename or delete that endpoint first. The check is made before anything is downloaded. Every `400` the feature names, the `507 OPERATION_FAILED` of a first install on a full disk and the `500 OPERATION_FAILED` of a delete the runtime refused carry their sentence as `data.message` (`{ text, vars }`, like a `409`), so a client can print it in its own language. **Consent is bound to the size shown.** `acknowledgedBytes` must be the `installBytes` of the chosen row: the model download plus the runtime download while the runtime is not unpacked yet. The server compares it with its own total before anything starts and answers `409 size-changed` when they differ, so a click confirms exactly the size the person saw. @@ -1775,12 +1777,13 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or - `tag?`: `{ kind, label }`, the one tag of the row. `kind` is `tested`, `best`, `tight`, `slow` or `small`; - `fit`: `good` (up to 0.85 of the budget) or `tight` (up to 0.98), with its label `fitLabel`; - `speed`: `ok`, `slow` (under 10 tokens per second) or `unknown`, with `speedLabel` (absent for `ok`) and `tokensPerSecond` (rounded to one decimal, absent for `unknown`; `speed` is decided on the unrounded figure, so `10` can be `slow`). `unknown` means the chip is in no row of the catalogue's chip table, or the catalogue has no speed figure for the model: a speed is never guessed; - - `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run. + - `tested`: we ran this model with this CLI on a Mac like this one (same chip, same memory size). Its `tokensPerSecond` is then the speed measured in that run; + - `library?`: the model at this context is a checked model on this machine already (see `install.library`). `active`: it is the model in use. `ready`: `POST /api/local-model/use` switches to it, and nothing is downloaded. Absent otherwise. - `budgetBytes` and `budgetGb`: the memory a model may use on this Mac (`0` for `not-macos` and `intel`, where none is computed). `budgetSource` is `estimated` (computed from the installed memory: all there is before the runtime ran), `wired-limit` (taken from `iogpu.wired_limit_mb`) or `measured` (the device total the runtime itself printed when it started; the GET uses it from the first install on). - `catalogUpdated`: ISO date on which the catalogue's rows were last checked. - `install`: where the install stands and what can be done about it. - `state`: `idle` (nothing installed, no job), `installing`, `paused` (Cancel or a server restart stopped the job; it never continues without a click), `failed` or `installed`. - - `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed. + - `step` and `steps[]` (present with `installing`, `paused` and `failed`): the step the job is in or stopped in, and the steps of this job in order, out of `runtime`, `model`, `verify`, `agent`, `connect`. `agent` is absent from a job whose agent CLI was already installed. A switch (`POST .../use`) is the two steps `verify` and `connect`. - `progress` (while a download runs, and as it stood on a paused or failed job): `completedBytes`, `totalBytes`, `completedGb`, `totalGb`, and while it moves `bytesPerSecond`, `mbPerSecond` (whole decimal megabytes per second as text) and `secondsLeft`. During the `model` step both byte figures come from the download's own lines, not from the catalogue. - `error` (with `failed`; with `installed` only when something blocks a session): `{ code, message: { text, vars? }, next, nextModel?, detail? }`. `next` is the ONE action that helps: `retry` (`POST .../resume`), `other-port` (`POST .../resume` or `.../runtime` with `newPort: true`), `next-model` (choose again; `nextModel` is `{ modelId, modelLabel }` when the server knows one), `start` (`POST .../runtime`) or `remove` (`DELETE`). `detail` is raw text for the curious (an installer's last lines), not translated. - `model` (`{ id, label }`), `cli`, `cliLabel`, `context`: the choice of the job or of the installation. `port`: the runtime's loopback port, once chosen. @@ -1790,7 +1793,9 @@ A probe that fails is an answer, not an error: a server that is not on macOS, or - `developerTools`: `present`, `missing` (the Mac has no Xcode or Command Line Tools, so macOS will offer to install them the first time an agent runs `git` or `python3`) or `unknown`. - `runtime`: `{ version, installed, downloadBytes, downloadGb, unpackedGb }`. `downloadBytes` is `0` once the runtime is unpacked. - `folder`: the absolute folder everything of this feature is kept in. - - `actions[]`: exactly what the server accepts right now, out of `install`, `cancel`, `resume`, `start`, `stop`, `remove` and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session). + - `library[]`: the models on this machine, the model in use first. Empty with `idle` (no runtime is there to ask). Each row: `id` (the name the runtime lists, `name:tag`), `label`, `context?` (the context it was checked with), `sizeBytes?` and `sizeGb?` (its size on disk as the runtime reports it, else as recorded when it was checked), `active` (it is the model in use), `verified` (Codeman checked it), `status` and `actions[]`. `status` is `ready`, `changed` (checked, and the runtime now reports another digest under the name), `missing` (checked, and the runtime no longer lists it) or `unverified` (the runtime lists it and Codeman never checked it: a model an earlier Remove kept, the model of a job that has not passed its check, or one another program pulled through the runtime's port). `actions` is what the server accepts for THAT row right now, out of `use` and `delete`; `delete` needs the running runtime, and a name that is not a plain `name:tag` is listed with no action. + - `store?`: `{ bytes, gb }`, what the model store takes on disk (the files of its blob folder, so shared layers count once and an interrupted download is included). Present in every state once it was measured; cached for about thirty seconds. + - `actions[]`: exactly what the server accepts right now, out of `install` (with `installed`: add another model), `cancel`, `resume`, `start`, `stop`, `remove` (with `idle` only while `store` holds something), `use` and `delete-model` (only while a row of `library` offers them) and `start-session` (not a route here: start a session on `endpointId`; offered only while the runtime runs and nothing blocks a session). Error codes (`install.error.code`), each with one next step: @@ -1800,9 +1805,11 @@ Error codes (`install.error.code`), each with one next step: | `start` | `runtime-stopped` (the runtime crashed three times in a row and is no longer restarted) | | `other-port` | `port-in-use` | | `next-model` | `model-not-found`, `does-not-fit`, `no-tool-call` | -| `remove` | `model-changed` (the model's digest is no longer the one stored at install time; a session on it is refused) | +| `remove` | `model-changed` (the model's digest is no longer the one stored when it was checked; a session on it is refused, and so is a switch to it) | -Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, when the model's digest changed, or for a model other than the installed one. +Two codes are only ever the reason of a refused session start, never on `install.error`: `runtime-not-running` (the person stopped the runtime) and `job-unfinished` (a job that changes the model in use is running or stopped: nothing is started until it ended `installed` again). + +Starting a session on the saved endpoint goes through the existing custom model launch (`POST /api/quick-start` with `customModel`, or `POST /api/sessions/:id/custom-model`). For the endpoint this feature manages both answer `OPERATION_FAILED` with one sentence while the runtime is not running, while a job that changes the model in use is running or stopped, when the model's digest changed, or for a model other than the one in use. Every computed size a person reads travels twice: as bytes, and as decimal GB text without the unit (`"needGb": "29.6"`). Show the text and add the unit; do not divide the bytes yourself, so one request never shows two different roundings. The installed memory is the exception named above: show `memClassGb`. diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index d280b477..dead6b93 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -720,11 +720,19 @@ Either way the actual launch (`runCustomModelEntry`) routes through `run()` itse **Remove** stops the child, undoes the connection FIRST (a refused write of the endpoint store or of the settings rejects before any file is deleted), restores the flag only when this feature switched it on AND no other endpoint is saved, then deletes the runtime and the private home and, when asked, the models (`removeLocalModelFiles()`, which refuses any path outside the root). ⚠️ While a live session uses the endpoint the answer is 409 `sessions-open` with the session names (`sessionNamesUsingEndpoint()`, read at the moment of the request). ⚠️ The coding agent stays: what its installer wrote is not ours to remove. +**The library** (one model in use, several on disk). `LocalModelStateFile.library` lists every model that PASSED the check through Codeman: `{ id, label, context, digest, bytes?, cli?, verifiedAt? }`, no path. The service upserts the entry as a fact when `verify` passes (where it stores the digest); the reducer only carries the list, through every event but `remove`. An `installed` file an earlier build wrote has no list: `libraryOf()` then counts its one model, and an install or a switch from it writes that model into the list before the state's `model` member becomes the new job's. ⚠️ ONE model is in use: the endpoint record still lists exactly one model, the runtime still holds one (`OLLAMA_MAX_LOADED_MODELS=1`), and `launchRefusalMessage()` still starts only the state's model with its stored digest. Never widen the record to "every checked model": the port has no login, and two sessions on two models would make the runtime swap 20 GB on every request. + +⚠️ Three verbs, each a row of the tables and never a special case. **Add**: `install` is also listed for `installed` in `INSTALL_TRANSITIONS`; the job is the same five steps, and the earlier model stays (the `replacedModelId` rule, `replacedModelFor()`, never names a model of the library: only an unchecked or partial download is deleted as a leftover). **Switch**: the event `use` (`installed`, `paused`, `failed`) starts a job of `USE_JOB_STEPS` (`verify`, `connect`) for a library entry, carrying its stored digest in `model.digest`; `stepVerify` compares that digest BEFORE it loads anything and again after the check, and a different or missing one fails with `model-changed`. A switch has no `model` step, so the leftovers of the job it replaced are deleted at the start of `verify` (`dropReplacedModel()`). Its coding agent is the entry's own when that is installed now, else the state's (`recheckCliInstalled()`), else a 400: a job that stopped while it installed another agent leaves the state naming one that is not there. **Delete** (`deleteLibraryModel()`) moves no state, so it is not an event; whether a state allows it is the `delete-model` action of its `INSTALL_STATE_TABLE` row. It takes a model NAME (the request schema and the service both hold it to `LOCAL_MODEL_ID_PATTERN`), requires that name in the server's own listing (the library or the runtime's tags), goes through the runtime's `/api/delete` only (never a file delete, never a path), and trusts only the runtime's answer that the model is gone, never an empty listing. ⚠️ It refuses the model of the installation or of the current job (`model-active`), a running job, and a runtime that is down or does not answer (`runtime-not-running`). + +⚠️ `install` from `installed` and `use` are refused with 409 `sessions-open` while live sessions use the endpoint, like Stop: both change which model the runtime holds and often restart it for another context. ⚠️ While a job that started from `installed` is unfinished (`installing`, `paused`, `failed`) the launch check refuses every start with `job-unfinished` (a code only a refused start says, like `runtime-not-running`), because it wants the state `installed`; the view then offers `use` on the other checked models, which is the way back. A switch is never offered for the model of the stopped job itself: Resume goes on with that one. + +**The view's two listings.** `install.library` (`buildLibraryRows()`, pure) joins the state's entries with what the RUNNING runtime lists (`/api/tags`): `ready`, `changed` (another digest under the name), `missing` (no longer listed) for a checked model, `unverified` for anything else the runtime lists (a model an earlier Remove kept, the model of a job that has not passed its check, one another program pulled through the unauthenticated port), which can be deleted and never switched to. A name that is not a plain `name:tag` is listed with no action. `install.store` is `measureModelStore()`: the sizes of the regular files directly in `models/blobs`, nothing followed, nothing outside the root. ⚠️ Neither is asked for on the one-second poll: `tagsCache` (5 s) and `storeCache` (30 s) are refreshed in the background, only a figure that was never read is waited for (300 ms at most), a runtime that does not answer keeps the last list (or every checked model would read as `missing` for a moment), and every verb and finished step that changes the store throws both away (`forgetStore()`). ⚠️ In `idle` there is no runtime to ask, so the list is empty and only `store` says that an earlier Remove kept models; `remove` is listed there only while it holds something, and `DELETE /api/local-model { removeModel: true }` is what frees it. `Candidate.library` (`active`, `ready`) is added by the ROUTE (`markLibraryCandidates()`), never by `recommend()`, which stays pure and knows nothing of what is installed. + **Developer tools**: `checkDeveloperTools()` runs ONLY `/usr/bin/xcode-select -p`, bounded, on darwin. ⚠️ Never `git`, `python3` or `xcrun` from the server: on a Mac without the tools each opens a system window. The view carries `developerTools`, and the window prints the one sentence when it is `missing`. **The endpoint store's writer** (`src/custom-model-hosts-writer.ts`): `mutateCustomModelHosts(fn)` is the shape of `mutateSettingsFile()`: one promise chain, a random temp name, a refusal (409) to write over a file that does not parse. ⚠️ EVERY writer of `custom-model-hosts.json` goes through it: the four endpoint routes, the sweep and the connect step (`test/custom-model-hosts-writer.test.ts` fails a `writeCustomModelHosts()` call anywhere else in `src/`). The routes used to read, change and write on their own through one shared temp name, so two creates sent together lost a record or failed, and a file that did not parse was read as `[]` and replaced by one record, with every other endpoint's API key gone. ⚠️ A route's check runs INSIDE the callback, and discover splices onto the record as it is after the probe, not the copy read before it. -Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event), `test/local-model-install-service.test.ts`, `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`. +Tests: `test/local-model-install-job.test.ts` (the tables and the reducer, every state with every event, the library rows), `test/local-model-install-service.test.ts` (the three library verbs and the two caches included), `test/local-model-install-e2e.test.ts` (the real modules over the fake runtime: install A, add B, switch back, delete B), `test/local-model-state-store.test.ts`, `test/local-model-connect.test.ts`, `test/custom-model-hosts-writer.test.ts`, `test/cli-install-runner.test.ts`, `test/routes/local-model-routes.test.ts`, `test/routes/local-model-launch-guard.test.ts`, and the runtime's own `test/local-model-runtime.test.ts`, `test/local-model-ollama-client.test.ts`. The fake runtime is `test/fixtures/fake-ollama.ts`. ### Approvals Inbox diff --git a/docs/local-models.md b/docs/local-models.md index 485c40cd..a90d4817 100644 --- a/docs/local-models.md +++ b/docs/local-models.md @@ -4,7 +4,9 @@ Codeman can set up a coding model that runs on the Mac the server runs on: no ac no API key, and the session's prompts and files go to a model on this Mac rather than a cloud provider (the coding agent itself may still check for updates). One button on the welcome screen opens a window that shows what fits this Mac, installs the choice and ends on a coding agent session that -runs on it. +runs on it. The same window then lists the models that are on this Mac with what each takes +on disk, adds another one, switches between them and deletes one +([The models on this Mac](#the-models-on-this-mac)). **Experimental.** The routes and their payloads may change in any release ([versioning-policy.md](versioning-policy.md), "Experimental local model routes"). Apple @@ -103,7 +105,9 @@ for a row it does not know. Its answer for each memory size is then pinned in ## What an install does, in order The person picks a model, presses "Install on this Mac" and confirms the sizes once. The -server then runs one job of five steps. The window polls `GET /api/local-model` once a +server then runs one job of five steps. The same job adds a second model to an installed one +("Add a model"): the runtime is there already, so it downloads the model alone, and the +earlier model stays on disk. The window polls `GET /api/local-model` once a second; the job runs in the server, so the window can be closed. | Step | What it does | @@ -244,10 +248,13 @@ Switching the setting on also starts that five-minute sweep for every other save existing custom model launch. Before a coding agent is STARTED on that endpoint the server checks, in this order: +- no job that changes the model in use is running or stopped: until an install or a switch + ended, nothing is started ("The model on this Mac is being changed and is not ready"); - the runtime child Codeman started is alive (whatever else answers on the port is not ours); -- the digest the runtime reports for the installed model is the one stored at install time; -- the model asked for is the one that was installed and checked. +- the digest the runtime reports for the model in use is the one stored when it was checked; +- the model asked for is the model in use. Another model that is on this Mac is not started + on the endpoint: switch to it in the window first. That check runs wherever a new agent process comes to be on the endpoint: a session created with the model, a model applied to a running session (which restarts its agent), the attach @@ -273,22 +280,75 @@ answer back from a file it writes in its own folder. The skill never installs a answers a confirmation, the runtime still answers one request at a time, and while a worker it opened is live, Stop and Remove are refused until it is closed. +## The models on this Mac + +The model store can hold several models. **One of them is the model in use**: the one the +endpoint record `local-ollama` lists, the runtime is started for and a session can be +started on. The runtime holds one model in memory at a time (`OLLAMA_MAX_LOADED_MODELS=1`), +so this is what the Mac can really do, and it keeps every rule above: one record, one +model, one stored digest per model. + +The installed screen lists what is on this Mac, the model in use first: + +| A row says | It means | +| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| In use | The model sessions start on | +| Ready | Codeman downloaded and checked it. **Use** switches to it | +| Not checked | The runtime lists it and Codeman never checked it: a model an earlier Remove kept, or one another program pulled through the runtime's port. It can only be deleted | +| Changed | It was checked, and the runtime now reports another digest under its name. It is never started; delete it and install it again | +| Missing | It was checked, and the runtime no longer lists it. **Delete** drops the entry | + +Each row shows the context the model was checked with and its size on disk, as the runtime +reports it. Under the list is what all downloads take together: the size of the files in +`local-model/models/blobs`, so layers two models share are counted once and an interrupted +download is included. The window asks the runtime for the list at most every five seconds +and measures the store at most every thirty, never on every poll. + +- **Add a model** opens the chooser. A model that is not on this Mac is installed like the + first one, with its sizes confirmed; when the job ends it is the model in use, and the + earlier model stays in the list. A row that is on this Mac already says so and is switched + to, without a download and without a confirm screen. +- **Use** switches to a checked model. It is a job of two steps, `verify` and `connect`: + the runtime is restarted with the context the model was checked with, the model is loaded + and checked again (the tiny chat, the GPU, the tool probe), its digest must still be the + stored one (else the switch fails with `model-changed` before the model is loaded), and + the endpoint record is rewritten. The coding agent is the one the model was checked with + when it is still installed, else the one of the installation. +- **Delete** removes one model through the runtime's own delete, after one question in the + row. The request carries the model's name, never a path, and the server deletes only a + name of its own listing. The model in use cannot be deleted: switch first, or Remove. It + needs the running runtime, so a stopped model shows no Delete until Start. + +Adding a model and switching are refused while a session uses the model ("Sessions are +still using this model"), like Stop: their coding agents were started with the other model's +name, and the runtime would swap models on every request. + +While a job that started from an installed model runs, or after it stopped (paused or +failed), no session is started: the checks above want a finished installation, and the +refusal says so (`job-unfinished`). The window +then lists the checked models with **Use**, which is the way back to the earlier one. Going +back deletes what the stopped job left of a model that never passed its check, the way a new +install does; a checked model is never deleted that way. + ## Stop, Start, Remove - **Stop** stops the runtime and keeps it from starting at the next boot. **Start** undoes that. - **Remove** stops the runtime, removes the endpoint record, switches the setting off again only when this feature switched it on and no other endpoint is saved, and deletes the - runtime and its private home. The downloaded model is deleted only when asked. The coding - agent stays. While a live session uses the model, Remove answers with the session names - instead. + runtime and its private home. The downloaded models (all of them) are deleted only when + asked. The coding agent stays. While a live session uses the model, Remove answers with + the session names instead. +- **Models a Remove kept** are not lost space: with nothing installed the window says how + much they still take and offers "Delete them", and after a new install they show in the + list as "Not checked", each with its own Delete. ## Where everything is All under the data directory (`~/.codeman`, or the instance's own): ``` -local-model.json the state file (mode 0600) +local-model.json the state file (mode 0600), with the list of checked models local-model/ runtime/ollama-0.40.15.tgz.partial the runtime archive while it downloads runtime/ollama-0.40.15/ the unpacked runtime (about 579 MB) @@ -313,9 +373,12 @@ in App Settings. | ------------ | ------------------------------------------------------------- | ---------------- | -------------------------------------- | | `idle` | stays idle | refused | install | | `installing` | becomes `paused` at the same step | becomes `paused` | cancel | -| `paused` | stays paused | refused | resume, install, remove | -| `failed` | stays failed, same error | refused | resume, install, remove | -| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, remove | +| `paused` | stays paused | refused | resume, install, use, delete a model, remove | +| `failed` | stays failed, same error | refused | resume, install, use, delete a model, remove | +| `installed` | stays installed; the runtime is started unless it was stopped | refused | start a session, stop or start, install (add a model), use, delete a model, remove | + +`use` and "delete a model" are listed only while a row of the list offers them, and in +`idle` "remove" is offered only while the model store still holds something. One job at a time. A job a server restart interrupted comes back paused and never continues without a click. Cancel interrupts the step in flight (the download, the pull, the request, @@ -357,6 +420,12 @@ or `python3`. on a Mac. "Recommended" appears only for the pair we ran, on the Mac we ran it on. - **The model stays in memory for ten minutes** after the last request, then the runtime unloads it. +- **One model runs at a time.** Several can be on disk; the runtime holds one, and a switch + restarts it and loads the other model (seconds to a minute). Close the sessions on the + model first: the window refuses the switch while one is open. +- **The disk figures are the runtime's.** A row's size is what the runtime reports for the + model, and the total is the size of its blob files. Neither was compared with Finder on a + Mac yet. ## Keeping the catalogue current diff --git a/docs/wiki/Custom-Model-Endpoints.md b/docs/wiki/Custom-Model-Endpoints.md index 4ee5aa59..fee21861 100644 --- a/docs/wiki/Custom-Model-Endpoints.md +++ b/docs/wiki/Custom-Model-Endpoints.md @@ -234,6 +234,11 @@ Good to know: but it is changed and removed in the local model window, not here. - Codeman starts the runtime with the server and stops it with it. **Stop** keeps it from starting next time. +- More than one model can be on the Mac, and one of them runs at a time. Once a model is + installed the window lists the models with what each takes on disk: **Add a model** + downloads another one, **Use** switches to a model that is already there (nothing is + downloaded; close the sessions on the model first), and **Delete** removes one, after one + question. The model in use cannot be deleted: switch first, or use **Remove**. - **Remove** deletes the runtime and the endpoint, and the model too if you tick that. The coding agent stays installed. - If the model on disk is no longer the one that was installed, Codeman refuses to start a diff --git a/src/web/CLAUDE.md b/src/web/CLAUDE.md index 9268b440..fd47f956 100644 --- a/src/web/CLAUDE.md +++ b/src/web/CLAUDE.md @@ -48,7 +48,7 @@ Frontend JS modules have `@fileoverview` with `@dependency`/`@loadorder` tags. L **Opened outside Codeman** (`foreign-sessions.js`, both home screens): the tmux sessions a human started, listed from `GET /api/mux/foreign` (root CLAUDE.md, Adopted tmux sessions). ONE renderer, `renderForeignSessions(container)`, fills the welcome block and the phone overview's, and every row goes through `textContent`, since a session name and a command line come from someone else's process. ⚠️ It polls only while a home screen is up (`startForeignPolling`/`stopForeignPolling` from `showWelcome()`/`hideWelcome()`) and the tab is visible (`_foreignPollTick()` is the one gate, and `visibilitychange` catches up; the cache TTL sits under the poll interval, so one tab's every tick is a full socket sweep), and the poll is local only: the docker and ssh scan costs one connection per target, so it runs once on the toggle and again on Rescan, and its rows and notes are kept apart (`_foreignWideSessions`, `_foreignWideNotes`) or the next local poll would wipe them. ⚠️ A 403 is a non-admin in multi-user mode, an answer and not a failed poll: `_foreignRefused` stops the polling and shows no block. ⚠️ `closeSession()` re-reads the list after an adopted tab's DELETE, since the home screen's own load races it and the row would say `Go to tab` until the next poll. ⚠️ An adopted tab has no kill outcome to offer, so the close dialog hides the red button and always tears the wrapper down. ⚠️ `session.remote` and `session.docker` are undefined on an adopted session, so a check that hides a row for those two usually needs `session.adopt` as well. Tests: `test/foreign-sessions-ui.test.ts`, `test/foreign-sessions-i18n.test.ts` (harvests every string from the real module). -**Run a model on this Mac** (`local-model-ui.js`; welcome button `#welcomeLocalModelBtn`, chooser window `#localModelModal`, App Settings > Models row `#localModelSettings`; server rules in the root CLAUDE.md, Local model): ONE window does everything (choose, confirm, progress, paused, failed, installed, no recommendation); the button and the Settings row only open it. Which screen shows follows `install.state` of `GET /api/local-model`; the page itself holds only the selection and whether the person is on the chooser or the confirm screen. ⚠️ The window is ONE frame for the whole flow: a fixed height on desktop (`.modal-content.local-model-window`), App Settings' boxed close button, the body scrolling between the header and a foot of its own (`#localModelWindowFoot`): every screen's last part (`.lm-foot`, or the `.lm-remove` question) is moved there by `_renderLocalModelWindow()`, so a button never moves away from under the pointer and is never clipped; a test that looks for a button looks in the frame, not the body. The short screens are centred in that frame on desktop and sit above the buttons on a phone (`data-screen` on the body). ⚠️ The memory bar (`.lm-bar`, `role="meter"`) is the one strong element: filled to the selected row's `budgetShare`, the model at its PEAK (the legend says so, and the installed line uses the same words beside what the model takes now, so the two figures are never read as a contradiction), the memory map (`Candidate.memory`: the model, its context and the brief peaks side by side, each named in the legend, on a track of `--text-muted` at about a third) in the feature's own colours (`--lm-*`: white for the feature, turned over to dark ink on the light skins, blue the context, green good, amber tight, orange slow, rose the brief peaks, deeper on the light skins), never `--accent`, which the owner rejected as too much blue (about 3:1 against the window, so the free part can be seen; a border token is 1.5:1 and the inputs' `--bg-input` is the window's own surface on the dark skins), 10px, no gradient or glow, and its width transition is the only motion (off under `prefers-reduced-motion`); a repaint starts each bar at the last width of its kind, so keep `_settleLocalModelBars()` after every body write. The figure beside it is an estimate and says "about" in whole GB (`localModelWholeGb()` on the server's text); only the download figure keeps tabular figures. ⚠️ The rows are the server's pick first, then the other models by download size, largest first (`localModelCandidates()`): the server's alternatives are its ranking, which no column shows. ⚠️ The download bar means ONE thing, the model download: at the start while the runtime is fetched, gone once the model is in (the step list carries verify, agent and connect); the first paint after an install finished shows "Start a session" alone (`justInstalled`), an error on an installed model is the headline with a grey dot, and Remove is quiet everywhere but red in its own question, never the accent. ⚠️ The window polls the GET once a second WHILE OPEN and never otherwise (no SSE event): `closeLocalModelWindow()` clears the timer, a background tab asks nothing, an unchanged answer repaints nothing, and a tick that moved only the download replaces `.lm-meter` alone, so a button is never swapped out under a click. A read that was already under way when a verb answered is dropped (`_localModelVerbs`), or the old state flashes back. ⚠️ The consent is bound to the size shown: the install request carries the row's `installBytes` as `acknowledgedBytes`, and a 409 `size-changed` shows `expectedGb` and sends `expectedBytes` next; never compute a total here. ⚠️ A button exists only for an action in `install.actions`, and after an error exactly one helpful action (`error.next`, already adapted by the server to the state: `start` on an installed model, `retry` on a job) plus a quiet Remove; an error's text is split after its first sentence (`localModelSentences()`, which knows the ideographic full stop) and only that sentence is the headline. A refusal the server names (a 400, a full disk) carries `data.message` like a 409 and is printed from it, never from the English `error` text. ⚠️ Every dynamic string goes in through `textContent` (model, coding agent and chip names, an error's detail, session names, the folder and the install command are someone else's text); the window body, the button and the row carry `data-i18n-skip` and the module translates its own sentences at render time, so a language or display-name change repaints from the module (the `` observer). ⚠️ Labels and sentences come from the payload (`{ text, vars }` through `localModelT`); the module holds no list of models or coding agents, compares no CLI id (the agent select's values are positions in the server's list, the id travels only as data into the install request and `selectCustomModelEntry`) and never contains the word "Recommended". ⚠️ `test/local-model-i18n.test.ts` harvests every `localModelT('...')` literal: a new string is a single-quoted literal with no apostrophe and gets a zh-CN entry, the only non-literal calls are the two payload ones, and a unit alone (`23.1 GB`) is not a translator call. ⚠️ Sizes are printed as sent (decimal GB text), bytes are never turned into a size; the installed memory alone is `memClassGb`. ⚠️ The model list is a real radio group (`role="radiogroup"`, roving `tabindex`, arrows, Home, End, the focus follows through the repaint's own focus restore, keyed by `data-lm-key`); the columns line up through `subgrid` with a fixed-track `@supports` fallback, and under 560px a row is two lines (mobile.css). ⚠️ The window has its OWN `FocusTrap` (`_localModelFocusTrap`, never `activeFocusTrap`: it opens on top of App Settings) and app.js claims its Escape before `closeAllPanels()`. ⚠️ Its buttons are the Git worktrees dialog's `.wt-btn` set, never `.btn-toolbar`: the skin block greys `.btn-toolbar.btn-primary` and the phone block pins every `.btn-toolbar` to 26px with `!important`. ⚠️ "Start a session" re-reads the settings and the CLI list first (`loadAppSettingsFromServer`, `loadCliListForSettings`: the server switched the endpoints flag on and may have installed the agent a moment ago), then calls the existing `selectCustomModelEntry(install.cli, install.endpointId)`. The button ships `hidden` and shows only once the server answered for a Mac ("Local model" once one is installed); button AND Settings group stay hidden for `not-macos`, for a refusal and while the first request is in flight; refused means `403` with `errorCode: FORBIDDEN` (the auth layer's `403 PASSWORD_CHANGE_REQUIRED` is a failed request, asked again on `codeman:me`). Remove asks first and deletes the downloaded model unless the box is cleared (nothing else in the UI can free that space later). Not on the phone overview. Tests: `test/local-model-ui.test.ts`, `test/local-model-i18n.test.ts`; the visual walk is a scratch Playwright script, not a test. +**Run a model on this Mac** (`local-model-ui.js`; welcome button `#welcomeLocalModelBtn`, chooser window `#localModelModal`, App Settings > Models row `#localModelSettings`; server rules in the root CLAUDE.md, Local model): ONE window does everything (choose, confirm, progress, paused, failed, installed, no recommendation); the button and the Settings row only open it. ⚠️ The models on this Mac (`install.library`) are ONE list builder, `_localModelLibraryEl()`: on the installed screen always (the model in use first, in the chooser's selected look, each row with the context it was checked with, a state tag and its size on disk), on a paused or failed screen only when a row offers something (the way back to an earlier model), never on the screen that ends an install. A row has "Use" and "Delete" only for what the server lists for THAT row (`row.actions`); Use is a switch with no confirm screen (nothing is downloaded), Delete asks once in place under its row (`.lm-lib-ask`, the one red button, the focus on Cancel), and the asked row keeps its buttons unseen (`.lm-lib-actions-away`) so the sizes do not jump sideways. Under the list: what all downloads take (`install.store`) and "Add a model", which opens the chooser from an installed model; there a row with `candidate.library` says "In use" or "On this Mac" and the main button becomes a disabled "In use" or "Use this model". With nothing installed and a store an earlier Remove kept, the chooser and the nothing-fits screen say what it takes and offer "Delete them" (`_localModelStoreEl()`). ⚠️ A screen that carries the list sets `data-long` on the body, which switches it from the centred short-screen layout (bottom-aligned on a phone) to top-aligned: a centred flex column that overflows cuts off its first lines. ⚠️ The row buttons live in `.lm-lib-actions`, not `.lm-buttons` (the screen's own buttons in the foot), and are a size smaller (`.lm-btn-row`). ⚠️ A job whose steps hold no `model` step is a switch (`localModelIsSwitch()`) and is named one ("Switching to", no download bar). Which screen shows follows `install.state` of `GET /api/local-model`; the page itself holds only the selection and whether the person is on the chooser or the confirm screen. ⚠️ The window is ONE frame for the whole flow: a fixed height on desktop (`.modal-content.local-model-window`), App Settings' boxed close button, the body scrolling between the header and a foot of its own (`#localModelWindowFoot`): every screen's last part (`.lm-foot`, or the `.lm-remove` question) is moved there by `_renderLocalModelWindow()`, so a button never moves away from under the pointer and is never clipped; a test that looks for a button looks in the frame, not the body. The short screens are centred in that frame on desktop and sit above the buttons on a phone (`data-screen` on the body). ⚠️ The memory bar (`.lm-bar`, `role="meter"`) is the one strong element: filled to the selected row's `budgetShare`, the model at its PEAK (the legend says so, and the installed line uses the same words beside what the model takes now, so the two figures are never read as a contradiction), the memory map (`Candidate.memory`: the model, its context and the brief peaks side by side, each named in the legend, on a track of `--text-muted` at about a third) in the feature's own colours (`--lm-*`: white for the feature, turned over to dark ink on the light skins, blue the context, green good, amber tight, orange slow, rose the brief peaks, deeper on the light skins), never `--accent`, which the owner rejected as too much blue (about 3:1 against the window, so the free part can be seen; a border token is 1.5:1 and the inputs' `--bg-input` is the window's own surface on the dark skins), 10px, no gradient or glow, and its width transition is the only motion (off under `prefers-reduced-motion`); a repaint starts each bar at the last width of its kind, so keep `_settleLocalModelBars()` after every body write. The figure beside it is an estimate and says "about" in whole GB (`localModelWholeGb()` on the server's text); only the download figure keeps tabular figures. ⚠️ The rows are the server's pick first, then the other models by download size, largest first (`localModelCandidates()`): the server's alternatives are its ranking, which no column shows. ⚠️ The download bar means ONE thing, the model download: at the start while the runtime is fetched, gone once the model is in (the step list carries verify, agent and connect); the first paint after an install finished shows "Start a session" alone (`justInstalled`), an error on an installed model is the headline with a grey dot, and Remove is quiet everywhere but red in its own question, never the accent. ⚠️ The window polls the GET once a second WHILE OPEN and never otherwise (no SSE event): `closeLocalModelWindow()` clears the timer, a background tab asks nothing, an unchanged answer repaints nothing, and a tick that moved only the download replaces `.lm-meter` alone, so a button is never swapped out under a click. A read that was already under way when a verb answered is dropped (`_localModelVerbs`), or the old state flashes back. ⚠️ The consent is bound to the size shown: the install request carries the row's `installBytes` as `acknowledgedBytes`, and a 409 `size-changed` shows `expectedGb` and sends `expectedBytes` next; never compute a total here. ⚠️ A button exists only for an action in `install.actions`, and after an error exactly one helpful action (`error.next`, already adapted by the server to the state: `start` on an installed model, `retry` on a job) plus a quiet Remove; an error's text is split after its first sentence (`localModelSentences()`, which knows the ideographic full stop) and only that sentence is the headline. A refusal the server names (a 400, a full disk) carries `data.message` like a 409 and is printed from it, never from the English `error` text. ⚠️ Every dynamic string goes in through `textContent` (model, coding agent and chip names, an error's detail, session names, the folder and the install command are someone else's text); the window body, the button and the row carry `data-i18n-skip` and the module translates its own sentences at render time, so a language or display-name change repaints from the module (the `` observer). ⚠️ Labels and sentences come from the payload (`{ text, vars }` through `localModelT`); the module holds no list of models or coding agents, compares no CLI id (the agent select's values are positions in the server's list, the id travels only as data into the install request and `selectCustomModelEntry`) and never contains the word "Recommended". ⚠️ `test/local-model-i18n.test.ts` harvests every `localModelT('...')` literal: a new string is a single-quoted literal with no apostrophe and gets a zh-CN entry, the only non-literal calls are the two payload ones, and a unit alone (`23.1 GB`) is not a translator call. ⚠️ Sizes are printed as sent (decimal GB text), bytes are never turned into a size; the installed memory alone is `memClassGb`. ⚠️ The model list is a real radio group (`role="radiogroup"`, roving `tabindex`, arrows, Home, End, the focus follows through the repaint's own focus restore, keyed by `data-lm-key`); the columns line up through `subgrid` with a fixed-track `@supports` fallback, and under 560px a row is two lines (mobile.css). ⚠️ The window has its OWN `FocusTrap` (`_localModelFocusTrap`, never `activeFocusTrap`: it opens on top of App Settings) and app.js claims its Escape before `closeAllPanels()`. ⚠️ Its buttons are the Git worktrees dialog's `.wt-btn` set, never `.btn-toolbar`: the skin block greys `.btn-toolbar.btn-primary` and the phone block pins every `.btn-toolbar` to 26px with `!important`. ⚠️ "Start a session" re-reads the settings and the CLI list first (`loadAppSettingsFromServer`, `loadCliListForSettings`: the server switched the endpoints flag on and may have installed the agent a moment ago), then calls the existing `selectCustomModelEntry(install.cli, install.endpointId)`. The button ships `hidden` and shows only once the server answered for a Mac ("Local model" once one is installed); button AND Settings group stay hidden for `not-macos`, for a refusal and while the first request is in flight; refused means `403` with `errorCode: FORBIDDEN` (the auth layer's `403 PASSWORD_CHANGE_REQUIRED` is a failed request, asked again on `codeman:me`). Remove asks first and deletes the downloaded model unless the box is cleared (nothing else in the UI can free that space later). Not on the phone overview. Tests: `test/local-model-ui.test.ts`, `test/local-model-i18n.test.ts`, and `test/local-model-ui.browser.test.ts` (real Chromium, not in the gate: the columns, the two-line phone rows, the long list that scrolls from its top); the visual walk is a scratch Playwright script, not a test. **Home-screen session order** (`CodemanSessionOrder` in constants.js, pure): BOTH home screens (phone overview, desktop rail) must order rows through this ONE comparator. Rank `needs` → `error` → `waiting` → `working` → `idle` → `done`. ⚠️ The tiebreak flips: states a session is still IN sort oldest-first, states it has STOPPED sort newest-first. ⚠️ The running group keys off `lastSubmitAt`, never `lastActivityAt` (a working pane repaints constantly). ⚠️ A 0 stamp means unknown and sorts last within its state. Final tiebreak is `orderIndex`, so the list never shuffles. The tab strip itself is NOT sorted by this. Test: `test/session-overview-order.test.ts`. → [architecture-invariants#home-screen-session-order](../../docs/architecture-invariants.md#home-screen-session-order)