diff --git a/CLAUDE.md b/CLAUDE.md index bb78756b..d0f72db6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -156,6 +156,8 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Auto-resume on usage limit** (opt-in per session, top of the Respawn tab): when Claude halts on a subscription limit, `usage-limit-patterns.ts` (pure, unit-tested) parses the reset time and `SessionAutoOps` arms a timer for reset+2min, then sends Esc + `continue`. ⚠️ Respawn cycles are blocked while paused (`isLimitPaused` guard in `onIdleDetected`), which is what prevents `/clear` from wiping the paused conversation. Claude-mode only. → [architecture-invariants#auto-resume-on-usage-limit](docs/architecture-invariants.md#auto-resume-on-usage-limit) +**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, `configBackupDir`, `configBackupKeepCount` 1-500 default 20, `configBackupKeepDays` 0-3650 default 30; `docs/wiki/Config-Backups.md`): timestamped snapshots `cfg-YYYYMMDD-HHMMSS/` (+ `manifest.json` with SHA-256s, `reason`, `appVersion`) of a FIXED allowlist (`CONFIG_BACKUP_FILES`: settings, clis, custom-model-hosts, webhook, intents, linked-cases, remote/docker hosts+cases, webviews, push-keys, users, `.env`) into `/backups/config` or a custom absolute folder. ⚠️ `state.json` is deliberately NOT tracked (rewritten per session event; it also holds cron jobs and respawn presets). One snapshot per real content change (digest of the file set vs the latest manifest), at startup + a 5-minute timer, skipped in test mode. ⚠️ Retention (`pruneConfigBackups`) only deletes folders matching the name pattern AND holding a parseable manifest, and never the newest; a custom folder equal to the data dir falls back to the default. ⚠️ Restore verifies every file's SHA-256 BEFORE writing, takes a `pre-restore` snapshot, writes atomically at 0600 and only allowlisted names; the id is validated against the name pattern (no traversal). Snapshots are 0700/0600 because `.env`/`users.json`/`custom-model-hosts.json` carry credentials; all routes are admin-only in multi-user mode. → [docs/wiki/Config-Backups.md](docs/wiki/Config-Backups.md) + **Plan-usage chip** (`showPlanUsageLimits`, per-device: desktop default **ON**, handhelds OFF): resolve DISPLAY only through `planUsageChipEnabled()` (settings-ui.js). The same setting is the server-side COLLECTION switch, read fresh by `readPlanUsageTelemetryEnabled()` (hooks-config.ts) at every claude create/respawn. ⚠️ An ABSENT key reads as ON in the reader; `GET /api/settings` must never write. ⚠️ A save sends `showPlanUsageLimits` ONLY when it flips the chip on that device (`planUsageCollectionFlip()`), or a phone switches collection off for every desktop. Claude data comes from the statusLine exporter, injected as an EPHEMERAL `claude --settings` flag (`resolveStatusLineCliCommand`), never written to disk, WRAPPING a user's own statusLine, posting to `POST /api/status-telemetry`. Codex comes from a read-only `account/rateLimits/read` poll (main bucket only). The statusline's `prompt_cache` object (Claude Code v2.1.251+) is kept by `StatusTelemetrySchema` (every documented scalar field, null-tolerant; `parsePromptCache()` in usage-telemetry.ts normalizes only the stored subset) and shown in the footer as `cache:until HH:MM` / `cache:cold` (only when the user has no statusline of their own) and in Session Options → Respawn (`#promptCacheStatus`, fed by `Session.promptCache` in `toState()` plus the change-only `session:promptCache` SSE event, worded by the pure `CodemanPromptCache.format`): warm-until/cold, the re-cache stake, the hit rate, the last miss's causes, and a `Context compacted at HH:MM` note after a compaction or a tool-result clearing (stamped on the first warm report whose `recache_tokens_if_cold` is null after one that carried it, never from `expected_rebuilds`, which Claude Code bumps only at the next request; NEVER an alert). ⚠️ `Session.setPromptCache()` stamps `compactedAt` onto a NEW object (never mutating the parsed report) and the route broadcasts the STORED object; the last report is restored on restart (constructor `promptCache`, like `displayModel`) and the readout is time-aware (a passed `expiresAt` reads cold, not stale-warm). It is the source of truth for the cache TTL, never the plan tier. → [architecture-invariants#plan-usage-chip-statusline-telemetry](docs/architecture-invariants.md#plan-usage-chip-statusline-telemetry), `docs/usage-limits-display-plan.md` **Orchestrator**: State machine that turns a user goal into a phased plan and drives it to completion: `idle → planning → approval → executing → verifying → (replanning) → completed/failed`. `OrchestratorLoop` (engine) delegates plan generation to `orchestrator-planner` and per-phase verification gates to `orchestrator-verifier`, executing phases via team agents/`task-queue`. State persists under the `orchestrator` key in `state.json`. Distinct from Ralph (single-session autonomous loop) — orchestrator coordinates multi-phase, multi-agent execution. See `docs/orchestrator-loop-architecture.md`. diff --git a/README.md b/README.md index d3ad103a..0cdb3b6a 100644 --- a/README.md +++ b/README.md @@ -479,6 +479,7 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt - **OSC 52 clipboard**: a program in a session that sets the clipboard with OSC 52 (Claude Code's `/copy`, a tmux yank, vim) puts the text on your browser's clipboard, in the main terminal and in every tile. Clipboard read requests are refused, and replaying a buffer never copies - **MCP server sync** _(opt-in)_: copies the user-level MCP servers each installed CLI has (Claude, Codex, Gemini, OpenCode, Antigravity, and GitHub Copilot CLI) into the others' own config files. Additive only, a server you switched off is never copied, **Preview** shows the changes first, and the previous file is kept as `.codeman-bak`. App Settings → Agents & CLIs; admin only in multi-user mode - **CJK / IME input** — full composition support for Chinese / Japanese / Korean, with Ctrl- and Alt-modified navigation keys passed through to the CLI +- **Config backups** — timestamped copies of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases, remote and Docker hosts, saved dashboards and, in multi-user mode, accounts), made at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder, how many to keep (1–500) and how long (days, 0 = no limit) are settings, with **Back up now** and a verified **Restore** in Settings → System. Retention only ever deletes folders Codeman made and never the newest. See the [wiki page](docs/wiki/Config-Backups.md) - **Plan usage in the header** — live Claude subscription usage (the 5-hour and weekly windows) from a statusline exporter Codeman hands to `claude` at spawn and never writes into your settings files, plus Codex limits from its own app-server; per device, on for desktops and off for phones - **Prompt-cache readout**: each Claude session shows whether its prompt cache is warm (`cache:until HH:MM`) or cold, in the footer (when you have no statusline of your own) and in Session Options → Respawn, with the cost of a re-cache and a note after a compaction. Display only; it survives a restart - **Session list, your way** — the header strip, a left sidebar with a filter box, or a vertical rail whose detailed rows carry created and state stamps and sort by activity; the phone home screen and the desktop home rail use the same order. On the rail, **Search sessions** narrows the list to the tabs that match, across every group, and a tab with an alert stays visible even when its name does not. Named **tab groups** collapse, reorder and take drags; a group's menu has **New session**, and a worker an agent spawns lands right after its parent (**Spawned tabs follow their parent**, off by default, keeps it moving with the parent). **Add to Focus** pins a shortcut to any tab in a section at the top of the rail, and **CLI Logos on Tabs** can be switched off per device diff --git a/docs/api-reference.md b/docs/api-reference.md index d638fd74..fa78ea91 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1156,9 +1156,9 @@ The result carries server **names** only, never `env` values, `headers` or file Timestamped snapshots of the files that hold what a user has customized (`src/config-backup.ts`). On by default; a snapshot is taken at startup and whenever the tracked files' content changes (checked every 5 minutes), so there is one entry per real change. All three routes are admin only in multi-user mode (`403`), because the files include credentials. Responses carry file names and sizes, never content. -Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `push-keys.json`, `users.json`, `.env`. Session state, logs, the hook secret and caches are not. +Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `remote-hosts.json`, `docker-hosts.json`, `docker-cases.json`, `webviews.json`, `push-keys.json`, `users.json`, `.env`. Session state (`state.json`, which also holds cron jobs and respawn presets), logs, the hook secret and caches are not. -Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `/backups/config`), `configBackupKeepCount` (1–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder of `0600` files plus a `manifest.json` with SHA-256 digests. +Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `/backups/config`), `configBackupKeepCount` (1–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, and a SHA-256 and size per file), all `0600`. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`. | Method | Path | Body | Notes | | ------ | ----------------------------------- | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | diff --git a/docs/wiki/Config-Backups.md b/docs/wiki/Config-Backups.md new file mode 100644 index 00000000..5b9f0ddc --- /dev/null +++ b/docs/wiki/Config-Backups.md @@ -0,0 +1,169 @@ +# Config Backups + +Codeman keeps timestamped copies of the files that hold what you have customized, so a bad +write, a hand edit or a botched restart can be undone. It is on by default, needs no setup, +and everything about it (where the copies go, how many are kept, how long) is a setting. + +## What is backed up + +A fixed list of files from the data directory (`~/.codeman/`, or the directory of the +[instance](Core-Concepts#instances) you run): + +| File | What it holds | +| ---- | ------------- | +| `settings.json` | Your preferences that sync across devices, including model and CLI settings. | +| `clis.json` | CLI enable/disable and custom CLI entries. | +| `custom-model-hosts.json` | Saved custom model endpoints. **Holds API keys.** | +| `webhook.json` | Webhook notification target. | +| `intents.json` | Read My Mind per-case profiles. | +| `linked-cases.json` | Registered cases. | +| `remote-hosts.json`, `docker-hosts.json`, `docker-cases.json` | Remote SSH and Docker overlay configuration. | +| `webviews.json` | Saved dashboard URLs. | +| `push-keys.json` | Web-push keys (regenerating them drops every device's subscription). | +| `users.json` | Multi-user accounts. **Holds password hashes.** | +| `.env` | Credentials the TUI client reads. **Holds a password.** | + +Only files that exist are included. This is an allowlist: nothing else in the data +directory is ever copied. + +**Not backed up:** `state.json` (sessions, and also cron jobs and respawn presets: it is rewritten +with every session event, so it would bury the real changes), logs, `mux-sessions.json`, the hook +secret, Docker seeds and caches. Back `state.json` up with your normal file backups if you rely on +cron jobs. + +## When a backup is made + +- **At startup**, so the state the server found is always on record. +- **Whenever the content changes.** The files are read and hashed every five minutes; a new + backup is written only if the set differs from the latest backup. An idle server writes + nothing, and the history is one entry per real change rather than one per timer tick. +- **On demand**, with **Back up now** (always writes one, even if nothing changed). +- **Before a restore**, as a `pre-restore` safety copy of the files about to be replaced. + +## Where they go and what they are called + +The default folder is `/backups/config` (`~/.codeman/backups/config`). You can point +it anywhere with an absolute path (a leading `~` is expanded). A path that is not absolute, or +that is the data directory itself, falls back to the default. + +Each backup is one folder named for the moment it was made, in the server's local time: + +``` +~/.codeman/backups/config/ + cfg-20261011-120305/ + manifest.json id, createdAt (epoch ms), reason, appVersion, SHA-256 + size per file + settings.json the original file names, unchanged + clis.json + custom-model-hosts.json + ... + cfg-20261011-123012/ + cfg-20261011-123012-2/ a second backup in the same second gets -2, -3, ... +``` + +| Part | Meaning | +| ---- | ------- | +| `cfg-YYYYMMDD-HHMMSS` | Date and time (server local time) the backup was made. A collision in the same second gets `-2`, `-3`, … | +| `manifest.json` | The record of the backup: `reason`, `appVersion`, and a SHA-256 and size for every file. | +| `reason` | `startup`, `auto` (a change was detected), `manual` (Back up now) or `pre-restore`. It is in the manifest and the Settings list, not in the folder name. | +| `.tmp-…` | A backup being written. It is renamed into place when complete and is never listed. | + +A folder counts as a backup only if its name matches that pattern **and** it has a valid +`manifest.json`. That is what makes it safe to point the folder at a directory that also holds +other things. + +## Retention + +Two limits, applied after every backup: + +| Setting | Range | Default | Meaning | +| ------- | ----- | ------- | ------- | +| **Keep this many backups** (`configBackupKeepCount`) | 1 – 500 | 20 | Keep the newest N; older ones are removed. | +| **Delete backups older than (days)** (`configBackupKeepDays`) | 0 – 3650 | 30 | Also remove anything older than this. `0` means no age limit. | + +Both apply together. Three guarantees: + +- **The newest backup is never removed**, whatever the numbers say, so there is always one to + go back to. +- **Only folders Codeman made are ever removed**, never anything else that happens to live in a + custom folder. +- Because backups are made on change, 20 backups reach back across 20 *changes*, however long + that took, not 20 timer ticks. + +## Settings → System → Config backups + +| Control | Setting | Default | +| ------- | ------- | ------- | +| Back up my configuration | `configBackupEnabled` | On (absent means on) | +| Backup folder | `configBackupDir` | empty = `/backups/config` | +| Keep this many backups | `configBackupKeepCount` | 20 | +| Delete backups older than (days) | `configBackupKeepDays` | 30 | +| **Back up now** | | | +| The list, with **Restore** per entry | | | + +The settings are saved server-side and read fresh, so a change applies on the next pass. The +list and **Back up now** use the **saved** folder and retention: apply your edits first. + +## Restoring + +From the list, press **Restore** on a backup and confirm. Codeman then: + +1. **Verifies** every file in the backup against the SHA-256 in its manifest. If any file is + missing or does not match, nothing is changed and you get an error. +2. **Takes a `pre-restore` backup** of the files about to be replaced, so a restore can itself + be undone. +3. **Replaces each file atomically** (written beside it, then renamed) with mode `0600`. + +Only the tracked names are ever written, whatever a manifest lists. + +Settings are re-read on the next request, so reload the page. `clis.json`, `users.json`, +`.env` and `push-keys.json` are read once per process: **restart Codeman** for those to take +effect (the response lists which). A restore does not touch running sessions. + +You can also restore by hand: the files in a backup folder are the originals, so copying one back +over the file in the data directory is equivalent (stop Codeman first if you are replacing +`clis.json` or `users.json`). + +## Security + +- Some tracked files carry credentials (`.env`, `users.json`, `custom-model-hosts.json`, + `push-keys.json`). Each backup folder is mode `0700` and its files `0600`, so the default + location exposes nothing the data directory does not already. +- **A custom folder is your decision.** Putting it on a shared drive, a synced folder or a + network share copies those credentials there, with whatever permissions that location has. + Choose a private location, or turn the feature off if you cannot. +- In [multi-user mode](Multi-User-Mode) all of it is admin only: the list, Back up now and + Restore. +- Listing returns file names and sizes, never contents. + +## HTTP API + +```bash +API=http://localhost:3000 +curl -s "$API/api/config-backups" | jq # settings, folder, list +curl -s -X POST "$API/api/config-backups" | jq # back up now +curl -s -X POST "$API/api/config-backups/cfg-20261011-120305/restore" | jq +``` + +See [HTTP API](HTTP-API) and the endpoint reference in `docs/api-reference.md`. + +## Troubleshooting + +- **The list is empty.** None of the tracked files existed yet, or backups are turned off. Press + **Back up now**. +- **"Backups are not available here".** You are not an admin in multi-user mode. +- **No new backup after I changed a setting.** Changes are picked up within five minutes; **Back up + now** takes one immediately. +- **A restore said a file does not match its checksum.** The backup was altered or damaged on + disk. Nothing was changed; restore an earlier one. +- **The backups folder is growing.** Lower **Keep this many backups**. Each backup is a few + kilobytes, but `custom-model-hosts.json` and `intents.json` can be larger. +- **I want my own off-machine copy.** Point the folder at a location your backup tool already + covers, mind the Security note above, and keep **Keep this many backups** high enough that + your tool sees each one. + +## Read next + +- [Settings Reference](Settings-Reference) - where the controls live. +- [Core Concepts](Core-Concepts) - what is on disk. +- [Security](Security) - what is protected and how. +- [Troubleshooting](Troubleshooting) - "my settings were reset". diff --git a/docs/wiki/Core-Concepts.md b/docs/wiki/Core-Concepts.md index 60da25e6..bc41e354 100644 --- a/docs/wiki/Core-Concepts.md +++ b/docs/wiki/Core-Concepts.md @@ -135,6 +135,7 @@ Everything Codeman knows lives under `~/.codeman/`: | `users.json` | Multi-user accounts, mode 0600. | | `push-*.json` | Web push keys and subscriptions. | | `certs/` | Self-signed TLS for `--https`. | +| `backups/config/` | Timestamped copies of the files above that you customized ([Config Backups](Config-Backups)). | None of it needs root, none of it leaves the machine, and deleting `~/.codeman/` resets Codeman to a fresh install without touching your code. diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index d14c60ea..50c108bb 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -113,6 +113,7 @@ codeman web # then open http://localhost:3000 | Page | What it answers | | --------------------------------------------- | -------------------------------------------------- | | [Running As A Service](Running-As-A-Service) | How do I keep it up across reboots, and update it? | +| [Config Backups](Config-Backups) | How do I get my settings back after they were changed or lost? | | [Troubleshooting](Troubleshooting) | Why is it doing that? | | [FAQ](FAQ) | The questions that keep coming up. | | [Contributing](Contributing) | How do I send a fix? | diff --git a/docs/wiki/Security.md b/docs/wiki/Security.md index 8dbaaa38..d7e169f1 100644 --- a/docs/wiki/Security.md +++ b/docs/wiki/Security.md @@ -107,6 +107,11 @@ browser and your server. Your agent CLI's traffic is its own, on your account. Two features send data outward, both off by default and both stated where they appear: voice dictation through your Claude login, and the Read My Mind prediction call. +[Config Backups](Config-Backups) copies some credential-bearing files (`.env`, `users.json`, +`custom-model-hosts.json`, `push-keys.json`) into a backup folder. The default folder is inside +the data directory, and every backup is mode `0700`/`0600`, so nothing new is exposed. Only a +folder you choose yourself can move them somewhere less private; admin only in multi-user mode. + ## Reporting a vulnerability **Never in a public issue.** diff --git a/docs/wiki/Settings-Reference.md b/docs/wiki/Settings-Reference.md index 2e7238ce..bf0a7800 100644 --- a/docs/wiki/Settings-Reference.md +++ b/docs/wiki/Settings-Reference.md @@ -190,6 +190,13 @@ Cloudflare tunnel controls including the tunnel URL. The **Diagnostics** group r tools with their versions and install hints (admin only in multi-user mode). In multi-user mode, the **Users** administration entry is injected here. +The **Config backups** group controls timestamped copies of your configuration files: an +on/off switch (on by default), the backup folder (default `~/.codeman/backups/config`), how +many backups to keep (1 to 500, default 20) and how many days to keep them (0 to 3650, default +30; 0 means no age limit), plus **Back up now** and a list with **Restore**. These are +server-side settings, read fresh. Details, naming and restore behaviour are on +[Config Backups](Config-Backups). + ## Session Options Per session, from the tab. @@ -243,3 +250,4 @@ Some things are configured before the server starts, not in the UI: - [The Dashboard](The-Dashboard) - what each control does once visible. - [Keeping Agents Running](Keeping-Agents-Running) - the Respawn panel in depth. - [Agent CLIs](Agent-CLIs) - model, effort, and permission modes. +- [Config Backups](Config-Backups) - copies of your settings, and how to restore them. diff --git a/docs/wiki/Troubleshooting.md b/docs/wiki/Troubleshooting.md index 114ca37d..aa3a83f4 100644 --- a/docs/wiki/Troubleshooting.md +++ b/docs/wiki/Troubleshooting.md @@ -131,6 +131,23 @@ That prompt comes from Claude Code, not Codeman. Codeman's default is to start w permission prompts skipped, which is what the security model is built around. If you would rather it prompted, change **App Settings → Agents & CLIs → Claude → Startup Mode**. +### My settings were reset, or a CLI I enabled is gone + +Something rewrote a file in `~/.codeman/` (a hand edit, a restart that loaded a bad file, a +test run pointed at the real data directory). If [Config Backups](Config-Backups) was on, +which is the default, you have copies: + +1. Open **Settings → System → Config backups** and pick the newest backup from before the + change. The list shows when each was made and what it holds. +2. **Restore** it, reload the page, and restart Codeman if the response says `clis.json`, + `users.json`, `.env` or `push-keys.json` were restored. + +Without the UI: the backups are plain folders in `~/.codeman/backups/config/` +(`cfg-YYYYMMDD-HHMMSS`); copy the file you need back over the one in `~/.codeman/`. + +Not covered: `state.json` (cron jobs and respawn presets live there). A backup only exists +from the moment the feature first ran, so a reset that happened before that cannot be undone. + ### Sessions vanished after a reboot Expected. tmux does not survive a reboot, so the sessions are gone. Conversations are not: diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 62e0e7a7..0b85b925 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -51,6 +51,7 @@ **Operating it** - [Running As A Service](Running-As-A-Service) +- [Config Backups](Config-Backups) - [Troubleshooting](Troubleshooting) - [FAQ](FAQ) - [Contributing](Contributing) diff --git a/src/config-backup.ts b/src/config-backup.ts index 84f60be1..9749c5d0 100644 --- a/src/config-backup.ts +++ b/src/config-backup.ts @@ -40,6 +40,9 @@ import { isAbsolute, join, resolve, sep } from 'node:path'; * The files (relative to the data dir) a snapshot holds: what a user configured, not what the server * regenerates (session state, logs, the hook secret, docker seeds, caches). * + * ⚠️ Not tracked: `state.json`. It also holds cron jobs and respawn presets, but it is rewritten with + * every session event, so tracking it would make a snapshot per event and bury the real changes. + * * ⚠️ `.env`, `users.json`, `custom-model-hosts.json` and `push-keys.json` carry credentials or keys. * They are already in the data dir with the same permissions; a snapshot folder is 0700 with 0600 * files, so the default location widens nothing. A custom location is the user's own choice. @@ -51,6 +54,10 @@ export const CONFIG_BACKUP_FILES = [ 'webhook.json', 'intents.json', 'linked-cases.json', + 'remote-hosts.json', + 'docker-hosts.json', + 'docker-cases.json', + 'webviews.json', 'push-keys.json', 'users.json', '.env',