diff --git a/.changeset/18dce807.md b/.changeset/18dce807.md new file mode 100644 index 00000000..9eb3a1b8 --- /dev/null +++ b/.changeset/18dce807.md @@ -0,0 +1,5 @@ +--- +"aicodeman": minor +--- + +Config backups: timestamped snapshots of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases and, in multi-user mode, accounts), taken at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder and the retention (keep N backups, drop older than N days) are configurable in Settings → System, with Back up now and Restore. Retention only deletes folders Codeman made, never the newest backup. A comparison view shows what changed between two backups, or between a backup and the running config, without ever showing a credential's value. diff --git a/CLAUDE.md b/CLAUDE.md index 9ba8719c..77b912b1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -157,6 +157,8 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Auto-resume on usage limit** (opt-in per session, top of the Respawn tab): when Claude halts on a subscription limit, `usage-limit-patterns.ts` (pure, unit-tested) parses the reset time and `SessionAutoOps` arms a timer for reset+2min, then sends Esc + `continue`. ⚠️ Respawn cycles are blocked while paused (`isLimitPaused` guard in `onIdleDetected`), which is what prevents `/clear` from wiping the paused conversation. Claude-mode only. → [architecture-invariants#auto-resume-on-usage-limit](docs/architecture-invariants.md#auto-resume-on-usage-limit) +**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, SYNCED; `docs/wiki/Config-Backups.md`): timestamped `cfg-*` snapshots of a FIXED allowlist of config files (`CONFIG_BACKUP_FILES`; `state.json` deliberately not) at startup and whenever their content changes, into `/backups/config` or `configBackupDir`. ⚠️ A field a tracked file rewrites on its own goes in `VOLATILE_KEYS`, or every tick makes a snapshot and the history shrinks to about 95 minutes. ⚠️ Retention deletes only this instance's folders that carry a valid manifest, never the newest, and ages a snapshot from its SUCCESSOR; a retention key ABSENT from `settings.json` means what the newest manifest recorded (`withRecordedRetention`), never the defaults. ⚠️ The instance is identified by the random id in `config-backup-id`, never the hostname (a Docker recreate changes it). ⚠️ A custom folder is probed BOUNDED (#516) before ANY read, the recorded-retention fallback included, and a pass reads the folder once (`runConfigBackupPass`). ⚠️ Restore verifies every SHA-256 and refuses symlinks and group/world bits before it writes anything, takes the undo copy first, and reports a partial failure as 500 with `restored`/`failed`/`safetySnapshot` in `data`; every path that wrote files runs `afterRestore()` (intents reload, CLI registry reload, admin audit), and Settings re-reads the form after a `settings.json` restore. ⚠️ Compare never returns a credential (`REDACT_ALL_FILES`, `SECRET_KEY_RE`). Admin only in multi-user mode. → [architecture-invariants#config-backups](docs/architecture-invariants.md#config-backups) + **Plan-usage chip** (`showPlanUsageLimits`, per-device: desktop default **ON**, handhelds OFF): resolve DISPLAY only through `planUsageChipEnabled()` (settings-ui.js). The same setting is the server-side COLLECTION switch, read fresh by `readPlanUsageTelemetryEnabled()` (hooks-config.ts) at every claude create/respawn. ⚠️ An ABSENT key reads as ON in the reader; `GET /api/settings` must never write. ⚠️ A save sends `showPlanUsageLimits` ONLY when it flips the chip on that device (`planUsageCollectionFlip()`), or a phone switches collection off for every desktop. Claude data comes from the statusLine exporter, injected as an EPHEMERAL `claude --settings` flag (`resolveStatusLineCliCommand`), never written to disk, WRAPPING a user's own statusLine, posting to `POST /api/status-telemetry`. Codex comes from a read-only `account/rateLimits/read` poll (main bucket only). Copilot comes from a read-only `GET https://api.github.com/copilot_internal/user` (`refreshCopilotPlanUsage()` in server.ts, at startup and every 10 minutes; `parseCopilotQuotaResponse()` keeps only the metered `premium_interactions` quota, as a `monthly` window with `used`/`limit`) sent with Copilot's OWN sign-in token (`resolveCopilotToken()`: `COPILOT_GITHUB_TOKEN`/`GH_TOKEN`/`GITHUB_TOKEN`, else the plain-text token in `/config.json`; a keychain sign-in shows no row). It is **on by default**, like the Codex poll: it rides `showPlanUsageLimits` (absent = ON), and Plan Usage Limits off stops the request. ⚠️ The gates live in `readCopilotPlanUsageIfEnabled()` (copilot-plan-usage.ts) in this order: the setting, then a token, then `isCopilotAvailable()`, whose resolver can run a login-shell probe that a host with no sign-in must never pay each tick. ⚠️ `redirect: 'error'`, the token is never logged, stored or broadcast, any failure is silently no row, and the row is admin-only on the wire like Codex's. The statusline's `prompt_cache` object (Claude Code v2.1.251+) is kept by `StatusTelemetrySchema` (every documented scalar field, null-tolerant; `parsePromptCache()` in usage-telemetry.ts normalizes only the stored subset) and shown in the footer as `cache:until HH:MM` / `cache:cold` (only when the user has no statusline of their own) and in Session Options → Respawn (`#promptCacheStatus`, fed by `Session.promptCache` in `toState()` plus the change-only `session:promptCache` SSE event, worded by the pure `CodemanPromptCache.format`): warm-until/cold, the re-cache stake, the hit rate, the last miss's causes, and a `Context compacted at HH:MM` note after a compaction or a tool-result clearing (stamped on the first warm report whose `recache_tokens_if_cold` is null after one that carried it, never from `expected_rebuilds`, which Claude Code bumps only at the next request; NEVER an alert). ⚠️ `Session.setPromptCache()` stamps `compactedAt` onto a NEW object (never mutating the parsed report) and the route broadcasts the STORED object; the last report is restored on restart (constructor `promptCache`, like `displayModel`) and the readout is time-aware (a passed `expiresAt` reads cold, not stale-warm). It is the source of truth for the cache TTL, never the plan tier. → [architecture-invariants#plan-usage-chip-statusline-telemetry](docs/architecture-invariants.md#plan-usage-chip-statusline-telemetry), `docs/usage-limits-display-plan.md` **Prompt-cache keep-warm** (`src/prompt-cache-keepwarm.ts`; per-session opt-in on the Respawn tab, app-wide default `promptCacheKeepWarm` + cap `promptCacheKeepWarmMaxHours` in App Settings → System → Automation, SYNCED, default OFF / 2 h; step 2 of Discussion #554): one keep-alive turn (`KEEP_WARM_NUDGE`, under 100 chars) five minutes before an idle claude session's cache would expire, so the next prompt reads the conversation from cache instead of re-writing it at 2x. Codeman owns the idle policy for these sessions: a launch with keep-warm on also carries `"idleCompaction": false`, folded LAST into the ONE `--settings` JSON (`session-cli-registry-bridge.ts`), decided once per create/respawn by `decideIdleCompactionOptOut()` (tmux-manager.ts: keep-warm on via `keepWarmOn()`, exporter `injected`, claude's `idleCompactionSetting` gate at 2.1.290, fail-closed like `--name`) and recorded as `MuxSession.idleCompactionOff`; a keep-warm-off launch stays byte-identical (`test/cli-registry-spawn-golden.test.ts`). Switched on after launch it is best-effort and the readout says idle compaction may still run until the next launch. ⚠️ The deadline is `Session.promptCache.expiresAt` from the statusline, never idle detection or the plan tier; a `5m` TTL is a hard stop, and so is a launch whose `statusLineExporter` is not `injected`. ⚠️ Every guard lives in the pure `decideKeepWarm()` (effective opt-in, where an agent-spawned session (`parentSessionId`) does not inherit the app default; exporter injected; warm; no `compactedAt` this gap; tokens at stake; live pane (pid AND no `paneExit`); no limit pause (auto-resume's, or a limit message on screen); no respawn/Ralph/task/waiter; inside the cap since `lastRealPromptAt`; idle; no dialog; EMPTY composer via `composerIsEmpty()`, continuation lines included, read from a STYLED capture so Claude Code's faint prompt suggestion is not taken for a draft; no recent keystroke); add a guard there, never in the service. ⚠️ The nudge IS a submit and moves `lastSubmitAt` (the response viewer's conversation correlation needs it); the window `beginKeepAliveTurn()` opens keeps it off `lastRealPromptAt` (the cap's own persisted clock) and opens its `stop`/`idle_prompt` hooks pre-acknowledged (`preAcknowledged` on `notePrompt`), so nothing alerts, and leaves an idle item already pending exactly as found (only a human spends an idle alert; `session:working`/`session:idle` carry `keepAlive: true` so the page's stuck timer ignores the turn); the keep-alive's own `idle_prompt` ends that window (`endKeepAliveTurn()`; `KEEP_WARM_TURN_WINDOW_MS` is only a ceiling, since a slow turn's idle prompt lands a minute after it ends), so does a nudge whose write failed (retried a minute later, never past the last-chance line), and so does any other submit: a `fromUser` write, a programmatic write of anything but the nudge, or a `prompt_submitted` hook whose prompt is not the nudge. Read My Mind drops its exact text (`isCapturablePrompt`). → [architecture-invariants#prompt-cache-keep-warm](docs/architecture-invariants.md#prompt-cache-keep-warm) @@ -173,7 +175,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Docker Compose deployment** (`docker/`): Codeman runs in a container and spawns Docker cases as **SIBLING** containers via the host socket, never nested. `resolveDockerDaemonMountSource()` maps HOME bind sources into the daemon's namespace (`CODEMAN_DOCKER_HOST_HOME`); `CODEMAN_CASES_PATH` makes workspaces resolve to the same absolute path on both sides. ⚠️ `CODEMAN_CASES_PATH` must move every consumer: resolve it only via `config/cases-dir.ts`. ⚠️ `.dockerignore` matches whole paths: keep `**/.env` or `docker/.env` secrets ship in the image. ⚠️ Long-form binds create missing sources ROOT-OWNED: `Start-Codeman.sh` pre-creates them, and `docker/entrypoint.sh` (root, `cap_add: [CHOWN, DAC_OVERRIDE, KILL, SETGID, SETUID]` against `cap_drop: ALL`, `KILL` for tini; pinned by the test) fixes ownership then drops to `PUID:PGID` via `setpriv`, never re-owning foreign dirs. ⚠️ Append `/opt/codeman-cli` and `~/.local/bin` to `PATH`, never prepend. ⚠️ `server.Dockerfile`, the compose file and `.env.example` feed the self-updater's environment gate (`docs/docker-self-update.md`). → [architecture-invariants#docker-compose-deployment](docs/architecture-invariants.md#docker-compose-deployment), `docs/docker-compose.md` -**CLI registry** (`src/config/cli-registry/`): every run mode is a `CliEntry` (discovery, launch argv template, env handling, `capabilities`, and the `overlays` behind remote/docker pane commands). **No code outside `stock.ts` may branch on a CLI id**: use a capability field or a NAMED PROFILE (`profiles.ts`); `test/cli-registry-no-id-branching.test.ts` and `test/frontend-cli-no-id-branching.test.ts` enforce it. ⚠️ Config holds typed argv tokens, never shell text; literals are validated at LOAD time and a bad one rejects the whole entry. ⚠️ Keep `external`, `hooks` and `altScreen` independent; never derive one from another. ⚠️ Config regexes (`discovery.version.regex`, `capabilities.workDetect.workingLine`, `capabilities.workDetect.watchingLine`, `capabilities.workDetect.awaitingLine`, `capabilities.workDetect.blockedLine`, `capabilities.modelDetect.screenLine`) must compile through `compileVersionRegex()`. ⚠️ `privilegedParams[].param` names a LAUNCH PARAM, not the legacy `Config` field (bridged only by `launch.legacyConfigAliases`); a wrong name silently clamps nothing. ⚠️ Resolve the registry AT CALL TIME, never in a module-level const. ⚠️ Remote claude/omp arms of `buildRemoteLaunchCommand` are not covered by the pane-command golden. `~/.codeman/clis.json` overrides entries. It is WRITTEN only by the opt-in CLI management routes (`cliManagementEnabled`, default OFF; `/api/clis`, `cli-registry-routes.ts`), and only through `mutateRegistryFile()` in `registry-writer.ts`, which serializes mutations and refuses (409) a file that does not parse or has group/world permission bits rather than overwriting it. Importing the registry still writes nothing. → [architecture-invariants#cli-registry](docs/architecture-invariants.md#cli-registry), `docs/cli-registry.md` +**CLI registry** (`src/config/cli-registry/`): every run mode is a `CliEntry` (discovery, launch argv template, env handling, `capabilities`, and the `overlays` behind remote/docker pane commands). **No code outside `stock.ts` may branch on a CLI id**: use a capability field or a NAMED PROFILE (`profiles.ts`); `test/cli-registry-no-id-branching.test.ts` and `test/frontend-cli-no-id-branching.test.ts` enforce it. ⚠️ Config holds typed argv tokens, never shell text; literals are validated at LOAD time and a bad one rejects the whole entry. ⚠️ Keep `external`, `hooks` and `altScreen` independent; never derive one from another. ⚠️ Config regexes (`discovery.version.regex`, `capabilities.workDetect.workingLine`, `capabilities.workDetect.watchingLine`, `capabilities.workDetect.awaitingLine`, `capabilities.workDetect.blockedLine`, `capabilities.modelDetect.screenLine`) must compile through `compileVersionRegex()`. ⚠️ `privilegedParams[].param` names a LAUNCH PARAM, not the legacy `Config` field (bridged only by `launch.legacyConfigAliases`); a wrong name silently clamps nothing. ⚠️ Resolve the registry AT CALL TIME, never in a module-level const. ⚠️ Remote claude/omp arms of `buildRemoteLaunchCommand` are not covered by the pane-command golden. `~/.codeman/clis.json` overrides entries. It is WRITTEN only by the opt-in CLI management routes (`cliManagementEnabled`, default OFF; `/api/clis`, `cli-registry-routes.ts`), and only through `mutateRegistryFile()` in `registry-writer.ts` (the one exception: a config-backup restore of `clis.json`, which then calls `reloadCliRegistry()`), which serializes mutations and refuses (409) a file that does not parse or has group/world permission bits rather than overwriting it. Importing the registry still writes nothing. → [architecture-invariants#cli-registry](docs/architecture-invariants.md#cli-registry), `docs/cli-registry.md` **External CLI modes (OpenCode, Codex, Gemini, Antigravity, Pi, Grok, DeepSeek, OMP, Copilot)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior off (Ralph tracker, BashToolParser, token parsing, ❯ readiness; readiness is output stabilization); work detection is per-CLI `capabilities.workDetect` data, not this gate. All nine **require tmux, no direct PTY fallback** (secrets go via socket-scoped `tmux setenv`, never the command line). ⚠️ `run*()` in `session-ui.js` MUST unwrap the `{success,data}` envelope. ⚠️ **Codex uses predictive write-through echo, never the buffer overlay**: `_predictHookOnData` must never `return` (wire path stays byte-identical), and flushed text and a bracketed paste must go out as separate delayed writes. ⚠️ **Pi**: no bypass flag, never invent one; `approveProjectTrust` executes repo code, so it is in the clamp's **materialize** branch; never wire `--api-key`. ⚠️ **Grok**: `alwaysApprove` is stripped for non-granted owners (only-if-sent). ⚠️ **DeepSeek**: the agent is a PROFILE (Run gates on `isDeepSeekRunnable()`); the permission switch is the `DSH_PERMISSION_MODE` env var, so `clampEnvOverridesForOwner()` must DROP `DSH_PERMISSION_MODE`, `DSH_HOME` and `DEEPSEEK_BASE_URL` for non-granted owners; `hooksAvailableForMode()` is per-SESSION for it (pass `sessionHookOptions(session)`) and is never a stand-in for `mode === 'claude'`; answers come from `deepseek-transcript.ts`, paired by header `cwd` + boot window, never newest-mtime. ⚠️ **OMP**: `OMP_AUTH_BROKER_URL`/`_TOKEN` are clamped the same way. ⚠️ **Copilot**: `allowAll` (`--yolo`) is a `privilegedParam` and `COPILOT_HOME`/`COPILOT_ALLOW_ALL`/`COPILOT_PROVIDER_*`/`COPILOT_MODEL` are privileged env keys, and so is every other `COPILOT_*` variable the CLI reads that switches approvals, runs code with no approval row or sends its sign-in token elsewhere (the 1.0.98 list is in `stock.ts`; the prefix admits whatever a later release adds, so re-read the bundle whenever the measured version moves); the resume id must start with a letter or digit (`path-segment`), since `--resume`'s value is optional and `--resume --yolo` would otherwise make `--yolo` its own flag; `--name` is dropped next to `--resume`/`--continue` (Copilot refuses the pair); its mouse DECSETs are stripped (`strip-mux-and-mouse`) so a drag selects and the wheel is forwarded by `_shouldForwardWheelToApp` while `cliMouseTracking` is on; `composerReadyMark` is the footer hint `/ commands · ? help`, NOT `❯` (the folder-trust dialog draws `❯ 1. Yes`); `last-response` reads the pane's visible screen (`transcriptNeedsScreenCapture`) through a per-CLI `PANE_DIALECTS` row, because its stream is cursor moves, not lines (a prompt row scrolled off by a tall answer reads as the answer's tail unless a banner/trust dialog is on screen, and typed composer text is not a prompt); Docker seeds no Copilot credential (`config.json` holds a plain-text token without a keyring). → [architecture-invariants#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp](docs/architecture-invariants.md#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp) @@ -305,7 +307,7 @@ One module per domain in `src/web/routes/` (plus a barrel; `ls src/web/routes/` ## State Files -All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) and `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back; both installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). +All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) and `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back; both installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). **Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored. diff --git a/README.md b/README.md index cc2f4096..b3fe7e7e 100644 --- a/README.md +++ b/README.md @@ -479,6 +479,7 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt - **OSC 52 clipboard**: a program in a session that sets the clipboard with OSC 52 (Claude Code's `/copy`, a tmux yank, vim) puts the text on your browser's clipboard, in the main terminal and in every tile. Clipboard read requests are refused, and replaying a buffer never copies - **MCP server sync** _(opt-in)_: copies the user-level MCP servers each installed CLI has (Claude, Codex, Gemini, OpenCode, Antigravity, and GitHub Copilot CLI) into the others' own config files. Additive only, a server you switched off is never copied, **Preview** shows the changes first, and the previous file is kept as `.codeman-bak`. App Settings → Agents & CLIs; admin only in multi-user mode - **CJK / IME input** — full composition support for Chinese / Japanese / Korean, with Ctrl- and Alt-modified navigation keys passed through to the CLI +- **Config backups** — timestamped copies of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases, remote and Docker hosts, saved dashboards and, in multi-user mode, accounts), made at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder, how many to keep (2–500) and how long (days, 0 = no limit) are settings, with **Back up now** and a verified **Restore** in Settings → System. Retention only ever deletes folders Codeman made and never the newest. See the [wiki page](docs/wiki/Config-Backups.md) - **Plan usage in the header** — live Claude subscription usage (the 5-hour and weekly windows) from a statusline exporter Codeman hands to `claude` at spawn and never writes into your settings files, plus Codex limits from its own app-server and the GitHub Copilot monthly premium-request quota (the server asks `api.github.com` for it every 10 minutes with Copilot's own sign-in token: on by default while Copilot CLI is installed and signed in, and turning off Plan Usage Limits stops it); per device, on for desktops and off for phones - **Prompt-cache readout**: each Claude session shows whether its prompt cache is warm (`cache:until HH:MM`) or cold, in the footer (when you have no statusline of your own) and in Session Options → Respawn, with the cost of a re-cache and a note after a compaction. Display only; it survives a restart - **Session list, your way** — the header strip, a left sidebar with a filter box, or a vertical rail whose detailed rows carry created and state stamps and sort by activity; the phone home screen and the desktop home rail use the same order. On the rail, **Search sessions** narrows the list to the tabs that match, across every group, and a tab with an alert stays visible even when its name does not. Named **tab groups** collapse, reorder and take drags; a group's menu has **New session**, and a worker an agent spawns lands right after its parent (**Spawned tabs follow their parent**, off by default, keeps it moving with the parent). **Add to Focus** pins a shortcut to any tab in a section at the top of the rail, and **CLI Logos on Tabs** can be switched off per device diff --git a/docs/api-reference.md b/docs/api-reference.md index d14cc05c..b3de823c 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1171,6 +1171,21 @@ Claude Code reports the main conversation's prompt-cache state on its statusline `status` is `{ enabled, idleCompactionOff, state, nextCheckAt?, reason?, nudges, lastNudgeAt?, budgetEndsAt? }` (`idleCompactionOff`: whether the session's current launch carries that opt-out; `budgetEndsAt` only while `enabled`) with `state` one of `off`, `armed` (waiting for the due time), `waiting` (due, deferred by a guard named in `reason`), `nudged` (sent, waiting for the next cache report) or `stopped` (`reason` says why). Two SSE events carry changes: `session:promptCache` (`{ sessionId, ...promptCache }`) and `session:keepWarm` (`{ sessionId, ...status }`), both change-only. The keep-alive turn's `hook:stop` and `hook:idle_prompt` broadcasts carry `acknowledgedReason: "cache keep-alive"`, no push is sent for them and the run summary does not record them; an idle approval already pending stays as it was (only a person spends its alert). Its `session:working` and `session:idle` broadcasts carry `keepAlive: true`. +## Config backups + +Timestamped snapshots of the files that hold what a user has customized (`src/config-backup.ts`). On by default; a snapshot is taken at startup and whenever the tracked files' content changes (checked every 5 minutes), so there is one entry per real change. All four routes are admin only in multi-user mode (`403`), because the files include credentials. Responses carry file names and sizes; only the diff returns values, and never a credential's. + +Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `remote-hosts.json`, `remote-cases.json`, `docker-hosts.json`, `docker-cases.json`, `webviews.json`, `push-keys.json`, `users.json`, `.env`. Session state (`state.json`, which also holds cron jobs and respawn presets), logs, the hook secret and caches are not. + +Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `/backups/config`), `configBackupKeepCount` (2–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). A key absent from `settings.json` means the retention the newest backup in the folder recorded, and the default only when none did. Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, the change-check `hash`, an `origin` id, the `keepCount` and `keepDays` in force, and a SHA-256 and size per file), all `0600`. A snapshot is never dated earlier than the instance's newest one, so a clock that read ahead once does not make every later pass write a copy. All four routes answer `503 OPERATION_FAILED` when a custom folder does not respond, and diff and restore answer `409` for a symlinked or group/world-accessible backup as well as a checksum mismatch. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`. + +| Method | Path | Body | Notes | +| ------ | ------------------------------------ | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `GET` | `/api/config-backups` | none | `settings` in effect, `dir`, `defaultDir`, `tracked[]` and `backups[]` (`id`, `createdAt`, `reason`, `appVersion`, `files[]`, `bytes`), newest first. | +| `POST` | `/api/config-backups` | none | Snapshot now, even if nothing changed (`reason: manual`), then apply retention. `status` is `created` or `empty` (nothing tracked exists yet); `removed[]` lists the ids retention deleted. `503` when a custom folder does not answer. | +| `GET` | `/api/config-backups/diff?from=&to=` | none | What differs between two states: `from`/`to` are a backup id or `current` (the running config; the default for `to`). `files[]` (`name`, `status` `added`/`removed`/`changed`/`unchanged`, `changes[]` of `path`, `kind`, `before`/`after`, `truncated`, `detail`) and `changed`. Credential values are never returned (`redacted: true`). `400` without `from`; `404` unknown id; `409` if a file fails its checksum. | +| `POST` | `/api/config-backups/:id/restore` | none | Put a snapshot's files back, atomically, after a `pre-restore` snapshot of the current ones (none when nothing changed since the newest backup). `404` unknown id; `409 CONFLICT` if a file fails its checksum (nothing changed). Returns `restored[]`, `untouched[]` (tracked files present now that the backup did not hold; left as they are), `safetySnapshot` (the `pre-restore` id, or the newest backup's when nothing had changed since it) and `restartRequired[]` (files the running server only reads at start: `.env`, `push-keys.json`, `settings.json`). A file that cannot be written (a full disk, a directory in its place) does not stop the others: the answer is then `500 OPERATION_FAILED` with the same fields in `data` plus `failed[]` (`name`, `error`); the files listed in `restored[]` are the backup's and are reloaded like a full restore. | + ## Webhook notifications Posts the Web Push events to ntfy, Slack, Discord or a generic JSON URL (Settings → Notifications). Off by default. The webhook URL is a bearer secret (anyone holding a Slack/Discord URL can post as it), so it lives in `~/.codeman/webhook.json` (0600), is **never returned**, and is kept out of `settings.json`. All three routes answer `403` for a non-admin in multi-user mode. diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index b8ce7fa1..fea87671 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -117,7 +117,7 @@ Further detail: ⚠️ An ADOPTED container may back SEVERAL cases at different Spawn commands are pinned as literal strings in `test/cli-registry-spawn-golden.test.ts`, remote/docker pane commands in `test/location-overlay-commands.test.ts`. ⚠️ That second golden no longer covers **remote claude or remote omp**: both now have their own arm in `buildRemoteLaunchCommand` (a `--session-id || --resume` pair, and `--continue`, so a respawn continues the same conversation) and never reach `defaultRemoteCommandForMode`, which is what that test asserts. Their real pins are `toContain` substrings in `test/tmux-manager.test.ts` and `test/remote-shared-sessions.test.ts`; changing either arm will NOT fail the golden. -⚠️ Anything reading the registry resolves it AT CALL TIME (`sessionModeSchema()`, `allowedEnvPrefixes()`, `dependencyRegistry()`, the resolvers' `searchDirs` thunks) — a module-level const freezes at first import, so a CLI enabled while the server ran moved the run menu but not that surface. `~/.codeman/clis.json` overrides any entry. Importing the registry still has no filesystem side effects: the only writer is `registry-writer.ts`, imported only by `cli-registry-routes.ts` (the opt-in CLI management routes, `cliManagementEnabled`, default OFF). ⚠️ Every write goes through `mutateRegistryFile()`: mutations run one at a time on a single promise chain, each with a unique temp file (three parallel toggles once lost two and 500'd on a shared temp name), and a file the READER would ignore (group/world permission bits) or quarantine (unparseable) is REFUSED with a 409 rather than rewritten, so one Settings click cannot replace a hand-edit or turn a refused file into trusted 0600 config. Only a missing file starts fresh. After a write the registry is reloaded, so changes apply without a restart. User guide: `docs/cli-registry.md`. +⚠️ Anything reading the registry resolves it AT CALL TIME (`sessionModeSchema()`, `allowedEnvPrefixes()`, `dependencyRegistry()`, the resolvers' `searchDirs` thunks) — a module-level const freezes at first import, so a CLI enabled while the server ran moved the run menu but not that surface. `~/.codeman/clis.json` overrides any entry. Importing the registry still has no filesystem side effects: the only writer is `registry-writer.ts`, imported only by `cli-registry-routes.ts` (the opt-in CLI management routes, `cliManagementEnabled`, default OFF). ⚠️ Every write goes through `mutateRegistryFile()` (the one exception is a config-backup restore of `clis.json`, which writes it atomically and calls `reloadCliRegistry()`): mutations run one at a time on a single promise chain, each with a unique temp file (three parallel toggles once lost two and 500'd on a shared temp name), and a file the READER would ignore (group/world permission bits) or quarantine (unparseable) is REFUSED with a 409 rather than rewritten, so one Settings click cannot replace a hand-edit or turn a refused file into trusted 0600 config. Only a missing file starts fresh. After a write the registry is reloaded, so changes apply without a restart. User guide: `docs/cli-registry.md`. ## Session data and lifecycle @@ -721,6 +721,36 @@ Matching semantics: a query containing `/` matches the relative path, otherwise ⚠️ Closing the preview must **pause and unload** the media (`_stopFilePreviewMedia` in panels-ui.js): dropping the overlay's `visible` class is `display:none` and nothing else, and a DETACHED `HTMLMediaElement` keeps playing, which is how the X button used to leave a video audible with no player to pause. +### Config backups + +**Config backups** (`src/config-backup.ts`, pure over the directories it is handed; `routes/config-backup-routes.ts`; the Config backups group in App Settings → System, `settings-ui.js`; contributed in #661, user guide `docs/wiki/Config-Backups.md`, endpoints in `docs/api-reference.md`): timestamped snapshots of the files that hold what a user customized, so a bad write can be undone. Settings (SYNCED): `configBackupEnabled` (absent = ON, since it protects against an event nobody schedules), `configBackupDir` (absolute, `~` expanded; empty, relative or the data dir itself = `/backups/config`), `configBackupKeepCount` (2 to 500, default 20), `configBackupKeepDays` (0 to 3650, default 30, `0` = no age limit). A snapshot is a `cfg-YYYYMMDD-HHMMSS` folder (server local time, `-2`, `-3`… on a same-second collision; mode 0700) holding the tracked files under their own names plus a `manifest.json` (all 0600): `id`, `createdAt`, `reason` (`startup`/`auto`/`manual`/`pre-restore`), `appVersion`, the change-check `hash`, the `origin`, the `keepCount`/`keepDays` in force, and a SHA-256 and size per file. It is built in a `.tmp-cfg-*` folder and renamed into place; a pass sweeps temp folders older than an hour. Passes run at startup and every 5 minutes (`CONFIG_BACKUP_INTERVAL_MS` in `server.ts`, not in test mode), and on **Back up now** (forced). + +⚠️ **The tracked list is a fixed allowlist** (`CONFIG_BACKUP_FILES`), never "everything in the data dir", and restore writes only those names whatever a manifest lists. `state.json` is deliberately NOT tracked: it is rewritten on every session event, so it would make a snapshot per event and bury the real changes. It also holds cron jobs and respawn presets, which is the documented gap. + +⚠️ **One snapshot per real change.** The change check compares a digest of the file set with the newest manifest's `hash`, looking through `VOLATILE_KEYS`, fields a tracked file rewrites on its own: `custom-model-hosts.json` `lastDiscoveredAt` (the discovery poll), `users.json` `lastLoginAt` (every Basic login), `settings.json` `lastUsedCase` (a session started in another case), `intents.json` `recentPrompts`/`updatedAt` (every captured prompt), `docker-cases.json` `lastClaudeSessionId` (a `/clear` in a Docker case) and `webviews.json` `lastOpenedAt` (opening a web tab). Each was found the same way: one tick-driven rewrite made a snapshot per tick, and the 20 kept copies covered about 95 minutes. A new self-rewritten field in a tracked file needs an entry there and a row in the `it.each` table of `test/config-backup.test.ts`. The snapshot still stores the bytes and restore verifies the raw digest; only the comparison looks through these keys (so Compare calls `intents.json` unchanged when only prompts differ, yet a restore drops the prompts captured since). + +⚠️ **Retention deletes only what this module made, and never the newest snapshot.** A folder counts only with the name pattern AND a parseable manifest of this origin, so a custom folder that also holds other things is safe to point at. A snapshot is aged from its SUCCESSOR's `createdAt` (its state was live until the next one was made): aging it by its own date deleted the last good copy on the first change after a quiet month, which is exactly the bad write the feature exists for. The keep count floor is 2 (`CONFIG_BACKUP_MIN_KEEP_COUNT`), since 1 deletes the state before every change. A new snapshot is never dated earlier than the newest one (`createdAt + 1`): one written while the clock read ahead (dual boot, a VM resumed with a stale clock) stayed newest after the correction, differed from every later pass, and made each one write a copy until real time caught up. The flip side, a clock once set to a future year, is a Troubleshooting entry (delete the future-dated folder). + +⚠️ **The origin is a random id in `/config-backup-id`, never the hostname.** Instances that share a folder (two machines on one synced folder, prod and a beta) must keep to their own snapshots, so the change check, list, prune, Compare and restore are all scoped to it; without that, each instance snapshotted on every tick and Restore offered the other machine's `users.json` and `.env`. The hostname was tried first and was wrong: a Docker Compose container gets a new one on every recreate, which unlisted every earlier snapshot, and since prune is origin-scoped they were never deleted either. The id is not the hook secret and is never tracked or restored. Listing never creates it (`readConfigBackupOrigin()`; only a pass or a restore does). A fresh or reinstalled data dir is a new owner; copying an `origin` from an old manifest into the file makes those snapshots visible again. + +⚠️ **Retention survives a bad `settings.json`.** Each manifest records the retention in force. `runConfigBackup()` reads `settings.json` itself, because `readJsonConfig` answers `{}` for any failure, including a read that lands inside the non-atomic write of `PUT /api/settings`: a file that does not parse still gets its snapshot but prunes nothing, and a retention key ABSENT from the file (a reset) means what the newest manifest that recorded one holds (`withRecordedRetention()`, per key, in the folder the file names, so re-entering a custom folder after a reset finds that folder's own history). `GET /api/config-backups` reports the same values the pass uses, the form fills from them, and Save sends a retention key only when it was saved before or edited there (`_configBackupRetentionPayload()`), so merely displaying the recorded retention never writes it back. Restoring `settings.json` keeps the four `configBackup*` keys at their live values (an old `configBackupDir` would point the list at another folder and hide the undo copy just made), except a retention key the live file lacks, which keeps the backup's. While `settings.json` does not parse a pass reads it as `{}`, so its snapshot goes to the default folder; nothing is widened. + +⚠️ **A custom folder is probed BOUNDED before anything reads it** (#516: it may be a network mount that stopped answering, and the GET runs every time App Settings opens). `probeBackupDir()` runs `probePathKind(dir, { pastCap: true })` before every pass and in every route, and `readEffectiveBackupSettings()` probes BEFORE the recorded-retention fallback lists the folder (that fallback once did a `readdirSync` first whenever only `configBackupDir` was set). `unknown` answers 503 `OPERATION_FAILED` through `describeUnknownPath()` and skips the pass; a missing custom folder is created only when its parent is present. The default folder needs no probe. `test/routes/config-backup-unreachable.test.ts` pins all four routes with a recording `node:fs` mock. + +⚠️ **One walk of the folder per pass.** `runConfigBackupPass()` reads the folder once and hands the names to the temp sweep and the manifests to the retention fallback, the change check and the prune (with the snapshot it just made prepended: it is always the newest, by the date floor above). Each of the three used to list the folder itself, so every manifest was read three times per pass, every five minutes, on what may be a network folder; restore lists once too. `test/routes/config-backup-one-walk.test.ts` counts the reads. + +⚠️ **Restore verifies everything before it writes anything.** Every file's SHA-256 against the manifest, and the id must match the name pattern (no traversal); a mismatch is a 409 with nothing changed. It also refuses a snapshot folder or file that is a symlink or has a group/world bit: this module only ever writes 0700/0600 real files, so such a thing was planted or loosened later, and rewriting it as 0600 would turn it into trusted config (`clis.json` selects the binaries Codeman spawns; `isUnsafePermissions` refuses the same bits there). The mode test is skipped where the filesystem keeps no modes (vfat/exFAT, CIFS, WSL `/mnt/d` without `metadata` report 0755/0777 for everything, and Restore would refuse every backup there): `folderKeepsModes()` decides by creating a 0600 probe file with a random name and `wx`, because the earlier fixed `.mode-probe-` name let a planted symlink be truncated and chmodded by one Compare click. Compare applies the same checks to the snapshots it reads. + +⚠️ **The undo copy comes first, and a partial restore is reported, never hidden.** A `pre-restore` snapshot of the live files precedes the writes; it is not forced, so when nothing changed since the newest backup that backup is the undo copy and is the one reported (a forced copy spent a retention slot on a duplicate). It records the retention in force, so it cannot hide that retention from the next pass. Each file is replaced atomically (temp + rename, 0600). A file that cannot be written (a full disk, a directory in its place) does not stop the others: `ConfigBackupPartialRestoreError` carries `restored`, `failed`, `untouched` and `safetySnapshot`, and the route answers 500 with the error envelope plus those fields (and `restartRequired`) in `data`, the shape the tab-layout 409 uses. The bare 500 it replaced named neither the files already put back nor the undo copy. + +⚠️ **What a restore wrote must reach the running server**, on the full and the partial path alike (`afterRestore()` in the route): `intentStore.reload()` (the store caches `intents.json` and rewrote it whole on the next captured prompt, reverting the restore), `reloadCliRegistry()` (a restore of `clis.json` is the one write that bypasses `mutateRegistryFile()`, see CLI registry), and in multi-user mode a `config_backup_restore` admin-audit entry (`users.json` holds accounts, password hashes, roles and bypass grants). `users.json` itself is re-read within a second by the user store. `.env`, `push-keys.json` and `settings.json` (whose watcher toggles only `PUT /api/settings` runs) apply after a restart: `RESTART_REQUIRED`, returned as `restartRequired`. Settings re-reads `/api/settings` into the form after any restore that wrote `settings.json`, partial ones included, because the form still holds the old values and the next Save would PUT them over the restored file. + +⚠️ **Compare never returns a credential.** `REDACT_ALL_FILES` (`.env`, `users.json`, `push-keys.json`, `webhook.json` whose URL is a bearer secret, `intents.json` whose prompts carry secrets) report changed key names only; anywhere else a key matching `SECRET_KEY_RE` (token, secret, password, credential, auth, `*key`) is reported as changed without its value, and URL values lose userinfo, query and fragment. Listing returns file names and sizes only. + +⚠️ **Admin only in multi-user mode**: all four routes answer 403 for a non-admin and the Settings group is hidden for them (`_applyDoctorAdminGate()`), because the files include credentials. + +Tests: `test/config-backup.test.ts`, `test/config-backup-origin.test.ts`, `test/config-backup-restore-ui.test.ts` (vm harness over the real `settings-ui.js`), `test/routes/config-backup-routes.test.ts`, `test/routes/config-backup-unreachable.test.ts`, `test/routes/config-backup-one-walk.test.ts`. + ## Frontend ### Command palette and shortcut registry diff --git a/docs/wiki/Config-Backups.md b/docs/wiki/Config-Backups.md new file mode 100644 index 00000000..8ca07216 --- /dev/null +++ b/docs/wiki/Config-Backups.md @@ -0,0 +1,256 @@ +# Config Backups + +Codeman keeps timestamped copies of the files that hold what you have customized, so a bad +write, a hand edit or a botched restart can be undone. It is on by default, needs no setup, +and everything about it (where the copies go, how many are kept, how long) is a setting. + +## What is backed up + +A fixed list of files from the data directory (`~/.codeman/`, or the directory of the +[instance](Core-Concepts#instances) you run): + +| File | What it holds | +| ---- | ------------- | +| `settings.json` | Your preferences that sync across devices, including model and CLI settings. | +| `clis.json` | CLI enable/disable and custom CLI entries. | +| `custom-model-hosts.json` | Saved custom model endpoints. **Holds API keys.** | +| `webhook.json` | Webhook notification target. | +| `intents.json` | Read My Mind per-case profiles. | +| `linked-cases.json` | Registered cases. | +| `remote-hosts.json`, `remote-cases.json`, `docker-hosts.json`, `docker-cases.json` | Remote SSH and Docker overlay configuration. | +| `webviews.json` | Saved dashboard URLs. | +| `push-keys.json` | Web-push keys (regenerating them drops every device's subscription). | +| `users.json` | Multi-user accounts. **Holds password hashes.** | +| `.env` | Credentials the TUI client reads. **Holds a password.** | + +Only files that exist are included. This is an allowlist: nothing else in the data +directory is ever copied. + +**Not backed up:** `state.json` (sessions, and also cron jobs and respawn presets: it is rewritten +with every session event, so it would bury the real changes), logs, `mux-sessions.json`, the hook +secret, Docker seeds and caches. Back `state.json` up with your normal file backups if you rely on +cron jobs. + +## When a backup is made + +- **At startup**, so the state the server found is always on record. +- **Whenever the content changes.** The files are read and hashed every five minutes; a new + backup is written only if the set differs from the latest backup. An idle server writes + nothing, and the history is one entry per real change rather than one per timer tick. + A field a file rewrites on its own is not a change, and the comparison looks through it (the + snapshot still stores the file exactly as written, and a restore puts those fields back as they + were): `custom-model-hosts.json` `lastDiscoveredAt` (the model-discovery poll, every five + minutes), `users.json` `lastLoginAt`, `settings.json` `lastUsedCase`, `intents.json` + `recentPrompts` and `updatedAt` (every captured prompt) and `docker-cases.json` + `lastClaudeSessionId` (a Docker case's conversation id after a `/clear`) and `webviews.json` + `lastOpenedAt` (opening a web tab). Without that, every + tick looked like an edit and 20 copies covered only about 100 minutes. So Compare shows + `intents.json` as unchanged when only prompts differ, yet a restore drops the prompts captured + since. +- **On demand**, with **Back up now** (always writes one, even if nothing changed). +- **Before a restore**, as a `pre-restore` safety copy of the files about to be replaced (unless + nothing changed since the newest backup). + +## Where they go and what they are called + +The default folder is `/backups/config` (`~/.codeman/backups/config`). You can point +it anywhere with an absolute path (a leading `~` is expanded). A path that is not absolute, or +that is the data directory itself, falls back to the default. + +Each backup is one folder named for the moment it was made, in the server's local time: + +``` +~/.codeman/backups/config/ + cfg-20261011-120305/ + manifest.json id, createdAt (epoch ms), reason, appVersion, hash, origin, + keepCount and keepDays in force, SHA-256 + size per file + settings.json the original file names, unchanged + clis.json + custom-model-hosts.json + ... + cfg-20261011-123012/ + cfg-20261011-123012-2/ a second backup in the same second gets -2, -3, ... +``` + +| Part | Meaning | +| ---- | ------- | +| `cfg-YYYYMMDD-HHMMSS` | Date and time (server local time) the backup was made. A collision in the same second gets `-2`, `-3`, … | +| `manifest.json` | The record of the backup: `reason`, `appVersion`, the change-check `hash`, the `origin` id, the retention in force (`keepCount`, `keepDays`), and a SHA-256 and size for every file. | +| `reason` | `startup`, `auto` (a change was detected), `manual` (Back up now) or `pre-restore`. It is in the manifest and the Settings list, not in the folder name. | +| `.tmp-…` | A backup being written. It is renamed into place when complete and is never listed. | + +A folder counts as a backup only if its name matches that pattern **and** it has a valid +`manifest.json`. That is what makes it safe to point the folder at a directory that also holds +other things. + +## Retention + +Two limits, applied after every backup: + +| Setting | Range | Default | Meaning | +| ------- | ----- | ------- | ------- | +| **Keep this many backups** (`configBackupKeepCount`) | 2 – 500 | 20 | Keep the newest N; older ones are removed. (1 is not offered: every change would delete the state before it.) | +| **Delete backups older than (days)** (`configBackupKeepDays`) | 0 – 3650 | 30 | Also remove a backup once it has been replaced for longer than this. `0` means no age limit. | + +Both apply together. Four guarantees: + +- **The newest backup is never removed**, whatever the numbers say, so there is always one to + go back to. +- **Only folders Codeman made are ever removed**, never anything else that happens to live in a + custom folder. +- **Age counts from when a backup was replaced**, not from when it was made. Backups are made on + change, so a config that stays the same for months has one backup whose own date keeps + ageing; it is the copy you need the day something goes wrong, so it stays until a newer one + has existed for longer than the limit. +- Because backups are made on change, 20 backups reach back across 20 *changes*, however long + that took, not 20 timer ticks. + +## Settings → System → Config backups + +| Control | Setting | Default | +| ------- | ------- | ------- | +| Back up my configuration | `configBackupEnabled` | On (absent means on) | +| Backup folder | `configBackupDir` | empty = `/backups/config` | +| Keep this many backups | `configBackupKeepCount` | 20 | +| Delete backups older than (days) | `configBackupKeepDays` | 30 | +| **Back up now** | | | +| The list, with **Restore**, **Compare with current** and **Changes in this backup** per entry | | | +| **Compare** … with … **Show changes** | | | + +The settings are saved server-side and read fresh, so a change applies on the next pass. The +list and **Back up now** use the **saved** folder and retention: apply your edits first. + +## Restoring + +From the list, press **Restore** on a backup and confirm. Codeman then: + +1. **Verifies** every file in the backup against the SHA-256 in its manifest. If any file is + missing or does not match, nothing is changed and you get an error. It also refuses a backup + folder or file that is a symlink or that group or others can read or write (Codeman never + makes one like that, and writing it back as `0600` would turn it into trusted config). The + permission test is skipped when the backup folder's filesystem does not keep file modes + (FAT/exFAT drives, SMB shares, WSL's `/mnt/d` without the `metadata` option report the same + mode for every file, so it would refuse every backup there). The symlink test is not. Compare + applies the same checks to the backups it reads. +2. **Takes a `pre-restore` backup** of the files about to be replaced, so a restore can itself + be undone. When nothing changed since the newest backup (the same comparison an automatic + backup makes), no copy is made and the response names that backup instead (`safetySnapshot`). +3. **Replaces each file atomically** (written beside it, then renamed) with mode `0600`. + +Only the tracked names are ever written, whatever a manifest lists. + +If a file cannot be written (a full disk, a folder where the file should be), the restore still +writes the others and then reports an error that names the files it restored, the ones it could +not write, and the backup that holds the files as they were before. The restored files are in +effect as after a full restore (Settings re-reads `settings.json` into the form), so fix the cause +and restore again, or restore that backup to go back. + +Restoring `settings.json` keeps the four backup settings (on or off, folder, how many, how many +days) at their current values, so a restore from before a folder change does not point the list +at the old folder and hide the `pre-restore` copy it just made. The exception is the two +retention values: where the live `settings.json` lacks one (a reset file), the restore keeps the +backup's own, key by key, so the next pass does not fall back to 20 copies and 30 days. The +`pre-restore` copy records the retention in force too. + +Settings are re-read on the next request, so reload the page. `.env` and `push-keys.json` are +read once per process: **restart Codeman** for those to take effect (the response lists which). +`clis.json` is reloaded by the restore itself and `users.json` is re-read within a second. Restoring `settings.json` also does not re-run the switches +that saving in Settings applies at once (file watchers and the like), so those apply after a +restart, and `settings.json` is listed with the restart-required files. A restore does not touch +running sessions. + +When the backup folder is shared (a synced folder used by two machines, or prod and a beta), +each snapshot records a random id that the instance made once and keeps in its data directory +(`config-backup-id`; not the hostname, which a Docker container changes on every recreate). Every +instance only snapshots, lists, prunes, compares and restores its own. Deleting that file makes +the instance a new owner, and its earlier snapshots stop being listed (they stay on disk). The +same happens after a reinstall of the data directory. To see them again, copy the `origin` value +from any of their `manifest.json` files into `/config-backup-id`. + +You can also restore by hand: the files in a backup folder are the originals, so copying one back +over the file in the data directory is equivalent (stop Codeman first if you are replacing +`clis.json` or `users.json`). + +## Comparing backups + +Before you restore, or to find out what changed and when, use **Compare** in the list: + +- **Compare with current** on a backup shows what differs between that backup and the config + running now. A restore puts back the files the backup holds; files created since are left as + they are, and the restore response names them (`untouched`). +- **Changes in this backup** shows what differs between the backup before it and this one. +- The **Compare … with … Show changes** pickers compare any two backups, or a backup with the + current config. + +Each file that differs is listed with its changed keys (`theme.mode`, `tags[1]`, a variable name +in `.env`) as before → after, plus files that appeared or vanished between the two states. Files +that are the same are listed on one line. A file that is not JSON shows only that it differs, and +a file with more than 200 changes is cut short, so use the files themselves for a full diff. The +self-rewritten fields listed under "When a backup is made" are not counted as changes. + +**Credential values are never shown.** `.env`, `users.json`, `push-keys.json`, `webhook.json` (the +URL is a bearer secret) and `intents.json` (captured prompts) list only the names that changed, and in every other file a key that looks like a token, secret, password, +credential or key is shown as "changed" without its value. URL values lose any `user:password@`, +query and fragment. + +## Security + +- Some tracked files carry credentials (`.env`, `users.json`, `custom-model-hosts.json`, + `push-keys.json`, `webhook.json`). Each backup folder is mode `0700` and its files `0600`, so the default + location exposes nothing the data directory does not already. +- **A custom folder is your decision.** Putting it on a shared drive, a synced folder or a + network share copies those credentials there, with whatever permissions that location has. + Choose a private location, or turn the feature off if you cannot. +- **Retention survives a bad `settings.json`.** If it does not parse, a pass still takes its + snapshot but deletes nothing. If a retention setting is missing (a reset file), the pass + prunes with the retention the newest backup that recorded one holds, not the 20 copies and 30 + days defaults, and Settings shows those values. Saving sends a retention value only if it was + saved before or you edited it there, so saving other settings (or re-entering a custom folder + after a reset) does not write the displayed 20 and 30 over it; the first pass in the re-entered + folder finds the retention its own backups recorded. While `settings.json` does not + parse, a pass reads it as empty, so its snapshot goes to the default folder even if a custom + folder is set or backups are switched off; nothing is widened (same modes, inside the data + directory). +- **Anyone who can write to a custom backup folder can plant a backup that Restore applies.** + Restore refuses symlinks and files others can write, but a file of your own user's making + passes, so keep the folder private. +- In [multi-user mode](Multi-User-Mode) all of it is admin only: the list, Back up now, + Restore and Compare, and each restore is written to the admin audit log + (`config_backup_restore`). +- Listing returns file names and sizes, never contents. A comparison returns changed values but + never a credential's (see [Comparing backups](#comparing-backups)). + +## HTTP API + +```bash +API=http://localhost:3000 +curl -s "$API/api/config-backups" | jq # settings, folder, list +curl -s -X POST "$API/api/config-backups" | jq # back up now +curl -s -X POST "$API/api/config-backups/cfg-20261011-120305/restore" | jq +curl -s "$API/api/config-backups/diff?from=cfg-20261011-120305" | jq # that backup vs the running config +curl -s "$API/api/config-backups/diff?from=cfg-20261011-120305&to=cfg-20261012-090000" | jq +``` + +See [HTTP API](HTTP-API) and the endpoint reference in `docs/api-reference.md`. + +## Troubleshooting + +- **The list is empty.** None of the tracked files existed yet, or backups are turned off. Press + **Back up now**. +- **"Backups are not available here".** You are not an admin in multi-user mode. +- **No new backup after I changed a setting.** Changes are picked up within five minutes; **Back up + now** takes one immediately. +- **A restore said a file does not match its checksum.** The backup was altered or damaged on + disk. Nothing was changed; restore an earlier one. +- **The backups folder is growing.** Lower **Keep this many backups**. Each backup is a few + kilobytes, but `custom-model-hosts.json` and `intents.json` can be larger. +- **I want my own off-machine copy.** Point the folder at a location your backup tool already + covers, mind the Security note above, and keep **Keep this many backups** high enough that + your tool sees each one. + +## Read next + +- [Settings Reference](Settings-Reference) - where the controls live. +- [Core Concepts](Core-Concepts) - what is on disk. +- [Security](Security) - what is protected and how. +- [Troubleshooting](Troubleshooting) - "my settings were reset". diff --git a/docs/wiki/Core-Concepts.md b/docs/wiki/Core-Concepts.md index 3aab2bae..6083b847 100644 --- a/docs/wiki/Core-Concepts.md +++ b/docs/wiki/Core-Concepts.md @@ -138,6 +138,7 @@ Everything Codeman knows lives under `~/.codeman/`: | `users.json` | Multi-user accounts, mode 0600. | | `push-*.json` | Web push keys and subscriptions. | | `certs/` | Self-signed TLS for `--https`. | +| `backups/config/` | Timestamped copies of the files above that you customized ([Config Backups](Config-Backups)). | None of it needs root, none of it leaves the machine, and deleting `~/.codeman/` resets Codeman to a fresh install without touching your code. diff --git a/docs/wiki/HTTP-API.md b/docs/wiki/HTTP-API.md index 7f72223e..ec887e90 100644 --- a/docs/wiki/HTTP-API.md +++ b/docs/wiki/HTTP-API.md @@ -173,6 +173,10 @@ curl -s "$API/api/subagents" | jq # background agents curl -s "$API/api/search?q=deploy" | jq # cross-session search curl -s "$API/api/mcp-sync" | jq # preview MCP server sync (opt-in: 403 until mcpSyncEnabled is on) curl -s -X POST "$API/api/mcp-sync" | jq # apply it: add missing servers to each CLI config, never edit/remove +curl -s "$API/api/config-backups" | jq # config snapshots: folder, retention, list (admin only in multi-user mode) +curl -s -X POST "$API/api/config-backups" | jq # back up the customized config files now +curl -s -X POST "$API/api/config-backups/cfg-20261011-120000/restore" | jq # put a snapshot back (after a pre-restore copy) +curl -s "$API/api/config-backups/diff?from=cfg-20261011-120000" | jq # what differs from the running config (`to=` another snapshot id to compare two) # with ID set to a session id: curl -s "$API/api/sessions/$ID/last-response" | jq -r '.data.text' # last answer, from the transcript (claude, codex, deepseek) diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index d14c60ea..50c108bb 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -113,6 +113,7 @@ codeman web # then open http://localhost:3000 | Page | What it answers | | --------------------------------------------- | -------------------------------------------------- | | [Running As A Service](Running-As-A-Service) | How do I keep it up across reboots, and update it? | +| [Config Backups](Config-Backups) | How do I get my settings back after they were changed or lost? | | [Troubleshooting](Troubleshooting) | Why is it doing that? | | [FAQ](FAQ) | The questions that keep coming up. | | [Contributing](Contributing) | How do I send a fix? | diff --git a/docs/wiki/Security.md b/docs/wiki/Security.md index 12f576d1..71b45c2a 100644 --- a/docs/wiki/Security.md +++ b/docs/wiki/Security.md @@ -114,6 +114,11 @@ never logged and redirects are refused. Turn off **Plan Usage Limits** in App Se Two features send data outward, both off by default and both stated where they appear: voice dictation through your Claude login, and the Read My Mind prediction call. +[Config Backups](Config-Backups) copies some credential-bearing files (`.env`, `users.json`, +`custom-model-hosts.json`, `push-keys.json`, `webhook.json`) into a backup folder. The default folder is inside +the data directory, and every backup is mode `0700`/`0600`, so nothing new is exposed. Only a +folder you choose yourself can move them somewhere less private; admin only in multi-user mode. + ## Reporting a vulnerability **Never in a public issue.** diff --git a/docs/wiki/Settings-Reference.md b/docs/wiki/Settings-Reference.md index 0002931b..89e09246 100644 --- a/docs/wiki/Settings-Reference.md +++ b/docs/wiki/Settings-Reference.md @@ -190,6 +190,14 @@ Cloudflare tunnel controls including the tunnel URL. The **Diagnostics** group r tools with their versions and install hints (admin only in multi-user mode). In multi-user mode, the **Users** administration entry is injected here. +The **Config backups** group controls timestamped copies of your configuration files: an +on/off switch (on by default), the backup folder (default `~/.codeman/backups/config`), how +many backups to keep (2 to 500, default 20) and how many days to keep them (0 to 3650, default +30; 0 means no age limit). A value that is not saved in `settings.json` (a reset file) is whatever +the newest backup in that folder recorded, and the default only when none did, plus **Back up now** and a list with **Restore**. These are +server-side settings, read fresh. Details, naming and restore behaviour are on +[Config Backups](Config-Backups). + ## Session Options Per session, from the tab. @@ -244,3 +252,4 @@ Some things are configured before the server starts, not in the UI: - [The Dashboard](The-Dashboard) - what each control does once visible. - [Keeping Agents Running](Keeping-Agents-Running) - the Respawn panel in depth. - [Agent CLIs](Agent-CLIs) - model, effort, and permission modes. +- [Config Backups](Config-Backups) - copies of your settings, and how to restore them. diff --git a/docs/wiki/Troubleshooting.md b/docs/wiki/Troubleshooting.md index ab74e79c..5a8fba2e 100644 --- a/docs/wiki/Troubleshooting.md +++ b/docs/wiki/Troubleshooting.md @@ -131,6 +131,32 @@ That prompt comes from Claude Code, not Codeman. Codeman's default is to start w permission prompts skipped, which is what the security model is built around. If you would rather it prompted, change **App Settings → Agents & CLIs → Claude → Startup Mode**. +### My settings were reset, or a CLI I enabled is gone + +Something rewrote a file in `~/.codeman/` (a hand edit, a restart that loaded a bad file, a +test run pointed at the real data directory). If [Config Backups](Config-Backups) was on, +which is the default, you have copies: + +1. Open **Settings → System → Config backups** and pick the newest backup from before the + change. The list shows when each was made and what it holds. +2. **Restore** it, reload the page, and restart Codeman if the response's `restartRequired` + list is not empty. + +A reset `settings.json` also drops the backup folder setting. If you use a custom folder, enter +it again and save before looking for the list; until then the list shows the default folder. +After a reinstall of the data directory the old backups are unlisted because the instance has a +new id; see [Config Backups](Config-Backups) for how to make them visible again. + +A backup is never dated earlier than the newest one, so if the clock was once set to a far +future date, every later backup keeps that date. Delete the future-dated backup folder to +return to normal dating. + +Without the UI: the backups are plain folders in `~/.codeman/backups/config/` +(`cfg-YYYYMMDD-HHMMSS`); copy the file you need back over the one in `~/.codeman/`. + +Not covered: `state.json` (cron jobs and respawn presets live there). A backup only exists +from the moment the feature first ran, so a reset that happened before that cannot be undone. + ### Sessions vanished after a reboot Expected. tmux does not survive a reboot, so the sessions are gone. Conversations are not: diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 62e0e7a7..0b85b925 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -51,6 +51,7 @@ **Operating it** - [Running As A Service](Running-As-A-Service) +- [Config Backups](Config-Backups) - [Troubleshooting](Troubleshooting) - [FAQ](FAQ) - [Contributing](Contributing) diff --git a/src/config-backup.ts b/src/config-backup.ts new file mode 100644 index 00000000..f9e315aa --- /dev/null +++ b/src/config-backup.ts @@ -0,0 +1,948 @@ +/** + * @fileoverview Config backups: timestamped snapshots of the files that hold what a user has + * customized, kept in a default-but-configurable folder with configurable retention. + * + * Why it exists: those files are plain JSON in the data dir, rewritten by the server, the Settings UI, + * the CLI and (once, by a mis-configured test run) by things that should never touch them, and there + * was no copy to go back to. A snapshot is taken at startup and whenever the tracked files' content + * changes, so the history is one entry per actual change rather than one per timer tick. + * + * Safety rules the code enforces rather than documents: + * - The tracked file list is a fixed allowlist, never "everything in the data dir". + * - Retention only ever deletes folders this module made (name pattern AND a parseable manifest), so + * pointing the backup folder at a directory that holds other things cannot delete them. + * - The newest snapshot is never pruned, whatever the retention numbers say. + * - A snapshot is built in a temp folder and renamed into place, so a crash leaves no half snapshot. + * - Restore verifies each file against the manifest's SHA-256 and takes a `pre-restore` snapshot first. + * - Snapshot folders are 0700 and their files 0600: some tracked files carry credentials. + * + * Pure of the server: every function takes the directories it works in. + * + * @module config-backup + */ + +import { createHash, randomUUID } from 'node:crypto'; +import { + chmodSync, + existsSync, + lstatSync, + mkdirSync, + readdirSync, + readFileSync, + renameSync, + rmSync, + statSync, + writeFileSync, +} from 'node:fs'; +import { homedir } from 'node:os'; +import { isAbsolute, join, resolve, sep } from 'node:path'; + +/** + * The files (relative to the data dir) a snapshot holds: what a user configured, not what the server + * regenerates (session state, logs, the hook secret, docker seeds, caches). + * + * ⚠️ Not tracked: `state.json`. It also holds cron jobs and respawn presets, but it is rewritten with + * every session event, so tracking it would make a snapshot per event and bury the real changes. + * + * ⚠️ `.env`, `users.json`, `custom-model-hosts.json` and `push-keys.json` carry credentials or keys. + * They are already in the data dir with the same permissions; a snapshot folder is 0700 with 0600 + * files, so the default location widens nothing. A custom location is the user's own choice. + */ +export const CONFIG_BACKUP_FILES = [ + 'settings.json', + 'clis.json', + 'custom-model-hosts.json', + 'webhook.json', + 'intents.json', + 'linked-cases.json', + 'remote-hosts.json', + 'remote-cases.json', + 'docker-hosts.json', + 'docker-cases.json', + 'webviews.json', + 'push-keys.json', + 'users.json', + '.env', +] as const; + +export const CONFIG_BACKUP_DEFAULTS = { enabled: true, keepCount: 20, keepDays: 30 } as const; +/** 1 would delete the state before every change, so a bad write could never be undone. */ +export const CONFIG_BACKUP_MIN_KEEP_COUNT = 2; +export const CONFIG_BACKUP_MAX_KEEP_COUNT = 500; +export const CONFIG_BACKUP_MAX_KEEP_DAYS = 3650; + +/** `cfg-YYYYMMDD-HHMMSS`, plus `-2`, `-3`… if two land in one second. */ +const SNAPSHOT_ID_RE = /^cfg-\d{8}-\d{6}(?:-\d{1,3})?$/; +const MANIFEST = 'manifest.json'; + +export interface ConfigBackupSettings { + enabled: boolean; + /** Absolute folder, or null for the default (`/backups/config`). */ + dir: string | null; + /** Keep at most this many snapshots (newest first). 2–500. */ + keepCount: number; + /** Also drop snapshots older than this many days; 0 = no age limit. */ + keepDays: number; +} + +export interface ConfigBackupFile { + name: string; + bytes: number; + sha256: string; +} + +export interface ConfigBackupManifest { + id: string; + createdAt: number; + reason: string; + appVersion: string; + /** Digest of the set of files, so an unchanged state is not snapshotted twice. */ + hash: string; + /** Id of the data dir that made it (see {@link configBackupOrigin}), so instances sharing one folder keep to their own snapshots. */ + origin?: string; + /** + * The retention in force when it was made. If `settings.json` later fails to parse or is reset, + * the next pass prunes with these instead of the defaults, so one bad write cannot shorten a + * history the user asked to keep. + */ + keepCount?: number; + keepDays?: number; + files: ConfigBackupFile[]; +} + +export interface ConfigBackupSummary { + id: string; + createdAt: number; + reason: string; + appVersion: string; + files: Array<{ name: string; bytes: number }>; + bytes: number; +} + +const intIn = (value: unknown, min: number, max: number, fallback: number): number => { + const n = typeof value === 'number' ? value : typeof value === 'string' && value.trim() ? Number(value) : NaN; + if (!Number.isFinite(n)) return fallback; + return Math.min(max, Math.max(min, Math.floor(n))); +}; + +/** `~`-expand and require an absolute path; anything else means "use the default". */ +export function normalizeBackupDir(raw: unknown, home: string = homedir()): string | null { + if (typeof raw !== 'string') return null; + let value = raw.trim(); + if (!value || value.includes('\0')) return null; + if (value === '~') value = home; + else if (value.startsWith('~/')) value = join(home, value.slice(2)); + if (!isAbsolute(value)) return null; + return resolve(value); +} + +/** Read the four `configBackup*` settings out of a settings.json object, clamped and defaulted. */ +export function readConfigBackupSettings(raw: Record | null | undefined): ConfigBackupSettings { + const s = raw ?? {}; + return { + // Absent means ON: the feature protects against an event nobody schedules. + enabled: s.configBackupEnabled !== false, + dir: normalizeBackupDir(s.configBackupDir), + keepCount: intIn( + s.configBackupKeepCount, + CONFIG_BACKUP_MIN_KEEP_COUNT, + CONFIG_BACKUP_MAX_KEEP_COUNT, + CONFIG_BACKUP_DEFAULTS.keepCount + ), + keepDays: intIn(s.configBackupKeepDays, 0, CONFIG_BACKUP_MAX_KEEP_DAYS, CONFIG_BACKUP_DEFAULTS.keepDays), + }; +} + +export function defaultConfigBackupDir(dataDir: string): string { + return join(dataDir, 'backups', 'config'); +} + +/** + * The folder snapshots go to: the configured one, or the default. A custom folder that IS the data dir + * falls back to the default, because snapshot folders and the live files must not share a directory. + */ +export function resolveConfigBackupDir(settings: ConfigBackupSettings, dataDir: string): string { + const fallback = defaultConfigBackupDir(dataDir); + const dir = settings.dir; + if (!dir) return fallback; + const data = resolve(dataDir); + if (dir === data) return fallback; + return dir; +} + +const ORIGIN_FILE = 'config-backup-id'; +const ORIGIN_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/; + +/** + * Who owns a snapshot: a random id made once and kept in the data dir. A hostname would not do (a + * Docker container gets a new one on every recreate, which orphaned every earlier snapshot), nor + * would the path alone (two machines can share a `$HOME` path). It is a file of its own, not the + * hook secret, and is not tracked or restored. If it cannot be written the id is not persisted, so + * snapshots made meanwhile are not listed later; the data dir being unwritable breaks far more. + */ +export function configBackupOrigin(dataDir: string): string { + const path = join(dataDir, ORIGIN_FILE); + try { + const existing = readFileSync(path, 'utf8').trim(); + if (ORIGIN_RE.test(existing)) return existing; + } catch { + /* missing: made below */ + } + const id = randomUUID(); + try { + mkdirSync(dataDir, { recursive: true }); + writeFileSync(path, id + '\n', { mode: 0o600 }); + } catch { + /* see above */ + } + return id; +} + +/** The id already made for this data dir, or null. Read-only: listing must not write a file. */ +export function readConfigBackupOrigin(dataDir: string): string | null { + try { + const existing = readFileSync(join(dataDir, ORIGIN_FILE), 'utf8').trim(); + return ORIGIN_RE.test(existing) ? existing : null; + } catch { + return null; + } +} + +const sha256 = (data: Buffer | string): string => createHash('sha256').update(data).digest('hex'); + +/** + * Keys a tracked file rewrites on its own schedule, with no change to anything the user set, so + * they must not make a snapshot look different. `custom-model-hosts.json` carries a + * `lastDiscoveredAt` per host that the model-discovery poll refreshes every 5 minutes: before this, + * the change check never matched and the snapshot folder filled with copies that differed by one + * timestamp, evicting every older snapshot within about 100 minutes at the default keep count. + * The snapshot itself still stores the file exactly as it was, and restore still verifies against + * the file's own digest; only the "has anything changed?" comparison looks through these keys. + */ +const VOLATILE_KEYS: Readonly> = { + 'custom-model-hosts.json': ['lastDiscoveredAt'], + // Rewritten by activity, not by anything the user set: a Basic-auth login (at most once a minute + // per user), a session started in another case, every captured prompt, and a Docker case's + // conversation id after a /clear, and a web tab being opened. + 'users.json': ['lastLoginAt'], + 'settings.json': ['lastUsedCase'], + 'intents.json': ['recentPrompts', 'updatedAt'], + 'docker-cases.json': ['lastClaudeSessionId'], + 'webviews.json': ['lastOpenedAt'], +}; + +function withoutKeys(value: unknown, drop: ReadonlySet): unknown { + if (Array.isArray(value)) return value.map((v) => withoutKeys(v, drop)); + if (value && typeof value === 'object') { + const out: Record = {}; + for (const [k, v] of Object.entries(value as Record)) { + if (!drop.has(k)) out[k] = withoutKeys(v, drop); + } + return out; + } + return value; +} + +/** Digest used only to decide whether two states are the same: the file's own, minus its volatile keys. */ +function changeDigest(name: string, data: Buffer, rawSha: string): string { + const volatile = VOLATILE_KEYS[name]; + if (!volatile) return rawSha; + try { + return sha256(JSON.stringify(withoutKeys(JSON.parse(data.toString('utf8')), new Set(volatile)))); + } catch { + return rawSha; // not JSON (or unreadable as such): compare the bytes, as for any other file + } +} + +function stamp(now: number): string { + const d = new Date(now); + const p = (n: number, w = 2) => String(n).padStart(w, '0'); + return `${d.getFullYear()}${p(d.getMonth() + 1)}${p(d.getDate())}-${p(d.getHours())}${p(d.getMinutes())}${p(d.getSeconds())}`; +} + +function readManifest(dir: string, id: string, origin?: string): ConfigBackupManifest | null { + if (!SNAPSHOT_ID_RE.test(id)) return null; + try { + const m = JSON.parse(readFileSync(join(dir, id, MANIFEST), 'utf8')) as ConfigBackupManifest; + if (!m || m.id !== id || !Array.isArray(m.files) || typeof m.createdAt !== 'number' || typeof m.hash !== 'string') { + return null; + } + if (origin !== undefined && m.origin !== origin) return null; + return m; + } catch { + return null; + } +} + +/** A folder's entry names; none when it cannot be read (absent, or not a folder). */ +function folderEntries(dir: string): string[] { + try { + return readdirSync(dir); + } catch { + return []; + } +} + +/** + * The manifests of the snapshots this module made among `names` in `dir`, newest first. Foreign + * folders and files are ignored, and so are other instances' snapshots when `origin` is given (see + * {@link configBackupOrigin}). Each one is a file read, so a pass does this once and passes the + * result on (see {@link runConfigBackupPass}). + */ +function manifestsIn(dir: string, names: readonly string[], origin?: string): ConfigBackupManifest[] { + const out: ConfigBackupManifest[] = []; + for (const id of names) { + const m = readManifest(dir, id, origin); + if (m) out.push(m); + } + return out.sort((a, b) => b.createdAt - a.createdAt || (a.id < b.id ? 1 : -1)); +} + +/** + * Snapshots this module made in `dir`, newest first. Foreign folders and files are ignored, and so + * are other instances' snapshots when `origin` is given (see {@link configBackupOrigin}). + */ +export function listConfigBackups(dir: string, origin?: string): ConfigBackupSummary[] { + return manifestsIn(dir, folderEntries(dir), origin).map((m) => { + const files = m.files.map((f) => ({ name: f.name, bytes: f.bytes })); + return { + id: m.id, + createdAt: m.createdAt, + reason: m.reason, + appVersion: m.appVersion, + files, + bytes: files.reduce((n, f) => n + f.bytes, 0), + }; + }); +} + +export type ConfigBackupResult = + | { status: 'created'; id: string; files: number; bytes: number } + | { status: 'unchanged'; id: string } + | { status: 'empty' }; + +export interface CreateConfigBackupOptions { + dataDir: string; + backupDir: string; + reason: string; + appVersion?: string; + /** Snapshot even if nothing changed (a manual request, a pre-restore safety copy). */ + force?: boolean; + /** Recorded in the manifest, see {@link ConfigBackupManifest.keepCount}. */ + keepCount?: number; + keepDays?: number; + now?: number; +} + +/** Snapshot the tracked files that exist, unless the latest snapshot already holds exactly this state. */ +export function createConfigBackup(opts: CreateConfigBackupOptions): ConfigBackupResult { + return snapshotConfig(opts).result; +} + +/** + * {@link createConfigBackup}, also returning the manifest it wrote, over this instance's snapshots + * when the caller has already listed them (`listed`, newest first): one walk of the folder per pass. + */ +function snapshotConfig( + opts: CreateConfigBackupOptions, + listed?: { origin: string; existing: ConfigBackupManifest[] } +): { result: ConfigBackupResult; manifest: ConfigBackupManifest | null } { + const now = opts.now ?? Date.now(); + const present: Array<{ name: string; data: Buffer; sha: string; change: string }> = []; + for (const name of CONFIG_BACKUP_FILES) { + try { + const path = join(opts.dataDir, name); + if (!statSync(path).isFile()) continue; + const data = readFileSync(path); + const sha = sha256(data); + present.push({ name, data, sha, change: changeDigest(name, data, sha) }); + } catch { + /* absent or unreadable: simply not part of this snapshot */ + } + } + if (present.length === 0) return { result: { status: 'empty' }, manifest: null }; + + const hash = sha256(present.map((f) => `${f.name}\0${f.change}`).join('\n')); + const origin = listed?.origin ?? configBackupOrigin(opts.dataDir); + const existing = listed?.existing ?? manifestsIn(opts.backupDir, folderEntries(opts.backupDir), origin); + if (!opts.force && existing[0] && existing[0].hash === hash) { + return { result: { status: 'unchanged', id: existing[0].id }, manifest: null }; + } + + // Never dated earlier than this instance's newest snapshot. One written while the clock read ahead + // (a dual-boot machine, a VM resumed with a stale clock) would otherwise stay newest after the + // correction, differ from every later pass, and have each one write a copy until real time caught up. + const createdAt = existing[0] ? Math.max(now, existing[0].createdAt + 1) : now; + mkdirSync(opts.backupDir, { recursive: true, mode: 0o700 }); + const base = `cfg-${stamp(createdAt)}`; + let id = base; + for (let n = 2; existsSync(join(opts.backupDir, id)); n++) id = `${base}-${n}`; + + const tmp = join(opts.backupDir, `.tmp-${id}-${process.pid}`); + rmSync(tmp, { recursive: true, force: true }); + mkdirSync(tmp, { mode: 0o700 }); + const manifest: ConfigBackupManifest = { + id, + createdAt, + reason: opts.reason, + appVersion: opts.appVersion ?? '', + hash, + origin, + ...(opts.keepCount !== undefined && { keepCount: opts.keepCount }), + ...(opts.keepDays !== undefined && { keepDays: opts.keepDays }), + files: present.map((f) => ({ name: f.name, bytes: f.data.length, sha256: f.sha })), + }; + try { + for (const f of present) writeFileSync(join(tmp, f.name), f.data, { mode: 0o600 }); + writeFileSync(join(tmp, MANIFEST), JSON.stringify(manifest, null, 2) + '\n', { mode: 0o600 }); + renameSync(tmp, join(opts.backupDir, id)); + try { + chmodSync(join(opts.backupDir, id), 0o700); + } catch { + /* best effort on filesystems without modes */ + } + } catch (err) { + rmSync(tmp, { recursive: true, force: true }); + throw err; + } + const bytes = present.reduce((n, f) => n + f.data.length, 0); + return { result: { status: 'created', id, files: present.length, bytes }, manifest }; +} + +/** Remove `.tmp-cfg-*` build folders a crashed snapshot left behind (older than an hour, so a live one on a shared folder is safe). */ +export function sweepConfigBackupTemps(dir: string, now: number = Date.now()): void { + sweepTemps(dir, folderEntries(dir), now); +} + +function sweepTemps(dir: string, names: readonly string[], now: number): void { + for (const n of names) { + if (!/^\.tmp-cfg-\d{8}-\d{6}(?:-\d{1,3})?-\d+$/.test(n)) continue; + const st = lstatSync(join(dir, n), { throwIfNoEntry: false }); + if (st && now - st.mtimeMs > 3_600_000) rmSync(join(dir, n), { recursive: true, force: true }); + } +} + +/** + * The retention recorded in this instance's newest snapshot, or null when it recorded none. `listed` + * is this instance's manifests in `dir`, newest first, when the caller already read them. + */ +export function latestBackupRetention( + dir: string, + origin: string, + listed?: readonly ConfigBackupManifest[] +): { keepCount: number; keepDays: number } | null { + // The newest manifest that recorded one: a snapshot from before this field existed must not hide it. + for (const m of listed ?? manifestsIn(dir, folderEntries(dir), origin)) { + if (typeof m.keepCount === 'number' && typeof m.keepDays === 'number') { + return { keepCount: m.keepCount, keepDays: m.keepDays }; + } + } + return null; +} + +/** + * The settings in effect, with each retention key that `raw` (the parsed settings.json) lacks, as + * after a reset, taken from what the newest backup recorded in `dir` instead of the defaults. It + * looks at the retention keys only (not the folder), so re-entering a custom folder after a reset + * finds that folder's own manifests. One place for the timer pass and for what the GET route + * reports, so they cannot disagree. + */ +export function withRecordedRetention( + settings: ConfigBackupSettings, + raw: Record, + dir: string, + origin: string | null, + listed?: readonly ConfigBackupManifest[] +): ConfigBackupSettings { + const lacksCount = !('configBackupKeepCount' in raw); + const lacksDays = !('configBackupKeepDays' in raw); + if ((!lacksCount && !lacksDays) || !origin) return settings; + const kept = latestBackupRetention(dir, origin, listed); + if (!kept) return settings; + return { + ...settings, + ...(lacksCount && { keepCount: kept.keepCount }), + ...(lacksDays && { keepDays: kept.keepDays }), + }; +} + +export interface PruneOptions { + keepCount: number; + keepDays: number; + /** Only this instance's snapshots are pruned; omitted = every snapshot in the folder. */ + origin?: string; + now?: number; +} + +/** + * Apply retention: keep the newest `keepCount`, and (when `keepDays` > 0) drop anything older than + * that. The newest snapshot always survives. Returns the ids removed. + */ +export function pruneConfigBackups(dir: string, opts: PruneOptions): string[] { + return pruneListed(dir, manifestsIn(dir, folderEntries(dir), opts.origin), opts); +} + +/** {@link pruneConfigBackups} over a listing already made (`all`, newest first). */ +function pruneListed(dir: string, all: ReadonlyArray<{ id: string; createdAt: number }>, opts: PruneOptions): string[] { + const now = opts.now ?? Date.now(); + const cutoff = opts.keepDays > 0 ? now - opts.keepDays * 86_400_000 : null; + const removed: string[] = []; + all.forEach((b, index) => { + if (index === 0) return; + const tooMany = index >= Math.max(1, opts.keepCount); + // A snapshot's state was live until the next one was made: age it from then. Aging by its own + // createdAt deleted the last good copy on the first change after a quiet month. + const tooOld = cutoff !== null && all[index - 1].createdAt < cutoff; + if (!tooMany && !tooOld) return; + // The listing already proved this is one of ours (name pattern + manifest). + const target = join(dir, b.id); + if (!target.startsWith(resolve(dir) + sep)) return; + try { + rmSync(target, { recursive: true, force: true }); + removed.push(b.id); + } catch { + /* leave it; the next pass retries */ + } + }); + return removed; +} + +export interface ConfigBackupPassOptions { + dataDir: string; + backupDir: string; + reason: string; + appVersion?: string; + /** Snapshot even if nothing changed (Back up now). */ + force?: boolean; + /** As read from settings.json ({@link readConfigBackupSettings}), before the recorded-retention fallback. */ + settings: ConfigBackupSettings; + /** The parsed settings.json (`{}` when it did not parse): the fallback tells an absent key from a set one. */ + raw: Record; + /** False when settings.json did not parse: the pass still snapshots, but deletes nothing on its say-so. */ + prune: boolean; + now?: number; +} + +/** + * One pass of the timer or of Back up now: sweep stale temp folders, snapshot when the state changed, + * apply retention. The folder is read and each manifest parsed ONCE, and that listing feeds the + * recorded-retention fallback, the change check and the prune. It used to be listed for each of the + * three, which on a network folder is every manifest read three times every five minutes. + */ +export function runConfigBackupPass(opts: ConfigBackupPassOptions): { + result: ConfigBackupResult; + removed: string[]; +} { + const names = folderEntries(opts.backupDir); + sweepTemps(opts.backupDir, names, Date.now()); + const origin = configBackupOrigin(opts.dataDir); + const existing = manifestsIn(opts.backupDir, names, origin); + // No retention in the file (reset, or unreadable): keep what the newest snapshot recorded. + const { keepCount, keepDays } = withRecordedRetention(opts.settings, opts.raw, opts.backupDir, origin, existing); + const { result, manifest } = snapshotConfig( + { + dataDir: opts.dataDir, + backupDir: opts.backupDir, + reason: opts.reason, + appVersion: opts.appVersion, + force: opts.force, + keepCount, + keepDays, + now: opts.now, + }, + { origin, existing } + ); + if (!opts.prune) return { result, removed: [] }; + // A snapshot made now is dated after existing[0], so it heads the listing retention counts. + const all = manifest ? [manifest, ...existing] : existing; + return { result, removed: pruneListed(opts.backupDir, all, { keepCount, keepDays, now: opts.now }) }; +} + +export class ConfigBackupNotFoundError extends Error {} +export class ConfigBackupCorruptError extends Error {} + +/** + * A restore that wrote some files and could not write others (a full disk, a directory where a file + * should be). Every file passed verification first, so the ones written are the backup's: the caller + * still has to reload what they feed and tell the user which files are which. + */ +export class ConfigBackupPartialRestoreError extends Error { + constructor( + readonly restored: string[], + readonly failed: Array<{ name: string; error: string }>, + readonly untouched: string[], + readonly safetySnapshot: string | null + ) { + const done = restored.length ? `${restored.join(', ')} restored` : 'nothing was restored'; + const undo = safetySnapshot ? `; the files from before the restore are in backup ${safetySnapshot}` : ''; + super( + `Restore incomplete: could not write ${failed.map((f) => f.name).join(', ')} (${failed[0]?.error}); ${done}${undo}` + ); + } +} + +/** + * Restore refuses what this module never writes. Snapshots are 0700/0600 real files, and a restore + * writes the file as 0600: a symlink, or a file with a group/world bit, was planted or loosened + * after the snapshot, and rewriting it as 0600 would turn it into trusted config (`clis.json` + * selects the binaries Codeman spawns; `isUnsafePermissions` refuses the same bits there). + * POSIX only, like `isUnsafePermissions`. The mode test is skipped (the symlink test is not) on a + * filesystem that does not keep modes, see {@link folderKeepsModes}. + */ +function assertTrustedSnapshotEntry(path: string, label: string, checkMode: boolean): void { + if (process.platform === 'win32') return; + const st = lstatSync(path, { throwIfNoEntry: false }); + if (!st) return; // missing is reported by the read that follows + if (st.isSymbolicLink()) throw new ConfigBackupCorruptError(`${label} is a symlink, so it is not used`); + if (checkMode && (st.mode & 0o077) !== 0) { + throw new ConfigBackupCorruptError(`${label} is readable or writable by group or others, so it is not used`); + } +} + +/** + * Whether the backup folder's filesystem keeps a file's mode: vfat/exFAT, CIFS shares and WSL's + * `/mnt/d` report 0755 or 0777 for everything, so the group/world test would refuse every backup + * there even though snapshots succeed (`createConfigBackup` already expects mode-less filesystems). + * Probed with a 0600 temp file made with `wx` under a random name; if the probe cannot run, the + * strict test stays on. + */ +function folderKeepsModes(dir: string): boolean { + if (process.platform === 'win32') return false; + // A random name and 'wx' (fail if anything is there): a planted symlink or directory with a + // guessable name must neither be written through nor break the probe. Only a file this call + // created is removed. + const probe = join(dir, `.mode-probe-${randomUUID()}`); + let created = false; + try { + writeFileSync(probe, '', { mode: 0o600, flag: 'wx' }); + created = true; + chmodSync(probe, 0o600); + return (lstatSync(probe).mode & 0o777 & 0o077) === 0; + } catch { + return true; + } finally { + if (created) rmSync(probe, { force: true }); + } +} + +/** The folder's own backup settings stay as they are now: restoring an old `configBackupDir` hides the `pre-restore` copy. */ +const BACKUP_SETTING_KEYS = ['configBackupEnabled', 'configBackupDir', 'configBackupKeepCount', 'configBackupKeepDays']; + +function keepCurrentBackupSettings(snapshot: Buffer, livePath: string): Buffer { + try { + const restored = JSON.parse(snapshot.toString('utf8')) as Record; + const live = JSON.parse(readFileSync(livePath, 'utf8')) as Record; + if (!restored || typeof restored !== 'object' || !live || typeof live !== 'object') return snapshot; + let changed = false; + // A reset live file lacks the retention keys. Taking "absent" for them would bring the defaults + // back (20 copies, 30 days) on the next pass, so the backup's own retention stays. + for (const key of BACKUP_SETTING_KEYS) { + if (!(key in live) && (key === 'configBackupKeepCount' || key === 'configBackupKeepDays')) continue; + if (JSON.stringify(restored[key]) === JSON.stringify(live[key])) continue; + changed = true; + if (key in live) restored[key] = live[key]; + else delete restored[key]; + } + if (!changed) return snapshot; // byte-identical to the snapshot when the folder settings did not move + return Buffer.from(JSON.stringify(restored, null, 2) + '\n'); + } catch { + return snapshot; // not JSON, or no live file to take the values from + } +} + +export interface RestoreResult { + restored: string[]; + /** Tracked files present now that the snapshot did not hold; a restore leaves them as they are. */ + untouched: string[]; + /** + * The snapshot holding the live files as they were before the restore: the `pre-restore` copy, or the + * newest backup when it already held exactly that state. Null only when no tracked file existed. + */ + safetySnapshot: string | null; +} + +/** + * Put a snapshot's files back into the data dir. Every file is checked against the manifest's digest + * BEFORE anything is written, a `pre-restore` snapshot of the live files is taken first (unless the + * newest backup already holds that state, which is then reported instead), and each file is replaced + * atomically (temp + rename) with mode 0600. A file that cannot be written does not stop the others; + * it throws {@link ConfigBackupPartialRestoreError} at the end, naming both lists. + */ +export function restoreConfigBackup(opts: { + dataDir: string; + backupDir: string; + id: string; + appVersion?: string; + now?: number; + /** Override the probe of whether the backup folder keeps file modes (tests; a mode-less filesystem). */ + keepsModes?: boolean; +}): RestoreResult { + const origin = readConfigBackupOrigin(opts.dataDir); + const manifest = origin ? readManifest(opts.backupDir, opts.id, origin) : null; + if (!manifest || !origin) throw new ConfigBackupNotFoundError(`No config backup named ${opts.id}`); + + const allowed = new Set(CONFIG_BACKUP_FILES); + const loaded: Array<{ name: string; data: Buffer }> = []; + const checkMode = opts.keepsModes ?? folderKeepsModes(opts.backupDir); + assertTrustedSnapshotEntry(join(opts.backupDir, opts.id), `Backup ${opts.id}`, checkMode); + for (const f of manifest.files) { + if (!allowed.has(f.name)) continue; + let data: Buffer; + assertTrustedSnapshotEntry(join(opts.backupDir, opts.id, f.name), `${f.name} in ${opts.id}`, checkMode); + try { + data = readFileSync(join(opts.backupDir, opts.id, f.name)); + } catch { + throw new ConfigBackupCorruptError(`${f.name} is missing from ${opts.id}`); + } + if (sha256(data) !== f.sha256) + throw new ConfigBackupCorruptError(`${f.name} in ${opts.id} does not match its checksum`); + loaded.push({ name: f.name, data }); + } + if (loaded.length === 0) throw new ConfigBackupCorruptError(`${opts.id} holds no restorable files`); + + // One walk of the folder serves both the recorded retention and the snapshot below. + const existing = manifestsIn(opts.backupDir, folderEntries(opts.backupDir), origin); + const { result: safety } = snapshotConfig( + { + dataDir: opts.dataDir, + backupDir: opts.backupDir, + reason: 'pre-restore', + appVersion: opts.appVersion, + // Not forced: when the live files equal the newest backup, that backup already is the undo copy, + // and a forced duplicate would spend a retention slot on nothing. + now: opts.now, + // The retention in force, so this newest snapshot does not hide it from the next pass. + ...(latestBackupRetention(opts.backupDir, origin, existing) ?? {}), + }, + { origin, existing } + ); + + mkdirSync(opts.dataDir, { recursive: true }); + const restored: string[] = []; + const failed: Array<{ name: string; error: string }> = []; + for (const f of loaded) { + const target = join(opts.dataDir, f.name); + const tmp = `${target}.restore-${process.pid}`; + try { + writeFileSync(tmp, f.name === 'settings.json' ? keepCurrentBackupSettings(f.data, target) : f.data, { + mode: 0o600, + }); + renameSync(tmp, target); + restored.push(f.name); + } catch (err) { + try { + rmSync(tmp, { force: true }); + } catch { + /* the temp is a leftover at worst; the failure below is what matters */ + } + // Keep going: each file is replaced on its own, and the files written before this one are the + // backup's already. Stopping here used to answer a bare 500 that named neither list. + failed.push({ name: f.name, error: err instanceof Error ? err.message : String(err) }); + } + } + // A restore never deletes: tracked files the snapshot did not hold stay as they are. + const untouched = CONFIG_BACKUP_FILES.filter( + (n) => + !restored.includes(n) && + !failed.some((x) => x.name === n) && + lstatSync(join(opts.dataDir, n), { throwIfNoEntry: false })?.isFile() + ); + const safetySnapshot = safety.status === 'empty' ? null : safety.id; + if (failed.length) throw new ConfigBackupPartialRestoreError(restored, failed, untouched, safetySnapshot); + return { restored, untouched, safetySnapshot }; +} + +/** The live files in the data dir, as a side of a diff. */ +export const CONFIG_DIFF_CURRENT = 'current'; + +export interface ConfigDiffChange { + /** Dotted path inside the file (`a.b[0].c`), or the variable name for `.env`. */ + path: string; + kind: 'added' | 'removed' | 'changed'; + /** JSON text of the value; absent when it did not exist on that side or `redacted` is set. */ + before?: string; + after?: string; + /** The value is a credential, so only the fact that it differs is reported. */ + redacted?: boolean; +} + +export interface ConfigDiffFile { + name: string; + /** `added` = only in `to`, `removed` = only in `from`. */ + status: 'added' | 'removed' | 'changed' | 'unchanged'; + changes: ConfigDiffChange[]; + /** More changes than `MAX_DIFF_CHANGES`; the rest are not listed. */ + truncated: boolean; + /** `keys` = per-key changes; `bytes` = the file is not JSON/env, so only "differs" is known. */ + detail: 'keys' | 'bytes'; +} + +export interface ConfigDiff { + from: string; + to: string; + files: ConfigDiffFile[]; + /** Files whose status is not `unchanged`. */ + changed: number; +} + +const MAX_DIFF_CHANGES = 200; +const MAX_DIFF_VALUE_CHARS = 200; + +/** Files whose every value is a credential or key: only names are ever shown. */ +const REDACT_ALL_FILES: ReadonlySet = new Set([ + '.env', + 'users.json', + 'push-keys.json', + 'webhook.json', + 'intents.json', +]); +/** Elsewhere a value is hidden when its own key looks like a credential. */ +const SECRET_KEY_RE = /(token|secret|passw|credential|auth|key$)/i; + +type Leaves = Map; + +function flattenJson(value: unknown, path: string, out: Leaves): void { + if (Array.isArray(value) && value.length > 0) { + value.forEach((v, i) => flattenJson(v, `${path}[${i}]`, out)); + } else if (value && typeof value === 'object' && Object.keys(value).length > 0) { + for (const [k, v] of Object.entries(value as Record)) { + flattenJson(v, path ? `${path}.${k}` : k, out); + } + } else { + out.set(path || '(root)', JSON.stringify(value) ?? 'null'); + } +} + +function parseEnv(text: string): Leaves { + const out: Leaves = new Map(); + for (const line of text.split(/\r?\n/)) { + const m = /^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_.-]*)\s*=(.*)$/.exec(line); + if (m) out.set(m[1], m[2].trim()); + } + return out; +} + +/** Per-key view of a tracked file, or null when it is not JSON/env text. */ +function leavesOf(name: string, data: Buffer): Leaves | null { + const text = data.toString('utf8'); + if (name === '.env') return parseEnv(text); + try { + const parsed = JSON.parse(text) as unknown; + const volatile = VOLATILE_KEYS[name]; + const out: Leaves = new Map(); + flattenJson(volatile ? withoutKeys(parsed, new Set(volatile)) : parsed, '', out); + return out; + } catch { + return null; + } +} + +/** A URL value loses its userinfo, query and fragment: they carry passwords and tokens. */ +function scrubUrl(jsonText: string): string { + try { + const v = JSON.parse(jsonText) as unknown; + if (typeof v !== 'string' || !/^[a-z][a-z0-9+.-]*:\/\//i.test(v)) return jsonText; + const u = new URL(v); + if (!u.username && !u.password && !u.search && !u.hash) return jsonText; + return JSON.stringify(`${u.protocol}//${u.host}${u.pathname}`); + } catch { + return jsonText; + } +} + +const clip = (s: string): string => (s.length > MAX_DIFF_VALUE_CHARS ? `${s.slice(0, MAX_DIFF_VALUE_CHARS)}…` : s); + +function diffLeaves(name: string, a: Leaves, b: Leaves): { changes: ConfigDiffChange[]; truncated: boolean } { + const all = REDACT_ALL_FILES.has(name); + const changes: ConfigDiffChange[] = []; + for (const path of [...new Set([...a.keys(), ...b.keys()])].sort()) { + const before = a.get(path); + const after = b.get(path); + if (before === after) continue; + const kind = before === undefined ? 'added' : after === undefined ? 'removed' : 'changed'; + if (all || SECRET_KEY_RE.test(path.split('.').pop() ?? '')) { + changes.push({ path, kind, redacted: true }); + } else { + changes.push({ + path, + kind, + ...(before !== undefined && { before: clip(scrubUrl(before)) }), + ...(after !== undefined && { after: clip(scrubUrl(after)) }), + }); + } + } + return { changes: changes.slice(0, MAX_DIFF_CHANGES), truncated: changes.length > MAX_DIFF_CHANGES }; +} + +/** The tracked files of one side, verified: a snapshot's against its manifest, the live ones as they are. */ +function loadDiffSide(side: string, dataDir: string, backupDir: string): Map { + const out = new Map(); + if (side === CONFIG_DIFF_CURRENT) { + for (const name of CONFIG_BACKUP_FILES) { + try { + const path = join(dataDir, name); + if (statSync(path).isFile()) out.set(name, readFileSync(path)); + } catch { + /* absent or unreadable: not part of this side */ + } + } + return out; + } + const origin = readConfigBackupOrigin(dataDir); + const manifest = origin ? readManifest(backupDir, side, origin) : null; + if (!manifest) throw new ConfigBackupNotFoundError(`No config backup named ${side}`); + const allowed = new Set(CONFIG_BACKUP_FILES); + const checkMode = folderKeepsModes(backupDir); + assertTrustedSnapshotEntry(join(backupDir, side), `Backup ${side}`, checkMode); + for (const f of manifest.files) { + if (!allowed.has(f.name)) continue; + let data: Buffer; + assertTrustedSnapshotEntry(join(backupDir, side, f.name), `${f.name} in ${side}`, checkMode); + try { + data = readFileSync(join(backupDir, side, f.name)); + } catch { + throw new ConfigBackupCorruptError(`${f.name} is missing from ${side}`); + } + if (sha256(data) !== f.sha256) + throw new ConfigBackupCorruptError(`${f.name} in ${side} does not match its checksum`); + out.set(f.name, data); + } + return out; +} + +/** + * What differs between two states of the tracked files; a state is a snapshot id or `current` + * (the live files). Never returns the value of a credential: see REDACT_ALL_FILES and SECRET_KEY_RE. + */ +export function diffConfigBackups(opts: { dataDir: string; backupDir: string; from: string; to: string }): ConfigDiff { + const a = loadDiffSide(opts.from, opts.dataDir, opts.backupDir); + const b = loadDiffSide(opts.to, opts.dataDir, opts.backupDir); + const files: ConfigDiffFile[] = []; + for (const name of CONFIG_BACKUP_FILES) { + const before = a.get(name); + const after = b.get(name); + if (!before && !after) continue; + const base = { name, changes: [] as ConfigDiffChange[], truncated: false }; + if (before && after && sha256(before) === sha256(after)) { + files.push({ ...base, status: 'unchanged', detail: 'keys' }); + continue; + } + const la = before ? leavesOf(name, before) : new Map(); + const lb = after ? leavesOf(name, after) : new Map(); + const status = !before ? 'added' : !after ? 'removed' : 'changed'; + if (!la || !lb) { + files.push({ ...base, status, detail: 'bytes' }); + continue; + } + const { changes, truncated } = diffLeaves(name, la, lb); + // Only volatile keys differed (a discovery timestamp): the same state as far as the user is concerned. + files.push({ + name, + status: changes.length || status !== 'changed' ? status : 'unchanged', + changes, + truncated, + detail: 'keys', + }); + } + return { from: opts.from, to: opts.to, files, changed: files.filter((f) => f.status !== 'unchanged').length }; +} diff --git a/src/intent-store.ts b/src/intent-store.ts index 2ea821ab..8055954e 100644 --- a/src/intent-store.ts +++ b/src/intent-store.ts @@ -112,6 +112,11 @@ export class IntentStore { // ----- Public API ----- + /** Drop the in-memory copy so the next call re-reads intents.json (after a config restore replaced it). */ + reload(): void { + this.profiles = null; + } + /** * The profile for a session's case. Never persists on read: an absent * profile returns an empty transient one (`updatedAt: 0`). diff --git a/src/web/public/index.html b/src/web/public/index.html index bcdec4d7..6da662b1 100644 --- a/src/web/public/index.html +++ b/src/web/public/index.html @@ -3258,6 +3258,48 @@ +
+

Config backups

server
+
+
+
+ Back up my configuration + Keeps timestamped copies of your settings, CLI overrides, custom endpoints, webhook, linked cases and (in multi-user mode) accounts. A copy is made at startup and whenever one of them changes. Some of these files hold credentials, so each copy is private to this user (mode 0700/0600). +
+ +
+
+
+ Backup folder + An absolute path (a leading ~ is expanded). Leave empty for the default shown. Only folders Codeman created there are ever pruned. +
+ +
+
+
+ Keep this many backups + 2 to 500 (with 1, every change would delete the state before it). The newest backup is always kept. +
+ +
+
+
+ Delete backups older than (days) + A backup is removed once a newer one has existed this many days. 0 keeps backups until the count above pushes them out. +
+ +
+
+
+ Backups + Folder and retention changes apply after you save. Restoring puts a backup's files back and makes a copy of the current ones first. Compare shows what changed between two backups, or between a backup and the running config; credential values are never shown. +
+ +
+
Loading…
+
+
+

Automation

synced
diff --git a/src/web/public/settings-ui.js b/src/web/public/settings-ui.js index 926ea984..5504719c 100644 --- a/src/web/public/settings-ui.js +++ b/src/web/public/settings-ui.js @@ -555,6 +555,12 @@ Object.assign(CodemanApp.prototype, { this._mcpSyncSavedOn = settings.mcpSyncEnabled === true; document.getElementById('appSettingsMcpSync').checked = this._mcpSyncSavedOn; this.applyMcpSyncVisibility(); + // Config backups: absent means ON, 20 copies, 30 days. + document.getElementById('appSettingsConfigBackup').checked = settings.configBackupEnabled !== false; + document.getElementById('appSettingsConfigBackupDir').value = settings.configBackupDir || ''; + document.getElementById('appSettingsConfigBackupKeepCount').value = settings.configBackupKeepCount ?? 20; + document.getElementById('appSettingsConfigBackupKeepDays').value = settings.configBackupKeepDays ?? 30; + this.loadConfigBackups(); this._applyDoctorAdminGate(); this.loadWebhook(); // Read My Mind: synced, default OFF (opt-in; capture + prediction cost real tokens). @@ -1365,10 +1371,178 @@ Object.assign(CodemanApp.prototype, { * wired to `codeman:me` for the same late-resolving role as the groups above. */ _applyDoctorAdminGate() { - const group = document.getElementById('doctorGroup'); - if (!group) return; const me = window.__codemanUser || {}; - group.style.display = me.multiUser && me.role !== 'admin' ? 'none' : ''; + for (const id of ['doctorGroup', 'configBackupGroup']) { + const group = document.getElementById(id); + if (group) group.style.display = me.multiUser && me.role !== 'admin' ? 'none' : ''; + } + }, + + // ── Config backups ────────────────────────────────────────────────────────── + // Server-side files, so these read the SAVED settings: a folder or retention edit applies after Save. + + /** + * The retention keys a Save sends: a value only once it was saved before or edited here. After a reset + * the form merely DISPLAYS what the passes use (the retention the backups recorded), and writing that back + * would replace it with whatever a different folder or an older default says. + */ + _configBackupRetentionPayload(prev) { + const count = document.getElementById('appSettingsConfigBackupKeepCount'); + const days = document.getElementById('appSettingsConfigBackupKeepDays'); + const out = {}; + if (prev.configBackupKeepCount !== undefined || count.dataset.touched) { + out.configBackupKeepCount = Math.min(500, Math.max(2, parseInt(count.value, 10) || 20)); + } + if (prev.configBackupKeepDays !== undefined || days.dataset.touched) { + const n = parseInt(days.value, 10); + out.configBackupKeepDays = Math.min(3650, Math.max(0, Number.isNaN(n) ? 30 : n)); + } + return out; + }, + + /** Fill the backup list, and show the default folder as the input's placeholder. */ + async loadConfigBackups() { + const out = this.$('configBackupList'); + if (!out) return; + const res = await this._api('/api/config-backups'); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + out.textContent = body?.error || 'Backups are not available here (admin only in multi-user mode).'; + return; + } + const data = body.data; + const dirInput = this.$('appSettingsConfigBackupDir') || document.getElementById('appSettingsConfigBackupDir'); + if (dirInput && data.defaultDir) dirInput.placeholder = `default: ${data.defaultDir}`; + // A reset settings.json has none of the four keys, but the passes keep the retention the newest backup + // recorded: show that, or a Save would write 20 / 30 over it. + const saved = this.loadAppSettingsFromStorage(); + const countInput = document.getElementById('appSettingsConfigBackupKeepCount'); + const daysInput = document.getElementById('appSettingsConfigBackupKeepDays'); + if (countInput && saved.configBackupKeepCount === undefined && data.settings) countInput.value = data.settings.keepCount; + if (daysInput && saved.configBackupKeepDays === undefined && data.settings) daysInput.value = data.settings.keepDays; + if (!data.backups.length) { + out.innerHTML = `No backups yet in ${escapeHtml(data.dir)}.`; + return; + } + const kb = (n) => (n < 1024 ? `${n} B` : `${(n / 1024).toFixed(1)} KB`); + this._configBackups = data.backups; + const rows = data.backups.map((b, i) => { + const when = new Date(b.createdAt).toLocaleString(); + const names = b.files.map((f) => escapeHtml(f.name)).join(', '); + const id = escapeHtml(b.id); + // The list is newest first, so the next row is the backup this one followed. + const prev = data.backups[i + 1]; + const vsPrev = prev ? `` : ''; + return `
  • ${escapeHtml(when)} (${escapeHtml(b.reason)}, ${kb(b.bytes)}) + + + ${vsPrev} +
    ${names}
    • `; + }); + const options = (sel) => [``] + .concat(data.backups.map((b) => ``)) + .join(''); + const picker = data.backups.length + ? `
    Compare with +
    ` + : ''; + out.innerHTML = `
    ${data.backups.length} backup${data.backups.length === 1 ? '' : 's'} in ${escapeHtml(data.dir)}
    ${picker}
      ${rows.join('')}
    `; + }, + + /** What differs between two states of the config files: a backup id or `current` (the running config). */ + async showConfigDiff(from, to) { + const out = this.$('configBackupDiff'); + if (!out) return; + out.textContent = 'Comparing…'; + const res = await this._api(`/api/config-backups/diff?from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}`); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + out.textContent = body?.error || 'Could not compare these'; + return; + } + const label = (id) => { + if (id === 'current') return 'the current config'; + const b = (this._configBackups || []).find((x) => x.id === id); + return b ? new Date(b.createdAt).toLocaleString() : id; + }; + const d = body.data; + const verb = { added: 'added', removed: 'removed', changed: 'changed' }; + const files = d.files.filter((f) => f.status !== 'unchanged').map((f) => { + let detail; + if (f.detail === 'bytes') { + detail = '
    Differs (not a key/value file, so no detail).
    '; + } else { + const rows = f.changes.map((c) => { + const val = (v) => `${escapeHtml(v ?? '')}`; + const text = c.redacted ? `${verb[c.kind]} (value hidden: credential)` + : c.kind === 'added' ? `added ${val(c.after)}` + : c.kind === 'removed' ? `removed (was ${val(c.before)})` + : `${val(c.before)} → ${val(c.after)}`; + return `
  • ${escapeHtml(c.path)}: ${text}
    • `; + }); + const more = f.truncated ? '
  • More changes not shown.
    • ' : ''; + detail = f.changes.length ? `
      ${rows.join('')}${more}
    ` : ''; + } + const whole = f.status === 'added' ? ' (new file)' : f.status === 'removed' ? ' (file gone)' : ''; + return `
    ${escapeHtml(f.name)}${whole}${detail}
    `; + }); + const same = d.files.filter((f) => f.status === 'unchanged').map((f) => escapeHtml(f.name)); + out.innerHTML = `
    ${escapeHtml(label(d.from))} → ${escapeHtml(label(d.to))}: ${ + d.changed ? `${d.changed} file${d.changed === 1 ? '' : 's'} differ` : 'no differences'}
    ${files.join('')}${ + same.length ? `
    Unchanged: ${same.join(', ')}
    ` : ''}`; + }, + + async backupConfigNow() { + const btn = this.$('configBackupNowBtn'); + if (btn) btn.disabled = true; + try { + const res = await this._apiPost('/api/config-backups', {}); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + if (!res || !res.ok || !body || body.success === false) { + this.showToast(body?.error || 'Backup failed', 'error'); + } else if (body.data.status === 'empty') { + this.showToast('Nothing to back up yet', 'warning'); + } else { + this.showToast(`Backed up ${body.data.files} file${body.data.files === 1 ? '' : 's'}`, 'success'); + } + } finally { + if (btn) btn.disabled = false; + this.loadConfigBackups(); + } + }, + + async restoreConfigBackup(id) { + const names = ((this._configBackups || []).find((b) => b.id === id)?.files || []).map((f) => f.name); + const secret = names.filter((n) => ['.env', 'users.json', 'push-keys.json', 'custom-model-hosts.json', 'webhook.json'].includes(n)); + const list = names.length ? `\n\nFiles replaced: ${names.join(', ')}` : ''; + const warn = secret.length ? `\n\nThis rolls back credentials and accounts (${secret.join(', ')}): passwords, disabled accounts and keys return to how they were then.` : ''; + if (!confirm(`Restore this backup? Its files replace the current ones (a copy of the current files is made first). Some files only take effect after a restart.${list}${warn}`)) return; + const res = await this._apiPost(`/api/config-backups/${encodeURIComponent(id)}/restore`, {}); + let body = null; + try { body = res ? await res.json() : null; } catch { /* fall through */ } + const ok = res && res.ok && body && body.success !== false; + const d = body?.data; + // A restore that failed partway still answers with what it wrote (`restored`, `failed`, `safetySnapshot`): + // those files are the backup's now and need the same refill as a full restore. + if (!ok && !Array.isArray(d?.restored)) { + this.showToast(body?.error || 'Restore failed', 'error'); + return; + } + const restart = d.restartRequired?.length ? ` Restart Codeman to apply: ${d.restartRequired.join(', ')}.` : ''; + // The form still holds the pre-restore values, and Save would PUT them over the restored file. + if (d.restored.includes('settings.json')) { + await this.loadAppSettingsFromServer(); + this.openAppSettings(); + } + if (ok) { + this.showToast(`Restored ${d.restored.length} file${d.restored.length === 1 ? '' : 's'}.${restart}`, 'success'); + } else { + this.showToast(`${body.error || 'Restore failed partway'}.${restart}`, 'error'); + } + this.loadConfigBackups(); }, /** Preview (apply=false) or run (apply=true) the MCP server sync across enabled CLIs. */ @@ -2744,6 +2918,9 @@ Object.assign(CodemanApp.prototype, { customModelEndpointsEnabled: document.getElementById('appSettingsCustomModelEndpoints').checked, cliManagementEnabled: document.getElementById('appSettingsCliManagement').checked, mcpSyncEnabled: document.getElementById('appSettingsMcpSync').checked, + configBackupEnabled: document.getElementById('appSettingsConfigBackup').checked, + configBackupDir: document.getElementById('appSettingsConfigBackupDir').value.trim(), + ...this._configBackupRetentionPayload(_prev), readMyMindEnabled: document.getElementById('appSettingsReadMyMind').checked, ultracodeFloatingWindows: document.getElementById('appSettingsUltracodeFloatingWindows').checked, showMultiMonitorButton: document.getElementById('appSettingsShowMultiMonitorButton').checked, diff --git a/src/web/routes/config-backup-routes.ts b/src/web/routes/config-backup-routes.ts new file mode 100644 index 00000000..5cf78eba --- /dev/null +++ b/src/web/routes/config-backup-routes.ts @@ -0,0 +1,298 @@ +/** + * @fileoverview Config backups (src/config-backup.ts). + * + * GET /api/config-backups — settings in effect, the folder, and the snapshots in it. + * POST /api/config-backups — snapshot now (a manual request always makes one). + * POST /api/config-backups/:id/restore — put a snapshot's files back (after a `pre-restore` snapshot). + * GET /api/config-backups/diff?from=&to= — what differs between two snapshots, or a snapshot and the + * running config (`current`, the default for `to`). + * + * The files include credentials (`.env`, `users.json`, endpoint keys), so in multi-user mode all four + * are admin only. Responses carry file NAMES and sizes, never file content, except the diff, which shows + * changed values but never a credential's (`.env`, `users.json`, `push-keys.json`, `webhook.json`, `intents.json` and any key that looks + * like a token/secret/password are reported as "changed" only). Restore replaces files the running server + * may have cached: `restartRequired` says which ones only apply after a restart. A restore that could not + * write every file answers 500 with the error envelope plus `data` (`restored`, `failed`, `untouched`, + * `safetySnapshot`, `restartRequired`): the files it wrote stay restored and are reloaded like a full one. + */ + +import { readFile } from 'node:fs/promises'; +import { createRequire } from 'node:module'; +import type { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify'; +import { ApiErrorCode, createErrorResponse, getErrorMessage, type ApiResponse } from '../../types.js'; +import { getAuthUser, isAdmin, readJsonConfig, SETTINGS_PATH } from '../route-helpers.js'; +import { appendAdminAudit } from '../admin-audit.js'; +import { reloadCliRegistry } from '../../config/cli-registry/registry.js'; +import { isMultiUserMode } from '../../config/multiuser.js'; +import { getDataDir } from '../../config/instance.js'; +import { intentStore } from '../../intent-store.js'; +import { describeUnknownPath, probePathKind } from '../../utils/bounded-path-probe.js'; +import { dirname } from 'node:path'; +import { + CONFIG_BACKUP_FILES, + CONFIG_DIFF_CURRENT, + ConfigBackupCorruptError, + ConfigBackupNotFoundError, + ConfigBackupPartialRestoreError, + defaultConfigBackupDir, + diffConfigBackups, + listConfigBackups, + readConfigBackupSettings, + resolveConfigBackupDir, + restoreConfigBackup, + runConfigBackupPass, + withRecordedRetention, + readConfigBackupOrigin, + type ConfigBackupResult, + type ConfigBackupSettings, + type ConfigBackupSummary, + type ConfigDiff, +} from '../../config-backup.js'; + +// Only recorded in each manifest; a missing package.json just leaves it blank. +const APP_VERSION: string = (() => { + try { + return (createRequire(import.meta.url)('../../../package.json') as { version?: string }).version ?? ''; + } catch { + return ''; + } +})(); + +/** Files whose running-process copy only reloads on restart (the rest are read per request; `users.json` + * is re-read within a second, `clis.json` is reloaded below, `intents.json` by `reload()`). */ +const RESTART_REQUIRED = new Set(['.env', 'push-keys.json', 'settings.json']); + +export interface ConfigBackupStatus { + settings: ConfigBackupSettings; + dir: string; + defaultDir: string; + tracked: readonly string[]; + backups: ConfigBackupSummary[]; +} + +/** + * The settings in effect and the folder they name. A custom folder is probed (bounded, #516) BEFORE + * the recorded-retention fallback reads it, so this throws `BackupDirUnavailableError` for a folder + * that does not answer and never lists it; `present` is false for a custom folder that does not exist yet. + */ +export async function readEffectiveBackupSettings(): Promise<{ + settings: ConfigBackupSettings; + dir: string; + present: boolean; +}> { + const raw = await readJsonConfig>(SETTINGS_PATH, 'settings.json', {}); + const base = readConfigBackupSettings(raw); + const dir = resolveConfigBackupDir(base, getDataDir()); + const present = await probeBackupDir(base, dir); + // The same fallback the timer pass uses, so the form never shows 20 / 30 while a pass keeps 100 / 0. + const settings = present ? withRecordedRetention(base, raw, dir, readConfigBackupOrigin(getDataDir())) : base; + return { settings, dir, present }; +} + +/** A user-chosen backup folder that is not safe to touch (bounded probe, as for case paths, #516). */ +export class BackupDirUnavailableError extends Error {} + +/** + * Probe a custom backup folder without blocking the event loop on a dead mount. The default folder + * needs no probe. Returns whether the folder exists; throws when it did not answer, or is not a + * folder. A missing custom folder is only creatable when its parent is present. + */ +export async function probeBackupDir(settings: ConfigBackupSettings, dir: string): Promise { + if (!settings.dir || dir === defaultConfigBackupDir(getDataDir())) return true; + const kind = await probePathKind(dir, { pastCap: true }); + if (kind === 'unknown') + throw new BackupDirUnavailableError(describeUnknownPath('Backup folder', dir, { pastCap: true })); + if (kind === 'file') throw new BackupDirUnavailableError(`Backup folder is a file, not a folder: ${dir}`); + if (kind === 'directory') return true; + const parent = await probePathKind(dirname(dir), { pastCap: true }); + if (parent === 'unknown') { + throw new BackupDirUnavailableError(describeUnknownPath('Backup folder', dirname(dir), { pastCap: true })); + } + if (parent !== 'directory') throw new BackupDirUnavailableError(`Backup folder's parent is missing: ${dirname(dir)}`); + return false; +} + +async function readSettingsForBackup(): Promise | 'unreadable'> { + try { + const parsed: unknown = JSON.parse(await readFile(SETTINGS_PATH, 'utf8')); + return parsed && typeof parsed === 'object' && !Array.isArray(parsed) + ? (parsed as Record) + : 'unreadable'; + } catch (err) { + return (err as NodeJS.ErrnoException).code === 'ENOENT' ? {} : 'unreadable'; + } +} + +/** Snapshot (when changed) and apply retention; shared by the timer and the manual route. */ +export async function runConfigBackup( + reason: string, + opts: { force?: boolean } = {} +): Promise<{ result: ConfigBackupResult; removed: string[] } | null> { + // Read settings.json directly: readJsonConfig answers {} for ANY failure, so a syntax error or a read + // that lands inside PUT /api/settings's non-atomic write used to prune with the defaults (20 / 30 + // days) and drop the longer history the user asked to keep. + const raw = await readSettingsForBackup(); + const unreadable = raw === 'unreadable'; + const fromFile = readConfigBackupSettings(unreadable ? {} : raw); + if (!fromFile.enabled && !opts.force) return null; + const dir = resolveConfigBackupDir(fromFile, getDataDir()); + await probeBackupDir(fromFile, dir); + return runConfigBackupPass({ + dataDir: getDataDir(), + backupDir: dir, + reason, + appVersion: APP_VERSION, + force: opts.force, + settings: fromFile, + raw: unreadable ? {} : raw, + // An unreadable settings.json still gets its snapshot, but no pass deletes anything on its say-so. + prune: !unreadable, + }); +} + +export function registerConfigBackupRoutes(app: FastifyInstance): void { + const deny = (req: FastifyRequest, reply: FastifyReply): ApiResponse | null => { + if (isMultiUserMode() && !isAdmin(req)) { + reply.code(403); + return createErrorResponse(ApiErrorCode.FORBIDDEN, 'Admin only in multi-user mode'); + } + return null; + }; + + app.get('/api/config-backups', async (req, reply): Promise> => { + const denied = deny(req, reply); + if (denied) return denied; + let eff: Awaited>; + try { + eff = await readEffectiveBackupSettings(); + } catch (err) { + reply.code(503); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, getErrorMessage(err)); + } + const { settings, dir, present } = eff; + // Read-only: a listing must not create config-backup-id, and without one nothing can be ours. + const origin = readConfigBackupOrigin(getDataDir()); + return { + success: true, + data: { + settings, + dir, + defaultDir: defaultConfigBackupDir(getDataDir()), + tracked: CONFIG_BACKUP_FILES, + backups: present && origin ? listConfigBackups(dir, origin) : [], + }, + }; + }); + + app.post('/api/config-backups', async (req, reply) => { + const denied = deny(req, reply); + if (denied) return denied; + try { + const out = await runConfigBackup('manual', { force: true }); + return { success: true, data: { ...out!.result, removed: out!.removed } }; + } catch (err) { + reply.code(err instanceof BackupDirUnavailableError ? 503 : 500); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Backup failed: ${getErrorMessage(err)}`); + } + }); + + app.get('/api/config-backups/diff', async (req, reply): Promise> => { + const denied = deny(req, reply); + if (denied) return denied; + const q = req.query as { from?: string; to?: string }; + if (typeof q.from !== 'string' || !q.from) { + reply.code(400); + return createErrorResponse(ApiErrorCode.INVALID_INPUT, '`from` is required (a backup id or "current")'); + } + try { + const { dir } = await readEffectiveBackupSettings(); + const data = diffConfigBackups({ + dataDir: getDataDir(), + backupDir: dir, + from: q.from, + to: typeof q.to === 'string' && q.to ? q.to : CONFIG_DIFF_CURRENT, + }); + return { success: true, data }; + } catch (err) { + if (err instanceof ConfigBackupNotFoundError) { + reply.code(404); + return createErrorResponse(ApiErrorCode.NOT_FOUND, err.message); + } + if (err instanceof ConfigBackupCorruptError) { + reply.code(409); + return createErrorResponse(ApiErrorCode.CONFLICT, err.message); + } + reply.code(err instanceof BackupDirUnavailableError ? 503 : 500); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Diff failed: ${getErrorMessage(err)}`); + } + }); + + /** + * What a restore that wrote files must do, whether or not every file made it: reload what the running + * server caches, audit it, and name the files that only apply after a restart. + */ + const afterRestore = (req: FastifyRequest, id: string, restored: string[], failed: string[] = []): string[] => { + // IntentStore caches the file and rewrites it whole on the next prompt, which would undo the restore. + if (restored.includes('intents.json')) intentStore.reload(); + // restoreConfigBackup writes clis.json directly (the one writer besides mutateRegistryFile). + if (restored.includes('clis.json')) reloadCliRegistry(); + // A restore of users.json replaces accounts, password hashes, roles and bypass grants. + if (isMultiUserMode()) { + void appendAdminAudit({ + admin: getAuthUser(req).username, + action: 'config_backup_restore', + target: id, + ip: req.ip, + detail: { restored, ...(failed.length > 0 && { failed }) }, + }); + } + return restored.filter((n) => RESTART_REQUIRED.has(n)); + }; + + app.post('/api/config-backups/:id/restore', async (req, reply) => { + const denied = deny(req, reply); + if (denied) return denied; + const { id } = req.params as { id: string }; + try { + const { dir } = await readEffectiveBackupSettings(); + const out = restoreConfigBackup({ dataDir: getDataDir(), backupDir: dir, id, appVersion: APP_VERSION }); + return { + success: true, + data: { + restored: out.restored, + untouched: out.untouched, + safetySnapshot: out.safetySnapshot, + restartRequired: afterRestore(req, id, out.restored), + }, + }; + } catch (err) { + if (err instanceof ConfigBackupNotFoundError) { + reply.code(404); + return createErrorResponse(ApiErrorCode.NOT_FOUND, err.message); + } + if (err instanceof ConfigBackupCorruptError) { + reply.code(409); + return createErrorResponse(ApiErrorCode.CONFLICT, `${err.message}. Nothing was changed.`); + } + if (err instanceof ConfigBackupPartialRestoreError) { + // Some files are the backup's now: the client must learn which (Settings re-reads a restored + // settings.json, or its next Save writes the old values back) and where the undo copy is. + const failedNames = err.failed.map((f) => f.name); + const restartRequired = afterRestore(req, id, err.restored, failedNames); + reply.code(500); + return { + ...createErrorResponse(ApiErrorCode.OPERATION_FAILED, err.message), + data: { + restored: err.restored, + failed: err.failed, + untouched: err.untouched, + safetySnapshot: err.safetySnapshot, + restartRequired, + }, + }; + } + reply.code(err instanceof BackupDirUnavailableError ? 503 : 500); + return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Restore failed: ${getErrorMessage(err)}`); + } + }); +} diff --git a/src/web/routes/index.ts b/src/web/routes/index.ts index aa1d721f..ad34f34d 100644 --- a/src/web/routes/index.ts +++ b/src/web/routes/index.ts @@ -32,6 +32,7 @@ export { registerVoiceRoutes } from './voice-routes.js'; export { registerWebviewRoutes, tryWebviewRefererFallback } from './webview-routes.js'; export { registerTabLayoutRoutes } from './tab-layout-routes.js'; export { registerMcpSyncRoutes } from './mcp-sync-routes.js'; +export { registerConfigBackupRoutes, runConfigBackup } from './config-backup-routes.js'; export { registerWebhookRoutes } from './webhook-routes.js'; export { registerDoctorRoutes } from './doctor-routes.js'; export { diff --git a/src/web/schemas.ts b/src/web/schemas.ts index 623038ea..af044cad 100644 --- a/src/web/schemas.ts +++ b/src/web/schemas.ts @@ -1525,6 +1525,18 @@ export const SettingsUpdateSchema = z * opt-in. While OFF, GET/POST /api/mcp-sync answer 403 and the Settings controls are hidden. */ mcpSyncEnabled: z.boolean().optional(), + /** + * Config backups (src/config-backup.ts): timestamped snapshots of the customized config files, + * taken at startup and whenever their content changes. SYNCED. Absent means ON (the feature + * protects against an event nobody schedules); the folder defaults to `/backups/config`. + */ + configBackupEnabled: z.boolean().optional(), + /** Absolute folder (a leading `~` is expanded); empty or absent = the default folder. */ + configBackupDir: z.string().max(1024).optional(), + /** Keep at most this many snapshots. */ + configBackupKeepCount: z.number().int().min(2).max(500).optional(), + /** Also drop snapshots older than this many days; 0 = no age limit. */ + configBackupKeepDays: z.number().int().min(0).max(3650).optional(), /** * Read My Mind predictor model override. Empty/absent = the AI-checker * default (opus: prediction quality is the product and it runs only on an diff --git a/src/web/server.ts b/src/web/server.ts index de30f762..2eb6f3a5 100644 --- a/src/web/server.ts +++ b/src/web/server.ts @@ -223,6 +223,8 @@ import { registerWebviewRoutes, registerTabLayoutRoutes, registerMcpSyncRoutes, + registerConfigBackupRoutes, + runConfigBackup, registerWebhookRoutes, registerDoctorRoutes, registerCustomModelRoutes, @@ -246,6 +248,10 @@ const SSE_CLIENT_ID_RE = /^[A-Za-z0-9_-]{8,64}$/; const CODEX_USAGE_POLL_INTERVAL_MS = 5 * 60_000; // Copilot's quota is read over the network from GitHub, and a monthly figure moves slowly. const COPILOT_USAGE_POLL_INTERVAL_MS = 10 * 60_000; + +// Config backups: one cheap read-and-hash of a handful of small files; a snapshot is written only when +// their content changed since the latest one, so this is one entry per real change, not per tick. +const CONFIG_BACKUP_INTERVAL_MS = 5 * 60_000; const CUSTOM_MODEL_REDISCOVER_INTERVAL_MS = 5 * 60_000; // Much shorter than the model-LIST refresh above on purpose: this catches an actual // eviction (a session's model no longer loaded, silently swapped out by another @@ -1243,6 +1249,7 @@ export class WebServer extends EventEmitter { registerWebviewRoutes(this.app, ctx, this.basePath); registerTabLayoutRoutes(this.app, ctx); registerMcpSyncRoutes(this.app); + registerConfigBackupRoutes(this.app); registerWebhookRoutes(this.app, { notifier: this.webhookNotifier, configDir: getDataDir(), @@ -3164,6 +3171,18 @@ export class WebServer extends EventEmitter { // $CODEMAN_HOOK_SECRET_FILE — hook curls cat that path at execution time. getHookSecret(); + // Config backups: a snapshot at start (the state the process found) and then on change. + if (!this.testMode) { + const backup = (reason: string) => + void runConfigBackup(reason).catch((err) => + console.warn('[config-backup] snapshot failed:', err instanceof Error ? err.message : err) + ); + backup('startup'); + this.cleanup.setInterval(() => backup('auto'), CONFIG_BACKUP_INTERVAL_MS, { + description: 'Config backup', + }); + } + // Main Codex subscription limits come from the signed-in local CLI. Keep // this read-only and host-scoped; multi-user SSE routing makes it admin-only. if (!this.testMode) { diff --git a/test/config-backup-origin.test.ts b/test/config-backup-origin.test.ts new file mode 100644 index 00000000..bf6f26c4 --- /dev/null +++ b/test/config-backup-origin.test.ts @@ -0,0 +1,59 @@ +/** + * @fileoverview The snapshot origin must survive a hostname change (a Docker container gets a new + * hostname on every recreate) and still differ between data dirs. + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +const host = vi.hoisted(() => ({ name: 'container-a' })); +vi.mock('node:os', async (importOriginal) => ({ + ...(await importOriginal()), + hostname: () => host.name, +})); + +import { + configBackupOrigin, + createConfigBackup, + listConfigBackups, + pruneConfigBackups, + restoreConfigBackup, +} from '../src/config-backup.js'; + +let root: string; +let data: string; +let backups: string; +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'cfg-origin-')); + data = join(root, 'data'); + backups = join(root, 'backups'); + mkdirSync(data, { recursive: true }); + writeFileSync(join(data, 'settings.json'), '{"a":1}'); + host.name = 'container-a'; +}); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +describe('configBackupOrigin', () => { + it('is made once, kept in the data dir, and differs between data dirs', () => { + const a = configBackupOrigin(data); + expect(configBackupOrigin(data)).toBe(a); + expect(readFileSync(join(data, 'config-backup-id'), 'utf8').trim()).toBe(a); + const other = join(root, 'other'); + expect(configBackupOrigin(other)).not.toBe(a); + }); + + it('keeps snapshots listed, restorable and prunable after the hostname changes', () => { + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: 1_000_000 }) as { + id: string; + }; + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'manual', force: true, now: 2_000_000 }); + host.name = 'container-b'; // the container was recreated + const origin = configBackupOrigin(data); + expect(listConfigBackups(backups, origin)).toHaveLength(2); + writeFileSync(join(data, 'settings.json'), '{"a":2}'); // a change, so the restore takes a pre-restore copy + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id })).not.toThrow(); + const removed = pruneConfigBackups(backups, { keepCount: 1, keepDays: 0, origin }); + expect(removed).toHaveLength(2); // 3 now: the restore took a pre-restore copy + }); +}); diff --git a/test/config-backup-restore-ui.test.ts b/test/config-backup-restore-ui.test.ts new file mode 100644 index 00000000..bb086859 --- /dev/null +++ b/test/config-backup-restore-ui.test.ts @@ -0,0 +1,188 @@ +/** + * @fileoverview Restoring settings.json from inside App Settings must refill the form, or Save + * would PUT the pre-restore values back over the restored file. Evaluates the real settings-ui.js + * against a stub app. + */ +import { describe, expect, it } from 'vitest'; +import { readFileSync } from 'node:fs'; +import vm from 'node:vm'; + +const SRC = readFileSync(new URL('../src/web/public/settings-ui.js', import.meta.url), 'utf-8'); + +function makeApp( + restored: string[], + confirmed = true, + reply: { ok: boolean; body: unknown } = { ok: true, body: { success: true, data: { restored, restartRequired: [] } } } +) { + const calls: string[] = []; + const confirms: string[] = []; + const toasts: Array<[string, string]> = []; + function CodemanApp(this: unknown) {} + const ctx = vm.createContext({ + CodemanApp, + window: {}, + document: {}, + confirm: (m: string) => (confirms.push(m), confirmed), + escapeHtml: (s: string) => s, + }); + vm.runInContext(SRC, ctx); + const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record; + Object.assign(app, { + _configBackups: [ + { + id: 'cfg-1', + files: [{ name: 'settings.json' }, { name: 'users.json' }, { name: '.env' }, { name: 'clis.json' }], + }, + ], + _apiPost: async () => ({ ok: reply.ok, json: async () => reply.body }), + loadAppSettingsFromServer: async () => void calls.push('reload-settings'), + openAppSettings: () => void calls.push('refill-form'), + loadConfigBackups: () => void calls.push('reload-list'), + showToast: (m: string, type: string) => void toasts.push([m, type]), + }); + return { app, calls, confirms, toasts }; +} + +describe('restoreConfigBackup (Settings UI)', () => { + it('re-reads settings into the form when settings.json was restored', async () => { + const { app, calls } = makeApp(['settings.json', 'clis.json']); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(calls).toEqual(['reload-settings', 'refill-form', 'reload-list']); + }); + + it('leaves the form alone when settings.json was not part of the restore', async () => { + const { app, calls } = makeApp(['clis.json']); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(calls).toEqual(['reload-list']); + }); + + it('a restore that failed partway still refills the form when settings.json was among the files written', async () => { + const { app, calls, toasts } = makeApp([], true, { + ok: false, + body: { + success: false, + errorCode: 'OPERATION_FAILED', + error: 'Restore incomplete: could not write clis.json (EISDIR); settings.json restored', + data: { + restored: ['settings.json'], + failed: [{ name: 'clis.json', error: 'EISDIR' }], + safetySnapshot: 'cfg-2', + restartRequired: ['settings.json'], + }, + }, + }); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(calls).toEqual(['reload-settings', 'refill-form', 'reload-list']); + expect(toasts).toHaveLength(1); + expect(toasts[0][1]).toBe('error'); + expect(toasts[0][0]).toContain('clis.json'); + }); + + it('a refused restore (nothing written) only reports the error', async () => { + const { app, calls, toasts } = makeApp([], true, { + ok: false, + body: { success: false, errorCode: 'CONFLICT', error: 'settings.json does not match its checksum' }, + }); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(calls).toEqual([]); + expect(toasts).toEqual([['settings.json does not match its checksum', 'error']]); + }); + + it('names the files being replaced and calls out the credential ones', async () => { + const { app, confirms } = makeApp(['settings.json'], false); + await (app.restoreConfigBackup as (id: string) => Promise)('cfg-1'); + expect(confirms[0]).toContain('settings.json, users.json, .env, clis.json'); + expect(confirms[0]).toContain('credentials'); + expect(confirms[0]).toMatch(/users\.json, \.env\)/); + }); +}); + +describe('loadConfigBackups (retention fields)', () => { + function load(saved: Record) { + const inputs: Record = { + appSettingsConfigBackupKeepCount: { value: '20', placeholder: '' }, + appSettingsConfigBackupKeepDays: { value: '30', placeholder: '' }, + appSettingsConfigBackupDir: { value: '', placeholder: '' }, + }; + function CodemanApp(this: unknown) {} + const ctx = vm.createContext({ + CodemanApp, + window: {}, + document: { getElementById: (id: string) => inputs[id] ?? null }, + escapeHtml: (s: string) => s, + }); + vm.runInContext(SRC, ctx); + const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record; + Object.assign(app, { + $: () => ({ textContent: '', innerHTML: '' }), + loadAppSettingsFromStorage: () => saved, + _api: async () => ({ + ok: true, + json: async () => ({ + success: true, + data: { settings: { keepCount: 100, keepDays: 0 }, dir: '/d', defaultDir: '/d', backups: [] }, + }), + }), + }); + return { app, inputs }; + } + + it('shows the retention the passes use when settings.json has none of the keys (a reset file)', async () => { + const { app, inputs } = load({}); + await (app.loadConfigBackups as () => Promise)(); + expect(inputs.appSettingsConfigBackupKeepCount.value).toBe(100); + expect(inputs.appSettingsConfigBackupKeepDays.value).toBe(0); + }); + + it('leaves the form alone when the saved settings state the retention', async () => { + const { app, inputs } = load({ configBackupKeepCount: 7, configBackupKeepDays: 9 }); + await (app.loadConfigBackups as () => Promise)(); + expect(inputs.appSettingsConfigBackupKeepCount.value).toBe('20'); // the open-settings code owns it + }); +}); + +describe('Save and the displayed retention', () => { + function payload( + prev: Record, + touched: { count?: boolean; days?: boolean }, + count = '100', + days = '0' + ) { + const inputs: Record }> = { + appSettingsConfigBackupKeepCount: { value: count, dataset: touched.count ? { touched: '1' } : {} }, + appSettingsConfigBackupKeepDays: { value: days, dataset: touched.days ? { touched: '1' } : {} }, + }; + function CodemanApp(this: unknown) {} + const ctx = vm.createContext({ + CodemanApp, + window: {}, + document: { getElementById: (id: string) => inputs[id] }, + }); + vm.runInContext(SRC, ctx); + const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record; + return (app._configBackupRetentionPayload as (p: Record) => Record).call( + app, + prev + ); + } + + it('leaves out a retention value that was only displayed (a reset settings.json)', () => { + expect(payload({}, {})).toEqual({}); + }); + + it('sends a value that was saved before, or edited here, and only that one', () => { + expect(payload({ configBackupKeepCount: 100 }, {})).toEqual({ configBackupKeepCount: 100 }); + expect(payload({}, { days: true })).toEqual({ configBackupKeepDays: 0 }); + expect(payload({}, { count: true, days: true }, '5', '7')).toEqual({ + configBackupKeepCount: 5, + configBackupKeepDays: 7, + }); + }); + + it('clamps like the form always did', () => { + expect(payload({}, { count: true, days: true }, '1', '99999')).toEqual({ + configBackupKeepCount: 2, + configBackupKeepDays: 3650, + }); + }); +}); diff --git a/test/config-backup.test.ts b/test/config-backup.test.ts new file mode 100644 index 00000000..2a7e1231 --- /dev/null +++ b/test/config-backup.test.ts @@ -0,0 +1,686 @@ +/** + * @fileoverview Config backups (src/config-backup.ts): what is tracked, when a snapshot is made, + * how retention prunes, and that restore verifies before it writes. Pure fs over temp dirs; + * port: N/A. + */ +import { + chmodSync, + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + symlinkSync, + statSync, + writeFileSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { + CONFIG_BACKUP_FILES, + ConfigBackupCorruptError, + ConfigBackupNotFoundError, + ConfigBackupPartialRestoreError, + createConfigBackup, + defaultConfigBackupDir, + diffConfigBackups, + listConfigBackups, + normalizeBackupDir, + pruneConfigBackups, + readConfigBackupSettings, + resolveConfigBackupDir, + restoreConfigBackup, + configBackupOrigin, + latestBackupRetention, + sweepConfigBackupTemps, +} from '../src/config-backup.js'; + +let root: string; +let data: string; +let backups: string; +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'cfg-backup-')); + data = join(root, 'data'); + backups = join(root, 'backups'); + mkdirSync(data); +}); +afterEach(() => rmSync(root, { recursive: true, force: true })); + +const put = (name: string, text: string) => writeFileSync(join(data, name), text); +const DAY = 86_400_000; +const T0 = Date.UTC(2026, 9, 1, 12, 0, 0); + +describe('settings', () => { + it('defaults to ON, 20 copies, 30 days, default folder', () => { + expect(readConfigBackupSettings({})).toEqual({ enabled: true, dir: null, keepCount: 20, keepDays: 30 }); + expect(readConfigBackupSettings(null).enabled).toBe(true); + }); + + it('only an explicit false turns it off, and numbers are clamped to their ranges', () => { + expect(readConfigBackupSettings({ configBackupEnabled: false }).enabled).toBe(false); + const s = readConfigBackupSettings({ configBackupKeepCount: 9999, configBackupKeepDays: -3 }); + expect(s.keepCount).toBe(500); + expect(s.keepDays).toBe(0); + expect(readConfigBackupSettings({ configBackupKeepCount: 0 }).keepCount).toBe(2); + expect(readConfigBackupSettings({ configBackupKeepCount: 'abc' }).keepCount).toBe(20); + expect(readConfigBackupSettings({ configBackupKeepCount: 7.9 }).keepCount).toBe(7); + }); + + it('expands ~ and refuses anything that is not an absolute path', () => { + expect(normalizeBackupDir('~/cfg', '/home/u')).toBe('/home/u/cfg'); + expect(normalizeBackupDir('/mnt/backups/codeman', '/home/u')).toBe('/mnt/backups/codeman'); + for (const bad of ['', ' ', 'relative/dir', './here', undefined, 42, '/x\0y']) { + expect(normalizeBackupDir(bad as never, '/home/u'), String(bad)).toBeNull(); + } + }); + + it('uses /backups/config by default and never the data dir itself', () => { + expect(resolveConfigBackupDir(readConfigBackupSettings({}), '/d')).toBe(defaultConfigBackupDir('/d')); + expect(resolveConfigBackupDir(readConfigBackupSettings({ configBackupDir: '/elsewhere' }), '/d')).toBe( + '/elsewhere' + ); + expect(resolveConfigBackupDir(readConfigBackupSettings({ configBackupDir: '/d' }), '/d')).toBe( + defaultConfigBackupDir('/d') + ); + }); +}); + +describe('createConfigBackup', () => { + it('snapshots only the tracked files that exist, with a manifest', () => { + put('settings.json', '{"a":1}'); + put('clis.json', '{"clis":{}}'); + put('state.json', '{"runtime":true}'); // not config: must not be copied + put('hook-secret', 'x'); // regenerated: must not be copied + const r = createConfigBackup({ + dataDir: data, + backupDir: backups, + reason: 'startup', + appVersion: '1.2.3', + now: T0, + }); + expect(r.status).toBe('created'); + const [b] = listConfigBackups(backups); + expect(b.files.map((f) => f.name).sort()).toEqual(['clis.json', 'settings.json']); + expect(b.reason).toBe('startup'); + expect(b.appVersion).toBe('1.2.3'); + expect(b.createdAt).toBe(T0); + expect(existsSync(join(backups, b.id, 'state.json'))).toBe(false); + expect(readFileSync(join(backups, b.id, 'settings.json'), 'utf8')).toBe('{"a":1}'); + }); + + it('tracks a fixed allowlist', () => { + expect([...CONFIG_BACKUP_FILES]).toContain('settings.json'); + expect([...CONFIG_BACKUP_FILES]).not.toContain('state.json'); + expect([...CONFIG_BACKUP_FILES]).not.toContain('hook-secret'); + }); + + it('makes no second snapshot while nothing changed, and one when something does', () => { + put('settings.json', '{"a":1}'); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + const again = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }); + expect(again).toEqual({ status: 'unchanged', id: (first as { id: string }).id }); + expect(listConfigBackups(backups)).toHaveLength(1); + + put('settings.json', '{"a":2}'); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'created' + ); + expect(listConfigBackups(backups)).toHaveLength(2); + }); + + it('does not count a self-refreshed discovery timestamp as a change', () => { + const hosts = (stamp: string, model = 'm1') => + JSON.stringify({ hosts: [{ id: 'h', models: [{ id: model }], lastDiscoveredAt: stamp }] }); + put('custom-model-hosts.json', hosts('2026-10-12T08:44:22.026Z')); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + + // The 5-minute discovery poll rewrites only the timestamp: still the same state. + put('custom-model-hosts.json', hosts('2026-10-12T08:49:22.025Z')); + const again = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }); + expect(again).toEqual({ status: 'unchanged', id: (first as { id: string }).id }); + expect(listConfigBackups(backups)).toHaveLength(1); + + // A real edit next to a refreshed timestamp is still a change, stored exactly as written. + const edited = hosts('2026-10-12T08:54:22.025Z', 'm2'); + put('custom-model-hosts.json', edited); + const third = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }); + expect(third.status).toBe('created'); + expect(readFileSync(join(backups, (third as { id: string }).id, 'custom-model-hosts.json'), 'utf8')).toBe(edited); + }); + + it.each([ + [ + 'webviews.json', + (n: number) => + JSON.stringify([ + { + id: 'w1', + name: 'Grafana', + url: 'http://localhost:3001', + embedMode: 'proxy', + trusted: false, + createdAt: 1, + lastOpenedAt: n, + }, + ]), + 'name', + ], + [ + 'docker-cases.json', + (n: number) => JSON.stringify([{ name: 'app', hostId: 'h', lastClaudeSessionId: `c${n}` }]), + 'hostId', + ], + ['users.json', (n: number) => JSON.stringify({ users: [{ name: 'a', role: 'admin', lastLoginAt: n }] }), 'role'], + ['settings.json', (n: number) => JSON.stringify({ displayName: 'x', lastUsedCase: `case-${n}` }), 'displayName'], + [ + 'intents.json', + (n: number) => JSON.stringify({ profiles: [{ key: 'k', goals: 'g', updatedAt: n, recentPrompts: [`p${n}`] }] }), + 'goals', + ], + ])('does not count activity in %s as a change, but does count an edit next to it', (name, make, editKey) => { + put(name, make(1)); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put(name, make(2)); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 })).toEqual({ + status: 'unchanged', + id: (first as { id: string }).id, + }); + put(name, make(3).replace(new RegExp(`"${editKey}":"[^"]*"`), `"${editKey}":"edited"`)); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'created' + ); + }); + + it('compares a tracked file that is not valid JSON by its bytes', () => { + put('custom-model-hosts.json', 'not json lastDiscoveredAt 1'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put('custom-model-hosts.json', 'not json lastDiscoveredAt 2'); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }).status).toBe( + 'created' + ); + }); + + it('force makes a snapshot even when unchanged, and two in one second get distinct ids', () => { + put('settings.json', '{}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'a', now: T0 }); + const forced = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'manual', force: true, now: T0 }); + expect(forced.status).toBe('created'); + const ids = listConfigBackups(backups).map((b) => b.id); + expect(new Set(ids).size).toBe(2); + }); + + it('reports empty, creating nothing, when no tracked file exists', () => { + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'x' })).toEqual({ status: 'empty' }); + expect(existsSync(backups)).toBe(false); + }); + + it('keeps snapshots private: 0700 folder and 0600 files', () => { + put('settings.json', '{}'); + put('.env', 'CODEMAN_PASSWORD=secret'); + const r = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'x', now: T0 }); + const id = (r as { id: string }).id; + expect(statSync(join(backups, id)).mode & 0o777).toBe(0o700); + expect(statSync(join(backups, id, '.env')).mode & 0o777).toBe(0o600); + expect(statSync(join(backups, id, 'manifest.json')).mode & 0o777).toBe(0o600); + }); + + it('leaves no temp folder behind', () => { + put('settings.json', '{}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'x', now: T0 }); + const stray = readdirSync(backups).filter((n) => n.startsWith('.tmp-')); + expect(stray).toEqual([]); + }); +}); + +describe('retention', () => { + const makeN = (n: number) => { + for (let i = 0; i < n; i++) { + put('settings.json', JSON.stringify({ i })); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + i * DAY }); + } + }; + + it('keeps the newest keepCount', () => { + makeN(6); + const removed = pruneConfigBackups(backups, { keepCount: 3, keepDays: 0, now: T0 + 6 * DAY }); + expect(removed).toHaveLength(3); + const left = listConfigBackups(backups); + expect(left).toHaveLength(3); + expect(left.map((b) => b.createdAt)).toEqual([T0 + 5 * DAY, T0 + 4 * DAY, T0 + 3 * DAY]); + }); + + it('drops anything older than keepDays, but never the newest', () => { + makeN(4); // T0 .. T0+3d + // At T0+100d everything is older than 30 days: the newest still survives. + const removed = pruneConfigBackups(backups, { keepCount: 50, keepDays: 30, now: T0 + 100 * DAY }); + expect(removed).toHaveLength(3); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([T0 + 3 * DAY]); + }); + + it('applies both limits together and keepDays 0 means no age limit', () => { + makeN(5); + expect(pruneConfigBackups(backups, { keepCount: 50, keepDays: 0, now: T0 + 999 * DAY })).toEqual([]); + const removed = pruneConfigBackups(backups, { keepCount: 4, keepDays: 3, now: T0 + 5 * DAY }); + // Newest first: T0+4d, +3d, +2d, +1d, T0. A snapshot ages from its successor: T0+1d was live + // until T0+2d, which sits exactly on the cutoff (kept); only T0 is over the count. + expect(removed.length).toBe(1); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([ + T0 + 4 * DAY, + T0 + 3 * DAY, + T0 + 2 * DAY, + T0 + DAY, + ]); + }); + + it('keeps the last good copy when the first change comes after a quiet month', () => { + put('settings.json', '{"theme":"good"}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'startup', now: T0 }); + put('settings.json', '{}'); // the bad write, 40 days later + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 40 * DAY }); + pruneConfigBackups(backups, { keepCount: 20, keepDays: 30, now: T0 + 40 * DAY }); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([T0 + 40 * DAY, T0]); + }); + + it('never deletes a folder it did not make', () => { + makeN(3); + mkdirSync(join(backups, 'photos')); + writeFileSync(join(backups, 'photos', 'a.jpg'), 'x'); + mkdirSync(join(backups, 'cfg-20200101-000000')); // right name, no manifest: not ours + writeFileSync(join(backups, 'notes.txt'), 'keep'); + pruneConfigBackups(backups, { keepCount: 1, keepDays: 1, now: T0 + 500 * DAY }); + expect(existsSync(join(backups, 'photos', 'a.jpg'))).toBe(true); + expect(existsSync(join(backups, 'cfg-20200101-000000'))).toBe(true); + expect(existsSync(join(backups, 'notes.txt'))).toBe(true); + expect(listConfigBackups(backups)).toHaveLength(1); + }); + + it('does nothing for a folder that does not exist', () => { + expect(pruneConfigBackups(join(root, 'nope'), { keepCount: 1, keepDays: 1 })).toEqual([]); + }); +}); + +describe('restoreConfigBackup', () => { + it('puts the files back, atomically, after taking a pre-restore snapshot', () => { + put('settings.json', '{"good":true}'); + put('clis.json', '{"clis":{"grok":{"enabled":true}}}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + + put('settings.json', '{"polluted":true}'); + rmSync(join(data, 'clis.json')); + + const r = restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id, now: T0 + 1000 }); + expect(r.restored.sort()).toEqual(['clis.json', 'settings.json']); + expect(readFileSync(join(data, 'settings.json'), 'utf8')).toBe('{"good":true}'); + expect(readFileSync(join(data, 'clis.json'), 'utf8')).toBe('{"clis":{"grok":{"enabled":true}}}'); + expect(statSync(join(data, 'settings.json')).mode & 0o777).toBe(0o600); + + // The state it replaced is recoverable. + expect(r.safetySnapshot).toBeTruthy(); + const safety = listConfigBackups(backups).find((b) => b.id === r.safetySnapshot)!; + expect(safety.reason).toBe('pre-restore'); + expect(readFileSync(join(backups, safety.id, 'settings.json'), 'utf8')).toBe('{"polluted":true}'); + }); + + it('refuses a snapshot whose file no longer matches its checksum, changing nothing', () => { + put('settings.json', '{"good":true}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + chmodSync(join(backups, made.id, 'settings.json'), 0o600); + writeFileSync(join(backups, made.id, 'settings.json'), '{"tampered":true}'); + put('settings.json', '{"live":true}'); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id })).toThrow( + ConfigBackupCorruptError + ); + expect(readFileSync(join(data, 'settings.json'), 'utf8')).toBe('{"live":true}'); + expect(listConfigBackups(backups)).toHaveLength(1); // no pre-restore snapshot for a refused restore + }); + + it('rejects unknown ids and anything that is not a snapshot name (no path traversal)', () => { + put('settings.json', '{}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + for (const id of ['cfg-19990101-000000', '../data', '..', 'photos', 'cfg-20261001-120000/../../x']) { + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id }), id).toThrow( + ConfigBackupNotFoundError + ); + } + }); + + it('reuses the newest backup as the undo copy when the live files already equal it', () => { + put('settings.json', '{"a":1}'); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + put('settings.json', '{"a":2}'); + const newest = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + DAY }) as { + id: string; + }; + const r = restoreConfigBackup({ dataDir: data, backupDir: backups, id: first.id, now: T0 + 2 * DAY }); + expect(r.safetySnapshot).toBe(newest.id); + expect(listConfigBackups(backups).map((b) => b.id)).toEqual([newest.id, first.id]); // no duplicate + expect(readFileSync(join(data, 'settings.json'), 'utf8')).toBe('{"a":1}'); + }); + + it('a file it cannot write does not stop the others, and the error names both lists', () => { + put('settings.json', '{"good":true}'); + put('clis.json', '{"clis":{}}'); + put('webhook.json', '{"enabled":false}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + put('settings.json', '{"polluted":true}'); + put('webhook.json', '{"enabled":true}'); + rmSync(join(data, 'clis.json')); + mkdirSync(join(data, 'clis.json')); // between the other two in the tracked list + let caught: unknown; + try { + restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id, now: T0 + 1000 }); + } catch (err) { + caught = err; + } + expect(caught).toBeInstanceOf(ConfigBackupPartialRestoreError); + const err = caught as ConfigBackupPartialRestoreError; + expect(err.restored).toEqual(['settings.json', 'webhook.json']); + expect(err.failed.map((f) => f.name)).toEqual(['clis.json']); + expect(err.safetySnapshot).toBeTruthy(); + expect(readFileSync(join(data, 'webhook.json'), 'utf8')).toBe('{"enabled":false}'); + expect(readdirSync(data).filter((n) => n.includes('.restore-'))).toEqual([]); // no temp left behind + }); + + it('only ever writes allowlisted names, even if a manifest lists others', () => { + put('settings.json', '{"a":1}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + const mpath = join(backups, made.id, 'manifest.json'); + const m = JSON.parse(readFileSync(mpath, 'utf8')); + m.files.push({ name: '../escape.json', bytes: 1, sha256: 'x' }, { name: 'state.json', bytes: 1, sha256: 'x' }); + writeFileSync(mpath, JSON.stringify(m)); + restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id }); + expect(existsSync(join(root, 'escape.json'))).toBe(false); + expect(existsSync(join(data, 'state.json'))).toBe(false); + }); +}); + +describe('diffConfigBackups', () => { + const snap = (now: number) => { + const r = createConfigBackup({ dataDir: data, backupDir: backups, reason: 't', force: true, now }); + if (r.status !== 'created') throw new Error('expected a snapshot'); + return r.id; + }; + const diff = (from: string, to: string) => diffConfigBackups({ dataDir: data, backupDir: backups, from, to }); + const file = (d: ReturnType, name: string) => d.files.find((f) => f.name === name)!; + + it('lists key-level changes between two backups', () => { + put('settings.json', JSON.stringify({ displayName: 'a', theme: { mode: 'dark' }, tags: ['x'] })); + const a = snap(T0); + put('settings.json', JSON.stringify({ displayName: 'b', theme: { mode: 'dark' }, tags: ['x', 'y'], extra: 1 })); + const b = snap(T0 + 1000); + const f = file(diff(a, b), 'settings.json'); + expect(f.status).toBe('changed'); + expect(f.changes).toEqual([ + { path: 'displayName', kind: 'changed', before: '"a"', after: '"b"' }, + { path: 'extra', kind: 'added', after: '1' }, + { path: 'tags[1]', kind: 'added', after: '"y"' }, + ]); + }); + + it('compares a backup with the running config, and reports files that appeared or vanished', () => { + put('settings.json', '{"a":1}'); + put('webhook.json', '{"url":"u"}'); + const a = snap(T0); + put('settings.json', '{"a":2}'); + rmSync(join(data, 'webhook.json')); + put('intents.json', '{"i":true}'); + const d = diff(a, 'current'); + expect(d.changed).toBe(3); + expect(file(d, 'settings.json').changes).toEqual([{ path: 'a', kind: 'changed', before: '1', after: '2' }]); + expect(file(d, 'webhook.json').status).toBe('removed'); + expect(file(d, 'intents.json').status).toBe('added'); + expect(diff(a, a).changed).toBe(0); + }); + + it('never returns a webhook URL, captured prompts, or URL userinfo and query', () => { + put('webhook.json', JSON.stringify({ url: 'https://hooks.slack.com/services/T0/B0/oldsecret' })); + put('intents.json', JSON.stringify({ profiles: [{ key: 'k', prompts: ['my password is pw1'] }] })); + put('settings.json', JSON.stringify({ baseUrl: 'https://user:pw@host/p?token=t1#f' })); + const a = snap(T0); + put('webhook.json', JSON.stringify({ url: 'https://hooks.slack.com/services/T0/B0/newsecret' })); + put('intents.json', JSON.stringify({ profiles: [{ key: 'k', prompts: ['my password is pw2'] }] })); + put('settings.json', JSON.stringify({ baseUrl: 'https://user:pw@host/p?token=t2#f' })); + const d = diff(a, 'current'); + const json = JSON.stringify(d); + for (const s of ['secret', 'pw1', 'pw2', 'user:pw', 'token=', 'hooks.slack']) expect(json).not.toContain(s); + expect(file(d, 'webhook.json').changes.every((c) => c.redacted)).toBe(true); + expect(file(d, 'intents.json').changes.every((c) => c.redacted)).toBe(true); + }); + + it('never returns a credential value, only that it differs', () => { + put('.env', 'SECRET_ONE=hunter2\nOTHER=abc\n'); + put('users.json', JSON.stringify({ admin: { name: 'root' } })); + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', apiKey: 'sk-old', url: 'http://x' }])); + const a = snap(T0); + put('.env', 'SECRET_ONE=hunter3\nOTHER=abd\n'); + put('users.json', JSON.stringify({ admin: { name: 'rooted' } })); + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', apiKey: 'sk-new', url: 'http://y' }])); + const d = diff(a, 'current'); + const json = JSON.stringify(d); + for (const secret of ['hunter', 'abc', 'abd', 'root', 'sk-old', 'sk-new']) expect(json).not.toContain(secret); + expect(file(d, '.env').changes.every((c) => c.redacted)).toBe(true); + expect(file(d, 'custom-model-hosts.json').changes).toEqual([ + { path: '[0].apiKey', kind: 'changed', redacted: true }, + { path: '[0].url', kind: 'changed', before: '"http://x"', after: '"http://y"' }, + ]); + }); + + it('does not count a self-refreshed discovery timestamp as a difference', () => { + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', lastDiscoveredAt: 1 }])); + const a = snap(T0); + put('custom-model-hosts.json', JSON.stringify([{ name: 'h', lastDiscoveredAt: 2 }])); + const d = diff(a, 'current'); + expect(file(d, 'custom-model-hosts.json').status).toBe('unchanged'); + expect(d.changed).toBe(0); + }); + + it('reports a file that is not JSON as differing, without its content', () => { + put('settings.json', 'not json at all'); + const a = snap(T0); + put('settings.json', 'still not json'); + const f = file(diff(a, 'current'), 'settings.json'); + expect(f).toMatchObject({ status: 'changed', detail: 'bytes', changes: [] }); + }); + + it('caps the changes listed per file', () => { + put('settings.json', '{}'); + const a = snap(T0); + put('settings.json', JSON.stringify(Object.fromEntries(Array.from({ length: 250 }, (_, i) => [`k${i}`, i])))); + const f = file(diff(a, 'current'), 'settings.json'); + expect(f.changes).toHaveLength(200); + expect(f.truncated).toBe(true); + }); + + it('refuses unknown ids, path traversal and a tampered snapshot', () => { + put('settings.json', '{"a":1}'); + const a = snap(T0); + expect(() => diff('cfg-20200101-000000', 'current')).toThrow(ConfigBackupNotFoundError); + expect(() => diff('../etc', 'current')).toThrow(ConfigBackupNotFoundError); + writeFileSync(join(backups, a, 'settings.json'), '{"a":99}'); + expect(() => diff(a, 'current')).toThrow(ConfigBackupCorruptError); + }); +}); + +describe('createConfigBackup: a snapshot dated ahead of the clock', () => { + it('is not written again on every pass once the clock is corrected', () => { + put('settings.json', '{"v":0}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put('settings.json', '{"v":1}'); + const ahead = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2 * 3_600_000 }); + expect(ahead.status).toBe('created'); + put('settings.json', '{"v":2}'); // the clock is back at T0 + 1 minute + const corrected = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 60_000 }); + expect(corrected.status).toBe('created'); + const [newest, second] = listConfigBackups(backups); + expect(newest.id).toBe((corrected as { id: string }).id); // dated no earlier than the one before it + expect(newest.createdAt).toBeGreaterThan(second.createdAt); + for (let i = 0; i < 24; i++) { + const r = createConfigBackup({ + dataDir: data, + backupDir: backups, + reason: 'auto', + now: T0 + 120_000 + i * 300_000, + }); + expect(r.status, `pass ${i}`).toBe('unchanged'); + } + expect(listConfigBackups(backups)).toHaveLength(3); + }); +}); + +describe('instances sharing one backup folder', () => { + it('keeps to its own snapshots: no churn, no prune, no restore of the other', () => { + const other = join(root, 'other-data'); + mkdirSync(other, { recursive: true }); + put('settings.json', '{"a":1}'); + writeFileSync(join(other, 'settings.json'), '{"b":2}'); + const mine = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + const theirs = createConfigBackup({ dataDir: other, backupDir: backups, reason: 'auto', now: T0 + 1000 }); + expect(theirs.status).toBe('created'); + // Each side's change check compares with its own newest snapshot, so a second tick is a no-op. + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'unchanged' + ); + expect(createConfigBackup({ dataDir: other, backupDir: backups, reason: 'auto', now: T0 + 3000 }).status).toBe( + 'unchanged' + ); + expect(listConfigBackups(backups, configBackupOrigin(data))).toHaveLength(1); + expect( + pruneConfigBackups(backups, { keepCount: 1, keepDays: 1, now: T0 + 999 * DAY, origin: configBackupOrigin(data) }) + ).toEqual([]); + expect(listConfigBackups(backups)).toHaveLength(2); + const theirId = (theirs as { id: string }).id; + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: theirId })).toThrow( + ConfigBackupNotFoundError + ); + expect((mine as { id: string }).id).not.toBe(theirId); + }); + + it('a restore reports tracked files it left alone, and sweeps only old temp folders', () => { + put('settings.json', '{"a":1}'); + const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + put('webhook.json', '{"url":"u"}'); + expect(restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id }).untouched).toEqual(['webhook.json']); + const stale = join(backups, '.tmp-cfg-20200101-000000-99999'); + mkdirSync(stale); + sweepConfigBackupTemps(backups, Date.now() + 2 * 3_600_000); + expect(existsSync(stale)).toBe(false); + }); +}); + +describe('restore trusts only what this module wrote', () => { + const made = () => + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }) as { id: string }; + + it('refuses a symlinked snapshot file and a snapshot file with group or world bits', () => { + put('settings.json', '{"a":1}'); + put('clis.json', '{"clis":{}}'); + const { id } = made(); + const file = join(backups, id, 'clis.json'); + chmodSync(file, 0o644); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).toThrow(ConfigBackupCorruptError); + + chmodSync(file, 0o600); + const real = join(root, 'elsewhere.json'); + writeFileSync(real, '{"clis":{}}'); + rmSync(file); + symlinkSync(real, file); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).toThrow(/symlink/); + // nothing was written and no pre-restore copy was taken for a refused restore + expect(listConfigBackups(backups)).toHaveLength(1); + }); + + it('skips the group/world bit test on a filesystem that does not keep modes, but never the symlink test', () => { + put('settings.json', '{"a":1}'); + put('clis.json', '{"clis":{}}'); + const { id } = made(); + const file = join(backups, id, 'clis.json'); + chmodSync(file, 0o644); // what vfat, CIFS and WSL /mnt/d report for every file + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id, keepsModes: false })).not.toThrow(); + + const real = join(root, 'elsewhere.json'); + writeFileSync(real, '{"clis":{}}'); + rmSync(file); + symlinkSync(real, file); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id, keepsModes: false })).toThrow(/symlink/); + }); + + it('the mode probe never writes through a planted symlink or trips over a planted directory', () => { + put('settings.json', '{"a":1}'); + const { id } = made(); + const target = join(root, 'victim.txt'); + writeFileSync(target, 'precious', { mode: 0o644 }); + symlinkSync(target, join(backups, `.mode-probe-${process.pid}`)); + mkdirSync(join(backups, '.mode-probe-0')); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).not.toThrow(); + expect(() => diffConfigBackups({ dataDir: data, backupDir: backups, from: id, to: 'current' })).not.toThrow(); + expect(readFileSync(target, 'utf8')).toBe('precious'); + expect(statSync(target).mode & 0o777).toBe(0o644); + }); + + it('compare applies the same check to a snapshot file', () => { + put('settings.json', '{"a":1}'); + const { id } = made(); + chmodSync(join(backups, id, 'settings.json'), 0o644); + expect(() => diffConfigBackups({ dataDir: data, backupDir: backups, from: id, to: 'current' })).toThrow( + ConfigBackupCorruptError + ); + }); + + it('refuses a snapshot folder that others can write to', () => { + put('settings.json', '{"a":1}'); + const { id } = made(); + chmodSync(join(backups, id), 0o777); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).toThrow(ConfigBackupCorruptError); + chmodSync(join(backups, id), 0o700); + expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id })).not.toThrow(); + }); + + it('keeps the backup retention when the live settings.json was reset and has none of the four keys', () => { + put('settings.json', JSON.stringify({ displayName: 'old', configBackupKeepCount: 100, configBackupKeepDays: 0 })); + const { id } = made(); + put('settings.json', '{}'); + restoreConfigBackup({ dataDir: data, backupDir: backups, id }); + expect(JSON.parse(readFileSync(join(data, 'settings.json'), 'utf8'))).toEqual({ + displayName: 'old', + configBackupKeepCount: 100, + configBackupKeepDays: 0, + }); + }); + + it('latestBackupRetention skips a newer snapshot that recorded none, and a pre-restore copy records it', () => { + put('settings.json', '{"a":1}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0, keepCount: 100, keepDays: 0 }); + put('settings.json', '{"a":2}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 }); // an older build + const origin = configBackupOrigin(data); + expect(latestBackupRetention(backups, origin)).toEqual({ keepCount: 100, keepDays: 0 }); + const id = listConfigBackups(backups, origin).at(-1)!.id; + // A change since the newest backup, so the restore makes a pre-restore copy rather than reusing it. + put('settings.json', '{"a":3}'); + const r = restoreConfigBackup({ dataDir: data, backupDir: backups, id, now: T0 + 2000 }); + expect(r.safetySnapshot).toBeTruthy(); + const pre = listConfigBackups(backups, origin).find((b) => b.id === r.safetySnapshot)!; + expect(pre.reason).toBe('pre-restore'); + expect(JSON.parse(readFileSync(join(backups, pre.id, 'manifest.json'), 'utf8'))).toMatchObject({ + keepCount: 100, + keepDays: 0, + }); + }); + + it('keeps the current folder and retention when settings.json is restored', () => { + put('settings.json', JSON.stringify({ displayName: 'old', configBackupDir: '/old', configBackupKeepCount: 5 })); + const { id } = made(); + put('settings.json', JSON.stringify({ displayName: 'new', configBackupDir: '/new', configBackupKeepDays: 7 })); + restoreConfigBackup({ dataDir: data, backupDir: backups, id }); + const after = JSON.parse(readFileSync(join(data, 'settings.json'), 'utf8')); + // the folder and days follow the live file; the live file has no keep count (reset), so the backup's 5 stays + expect(after).toEqual({ + displayName: 'old', + configBackupDir: '/new', + configBackupKeepCount: 5, + configBackupKeepDays: 7, + }); + }); +}); diff --git a/test/routes/config-backup-one-walk.test.ts b/test/routes/config-backup-one-walk.test.ts new file mode 100644 index 00000000..2a690a41 --- /dev/null +++ b/test/routes/config-backup-one-walk.test.ts @@ -0,0 +1,94 @@ +/** + * @fileoverview A backup pass reads the backup folder once: one directory listing and one read per + * manifest, shared by the recorded-retention fallback, the change check and the prune. Each used to + * list the folder itself, which on a network folder meant every manifest read three times every five + * minutes (found in review of #661). + * + * ⚠️ test/setup.ts gives the whole FILE one temp HOME. Port: N/A (no server). + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { mkdirSync, rmSync, writeFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; + +const rec = vi.hoisted(() => ({ dir: '', calls: [] as string[] })); + +vi.mock('node:fs', async (importOriginal) => { + const actual = await importOriginal(); + const record = unknown>(name: string, real: F): F => + ((path: unknown, ...rest: unknown[]) => { + if (rec.dir && String(path).startsWith(rec.dir)) rec.calls.push(`${name} ${String(path)}`); + return (real as unknown as (...a: unknown[]) => unknown)(path, ...rest); + }) as unknown as F; + const wrapped = { + ...actual, + readdirSync: record('readdirSync', actual.readdirSync), + readFileSync: record('readFileSync', actual.readFileSync), + }; + return { ...wrapped, default: wrapped }; +}); + +import { runConfigBackup } from '../../src/web/routes/config-backup-routes.js'; +import { SETTINGS_PATH } from '../../src/web/route-helpers.js'; +import { getDataDir } from '../../src/config/instance.js'; +import { createConfigBackup, defaultConfigBackupDir, listConfigBackups } from '../../src/config-backup.js'; + +const dir = () => defaultConfigBackupDir(getDataDir()); +const DAY = 86_400_000; +const BASE = Date.UTC(2025, 0, 1); + +beforeEach(() => { + rec.dir = ''; + rec.calls.length = 0; + rmSync(dir(), { recursive: true, force: true }); + mkdirSync(dirname(SETTINGS_PATH), { recursive: true }); +}); +afterEach(() => { + rec.dir = ''; + rmSync(dir(), { recursive: true, force: true }); + rmSync(SETTINGS_PATH, { force: true }); +}); + +describe('one backup pass', () => { + it('lists the folder once and reads each manifest once', async () => { + // Only the OLDEST snapshot recorded a retention, so the fallback has to look through every one. + for (let i = 0; i < 6; i++) { + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: `v${i}` })); + createConfigBackup({ + dataDir: getDataDir(), + backupDir: dir(), + reason: 'auto', + now: BASE + i * DAY, + ...(i === 0 && { keepCount: 100, keepDays: 0 }), + }); + } + // A change (so the pass writes a snapshot and then prunes) and no retention keys (the fallback reads). + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: 'changed' })); + rec.dir = dir(); + const out = await runConfigBackup('auto'); + rec.dir = ''; + + expect(out?.result.status).toBe('created'); + expect(out?.removed).toEqual([]); // the recorded 100 / no age limit, not the defaults + expect(rec.calls.filter((c) => c.startsWith('readdirSync'))).toHaveLength(1); + const manifestReads = rec.calls.filter((c) => c.startsWith('readFileSync') && c.endsWith('manifest.json')); + expect(manifestReads).toHaveLength(6); + expect(new Set(manifestReads).size).toBe(6); + expect(listConfigBackups(dir())).toHaveLength(7); + }); + + it('counts the snapshot it just made when it prunes', async () => { + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepCount: 2, configBackupKeepDays: 0, v: 0 })); + for (let i = 0; i < 3; i++) { + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepCount: 2, configBackupKeepDays: 0, v: i })); + createConfigBackup({ dataDir: getDataDir(), backupDir: dir(), reason: 'auto', now: BASE + i * DAY }); + } + const before = listConfigBackups(dir()).map((b) => b.id); + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepCount: 2, configBackupKeepDays: 0, v: 9 })); + const out = await runConfigBackup('auto'); + expect(out?.result.status).toBe('created'); + const kept = listConfigBackups(dir()).map((b) => b.id); + // keep 2: the new one and the newest before it; the two older ones go + expect(kept).toEqual([(out?.result as { id: string }).id, before[0]]); + expect(out?.removed.sort()).toEqual([before[1], before[2]].sort()); + }); +}); diff --git a/test/routes/config-backup-routes.test.ts b/test/routes/config-backup-routes.test.ts new file mode 100644 index 00000000..3eb25547 --- /dev/null +++ b/test/routes/config-backup-routes.test.ts @@ -0,0 +1,330 @@ +/** + * @fileoverview /api/config-backups: list, back up now, restore. Settings are read fresh from + * settings.json, so a folder or retention change applies on the next request. + * + * ⚠️ test/setup.ts gives the whole FILE one temp HOME (and so one data dir). Port: N/A (inject()). + */ +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { createRouteTestHarness } from './_route-test-utils.js'; +import { registerConfigBackupRoutes, runConfigBackup } from '../../src/web/routes/config-backup-routes.js'; +import { SETTINGS_PATH } from '../../src/web/route-helpers.js'; +import { getDataDir } from '../../src/config/instance.js'; +import { createConfigBackup, defaultConfigBackupDir, listConfigBackups } from '../../src/config-backup.js'; +import { intentStore } from '../../src/intent-store.js'; + +let custom: string; +const settings = (extra: Record = {}) => { + mkdirSync(dirname(SETTINGS_PATH), { recursive: true }); + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: 'before', ...extra })); +}; + +beforeEach(() => { + custom = mkdtempSync(join(tmpdir(), 'cfg-backup-route-')); + rmSync(defaultConfigBackupDir(getDataDir()), { recursive: true, force: true }); + settings(); +}); +afterEach(() => { + delete process.env.CODEMAN_MULTIUSER; + rmSync(custom, { recursive: true, force: true }); + rmSync(SETTINGS_PATH, { force: true }); + rmSync(defaultConfigBackupDir(getDataDir()), { recursive: true, force: true }); +}); + +describe('/api/config-backups', () => { + it('lists the effective settings, the default folder and the tracked files', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method: 'GET', url: '/api/config-backups' }); + expect(res.statusCode).toBe(200); + const d = res.json().data; + expect(d.settings).toEqual({ enabled: true, dir: null, keepCount: 20, keepDays: 30 }); + expect(d.dir).toBe(defaultConfigBackupDir(getDataDir())); + expect(d.tracked).toContain('settings.json'); + expect(d.backups).toEqual([]); + }); + + it('backs up now into the default folder and lists it', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const made = await app.inject({ method: 'POST', url: '/api/config-backups' }); + expect(made.statusCode).toBe(200); + expect(made.json().data.status).toBe('created'); + const list = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data; + expect(list.backups).toHaveLength(1); + expect(list.backups[0].reason).toBe('manual'); + expect(list.backups[0].files.map((f: { name: string }) => f.name)).toContain('settings.json'); + // Names and sizes only, never content. + expect(JSON.stringify(list)).not.toContain('before'); + }); + + it('follows a custom folder and retention from settings on the next request', async () => { + const dir = join(custom, 'snap'); + settings({ configBackupDir: dir, configBackupKeepCount: 2, configBackupKeepDays: 0 }); + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + for (let i = 0; i < 4; i++) { + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupDir: dir, configBackupKeepCount: 2, i })); + await app.inject({ method: 'POST', url: '/api/config-backups' }); + await new Promise((r) => setTimeout(r, 5)); + } + const d = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data; + expect(d.dir).toBe(dir); + expect(d.backups).toHaveLength(2); + expect(existsSync(defaultConfigBackupDir(getDataDir()))).toBe(false); + }); + + it('restores a backup, reports what needs a restart and keeps a pre-restore copy', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + writeFileSync(join(dirname(SETTINGS_PATH), 'clis.json'), '{"schemaVersion":1,"clis":{}}', { mode: 0o600 }); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: 'polluted' })); + writeFileSync(join(dirname(SETTINGS_PATH), 'clis.json'), '{"clis":{"evil":{}}}'); + + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + expect(res.statusCode).toBe(200); + const d = res.json().data; + expect(d.restored).toEqual(expect.arrayContaining(['settings.json', 'clis.json'])); + expect(d.restartRequired).not.toContain('clis.json'); // reloaded by the restore itself + expect(d.restartRequired).toContain('settings.json'); // its watcher toggles are not re-run by a restore + expect(d.safetySnapshot).toBeTruthy(); + expect(JSON.parse(readFileSync(SETTINGS_PATH, 'utf8')).displayName).toBe('before'); + }); + + it('a restore that fails partway names what it restored, what failed and the pre-restore copy', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const clis = join(dirname(SETTINGS_PATH), 'clis.json'); + writeFileSync(clis, '{"schemaVersion":1,"clis":{}}', { mode: 0o600 }); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + writeFileSync(SETTINGS_PATH, JSON.stringify({ displayName: 'polluted' })); + // A file the server cannot replace: settings.json (first in the list) goes back, clis.json cannot. + rmSync(clis); + mkdirSync(clis); + try { + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + expect(res.statusCode).toBe(500); + const body = res.json(); + expect(body).toMatchObject({ success: false, errorCode: 'OPERATION_FAILED' }); + expect(body.error).toContain('clis.json'); + expect(body.data.restored).toContain('settings.json'); + expect(body.data.failed.map((f: { name: string }) => f.name)).toEqual(['clis.json']); + expect(body.data.restartRequired).toContain('settings.json'); + expect(JSON.parse(readFileSync(SETTINGS_PATH, 'utf8')).displayName).toBe('before'); + // The undo copy exists, holds the state the restore replaced, and is the one named. + const safety = body.data.safetySnapshot; + expect(safety).toBeTruthy(); + expect(body.error).toContain(safety); + const listed = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.backups; + expect(listed.find((b: { id: string }) => b.id === safety)?.reason).toBe('pre-restore'); + } finally { + rmSync(clis, { recursive: true, force: true }); + } + }); + + it('a restored intents.json is not reverted by the next captured prompt', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const dir = mkdtempSync(join(tmpdir(), 'cfg-backup-intent-')); + try { + intentStore.recordPrompt(undefined, dir, 's1', 'first prompt about the build pipeline'); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + intentStore.recordPrompt(undefined, dir, 's1', 'second prompt about the build pipeline'); + await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + intentStore.recordPrompt(undefined, dir, 's1', 'third prompt about the build pipeline'); + expect(intentStore.getProfile(undefined, dir).recentPrompts).toHaveLength(2); + } finally { + intentStore.deleteProfile(undefined, dir); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('refuses a custom folder whose parent is missing instead of creating it', async () => { + settings({ configBackupDir: join(custom, 'no', 'such', 'dir') }); + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method: 'POST', url: '/api/config-backups' }); + expect(res.statusCode).toBe(503); + expect(existsSync(join(custom, 'no'))).toBe(false); + }); + + it('answers 404 for an unknown backup and 404 for a traversal attempt', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + for (const id of ['cfg-19990101-000000', '..%2F..%2Fetc', 'photos']) { + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + expect(res.statusCode, id).toBe(404); + } + }); + + it('is admin only in multi-user mode: a non-admin is refused on every verb', async () => { + process.env.CODEMAN_MULTIUSER = '1'; + const { app } = await createRouteTestHarness(registerConfigBackupRoutes, { + authUser: { username: 'bob', role: 'user' }, + }); + for (const [method, url] of [ + ['GET', '/api/config-backups'], + ['POST', '/api/config-backups'], + ['POST', '/api/config-backups/cfg-20261001-120000/restore'], + ['GET', '/api/config-backups/diff?from=cfg-20261001-120000'], + ] as const) { + expect((await app.inject({ method, url })).statusCode, `${method} ${url}`).toBe(403); + } + }); + + it('multi-user: a restore is written to the admin audit log', async () => { + process.env.CODEMAN_MULTIUSER = '1'; + const { app } = await createRouteTestHarness(registerConfigBackupRoutes, { + authUser: { username: 'root', role: 'admin' }, + }); + const id = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data.id; + await app.inject({ method: 'POST', url: `/api/config-backups/${id}/restore` }); + const auditPath = join(getDataDir(), 'admin-audit.jsonl'); + const written = () => existsSync(auditPath) && readFileSync(auditPath, 'utf8').includes('config_backup_restore'); + for (let i = 0; i < 40 && !written(); i++) await new Promise((r) => setTimeout(r, 25)); + const last = JSON.parse(readFileSync(auditPath, 'utf8').trim().split('\n').pop()!); + expect(last).toMatchObject({ admin: 'root', action: 'config_backup_restore', target: id }); + expect(last.detail.restored).toContain('settings.json'); + rmSync(auditPath, { force: true }); + }); + + it('multi-user: an admin is allowed', async () => { + process.env.CODEMAN_MULTIUSER = '1'; + const { app } = await createRouteTestHarness(registerConfigBackupRoutes, { + authUser: { username: 'root', role: 'admin' }, + }); + expect((await app.inject({ method: 'GET', url: '/api/config-backups' })).json().success).toBe(true); + }); +}); + +describe('GET /api/config-backups/diff', () => { + it('compares a backup with the running config, defaulting `to` to current', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const made = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data; + settings({ displayName: 'after' }); + const res = await app.inject({ method: 'GET', url: `/api/config-backups/diff?from=${made.id}` }); + expect(res.statusCode).toBe(200); + const d = res.json().data; + expect(d.to).toBe('current'); + expect(d.files.find((f: { name: string }) => f.name === 'settings.json').changes).toEqual([ + { path: 'displayName', kind: 'changed', before: '"before"', after: '"after"' }, + ]); + }); + + it('400 without `from`, 404 for an unknown backup', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + expect((await app.inject({ method: 'GET', url: '/api/config-backups/diff' })).statusCode).toBe(400); + const missing = await app.inject({ method: 'GET', url: '/api/config-backups/diff?from=cfg-20200101-000000' }); + expect(missing.statusCode).toBe(404); + }); +}); + +describe('retention survives an unreadable or reset settings.json', () => { + const DAY = 86_400_000; + const BASE = Date.UTC(2025, 0, 1); + const dir = () => defaultConfigBackupDir(getDataDir()); + /** 30 snapshots a day apart, made under a long-history retention (100 copies, no age limit). */ + function seed(withKeys = true) { + settings({ configBackupKeepCount: 100, configBackupKeepDays: 0 }); + for (let i = 0; i < 30; i++) { + writeFileSync( + SETTINGS_PATH, + JSON.stringify( + withKeys + ? { displayName: `v${i}`, configBackupKeepCount: 100, configBackupKeepDays: 0 } + : { displayName: `v${i}` } + ) + ); + createConfigBackup({ + dataDir: getDataDir(), + backupDir: dir(), + reason: 'auto', + now: BASE + i * DAY, + keepCount: 100, + keepDays: 0, + }); + } + expect(listConfigBackups(dir())).toHaveLength(30); + } + + it('a settings.json that does not parse still gets a snapshot but prunes nothing', async () => { + seed(); + writeFileSync(SETTINGS_PATH, '{ "displayName": "half-written", }'); + const out = await runConfigBackup('auto'); + expect(out?.result.status).toBe('created'); + expect(out?.removed).toEqual([]); + expect(listConfigBackups(dir())).toHaveLength(31); + }); + + it('a reset settings.json prunes with the retention the newest snapshot recorded, not the defaults', async () => { + seed(); + writeFileSync(SETTINGS_PATH, '{}'); + const out = await runConfigBackup('auto'); + expect(out?.result.status).toBe('created'); + // aged manifests (a year old) and keepDays 0: the defaults (20 copies, 30 days) would leave 20 + expect(listConfigBackups(dir())).toHaveLength(31); + }); + + it.each([true, false])( + 'a restore after a reset keeps the retention (the backup %s the four keys)', + async (withKeys) => { + seed(withKeys); + const oldest = listConfigBackups(dir()).at(-1)!.id; + writeFileSync(SETTINGS_PATH, '{}'); // the reset + expect((await runConfigBackup('auto'))?.removed).toEqual([]); + + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method: 'POST', url: `/api/config-backups/${oldest}/restore` }); + expect(res.statusCode).toBe(200); + // the restored file carries the retention, or the recorded one still applies via the pre-restore copy + const out = await runConfigBackup('auto'); + expect(out?.removed).toEqual([]); + expect(listConfigBackups(dir()).length).toBeGreaterThanOrEqual(32); + const status = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.settings; + expect(status).toMatchObject({ keepCount: 100, keepDays: 0 }); // what the form shows is what the pass uses + } + ); + + it("re-entering a custom folder after a reset finds that folder's recorded retention", async () => { + const folder = join(custom, 'kept'); + settings({ configBackupDir: folder, configBackupKeepCount: 100, configBackupKeepDays: 0 }); + for (let i = 0; i < 30; i++) { + createConfigBackup({ + dataDir: getDataDir(), + backupDir: folder, + reason: 'auto', + force: true, + now: BASE + i * DAY, + keepCount: 100, + keepDays: 0, + }); + } + writeFileSync(SETTINGS_PATH, '{}'); // the reset drops the folder too + await runConfigBackup('auto'); // goes to the default folder, which holds none of those manifests + // the user enters the folder again and saves; the form no longer sends retention it only displayed + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupDir: folder, configBackupEnabled: true })); + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const reported = (await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.settings; + expect(reported).toMatchObject({ keepCount: 100, keepDays: 0 }); + const out = await runConfigBackup('auto'); + expect(out?.removed).toEqual([]); + expect(listConfigBackups(folder).length).toBeGreaterThanOrEqual(30); + }); + + it('only the retention key the file lacks falls back to the recorded one', async () => { + seed(); + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepDays: 3650 })); + await runConfigBackup('auto'); + expect(listConfigBackups(dir())).toHaveLength(31); // keepCount 100 from the manifests, not the default 20 + }); + + it('a settings.json that sets the retention still wins', async () => { + seed(); + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupKeepCount: 5, configBackupKeepDays: 0 })); + await runConfigBackup('auto'); + expect(listConfigBackups(dir())).toHaveLength(5); + }); + + it('listing never creates config-backup-id', async () => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const idFile = join(getDataDir(), 'config-backup-id'); + rmSync(idFile, { force: true }); + expect((await app.inject({ method: 'GET', url: '/api/config-backups' })).json().data.backups).toEqual([]); + expect(existsSync(idFile)).toBe(false); + }); +}); diff --git a/test/routes/config-backup-unreachable.test.ts b/test/routes/config-backup-unreachable.test.ts new file mode 100644 index 00000000..97afb391 --- /dev/null +++ b/test/routes/config-backup-unreachable.test.ts @@ -0,0 +1,78 @@ +/** + * @fileoverview A custom backup folder that stops answering (a network mount) must never be listed or + * read before the bounded probe says it is there: GET, Back up now, diff and restore answer 503 without + * touching it. + * With only `configBackupDir` in settings.json the recorded-retention fallback would otherwise + * `readdirSync` the folder first (found in review of #661). + * + * ⚠️ test/setup.ts gives the whole FILE one temp HOME. Port: N/A (inject()). + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; + +const dead = vi.hoisted(() => ({ dir: '', touched: [] as string[] })); + +vi.mock('node:fs', async (importOriginal) => { + const actual = await importOriginal(); + const record = unknown>(name: string, real: F): F => + ((path: unknown, ...rest: unknown[]) => { + if (dead.dir && String(path).startsWith(dead.dir)) dead.touched.push(`${name} ${String(path)}`); + return (real as unknown as (...a: unknown[]) => unknown)(path, ...rest); + }) as unknown as F; + const wrapped = { + ...actual, + readdirSync: record('readdirSync', actual.readdirSync), + readFileSync: record('readFileSync', actual.readFileSync), + statSync: record('statSync', actual.statSync), + lstatSync: record('lstatSync', actual.lstatSync), + existsSync: record('existsSync', actual.existsSync), + }; + return { ...wrapped, default: wrapped }; +}); + +vi.mock('../../src/utils/bounded-path-probe.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + probePathKind: async (path: string, options?: { pastCap?: boolean }) => + dead.dir && path.startsWith(dead.dir) ? 'unknown' : actual.probePathKind(path, options), + }; +}); + +import { createRouteTestHarness } from './_route-test-utils.js'; +import { registerConfigBackupRoutes } from '../../src/web/routes/config-backup-routes.js'; +import { SETTINGS_PATH } from '../../src/web/route-helpers.js'; +import { getDataDir } from '../../src/config/instance.js'; +import { configBackupOrigin } from '../../src/config-backup.js'; + +let root: string; +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'cfg-backup-dead-')); + dead.dir = join(root, 'mount'); + dead.touched.length = 0; + configBackupOrigin(getDataDir()); // an instance that already has an id: the fallback has something to look for + mkdirSync(dirname(SETTINGS_PATH), { recursive: true }); + // Only the folder: no retention keys, the shape that makes the recorded-retention fallback want to read it. + writeFileSync(SETTINGS_PATH, JSON.stringify({ configBackupDir: dead.dir })); +}); +afterEach(() => { + dead.dir = ''; + rmSync(root, { recursive: true, force: true }); + rmSync(SETTINGS_PATH, { force: true }); +}); + +describe('a custom backup folder that does not answer', () => { + it.each([ + ['GET', '/api/config-backups'], + ['POST', '/api/config-backups'], + ['GET', '/api/config-backups/diff?from=cfg-20261001-120000'], + ['POST', '/api/config-backups/cfg-20261001-120000/restore'], + ] as const)('%s %s answers 503 without reading the folder', async (method, url) => { + const { app } = await createRouteTestHarness(registerConfigBackupRoutes); + const res = await app.inject({ method, url }); + expect(res.statusCode).toBe(503); + expect(dead.touched).toEqual([]); + }); +});