mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-18 12:19:33 +01:00
fix(config-backup): origin is a stored random id, Settings refills after a settings restore, clis.json reloads
Review round 2 on #661: the hostname in the origin orphaned every snapshot when a Docker container was recreated, so the origin is a random id kept in <data dir>/config-backup-id; after restoring settings.json the Settings form is re-read from /api/settings so Save cannot write the old values back; the restore confirm names the files and the credential ones; clis.json is reloaded (reloadCliRegistry) and users.json needs no restart; API reference and route docs corrected.
This commit is contained in:
1 parent
247740fc77
commit
8b27e4beb6
10 files changed
+186
-23
No files matched your search
@@ -156,7 +156,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Auto-resume on usage limit** (opt-in per session, top of the Respawn tab): when Claude halts on a subscription limit, `usage-limit-patterns.ts` (pure, unit-tested) parses the reset time and `SessionAutoOps` arms a timer for reset+2min, then sends Esc + `continue`. ⚠️ Respawn cycles are blocked while paused (`isLimitPaused` guard in `onIdleDetected`), which is what prevents `/clear` from wiping the paused conversation. Claude-mode only. → [architecture-invariants#auto-resume-on-usage-limit](docs/architecture-invariants.md#auto-resume-on-usage-limit)
|
||||
|
||||
**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, `configBackupDir`, `configBackupKeepCount` 1-500 default 20, `configBackupKeepDays` 0-3650 default 30; `docs/wiki/Config-Backups.md`): timestamped snapshots `cfg-YYYYMMDD-HHMMSS/` (+ `manifest.json` with SHA-256s, `reason`, `appVersion`) of a FIXED allowlist (`CONFIG_BACKUP_FILES`: settings, clis, custom-model-hosts, webhook, intents, linked-cases, remote/docker hosts+cases (incl. `remote-cases.json`), webviews, push-keys, users, `.env`) into `<data dir>/backups/config` or a custom absolute folder. ⚠️ `state.json` is deliberately NOT tracked (rewritten per session event; it also holds cron jobs and respawn presets). One snapshot per real content change (digest of the file set vs the latest manifest), at startup + a 5-minute timer, skipped in test mode. ⚠️ Retention (`pruneConfigBackups`) only deletes folders matching the name pattern AND holding a parseable manifest, and never the newest; a custom folder equal to the data dir falls back to the default. ⚠️ Restore verifies every file's SHA-256 BEFORE writing, takes a `pre-restore` snapshot, writes atomically at 0600 and only allowlisted names; the id is validated against the name pattern (no traversal). Snapshots are 0700/0600 because `.env`/`users.json`/`custom-model-hosts.json` carry credentials; all routes are admin-only in multi-user mode. ⚠️ A custom folder is probed with `probePathKind(dir, { pastCap: true })` before every pass and in each route (#516: it may be a network mount; `unknown` skips the pass / answers 503 `OPERATION_FAILED`), and a missing custom folder is only created when its parent is present. ⚠️ Each manifest records an `origin` (hostname + data dir); change check, prune, list, restore and diff are scoped to it so instances sharing one folder do not snapshot or prune each other. Restoring `intents.json` calls `intentStore.reload()`; `settings.json` applies its watcher toggles after a restart. → [docs/wiki/Config-Backups.md](docs/wiki/Config-Backups.md)
|
||||
**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, `configBackupDir`, `configBackupKeepCount` 1-500 default 20, `configBackupKeepDays` 0-3650 default 30; `docs/wiki/Config-Backups.md`): timestamped snapshots `cfg-YYYYMMDD-HHMMSS/` (+ `manifest.json` with SHA-256s, `reason`, `appVersion`) of a FIXED allowlist (`CONFIG_BACKUP_FILES`: settings, clis, custom-model-hosts, webhook, intents, linked-cases, remote/docker hosts+cases (incl. `remote-cases.json`), webviews, push-keys, users, `.env`) into `<data dir>/backups/config` or a custom absolute folder. ⚠️ `state.json` is deliberately NOT tracked (rewritten per session event; it also holds cron jobs and respawn presets). One snapshot per real content change (digest of the file set vs the latest manifest), at startup + a 5-minute timer, skipped in test mode. ⚠️ Retention (`pruneConfigBackups`) only deletes folders matching the name pattern AND holding a parseable manifest, and never the newest; a custom folder equal to the data dir falls back to the default. ⚠️ Restore verifies every file's SHA-256 BEFORE writing, takes a `pre-restore` snapshot, writes atomically at 0600 and only allowlisted names; the id is validated against the name pattern (no traversal). Snapshots are 0700/0600 because `.env`/`users.json`/`custom-model-hosts.json` carry credentials; all routes are admin-only in multi-user mode. ⚠️ A custom folder is probed with `probePathKind(dir, { pastCap: true })` before every pass and in each route (#516: it may be a network mount; `unknown` skips the pass / answers 503 `OPERATION_FAILED`), and a missing custom folder is only created when its parent is present. ⚠️ Each manifest records an `origin`, a random id kept in `<data dir>/config-backup-id` (NOT the hostname: a Docker container gets a new one per recreate, which orphaned every snapshot); change check, prune, list, restore and diff are scoped to it so instances sharing one folder do not snapshot or prune each other. Restoring `intents.json` calls `intentStore.reload()`; `settings.json` applies its watcher toggles after a restart; Settings re-reads `/api/settings` into the form after a restore of it, since Save would otherwise PUT the old values back. ⚠️ Restoring `clis.json` writes it directly (the one writer besides `mutateRegistryFile()`) and then calls `reloadCliRegistry()`. → [docs/wiki/Config-Backups.md](docs/wiki/Config-Backups.md)
|
||||
|
||||
**Plan-usage chip** (`showPlanUsageLimits`, per-device: desktop default **ON**, handhelds OFF): resolve DISPLAY only through `planUsageChipEnabled()` (settings-ui.js). The same setting is the server-side COLLECTION switch, read fresh by `readPlanUsageTelemetryEnabled()` (hooks-config.ts) at every claude create/respawn. ⚠️ An ABSENT key reads as ON in the reader; `GET /api/settings` must never write. ⚠️ A save sends `showPlanUsageLimits` ONLY when it flips the chip on that device (`planUsageCollectionFlip()`), or a phone switches collection off for every desktop. Claude data comes from the statusLine exporter, injected as an EPHEMERAL `claude --settings` flag (`resolveStatusLineCliCommand`), never written to disk, WRAPPING a user's own statusLine, posting to `POST /api/status-telemetry`. Codex comes from a read-only `account/rateLimits/read` poll (main bucket only). The statusline's `prompt_cache` object (Claude Code v2.1.251+) is kept by `StatusTelemetrySchema` (every documented scalar field, null-tolerant; `parsePromptCache()` in usage-telemetry.ts normalizes only the stored subset) and shown in the footer as `cache:until HH:MM` / `cache:cold` (only when the user has no statusline of their own) and in Session Options → Respawn (`#promptCacheStatus`, fed by `Session.promptCache` in `toState()` plus the change-only `session:promptCache` SSE event, worded by the pure `CodemanPromptCache.format`): warm-until/cold, the re-cache stake, the hit rate, the last miss's causes, and a `Context compacted at HH:MM` note after a compaction or a tool-result clearing (stamped on the first warm report whose `recache_tokens_if_cold` is null after one that carried it, never from `expected_rebuilds`, which Claude Code bumps only at the next request; NEVER an alert). ⚠️ `Session.setPromptCache()` stamps `compactedAt` onto a NEW object (never mutating the parsed report) and the route broadcasts the STORED object; the last report is restored on restart (constructor `promptCache`, like `displayModel`) and the readout is time-aware (a passed `expiresAt` reads cold, not stale-warm). It is the source of truth for the cache TTL, never the plan tier. → [architecture-invariants#plan-usage-chip-statusline-telemetry](docs/architecture-invariants.md#plan-usage-chip-statusline-telemetry), `docs/usage-limits-display-plan.md`
|
||||
|
||||
@@ -172,7 +172,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Docker Compose deployment** (`docker/`): Codeman runs in a container and spawns Docker cases as **SIBLING** containers via the host socket, never nested. `resolveDockerDaemonMountSource()` maps HOME bind sources into the daemon's namespace (`CODEMAN_DOCKER_HOST_HOME`); `CODEMAN_CASES_PATH` makes workspaces resolve to the same absolute path on both sides. ⚠️ `CODEMAN_CASES_PATH` must move every consumer: resolve it only via `config/cases-dir.ts`. ⚠️ `.dockerignore` matches whole paths: keep `**/.env` or `docker/.env` secrets ship in the image. ⚠️ Long-form binds create missing sources ROOT-OWNED: `Start-Codeman.sh` pre-creates them, and `docker/entrypoint.sh` (root, `cap_add: [CHOWN, DAC_OVERRIDE, KILL, SETGID, SETUID]` against `cap_drop: ALL`, `KILL` for tini; pinned by the test) fixes ownership then drops to `PUID:PGID` via `setpriv`, never re-owning foreign dirs. ⚠️ Append `/opt/codeman-cli` and `~/.local/bin` to `PATH`, never prepend. ⚠️ `server.Dockerfile`, the compose file and `.env.example` feed the self-updater's environment gate (`docs/docker-self-update.md`). → [architecture-invariants#docker-compose-deployment](docs/architecture-invariants.md#docker-compose-deployment), `docs/docker-compose.md`
|
||||
|
||||
**CLI registry** (`src/config/cli-registry/`): every run mode is a `CliEntry` (discovery, launch argv template, env handling, `capabilities`, and the `overlays` behind remote/docker pane commands). **No code outside `stock.ts` may branch on a CLI id**: use a capability field or a NAMED PROFILE (`profiles.ts`); `test/cli-registry-no-id-branching.test.ts` and `test/frontend-cli-no-id-branching.test.ts` enforce it. ⚠️ Config holds typed argv tokens, never shell text; literals are validated at LOAD time and a bad one rejects the whole entry. ⚠️ Keep `external`, `hooks` and `altScreen` independent; never derive one from another. ⚠️ Config regexes (`discovery.version.regex`, `capabilities.workDetect.workingLine`, `capabilities.workDetect.watchingLine`, `capabilities.workDetect.awaitingLine`, `capabilities.workDetect.blockedLine`, `capabilities.modelDetect.screenLine`) must compile through `compileVersionRegex()`. ⚠️ `privilegedParams[].param` names a LAUNCH PARAM, not the legacy `<Mode>Config` field (bridged only by `launch.legacyConfigAliases`); a wrong name silently clamps nothing. ⚠️ Resolve the registry AT CALL TIME, never in a module-level const. ⚠️ Remote claude/omp arms of `buildRemoteLaunchCommand` are not covered by the pane-command golden. `~/.codeman/clis.json` overrides entries. It is WRITTEN only by the opt-in CLI management routes (`cliManagementEnabled`, default OFF; `/api/clis`, `cli-registry-routes.ts`), and only through `mutateRegistryFile()` in `registry-writer.ts`, which serializes mutations and refuses (409) a file that does not parse or has group/world permission bits rather than overwriting it. Importing the registry still writes nothing. → [architecture-invariants#cli-registry](docs/architecture-invariants.md#cli-registry), `docs/cli-registry.md`
|
||||
**CLI registry** (`src/config/cli-registry/`): every run mode is a `CliEntry` (discovery, launch argv template, env handling, `capabilities`, and the `overlays` behind remote/docker pane commands). **No code outside `stock.ts` may branch on a CLI id**: use a capability field or a NAMED PROFILE (`profiles.ts`); `test/cli-registry-no-id-branching.test.ts` and `test/frontend-cli-no-id-branching.test.ts` enforce it. ⚠️ Config holds typed argv tokens, never shell text; literals are validated at LOAD time and a bad one rejects the whole entry. ⚠️ Keep `external`, `hooks` and `altScreen` independent; never derive one from another. ⚠️ Config regexes (`discovery.version.regex`, `capabilities.workDetect.workingLine`, `capabilities.workDetect.watchingLine`, `capabilities.workDetect.awaitingLine`, `capabilities.workDetect.blockedLine`, `capabilities.modelDetect.screenLine`) must compile through `compileVersionRegex()`. ⚠️ `privilegedParams[].param` names a LAUNCH PARAM, not the legacy `<Mode>Config` field (bridged only by `launch.legacyConfigAliases`); a wrong name silently clamps nothing. ⚠️ Resolve the registry AT CALL TIME, never in a module-level const. ⚠️ Remote claude/omp arms of `buildRemoteLaunchCommand` are not covered by the pane-command golden. `~/.codeman/clis.json` overrides entries. It is WRITTEN only by the opt-in CLI management routes (`cliManagementEnabled`, default OFF; `/api/clis`, `cli-registry-routes.ts`), and only through `mutateRegistryFile()` in `registry-writer.ts` (the one exception: a config-backup restore of `clis.json`, which then calls `reloadCliRegistry()`), which serializes mutations and refuses (409) a file that does not parse or has group/world permission bits rather than overwriting it. Importing the registry still writes nothing. → [architecture-invariants#cli-registry](docs/architecture-invariants.md#cli-registry), `docs/cli-registry.md`
|
||||
|
||||
**External CLI modes (OpenCode, Codex, Gemini, Antigravity, Pi, Grok, DeepSeek, OMP, Copilot)**: `isExternalCliMode()` in `session.ts` gates Claude-specific behavior off (Ralph tracker, BashToolParser, token parsing, ❯ readiness; readiness is output stabilization); work detection is per-CLI `capabilities.workDetect` data, not this gate. All nine **require tmux, no direct PTY fallback** (secrets go via socket-scoped `tmux setenv`, never the command line). ⚠️ `run*()` in `session-ui.js` MUST unwrap the `{success,data}` envelope. ⚠️ **Codex uses predictive write-through echo, never the buffer overlay**: `_predictHookOnData` must never `return` (wire path stays byte-identical), and flushed text and a bracketed paste must go out as separate delayed writes. ⚠️ **Pi**: no bypass flag, never invent one; `approveProjectTrust` executes repo code, so it is in the clamp's **materialize** branch; never wire `--api-key`. ⚠️ **Grok**: `alwaysApprove` is stripped for non-granted owners (only-if-sent). ⚠️ **DeepSeek**: the agent is a PROFILE (Run gates on `isDeepSeekRunnable()`); the permission switch is the `DSH_PERMISSION_MODE` env var, so `clampEnvOverridesForOwner()` must DROP `DSH_PERMISSION_MODE`, `DSH_HOME` and `DEEPSEEK_BASE_URL` for non-granted owners; `hooksAvailableForMode()` is per-SESSION for it (pass `sessionHookOptions(session)`) and is never a stand-in for `mode === 'claude'`; answers come from `deepseek-transcript.ts`, paired by header `cwd` + boot window, never newest-mtime. ⚠️ **OMP**: `OMP_AUTH_BROKER_URL`/`_TOKEN` are clamped the same way. ⚠️ **Copilot**: `allowAll` (`--yolo`) is a `privilegedParam` and `COPILOT_HOME`/`COPILOT_ALLOW_ALL`/`COPILOT_PROVIDER_*`/`COPILOT_MODEL` are privileged env keys, and so is every other `COPILOT_*` variable the CLI reads that switches approvals, runs code with no approval row or sends its sign-in token elsewhere (the 1.0.98 list is in `stock.ts`; the prefix admits whatever a later release adds, so re-read the bundle whenever the measured version moves); the resume id must start with a letter or digit (`path-segment`), since `--resume`'s value is optional and `--resume --yolo` would otherwise make `--yolo` its own flag; `--name` is dropped next to `--resume`/`--continue` (Copilot refuses the pair); its mouse DECSETs are stripped (`strip-mux-and-mouse`) so a drag selects and the wheel is forwarded by `_shouldForwardWheelToApp` while `cliMouseTracking` is on; `composerReadyMark` is the footer hint `/ commands · ? help`, NOT `❯` (the folder-trust dialog draws `❯ 1. Yes`); `last-response` reads the pane's visible screen (`transcriptNeedsScreenCapture`) through a per-CLI `PANE_DIALECTS` row, because its stream is cursor moves, not lines (a prompt row scrolled off by a tall answer reads as the answer's tail unless a banner/trust dialog is on screen, and typed composer text is not a prompt); Docker seeds no Copilot credential (`config.json` holds a plain-text token without a keyring). → [architecture-invariants#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp](docs/architecture-invariants.md#external-cli-modes-opencode-codex-gemini-antigravity-pi-grok-deepseek-omp)
|
||||
|
||||
|
||||
@@ -1154,18 +1154,18 @@ The result carries server **names** only, never `env` values, `headers` or file
|
||||
|
||||
## Config backups
|
||||
|
||||
Timestamped snapshots of the files that hold what a user has customized (`src/config-backup.ts`). On by default; a snapshot is taken at startup and whenever the tracked files' content changes (checked every 5 minutes), so there is one entry per real change. All three routes are admin only in multi-user mode (`403`), because the files include credentials. Responses carry file names and sizes, never content.
|
||||
Timestamped snapshots of the files that hold what a user has customized (`src/config-backup.ts`). On by default; a snapshot is taken at startup and whenever the tracked files' content changes (checked every 5 minutes), so there is one entry per real change. All four routes are admin only in multi-user mode (`403`), because the files include credentials. Responses carry file names and sizes; only the diff returns values, and never a credential's.
|
||||
|
||||
Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `remote-hosts.json`, `docker-hosts.json`, `docker-cases.json`, `webviews.json`, `push-keys.json`, `users.json`, `.env`. Session state (`state.json`, which also holds cron jobs and respawn presets), logs, the hook secret and caches are not.
|
||||
Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `remote-hosts.json`, `remote-cases.json`, `docker-hosts.json`, `docker-cases.json`, `webviews.json`, `push-keys.json`, `users.json`, `.env`. Session state (`state.json`, which also holds cron jobs and respawn presets), logs, the hook secret and caches are not.
|
||||
|
||||
Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `<data dir>/backups/config`), `configBackupKeepCount` (1–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, and a SHA-256 and size per file), all `0600`. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`.
|
||||
Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `<data dir>/backups/config`), `configBackupKeepCount` (1–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, an `origin` id, and a SHA-256 and size per file), all `0600`. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`.
|
||||
|
||||
| Method | Path | Body | Notes |
|
||||
| ------ | ------------------------------------ | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `GET` | `/api/config-backups` | none | `settings` in effect, `dir`, `defaultDir`, `tracked[]` and `backups[]` (`id`, `createdAt`, `reason`, `appVersion`, `files[]`, `bytes`), newest first. |
|
||||
| `POST` | `/api/config-backups` | none | Snapshot now, even if nothing changed (`reason: manual`), then apply retention. `status` is `created` or `empty` (nothing tracked exists yet). |
|
||||
| `POST` | `/api/config-backups` | none | Snapshot now, even if nothing changed (`reason: manual`), then apply retention. `status` is `created` or `empty` (nothing tracked exists yet); `removed[]` lists the ids retention deleted. `503` when a custom folder does not answer. |
|
||||
| `GET` | `/api/config-backups/diff?from=&to=` | none | What differs between two states: `from`/`to` are a backup id or `current` (the running config; the default for `to`). `files[]` (`name`, `status` `added`/`removed`/`changed`/`unchanged`, `changes[]` of `path`, `kind`, `before`/`after`, `truncated`, `detail`) and `changed`. Credential values are never returned (`redacted: true`). `400` without `from`; `404` unknown id; `409` if a file fails its checksum. |
|
||||
| `POST` | `/api/config-backups/:id/restore` | none | Put a snapshot's files back, atomically, after a `pre-restore` snapshot of the current ones. `404` unknown id; `409 CONFLICT` if a file fails its checksum (nothing changed). `restartRequired[]` names files the running server only reloads on restart. |
|
||||
| `POST` | `/api/config-backups/:id/restore` | none | Put a snapshot's files back, atomically, after a `pre-restore` snapshot of the current ones. `404` unknown id; `409 CONFLICT` if a file fails its checksum (nothing changed). Returns `restored[]`, `untouched[]` (tracked files present now that the backup did not hold; left as they are), `safetySnapshot` (the `pre-restore` id) and `restartRequired[]` (files the running server only reads at start: `.env`, `push-keys.json`, `settings.json`). |
|
||||
|
||||
## Webhook notifications
|
||||
|
||||
|
||||
@@ -113,7 +113,7 @@ Further detail: ⚠️ An ADOPTED container may back SEVERAL cases at different
|
||||
|
||||
Spawn commands are pinned as literal strings in `test/cli-registry-spawn-golden.test.ts`, remote/docker pane commands in `test/location-overlay-commands.test.ts`. ⚠️ That second golden no longer covers **remote claude or remote omp**: both now have their own arm in `buildRemoteLaunchCommand` (a `--session-id || --resume` pair, and `--continue`, so a respawn continues the same conversation) and never reach `defaultRemoteCommandForMode`, which is what that test asserts. Their real pins are `toContain` substrings in `test/tmux-manager.test.ts` and `test/remote-shared-sessions.test.ts`; changing either arm will NOT fail the golden.
|
||||
|
||||
⚠️ Anything reading the registry resolves it AT CALL TIME (`sessionModeSchema()`, `allowedEnvPrefixes()`, `dependencyRegistry()`, the resolvers' `searchDirs` thunks) — a module-level const freezes at first import, so a CLI enabled while the server ran moved the run menu but not that surface. `~/.codeman/clis.json` overrides any entry. Importing the registry still has no filesystem side effects: the only writer is `registry-writer.ts`, imported only by `cli-registry-routes.ts` (the opt-in CLI management routes, `cliManagementEnabled`, default OFF). ⚠️ Every write goes through `mutateRegistryFile()`: mutations run one at a time on a single promise chain, each with a unique temp file (three parallel toggles once lost two and 500'd on a shared temp name), and a file the READER would ignore (group/world permission bits) or quarantine (unparseable) is REFUSED with a 409 rather than rewritten, so one Settings click cannot replace a hand-edit or turn a refused file into trusted 0600 config. Only a missing file starts fresh. After a write the registry is reloaded, so changes apply without a restart. User guide: `docs/cli-registry.md`.
|
||||
⚠️ Anything reading the registry resolves it AT CALL TIME (`sessionModeSchema()`, `allowedEnvPrefixes()`, `dependencyRegistry()`, the resolvers' `searchDirs` thunks) — a module-level const freezes at first import, so a CLI enabled while the server ran moved the run menu but not that surface. `~/.codeman/clis.json` overrides any entry. Importing the registry still has no filesystem side effects: the only writer is `registry-writer.ts`, imported only by `cli-registry-routes.ts` (the opt-in CLI management routes, `cliManagementEnabled`, default OFF). ⚠️ Every write goes through `mutateRegistryFile()` (the one exception is a config-backup restore of `clis.json`, which writes it atomically and calls `reloadCliRegistry()`): mutations run one at a time on a single promise chain, each with a unique temp file (three parallel toggles once lost two and 500'd on a shared temp name), and a file the READER would ignore (group/world permission bits) or quarantine (unparseable) is REFUSED with a 409 rather than rewritten, so one Settings click cannot replace a hand-edit or turn a refused file into trusted 0600 config. Only a missing file starts fresh. After a write the registry is reloaded, so changes apply without a restart. User guide: `docs/cli-registry.md`.
|
||||
|
||||
## Session data and lifecycle
|
||||
|
||||
|
||||
@@ -122,14 +122,17 @@ Only the tracked names are ever written, whatever a manifest lists.
|
||||
|
||||
Settings are re-read on the next request, so reload the page. `clis.json`, `users.json`,
|
||||
`.env` and `push-keys.json` are read once per process: **restart Codeman** for those to take
|
||||
effect (the response lists which). Restoring `settings.json` also does not re-run the switches
|
||||
effect (the response lists which). `clis.json` is reloaded by the restore itself and `users.json`
|
||||
is re-read within a second. Restoring `settings.json` also does not re-run the switches
|
||||
that saving in Settings applies at once (file watchers and the like), so those apply after a
|
||||
restart, and `settings.json` is listed with the restart-required files. A restore does not touch
|
||||
running sessions.
|
||||
|
||||
When the backup folder is shared (a synced folder used by two machines, or prod and a beta),
|
||||
each snapshot records the host and data directory that made it, and every instance only
|
||||
snapshots, lists, prunes, compares and restores its own.
|
||||
each snapshot records a random id that the instance made once and keeps in its data directory
|
||||
(`config-backup-id`; not the hostname, which a Docker container changes on every recreate). Every
|
||||
instance only snapshots, lists, prunes, compares and restores its own. Deleting that file makes
|
||||
the instance a new owner, and its earlier snapshots stop being listed (they stay on disk).
|
||||
|
||||
You can also restore by hand: the files in a backup folder are the originals, so copying one back
|
||||
over the file in the data directory is equivalent (stop Codeman first if you are replacing
|
||||
|
||||
+30
-5
@@ -21,7 +21,7 @@
|
||||
* @module config-backup
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto';
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import {
|
||||
chmodSync,
|
||||
existsSync,
|
||||
@@ -34,7 +34,7 @@ import {
|
||||
statSync,
|
||||
writeFileSync,
|
||||
} from 'node:fs';
|
||||
import { homedir, hostname } from 'node:os';
|
||||
import { homedir } from 'node:os';
|
||||
import { isAbsolute, join, resolve, sep } from 'node:path';
|
||||
|
||||
/**
|
||||
@@ -96,7 +96,7 @@ export interface ConfigBackupManifest {
|
||||
appVersion: string;
|
||||
/** Digest of the set of files, so an unchanged state is not snapshotted twice. */
|
||||
hash: string;
|
||||
/** Host and data dir that made it, so instances sharing one folder keep to their own snapshots. */
|
||||
/** Id of the data dir that made it (see {@link configBackupOrigin}), so instances sharing one folder keep to their own snapshots. */
|
||||
origin?: string;
|
||||
files: ConfigBackupFile[];
|
||||
}
|
||||
@@ -156,8 +156,33 @@ export function resolveConfigBackupDir(settings: ConfigBackupSettings, dataDir:
|
||||
return dir;
|
||||
}
|
||||
|
||||
/** Who owns a snapshot: this machine's name plus the data dir (prod and a beta differ by the latter). */
|
||||
export const configBackupOrigin = (dataDir: string): string => `${hostname()}:${resolve(dataDir)}`;
|
||||
const ORIGIN_FILE = 'config-backup-id';
|
||||
const ORIGIN_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
|
||||
/**
|
||||
* Who owns a snapshot: a random id made once and kept in the data dir. A hostname would not do (a
|
||||
* Docker container gets a new one on every recreate, which orphaned every earlier snapshot), nor
|
||||
* would the path alone (two machines can share a `$HOME` path). It is a file of its own, not the
|
||||
* hook secret, and is not tracked or restored. If it cannot be written the id is not persisted, so
|
||||
* snapshots made meanwhile are not listed later; the data dir being unwritable breaks far more.
|
||||
*/
|
||||
export function configBackupOrigin(dataDir: string): string {
|
||||
const path = join(dataDir, ORIGIN_FILE);
|
||||
try {
|
||||
const existing = readFileSync(path, 'utf8').trim();
|
||||
if (ORIGIN_RE.test(existing)) return existing;
|
||||
} catch {
|
||||
/* missing: made below */
|
||||
}
|
||||
const id = randomUUID();
|
||||
try {
|
||||
mkdirSync(dataDir, { recursive: true });
|
||||
writeFileSync(path, id + '\n', { mode: 0o600 });
|
||||
} catch {
|
||||
/* see above */
|
||||
}
|
||||
return id;
|
||||
}
|
||||
|
||||
const sha256 = (data: Buffer | string): string => createHash('sha256').update(data).digest('hex');
|
||||
|
||||
|
||||
@@ -1482,7 +1482,11 @@ Object.assign(CodemanApp.prototype, {
|
||||
},
|
||||
|
||||
async restoreConfigBackup(id) {
|
||||
if (!confirm('Restore this backup? Its files replace the current ones (a copy of the current files is made first). Some files only take effect after a restart.')) return;
|
||||
const names = ((this._configBackups || []).find((b) => b.id === id)?.files || []).map((f) => f.name);
|
||||
const secret = names.filter((n) => ['.env', 'users.json', 'push-keys.json', 'custom-model-hosts.json', 'webhook.json'].includes(n));
|
||||
const list = names.length ? `\n\nFiles replaced: ${names.join(', ')}` : '';
|
||||
const warn = secret.length ? `\n\nThis rolls back credentials and accounts (${secret.join(', ')}): passwords, disabled accounts and keys return to how they were then.` : '';
|
||||
if (!confirm(`Restore this backup? Its files replace the current ones (a copy of the current files is made first). Some files only take effect after a restart.${list}${warn}`)) return;
|
||||
const res = await this._apiPost(`/api/config-backups/${encodeURIComponent(id)}/restore`, {});
|
||||
let body = null;
|
||||
try { body = res ? await res.json() : null; } catch { /* fall through */ }
|
||||
@@ -1492,7 +1496,12 @@ Object.assign(CodemanApp.prototype, {
|
||||
}
|
||||
const d = body.data;
|
||||
const restart = d.restartRequired?.length ? ` Restart Codeman to apply: ${d.restartRequired.join(', ')}.` : '';
|
||||
this.showToast(`Restored ${d.restored.length} file${d.restored.length === 1 ? '' : 's'}. Reload this page to see settings.${restart}`, 'success');
|
||||
// The form still holds the pre-restore values, and Save would PUT them over the restored file.
|
||||
if (d.restored.includes('settings.json')) {
|
||||
await this.loadAppSettingsFromServer();
|
||||
this.openAppSettings();
|
||||
}
|
||||
this.showToast(`Restored ${d.restored.length} file${d.restored.length === 1 ? '' : 's'}.${restart}`, 'success');
|
||||
this.loadConfigBackups();
|
||||
},
|
||||
|
||||
|
||||
@@ -7,9 +7,9 @@
|
||||
* GET /api/config-backups/diff?from=&to= — what differs between two snapshots, or a snapshot and the
|
||||
* running config (`current`, the default for `to`).
|
||||
*
|
||||
* The files include credentials (`.env`, `users.json`, endpoint keys), so in multi-user mode all three
|
||||
* The files include credentials (`.env`, `users.json`, endpoint keys), so in multi-user mode all four
|
||||
* are admin only. Responses carry file NAMES and sizes, never file content, except the diff, which shows
|
||||
* changed values but never a credential's (`.env`, `users.json`, `push-keys.json` and any key that looks
|
||||
* changed values but never a credential's (`.env`, `users.json`, `push-keys.json`, `webhook.json`, `intents.json` and any key that looks
|
||||
* like a token/secret/password are reported as "changed" only). Restore replaces files the running server
|
||||
* may have cached: `restartRequired` says which ones only apply after a restart.
|
||||
*/
|
||||
@@ -18,6 +18,7 @@ import { createRequire } from 'node:module';
|
||||
import type { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify';
|
||||
import { ApiErrorCode, createErrorResponse, getErrorMessage, type ApiResponse } from '../../types.js';
|
||||
import { isAdmin, readJsonConfig, SETTINGS_PATH } from '../route-helpers.js';
|
||||
import { reloadCliRegistry } from '../../config/cli-registry/registry.js';
|
||||
import { isMultiUserMode } from '../../config/multiuser.js';
|
||||
import { getDataDir } from '../../config/instance.js';
|
||||
import { intentStore } from '../../intent-store.js';
|
||||
@@ -53,8 +54,9 @@ const APP_VERSION: string = (() => {
|
||||
}
|
||||
})();
|
||||
|
||||
/** Files whose running-process copy only reloads on restart (the rest are read per request). */
|
||||
const RESTART_REQUIRED = new Set<string>(['clis.json', 'users.json', '.env', 'push-keys.json', 'settings.json']);
|
||||
/** Files whose running-process copy only reloads on restart (the rest are read per request; `users.json`
|
||||
* is re-read within a second, `clis.json` is reloaded below, `intents.json` by `reload()`). */
|
||||
const RESTART_REQUIRED = new Set<string>(['.env', 'push-keys.json', 'settings.json']);
|
||||
|
||||
export interface ConfigBackupStatus {
|
||||
settings: ConfigBackupSettings;
|
||||
@@ -203,6 +205,8 @@ export function registerConfigBackupRoutes(app: FastifyInstance): void {
|
||||
const out = restoreConfigBackup({ dataDir: getDataDir(), backupDir: dir, id, appVersion: APP_VERSION });
|
||||
// IntentStore caches the file and rewrites it whole on the next prompt, which would undo the restore.
|
||||
if (out.restored.includes('intents.json')) intentStore.reload();
|
||||
// restoreConfigBackup writes clis.json directly (the one writer besides mutateRegistryFile).
|
||||
if (out.restored.includes('clis.json')) reloadCliRegistry();
|
||||
return {
|
||||
success: true,
|
||||
data: {
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
/**
|
||||
* @fileoverview The snapshot origin must survive a hostname change (a Docker container gets a new
|
||||
* hostname on every recreate) and still differ between data dirs.
|
||||
*/
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
|
||||
const host = vi.hoisted(() => ({ name: 'container-a' }));
|
||||
vi.mock('node:os', async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import('node:os')>()),
|
||||
hostname: () => host.name,
|
||||
}));
|
||||
|
||||
import {
|
||||
configBackupOrigin,
|
||||
createConfigBackup,
|
||||
listConfigBackups,
|
||||
pruneConfigBackups,
|
||||
restoreConfigBackup,
|
||||
} from '../src/config-backup.js';
|
||||
|
||||
let root: string;
|
||||
let data: string;
|
||||
let backups: string;
|
||||
beforeEach(() => {
|
||||
root = mkdtempSync(join(tmpdir(), 'cfg-origin-'));
|
||||
data = join(root, 'data');
|
||||
backups = join(root, 'backups');
|
||||
mkdirSync(data, { recursive: true });
|
||||
writeFileSync(join(data, 'settings.json'), '{"a":1}');
|
||||
host.name = 'container-a';
|
||||
});
|
||||
afterEach(() => rmSync(root, { recursive: true, force: true }));
|
||||
|
||||
describe('configBackupOrigin', () => {
|
||||
it('is made once, kept in the data dir, and differs between data dirs', () => {
|
||||
const a = configBackupOrigin(data);
|
||||
expect(configBackupOrigin(data)).toBe(a);
|
||||
expect(readFileSync(join(data, 'config-backup-id'), 'utf8').trim()).toBe(a);
|
||||
const other = join(root, 'other');
|
||||
expect(configBackupOrigin(other)).not.toBe(a);
|
||||
});
|
||||
|
||||
it('keeps snapshots listed, restorable and prunable after the hostname changes', () => {
|
||||
const made = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: 1_000_000 }) as {
|
||||
id: string;
|
||||
};
|
||||
createConfigBackup({ dataDir: data, backupDir: backups, reason: 'manual', force: true, now: 2_000_000 });
|
||||
host.name = 'container-b'; // the container was recreated
|
||||
const origin = configBackupOrigin(data);
|
||||
expect(listConfigBackups(backups, origin)).toHaveLength(2);
|
||||
expect(() => restoreConfigBackup({ dataDir: data, backupDir: backups, id: made.id })).not.toThrow();
|
||||
const removed = pruneConfigBackups(backups, { keepCount: 1, keepDays: 0, origin });
|
||||
expect(removed).toHaveLength(2); // 3 now: the restore took a pre-restore copy
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,64 @@
|
||||
/**
|
||||
* @fileoverview Restoring settings.json from inside App Settings must refill the form, or Save
|
||||
* would PUT the pre-restore values back over the restored file. Evaluates the real settings-ui.js
|
||||
* against a stub app.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import vm from 'node:vm';
|
||||
|
||||
const SRC = readFileSync(new URL('../src/web/public/settings-ui.js', import.meta.url), 'utf-8');
|
||||
|
||||
function makeApp(restored: string[], confirmed = true) {
|
||||
const calls: string[] = [];
|
||||
const confirms: string[] = [];
|
||||
function CodemanApp(this: unknown) {}
|
||||
const ctx = vm.createContext({
|
||||
CodemanApp,
|
||||
window: {},
|
||||
document: {},
|
||||
confirm: (m: string) => (confirms.push(m), confirmed),
|
||||
escapeHtml: (s: string) => s,
|
||||
});
|
||||
vm.runInContext(SRC, ctx);
|
||||
const app = Object.create((CodemanApp as unknown as { prototype: object }).prototype) as Record<string, unknown>;
|
||||
Object.assign(app, {
|
||||
_configBackups: [
|
||||
{
|
||||
id: 'cfg-1',
|
||||
files: [{ name: 'settings.json' }, { name: 'users.json' }, { name: '.env' }, { name: 'clis.json' }],
|
||||
},
|
||||
],
|
||||
_apiPost: async () => ({
|
||||
ok: true,
|
||||
json: async () => ({ success: true, data: { restored, restartRequired: [] } }),
|
||||
}),
|
||||
loadAppSettingsFromServer: async () => void calls.push('reload-settings'),
|
||||
openAppSettings: () => void calls.push('refill-form'),
|
||||
loadConfigBackups: () => void calls.push('reload-list'),
|
||||
showToast: () => {},
|
||||
});
|
||||
return { app, calls, confirms };
|
||||
}
|
||||
|
||||
describe('restoreConfigBackup (Settings UI)', () => {
|
||||
it('re-reads settings into the form when settings.json was restored', async () => {
|
||||
const { app, calls } = makeApp(['settings.json', 'clis.json']);
|
||||
await (app.restoreConfigBackup as (id: string) => Promise<void>)('cfg-1');
|
||||
expect(calls).toEqual(['reload-settings', 'refill-form', 'reload-list']);
|
||||
});
|
||||
|
||||
it('leaves the form alone when settings.json was not part of the restore', async () => {
|
||||
const { app, calls } = makeApp(['clis.json']);
|
||||
await (app.restoreConfigBackup as (id: string) => Promise<void>)('cfg-1');
|
||||
expect(calls).toEqual(['reload-list']);
|
||||
});
|
||||
|
||||
it('names the files being replaced and calls out the credential ones', async () => {
|
||||
const { app, confirms } = makeApp(['settings.json'], false);
|
||||
await (app.restoreConfigBackup as (id: string) => Promise<void>)('cfg-1');
|
||||
expect(confirms[0]).toContain('settings.json, users.json, .env, clis.json');
|
||||
expect(confirms[0]).toContain('credentials');
|
||||
expect(confirms[0]).toMatch(/users\.json, \.env\)/);
|
||||
});
|
||||
});
|
||||
@@ -84,7 +84,7 @@ describe('/api/config-backups', () => {
|
||||
expect(res.statusCode).toBe(200);
|
||||
const d = res.json().data;
|
||||
expect(d.restored).toEqual(expect.arrayContaining(['settings.json', 'clis.json']));
|
||||
expect(d.restartRequired).toContain('clis.json');
|
||||
expect(d.restartRequired).not.toContain('clis.json'); // reloaded by the restore itself
|
||||
expect(d.restartRequired).toContain('settings.json'); // its watcher toggles are not re-run by a restore
|
||||
expect(d.safetySnapshot).toBeTruthy();
|
||||
expect(JSON.parse(readFileSync(SETTINGS_PATH, 'utf8')).displayName).toBe('before');
|
||||
|
||||
Reference in new issue
Block a user