diff --git a/CLAUDE.md b/CLAUDE.md index 93e73e78..bcdefb3d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -156,7 +156,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Auto-resume on usage limit** (opt-in per session, top of the Respawn tab): when Claude halts on a subscription limit, `usage-limit-patterns.ts` (pure, unit-tested) parses the reset time and `SessionAutoOps` arms a timer for reset+2min, then sends Esc + `continue`. ⚠️ Respawn cycles are blocked while paused (`isLimitPaused` guard in `onIdleDetected`), which is what prevents `/clear` from wiping the paused conversation. Claude-mode only. → [architecture-invariants#auto-resume-on-usage-limit](docs/architecture-invariants.md#auto-resume-on-usage-limit) -**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, `configBackupDir`, `configBackupKeepCount` 1-500 default 20, `configBackupKeepDays` 0-3650 default 30; `docs/wiki/Config-Backups.md`): timestamped snapshots `cfg-YYYYMMDD-HHMMSS/` (+ `manifest.json` with SHA-256s, `reason`, `appVersion`) of a FIXED allowlist (`CONFIG_BACKUP_FILES`: settings, clis, custom-model-hosts, webhook, intents, linked-cases, remote/docker hosts+cases (incl. `remote-cases.json`), webviews, push-keys, users, `.env`) into `/backups/config` or a custom absolute folder. ⚠️ `state.json` is deliberately NOT tracked (rewritten per session event; it also holds cron jobs and respawn presets). One snapshot per real content change (digest of the file set vs the latest manifest), at startup + a 5-minute timer, skipped in test mode. ⚠️ Retention (`pruneConfigBackups`) only deletes folders matching the name pattern AND holding a parseable manifest, and never the newest; a custom folder equal to the data dir falls back to the default. ⚠️ Restore verifies every file's SHA-256 BEFORE writing, takes a `pre-restore` snapshot, writes atomically at 0600 and only allowlisted names; the id is validated against the name pattern (no traversal). Snapshots are 0700/0600 because `.env`/`users.json`/`custom-model-hosts.json` carry credentials; all routes are admin-only in multi-user mode. ⚠️ A custom folder is probed with `probePathKind(dir, { pastCap: true })` before every pass and in each route (#516: it may be a network mount; `unknown` skips the pass / answers 503 `OPERATION_FAILED`), and a missing custom folder is only created when its parent is present. ⚠️ Each manifest records an `origin`, a random id kept in `/config-backup-id` (NOT the hostname: a Docker container gets a new one per recreate, which orphaned every snapshot); change check, prune, list, restore and diff are scoped to it so instances sharing one folder do not snapshot or prune each other. Restoring `intents.json` calls `intentStore.reload()`; `settings.json` applies its watcher toggles after a restart; Settings re-reads `/api/settings` into the form after a restore of it, since Save would otherwise PUT the old values back. ⚠️ Restoring `clis.json` writes it directly (the one writer besides `mutateRegistryFile()`) and then calls `reloadCliRegistry()`. → [docs/wiki/Config-Backups.md](docs/wiki/Config-Backups.md) +**Config backups** (`src/config-backup.ts`, `routes/config-backup-routes.ts`; `configBackupEnabled` absent = ON, `configBackupDir`, `configBackupKeepCount` 2-500 default 20, `configBackupKeepDays` 0-3650 default 30; `docs/wiki/Config-Backups.md`): timestamped snapshots `cfg-YYYYMMDD-HHMMSS/` (+ `manifest.json` with SHA-256s, `reason`, `appVersion`) of a FIXED allowlist (`CONFIG_BACKUP_FILES`: settings, clis, custom-model-hosts, webhook, intents, linked-cases, remote/docker hosts+cases (incl. `remote-cases.json`), webviews, push-keys, users, `.env`) into `/backups/config` or a custom absolute folder. ⚠️ `state.json` is deliberately NOT tracked (rewritten per session event; it also holds cron jobs and respawn presets). One snapshot per real content change (digest of the file set vs the latest manifest), at startup + a 5-minute timer, skipped in test mode. ⚠️ Retention (`pruneConfigBackups`) only deletes folders matching the name pattern AND holding a parseable manifest, and never the newest; a custom folder equal to the data dir falls back to the default. ⚠️ Restore verifies every file's SHA-256 BEFORE writing, takes a `pre-restore` snapshot, writes atomically at 0600 and only allowlisted names; the id is validated against the name pattern (no traversal). Snapshots are 0700/0600 because `.env`/`users.json`/`custom-model-hosts.json` carry credentials; all routes are admin-only in multi-user mode. ⚠️ A custom folder is probed with `probePathKind(dir, { pastCap: true })` before every pass and in each route (#516: it may be a network mount; `unknown` skips the pass / answers 503 `OPERATION_FAILED`), and a missing custom folder is only created when its parent is present. ⚠️ Each manifest records an `origin`, a random id kept in `/config-backup-id` (NOT the hostname: a Docker container gets a new one per recreate, which orphaned every snapshot); change check, prune, list, restore and diff are scoped to it so instances sharing one folder do not snapshot or prune each other. Restoring `intents.json` calls `intentStore.reload()`; `settings.json` applies its watcher toggles after a restart; Settings re-reads `/api/settings` into the form after a restore of it, since Save would otherwise PUT the old values back. ⚠️ Restoring `clis.json` writes it directly (the one writer besides `mutateRegistryFile()`) and then calls `reloadCliRegistry()`. → [docs/wiki/Config-Backups.md](docs/wiki/Config-Backups.md) **Plan-usage chip** (`showPlanUsageLimits`, per-device: desktop default **ON**, handhelds OFF): resolve DISPLAY only through `planUsageChipEnabled()` (settings-ui.js). The same setting is the server-side COLLECTION switch, read fresh by `readPlanUsageTelemetryEnabled()` (hooks-config.ts) at every claude create/respawn. ⚠️ An ABSENT key reads as ON in the reader; `GET /api/settings` must never write. ⚠️ A save sends `showPlanUsageLimits` ONLY when it flips the chip on that device (`planUsageCollectionFlip()`), or a phone switches collection off for every desktop. Claude data comes from the statusLine exporter, injected as an EPHEMERAL `claude --settings` flag (`resolveStatusLineCliCommand`), never written to disk, WRAPPING a user's own statusLine, posting to `POST /api/status-telemetry`. Codex comes from a read-only `account/rateLimits/read` poll (main bucket only). The statusline's `prompt_cache` object (Claude Code v2.1.251+) is kept by `StatusTelemetrySchema` (every documented scalar field, null-tolerant; `parsePromptCache()` in usage-telemetry.ts normalizes only the stored subset) and shown in the footer as `cache:until HH:MM` / `cache:cold` (only when the user has no statusline of their own) and in Session Options → Respawn (`#promptCacheStatus`, fed by `Session.promptCache` in `toState()` plus the change-only `session:promptCache` SSE event, worded by the pure `CodemanPromptCache.format`): warm-until/cold, the re-cache stake, the hit rate, the last miss's causes, and a `Context compacted at HH:MM` note after a compaction or a tool-result clearing (stamped on the first warm report whose `recache_tokens_if_cold` is null after one that carried it, never from `expected_rebuilds`, which Claude Code bumps only at the next request; NEVER an alert). ⚠️ `Session.setPromptCache()` stamps `compactedAt` onto a NEW object (never mutating the parsed report) and the route broadcasts the STORED object; the last report is restored on restart (constructor `promptCache`, like `displayModel`) and the readout is time-aware (a passed `expiresAt` reads cold, not stale-warm). It is the source of truth for the cache TTL, never the plan tier. → [architecture-invariants#plan-usage-chip-statusline-telemetry](docs/architecture-invariants.md#plan-usage-chip-statusline-telemetry), `docs/usage-limits-display-plan.md` diff --git a/docs/api-reference.md b/docs/api-reference.md index dc2fa42a..45553419 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -1158,7 +1158,7 @@ Timestamped snapshots of the files that hold what a user has customized (`src/co Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.json`, `custom-model-hosts.json`, `webhook.json`, `intents.json`, `linked-cases.json`, `remote-hosts.json`, `remote-cases.json`, `docker-hosts.json`, `docker-cases.json`, `webviews.json`, `push-keys.json`, `users.json`, `.env`. Session state (`state.json`, which also holds cron jobs and respawn presets), logs, the hook secret and caches are not. -Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `/backups/config`), `configBackupKeepCount` (1–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, an `origin` id, and a SHA-256 and size per file), all `0600`. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`. +Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `/backups/config`), `configBackupKeepCount` (2–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, an `origin` id, and a SHA-256 and size per file), all `0600`. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`. | Method | Path | Body | Notes | | ------ | ------------------------------------ | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | diff --git a/docs/wiki/Config-Backups.md b/docs/wiki/Config-Backups.md index 0f819f4a..041f7110 100644 --- a/docs/wiki/Config-Backups.md +++ b/docs/wiki/Config-Backups.md @@ -81,8 +81,8 @@ Two limits, applied after every backup: | Setting | Range | Default | Meaning | | ------- | ----- | ------- | ------- | -| **Keep this many backups** (`configBackupKeepCount`) | 1 – 500 | 20 | Keep the newest N; older ones are removed. | -| **Delete backups older than (days)** (`configBackupKeepDays`) | 0 – 3650 | 30 | Also remove anything older than this. `0` means no age limit. | +| **Keep this many backups** (`configBackupKeepCount`) | 2 – 500 | 20 | Keep the newest N; older ones are removed. (1 is not offered: every change would delete the state before it.) | +| **Delete backups older than (days)** (`configBackupKeepDays`) | 0 – 3650 | 30 | Also remove a backup once it has been replaced for longer than this. `0` means no age limit. | Both apply together. Three guarantees: @@ -90,6 +90,10 @@ Both apply together. Three guarantees: go back to. - **Only folders Codeman made are ever removed**, never anything else that happens to live in a custom folder. +- **Age counts from when a backup was replaced**, not from when it was made. Backups are made on + change, so a config that stays the same for months has one backup whose own date keeps + ageing; it is the copy you need the day something goes wrong, so it stays until a newer one + has existed for longer than the limit. - Because backups are made on change, 20 backups reach back across 20 *changes*, however long that took, not 20 timer ticks. @@ -120,10 +124,9 @@ From the list, press **Restore** on a backup and confirm. Codeman then: Only the tracked names are ever written, whatever a manifest lists. -Settings are re-read on the next request, so reload the page. `clis.json`, `users.json`, -`.env` and `push-keys.json` are read once per process: **restart Codeman** for those to take -effect (the response lists which). `clis.json` is reloaded by the restore itself and `users.json` -is re-read within a second. Restoring `settings.json` also does not re-run the switches +Settings are re-read on the next request, so reload the page. `.env` and `push-keys.json` are +read once per process: **restart Codeman** for those to take effect (the response lists which). +`clis.json` is reloaded by the restore itself and `users.json` is re-read within a second. Restoring `settings.json` also does not re-run the switches that saving in Settings applies at once (file watchers and the like), so those apply after a restart, and `settings.json` is listed with the restart-required files. A restore does not touch running sessions. diff --git a/src/config-backup.ts b/src/config-backup.ts index 9ac01bad..a583dcf8 100644 --- a/src/config-backup.ts +++ b/src/config-backup.ts @@ -66,6 +66,8 @@ export const CONFIG_BACKUP_FILES = [ ] as const; export const CONFIG_BACKUP_DEFAULTS = { enabled: true, keepCount: 20, keepDays: 30 } as const; +/** 1 would delete the state before every change, so a bad write could never be undone. */ +export const CONFIG_BACKUP_MIN_KEEP_COUNT = 2; export const CONFIG_BACKUP_MAX_KEEP_COUNT = 500; export const CONFIG_BACKUP_MAX_KEEP_DAYS = 3650; @@ -77,7 +79,7 @@ export interface ConfigBackupSettings { enabled: boolean; /** Absolute folder, or null for the default (`/backups/config`). */ dir: string | null; - /** Keep at most this many snapshots (newest first). 1–500. */ + /** Keep at most this many snapshots (newest first). 2–500. */ keepCount: number; /** Also drop snapshots older than this many days; 0 = no age limit. */ keepDays: number; @@ -134,7 +136,12 @@ export function readConfigBackupSettings(raw: Record | null | u // Absent means ON: the feature protects against an event nobody schedules. enabled: s.configBackupEnabled !== false, dir: normalizeBackupDir(s.configBackupDir), - keepCount: intIn(s.configBackupKeepCount, 1, CONFIG_BACKUP_MAX_KEEP_COUNT, CONFIG_BACKUP_DEFAULTS.keepCount), + keepCount: intIn( + s.configBackupKeepCount, + CONFIG_BACKUP_MIN_KEEP_COUNT, + CONFIG_BACKUP_MAX_KEEP_COUNT, + CONFIG_BACKUP_DEFAULTS.keepCount + ), keepDays: intIn(s.configBackupKeepDays, 0, CONFIG_BACKUP_MAX_KEEP_DAYS, CONFIG_BACKUP_DEFAULTS.keepDays), }; } @@ -197,6 +204,11 @@ const sha256 = (data: Buffer | string): string => createHash('sha256').update(da */ const VOLATILE_KEYS: Readonly> = { 'custom-model-hosts.json': ['lastDiscoveredAt'], + // Rewritten by activity, not by anything the user set: a Basic-auth login (at most once a minute + // per user), a session started in another case, and every captured prompt. + 'users.json': ['lastLoginAt'], + 'settings.json': ['lastUsedCase'], + 'intents.json': ['recentPrompts', 'updatedAt'], }; function withoutKeys(value: unknown, drop: ReadonlySet): unknown { @@ -378,7 +390,9 @@ export function pruneConfigBackups(dir: string, opts: PruneOptions): string[] { all.forEach((b, index) => { if (index === 0) return; const tooMany = index >= Math.max(1, opts.keepCount); - const tooOld = cutoff !== null && b.createdAt < cutoff; + // A snapshot's state was live until the next one was made: age it from then. Aging by its own + // createdAt deleted the last good copy on the first change after a quiet month. + const tooOld = cutoff !== null && all[index - 1].createdAt < cutoff; if (!tooMany && !tooOld) return; // listConfigBackups already proved this is one of ours (name pattern + manifest). const target = join(dir, b.id); diff --git a/src/web/public/index.html b/src/web/public/index.html index aac43e48..a671bb9c 100644 --- a/src/web/public/index.html +++ b/src/web/public/index.html @@ -3262,9 +3262,9 @@
Keep this many backups - 1 to 500. The newest backup is always kept. + 2 to 500 (with 1, every change would delete the state before it). The newest backup is always kept.
- +
diff --git a/src/web/public/settings-ui.js b/src/web/public/settings-ui.js index a09119d3..30a475b8 100644 --- a/src/web/public/settings-ui.js +++ b/src/web/public/settings-ui.js @@ -2880,7 +2880,7 @@ Object.assign(CodemanApp.prototype, { mcpSyncEnabled: document.getElementById('appSettingsMcpSync').checked, configBackupEnabled: document.getElementById('appSettingsConfigBackup').checked, configBackupDir: document.getElementById('appSettingsConfigBackupDir').value.trim(), - configBackupKeepCount: Math.min(500, Math.max(1, parseInt(document.getElementById('appSettingsConfigBackupKeepCount').value, 10) || 20)), + configBackupKeepCount: Math.min(500, Math.max(2, parseInt(document.getElementById('appSettingsConfigBackupKeepCount').value, 10) || 20)), configBackupKeepDays: Math.min(3650, Math.max(0, Number.isNaN(parseInt(document.getElementById('appSettingsConfigBackupKeepDays').value, 10)) ? 30 : parseInt(document.getElementById('appSettingsConfigBackupKeepDays').value, 10))), readMyMindEnabled: document.getElementById('appSettingsReadMyMind').checked, ultracodeFloatingWindows: document.getElementById('appSettingsUltracodeFloatingWindows').checked, diff --git a/src/web/schemas.ts b/src/web/schemas.ts index bf235521..4223310b 100644 --- a/src/web/schemas.ts +++ b/src/web/schemas.ts @@ -1530,7 +1530,7 @@ export const SettingsUpdateSchema = z /** Absolute folder (a leading `~` is expanded); empty or absent = the default folder. */ configBackupDir: z.string().max(1024).optional(), /** Keep at most this many snapshots. */ - configBackupKeepCount: z.number().int().min(1).max(500).optional(), + configBackupKeepCount: z.number().int().min(2).max(500).optional(), /** Also drop snapshots older than this many days; 0 = no age limit. */ configBackupKeepDays: z.number().int().min(0).max(3650).optional(), /** diff --git a/test/config-backup.test.ts b/test/config-backup.test.ts index 4e3ffcc7..bcd17295 100644 --- a/test/config-backup.test.ts +++ b/test/config-backup.test.ts @@ -60,7 +60,7 @@ describe('settings', () => { const s = readConfigBackupSettings({ configBackupKeepCount: 9999, configBackupKeepDays: -3 }); expect(s.keepCount).toBe(500); expect(s.keepDays).toBe(0); - expect(readConfigBackupSettings({ configBackupKeepCount: 0 }).keepCount).toBe(1); + expect(readConfigBackupSettings({ configBackupKeepCount: 0 }).keepCount).toBe(2); expect(readConfigBackupSettings({ configBackupKeepCount: 'abc' }).keepCount).toBe(20); expect(readConfigBackupSettings({ configBackupKeepCount: 7.9 }).keepCount).toBe(7); }); @@ -147,6 +147,28 @@ describe('createConfigBackup', () => { expect(readFileSync(join(backups, (third as { id: string }).id, 'custom-model-hosts.json'), 'utf8')).toBe(edited); }); + it.each([ + ['users.json', (n: number) => JSON.stringify({ users: [{ name: 'a', role: 'admin', lastLoginAt: n }] }), 'role'], + ['settings.json', (n: number) => JSON.stringify({ displayName: 'x', lastUsedCase: `case-${n}` }), 'displayName'], + [ + 'intents.json', + (n: number) => JSON.stringify({ profiles: [{ key: 'k', goals: 'g', updatedAt: n, recentPrompts: [`p${n}`] }] }), + 'goals', + ], + ])('does not count activity in %s as a change, but does count an edit next to it', (name, make, editKey) => { + put(name, make(1)); + const first = createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); + put(name, make(2)); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 1000 })).toEqual({ + status: 'unchanged', + id: (first as { id: string }).id, + }); + put(name, make(3).replace(new RegExp(`"${editKey}":"[^"]*"`), `"${editKey}":"edited"`)); + expect(createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 2000 }).status).toBe( + 'created' + ); + }); + it('compares a tracked file that is not valid JSON by its bytes', () => { put('custom-model-hosts.json', 'not json lastDiscoveredAt 1'); createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 }); @@ -217,10 +239,24 @@ describe('retention', () => { makeN(5); expect(pruneConfigBackups(backups, { keepCount: 50, keepDays: 0, now: T0 + 999 * DAY })).toEqual([]); const removed = pruneConfigBackups(backups, { keepCount: 4, keepDays: 3, now: T0 + 5 * DAY }); - // Newest first: T0+4d, +3d, +2d, +1d, T0. The oldest is over the count AND past the cutoff - // (T0+2d); T0+1d is past the cutoff; T0+2d sits exactly on it and is kept. - expect(removed.length).toBe(2); - expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([T0 + 4 * DAY, T0 + 3 * DAY, T0 + 2 * DAY]); + // Newest first: T0+4d, +3d, +2d, +1d, T0. A snapshot ages from its successor: T0+1d was live + // until T0+2d, which sits exactly on the cutoff (kept); only T0 is over the count. + expect(removed.length).toBe(1); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([ + T0 + 4 * DAY, + T0 + 3 * DAY, + T0 + 2 * DAY, + T0 + DAY, + ]); + }); + + it('keeps the last good copy when the first change comes after a quiet month', () => { + put('settings.json', '{"theme":"good"}'); + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'startup', now: T0 }); + put('settings.json', '{}'); // the bad write, 40 days later + createConfigBackup({ dataDir: data, backupDir: backups, reason: 'auto', now: T0 + 40 * DAY }); + pruneConfigBackups(backups, { keepCount: 20, keepDays: 30, now: T0 + 40 * DAY }); + expect(listConfigBackups(backups).map((b) => b.createdAt)).toEqual([T0 + 40 * DAY, T0]); }); it('never deletes a folder it did not make', () => {