mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-19 04:39:33 +01:00
feat(config-backup): compare backups with each other and with the running config
Settings -> System -> Config backups gains a Compare view: a backup against the running config, a backup against the one before it, or any two backups. Each differing file lists its changed keys as before/after, plus files that appeared or vanished. GET /api/config-backups/diff?from=&to= serves it (`to` defaults to `current`; admin only in multi-user mode). Credential values are never returned: .env, users.json and push-keys.json report names only, and any other key that looks like a token, secret, password, credential or key is reported as changed without its value. A non-JSON file reports only that it differs, a file is cut at 200 changes, and a self-refreshed discovery timestamp does not count as a difference.
This commit is contained in:
1 parent
06182c6fce
commit
a2e49eb26d
10 files changed
+426
-16
No files matched your search
@@ -2,4 +2,4 @@
|
||||
"aicodeman": minor
|
||||
---
|
||||
|
||||
Config backups: timestamped snapshots of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases and, in multi-user mode, accounts), taken at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder and the retention (keep N backups, drop older than N days) are configurable in Settings → System, with Back up now and Restore. Retention only deletes folders Codeman made, never the newest backup.
|
||||
Config backups: timestamped snapshots of the files that hold your customization (settings, CLI overrides, custom endpoints, webhook, linked cases and, in multi-user mode, accounts), taken at startup and whenever one changes. On by default into `~/.codeman/backups/config`; the folder and the retention (keep N backups, drop older than N days) are configurable in Settings → System, with Back up now and Restore. Retention only deletes folders Codeman made, never the newest backup. A comparison view shows what changed between two backups, or between a backup and the running config, without ever showing a credential's value.
|
||||
@@ -1160,11 +1160,12 @@ Tracked files (a fixed list, relative to the data dir): `settings.json`, `clis.j
|
||||
|
||||
Settings (synced, in `settings.json`): `configBackupEnabled` (absent = on), `configBackupDir` (absolute path, `~` expanded; empty = `<data dir>/backups/config`), `configBackupKeepCount` (1–500, default 20), `configBackupKeepDays` (0–3650, default 30; `0` = no age limit). Retention only ever deletes snapshot folders Codeman made there, and never the newest one. Each snapshot is a `0700` folder named `cfg-YYYYMMDD-HHMMSS` (server local time; `-2`, `-3`… on a same-second collision) holding the tracked files under their own names plus a `manifest.json` (`id`, `createdAt`, `reason` of `startup`/`auto`/`manual`/`pre-restore`, `appVersion`, and a SHA-256 and size per file), all `0600`. Only folders with that name and a valid manifest are listed, restored or pruned. Full guide: `docs/wiki/Config-Backups.md`.
|
||||
|
||||
| Method | Path | Body | Notes |
|
||||
| ------ | ----------------------------------- | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `GET` | `/api/config-backups` | none | `settings` in effect, `dir`, `defaultDir`, `tracked[]` and `backups[]` (`id`, `createdAt`, `reason`, `appVersion`, `files[]`, `bytes`), newest first. |
|
||||
| `POST` | `/api/config-backups` | none | Snapshot now, even if nothing changed (`reason: manual`), then apply retention. `status` is `created` or `empty` (nothing tracked exists yet). |
|
||||
| `POST` | `/api/config-backups/:id/restore` | none | Put a snapshot's files back, atomically, after a `pre-restore` snapshot of the current ones. `404` unknown id; `409 CONFLICT` if a file fails its checksum (nothing changed). `restartRequired[]` names files the running server only reloads on restart. |
|
||||
| Method | Path | Body | Notes |
|
||||
| ------ | ------------------------------------ | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `GET` | `/api/config-backups` | none | `settings` in effect, `dir`, `defaultDir`, `tracked[]` and `backups[]` (`id`, `createdAt`, `reason`, `appVersion`, `files[]`, `bytes`), newest first. |
|
||||
| `POST` | `/api/config-backups` | none | Snapshot now, even if nothing changed (`reason: manual`), then apply retention. `status` is `created` or `empty` (nothing tracked exists yet). |
|
||||
| `GET` | `/api/config-backups/diff?from=&to=` | none | What differs between two states: `from`/`to` are a backup id or `current` (the running config; the default for `to`). `files[]` (`name`, `status` `added`/`removed`/`changed`/`unchanged`, `changes[]` of `path`, `kind`, `before`/`after`, `truncated`, `detail`) and `changed`. Credential values are never returned (`redacted: true`). `400` without `from`; `404` unknown id; `409` if a file fails its checksum. |
|
||||
| `POST` | `/api/config-backups/:id/restore` | none | Put a snapshot's files back, atomically, after a `pre-restore` snapshot of the current ones. `404` unknown id; `409 CONFLICT` if a file fails its checksum (nothing changed). `restartRequired[]` names files the running server only reloads on restart. |
|
||||
|
||||
## Webhook notifications
|
||||
|
||||
|
||||
@@ -102,7 +102,8 @@ Both apply together. Three guarantees:
|
||||
| Keep this many backups | `configBackupKeepCount` | 20 |
|
||||
| Delete backups older than (days) | `configBackupKeepDays` | 30 |
|
||||
| **Back up now** | | |
|
||||
| The list, with **Restore** per entry | | |
|
||||
| The list, with **Restore**, **Compare with current** and **Changes in this backup** per entry | | |
|
||||
| **Compare** … with … **Show changes** | | |
|
||||
|
||||
The settings are saved server-side and read fresh, so a change applies on the next pass. The
|
||||
list and **Back up now** use the **saved** folder and retention: apply your edits first.
|
||||
@@ -127,6 +128,26 @@ You can also restore by hand: the files in a backup folder are the originals, so
|
||||
over the file in the data directory is equivalent (stop Codeman first if you are replacing
|
||||
`clis.json` or `users.json`).
|
||||
|
||||
## Comparing backups
|
||||
|
||||
Before you restore, or to find out what changed and when, use **Compare** in the list:
|
||||
|
||||
- **Compare with current** on a backup shows what differs between that backup and the config
|
||||
running now, which is also what a restore of it would undo.
|
||||
- **Changes in this backup** shows what differs between the backup before it and this one.
|
||||
- The **Compare … with … Show changes** pickers compare any two backups, or a backup with the
|
||||
current config.
|
||||
|
||||
Each file that differs is listed with its changed keys (`theme.mode`, `tags[1]`, a variable name
|
||||
in `.env`) as before → after, plus files that appeared or vanished between the two states. Files
|
||||
that are the same are listed on one line. A file that is not JSON shows only that it differs, and
|
||||
a file with more than 200 changes is cut short, so use the files themselves for a full diff. A
|
||||
self-refreshed discovery timestamp in `custom-model-hosts.json` is not counted as a change.
|
||||
|
||||
**Credential values are never shown.** `.env`, `users.json` and `push-keys.json` list only the
|
||||
names that changed, and in every other file a key that looks like a token, secret, password,
|
||||
credential or key is shown as "changed" without its value.
|
||||
|
||||
## Security
|
||||
|
||||
- Some tracked files carry credentials (`.env`, `users.json`, `custom-model-hosts.json`,
|
||||
@@ -135,9 +156,10 @@ over the file in the data directory is equivalent (stop Codeman first if you are
|
||||
- **A custom folder is your decision.** Putting it on a shared drive, a synced folder or a
|
||||
network share copies those credentials there, with whatever permissions that location has.
|
||||
Choose a private location, or turn the feature off if you cannot.
|
||||
- In [multi-user mode](Multi-User-Mode) all of it is admin only: the list, Back up now and
|
||||
Restore.
|
||||
- Listing returns file names and sizes, never contents.
|
||||
- In [multi-user mode](Multi-User-Mode) all of it is admin only: the list, Back up now,
|
||||
Restore and Compare.
|
||||
- Listing returns file names and sizes, never contents. A comparison returns changed values but
|
||||
never a credential's (see [Comparing backups](#comparing-backups)).
|
||||
|
||||
## HTTP API
|
||||
|
||||
@@ -146,6 +168,8 @@ API=http://localhost:3000
|
||||
curl -s "$API/api/config-backups" | jq # settings, folder, list
|
||||
curl -s -X POST "$API/api/config-backups" | jq # back up now
|
||||
curl -s -X POST "$API/api/config-backups/cfg-20261011-120305/restore" | jq
|
||||
curl -s "$API/api/config-backups/diff?from=cfg-20261011-120305" | jq # that backup vs the running config
|
||||
curl -s "$API/api/config-backups/diff?from=cfg-20261011-120305&to=cfg-20261012-090000" | jq
|
||||
```
|
||||
|
||||
See [HTTP API](HTTP-API) and the endpoint reference in `docs/api-reference.md`.
|
||||
|
||||
@@ -176,6 +176,7 @@ curl -s -X POST "$API/api/mcp-sync" | jq # apply it: add missing ser
|
||||
curl -s "$API/api/config-backups" | jq # config snapshots: folder, retention, list (admin only in multi-user mode)
|
||||
curl -s -X POST "$API/api/config-backups" | jq # back up the customized config files now
|
||||
curl -s -X POST "$API/api/config-backups/cfg-20261011-120000/restore" | jq # put a snapshot back (after a pre-restore copy)
|
||||
curl -s "$API/api/config-backups/diff?from=cfg-20261011-120000" | jq # what differs from the running config (`to=` another snapshot id to compare two)
|
||||
|
||||
# with ID set to a session id:
|
||||
curl -s "$API/api/sessions/$ID/last-response" | jq -r '.data.text' # last answer, from the transcript (claude, codex, deepseek)
|
||||
|
||||
@@ -398,3 +398,175 @@ export function restoreConfigBackup(opts: {
|
||||
}
|
||||
return { restored, safetySnapshot: safety.status === 'created' ? safety.id : null };
|
||||
}
|
||||
|
||||
/** The live files in the data dir, as a side of a diff. */
|
||||
export const CONFIG_DIFF_CURRENT = 'current';
|
||||
|
||||
export interface ConfigDiffChange {
|
||||
/** Dotted path inside the file (`a.b[0].c`), or the variable name for `.env`. */
|
||||
path: string;
|
||||
kind: 'added' | 'removed' | 'changed';
|
||||
/** JSON text of the value; absent when it did not exist on that side or `redacted` is set. */
|
||||
before?: string;
|
||||
after?: string;
|
||||
/** The value is a credential, so only the fact that it differs is reported. */
|
||||
redacted?: boolean;
|
||||
}
|
||||
|
||||
export interface ConfigDiffFile {
|
||||
name: string;
|
||||
/** `added` = only in `to`, `removed` = only in `from`. */
|
||||
status: 'added' | 'removed' | 'changed' | 'unchanged';
|
||||
changes: ConfigDiffChange[];
|
||||
/** More changes than `MAX_DIFF_CHANGES`; the rest are not listed. */
|
||||
truncated: boolean;
|
||||
/** `keys` = per-key changes; `bytes` = the file is not JSON/env, so only "differs" is known. */
|
||||
detail: 'keys' | 'bytes';
|
||||
}
|
||||
|
||||
export interface ConfigDiff {
|
||||
from: string;
|
||||
to: string;
|
||||
files: ConfigDiffFile[];
|
||||
/** Files whose status is not `unchanged`. */
|
||||
changed: number;
|
||||
}
|
||||
|
||||
const MAX_DIFF_CHANGES = 200;
|
||||
const MAX_DIFF_VALUE_CHARS = 200;
|
||||
|
||||
/** Files whose every value is a credential or key: only names are ever shown. */
|
||||
const REDACT_ALL_FILES: ReadonlySet<string> = new Set(['.env', 'users.json', 'push-keys.json']);
|
||||
/** Elsewhere a value is hidden when its own key looks like a credential. */
|
||||
const SECRET_KEY_RE = /(token|secret|passw|credential|auth|key$)/i;
|
||||
|
||||
type Leaves = Map<string, string>;
|
||||
|
||||
function flattenJson(value: unknown, path: string, out: Leaves): void {
|
||||
if (Array.isArray(value) && value.length > 0) {
|
||||
value.forEach((v, i) => flattenJson(v, `${path}[${i}]`, out));
|
||||
} else if (value && typeof value === 'object' && Object.keys(value).length > 0) {
|
||||
for (const [k, v] of Object.entries(value as Record<string, unknown>)) {
|
||||
flattenJson(v, path ? `${path}.${k}` : k, out);
|
||||
}
|
||||
} else {
|
||||
out.set(path || '(root)', JSON.stringify(value) ?? 'null');
|
||||
}
|
||||
}
|
||||
|
||||
function parseEnv(text: string): Leaves {
|
||||
const out: Leaves = new Map();
|
||||
for (const line of text.split(/\r?\n/)) {
|
||||
const m = /^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_.-]*)\s*=(.*)$/.exec(line);
|
||||
if (m) out.set(m[1], m[2].trim());
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Per-key view of a tracked file, or null when it is not JSON/env text. */
|
||||
function leavesOf(name: string, data: Buffer): Leaves | null {
|
||||
const text = data.toString('utf8');
|
||||
if (name === '.env') return parseEnv(text);
|
||||
try {
|
||||
const parsed = JSON.parse(text) as unknown;
|
||||
const volatile = VOLATILE_KEYS[name];
|
||||
const out: Leaves = new Map();
|
||||
flattenJson(volatile ? withoutKeys(parsed, new Set(volatile)) : parsed, '', out);
|
||||
return out;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const clip = (s: string): string => (s.length > MAX_DIFF_VALUE_CHARS ? `${s.slice(0, MAX_DIFF_VALUE_CHARS)}…` : s);
|
||||
|
||||
function diffLeaves(name: string, a: Leaves, b: Leaves): { changes: ConfigDiffChange[]; truncated: boolean } {
|
||||
const all = REDACT_ALL_FILES.has(name);
|
||||
const changes: ConfigDiffChange[] = [];
|
||||
for (const path of [...new Set([...a.keys(), ...b.keys()])].sort()) {
|
||||
const before = a.get(path);
|
||||
const after = b.get(path);
|
||||
if (before === after) continue;
|
||||
const kind = before === undefined ? 'added' : after === undefined ? 'removed' : 'changed';
|
||||
if (all || SECRET_KEY_RE.test(path.split('.').pop() ?? '')) {
|
||||
changes.push({ path, kind, redacted: true });
|
||||
} else {
|
||||
changes.push({
|
||||
path,
|
||||
kind,
|
||||
...(before !== undefined && { before: clip(before) }),
|
||||
...(after !== undefined && { after: clip(after) }),
|
||||
});
|
||||
}
|
||||
}
|
||||
return { changes: changes.slice(0, MAX_DIFF_CHANGES), truncated: changes.length > MAX_DIFF_CHANGES };
|
||||
}
|
||||
|
||||
/** The tracked files of one side, verified: a snapshot's against its manifest, the live ones as they are. */
|
||||
function loadDiffSide(side: string, dataDir: string, backupDir: string): Map<string, Buffer> {
|
||||
const out = new Map<string, Buffer>();
|
||||
if (side === CONFIG_DIFF_CURRENT) {
|
||||
for (const name of CONFIG_BACKUP_FILES) {
|
||||
try {
|
||||
const path = join(dataDir, name);
|
||||
if (statSync(path).isFile()) out.set(name, readFileSync(path));
|
||||
} catch {
|
||||
/* absent or unreadable: not part of this side */
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
const manifest = readManifest(backupDir, side);
|
||||
if (!manifest) throw new ConfigBackupNotFoundError(`No config backup named ${side}`);
|
||||
const allowed = new Set<string>(CONFIG_BACKUP_FILES);
|
||||
for (const f of manifest.files) {
|
||||
if (!allowed.has(f.name)) continue;
|
||||
let data: Buffer;
|
||||
try {
|
||||
data = readFileSync(join(backupDir, side, f.name));
|
||||
} catch {
|
||||
throw new ConfigBackupCorruptError(`${f.name} is missing from ${side}`);
|
||||
}
|
||||
if (sha256(data) !== f.sha256)
|
||||
throw new ConfigBackupCorruptError(`${f.name} in ${side} does not match its checksum`);
|
||||
out.set(f.name, data);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* What differs between two states of the tracked files; a state is a snapshot id or `current`
|
||||
* (the live files). Never returns the value of a credential: see REDACT_ALL_FILES and SECRET_KEY_RE.
|
||||
*/
|
||||
export function diffConfigBackups(opts: { dataDir: string; backupDir: string; from: string; to: string }): ConfigDiff {
|
||||
const a = loadDiffSide(opts.from, opts.dataDir, opts.backupDir);
|
||||
const b = loadDiffSide(opts.to, opts.dataDir, opts.backupDir);
|
||||
const files: ConfigDiffFile[] = [];
|
||||
for (const name of CONFIG_BACKUP_FILES) {
|
||||
const before = a.get(name);
|
||||
const after = b.get(name);
|
||||
if (!before && !after) continue;
|
||||
const base = { name, changes: [] as ConfigDiffChange[], truncated: false };
|
||||
if (before && after && sha256(before) === sha256(after)) {
|
||||
files.push({ ...base, status: 'unchanged', detail: 'keys' });
|
||||
continue;
|
||||
}
|
||||
const la = before ? leavesOf(name, before) : new Map<string, string>();
|
||||
const lb = after ? leavesOf(name, after) : new Map<string, string>();
|
||||
const status = !before ? 'added' : !after ? 'removed' : 'changed';
|
||||
if (!la || !lb) {
|
||||
files.push({ ...base, status, detail: 'bytes' });
|
||||
continue;
|
||||
}
|
||||
const { changes, truncated } = diffLeaves(name, la, lb);
|
||||
// Only volatile keys differed (a discovery timestamp): the same state as far as the user is concerned.
|
||||
files.push({
|
||||
name,
|
||||
status: changes.length || status !== 'changed' ? status : 'unchanged',
|
||||
changes,
|
||||
truncated,
|
||||
detail: 'keys',
|
||||
});
|
||||
}
|
||||
return { from: opts.from, to: opts.to, files, changed: files.filter((f) => f.status !== 'unchanged').length };
|
||||
}
|
||||
@@ -3276,7 +3276,7 @@
|
||||
<div class="set-row" data-search="config backup now restore list">
|
||||
<div class="set-row-text">
|
||||
<span class="set-row-label">Backups</span>
|
||||
<span class="set-row-desc">Folder and retention changes apply after you save. Restoring puts a backup's files back and makes a copy of the current ones first.</span>
|
||||
<span class="set-row-desc">Folder and retention changes apply after you save. Restoring puts a backup's files back and makes a copy of the current ones first. Compare shows what changed between two backups, or between a backup and the running config; credential values are never shown.</span>
|
||||
</div>
|
||||
<button class="btn-toolbar btn-sm btn-primary" id="configBackupNowBtn" onclick="app.backupConfigNow()">Back up now</button>
|
||||
</div>
|
||||
|
||||
@@ -1393,14 +1393,72 @@ Object.assign(CodemanApp.prototype, {
|
||||
return;
|
||||
}
|
||||
const kb = (n) => (n < 1024 ? `${n} B` : `${(n / 1024).toFixed(1)} KB`);
|
||||
const rows = data.backups.map((b) => {
|
||||
this._configBackups = data.backups;
|
||||
const rows = data.backups.map((b, i) => {
|
||||
const when = new Date(b.createdAt).toLocaleString();
|
||||
const names = b.files.map((f) => escapeHtml(f.name)).join(', ');
|
||||
const id = escapeHtml(b.id);
|
||||
// The list is newest first, so the next row is the backup this one followed.
|
||||
const prev = data.backups[i + 1];
|
||||
const vsPrev = prev ? `<button class="btn-toolbar btn-sm" onclick="app.showConfigDiff('${escapeHtml(prev.id)}','${id}')">Changes in this backup</button>` : '';
|
||||
return `<li><b>${escapeHtml(when)}</b> <span class="set-row-desc">(${escapeHtml(b.reason)}, ${kb(b.bytes)})</span>
|
||||
<button class="btn-toolbar btn-sm" onclick="app.restoreConfigBackup('${escapeHtml(b.id)}')">Restore</button>
|
||||
<button class="btn-toolbar btn-sm" onclick="app.restoreConfigBackup('${id}')">Restore</button>
|
||||
<button class="btn-toolbar btn-sm" onclick="app.showConfigDiff('${id}','current')">Compare with current</button>
|
||||
${vsPrev}
|
||||
<div class="set-row-desc">${names}</div></li>`;
|
||||
});
|
||||
out.innerHTML = `<div>${data.backups.length} backup${data.backups.length === 1 ? '' : 's'} in <code>${escapeHtml(data.dir)}</code></div><ul>${rows.join('')}</ul>`;
|
||||
const options = (sel) => [`<option value="current"${sel === 'current' ? ' selected' : ''}>Current config</option>`]
|
||||
.concat(data.backups.map((b) => `<option value="${escapeHtml(b.id)}"${sel === b.id ? ' selected' : ''}>${escapeHtml(new Date(b.createdAt).toLocaleString())} (${escapeHtml(b.reason)})</option>`))
|
||||
.join('');
|
||||
const picker = data.backups.length
|
||||
? `<div class="set-row-desc">Compare <select id="configDiffFrom">${options(data.backups[0].id)}</select> with <select id="configDiffTo">${options('current')}</select>
|
||||
<button class="btn-toolbar btn-sm" onclick="app.showConfigDiff(app.$('configDiffFrom').value, app.$('configDiffTo').value)">Show changes</button></div>`
|
||||
: '';
|
||||
out.innerHTML = `<div>${data.backups.length} backup${data.backups.length === 1 ? '' : 's'} in <code>${escapeHtml(data.dir)}</code></div>${picker}<ul>${rows.join('')}</ul><div id="configBackupDiff"></div>`;
|
||||
},
|
||||
|
||||
/** What differs between two states of the config files: a backup id or `current` (the running config). */
|
||||
async showConfigDiff(from, to) {
|
||||
const out = this.$('configBackupDiff');
|
||||
if (!out) return;
|
||||
out.textContent = 'Comparing…';
|
||||
const res = await this._api(`/api/config-backups/diff?from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}`);
|
||||
let body = null;
|
||||
try { body = res ? await res.json() : null; } catch { /* fall through */ }
|
||||
if (!res || !res.ok || !body || body.success === false) {
|
||||
out.textContent = body?.error || 'Could not compare these';
|
||||
return;
|
||||
}
|
||||
const label = (id) => {
|
||||
if (id === 'current') return 'the current config';
|
||||
const b = (this._configBackups || []).find((x) => x.id === id);
|
||||
return b ? new Date(b.createdAt).toLocaleString() : id;
|
||||
};
|
||||
const d = body.data;
|
||||
const verb = { added: 'added', removed: 'removed', changed: 'changed' };
|
||||
const files = d.files.filter((f) => f.status !== 'unchanged').map((f) => {
|
||||
let detail;
|
||||
if (f.detail === 'bytes') {
|
||||
detail = '<div class="set-row-desc">Differs (not a key/value file, so no detail).</div>';
|
||||
} else {
|
||||
const rows = f.changes.map((c) => {
|
||||
const val = (v) => `<code>${escapeHtml(v ?? '')}</code>`;
|
||||
const text = c.redacted ? `${verb[c.kind]} <span class="set-row-desc">(value hidden: credential)</span>`
|
||||
: c.kind === 'added' ? `added ${val(c.after)}`
|
||||
: c.kind === 'removed' ? `removed (was ${val(c.before)})`
|
||||
: `${val(c.before)} → ${val(c.after)}`;
|
||||
return `<li><code>${escapeHtml(c.path)}</code>: ${text}</li>`;
|
||||
});
|
||||
const more = f.truncated ? '<li class="set-row-desc">More changes not shown.</li>' : '';
|
||||
detail = f.changes.length ? `<ul>${rows.join('')}${more}</ul>` : '';
|
||||
}
|
||||
const whole = f.status === 'added' ? ' (new file)' : f.status === 'removed' ? ' (file gone)' : '';
|
||||
return `<div><b>${escapeHtml(f.name)}</b>${whole}${detail}</div>`;
|
||||
});
|
||||
const same = d.files.filter((f) => f.status === 'unchanged').map((f) => escapeHtml(f.name));
|
||||
out.innerHTML = `<div><b>${escapeHtml(label(d.from))}</b> → <b>${escapeHtml(label(d.to))}</b>: ${
|
||||
d.changed ? `${d.changed} file${d.changed === 1 ? '' : 's'} differ` : 'no differences'}</div>${files.join('')}${
|
||||
same.length ? `<div class="set-row-desc">Unchanged: ${same.join(', ')}</div>` : ''}`;
|
||||
},
|
||||
|
||||
async backupConfigNow() {
|
||||
|
||||
@@ -4,10 +4,14 @@
|
||||
* GET /api/config-backups — settings in effect, the folder, and the snapshots in it.
|
||||
* POST /api/config-backups — snapshot now (a manual request always makes one).
|
||||
* POST /api/config-backups/:id/restore — put a snapshot's files back (after a `pre-restore` snapshot).
|
||||
* GET /api/config-backups/diff?from=&to= — what differs between two snapshots, or a snapshot and the
|
||||
* running config (`current`, the default for `to`).
|
||||
*
|
||||
* The files include credentials (`.env`, `users.json`, endpoint keys), so in multi-user mode all three
|
||||
* are admin only. Responses carry file NAMES and sizes, never file content. Restore replaces files the
|
||||
* running server may have cached: `restartRequired` says which ones only apply after a restart.
|
||||
* are admin only. Responses carry file NAMES and sizes, never file content, except the diff, which shows
|
||||
* changed values but never a credential's (`.env`, `users.json`, `push-keys.json` and any key that looks
|
||||
* like a token/secret/password are reported as "changed" only). Restore replaces files the running server
|
||||
* may have cached: `restartRequired` says which ones only apply after a restart.
|
||||
*/
|
||||
|
||||
import { createRequire } from 'node:module';
|
||||
@@ -18,10 +22,12 @@ import { isMultiUserMode } from '../../config/multiuser.js';
|
||||
import { getDataDir } from '../../config/instance.js';
|
||||
import {
|
||||
CONFIG_BACKUP_FILES,
|
||||
CONFIG_DIFF_CURRENT,
|
||||
ConfigBackupCorruptError,
|
||||
ConfigBackupNotFoundError,
|
||||
createConfigBackup,
|
||||
defaultConfigBackupDir,
|
||||
diffConfigBackups,
|
||||
listConfigBackups,
|
||||
pruneConfigBackups,
|
||||
readConfigBackupSettings,
|
||||
@@ -30,6 +36,7 @@ import {
|
||||
type ConfigBackupResult,
|
||||
type ConfigBackupSettings,
|
||||
type ConfigBackupSummary,
|
||||
type ConfigDiff,
|
||||
} from '../../config-backup.js';
|
||||
|
||||
// Only recorded in each manifest; a missing package.json just leaves it blank.
|
||||
@@ -113,6 +120,37 @@ export function registerConfigBackupRoutes(app: FastifyInstance): void {
|
||||
}
|
||||
});
|
||||
|
||||
app.get('/api/config-backups/diff', async (req, reply): Promise<ApiResponse<ConfigDiff>> => {
|
||||
const denied = deny(req, reply);
|
||||
if (denied) return denied;
|
||||
const q = req.query as { from?: string; to?: string };
|
||||
if (typeof q.from !== 'string' || !q.from) {
|
||||
reply.code(400);
|
||||
return createErrorResponse(ApiErrorCode.INVALID_INPUT, '`from` is required (a backup id or "current")');
|
||||
}
|
||||
try {
|
||||
const { dir } = await readEffectiveBackupSettings();
|
||||
const data = diffConfigBackups({
|
||||
dataDir: getDataDir(),
|
||||
backupDir: dir,
|
||||
from: q.from,
|
||||
to: typeof q.to === 'string' && q.to ? q.to : CONFIG_DIFF_CURRENT,
|
||||
});
|
||||
return { success: true, data };
|
||||
} catch (err) {
|
||||
if (err instanceof ConfigBackupNotFoundError) {
|
||||
reply.code(404);
|
||||
return createErrorResponse(ApiErrorCode.NOT_FOUND, err.message);
|
||||
}
|
||||
if (err instanceof ConfigBackupCorruptError) {
|
||||
reply.code(409);
|
||||
return createErrorResponse(ApiErrorCode.CONFLICT, err.message);
|
||||
}
|
||||
reply.code(500);
|
||||
return createErrorResponse(ApiErrorCode.OPERATION_FAILED, `Diff failed: ${getErrorMessage(err)}`);
|
||||
}
|
||||
});
|
||||
|
||||
app.post('/api/config-backups/:id/restore', async (req, reply) => {
|
||||
const denied = deny(req, reply);
|
||||
if (denied) return denied;
|
||||
|
||||
@@ -23,6 +23,7 @@ import {
|
||||
ConfigBackupNotFoundError,
|
||||
createConfigBackup,
|
||||
defaultConfigBackupDir,
|
||||
diffConfigBackups,
|
||||
listConfigBackups,
|
||||
normalizeBackupDir,
|
||||
pruneConfigBackups,
|
||||
@@ -295,3 +296,95 @@ describe('restoreConfigBackup', () => {
|
||||
expect(existsSync(join(data, 'state.json'))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('diffConfigBackups', () => {
|
||||
const snap = (now: number) => {
|
||||
const r = createConfigBackup({ dataDir: data, backupDir: backups, reason: 't', force: true, now });
|
||||
if (r.status !== 'created') throw new Error('expected a snapshot');
|
||||
return r.id;
|
||||
};
|
||||
const diff = (from: string, to: string) => diffConfigBackups({ dataDir: data, backupDir: backups, from, to });
|
||||
const file = (d: ReturnType<typeof diff>, name: string) => d.files.find((f) => f.name === name)!;
|
||||
|
||||
it('lists key-level changes between two backups', () => {
|
||||
put('settings.json', JSON.stringify({ displayName: 'a', theme: { mode: 'dark' }, tags: ['x'] }));
|
||||
const a = snap(T0);
|
||||
put('settings.json', JSON.stringify({ displayName: 'b', theme: { mode: 'dark' }, tags: ['x', 'y'], extra: 1 }));
|
||||
const b = snap(T0 + 1000);
|
||||
const f = file(diff(a, b), 'settings.json');
|
||||
expect(f.status).toBe('changed');
|
||||
expect(f.changes).toEqual([
|
||||
{ path: 'displayName', kind: 'changed', before: '"a"', after: '"b"' },
|
||||
{ path: 'extra', kind: 'added', after: '1' },
|
||||
{ path: 'tags[1]', kind: 'added', after: '"y"' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('compares a backup with the running config, and reports files that appeared or vanished', () => {
|
||||
put('settings.json', '{"a":1}');
|
||||
put('webhook.json', '{"url":"u"}');
|
||||
const a = snap(T0);
|
||||
put('settings.json', '{"a":2}');
|
||||
rmSync(join(data, 'webhook.json'));
|
||||
put('intents.json', '{"i":true}');
|
||||
const d = diff(a, 'current');
|
||||
expect(d.changed).toBe(3);
|
||||
expect(file(d, 'settings.json').changes).toEqual([{ path: 'a', kind: 'changed', before: '1', after: '2' }]);
|
||||
expect(file(d, 'webhook.json').status).toBe('removed');
|
||||
expect(file(d, 'intents.json').status).toBe('added');
|
||||
expect(diff(a, a).changed).toBe(0);
|
||||
});
|
||||
|
||||
it('never returns a credential value, only that it differs', () => {
|
||||
put('.env', 'SECRET_ONE=hunter2\nOTHER=abc\n');
|
||||
put('users.json', JSON.stringify({ admin: { name: 'root' } }));
|
||||
put('custom-model-hosts.json', JSON.stringify([{ name: 'h', apiKey: 'sk-old', url: 'http://x' }]));
|
||||
const a = snap(T0);
|
||||
put('.env', 'SECRET_ONE=hunter3\nOTHER=abd\n');
|
||||
put('users.json', JSON.stringify({ admin: { name: 'rooted' } }));
|
||||
put('custom-model-hosts.json', JSON.stringify([{ name: 'h', apiKey: 'sk-new', url: 'http://y' }]));
|
||||
const d = diff(a, 'current');
|
||||
const json = JSON.stringify(d);
|
||||
for (const secret of ['hunter', 'abc', 'abd', 'root', 'sk-old', 'sk-new']) expect(json).not.toContain(secret);
|
||||
expect(file(d, '.env').changes.every((c) => c.redacted)).toBe(true);
|
||||
expect(file(d, 'custom-model-hosts.json').changes).toEqual([
|
||||
{ path: '[0].apiKey', kind: 'changed', redacted: true },
|
||||
{ path: '[0].url', kind: 'changed', before: '"http://x"', after: '"http://y"' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('does not count a self-refreshed discovery timestamp as a difference', () => {
|
||||
put('custom-model-hosts.json', JSON.stringify([{ name: 'h', lastDiscoveredAt: 1 }]));
|
||||
const a = snap(T0);
|
||||
put('custom-model-hosts.json', JSON.stringify([{ name: 'h', lastDiscoveredAt: 2 }]));
|
||||
const d = diff(a, 'current');
|
||||
expect(file(d, 'custom-model-hosts.json').status).toBe('unchanged');
|
||||
expect(d.changed).toBe(0);
|
||||
});
|
||||
|
||||
it('reports a file that is not JSON as differing, without its content', () => {
|
||||
put('settings.json', 'not json at all');
|
||||
const a = snap(T0);
|
||||
put('settings.json', 'still not json');
|
||||
const f = file(diff(a, 'current'), 'settings.json');
|
||||
expect(f).toMatchObject({ status: 'changed', detail: 'bytes', changes: [] });
|
||||
});
|
||||
|
||||
it('caps the changes listed per file', () => {
|
||||
put('settings.json', '{}');
|
||||
const a = snap(T0);
|
||||
put('settings.json', JSON.stringify(Object.fromEntries(Array.from({ length: 250 }, (_, i) => [`k${i}`, i]))));
|
||||
const f = file(diff(a, 'current'), 'settings.json');
|
||||
expect(f.changes).toHaveLength(200);
|
||||
expect(f.truncated).toBe(true);
|
||||
});
|
||||
|
||||
it('refuses unknown ids, path traversal and a tampered snapshot', () => {
|
||||
put('settings.json', '{"a":1}');
|
||||
const a = snap(T0);
|
||||
expect(() => diff('cfg-20200101-000000', 'current')).toThrow(ConfigBackupNotFoundError);
|
||||
expect(() => diff('../etc', 'current')).toThrow(ConfigBackupNotFoundError);
|
||||
writeFileSync(join(backups, a, 'settings.json'), '{"a":99}');
|
||||
expect(() => diff(a, 'current')).toThrow(ConfigBackupCorruptError);
|
||||
});
|
||||
});
|
||||
@@ -106,6 +106,7 @@ describe('/api/config-backups', () => {
|
||||
['GET', '/api/config-backups'],
|
||||
['POST', '/api/config-backups'],
|
||||
['POST', '/api/config-backups/cfg-20261001-120000/restore'],
|
||||
['GET', '/api/config-backups/diff?from=cfg-20261001-120000'],
|
||||
] as const) {
|
||||
expect((await app.inject({ method, url })).statusCode, `${method} ${url}`).toBe(403);
|
||||
}
|
||||
@@ -119,3 +120,25 @@ describe('/api/config-backups', () => {
|
||||
expect((await app.inject({ method: 'GET', url: '/api/config-backups' })).json().success).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GET /api/config-backups/diff', () => {
|
||||
it('compares a backup with the running config, defaulting `to` to current', async () => {
|
||||
const { app } = await createRouteTestHarness(registerConfigBackupRoutes);
|
||||
const made = (await app.inject({ method: 'POST', url: '/api/config-backups' })).json().data;
|
||||
settings({ displayName: 'after' });
|
||||
const res = await app.inject({ method: 'GET', url: `/api/config-backups/diff?from=${made.id}` });
|
||||
expect(res.statusCode).toBe(200);
|
||||
const d = res.json().data;
|
||||
expect(d.to).toBe('current');
|
||||
expect(d.files.find((f: { name: string }) => f.name === 'settings.json').changes).toEqual([
|
||||
{ path: 'displayName', kind: 'changed', before: '"before"', after: '"after"' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('400 without `from`, 404 for an unknown backup', async () => {
|
||||
const { app } = await createRouteTestHarness(registerConfigBackupRoutes);
|
||||
expect((await app.inject({ method: 'GET', url: '/api/config-backups/diff' })).statusCode).toBe(400);
|
||||
const missing = await app.inject({ method: 'GET', url: '/api/config-backups/diff?from=cfg-20200101-000000' });
|
||||
expect(missing.statusCode).toBe(404);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user