diff --git a/CLAUDE.md b/CLAUDE.md index 42c8f0bd..fb3fa661 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -134,7 +134,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Agent CLI (`codeman agent …`, `src/cli-agent.ts`)**: the command-line client for the wait primitives and the session verbs, for agents the skill does not reach (pi, shell, no `jq`; `seedAgentSessionPreamble` covers every local session whose CLI takes the skill, `capabilities.agentSkillDir`; every pane still exports `CODEMAN_MUX`/`CODEMAN_SESSION_ID`/`CODEMAN_API_URL`). It is an HTTP CLIENT: inspection and guarded stage/submit use the ownership-checked `agent-session-routes.ts` module (experimental, behind `agentCoordinationEnabled`, SYNCED, default OFF: all three routes answer `403 FORBIDDEN` naming it while off, and the CLI shows that verbatim); the other verbs reuse existing routes, with no second transport (the plan's "Socket API" non-goal stands), every call carries `X-Codeman-Parent-Session`, only `spawn`'s quick-start carries `X-Codeman-Agent-Origin` (pinned in test/cli-agent.test.ts), while `spawn` without a case (cwd or `--working-dir`) is `POST /api/v1/sessions` plus `/interactive` or `/shell` without that header, so no agent-case marker, and refuses (exit 4, before any request) an empty `--working-dir` and a bare spawn from a HOME cwd, since it always sends an explicit `workingDir` and the server's no-`workingDir` hooks skip cannot apply (test/cli-agent-project-spawn.test.ts), auth (`src/codeman-credentials.ts`, shared with `codeman attach` and the TUI) from `CODEMAN_PASSWORD` or the data dir's `.env` (the same reader `codeman attach` uses — `readCodemanEnvFile`). Verbs return exit codes (`0` ok / `1` error / `2` timeout / `3` exited / `4` refused) instead of calling `process.exit`, so `test/cli-agent.test.ts` drives every verb against a recording fake transport plus the real `httpRequest` against a local server. ⚠️ Three guards live in code, not prose, and the tests pin them: refuse without `CODEMAN_MUX=1`+`CODEMAN_API_URL` (never guess a URL); `send` transmits printable text + `\r` only (`inputRefusal`: any byte < 0x20 or DEL refuses BEFORE the transport — opencode's `Ctrl+C` is `app_exit`), ESC exists solely as `interrupt` with no `\r`; `rm` fails closed (`deleteRefusal`: empty id, self shorter than 8, prefix match in EITHER direction, re-checked on the resolved id). ⚠️ The routes accept FULL ids only — an 8-char prefix is a 404 (measured live) — while `ls` prints prefixes, so `resolveSessionId` maps a short id through `GET /api/v1/sessions` and refuses an ambiguous one. It also refuses any id under 8 characters (exit 4, on every verb, before any request), the same floor as the server's `PARENT_SESSION_ID_MIN_PREFIX`, so `rm 9` can never pick an arbitrary session. ⚠️ A 400 for `--until stop` on a hook-less mode is passed through verbatim, never papered over: the marker path is the answer there. `spawn` on a readiness timeout exits 2 and LEAVES the session for inspection (the skill's `spawn_worker` deletes it; the CLI's caller is often a human); its stdout is the id ALONE (prose on stderr) so `SID=$(…)` works. ⚠️ Two server answers that look like success and are not: `delivered:false` without `duplicate` (the bytes went into a dead pane — the field exists so a client says "restart" instead of "wait longer") and `wait.ended:true, signal:null` (the worker died during a `stop`/`idle` wait: the registry satisfies only waiters that listed `exit`, then cancels the rest) — both exit 3, checked BEFORE the happy paths in `waitExitCode`. → README "`codeman agent`", `docs/api-reference.md` -**Discord bridge (`src/discord-bridge.ts`, `src/web/routes/discord-routes.ts`, `src/cli-notify.ts`)**: opt-in two-way Discord (a bot, not the one-way webhook), one thread per tab. Split like `webhook-notify.ts`: pure helpers, a 0600 store (`discord.json` holds the bot token, is never returned by these routes and is on the sensitive-path list; `discord-bridge.json` maps sessionId to thread id), then IO (`DiscordApi` REST with injected fetch, `DiscordGateway` inbound WebSocket with injected socket, `DiscordBridge` rules). Outbound hangs off `WebServer.sendPushNotifications` beside the webhook (so it works with zero push subscriptions) and tab events off `WebServer.broadcast` (`bridgeSessionEvent`); a finished turn (Claude's idle and stop hooks, DeepSeek's stop; the other CLIs raise no turn end, so theirs arrive with an error, a blocked respawn, a crash loop or `codeman notify`) or an error posts the CLI's WHOLE last message as markdown, read from its transcript through the response viewer's reader (`WebServer.lastResponseFor`, set by the session routes: the CLI's own transcript, or the pane segmented into turns for the CLIs that keep none; never for a shell), split over posts of at most 1900 characters and not posted twice (keyed on the text plus its turn, recorded only after the first post succeeds), while a dialog and every CLI without a transcript post the RENDERED tmux screen (`screenTail`, with the composer row and the status lines under it cut), never the raw stream, which a full-screen UI repaints in place; `staleThreads` (default `keep`) archives or deletes threads the bot owns that no tab maps to, after boot and then daily, never one under an hour old, never with an empty map and at most 25 per run. Inbound replies are typed through `session.writeViaMux`. ⚠️ Guards that live in code and tests pin: only `allowedUserIds`, in `guildId`, inside a thread the bridge created, are acted on, everything else is dropped unread; a reply is one printable line plus Enter (a lone digit while a permission or question is pending goes as the key alone, since the dialog acts on it), capped and rate limited, and `discord-notify` is a per-session token bucket (6 a minute, 429 `RATE_LIMITED`; `fileProblem` probes the folder with the bounded probe first); posts send `allowed_mentions` limited to the allowlisted ids; received files go only from Discord's own HTTPS hosts (max 5 x 8 MB, names reduced to `[A-Za-z0-9._-]`) into `/.codeman-discord/`, and a sent file's REAL path must lie inside the tab's folder (`fileProblem`, symlinks followed); a Discord failure never reaches a session; the alert tail is a function the bridge calls only once it will post (no tmux exec per event while off); a delete reaches the bridge from the cleanup path with `killMux` true, never from `session:deleted`, which a detach also fires; `.codeman-discord/` is probed with the bounded probe, refused if a symlink, and ignores itself with a `.gitignore` of `*`; a config restore of `discord.json` reloads the running bridge (`afterRestore`). The startup hint ("how to reach the user") rides the CLI registry as the engine value `appendSystemPrompt` (Claude's `--append-system-prompt`, undefined unless the bridge and `startupHint` are on, so every other launch is byte-identical) No other CLI is told. `codeman notify "" [--urgent] [--file ]` wraps `POST /api/sessions/:id/discord-notify`, which accepts a unique 8-character id prefix (a Docker case exports it cut). → `docs/discord-bridge.md`, `docs/wiki/Discord-Bridge.md`, `docs/api-reference.md` +**Discord bridge (`src/discord-bridge.ts`, `src/web/routes/discord-routes.ts`, `src/cli-notify.ts`)**: opt-in two-way Discord (a bot, not the one-way webhook), one thread per tab. Split like `webhook-notify.ts`: pure helpers, a 0600 store (`discord.json` holds the bot token, is never returned by these routes and is on the sensitive-path list; `discord-bridge.json` maps sessionId to thread id), then IO (`DiscordApi` REST with injected fetch, `DiscordGateway` inbound WebSocket with injected socket, `DiscordBridge` rules). Outbound hangs off `WebServer.sendPushNotifications` beside the webhook (so it works with zero push subscriptions) and tab events off `WebServer.broadcast` (`bridgeSessionEvent`); a finished turn (Claude's idle and stop hooks, DeepSeek's stop; the other CLIs raise no turn end, so theirs arrive with an error, a blocked respawn, a crash loop or `codeman notify`) or an error posts the CLI's WHOLE last message as markdown, read from its transcript through the response viewer's reader (`WebServer.lastResponseFor`, set by the session routes: the CLI's own transcript, or the pane segmented into turns for the CLIs that keep none; never for a shell), split over posts of at most 1900 characters and not posted twice (keyed on the text plus its turn, recorded only after the first post succeeds), while a dialog and every CLI without a transcript post the RENDERED tmux screen (`screenTail`, with the composer row and the status lines under it cut), never the raw stream, which a full-screen UI repaints in place; `staleThreads` (default `keep`) archives or deletes threads the bot owns that no tab maps to, after boot and then daily, never one under an hour old, never with an empty map and at most 25 per run. Inbound replies are typed through `session.writeViaMux`. An allowed reply is claimed by message ID before rate-limit or exited-agent checks, attachments or typing; a store failure never costs a reply. A file that could not be read is never overwritten. ⚠️ Guards that live in code and tests pin: only `allowedUserIds`, in `guildId`, inside a thread the bridge created, are acted on, everything else is dropped unread; a reply is one printable line plus Enter (a lone digit while a permission or question is pending goes as the key alone, since the dialog acts on it), capped and rate limited, and `discord-notify` is a per-session token bucket (6 a minute, 429 `RATE_LIMITED`; `fileProblem` probes the folder with the bounded probe first); posts send `allowed_mentions` limited to the allowlisted ids; received files go only from Discord's own HTTPS hosts (max 5 x 8 MB, names reduced to `[A-Za-z0-9._-]`) into `/.codeman-discord/`, and a sent file's REAL path must lie inside the tab's folder (`fileProblem`, symlinks followed); a Discord failure never reaches a session; the alert tail is a function the bridge calls only once it will post (no tmux exec per event while off); a delete reaches the bridge from the cleanup path with `killMux` true, never from `session:deleted`, which a detach also fires; `.codeman-discord/` is probed with the bounded probe, refused if a symlink, and ignores itself with a `.gitignore` of `*`; a config restore of `discord.json` reloads the running bridge (`afterRestore`). The startup hint ("how to reach the user") rides the CLI registry as the engine value `appendSystemPrompt` (Claude's `--append-system-prompt`, undefined unless the bridge and `startupHint` are on, so every other launch is byte-identical) No other CLI is told. `codeman notify "" [--urgent] [--file ]` wraps `POST /api/sessions/:id/discord-notify`, which accepts a unique 8-character id prefix (a Docker case exports it cut). → `docs/discord-bridge.md`, `docs/wiki/Discord-Bridge.md`, `docs/api-reference.md` **MCP server (`codeman mcp`, `src/mcp-server.ts`)**: a Model Context Protocol server on stdio for any MCP client, a CLIENT of the HTTP API like `codeman agent` (reuses its `httpRequest`, `inputRefusal`, `buildSendBody`; the MCP server adds no routes). Seven tools: `list_sessions`, `start_session`, `send_prompt`, `wait_for_session`, `read_output`, `interrupt_session`, `delete_session`. ⚠️ Dual-era: answers the legacy `initialize` handshake (2024-11-05 to 2025-11-25) AND the 2026-07-28 revision (per-request `_meta` with `protocolVersion` + `clientCapabilities`, `server/discover`, `resultType: "complete"` on every modern result); a modern `tools/list`/`server/discover` MUST carry `ttlMs` + `cacheScope` (Claude Code 2.1.296 drops every tool without them: measured). ⚠️ stdout is protocol only; logs go to stderr. ⚠️ The URL comes from `--url`/`CODEMAN_API_URL`, never a guess, and unconfigured it still lists tools (registries introspect) while every call answers how to configure it. ⚠️ Prompt/interrupt/delete only sessions the PROCESS started (in-memory allowlist, so a restart fails closed) or, inside a pane, the calling session's own children, unless `--allow-any-session`; never its own session. ⚠️ `read_output` returns text ONLY, no `structuredContent` (a client that prefers it showed the model metadata instead of the output: measured). Waits default 45 s (Codex's MCP tool timeout is 60 s). Registry manifest `config/mcp-registry/server.json` (`mcpName` in package.json, version kept by `scripts/sync-plugin.mjs`). Tests: `test/mcp-server.test.ts`. @@ -336,7 +336,7 @@ One module per domain in `src/web/routes/` (plus a barrel; `ls src/web/routes/` ## State Files -All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id), `local-model.json` (local model install: state, step, model, port, pid, digest, the checked models of the library, the previous `customModelEndpointsEnabled` value; mode 0600, holds no path, not part of a config backup) + `local-model/` (the pinned runtime, the model store and the runtime's private HOME, removed by `DELETE /api/local-model`), `agent-skill-default-migrated.json` (written once by the 1.50.0 boot migration of `agentSkillEnabled`; while it exists the migration never runs again, so deleting it would switch a stored off on at the next start; never tracked or restored), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back), `cli-installed` (empty, touched by `install.sh` when it installs a CLI: a running server reads only its mtime, and a change ends that CLI's negative-cache backoff) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all four installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). +All in `~/.codeman/`: `state.json` (sessions, settings, respawn, orchestrator, cron jobs/runs, owner tab layouts, case default groups), `mux-sessions.json` (tmux recovery), `settings.json` (user prefs), `push-keys.json` + `push-subscriptions.json`, `session-lifecycle.jsonl` (audit log), `update-status.json` (self-updater progress, polled across the service restart), `docker-env-applied.json` (Compose deployment only: sha256 of the Dockerfile + compose file the running container was built from, written by `Start-Codeman.sh`, read by the self-updater's environment gate), `docker-build-source.json` (Compose deployment only: the checkout's HEAD commit and `package-lock.json` hash the `codeman-node-modules`/`codeman-dist` volumes currently reflect, written by both `Start-Codeman.sh` and a successful in-place self-update, compared to detect and refresh a volume left stale by an externally-triggered rebuild), `linked-cases.json`, `webviews.json` (saved web-tab dashboard URLs), `remote-hosts.json` + `remote-cases.json`, `docker-hosts.json` + `docker-cases.json` + `docker-exports/`, `subagent-window-states.json` + `subagent-parents.json` (subagent window layout, GET/PUT `/api/subagent-window-states`/`-parents`), `hook-secret` (per-instance), `users.json` (multi-user, mode 0600) + `admin-audit.jsonl`, `intents.json` (Read My Mind intent profiles, mode 0600), `agent-inbox.json` (agent inbox snapshot, only while `agentInboxPersistEnabled` is on, mode 0600), `edit-history/.json` (Edit history panel, one store per session, every diff an agent wrote, mode 0600), `webhook.json` (webhook notifications: enabled/service/scope plus the ntfy/Slack/Discord URL, a bearer secret, so mode 0600, kept out of `settings.json` and never returned by `/api/webhook`, which is admin-only in multi-user mode), `discord.json` (Discord bridge: bot token, a bearer secret, so mode 0600, never returned by `/api/discord`, admin-only in multi-user mode) + `discord-bridge.json` (sessionId to Discord thread id) + `discord-reply-dedupe.json` (replayed-reply guard: at most 2,048 Discord message IDs with timestamps, kept 7 days, mode 0600; a file that could not be read is never overwritten), `local-model.json` (local model install: state, step, model, port, pid, digest, the checked models of the library, the previous `customModelEndpointsEnabled` value; mode 0600, holds no path, not part of a config backup) + `local-model/` (the pinned runtime, the model store and the runtime's private HOME, removed by `DELETE /api/local-model`), `agent-skill-default-migrated.json` (written once by the 1.50.0 boot migration of `agentSkillEnabled`; while it exists the migration never runs again, so deleting it would switch a stored off on at the next start; never tracked or restored), `config-backup-id` (the random id config backups record as each manifest's `origin`, so instances sharing a backup folder keep to their own snapshots; mode 0600, never tracked or restored, and deleting it unlists this instance's earlier snapshots), `backups/config/` (the default config-backup folder, moved by `configBackupDir`: `cfg-YYYYMMDD-HHMMSS/` snapshots, 0700 with 0600 files, each with a `manifest.json` of SHA-256s, the origin and the retention in force (`keepCount`/`keepDays`), which a pass falls back to when `settings.json` lacks a retention key), `certs/` (self-signed TLS for `--https`), `.env` (CODEMAN_USERNAME/PASSWORD fallback for the `codeman attach` CLI), `install.log` (installer step output, written by `install.sh`'s `run_step`) `tailscale-rename` (the node name before `install.sh` renamed it, so uninstall can offer it back), `cli-installed` (empty, touched by `install.sh` when it installs a CLI: a running server reads only its mtime, and a change ends that CLI's negative-cache backoff) and `runtime/` (the Node.js and tmux `install.sh` downloaded on its no-admin route, linked from `~/.local/bin`; all four installer-route only). Transient: `self-update-runner.sh`. Multi-user case spaces live OUTSIDE the data dir at `~/codeman-users//cases` (shared across instances like `~/codeman-cases`, override `CODEMAN_USER_SPACES_DIR`). **Generated top-level dirs** (all gitignored — don't edit or commit): `dist/` (esbuild output), `out/`, `coverage/`, `test-results/`, `tmp/`, `screenshots-echo-diag/`. The committed gesture bundle (`src/web/public/gesture/gesture-codeman.js`) IS tracked, but its runtime wasm/model assets (`src/web/public/gesture/wasm/`, `*.task`) are fetched and gitignored. diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index 7cb6b913..a10b1f72 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -1555,3 +1555,7 @@ updates `lastSubmitAt` just like any other Enter. - **Files.** Received files come only from Discord's HTTPS hosts into `/.codeman-discord/` (bounded probe, refused if a symlink, self-ignoring). A sent file's real path must lie inside the tab's folder; both the folder and the file are probed with the bounded probe first. - **The whole last message.** A finished turn or an error posts the CLI's whole last message, not the screen: `WebServer.lastResponseFor` is the response viewer's reader (the body of `GET /api/sessions/:id/last-response`, registered on the route context by `registerSessionRoutes`), so the source is the same per-CLI transcript, or the pane split into turns where a CLI keeps none; a dialog and a shell keep the screen tail (its composer row and the status lines under it cut, never a `❯ 1.` option row). The read is bounded to 4 s and any failure or empty read falls back to the screen. It is split into at most 8 posts of 1,900 characters, a cut code fence closed and reopened, and Markdown tables become aligned code blocks (the separator test strips whitespace first: a regex over a run of spaces is quadratic, and the text is steerable by whatever the agent read). The repeat check is keyed on the text plus the turn it came from (the transcript stamp, else the last submit) and is recorded only after the first post succeeds, so a failed post is sent again and a later turn ending on the same words is not dropped. Only some CLIs raise a turn end (`/api/hook-event`: Claude's `stop` and `idle_prompt`, DeepSeek's `stop`), so the others reach this path through errors, a blocked respawn or a crash loop. - **Cost guards.** The alert tail is a function called only once a post will happen (no tmux exec while off), and `discord-notify` is a per-session token bucket (6 a minute, `429 RATE_LIMITED`). + +### Discord reply replay metadata + +`discord-reply-dedupe.json` is a private 0600 data-directory file containing at most 2,048 reply IDs and timestamps retained for seven days. An allowed reply is claimed by message ID before rate-limit or exited-agent checks, attachments, or terminal input. Store failures log once and fall back to memory; they never cost a reply. An unreadable or unsafe file is never overwritten, and reads retry later. Invalid or oversized regular-file ID-only content warns and heals on the next write. Recovery keeps older disk rows before newer outage claims so bounded eviction drops the oldest IDs first. Ambiguous terminal input is never replayed automatically. diff --git a/docs/discord-bridge.md b/docs/discord-bridge.md index 86210d3e..351aef07 100644 --- a/docs/discord-bridge.md +++ b/docs/discord-bridge.md @@ -135,3 +135,9 @@ The text explains `codeman notify "message" --urgent`. `codeman` is on the PATH | Posts arrive but replies do nothing | Message Content Intent is off, or your user ID is not in the allowlist | | Thread is not deleted | The bot lacks Manage Threads on the parent channel | | Nothing arrives on the phone | Check the Discord app's notification setting for the server | + +## Reply replay protection + +The bridge remembers up to 2,048 recent Discord reply IDs for seven days in the private `discord-reply-dedupe.json` metadata file. It claims an allowed reply before rate-limit or exited-agent checks, downloading attachments, or typing into its session, so a gateway replay normally cannot type it twice, including after a restart. The file contains IDs and timestamps only, with mode 0600. + +This is best-effort replay protection, not an exactly-once guarantee. A full disk, unreadable file, or invalid metadata logs one content-free warning and falls back to in-memory checks. Invalid or oversized regular-file ID metadata is replaced on the next write. Files that could not be read are left untouched. Reads and writes retry on later replies; a restart during a metadata outage can lose those checks. At capacity the oldest entries are evicted, so capacity never blocks new replies. Notifications retain their existing delivery and retry behavior; this store never suppresses an outbound alert or replays terminal input. diff --git a/src/discord-bridge.ts b/src/discord-bridge.ts index a8703371..78560ea6 100644 --- a/src/discord-bridge.ts +++ b/src/discord-bridge.ts @@ -4,7 +4,10 @@ * Split like webhook-notify.ts, so the parts that matter are testable without a network: * - pure: `sanitiseReply`, `threadNameFor`, `buildPost`, `discordConfigProblem`, `maskToken` * - store: `~/.codeman/discord.json` (bot token, 0600 via tmp+rename) and - * `~/.codeman/discord-bridge.json` (sessionId -> threadId, not secret) + * `~/.codeman/discord-bridge.json` (sessionId -> threadId, not secret) and + * `~/.codeman/discord-reply-dedupe.json` (bounded reply IDs/timestamps, private). + * Allowed replies are claimed before attachments or typing; store failures never cost a reply. + * An unreadable file is never overwritten; invalid readable ID metadata can heal. * - IO: `DiscordApi` (REST, injected fetch), `DiscordGateway` (inbound WebSocket, injected * socket factory) and `DiscordBridge` (the rules that tie them to sessions) * @@ -27,6 +30,7 @@ import { join } from 'node:path'; import { probePathKind } from './utils/bounded-path-probe.js'; import { DISCORD_RECEIVED_DIR } from './web/paste-image-gc.js'; import WebSocket from 'ws'; +import { DiscordReplyDedupe } from './discord-reply-dedupe.js'; import { WEBHOOK_SCOPES, type WebhookUrgency } from './types/push.js'; import { DISCORD_ON_DELETE, @@ -824,6 +828,7 @@ export interface DiscordBridgeDeps { } export class DiscordBridge { + private readonly replies: DiscordReplyDedupe; private cfg: DiscordConfig = { ...DEFAULT_DISCORD_CONFIG }; private threads: Record = {}; private loaded = false; @@ -857,6 +862,7 @@ export class DiscordBridge { constructor(private readonly deps: DiscordBridgeDeps) { this.now = deps.now ?? Date.now; + this.replies = new DiscordReplyDedupe(deps.configDir, this.now); this.api = new DiscordApi(() => this.cfg.token, deps.fetchImpl, deps.sleep); } @@ -1301,6 +1307,7 @@ export class DiscordBridge { if (!this.cfg.allowedUserIds.includes(m.author?.id)) return; const sessionId = Object.keys(this.threads).find((k) => this.threads[k] === m.channel_id); if (!sessionId) return; + if (!(await this.replies.claim(m.guild_id, m.channel_id, m.id))) return; const t = this.now(); const prev = this.lastReply.get(sessionId); if (prev !== undefined && t - prev < REPLY_MIN_INTERVAL_MS) { diff --git a/src/discord-reply-dedupe.ts b/src/discord-reply-dedupe.ts new file mode 100644 index 00000000..ea938ec3 --- /dev/null +++ b/src/discord-reply-dedupe.ts @@ -0,0 +1,91 @@ +/** @fileoverview Best-effort replay protection. A metadata-store failure must never block a person's reply. */ +import fs from 'node:fs/promises'; +import { join } from 'node:path'; +import { randomUUID } from 'node:crypto'; + +const LIMIT = 2048; +const RETENTION_MS = 7 * 24 * 60 * 60_000; +export class DiscordReplyDedupe { + private seen = new Map(); + private loaded = false; + private dirty = false; + private warned = false; + private chain: Promise = Promise.resolve(); + constructor( + private readonly dir: string, + private readonly now: () => number = Date.now, + private readonly warn: () => void = () => + console.warn('Discord reply metadata unavailable (discord-reply-dedupe.json); using in-memory deduplication') + ) {} + claim(guild: string, channel: string, message: string): Promise { + const run = this.chain.then(() => this.take(guild, channel, message)); + this.chain = run.catch(() => undefined); + return run; + } + private async take(guild: string, channel: string, message: string): Promise { + // Restored legacy configurations and malformed gateway IDs must not poison the store. + if (![guild, channel, message].every((id) => /^\d{15,25}$/.test(id))) return true; + const path = join(this.dir, 'discord-reply-dedupe.json'); + const cutoff = this.now() - RETENTION_MS; + if (!this.loaded) { + let contentRead = false; + try { + const st = await fs.lstat(path); + if (!st.isFile() || st.isSymbolicLink()) throw Object.assign(new Error('Unsafe metadata'), { code: 'EINVAL' }); + if (st.size > 512 * 1024) { + contentRead = true; // A known oversized regular file is invalid metadata, not an IO failure. + throw new Error('Oversized metadata'); + } + const text = await fs.readFile(path, 'utf8'); + contentRead = true; + const rows: unknown = JSON.parse(text); + if (!Array.isArray(rows) || rows.length > LIMIT) throw new Error('Invalid metadata'); + const recovered = new Map(); + for (const row of rows) { + if (!Array.isArray(row) || !/^\d{15,25}:\d{15,25}:\d{15,25}$/.test(row[0]) || !Number.isFinite(row[1])) + throw new Error('Invalid metadata'); + if (row[1] >= cutoff) recovered.set(row[0], row[1]); + } + // Older disk rows precede newer claims made while reads were unavailable. + this.seen = new Map([...recovered, ...this.seen]); + this.loaded = true; + } catch (err) { + if ((err as NodeJS.ErrnoException).code === 'ENOENT') this.loaded = true; + else { + this.report(); + // Invalid content is recoverable ID-only metadata; unreadable/unsafe files stay untouched. + if (contentRead && !(err as NodeJS.ErrnoException).code) { + this.loaded = true; + this.dirty = true; + } + } + } + } + for (const [key, at] of this.seen) if (at < cutoff) this.seen.delete(key); + const key = `${guild}:${channel}:${message}`; + const fresh = !this.seen.has(key); + if (fresh) { + this.seen.set(key, this.now()); + this.dirty = true; + } // Before awaiting IO or typing, including during an outage. + while (this.seen.size > LIMIT) this.seen.delete(this.seen.keys().next().value!); + if (!this.loaded || !this.dirty) return fresh; // Do not overwrite a journal we could not read. + const tmp = `${path}.${randomUUID()}.tmp`; + try { + await fs.mkdir(this.dir, { recursive: true }); + await fs.writeFile(tmp, JSON.stringify([...this.seen]), { mode: 0o600, flag: 'wx' }); + await fs.rename(tmp, path); + this.warned = false; + this.dirty = false; + } catch { + this.report(); + } finally { + await fs.rm(tmp, { force: true }).catch(() => undefined); + } + return fresh; + } + private report(): void { + if (!this.warned) this.warn(); + this.warned = true; + } +} diff --git a/test/discord-bridge.test.ts b/test/discord-bridge.test.ts index 32db32d7..fec31c71 100644 --- a/test/discord-bridge.test.ts +++ b/test/discord-bridge.test.ts @@ -448,6 +448,72 @@ describe('DiscordBridge', () => { return b; } + it('deduplicates replayed real gateway IDs across restart while new replies still work', async () => { + const b = await ready(); + const m = reply({ id: '555555555555555555' }); + await b.onGatewayMessage(m); + b.stop(); + t += 10_000; + const next = await ready(); + await next.onGatewayMessage(m); + expect(sendInput).toHaveBeenCalledTimes(1); + await next.onGatewayMessage(reply({ id: '555555555555555556', content: 'continue' })); + expect(sendInput).toHaveBeenCalledTimes(2); + next.stop(); + }); + + it('claims rate-limited snowflake replies so replays remain refused across restart', async () => { + const b = await ready(); + await b.onGatewayMessage(reply({ id: '555555555555555555' })); + t += 200; + const refused = reply({ id: '555555555555555556', content: 'too soon' }); + await b.onGatewayMessage(refused); + expect(sendInput).toHaveBeenCalledTimes(1); + expect(posts.some((p) => p.path.includes(encodeURIComponent('⏳')))).toBe(true); + t += 5_000; + const count = posts.length; + await b.onGatewayMessage(refused); + expect(sendInput).toHaveBeenCalledTimes(1); + expect(posts).toHaveLength(count); + b.stop(); + const next = await ready(); + await next.onGatewayMessage(refused); + expect(sendInput).toHaveBeenCalledTimes(1); + await next.onGatewayMessage(reply({ id: '555555555555555557', content: 'new attempt' })); + expect(sendInput).toHaveBeenCalledTimes(2); + next.stop(); + }); + + it('ignores an immediate replay without adding an hourglass to a successful reply', async () => { + const b = await ready(); + const m = reply({ id: '555555555555555555' }); + await b.onGatewayMessage(m); + t += 200; + await b.onGatewayMessage(m); + expect(sendInput).toHaveBeenCalledTimes(1); + expect(posts.filter((p) => p.method === 'PUT')).toHaveLength(1); + expect(posts.some((p) => p.path.includes(encodeURIComponent('⏳')))).toBe(false); + b.stop(); + }); + + it('delivers replies and alerts when metadata is corrupt and recovers without restarting', async () => { + const b = await ready(); + writeFileSync(join(dir, 'discord-reply-dedupe.json'), '{'); + const log = vi.spyOn(console, 'warn').mockImplementation(() => undefined); + try { + await b.onGatewayMessage(reply({ id: '555555555555555555' })); + expect(sendInput).toHaveBeenCalledTimes(1); + t += 10_000; + expect((await b.notify(MSG))?.ok).toBe(true); + rmSync(join(dir, 'discord-reply-dedupe.json')); + await b.onGatewayMessage(reply({ id: '555555555555555556' })); + expect(sendInput).toHaveBeenCalledTimes(2); + expect(log).toHaveBeenCalledTimes(1); + } finally { + log.mockRestore(); + b.stop(); + } + }); it('types an allowed reply into the mapped session and ticks it', async () => { const b = await ready(); await b.onGatewayMessage(reply()); @@ -1143,7 +1209,7 @@ describe('DiscordBridge files and startup hint', () => { it('types a reply that waits on a download before a later text reply', async () => { const b = await make(); const slow = b.onGatewayMessage(reply({ content: 'first' })); - await new Promise((r) => setTimeout(r, 0)); // the first reply is under way (gate passed) before the clock moves + await vi.waitFor(() => expect(downloads).toHaveLength(1)); // wait for the claim and rate-limit gate t += 5_000; const fast = b.onGatewayMessage(reply({ id: '555555555555555556', content: 'second', attachments: [] })); await Promise.all([slow, fast]); @@ -1157,11 +1223,11 @@ describe('DiscordBridge files and startup hint', () => { await b.onGatewayMessage(reply({ content: '2', attachments: [] })); expect(sent.at(-1)?.[1]).toBe('2'); t += 5_000; - await b.onGatewayMessage(reply({ content: 'yes', attachments: [] })); + await b.onGatewayMessage(reply({ id: '555555555555555556', content: 'yes', attachments: [] })); expect(sent.at(-1)?.[1]).toBe('yes\r'); t += 5_000; waiting = false; - await b.onGatewayMessage(reply({ content: '2', attachments: [] })); + await b.onGatewayMessage(reply({ id: '555555555555555557', content: '2', attachments: [] })); expect(sent.at(-1)?.[1]).toBe('2\r'); b.stop(); }); diff --git a/test/discord-reply-dedupe.test.ts b/test/discord-reply-dedupe.test.ts new file mode 100644 index 00000000..2c51d050 --- /dev/null +++ b/test/discord-reply-dedupe.test.ts @@ -0,0 +1,133 @@ +import { mkdtemp, readFile, rm, writeFile, stat } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, beforeEach, expect, it, vi } from 'vitest'; +import { DiscordReplyDedupe } from '../src/discord-reply-dedupe.js'; +import fs from 'node:fs/promises'; +let dir: string; +const guild = '111111111111111111', + channel = '222222222222222222', + message = '333333333333333333'; +beforeEach(async () => { + dir = await mkdtemp(join(tmpdir(), 'discord-replies-')); +}); +afterEach(async () => { + await rm(dir, { recursive: true, force: true }); +}); +it('claims once across concurrent events and restarts, without storing message content', async () => { + const store = new DiscordReplyDedupe(dir); + expect(await Promise.all([store.claim(guild, channel, message), store.claim(guild, channel, message)])).toEqual([ + true, + false, + ]); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); + expect((await stat(join(dir, 'discord-reply-dedupe.json'))).mode & 0o777).toBe(0o600); + expect(JSON.parse(await readFile(join(dir, 'discord-reply-dedupe.json'), 'utf8'))).toEqual([ + [`${guild}:${channel}:${message}`, expect.any(Number)], + ]); +}); +it('falls back on corrupt metadata, logs once, retries after recovery, and retains memory claims', async () => { + const path = join(dir, 'discord-reply-dedupe.json'); + await writeFile(path, '{'); + const warn = vi.fn(), + store = new DiscordReplyDedupe(dir, Date.now, warn); + expect(await store.claim(guild, channel, message)).toBe(true); + expect(await store.claim(guild, channel, message)).toBe(false); + expect(warn).toHaveBeenCalledTimes(1); + await rm(path); + expect(await store.claim(guild, channel, '444444444444444444')).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); +}); +it('does not poison metadata with incomplete restored configuration', async () => { + const store = new DiscordReplyDedupe(dir); + expect(await store.claim('', channel, message)).toBe(true); + expect(await store.claim(guild, channel, message)).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); +}); +it('keeps accepting new replies during ENOSPC and persists memory claims when writes recover', async () => { + const warn = vi.fn(); + const store = new DiscordReplyDedupe(dir, Date.now, warn); + const rename = vi.spyOn(fs, 'rename').mockRejectedValueOnce(Object.assign(new Error('full'), { code: 'ENOSPC' })); + try { + expect(await store.claim(guild, channel, message)).toBe(true); + expect(await store.claim(guild, channel, message)).toBe(false); + expect(await store.claim(guild, channel, '444444444444444444')).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); + expect(warn).toHaveBeenCalledTimes(1); + } finally { + rename.mockRestore(); + } +}); +it('evicts old records rather than refusing replies at capacity', async () => { + await writeFile( + join(dir, 'discord-reply-dedupe.json'), + JSON.stringify( + Array.from({ length: 2048 }, (_, i) => [ + `${guild}:${channel}:${String(100000000000000000n + BigInt(i))}`, + Date.now(), + ]) + ) + ); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(true); + expect(JSON.parse(await readFile(join(dir, 'discord-reply-dedupe.json'), 'utf8'))).toHaveLength(2048); +}); + +it.each(['{', '{}', '[null]'])( + 'heals readable invalid metadata %s and keeps the claim across restart', + async (invalid) => { + const path = join(dir, 'discord-reply-dedupe.json'); + await writeFile(path, invalid); + const warn = vi.fn(), + store = new DiscordReplyDedupe(dir, Date.now, warn); + expect(await store.claim(guild, channel, message)).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); + expect(warn).toHaveBeenCalledTimes(1); + } +); + +it('retains newer outage claims when disk recovery reaches capacity, without overwriting an unread file', async () => { + const path = join(dir, 'discord-reply-dedupe.json'); + const rows = Array.from({ length: 2048 }, (_, i) => [ + `${guild}:${channel}:${100000000000000000n + BigInt(i)}`, + Date.now(), + ]); + const original = JSON.stringify(rows); + await writeFile(path, original); + const warn = vi.fn(), + store = new DiscordReplyDedupe(dir, Date.now, warn); + const read = vi + .spyOn(fs, 'readFile') + .mockRejectedValueOnce(Object.assign(new Error('descriptor limit'), { code: 'EMFILE' })); + expect(await store.claim(guild, channel, message)).toBe(true); + read.mockRestore(); + expect(await readFile(path, 'utf8')).toBe(original); + expect(await store.claim(guild, channel, '444444444444444444')).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); + const saved = JSON.parse(await readFile(path, 'utf8')); + expect(saved).toHaveLength(2048); + expect(saved.some((row: [string, number]) => row[0] === rows[0][0])).toBe(false); +}); + +it('heals an over-cap readable store rather than remaining memory-only until manual deletion', async () => { + const path = join(dir, 'discord-reply-dedupe.json'); + await writeFile( + path, + JSON.stringify( + Array.from({ length: 2049 }, (_, i) => [`${guild}:${channel}:${100000000000000000n + BigInt(i)}`, Date.now()]) + ) + ); + const warn = vi.fn(); + expect(await new DiscordReplyDedupe(dir, Date.now, warn).claim(guild, channel, message)).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); + expect(warn).toHaveBeenCalledTimes(1); +}); + +it('heals oversized regular metadata and persists replay protection across restart', async () => { + const path = join(dir, 'discord-reply-dedupe.json'); + await writeFile(path, ' '.repeat(512 * 1024 + 1)); + const warn = vi.fn(); + expect(await new DiscordReplyDedupe(dir, Date.now, warn).claim(guild, channel, message)).toBe(true); + expect(await new DiscordReplyDedupe(dir).claim(guild, channel, message)).toBe(false); + expect((await stat(path)).size).toBeLessThan(512 * 1024); + expect(warn).toHaveBeenCalledTimes(1); +});