mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-10-20 09:09:35 +02:00
test(uploads): pin the browser's copies of the document list to ALLOWED_DOCUMENT_EXTS
The browser cannot import the server's document allowlist, so the classifier, the three pickers' accept lists and the two file-path link patterns each carry a hand-kept copy. A missed copy fails silently: a file greyed out in the picker, an upload dropped client-side, or a pasted path that is no longer a link. Hoist the set to module scope so the test reads the shipped list.
This commit is contained in:
1 parent
ce809eac41
commit
7e8ed90e2d
3 files changed
+138
-7
No files matched your search
@@ -215,7 +215,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Attachments** (live external document references; all wiring in `file-routes.ts`): a **registry** maps a stable `attachmentId` to a realpath-resolved, extension-allowlisted absolute path, so browser requests never carry arbitrary absolute paths. ⚠️ The **magic-link scanner** (`codeman://attach?...` in terminal output) is **prompt-injectable**, so its scan path is force-confined to the session workspace; a hostile prompt could otherwise exfiltrate arbitrary host files over SSE. The security gate is an extension **allowlist**, not a blocklist. `document-conversion-limiter.ts` caps converter spawns globally: without it, N large docs detected at once fork N multi-minute processes, which is a resource-exhaustion vector. → [architecture-invariants#attachments](docs/architecture-invariants.md#attachments)
|
||||
|
||||
**Prompt uploads** (`POST /api/sessions/:id/paste-image`: clipboard paste, drop, the Compose Attach key and the phone toolbar's Attach button; client wiring in `image-input.js` and `keyboard-accessory.js`): a pick is written to `<workspace>/.codeman-uploads/` and its path lands on the prompt as unsent text. The folder is IN the workspace because that is the only path identical for a local and a container agent, FLAT because `<workspace>/.codeman/` would BE the data dir when the workspace is the home directory, and self-ignoring through a `.gitignore` of `*` written ONCE with `wx` (a file already there is the user's and is never overwritten; the user's own ignore file is never touched). ⚠️ The directory names live ONLY in `paste-image-gc.ts` (`UPLOAD_DIR_NAMES`): the pre-move `.claude-images/` receives nothing but stays readable for one release, so the GC sweep and the delete cleanup (through `uploadDirs()`) and the image watcher's ignore filter (reading the names) cover BOTH; retire the legacy entry there, never by hand elsewhere. ⚠️ `uploadDirs()` is async and probes the workspace BOUNDED first (`probePathKind()`, the #516 rule: an `unknown` path is skipped, never touched; the sweep keeps the stall cap and the delete passes `pastCap`), then returns only REAL directories (lstat, so a planted symlink is never one), none that is or contains `getDataDir()` (one strictly inside it only holds uploads and is listed), and none for a remote session, since both consumers delete; the check is made at list time, and the same-user race after it is accepted. Images are buffered in memory under `MAX_PASTE_IMAGE_BYTES` (50MB) after a client-side canvas re-encode; a DOCUMENT (pdf, docx, pptx, xlsx, zip and the text family txt/md/csv/tsv/json/log/yaml/yml/xml; the curated `ALLOWED_DOCUMENT_EXTS`, never `SUPPORTED_ATTACHMENT_EXTENSIONS` or `EDITABLE_EXTENSIONS`, which carry sh/py/js/html) is admitted by its name's extension alone, buffered like an image, judged on its bytes by the same fail-closed `imageMagicMatchesExt` (`%PDF-`, `PK\x03\x04` for the OOXML trio and zip, UTF-8 without a NUL byte for text: a SIGNATURE or ENCODING check, never a structure or content check, so generic zip bytes pass as `.docx` and a script renamed `.txt` is text; the sh/py/js/html exclusion is a SCOPE choice by extension, keeping the server from storing what it would serve or run, not a content filter: the agent can already write any of those) and saved as `paste-<ts>-<hex>-<stem><ext>` with a sanitized stem (`[A-Za-z0-9._-]`, 40 chars, a wholly non-ASCII name leaves no stem; the `paste-` prefix is what the GC matches). A VIDEO is never buffered: `?kind=video` streams it to disk under `MAX_PASTE_VIDEO_BYTES` (1GB, env `CODEMAN_MAX_PASTE_VIDEO_BYTES`), judges the container on the first 12 bytes as they arrive (ISO BMFF / QuickTime atom, EBML for webm, Ogg for ogv; a mislabeled body is drained unwritten from there, never parked on disk first) and unlinks on any failure, a dead transfer included. ⚠️ The flag is the ONLY way onto that path and admits video types alone (the registry's `VIDEO_ATTACHMENT_EXTENSIONS`, single-sourced: mp4/webm/mov/m4v/ogv, so an uploaded video is one the file viewer can also play; 3gp/mkv are out, since a phone's camera roll produces neither and neither previews), so an image can never ride the larger cap through memory; without it the route is byte-identical to the image-only version. Busboy takes its size limit before the part is seen, which is why the choice rides the URL and not the file. ⚠️ A video draws on its OWN per-(IP, session) rate bucket (`consumeVideoToken`, 3/min) beside the image bucket (`consumePasteToken`, 30/min): at 1GB each, 30/min would be 30GB of disk a minute, so the lower rate bounds it to 3GB; the image path keeps the 30/min bucket and is unchanged without the flag. A remote (SSH) session is refused (400) before any disk touch: its `workingDir` is the remote path, so the file would land on THIS host where the agent cannot read it (Docker is fine, the workspace is bind-mounted at the same absolute path). ⚠️ A live session's workspace can still sit on a network mount that stopped answering, so the route probes it BOUNDED (`probePathKind(session.workingDir, { pastCap: true })`, the #516 rule) before the first disk touch and refuses an `unknown` mount through `describeUnknownPath()` (OPERATION_FAILED), the way session create does, so a mount already unreachable cannot freeze a libuv worker on every upload (a mount that dies after the probe is the same residual race create accepts). ⚠️ Dispatch mirrors the client: an `image/*` MIME takes the IMAGE path whatever the name says (a GIF named `animation.txt` stays a GIF), a document extension takes the document branch, and anything else takes the image path exactly as before, where the HEIC sniff still recovers a HEIC under a generic name; a foreign extension with no image MIME gets its `Unsupported file type` 400 only AFTER that sLine truncated
|
||||
**Prompt uploads** (`POST /api/sessions/:id/paste-image`: clipboard paste, drop, the Compose Attach key and the phone toolbar's Attach button; client wiring in `image-input.js` and `keyboard-accessory.js`): a pick is written to `<workspace>/.codeman-uploads/` and its path lands on the prompt as unsent text. The folder is IN the workspace because that is the only path identical for a local and a container agent, FLAT because `<workspace>/.codeman/` would BE the data dir when the workspace is the home directory, and self-ignoring through a `.gitignore` of `*` written ONCE with `wx` (a file already there is the user's and is never overwritten; the user's own ignore file is never touched). ⚠️ The directory names live ONLY in `paste-image-gc.ts` (`UPLOAD_DIR_NAMES`): the pre-move `.claude-images/` receives nothing but stays readable for one release, so the GC sweep and the delete cleanup (through `uploadDirs()`) and the image watcher's ignore filter (reading the names) cover BOTH; retire the legacy entry there, never by hand elsewhere. ⚠️ `uploadDirs()` is async and probes the workspace BOUNDED first (`probePathKind()`, the #516 rule: an `unknown` path is skipped, never touched; the sweep keeps the stall cap and the delete passes `pastCap`), then returns only REAL directories (lstat, so a planted symlink is never one), none that is or contains `getDataDir()` (one strictly inside it only holds uploads and is listed), and none for a remote session, since both consumers delete; the check is made at list time, and the same-user race after it is accepted. Images are buffered in memory under `MAX_PASTE_IMAGE_BYTES` (50MB) after a client-side canvas re-encode; a DOCUMENT (pdf, docx, pptx, xlsx, zip and the text family txt/md/csv/tsv/json/log/yaml/yml/xml; the curated `ALLOWED_DOCUMENT_EXTS`, never `SUPPORTED_ATTACHMENT_EXTENSIONS` or `EDITABLE_EXTENSIONS`, which carry sh/py/js/html) is admitted by its name's extension alone, buffered like an image, judged on its bytes by the same fail-closed `imageMagicMatchesExt` (`%PDF-`, `PK\x03\x04` for the OOXML trio and zip, UTF-8 without a NUL byte for text: a SIGNATURE or ENCODING check, never a structure or content check, so generic zip bytes pass as `.docx` and a script renamed `.txt` is text; the sh/py/js/html exclusion is a SCOPE choice by extension, keeping the server from storing what it would serve or run, not a content filter: the agent can already write any of those) and saved as `paste-<ts>-<hex>-<stem><ext>` with a sanitized stem (`[A-Za-z0-9._-]`, 40 chars, a wholly non-ASCII name leaves no stem; the `paste-` prefix is what the GC matches). A VIDEO is never buffered: `?kind=video` streams it to disk under `MAX_PASTE_VIDEO_BYTES` (1GB, env `CODEMAN_MAX_PASTE_VIDEO_BYTES`), judges the container on the first 12 bytes as they arrive (ISO BMFF / QuickTime atom, EBML for webm, Ogg for ogv; a mislabeled body is drained unwritten from there, never parked on disk first) and unlinks on any failure, a dead transfer included. ⚠️ The flag is the ONLY way onto that path and admits video types alone (the registry's `VIDEO_ATTACHMENT_EXTENSIONS`, single-sourced: mp4/webm/mov/m4v/ogv, so an uploaded video is one the file viewer can also play; 3gp/mkv are out, since a phone's camera roll produces neither and neither previews), so an image can never ride the larger cap through memory; without it the route is byte-identical to the image-only version. Busboy takes its size limit before the part is seen, which is why the choice rides the URL and not the file. ⚠️ A video draws on its OWN per-(IP, session) rate bucket (`consumeVideoToken`, 3/min) beside the image bucket (`consumePasteToken`, 30/min): at 1GB each, 30/min would be 30GB of disk a minute, so the lower rate bounds it to 3GB; the image path keeps the 30/min bucket and is unchanged without the flag. A remote (SSH) session is refused (400) before any disk touch: its `workingDir` is the remote path, so the file would land on THIS host where the agent cannot read it (Docker is fine, the workspace is bind-mounted at the same absolute path). ⚠️ A live session's workspace can still sit on a network mount that stopped answering, so the route probes it BOUNDED (`probePathKind(session.workingDir, { pastCap: true })`, the #516 rule) before the first disk touch and refuses an `unknown` mount through `describeUnknownPath()` (OPERATION_FAILED), the way session create does, so a mount already unreachable cannot freeze a libuv worker on every upload (a mount that dies after the probe is the same residual race create accepts). ⚠️ Dispatch mirrors the client: an `image/*` MIME takes the IMAGE path whatever the name says (a GIF named `animation.txt` stays a GIF), a document extension takes the document branch, and anything else takes the image path exactly as before, where the HEIC sniff still recovers a HEIC under a generic name; a foreign extension with no image MIME gets its `Unsupported file type` 400 only AFTER that sLine truncated
|
||||
|
||||
**File-path links (terminal + chat)**: a path an agent prints is clickable on BOTH surfaces and opens the file-preview overlay. ⚠️ ONE pattern (`FILE_PATH_LINK_PATTERN` / `absoluteFilePathPattern()` in constants.js) feeds the xterm link provider AND `_linkifyFilePaths()`, a fresh instance per call (`lastIndex`). Both call `findFilePathLinks()`, which adds `~/`, relative and folder paths without overlapping a rooted match or a URL; a relative path resolves against `workingDir` (`resolveLinkedFilePath`), `~/` only server-side on the session's host (attachment route), and `terminalPathLine()` joins a path a program hard-wrapped short of the edge. ⚠️ A folder link opens the **folder grid**: `POST /api/sessions/:id/folders` guards + GRANTS the folder (`folder-listing.ts`, own registry, never the 200-capped attachment registry), files come by bare name via `…/folders/:folderId/files/:name/*` through `resolveServableAttachmentPath` plus the same extension/regular-file gate on the symlink's resolved target, so no browser request carries a path; `~/` is refused in a Docker case (the container's home is not this host's). The chat linkifier walks TEXT NODES with DOM APIs, never rebuilds sanitized markup as a string. ⚠️ An out-of-workspace path goes through the ATTACHMENT routes (`POST /api/sessions/:id/attachments` with `notify: false`), never by widening `file-content`/`file-raw` or `file-stream-manager`'s `tail -f` allowlist. ⚠️ `TEXT_ATTACHMENT_EXTENSIONS` IS `EDITABLE_EXTENSIONS` (never a second list), and widening READ must never widen RUN: `html`/`htm`/`svg` stay download-only, other text is inert `text/plain`+`nosniff`. ⚠️ The ONE place HTML renders is `/html-view/<cap>/...` (`html-view-routes.ts`, `html-view-capabilities.ts`): `POST /api/sessions/:id/html-view` mints a capability for the file's own directory, every response carries `Content-Security-Policy: sandbox` WITHOUT `allow-same-origin` (opaque origin even as a top-level tab, so the page can never read Codeman or drive its API) and with NO fetch directive (open network, owner decision 2026-10-13: never restrict it or add `allow-same-origin` without the owner), and only non-dot, asset-allowlisted files under that directory are served (GET/HEAD only; capabilities revoked with the web-tab ones on logout). That tree is served recursively, so minting is refused for a hidden directory or a broad root (`isBroadHtmlViewRoot`); the preview renders by default and the **Page** pill (or a refused mint) falls through to the source view and its Edit button. Never serve HTML same-origin anywhere else. Tests: `test/routes/html-view-routes.test.ts`. Media extensions are single-sourced in `attachment-registry.ts`. ⚠️ **XLSX previews parse in the BROWSER**, never on the server; the admission limits that keep a hostile workbook from freezing the worker live in `src/web/CLAUDE.md` (Spreadsheet preview). → [architecture-invariants#file-path-links-terminal--response-viewer](docs/architecture-invariants.md#file-path-links-terminal--response-viewer)
|
||||
|
||||
|
||||
@@ -330,6 +330,16 @@ export function stripInkRedrawBloat(buffer: string): string {
|
||||
*/
|
||||
const TEXT_UPLOAD_EXTS = new Set(['.txt', '.md', '.csv', '.tsv', '.json', '.log', '.yaml', '.yml', '.xml']);
|
||||
|
||||
/**
|
||||
* Documents a user hands an agent to read: the office formats plus the text
|
||||
* family. Curated on purpose, never SUPPORTED_ATTACHMENT_EXTENSIONS or
|
||||
* EDITABLE_EXTENSIONS, which carry sh/py/js/html; a prompt upload is a thing
|
||||
* to read, not a thing to run or serve. Each is judged on its bytes by
|
||||
* imageMagicMatchesExt (signature, or UTF-8 without NUL for text). The browser
|
||||
* keeps its own copies (test/upload-document-list-parity.test.ts pins them).
|
||||
*/
|
||||
export const ALLOWED_DOCUMENT_EXTS = new Set(['.pdf', '.docx', '.pptx', '.xlsx', '.zip', ...TEXT_UPLOAD_EXTS]);
|
||||
|
||||
/**
|
||||
* Validate upload bytes against a declared extension. Sniffs the first ~12 bytes
|
||||
* for a known magic-number signature (a text file is instead required to be
|
||||
@@ -5447,12 +5457,6 @@ export function registerSessionRoutes(
|
||||
// so an uploaded video is a type that previews. 3gp/mkv are deliberately out: they
|
||||
// would upload but not preview, and a phone's camera roll produces neither.
|
||||
const ALLOWED_VIDEO_EXTS = new Set([...VIDEO_ATTACHMENT_EXTENSIONS].map((e) => `.${e}`));
|
||||
// Documents a user hands an agent to read: the office formats plus the text
|
||||
// family. Curated on purpose, never SUPPORTED_ATTACHMENT_EXTENSIONS or
|
||||
// EDITABLE_EXTENSIONS, which carry sh/py/js/html; a prompt upload is a thing
|
||||
// to read, not a thing to run or serve. Each is judged on its bytes by
|
||||
// imageMagicMatchesExt (signature, or UTF-8 without NUL for text).
|
||||
const ALLOWED_DOCUMENT_EXTS = new Set(['.pdf', '.docx', '.pptx', '.xlsx', '.zip', ...TEXT_UPLOAD_EXTS]);
|
||||
const VIDEO_MIME_EXT: Record<string, string> = {
|
||||
mp4: '.mp4',
|
||||
quicktime: '.mov',
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
/**
|
||||
* @fileoverview Prompt-upload document list parity: server ⇄ browser copies.
|
||||
*
|
||||
* The server admits a document upload by extension through
|
||||
* `ALLOWED_DOCUMENT_EXTS` (session-routes.ts). The browser cannot import it, so
|
||||
* the list is copied by hand into:
|
||||
*
|
||||
* - `_promptAttachKind()` (image-input.js), the one classifier every pick,
|
||||
* drop and paste reads: an extension missing there is dropped client-side
|
||||
* with "Unsupported file type", an extra one uploads only to be refused.
|
||||
* - the `accept=` of the toolbar Attach input (index.html `#attachFileInput`)
|
||||
* and of the two Compose / paste-dialog pickers (keyboard-accessory.js
|
||||
* `.paste-file-input`): an extension missing there is greyed out in the
|
||||
* picker, so the user cannot pick the file at all.
|
||||
* - the two file-path link patterns (constants.js): an uploaded document's
|
||||
* path lands on the prompt and comes back in the agent's output, and an
|
||||
* extension missing there leaves that path as plain text. tsv and zip were
|
||||
* added there by hand once already.
|
||||
*
|
||||
* Same idea as test/media-extension-parity.test.ts: the server set is imported,
|
||||
* the browser copies are read from the shipped files. The classifier and the
|
||||
* link patterns are run, not just read. No port needed.
|
||||
*/
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { resolve } from 'node:path';
|
||||
import vm from 'node:vm';
|
||||
import { JSDOM } from 'jsdom';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { ALLOWED_DOCUMENT_EXTS } from '../src/web/routes/session-routes.js';
|
||||
|
||||
const PUBLIC = resolve(import.meta.dirname, '../src/web/public');
|
||||
const read = (f: string) => readFileSync(resolve(PUBLIC, f), 'utf8');
|
||||
|
||||
/** Server extensions without the dot, e.g. `pdf`. */
|
||||
const DOCUMENT_EXTS = [...ALLOWED_DOCUMENT_EXTS].map((e) => e.replace(/^\./, '')).sort();
|
||||
|
||||
/** The document extensions an `accept=` value lists (the `image/*` and `video/*` entries dropped). */
|
||||
function acceptDocumentExts(accept: string): string[] {
|
||||
return accept
|
||||
.split(',')
|
||||
.map((t) => t.trim())
|
||||
.filter((t) => t.startsWith('.'))
|
||||
.map((t) => t.slice(1))
|
||||
.sort();
|
||||
}
|
||||
|
||||
type AttachKind = (file: { name: string; type: string }) => string | null;
|
||||
|
||||
function loadPromptAttachKind(): { kind: AttachKind; method: string } {
|
||||
const source = read('image-input.js');
|
||||
const context = vm.createContext({});
|
||||
vm.runInContext('class CodemanApp {}', context);
|
||||
vm.runInContext(source, context, { filename: 'image-input.js' });
|
||||
const app = new (vm.runInContext('CodemanApp', context) as new () => { _promptAttachKind: AttachKind })();
|
||||
// The live method's own text, comments stripped, so a stale copy elsewhere cannot stand in for it.
|
||||
const method = app._promptAttachKind.toString().replace(/\/\*[\s\S]*?\*\/|\/\/.*$/gm, '');
|
||||
return { kind: (file) => app._promptAttachKind(file), method };
|
||||
}
|
||||
|
||||
describe('prompt-upload document list parity (ALLOWED_DOCUMENT_EXTS ⇄ browser copies)', () => {
|
||||
it('image-input.js _promptAttachKind classifies every server document extension as a document', () => {
|
||||
const { kind } = loadPromptAttachKind();
|
||||
// No MIME: the extension alone decides, as on the server.
|
||||
const missing = DOCUMENT_EXTS.filter((ext) => kind({ name: `report.${ext}`, type: '' }) !== 'document');
|
||||
expect(missing, `image-input.js _promptAttachKind() does not classify as document: ${missing.join(', ')}`).toEqual(
|
||||
[]
|
||||
);
|
||||
});
|
||||
|
||||
it('image-input.js _promptAttachKind lists no document extension the server refuses', () => {
|
||||
const { kind, method } = loadPromptAttachKind();
|
||||
const found = [...method.matchAll(/if \(\[([^\]]+)\]\.includes\(ext\)\) \{\s*return 'document';/g)];
|
||||
expect(found.length, 'expected one document extension list in image-input.js _promptAttachKind()').toBe(1);
|
||||
const clientExts = [...found[0][1].matchAll(/'([^']+)'/g)].map((q) => q[1]);
|
||||
// And it is the list that actually returns 'document'.
|
||||
for (const ext of clientExts) expect(kind({ name: `report.${ext}`, type: '' })).toBe('document');
|
||||
expect(clientExts.sort(), 'image-input.js _promptAttachKind() document list').toEqual(DOCUMENT_EXTS);
|
||||
});
|
||||
|
||||
it('index.html #attachFileInput accept= lists exactly the server document extensions', () => {
|
||||
const input = new JSDOM(read('index.html')).window.document.getElementById('attachFileInput');
|
||||
expect(input, '#attachFileInput not found in index.html').not.toBeNull();
|
||||
const exts = acceptDocumentExts(input!.getAttribute('accept') ?? '');
|
||||
expect(exts, 'index.html #attachFileInput accept=').toEqual(DOCUMENT_EXTS);
|
||||
});
|
||||
|
||||
it('keyboard-accessory.js: both .paste-file-input accept= list exactly the server document extensions', () => {
|
||||
const accepts = [...read('keyboard-accessory.js').matchAll(/class="paste-file-input" accept="([^"]*)"/g)].map(
|
||||
(m) => m[1]
|
||||
);
|
||||
// The Compose picker and the paste dialog's picker.
|
||||
expect(accepts.length, 'expected two .paste-file-input pickers in keyboard-accessory.js').toBe(2);
|
||||
accepts.forEach((accept, i) => {
|
||||
expect(acceptDocumentExts(accept), `keyboard-accessory.js .paste-file-input #${i + 1} accept=`).toEqual(
|
||||
DOCUMENT_EXTS
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('constants.js file-path link patterns link an uploaded document path', () => {
|
||||
const context = vm.createContext({});
|
||||
vm.runInContext(
|
||||
`${read('constants.js')}\nglobalThis.__p = { FILE_PATH_LINK_PATTERN, RELATIVE_FILE_PATH_LINK_PATTERN };`,
|
||||
context,
|
||||
{ filename: 'constants.js' }
|
||||
);
|
||||
const { FILE_PATH_LINK_PATTERN, RELATIVE_FILE_PATH_LINK_PATTERN } = (
|
||||
context as unknown as { __p: Record<string, RegExp> }
|
||||
).__p;
|
||||
|
||||
it('FILE_PATH_LINK_PATTERN matches a rooted path for every document extension', () => {
|
||||
const missing = DOCUMENT_EXTS.filter((ext) => {
|
||||
const path = `/home/u/case/.codeman-uploads/paste-1-ab-report.${ext}`;
|
||||
return new RegExp(FILE_PATH_LINK_PATTERN.source, 'g').exec(path)?.[1] !== path;
|
||||
});
|
||||
expect(missing, `constants.js FILE_PATH_LINK_PATTERN does not link: ${missing.join(', ')}`).toEqual([]);
|
||||
});
|
||||
|
||||
it('RELATIVE_FILE_PATH_LINK_PATTERN matches a relative path for every document extension', () => {
|
||||
const missing = DOCUMENT_EXTS.filter((ext) => {
|
||||
const path = `.codeman-uploads/paste-1-ab-report.${ext}`;
|
||||
return new RegExp(RELATIVE_FILE_PATH_LINK_PATTERN.source, 'g').exec(`saved ${path}`)?.[2] !== path;
|
||||
});
|
||||
expect(missing, `constants.js RELATIVE_FILE_PATH_LINK_PATTERN does not link: ${missing.join(', ')}`).toEqual([]);
|
||||
});
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user