diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index c7b8969d..4d7bfc35 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -53,7 +53,7 @@ Pure helpers unit-tested in `test/base-path.test.ts` + `test/webview-proxy.test. ⚠️ **`agent_working` is a hook event with no Claude Code hook behind it** (157th SSE constant). It exists because a harness turn cannot run while one of its own modal approvals is on screen, so "the agent started working" proves a dialog was answered in the terminal. It joins `APPROVAL_RESOLVING_EVENTS`; without it a dsh session's red alert would survive until the next `stop`, the exact stuck-alert bug the claude path already had to fix once — and the pane-capture staleness sweep that fixed it there is Claude-dialog-shaped and cannot help here. -⚠️ **Answers are READ FROM DISK for this mode, not scraped off the pane** (`src/deepseek-transcript.ts`, behind `GET /api/sessions/:id/last-response`). dsh writes a structured JSONL transcript at `$DSH_HOME/sessions///session.jsonl.zstd`, so it belongs with claude and codex rather than with the pane-segmented modes — and for dsh specifically the segmenter was not merely coarse but WRONG: dsh-TUI paints a full-screen splash, so a `last-response` call on a fresh dsh session returned its ASCII-art logo, which anything polling for a worker's first answer reads as an answer. Four mechanisms in that file are load-bearing. **(1)** dsh appends **one zstd FRAME per write**, and Node's `zlib` zstd decoder — one-shot AND streaming — stops at the first frame end: a real 56-line transcript decoded as 1 line / 158 bytes, i.e. the session header alone, so every call would have reported "nothing said yet" forever. `zstdFrameRanges()` walks frame and block headers (no decompression) to find exact boundaries and decompresses each frame; splitting on the 4-byte magic instead would corrupt everything after a magic sequence that happens to occur inside compressed data. zstd itself is resolved at RUNTIME (`zstdSupported()`), because it landed in Node 22.15 while the project floor is 22.0 and `@types/node` still does not declare it; without it the mode falls back to the pane exactly as before, which is why `readDeepSeekLastResponse()` distinguishes `null` ("this reader cannot run here") from an empty result ("read fine, nothing said yet"). **(2)** Every turn also records a plugin-sourced `user/message` (dsh's runtime-context snapshot: sandbox policy, approval policy, cwd), so only `source.kind === 'user'` is a real prompt. **(3)** A turn that ends in `reason.kind: 'error'` is surfaced as `Turn error: …` (and a non-error early stop such as `max-tokens` as `Turn ended: …`) rather than as an empty string, which an agent reads as "still thinking" through fifteen polls. **(4)** Reply text is assembled per (turn, step): a finalized `assistant/message` wins, and the streamed `assistant/chunk` / `text-chunks` deltas are consulted ONLY for a step that never finalized (so a partial answer is readable mid-turn without ever being appended twice) — ⚠️ and "finalized" is tracked as a SET of steps, not as non-empty text, because a step whose whole reply was reasoning strips to `''` at the `` boundary and would otherwise resurrect the raw, unstripped deltas in its place (measured on a real conversation). ⚠️ Session→transcript pairing is by the transcript's own header `cwd` plus a ±60 s boot window against the Codeman session's `createdAt`, never by reproducing dsh's directory mangling (which already has two forms on disk, `` and `session-`) and never by newest-mtime alone: mtime alone handed a freshly spawned worker its PREDECESSOR's answer in the same case directory, which is worse than saying nothing because an agent cannot tell a stale answer from a fresh one. The `DSH_HOME` override reaches the reader through the narrow `Session.deepSeekHomeOverride` getter rather than an `envOverrides` accessor, since that map can hold provider credentials; it is ephemeral by design (never persisted), so a session that overrode it and outlived a server restart resolves the default tree and reads as "nothing said yet". Tests: `test/deepseek-transcript.test.ts`. +⚠️ **Answers are READ FROM DISK for this mode, not scraped off the pane** (`src/deepseek-transcript.ts`, behind `GET /api/sessions/:id/last-response`). dsh writes a structured JSONL transcript at `$DSH_HOME/sessions///session.jsonl.zstd`, so it belongs with claude and codex rather than with the pane-segmented modes — and for dsh specifically the segmenter was not merely coarse but WRONG: dsh-TUI paints a full-screen splash, so a `last-response` call on a fresh dsh session returned its ASCII-art logo, which anything polling for a worker's first answer reads as an answer. Four mechanisms in that file are load-bearing. **(1)** dsh appends **one zstd FRAME per write**, and Node's `zlib` zstd decoder (one-shot AND streaming) before 24.22 and 26.10 (nodejs/node#65865) stops at the first frame end: a real 56-line transcript decoded as 1 line / 158 bytes, i.e. the session header alone, so every call would have reported "nothing said yet" forever. `zstdFrameRanges()` walks frame and block headers (no decompression) to find exact boundaries and decompresses each frame; splitting on the 4-byte magic instead would corrupt everything after a magic sequence that happens to occur inside compressed data. zstd itself is resolved at RUNTIME (`zstdSupported()`), because it landed in Node 22.15 while the project floor is 22.0 and `@types/node` still does not declare it; without it the mode falls back to the pane exactly as before, which is why `readDeepSeekLastResponse()` distinguishes `null` ("this reader cannot run here") from an empty result ("read fine, nothing said yet"). **(2)** Every turn also records a plugin-sourced `user/message` (dsh's runtime-context snapshot: sandbox policy, approval policy, cwd), so only `source.kind === 'user'` is a real prompt. **(3)** A turn that ends in `reason.kind: 'error'` is surfaced as `Turn error: …` (and a non-error early stop such as `max-tokens` as `Turn ended: …`) rather than as an empty string, which an agent reads as "still thinking" through fifteen polls. **(4)** Reply text is assembled per (turn, step): a finalized `assistant/message` wins, and the streamed `assistant/chunk` / `text-chunks` deltas are consulted ONLY for a step that never finalized (so a partial answer is readable mid-turn without ever being appended twice) — ⚠️ and "finalized" is tracked as a SET of steps, not as non-empty text, because a step whose whole reply was reasoning strips to `''` at the `` boundary and would otherwise resurrect the raw, unstripped deltas in its place (measured on a real conversation). ⚠️ Session→transcript pairing is by the transcript's own header `cwd` plus a ±60 s boot window against the Codeman session's `createdAt`, never by reproducing dsh's directory mangling (which already has two forms on disk, `` and `session-`) and never by newest-mtime alone: mtime alone handed a freshly spawned worker its PREDECESSOR's answer in the same case directory, which is worse than saying nothing because an agent cannot tell a stale answer from a fresh one. The `DSH_HOME` override reaches the reader through the narrow `Session.deepSeekHomeOverride` getter rather than an `envOverrides` accessor, since that map can hold provider credentials; it is ephemeral by design (never persisted), so a session that overrode it and outlived a server restart resolves the default tree and reads as "nothing said yet". Tests: `test/deepseek-transcript.test.ts`. ⚠️ **This is also what makes dsh the one non-claude mode the `codeman` agent skill drives like claude** (`skills/codeman/preamble.sh`, preamble 1.20.0): with a real end-of-turn signal AND a real transcript, `spawn_workers alpha beta:deepseek` is a mixed fleet in one call and `sendwait`/`last_text` need no per-mode variant. Two traps are handled in the preamble rather than left to the agent. **Readiness is not the stop signal**: the harness reports `idle` at BOOT roughly 300 ms before its composer paints (measured 2.26 s vs 2.56 s after spawn, twice), so a send-and-wait fired straight after `quick-start` resolves on the boot edge, reports a turn that never ran, and strands the prompt in a pane that was not yet accepting input — `spawn_worker`'s dsh branch gates on the composer glyph (`❯`, overridable via `DSH_READY_MARK`) instead, after which the boot edge is spent and unobservable. And `sendwait` asks for `wait:"stop,exit"` rather than the `wait:true` default set, because that set also carries `idle`, which for any external CLI is inferred from output stabilization: on a dsh worker whose TUI repaints rarely, a re-wait resolved in 0 ms with `signal:"idle"` on a turn that had three minutes left to run. The skill also sends `deepSeekConfig.permissionMode: 'danger-full-access'` for its own workers, matching what the Run button sends, because the harness default still asks and a worker parked on an approval row cannot finish a fan-out (the multi-user clamp still applies). diff --git a/docs/deepseek-integration.md b/docs/deepseek-integration.md index 874f6fd1..02af1002 100644 --- a/docs/deepseek-integration.md +++ b/docs/deepseek-integration.md @@ -235,9 +235,11 @@ worker's first answer reads as an answer. Three things about the file shaped the reader (`src/deepseek-transcript.ts`): - **It is one zstd FRAME per append, not one zstd stream.** `zstd -dc` decodes all - of them, Node's `zlib` zstd decoder stops at the first: a real 56-line - transcript came back as 1 line. The reader walks frame headers itself. On a Node - older than 22.15 (no zstd at all) the mode falls back to the pane, as before. + of them, Node's `zlib` zstd decoder before 24.22 and 26.10 stops at the first: + a real 56-line transcript came back as 1 line. The reader walks frame headers + itself, which stays correct on a Node that decodes every frame + (nodejs/node#65865). On a Node older than 22.15 (no zstd at all) the mode + falls back to the pane, as before. - **Not every `user/message` is the user.** Each turn also records a plugin-sourced runtime-context snapshot; only `source.kind === 'user'` is a prompt. diff --git a/src/deepseek-transcript.ts b/src/deepseek-transcript.ts index dc374f10..525188f7 100644 --- a/src/deepseek-transcript.ts +++ b/src/deepseek-transcript.ts @@ -26,13 +26,15 @@ * **1. One zstd FRAME per append, not one zstd stream.** The file is * `session.jsonl.zstd`, and dsh appends by compressing each batch of lines into * its own frame and writing it at the end. `zstd -dc` handles that (frames - * concatenate by definition), but Node's `zlib.zstdDecompress()` and - * `createZstdDecompress()` both stop at the first frame end: measured on a real - * 56-line transcript, Node returned 158 bytes / 1 line where the CLI returned - * 43,747 bytes / 56 lines. That is a silent truncation to the session header — - * every call would have reported "no answer yet" forever. `decodeZstdFrames()` - * below walks the frame headers itself and decompresses each frame, and - * `test/deepseek-transcript.test.ts` pins it against multi-frame fixtures. + * concatenate by definition), but before Node 24.22 and 26.10 + * (nodejs/node#65865) `zlib.zstdDecompress()` and `createZstdDecompress()` both + * stop at the first frame end, and the supported range still includes those + * versions: measured on a real 56-line transcript, Node returned 158 bytes / + * 1 line where the CLI returned 43,747 bytes / 56 lines. That is a silent + * truncation to the session header — every call would have reported "no answer + * yet" forever. `decodeZstdFrames()` below walks the frame headers itself and + * decompresses each frame, and `test/deepseek-transcript.test.ts` pins it + * against multi-frame fixtures. * * **2. The user's prompts are mixed with injected context.** Every turn also * writes a `user/message` whose source is a plugin (the runtime-context diff --git a/test/deepseek-transcript.test.ts b/test/deepseek-transcript.test.ts index e0837070..292aac14 100644 --- a/test/deepseek-transcript.test.ts +++ b/test/deepseek-transcript.test.ts @@ -4,10 +4,11 @@ * Two of these assertions exist because the obvious implementation was measured * to be wrong against real files: * - * - dsh appends ONE ZSTD FRAME PER WRITE, and Node's `zlib` zstd decoder stops - * at the first frame end. A 56-line transcript decoded as 1 line / 158 bytes, - * which reads as "the worker never answered" rather than as an error. The - * multi-frame fixtures below are the guard. + * - dsh appends ONE ZSTD FRAME PER WRITE, and Node's `zlib` zstd decoder + * before 24.22 and 26.10 stops at the first frame end. A 56-line transcript + * decoded as 1 line / 158 bytes, which reads as "the worker never answered" + * rather than as an error. Newer Node decodes every frame (nodejs/node#65865), + * so the multi-frame fixtures below pin the walker itself, not Node's decoder. * - a fresh worker in a case directory that had been used before answered its * first `last-response` with the PREVIOUS session's reply. Session-to- * transcript pairing is therefore its own describe block. @@ -67,10 +68,6 @@ describe.skipIf(!zstdSupported())('zstd frame walking', () => { const lines = Array.from({ length: 40 }, (_, i) => JSON.stringify({ type: 'noise', seq: i })); const buf = framed(lines); - // The one-shot decoder is what this module had to replace. - const oneShot = (zlib as unknown as { zstdDecompressSync?: (b: Buffer) => Buffer }).zstdDecompressSync!(buf); - expect(oneShot.toString('utf8').trim().split('\n')).toHaveLength(1); - expect(zstdFrameRanges(buf)).toHaveLength(40); expect(decodeZstdFrames(buf).trim().split('\n')).toHaveLength(40); });