diff --git a/.changeset/offline-codex-history.md b/.changeset/offline-codex-history.md index 57090231..9ed54d37 100644 --- a/.changeset/offline-codex-history.md +++ b/.changeset/offline-codex-history.md @@ -2,4 +2,4 @@ 'aicodeman': minor --- -Add offline `codeman history read` for saved Codex transcripts by file, native session ID, or conversation family, with whole-turn topic/date filters and bounded local reads and output. +Add offline `codeman history read` for saved Codex transcripts by file, native session ID, or conversation family (spawned sub-agents and forks included), with whole-turn topic and UTC date filters. A forked or spawned thread's copy of its parent's history is dropped, so every record belongs to the thread that wrote it. Store attribution has its own budget (`--metadata-bytes`), separate from the transcript budget (`--scan-bytes`), and every bound is raisable (`--max-entries` too), so id lookups keep working on stores of thousands of rollouts. A store file that cannot be attributed, an unreadable directory, or a rollout a running Codex is still appending to no longer fails the lookup; each is reported through `complete: false`. Assistant records carry Codex's own phase and user records none. The Past Sessions scanner and the offline reader now share one Codex rollout format module, including the injected-context rule. diff --git a/README.md b/README.md index bb3c142c..3fc1a495 100644 --- a/README.md +++ b/README.md @@ -996,6 +996,18 @@ codeman agent rm "$SID" # any session except t Rules the commands enforce rather than document: they refuse outside a Codeman session and never guess a URL; `send` transmits printable text plus Enter only (a control byte such as `Ctrl+C` is `app_exit` in opencode — ESC exists solely as `interrupt`, which never appends Enter); `rm` refuses an empty id, an unprovable self id and a prefix match in either direction. Ids may be the 8-character prefixes `ls` prints (resolved through the list; an ambiguous prefix refuses, anything shorter than 8 characters refuses on every verb). A prompt that starts with `-` goes after `--` (`send "$SID" -- "- fix the bug"`). The echo of the prompt you sent is output too: a `--match` marker that appears verbatim in the prompt matches at once, before the worker has done anything, so the prompt asks for it in halves. A remote session whose host is asleep answers a fire-and-forget `send` with `buffered` (Codeman wakes the host and types the prompt once the pane is back) or `dropped` (over the wake buffer's cap, nothing will be typed: exit `1`). Exit codes: `0` delivered/matched/signal, `1` error, `2` timeout, `3` the worker exited or the wait ended without an answer (`delivered:false`, `ended:true`), `4` refused. `spawn` prints the id alone on stdout (prose goes to stderr), so `SID=$(…)` captures exactly the id. `--json` prints the envelope's `data` for every verb. `--until stop` on a mode without hook signals is the server's 400, passed through — the marker path (`--match`) is the answer there, exactly as for the skill. +### `codeman history read`: saved Codex conversations, offline + +`codeman history read` reads Codex's own saved rollouts (`$CODEX_HOME/sessions`, else `~/.codex/sessions`) straight from disk: no server, no managed session, nothing written. Unlike `codeman agent` it works anywhere, including on a machine where Codeman is not running. + +```bash +codeman history read --session-id 019fc604 # one conversation, by native id or 8+ character prefix +codeman history read --root-session-id --topic auth # a conversation and every sub-agent or fork under it +codeman history read --file /path/to/rollout.jsonl --format text +``` + +Filters keep whole user/assistant turns (`--topic` is a literal, case-insensitive match; `--after`/`--before` are inclusive UTC dates). A forked or spawned thread's copy of its parent's history is dropped, so every record belongs to the thread that wrote it. Output is JSON by default and carries `complete: false` whenever the evidence is partial (a file that could not be attributed, a record a running Codex is still writing, an unknown ancestor). Every bound fails closed and is raisable: `--metadata-bytes` (attributing the store, default 1 GiB), `--max-entries` (default 65536), `--scan-bytes` (the selected transcripts, default 16 MiB) and `--max-bytes` (output, default 8192). The full reference is [`skills/codeman/reference/history.md`](skills/codeman/reference/history.md). + ### MCP server (`codeman mcp`): drive Codeman from any MCP client `codeman mcp` is a [Model Context Protocol](https://modelcontextprotocol.io) server on stdio, so any MCP client (Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor, ...) can start Codeman sessions in any installed agent CLI, prompt them, wait for them, read their answers and clean up, from inside Codeman or from outside it. Like `codeman agent`, it is a client of the HTTP API: no new route, auth and ownership unchanged. diff --git a/docs/offline-history.md b/docs/offline-history.md deleted file mode 100644 index 219aabc1..00000000 --- a/docs/offline-history.md +++ /dev/null @@ -1,52 +0,0 @@ -# Saved Codex history (offline) - -Use `codeman history read` for retained local Codex JSONL transcripts. This command -works outside a managed session and never calls the HTTP API. Do not bootstrap -API credentials or require `CODEMAN_MUX` for this read-only job. - -Choose exactly one selector: - -```bash -codeman history read --file /path/to/rollout.jsonl -codeman history read --session-id aaaaaaaa --sessions-root /path/to/sessions -codeman history read --root-session-id aaaaaaaa-1111-4111-8111-111111111111 \ - --topic 'authentication' --after 2026-10-01 --before 2026-10-24 \ - --max-bytes 65536 -``` - -`--session-id` accepts a native conversation UUID or unique prefix of at least -8 characters; identity comes from metadata, not filenames. `--root-session-id` -requires a full native UUID and includes transitive descendants found in metadata. -The default store is `$CODEX_HOME/sessions`, or `~/.codex/sessions` when unset. -`--file` selects one file and does not search for its ancestors or children. - -Topic matching is a case-insensitive literal substring. Dates are inclusive -`YYYY-MM-DD` bounds. Each filter selects whole user/assistant turns; a turn is -retained if it has a date match and a topic match, which may be different messages -in that turn. Tool calls/results and injected context are omitted. Adjacent duplicate -native representations are collapsed; repeated messages from the same -representation are retained. - -JSON is the default (`--json` or `--format json`); `--format text` is available. -Results identify source paths, native session/root IDs, timestamps, roles and phases, -plus scan counters and `complete`. A root ID is `null` when ancestry cannot be proven. -Report that uncertainty; never relabel the queried child as its own root. -`complete: false` reports malformed or partial records, skipped links, unattributed -files or missing ancestors. It does not prove the store includes every conversation -ever created, and a filtered result is only evidence for its selected scope. - -Reads require regular UTF8 files and reject final-component symlinks, conflicting -metadata, ambiguous IDs, cyclic ancestry, invalid selectors and changed selected -files. Directory traversal skips symlinks. Parent directories of an explicitly -supplied path must be trusted; this is not a filesystem sandbox. - -The default total metadata/transcript scan budget is 16 MiB; `--scan-bytes` allows -1–67108864 bytes. Directory scans are capped at 8192 entries and depth 8. The output -budget is 8192 bytes by default; `--max-bytes` allows 1–1048576 bytes, including the -final newline. Exhausted budgets fail with an error: output is never silently -truncated or split into partial JSON or turns. Narrow the store/filters or explicitly -raise the relevant budget. Missing evidence is an error, not an empty success. - -This command does not inspect live panes, stage prompts, submit input or manage -sessions. Live session operations use the separate `codeman agent` commands and -require the managed-session environment and running server. diff --git a/docs/wiki/Codeman-Skill.md b/docs/wiki/Codeman-Skill.md index dcd57f08..6417606e 100644 --- a/docs/wiki/Codeman-Skill.md +++ b/docs/wiki/Codeman-Skill.md @@ -123,8 +123,11 @@ message a worker directly instead of typing into its terminal. ## What it will and will not do -- **It self-gates.** Outside a Codeman session (`CODEMAN_MUX` unset) it refuses to act and - never guesses an API URL, so a global install costs an unrelated agent session nothing. +- **It self-gates.** Outside a Codeman session (`CODEMAN_MUX` unset) it refuses every live + API operation and never guesses an API URL, so a global install cannot drive a server it + is not part of. The one exception is reading saved Codex conversations with + `codeman history read` (see [Driving Codeman From An Agent](Driving-Codeman-From-An-Agent#reading-saved-codex-conversations)), + a local, read-only command that needs no server or session. - **Unprompted, it may only** start sessions, prompt them, and delete the ones **it created in that conversation, by exact id**, behind a guard that refuses to delete the agent's own session. @@ -153,6 +156,7 @@ The skill checks for both and says so instead of acting. | [`reference/recipes.md`](https://github.com/Ark0N/Codeman/blob/master/skills/codeman/reference/recipes.md) | Worked flows: claude, DeepSeek Harness and shell workers, fan-out, watching a blocked worker, messaging. Loaded on demand. | | [`reference/endpoints.md`](https://github.com/Ark0N/Codeman/blob/master/skills/codeman/reference/endpoints.md) | Endpoint tables, error codes, the signals each mode emits, capacity limits. Loaded on demand. | | [`reference/messaging.md`](https://github.com/Ark0N/Codeman/blob/master/skills/codeman/reference/messaging.md) | Talking to claude workers directly through Claude Code's cross-session messaging. Loaded on demand. | +| [`reference/history.md`](https://github.com/Ark0N/Codeman/blob/master/skills/codeman/reference/history.md) | Reading saved Codex conversations offline with `codeman history read`: selectors, filters, forks, budgets. Loaded on demand. | Only `SKILL.md` sits in the agent's context; the references load when a job needs them, so a quick run stays quick. diff --git a/docs/wiki/Driving-Codeman-From-An-Agent.md b/docs/wiki/Driving-Codeman-From-An-Agent.md index 6b2b337f..6e7ce85c 100644 --- a/docs/wiki/Driving-Codeman-From-An-Agent.md +++ b/docs/wiki/Driving-Codeman-From-An-Agent.md @@ -70,6 +70,32 @@ codeman agent rm "$SID" # any session except t - **Exit codes** are the same for every verb: `0` done, `1` error, `2` timeout, `3` the worker exited, `4` refused. `--json` prints the response's `data`. +## Reading saved Codex conversations + +`codeman history read` is the one command here that needs no server and no Codeman +session: it reads Codex's own saved rollouts (`$CODEX_HOME/sessions`, else +`~/.codex/sessions`) from disk and writes nothing. + +```bash +codeman history read --session-id 019fc604 # one conversation, by id or 8+ character prefix +codeman history read --root-session-id --topic auth # it and every sub-agent or fork under it +codeman history read --file /path/to/rollout.jsonl --format text +``` + +- **Filters** keep whole user/assistant turns: `--topic` is a literal, case-insensitive + match, and `--after`/`--before` are inclusive UTC dates. +- **Forks and sub-agents**: a child's copy of its parent's history is dropped, so every + record belongs to the thread that wrote it. +- **`complete: false`** in the JSON means the evidence is partial: a store file that could + not be attributed, a record a running Codex is still writing, an ancestor not found. +- **Budgets** fail closed and are raisable: `--metadata-bytes` (attributing the store, + default 1 GiB), `--max-entries` (65536), `--scan-bytes` (the selected transcripts, + 16 MiB) and `--max-bytes` (output, 8192). `--json` and `--format text` together are an + error. + +The full reference ships with the skill as +[`reference/history.md`](https://github.com/Ark0N/Codeman/blob/master/skills/codeman/reference/history.md). + ## The MCP server `codeman mcp` is a [Model Context Protocol](https://modelcontextprotocol.io) server, so any MCP diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md index 50c108bb..3a0cd566 100644 --- a/docs/wiki/Home.md +++ b/docs/wiki/Home.md @@ -42,7 +42,7 @@ codeman web # then open http://localhost:3000 **Driving it from code** - [The Codeman Skill](Codeman-Skill) - one prompt, a whole crew: an agent starting and supervising worker sessions. -- [Driving Codeman From An Agent](Driving-Codeman-From-An-Agent) - the `codeman agent` commands, the MCP server, wait primitives. +- [Driving Codeman From An Agent](Driving-Codeman-From-An-Agent) - the `codeman agent` commands, the MCP server, wait primitives, reading saved Codex conversations offline. - [HTTP API](HTTP-API) - the envelope, auth, the endpoint map, SSE events. - [Hooks And Integrations](Hooks-And-Integrations) - events flowing back into Codeman. diff --git a/plugins/codeman/skills/codeman/reference/history.md b/plugins/codeman/skills/codeman/reference/history.md index 0fa87205..bcdfc92c 100644 --- a/plugins/codeman/skills/codeman/reference/history.md +++ b/plugins/codeman/skills/codeman/reference/history.md @@ -16,36 +16,62 @@ codeman history read --root-session-id aaaaaaaa-1111-4111-8111-111111111111 \ `--session-id` accepts a native conversation UUID or unique prefix of at least 8 characters; identity comes from metadata, not filenames. `--root-session-id` -requires a full native UUID and includes transitive descendants found in metadata. -The default store is `$CODEX_HOME/sessions`, or `~/.codex/sessions` when unset. -`--file` selects one file and does not search for its ancestors or children. +requires a full native UUID and includes transitive descendants found in metadata +(spawned sub-agents and forks). The default store is `$CODEX_HOME/sessions`, or +`~/.codex/sessions` when unset; a store directory that is itself a symlink is +followed, links inside it are skipped. `--file` selects one file and does not +search for its ancestors or children. Topic matching is a case-insensitive literal substring. Dates are inclusive -`YYYY-MM-DD` bounds. Each filter selects whole user/assistant turns; a turn is -retained if it has a date match and a topic match, which may be different messages -in that turn. Tool calls/results and injected context are omitted. Adjacent duplicate -native representations are collapsed; repeated messages from the same -representation are retained. +`YYYY-MM-DD` bounds compared as UTC calendar dates. Each filter selects whole +user/assistant turns; a turn is retained if it has a date match and a topic match, +which may be different messages in that turn. Tool calls/results and injected +context (rows Codex adds that start with `#` or `<`, such as AGENTS.md and +``) are omitted. Adjacent duplicate native representations +are collapsed; repeated messages from the same representation are retained. -JSON is the default (`--json` or `--format json`); `--format text` is available. +A forked or spawned thread's file starts with a copy of its parent's history. That +copy is dropped (it is in the parent's own file), so each record belongs to the +thread that wrote it. A later `session_meta` for any thread that is not a declared +ancestor still refuses the file. + +JSON is the default (`--json` or `--format json`); `--format text` prints one block +per record. `--json` together with `--format text` is an error. Results identify source paths, native session/root IDs, timestamps, roles and phases, -plus scan counters and `complete`. A root ID is `null` when ancestry cannot be proven. +plus scan counters and `complete`. `phase` is Codex's own (`commentary` or +`final_answer`) on assistant records and `null` on user records or when none was +recorded. A root ID is `null` when ancestry cannot be proven. Report that uncertainty; never relabel the queried child as its own root. -`complete: false` reports malformed or partial records, skipped links, unattributed -files or missing ancestors. It does not prove the store includes every conversation -ever created, and a filtered result is only evidence for its selected scope. +`complete: false` reports malformed or partial records (a running Codex may be +mid-write), skipped links, unattributed files, unreadable directories, missing +ancestors, or an inherited copy whose end could not be found or proven. It does not +prove the store includes every conversation ever created, and a filtered result is +only evidence for its selected scope. Reads require regular UTF8 files and reject final-component symlinks, conflicting -metadata, ambiguous IDs, cyclic ancestry, invalid selectors and changed selected -files. Directory traversal skips symlinks. Parent directories of an explicitly -supplied path must be trusted; this is not a filesystem sandbox. +metadata, ambiguous IDs, cyclic ancestry and invalid selectors. A store file that +cannot be attributed (invalid UTF8 in its head, no metadata in its first 128 KiB, +changing while its head is read) is counted in `unattributedFiles` instead of +failing the lookup. A selected rollout that a running Codex is appending to is read +as a snapshot up to its last complete line; one that was replaced or truncated +fails. Parent directories of an explicitly supplied path must be trusted; this is +not a filesystem sandbox. -The default total metadata/transcript scan budget is 16 MiB; `--scan-bytes` allows -1–67108864 bytes. Directory scans are capped at 8192 entries and depth 8. The output -budget is 8192 bytes by default; `--max-bytes` allows 1–1048576 bytes, including the -final newline. Exhausted budgets fail with an error: output is never silently -truncated or split into partial JSON or turns. Narrow the store/filters or explicitly -raise the relevant budget. Missing evidence is an error, not an empty success. +Budgets, each failing with an error rather than a partial answer: + +| Flag | Bounds | Default | Maximum | +| ---- | ------ | ------- | ------- | +| `--metadata-bytes` | head reads attributing every rollout in the store | 1 GiB | 16 GiB | +| `--max-entries` | directory entries visited | 65536 | 1048576 | +| `--scan-bytes` | the selected transcripts | 16 MiB | 256 MiB | +| `--max-bytes` | output, including the final newline | 8192 | 1048576 | + +Each rollout's `session_meta` carries Codex's base instructions (about 20 KiB), so +attributing a store costs about that much per rollout, charged to +`--metadata-bytes`, never to `--scan-bytes`. Directory depth is capped at 8 (Codex +writes `YYYY/MM/DD`). Output is never silently truncated or split into partial +JSON or turns. Narrow the store/filters or explicitly raise the relevant budget. +Missing evidence is an error, not an empty success. This command does not inspect live panes, stage prompts, submit input or manage sessions. For live operations return to the managed-session guard in the skill. diff --git a/skills/codeman/reference/history.md b/skills/codeman/reference/history.md index 0fa87205..bcdfc92c 100644 --- a/skills/codeman/reference/history.md +++ b/skills/codeman/reference/history.md @@ -16,36 +16,62 @@ codeman history read --root-session-id aaaaaaaa-1111-4111-8111-111111111111 \ `--session-id` accepts a native conversation UUID or unique prefix of at least 8 characters; identity comes from metadata, not filenames. `--root-session-id` -requires a full native UUID and includes transitive descendants found in metadata. -The default store is `$CODEX_HOME/sessions`, or `~/.codex/sessions` when unset. -`--file` selects one file and does not search for its ancestors or children. +requires a full native UUID and includes transitive descendants found in metadata +(spawned sub-agents and forks). The default store is `$CODEX_HOME/sessions`, or +`~/.codex/sessions` when unset; a store directory that is itself a symlink is +followed, links inside it are skipped. `--file` selects one file and does not +search for its ancestors or children. Topic matching is a case-insensitive literal substring. Dates are inclusive -`YYYY-MM-DD` bounds. Each filter selects whole user/assistant turns; a turn is -retained if it has a date match and a topic match, which may be different messages -in that turn. Tool calls/results and injected context are omitted. Adjacent duplicate -native representations are collapsed; repeated messages from the same -representation are retained. +`YYYY-MM-DD` bounds compared as UTC calendar dates. Each filter selects whole +user/assistant turns; a turn is retained if it has a date match and a topic match, +which may be different messages in that turn. Tool calls/results and injected +context (rows Codex adds that start with `#` or `<`, such as AGENTS.md and +``) are omitted. Adjacent duplicate native representations +are collapsed; repeated messages from the same representation are retained. -JSON is the default (`--json` or `--format json`); `--format text` is available. +A forked or spawned thread's file starts with a copy of its parent's history. That +copy is dropped (it is in the parent's own file), so each record belongs to the +thread that wrote it. A later `session_meta` for any thread that is not a declared +ancestor still refuses the file. + +JSON is the default (`--json` or `--format json`); `--format text` prints one block +per record. `--json` together with `--format text` is an error. Results identify source paths, native session/root IDs, timestamps, roles and phases, -plus scan counters and `complete`. A root ID is `null` when ancestry cannot be proven. +plus scan counters and `complete`. `phase` is Codex's own (`commentary` or +`final_answer`) on assistant records and `null` on user records or when none was +recorded. A root ID is `null` when ancestry cannot be proven. Report that uncertainty; never relabel the queried child as its own root. -`complete: false` reports malformed or partial records, skipped links, unattributed -files or missing ancestors. It does not prove the store includes every conversation -ever created, and a filtered result is only evidence for its selected scope. +`complete: false` reports malformed or partial records (a running Codex may be +mid-write), skipped links, unattributed files, unreadable directories, missing +ancestors, or an inherited copy whose end could not be found or proven. It does not +prove the store includes every conversation ever created, and a filtered result is +only evidence for its selected scope. Reads require regular UTF8 files and reject final-component symlinks, conflicting -metadata, ambiguous IDs, cyclic ancestry, invalid selectors and changed selected -files. Directory traversal skips symlinks. Parent directories of an explicitly -supplied path must be trusted; this is not a filesystem sandbox. +metadata, ambiguous IDs, cyclic ancestry and invalid selectors. A store file that +cannot be attributed (invalid UTF8 in its head, no metadata in its first 128 KiB, +changing while its head is read) is counted in `unattributedFiles` instead of +failing the lookup. A selected rollout that a running Codex is appending to is read +as a snapshot up to its last complete line; one that was replaced or truncated +fails. Parent directories of an explicitly supplied path must be trusted; this is +not a filesystem sandbox. -The default total metadata/transcript scan budget is 16 MiB; `--scan-bytes` allows -1–67108864 bytes. Directory scans are capped at 8192 entries and depth 8. The output -budget is 8192 bytes by default; `--max-bytes` allows 1–1048576 bytes, including the -final newline. Exhausted budgets fail with an error: output is never silently -truncated or split into partial JSON or turns. Narrow the store/filters or explicitly -raise the relevant budget. Missing evidence is an error, not an empty success. +Budgets, each failing with an error rather than a partial answer: + +| Flag | Bounds | Default | Maximum | +| ---- | ------ | ------- | ------- | +| `--metadata-bytes` | head reads attributing every rollout in the store | 1 GiB | 16 GiB | +| `--max-entries` | directory entries visited | 65536 | 1048576 | +| `--scan-bytes` | the selected transcripts | 16 MiB | 256 MiB | +| `--max-bytes` | output, including the final newline | 8192 | 1048576 | + +Each rollout's `session_meta` carries Codex's base instructions (about 20 KiB), so +attributing a store costs about that much per rollout, charged to +`--metadata-bytes`, never to `--scan-bytes`. Directory depth is capped at 8 (Codex +writes `YYYY/MM/DD`). Output is never silently truncated or split into partial +JSON or turns. Narrow the store/filters or explicitly raise the relevant budget. +Missing evidence is an error, not an empty success. This command does not inspect live panes, stage prompts, submit input or manage sessions. For live operations return to the managed-session guard in the skill.