feat(history): COD-549 read saved Codex conversations offline

This commit is contained in:
Aamer Akhter committed 2026-10-24 20:43:17 -04:00
1 parent 3e06f3c141
commit e8ea8b0715
10 files changed
+960 -8

No files matched your search

+5
View File
@@ -0,0 +1,5 @@
---
'aicodeman': minor
---
Add offline `codeman history read` for saved Codex transcripts by file, native session ID, or conversation family, with whole-turn topic/date filters and bounded local reads and output.
+52
View File
@@ -0,0 +1,52 @@
# Saved Codex history (offline)
Use `codeman history read` for retained local Codex JSONL transcripts. This command
works outside a managed session and never calls the HTTP API. Do not bootstrap
API credentials or require `CODEMAN_MUX` for this read-only job.
Choose exactly one selector:
```bash
codeman history read --file /path/to/rollout.jsonl
codeman history read --session-id aaaaaaaa --sessions-root /path/to/sessions
codeman history read --root-session-id aaaaaaaa-1111-4111-8111-111111111111 \
--topic 'authentication' --after 2026-10-01 --before 2026-10-24 \
--max-bytes 65536
```
`--session-id` accepts a native conversation UUID or unique prefix of at least
8 characters; identity comes from metadata, not filenames. `--root-session-id`
requires a full native UUID and includes transitive descendants found in metadata.
The default store is `$CODEX_HOME/sessions`, or `~/.codex/sessions` when unset.
`--file` selects one file and does not search for its ancestors or children.
Topic matching is a case-insensitive literal substring. Dates are inclusive
`YYYY-MM-DD` bounds. Each filter selects whole user/assistant turns; a turn is
retained if it has a date match and a topic match, which may be different messages
in that turn. Tool calls/results and injected context are omitted. Adjacent duplicate
native representations are collapsed; repeated messages from the same
representation are retained.
JSON is the default (`--json` or `--format json`); `--format text` is available.
Results identify source paths, native session/root IDs, timestamps, roles and phases,
plus scan counters and `complete`. A root ID is `null` when ancestry cannot be proven.
Report that uncertainty; never relabel the queried child as its own root.
`complete: false` reports malformed or partial records, skipped links, unattributed
files or missing ancestors. It does not prove the store includes every conversation
ever created, and a filtered result is only evidence for its selected scope.
Reads require regular UTF8 files and reject final-component symlinks, conflicting
metadata, ambiguous IDs, cyclic ancestry, invalid selectors and changed selected
files. Directory traversal skips symlinks. Parent directories of an explicitly
supplied path must be trusted; this is not a filesystem sandbox.
The default total metadata/transcript scan budget is 16 MiB; `--scan-bytes` allows
1–67108864 bytes. Directory scans are capped at 8192 entries and depth 8. The output
budget is 8192 bytes by default; `--max-bytes` allows 1–1048576 bytes, including the
final newline. Exhausted budgets fail with an error: output is never silently
truncated or split into partial JSON or turns. Narrow the store/filters or explicitly
raise the relevant budget. Missing evidence is an error, not an empty success.
This command does not inspect live panes, stage prompts, submit input or manage
sessions. Live session operations use the separate `codeman agent` commands and
require the managed-session environment and running server.
+11 -4
View File
@@ -8,12 +8,19 @@ description: >-
read their output, and clean up; where available, message claude workers directly
(Claude Code cross-session messaging). Use when asked to orchestrate or parallelize
work across Codeman sessions, get a second opinion from another CLI or model, watch
another session, or start and manage workers. Only usable inside a Codeman-managed
session (CODEMAN_MUX=1); refuse to act otherwise.
another session, start and manage workers, or retrieve saved local Codex history
by file/native ID or root and descendants with topic/date filters. Live API
operations require a Codeman-managed session (CODEMAN_MUX=1). Offline history
reads do not require a server or managed session.
---
# Driving Codeman from inside a session
For **saved Codex conversation history**, use `codeman history read` and
[reference/history.md](reference/history.md). This is a read-only local command:
route to it **before §0**, including outside a managed session. It needs no API
URL, credentials or running server. All live API operations below retain §0.
You are an agent running inside a Codeman-managed terminal session. Codeman is the
server that spawned you; its HTTP API can start, prompt, watch, and delete other
sessions.
@@ -43,8 +50,8 @@ machine has no `jq` for the preamble below.
## 0. Guard and bootstrap
If `CODEMAN_MUX` is not `1`, **stop and say so**. Do not guess an API URL; a server
you are not part of is not yours to drive.
For live API operations, if `CODEMAN_MUX` is not `1`, **stop and say so**.
Do not guess an API URL; a server you are not part of is not yours to drive.
Two failures come from your CLI, not from Codeman: name them to the user instead of
retrying.
@@ -0,0 +1,51 @@
# Saved Codex history (offline)
Use `codeman history read` for retained local Codex JSONL transcripts. This command
works outside a managed session and never calls the HTTP API. Do not bootstrap
API credentials or require `CODEMAN_MUX` for this read-only job.
Choose exactly one selector:
```bash
codeman history read --file /path/to/rollout.jsonl
codeman history read --session-id aaaaaaaa --sessions-root /path/to/sessions
codeman history read --root-session-id aaaaaaaa-1111-4111-8111-111111111111 \
--topic 'authentication' --after 2026-10-01 --before 2026-10-24 \
--max-bytes 65536
```
`--session-id` accepts a native conversation UUID or unique prefix of at least
8 characters; identity comes from metadata, not filenames. `--root-session-id`
requires a full native UUID and includes transitive descendants found in metadata.
The default store is `$CODEX_HOME/sessions`, or `~/.codex/sessions` when unset.
`--file` selects one file and does not search for its ancestors or children.
Topic matching is a case-insensitive literal substring. Dates are inclusive
`YYYY-MM-DD` bounds. Each filter selects whole user/assistant turns; a turn is
retained if it has a date match and a topic match, which may be different messages
in that turn. Tool calls/results and injected context are omitted. Adjacent duplicate
native representations are collapsed; repeated messages from the same
representation are retained.
JSON is the default (`--json` or `--format json`); `--format text` is available.
Results identify source paths, native session/root IDs, timestamps, roles and phases,
plus scan counters and `complete`. A root ID is `null` when ancestry cannot be proven.
Report that uncertainty; never relabel the queried child as its own root.
`complete: false` reports malformed or partial records, skipped links, unattributed
files or missing ancestors. It does not prove the store includes every conversation
ever created, and a filtered result is only evidence for its selected scope.
Reads require regular UTF8 files and reject final-component symlinks, conflicting
metadata, ambiguous IDs, cyclic ancestry, invalid selectors and changed selected
files. Directory traversal skips symlinks. Parent directories of an explicitly
supplied path must be trusted; this is not a filesystem sandbox.
The default total metadata/transcript scan budget is 16 MiB; `--scan-bytes` allows
1–67108864 bytes. Directory scans are capped at 8192 entries and depth 8. The output
budget is 8192 bytes by default; `--max-bytes` allows 1–1048576 bytes, including the
final newline. Exhausted budgets fail with an error: output is never silently
truncated or split into partial JSON or turns. Narrow the store/filters or explicitly
raise the relevant budget. Missing evidence is an error, not an empty success.
This command does not inspect live panes, stage prompts, submit input or manage
sessions. For live operations return to the managed-session guard in the skill.
+11 -4
View File
@@ -8,12 +8,19 @@ description: >-
read their output, and clean up; where available, message claude workers directly
(Claude Code cross-session messaging). Use when asked to orchestrate or parallelize
work across Codeman sessions, get a second opinion from another CLI or model, watch
another session, or start and manage workers. Only usable inside a Codeman-managed
session (CODEMAN_MUX=1); refuse to act otherwise.
another session, start and manage workers, or retrieve saved local Codex history
by file/native ID or root and descendants with topic/date filters. Live API
operations require a Codeman-managed session (CODEMAN_MUX=1). Offline history
reads do not require a server or managed session.
---
# Driving Codeman from inside a session
For **saved Codex conversation history**, use `codeman history read` and
[reference/history.md](reference/history.md). This is a read-only local command:
route to it **before §0**, including outside a managed session. It needs no API
URL, credentials or running server. All live API operations below retain §0.
You are an agent running inside a Codeman-managed terminal session. Codeman is the
server that spawned you; its HTTP API can start, prompt, watch, and delete other
sessions.
@@ -43,8 +50,8 @@ machine has no `jq` for the preamble below.
## 0. Guard and bootstrap
If `CODEMAN_MUX` is not `1`, **stop and say so**. Do not guess an API URL; a server
you are not part of is not yours to drive.
For live API operations, if `CODEMAN_MUX` is not `1`, **stop and say so**.
Do not guess an API URL; a server you are not part of is not yours to drive.
Two failures come from your CLI, not from Codeman: name them to the user instead of
retrying.
+51
View File
@@ -0,0 +1,51 @@
# Saved Codex history (offline)
Use `codeman history read` for retained local Codex JSONL transcripts. This command
works outside a managed session and never calls the HTTP API. Do not bootstrap
API credentials or require `CODEMAN_MUX` for this read-only job.
Choose exactly one selector:
```bash
codeman history read --file /path/to/rollout.jsonl
codeman history read --session-id aaaaaaaa --sessions-root /path/to/sessions
codeman history read --root-session-id aaaaaaaa-1111-4111-8111-111111111111 \
--topic 'authentication' --after 2026-10-01 --before 2026-10-24 \
--max-bytes 65536
```
`--session-id` accepts a native conversation UUID or unique prefix of at least
8 characters; identity comes from metadata, not filenames. `--root-session-id`
requires a full native UUID and includes transitive descendants found in metadata.
The default store is `$CODEX_HOME/sessions`, or `~/.codex/sessions` when unset.
`--file` selects one file and does not search for its ancestors or children.
Topic matching is a case-insensitive literal substring. Dates are inclusive
`YYYY-MM-DD` bounds. Each filter selects whole user/assistant turns; a turn is
retained if it has a date match and a topic match, which may be different messages
in that turn. Tool calls/results and injected context are omitted. Adjacent duplicate
native representations are collapsed; repeated messages from the same
representation are retained.
JSON is the default (`--json` or `--format json`); `--format text` is available.
Results identify source paths, native session/root IDs, timestamps, roles and phases,
plus scan counters and `complete`. A root ID is `null` when ancestry cannot be proven.
Report that uncertainty; never relabel the queried child as its own root.
`complete: false` reports malformed or partial records, skipped links, unattributed
files or missing ancestors. It does not prove the store includes every conversation
ever created, and a filtered result is only evidence for its selected scope.
Reads require regular UTF8 files and reject final-component symlinks, conflicting
metadata, ambiguous IDs, cyclic ancestry, invalid selectors and changed selected
files. Directory traversal skips symlinks. Parent directories of an explicitly
supplied path must be trusted; this is not a filesystem sandbox.
The default total metadata/transcript scan budget is 16 MiB; `--scan-bytes` allows
1–67108864 bytes. Directory scans are capped at 8192 entries and depth 8. The output
budget is 8192 bytes by default; `--max-bytes` allows 1–1048576 bytes, including the
final newline. Exhausted budgets fail with an error: output is never silently
truncated or split into partial JSON or turns. Narrow the store/filters or explicitly
raise the relevant budget. Missing evidence is an error, not an empty success.
This command does not inspect live panes, stage prompts, submit input or manage
sessions. For live operations return to the managed-session guard in the skill.
+47
View File
@@ -0,0 +1,47 @@
/** Local offline transcript commands. Deliberately separate from the agent API guard. */
import type { Command } from 'commander';
import { formatLocalHistory, HISTORY_DEFAULT_SCAN_BYTES, readLocalHistory } from './local-history.js';
export function registerHistoryCommands(program: Command): Command {
const history = program
.command('history')
.description('Read local saved native transcripts without a running server');
history
.command('read')
.description('Read Codex history by file, native id, or root conversation and descendants')
.option('--file <path>', 'Explicit local JSONL transcript (regular file only)')
.option('--session-id <id>', 'Native Codex conversation UUID or unique prefix (at least 8 characters)')
.option('--root-session-id <uuid>', 'Native root UUID, including transitive child conversations')
.option('--sessions-root <path>', 'Local Codex sessions directory (default: CODEX_HOME/sessions)')
.option('--topic <text>', 'Case-insensitive literal match, preserving whole matching turns')
.option('--after <date>', 'Inclusive YYYY-MM-DD lower bound; preserves whole matching turns')
.option('--before <date>', 'Inclusive YYYY-MM-DD upper bound; preserves whole matching turns')
.option('--max-bytes <n>', 'Maximum complete output bytes (1–1048576)', '8192')
.option(
'--scan-bytes <n>',
'Total metadata and selected transcript read budget (1–67108864)',
String(HISTORY_DEFAULT_SCAN_BYTES)
)
.option('--format <format>', 'json or text', 'json')
.option('--json', 'JSON output (also the default)')
.action(async (options: Record<string, string | boolean | undefined>) => {
try {
const format = options.json ? 'json' : options.format;
if (format !== 'json' && format !== 'text') throw new Error('--format must be json or text');
const result = await readLocalHistory({
file: options.file as string | undefined,
sessionId: options.sessionId as string | undefined,
rootSessionId: options.rootSessionId as string | undefined,
sessionsRoot: options.sessionsRoot as string | undefined,
topic: options.topic as string | undefined,
after: options.after as string | undefined,
before: options.before as string | undefined,
scanBytes: Number(options.scanBytes),
});
program.configureOutput().writeOut?.(`${formatLocalHistory(result, format, Number(options.maxBytes))}\n`);
} catch (error) {
program.error(error instanceof Error ? error.message : String(error), { exitCode: 1, code: 'history.read' });
}
});
return history;
}
+2
View File
@@ -29,6 +29,7 @@ import {
import type { AgentSkillDir } from './config/cli-registry/types.js';
import { getCli, listClis } from './config/cli-registry/registry.js';
import { registerAgentCommands } from './cli-agent.js';
import { registerHistoryCommands } from './cli-history.js';
import { resolveMcpConfig, runMcpStdio } from './mcp-server.js';
import { getSessionManager } from './session-manager.js';
import { getTaskQueue } from './task-queue.js';
@@ -324,6 +325,7 @@ skillCmd
// ============ Agent Commands (session-to-session, any CLI mode) ============
registerAgentCommands(program);
registerHistoryCommands(program);
// ============ MCP server (any MCP client drives Codeman) ============
+484
View File
@@ -0,0 +1,484 @@
/** Offline, local-operator Codex history. No HTTP routes or running session required. */
import { constants } from 'node:fs';
import { isUtf8 } from 'node:buffer';
import { lstat, open, opendir, type FileHandle } from 'node:fs/promises';
import { homedir } from 'node:os';
import { join, resolve } from 'node:path';
export const HISTORY_DEFAULT_SCAN_BYTES = 16 * 1024 * 1024;
export const HISTORY_MAX_SCAN_BYTES = 64 * 1024 * 1024;
export const HISTORY_MAX_OUTPUT_BYTES = 1024 * 1024;
const ID = /^[0-9a-f]{8}(?:[0-9a-f-]{0,28})$/i;
const FULL_ID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
const METADATA_BYTES = 128 * 1024;
type Json = Record<string, unknown>;
export interface LocalHistoryOptions {
file?: string;
sessionId?: string;
rootSessionId?: string;
sessionsRoot?: string;
topic?: string;
after?: string;
before?: string;
scanBytes?: number;
maxEntries?: number;
}
export interface HistoryRecord {
rootSessionId: string | null;
sessionId: string;
timestamp: string;
role: 'user' | 'assistant';
phase: string;
message: string;
}
interface Source {
path: string;
sessionId: string;
parentId?: string;
rootId?: string;
timestamp: string;
fingerprint?: string;
}
export interface LocalHistoryResult {
source: 'codex-jsonl';
complete: boolean;
sessions: Array<{ sessionId: string; rootSessionId: string | null; parentSessionId?: string; path: string }>;
records: HistoryRecord[];
scan: {
bytesRead: number;
entries: number;
malformedRecords: number;
partialRecords: number;
skippedSymlinks: number;
unattributedFiles: number;
};
}
interface Budget {
remaining: number;
entriesRemaining: number;
scan: LocalHistoryResult['scan'];
}
function object(value: unknown): Json {
return value && typeof value === 'object' && !Array.isArray(value) ? (value as Json) : {};
}
function string(value: unknown): string | undefined {
return typeof value === 'string' ? value : undefined;
}
function validDate(value: string | undefined): boolean {
return (
value === undefined ||
(/^\d{4}-\d{2}-\d{2}$/.test(value) &&
Number.isFinite(Date.parse(value)) &&
new Date(value).toISOString().slice(0, 10) === value)
);
}
function validate(options: LocalHistoryOptions): void {
if ([options.file, options.sessionId, options.rootSessionId].filter((v) => v !== undefined).length !== 1)
throw new Error('Select exactly one of --file, --session-id, or --root-session-id');
if (options.file !== undefined && !options.file) throw new Error('A nonempty file is required');
if (options.sessionId !== undefined && !ID.test(options.sessionId)) throw new Error('Invalid native session id');
if (options.rootSessionId !== undefined && !FULL_ID.test(options.rootSessionId))
throw new Error('Invalid root session id');
if (
!validDate(options.after) ||
!validDate(options.before) ||
(options.after && options.before && options.after > options.before)
)
throw new Error('Invalid date or date range (use YYYY-MM-DD)');
if (options.topic !== undefined && !options.topic.trim()) throw new Error('Invalid empty topic');
for (const [value, max] of [
[options.scanBytes ?? HISTORY_DEFAULT_SCAN_BYTES, HISTORY_MAX_SCAN_BYTES],
[options.maxEntries ?? 8192, 65536],
])
if (!Number.isSafeInteger(value) || value < 1 || value > max) throw new Error('Invalid scan budget');
}
async function regularFile(path: string): Promise<FileHandle> {
// O_NOFOLLOW prevents a final-component link swap between lstat and open. fstat
// validates the opened descriptor, not a previously inspected pathname.
if (!(await lstat(path)).isFile()) throw new Error('History source must be a regular file, not a symlink');
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
if (!(await handle.stat()).isFile()) {
await handle.close();
throw new Error('History source must be a regular file');
}
return handle;
}
async function readChunk(handle: FileHandle, position: number, size: number, budget: Budget): Promise<Buffer> {
if (size > budget.remaining)
throw new Error('History scan byte budget exhausted; narrow the store or raise --scan-bytes');
const buffer = Buffer.alloc(size);
const { bytesRead } = await handle.read(buffer, 0, size, position);
budget.remaining -= bytesRead;
budget.scan.bytesRead += bytesRead;
return buffer.subarray(0, bytesRead);
}
function metadata(value: unknown, path: string): Source | undefined {
const entry = object(value);
if (entry.type !== 'session_meta') return undefined;
const p = object(entry.payload);
const sessionId = string(p.id) ?? string(p.session_id);
if (!sessionId || !FULL_ID.test(sessionId)) return undefined;
const source = object(p.source);
const spawn = object(object(source.subagent).thread_spawn);
const parentId = string(p.parent_thread_id) ?? string(p.parent_session_id) ?? string(spawn.parent_thread_id);
const declaredRoot = string(p.root_session_id) ?? string(p.session_id);
return {
path,
sessionId: sessionId.toLowerCase(),
parentId: parentId && FULL_ID.test(parentId) ? parentId.toLowerCase() : undefined,
rootId:
declaredRoot && declaredRoot !== sessionId && FULL_ID.test(declaredRoot) ? declaredRoot.toLowerCase() : undefined,
timestamp: string(p.timestamp) ?? string(entry.timestamp) ?? '',
};
}
async function readMetadata(path: string, budget: Budget): Promise<Source | undefined> {
const handle = await regularFile(path);
try {
const before = await handle.stat();
const parts: Buffer[] = [];
let offset = 0;
let parsed = 0;
while (offset < before.size && offset < METADATA_BYTES) {
const chunk = await readChunk(
handle,
offset,
Math.min(4096, before.size - offset, METADATA_BYTES - offset),
budget
);
if (!chunk.length) break;
parts.push(chunk);
offset += chunk.length;
const buffer = Buffer.concat(parts);
let end = buffer.indexOf(10, parsed);
while (end >= 0) {
const line = buffer.subarray(parsed, end);
if (!isUtf8(line)) throw new Error('History metadata contains invalid UTF8');
try {
const found = metadata(JSON.parse(line.toString('utf8')), path);
if (found) {
const after = await handle.stat();
const fingerprint = (value: typeof before) =>
`${value.dev}:${value.ino}:${value.size}:${value.mtimeMs}:${value.ctimeMs}`;
if (fingerprint(before) !== fingerprint(after))
throw new Error('History source changed during metadata scan');
return { ...found, fingerprint: fingerprint(before) };
}
} catch (error) {
if (!(error instanceof SyntaxError)) throw error;
}
parsed = end + 1;
end = buffer.indexOf(10, parsed);
}
}
if (offset < before.size)
throw new Error('History metadata scan budget exhausted; cannot prove family completeness');
return undefined;
} finally {
await handle.close();
}
}
async function findSources(root: string, budget: Budget): Promise<Source[]> {
if (!(await lstat(root)).isDirectory()) throw new Error('Sessions root must be a real directory, not a symlink');
const sources: Source[] = [];
const visit = async (dir: string, depth: number): Promise<void> => {
if (depth > 8) throw new Error('History directory scan depth budget exhausted');
const entries = await opendir(dir);
for await (const entry of entries) {
if (--budget.entriesRemaining < 0) throw new Error('History directory scan entry budget exhausted');
budget.scan.entries++;
const path = join(dir, entry.name);
if (entry.isSymbolicLink()) {
budget.scan.skippedSymlinks++;
continue;
}
if (entry.isDirectory()) {
// Recheck immediately before descending; never intentionally follow links.
if (!(await lstat(path)).isDirectory()) throw new Error('History directory changed during scan');
await visit(path, depth + 1);
} else if (entry.isFile() && entry.name.endsWith('.jsonl')) {
const source = await readMetadata(path, budget);
if (source) sources.push(source);
else budget.scan.unattributedFiles++;
}
}
};
await visit(root, 0);
return sources;
}
function contentText(value: unknown): string {
if (typeof value === 'string') return value;
if (!Array.isArray(value)) return '';
return value
.map((block) => {
const item = object(block);
return typeof item.type === 'string' && ['text', 'input_text', 'output_text'].includes(item.type.toLowerCase())
? (string(item.text) ?? '')
: '';
})
.filter(Boolean)
.join('\n');
}
interface Message {
representation: 'event' | 'modern' | 'response';
role: 'user' | 'assistant';
message: string;
phase: string;
timestamp: string;
}
function message(value: unknown): Message | undefined {
const entry = object(value);
const p = object(entry.payload);
if (entry.isSidechain || p.isSidechain) return undefined;
let role: Message['role'] | undefined;
let representation: Message['representation'] = 'event';
let text = '';
let phase = string(p.phase) ?? 'progress';
if (entry.type === 'event_msg') {
if (p.type === 'user_message' || p.type === 'agent_message') {
role = p.type === 'user_message' ? 'user' : 'assistant';
text = contentText(p.message);
} else if (p.type === 'item_completed') {
representation = 'modern';
const item = object(p.item);
if (item.type === 'UserMessage' || item.type === 'AgentMessage') {
role = item.type === 'UserMessage' ? 'user' : 'assistant';
text = contentText(item.content);
phase = string(item.phase) ?? phase;
}
}
} else if (
entry.type === 'response_item' &&
(p.type === 'message' || p.type === undefined) &&
(p.role === 'user' || p.role === 'assistant')
) {
role = p.role;
representation = 'response';
text = contentText(p.content);
// Codex context is not a human turn. Do not omit arbitrary markdown prompts.
if (
role === 'user' &&
/^(?:# AGENTS\.md instructions|<environment_context>|<INSTRUCTIONS>|<turn_aborted>)/.test(text)
)
return undefined;
}
if (!role || !text) return undefined;
return {
role,
message: text,
phase,
representation,
timestamp: string(entry.timestamp) ?? string(p.timestamp) ?? '',
};
}
async function readMessages(source: Source, budget: Budget): Promise<Message[]> {
const handle = await regularFile(source.path);
try {
const before = await handle.stat();
if (`${before.dev}:${before.ino}:${before.size}:${before.mtimeMs}:${before.ctimeMs}` !== source.fingerprint)
throw new Error('History source changed after attribution; retry explicitly');
const buffer = await readChunk(handle, 0, before.size, budget);
const after = await handle.stat();
if (
buffer.length !== before.size ||
before.size !== after.size ||
before.mtimeMs !== after.mtimeMs ||
before.ctimeMs !== after.ctimeMs
)
throw new Error('History source changed while reading; retry explicitly');
if (!isUtf8(buffer)) throw new Error('History source contains invalid UTF8');
const lines = buffer.toString('utf8').split('\n');
const partial = lines.pop();
if (partial) budget.scan.partialRecords++;
const records: Message[] = [];
let representations = new Set<Message['representation']>();
for (const line of lines) {
if (!line.trim()) continue;
let parsed: unknown;
try {
parsed = JSON.parse(line);
} catch {
budget.scan.malformedRecords++;
continue;
}
if (object(parsed).type === 'session_meta') {
const declared = metadata(parsed, source.path);
if (
!declared ||
declared.sessionId !== source.sessionId ||
declared.parentId !== source.parentId ||
declared.rootId !== source.rootId
)
throw new Error('History source contains conflicting native metadata; attribution refused');
continue;
}
const item = message(parsed);
if (!item) continue;
const previous = records.at(-1);
// Native events and response_item commonly duplicate one accepted message.
if (
previous?.role === item.role &&
previous.message === item.message &&
!representations.has(item.representation)
) {
representations.add(item.representation);
if (!previous.timestamp) previous.timestamp = item.timestamp;
if (item.phase === 'final_answer') previous.phase = item.phase;
continue;
}
records.push(item);
representations = new Set([item.representation]);
}
return records;
} finally {
await handle.close();
}
}
function selectedTurns(messages: Message[], options: LocalHistoryOptions): Message[] {
const turns: Message[][] = [];
for (const item of messages) {
if (item.role === 'user' || !turns.length) turns.push([]);
turns.at(-1)!.push(item);
}
const needle = options.topic?.trim().toLocaleLowerCase();
return turns
.filter((turn) => {
const dateMatches =
(!options.after && !options.before) ||
turn.some((item) => {
const date = item.timestamp.slice(0, 10);
return (
validDate(date) && (!options.after || date >= options.after) && (!options.before || date <= options.before)
);
});
return dateMatches && (!needle || turn.some((item) => item.message.toLocaleLowerCase().includes(needle)));
})
.flat();
}
/** Native ancestry is metadata evidence, not the conversation selected by a query. */
function nativeRoot(source: Source, sources: Source[]): string | null {
const seen = new Set<string>();
let current = source;
while (true) {
if (seen.has(current.sessionId)) throw new Error('Native history contains cyclic ancestry');
seen.add(current.sessionId);
if (current.rootId) return current.rootId;
if (!current.parentId) return current.sessionId;
const parents = sources.filter((candidate) => candidate.sessionId === current.parentId);
if (parents.length === 0) return null;
if (parents.length > 1) throw new Error('Native ancestor id is ambiguous');
current = parents[0];
}
}
export async function readLocalHistory(options: LocalHistoryOptions): Promise<LocalHistoryResult> {
validate(options);
const scan: LocalHistoryResult['scan'] = {
bytesRead: 0,
entries: 0,
malformedRecords: 0,
partialRecords: 0,
skippedSymlinks: 0,
unattributedFiles: 0,
};
const budget: Budget = {
remaining: options.scanBytes ?? HISTORY_DEFAULT_SCAN_BYTES,
entriesRemaining: options.maxEntries ?? 8192,
scan,
};
let sources: Source[];
let attributionSources: Source[];
if (options.file !== undefined) {
const source = await readMetadata(resolve(options.file), budget);
if (!source) throw new Error('History source is missing valid native session metadata');
sources = [source];
attributionSources = sources;
} else {
const root = options.sessionsRoot ?? join(process.env.CODEX_HOME || join(homedir(), '.codex'), 'sessions');
const all = await findSources(resolve(root), budget);
const wanted = (options.sessionId ?? options.rootSessionId)!.toLowerCase();
const matches = all.filter((source) => source.sessionId.toLowerCase().startsWith(wanted));
if (!matches.length) throw new Error('Native session missing from history store');
if (matches.length !== 1) throw new Error('Native session id is ambiguous');
const target = matches[0];
attributionSources = all;
const queryRootId = target.sessionId;
sources = [target];
if (options.rootSessionId) {
const selected = new Set([target.sessionId]);
let changed = true;
while (changed) {
changed = false;
for (const source of all) {
if (
!selected.has(source.sessionId) &&
(source.rootId === queryRootId || (source.parentId && selected.has(source.parentId)))
) {
selected.add(source.sessionId);
sources.push(source);
changed = true;
}
}
}
for (const source of sources) {
if (all.filter((other) => other.sessionId === source.sessionId).length !== 1)
throw new Error('Native family contains ambiguous duplicated session ids');
}
}
}
sources.sort((a, b) => a.timestamp.localeCompare(b.timestamp) || a.sessionId.localeCompare(b.sessionId));
const roots = new Map(sources.map((source) => [source.sessionId, nativeRoot(source, attributionSources)]));
const records: HistoryRecord[] = [];
for (const source of sources) {
records.push(
...selectedTurns(await readMessages(source, budget), options).map((item) => ({
rootSessionId: roots.get(source.sessionId) ?? null,
sessionId: source.sessionId,
role: item.role,
message: item.message,
phase: item.phase,
timestamp: item.timestamp,
}))
);
}
if (!records.length) throw new Error('No matching user/assistant evidence found');
return {
source: 'codex-jsonl',
complete:
scan.malformedRecords === 0 &&
scan.partialRecords === 0 &&
scan.skippedSymlinks === 0 &&
scan.unattributedFiles === 0 &&
[...roots.values()].every((root) => root !== null),
sessions: sources.map((source) => ({
sessionId: source.sessionId,
rootSessionId: roots.get(source.sessionId) ?? null,
...(source.parentId ? { parentSessionId: source.parentId } : {}),
path: source.path,
})),
records,
scan,
};
}
/** Output never splits a JSON document, UTF8 character, record, or selected turn. */
export function formatLocalHistory(
result: LocalHistoryResult,
format: 'json' | 'text' = 'json',
maxBytes = 8192
): string {
if (!Number.isSafeInteger(maxBytes) || maxBytes < 1 || maxBytes > HISTORY_MAX_OUTPUT_BYTES)
throw new Error('Invalid output byte budget');
const body =
format === 'json'
? JSON.stringify(result)
: [
`source=${result.source} complete=${result.complete} scannedBytes=${result.scan.bytesRead}`,
...result.records.map(
(row) =>
`[root=${row.rootSessionId} session=${row.sessionId} ${row.timestamp} ${row.role} ${row.phase}]\n${row.message}`
),
].join('\n\n');
if (Buffer.byteLength(body) + 1 > maxBytes)
throw new Error('History output byte budget exceeded; narrow filters or raise --max-bytes');
return body;
}
+246
View File
@@ -0,0 +1,246 @@
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { mkdtemp, mkdir, rm, symlink, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import * as history from '../src/local-history.js';
import { Command } from 'commander';
import { registerHistoryCommands } from '../src/cli-history.js';
const ROOT = 'aaaaaaaa-1111-4111-8111-111111111111';
const CHILD = 'bbbbbbbb-1111-4111-8111-111111111111';
const GRANDCHILD = 'cccccccc-1111-4111-8111-111111111111';
let dir: string;
const meta = (id: string, parent?: string) => ({
type: 'session_meta',
payload: {
id,
timestamp: '2026-10-24T01:00:00Z',
...(parent ? { source: { subagent: { thread_spawn: { parent_thread_id: parent } } } } : {}),
},
});
const row = (role: string, text: string, timestamp = '2026-10-24T01:00:00Z') => ({
type: 'response_item',
timestamp,
payload: { type: 'message', role, content: [{ type: role === 'user' ? 'input_text' : 'output_text', text }] },
});
async function fixture(id: string, records: unknown[], parent?: string) {
const path = join(dir, `rollout-${id}.jsonl`);
await writeFile(path, [meta(id, parent), ...records].map((r) => JSON.stringify(r)).join('\n') + '\n');
return path;
}
beforeEach(async () => {
dir = await mkdtemp(join(tmpdir(), 'local-history-'));
});
afterEach(async () => {
await rm(dir, { recursive: true, force: true });
});
describe('offline native history', () => {
it('reads a native transcript without a server and preserves identity and timestamps', async () => {
const file = await fixture(ROOT, [row('user', 'question'), row('assistant', 'answer')]);
const result = await history.readLocalHistory({ file });
expect(result.records.map((r) => r.message)).toEqual(['question', 'answer']);
expect(result.records[0]).toMatchObject({
sessionId: ROOT,
rootSessionId: ROOT,
timestamp: '2026-10-24T01:00:00Z',
});
expect(result.complete).toBe(true);
});
it('resolves metadata identities rather than filenames', async () => {
await fixture(ROOT, [row('assistant', 'yes')]);
const result = await history.readLocalHistory({ sessionId: ROOT.slice(0, 8), sessionsRoot: dir });
expect(result.records[0].sessionId).toBe(ROOT);
});
it('rejects ambiguous native prefixes', async () => {
await fixture(ROOT, [row('assistant', 'one')]);
await fixture('aaaaaaaa-2222-4222-8222-222222222222', [row('assistant', 'two')]);
await expect(history.readLocalHistory({ sessionId: 'aaaaaaaa', sessionsRoot: dir })).rejects.toThrow(/ambiguous/);
});
it('gathers transitive children and retains a whole topic/date matching turn', async () => {
await fixture(ROOT, [row('user', 'other'), row('assistant', 'unrelated')]);
await fixture(
CHILD,
[row('user', 'topic question'), row('assistant', 'answer after midnight', '2026-10-25T00:01:00Z')],
ROOT
);
await fixture(GRANDCHILD, [row('user', 'topic child'), row('assistant', 'child answer')], CHILD);
const result = await history.readLocalHistory({
rootSessionId: ROOT,
sessionsRoot: dir,
topic: 'TOPIC',
before: '2026-10-24',
});
expect(result.sessions).toHaveLength(3);
expect(result.records.map((r) => r.message)).toEqual([
'topic question',
'answer after midnight',
'topic child',
'child answer',
]);
expect(result.records.every((r) => r.rootSessionId === ROOT)).toBe(true);
});
it('resolves the real native root when querying a child by id', async () => {
await fixture(ROOT, [row('assistant', 'root')]);
await fixture(CHILD, [row('assistant', 'child')], ROOT);
await fixture(GRANDCHILD, [row('assistant', 'grandchild')], CHILD);
const result = await history.readLocalHistory({ sessionId: GRANDCHILD, sessionsRoot: dir });
expect(result.records[0].rootSessionId).toBe(ROOT);
expect(result.sessions[0]).toMatchObject({ parentSessionId: CHILD, rootSessionId: ROOT });
});
it('labels a file-only child root unknown rather than inventing an ancestor', async () => {
const file = await fixture(CHILD, [row('assistant', 'child')], ROOT);
const result = await history.readLocalHistory({ file });
expect(result.records[0].rootSessionId).toBeNull();
expect(result.sessions[0]).toMatchObject({ parentSessionId: ROOT, rootSessionId: null });
});
it('labels missing ancestors unknown and refuses cyclic ancestry', async () => {
await fixture(CHILD, [row('assistant', 'child')], ROOT);
expect(
(await history.readLocalHistory({ sessionId: CHILD, sessionsRoot: dir })).records[0].rootSessionId
).toBeNull();
await fixture(ROOT, [row('assistant', 'root')], CHILD);
await expect(history.readLocalHistory({ sessionId: CHILD, sessionsRoot: dir })).rejects.toThrow(/cyclic/);
});
it('supports legacy shared session_id genealogy', async () => {
await fixture(ROOT, [row('assistant', 'root')]);
await writeFile(
join(dir, 'legacy.jsonl'),
[{ type: 'session_meta', payload: { id: CHILD, session_id: ROOT } }, row('assistant', 'legacy child')]
.map((r) => JSON.stringify(r))
.join('\n') + '\n'
);
expect((await history.readLocalHistory({ rootSessionId: ROOT, sessionsRoot: dir })).sessions).toHaveLength(2);
});
it('parses modern events and removes adjacent duplicate representations without erasing repeated turns', async () => {
const file = await fixture(ROOT, [
row('user', 'question'),
{
type: 'event_msg',
payload: {
type: 'item_completed',
item: { type: 'UserMessage', content: [{ type: 'Text', text: 'question' }] },
},
},
row('assistant', 'answer'),
{
type: 'event_msg',
payload: {
type: 'item_completed',
item: { type: 'AgentMessage', phase: 'final_answer', content: [{ type: 'Text', text: 'answer' }] },
},
},
{ type: 'event_msg', payload: { type: 'user_message', message: 'question' } },
{ type: 'event_msg', payload: { type: 'agent_message', message: 'answer' } },
]);
expect((await history.readLocalHistory({ file })).records.map((r) => r.message)).toEqual([
'question',
'answer',
'question',
'answer',
]);
});
it('preserves genuinely repeated identical messages from the same representation', async () => {
const file = await fixture(ROOT, [row('user', 'repeat'), row('user', 'repeat'), row('assistant', 'answer')]);
expect((await history.readLocalHistory({ file })).records.map((r) => r.message)).toEqual([
'repeat',
'repeat',
'answer',
]);
});
it('labels unreadable metadata as incomplete rather than hiding possible children', async () => {
await fixture(ROOT, [row('assistant', 'root')]);
await writeFile(join(dir, 'unattributed.jsonl'), '{bad metadata}\n');
const result = await history.readLocalHistory({ rootSessionId: ROOT, sessionsRoot: dir });
expect(result.complete).toBe(false);
expect(result.scan.unattributedFiles).toBe(1);
});
it('refuses conflicting session metadata instead of attributing another conversation answer', async () => {
const file = await fixture(ROOT, [row('user', 'root question'), meta(CHILD), row('assistant', 'child answer')]);
await expect(history.readLocalHistory({ file })).rejects.toThrow(/conflicting.*metadata/);
});
it('accepts repeated metadata only when native identity and genealogy agree', async () => {
const file = await fixture(ROOT, [row('user', 'question'), meta(ROOT), row('assistant', 'answer')]);
expect((await history.readLocalHistory({ file })).records.map((record) => record.message)).toEqual([
'question',
'answer',
]);
await fixture(CHILD, [meta(CHILD, GRANDCHILD), row('assistant', 'child answer')], ROOT);
await expect(history.readLocalHistory({ sessionId: CHILD, sessionsRoot: dir })).rejects.toThrow(
/conflicting.*metadata/
);
});
it('labels skipped malformed and pending partial records', async () => {
const file = await fixture(ROOT, [row('assistant', 'complete')]);
const { appendFile } = await import('node:fs/promises');
await appendFile(file, 'not json\n{"type":');
const result = await history.readLocalHistory({ file });
expect(result.complete).toBe(false);
expect(result.scan.malformedRecords).toBe(1);
expect(result.scan.partialRecords).toBe(1);
expect(result.records[0].message).toBe('complete');
});
it('rejects invalid UTF8, nonregular inputs and symlinks', async () => {
const file = await fixture(ROOT, [row('assistant', 'answer')]);
const linked = join(dir, 'linked.jsonl');
await symlink(file, linked);
await expect(history.readLocalHistory({ file: linked })).rejects.toThrow(/regular|symlink/);
await expect(history.readLocalHistory({ file: dir })).rejects.toThrow(/regular/);
await writeFile(file, Buffer.from([0xff, 0x0a]));
await expect(history.readLocalHistory({ file })).rejects.toThrow(/UTF8/);
});
it('fails closed for byte budgets and preserves valid JSON for successful output', async () => {
const file = await fixture(ROOT, [row('assistant', '😀'.repeat(80))]);
await expect(history.readLocalHistory({ file, scanBytes: 32 })).rejects.toThrow(/scan.*budget/);
const result = await history.readLocalHistory({ file });
expect(() => history.formatLocalHistory(result, 'json', 40)).toThrow(/output.*budget/);
const output = history.formatLocalHistory(result, 'json', 8192);
expect(Buffer.byteLength(output)).toBeLessThanOrEqual(8192);
expect(JSON.parse(output).records[0].message).toBe('😀'.repeat(80));
});
it.each([{ after: '2026-02-30' }, { before: 'bad' }, { after: '2026-10-25', before: '2026-10-24' }, { topic: ' ' }])(
'rejects invalid filters %j',
async (filter) => {
const file = await fixture(ROOT, [row('assistant', 'answer')]);
await expect(history.readLocalHistory({ file, ...filter })).rejects.toThrow(/date|topic/);
}
);
it('rejects missing selectors, multiple selectors, empty matches and absent native ids', async () => {
const file = await fixture(ROOT, [row('assistant', 'answer')]);
await expect(history.readLocalHistory({})).rejects.toThrow(/exactly one/);
await expect(history.readLocalHistory({ file, sessionId: ROOT })).rejects.toThrow(/exactly one/);
await expect(history.readLocalHistory({ file, topic: 'absent' })).rejects.toThrow(/evidence/);
await expect(history.readLocalHistory({ sessionId: CHILD, sessionsRoot: dir })).rejects.toThrow(/missing/);
});
it('rejects exhausted directory traversal rather than returning an incomplete family', async () => {
await fixture(ROOT, [row('assistant', 'root')]);
await mkdir(join(dir, 'nested'));
await fixture(CHILD, [row('assistant', 'child')], ROOT);
await expect(history.readLocalHistory({ rootSessionId: ROOT, sessionsRoot: dir, maxEntries: 1 })).rejects.toThrow(
/scan.*budget/
);
});
it('executes the offline CLI with complete JSON and no internal parser fields', async () => {
const file = await fixture(ROOT, [row('assistant', 'offline')]);
let output = '';
const program = new Command().exitOverride().configureOutput({
writeOut: (text) => {
output += text;
},
writeErr: () => {},
});
registerHistoryCommands(program);
await program.parseAsync(['history', 'read', '--file', file, '--json'], { from: 'user' });
const result = JSON.parse(output);
expect(result.records[0].message).toBe('offline');
expect(result.records[0]).not.toHaveProperty('representation');
await expect(
program.parseAsync(['history', 'read', '--file', file, '--session-id', ROOT], { from: 'user' })
).rejects.toThrow(/exactly one/);
});
it('registers offline commands and reports invalid selectors without process exit', async () => {
const program = new Command().exitOverride();
registerHistoryCommands(program);
expect(program.commands.find((c) => c.name() === 'history')?.commands.map((c) => c.name())).toEqual(['read']);
});
});