test(story-mode): real sessions as a regression corpus

Nine transcripts from the review rounds, each a real Claude Code worker
building a small program (a Python CLI with a fix loop, a web page with
node:test, a bash script with a background helper, a long turn with a
prompt added mid-turn, teammates then an interrupt, a question then tests
first then a push, a text plan with a scratch dry run, /compact then plan
mode, a test failed on purpose). Each is cut down to the lines and fields
the reducer reads, with every path, user name and email rewritten and the
file contents of Reads left out; replaying them before and after gives
the same pages.

The expected file beside each pins every page (panel kinds, captions,
balloons, counts), so a change that alters a story fails with a diff of
exactly what changed; STORY_CORPUS_UPDATE=1 regenerates them. The beats
the reviews were about are also stated outright.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-10-18 11:45:41 +02:00
1 parent f4898a626f
commit f464548541
19 files changed
+2313

No files matched your search

@@ -0,0 +1,92 @@
[
{
"prompt": "Build a small Python command-line tool wcount.py in this folder that prints the line, word and chara",
"origin": "human",
"complete": true,
"interrupted": false,
"durationMs": 192853,
"toolCalls": 13,
"filesChanged": 3,
"added": 220,
"removed": 1,
"helpers": 0,
"helpersRunning": 0,
"panels": [
{
"kind": "mission",
"ok": null,
"story": "You give Claude a job.",
"director": "Prompt · 52 words",
"balloon": "Build a small Python command-line tool wcount.py in this folder that prints the line, word and character counts of the files given as arguments, like wc, with a --json flag that prints the counts as JSON, plus pytest te…",
"tests": null,
"events": 1
},
{
"kind": "investigate",
"ok": null,
"story": "Claude reads 1 file to find its way around.",
"director": "Bash ×2 · ls -la && git log --oneline -5 2>&1; git branch -a; python3… · 1 error",
"balloon": null,
"tests": null,
"events": 2
},
{
"kind": "backup",
"ok": null,
"story": "Claude asks the advisor for a second opinion.",
"director": "Advisor · advisor",
"balloon": null,
"tests": null,
"events": 1
},
{
"kind": "loop",
"ok": true,
"story": "Fix and test, 2 rounds. It passes in the end.",
"director": "3 checks · 5 edits · +218 −1 · last: ok",
"balloon": "All 19 tests pass, ruff check is clean, and the output matches real wc column for column.",
"tests": {
"passed": 19,
"failed": 0
},
"events": 7
},
{
"kind": "save",
"ok": true,
"story": "Claude saves the work.",
"director": "git commit -q -m \"chore: ignore Python and pytest caches · writes 1 file",
"balloon": null,
"tests": null,
"events": 1
},
{
"kind": "backup",
"ok": null,
"story": "Claude asks the advisor for a second opinion.",
"director": "Advisor · advisor",
"balloon": null,
"tests": null,
"events": 1
},
{
"kind": "run",
"ok": null,
"story": "Claude works in the terminal.",
"director": "Bash · python3 wcount.py nope nope2; echo \"exit=$?\"; wc nope nope2…",
"balloon": null,
"tests": null,
"events": 1
},
{
"kind": "complete",
"ok": null,
"story": "Done in 3 minutes.",
"director": "turn_duration · 3 min 13 s · 13 tool calls · 3 files changed",
"balloon": "I built wcount.py and test_wcount.py.",
"tests": null,
"events": 0
}
]
}
]