Merge #614 fix(files): prevent large folders starving File Viewer results

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/api-reference.md

@@ -1014,6 +1014,21 @@ while a `q` containing `/` is matched against paths relative to the folder walke
 also carries `worktree`, the same `{ name, path }`, whenever the session has a `liveWorktree`,
 whichever folder was walked and with or without `root=live`.
 
+The walk lists at most 5,000 files and folders and opens no more folders once it has read 250,000
+directory entries (the folder that crosses that line is read whole). It sets `truncated` when it
+stops with anything left, which includes a folder it never opened, even if that folder would have
+turned out to be empty. It is a per-directory round-robin (breadth-first): each folder found so far
+gets an equal share of what is left before any of them gets more, and a folder's own files count
+as one more share beside the subfolders it has just found, so one huge folder cannot use up the
+listing limit before its later siblings or its own subfolders are read, and the `matches` of a `q`
+search come back shallow-first. A folder made of thousands of small subfolders can still use most
+of the limit one level down, and folders that add up to the read limit can leave later ones
+unread. In the tree, a directory node either limit stopped short of also carries
+`incomplete: true`: its `children` hold what was listed before the limit, which can be nothing.
+The key is absent on every other node. A folder below the start of the walk that does not answer a
+bounded probe (a mount that stopped responding) is skipped like an unreadable one: it is listed
+with empty `children` and no `incomplete`.
+
 ## Session model (`displayModel`)
 
 Session state (`GET /api/v1/sessions`, the `session:updated` event) carries the model a

docs/architecture-invariants.md

@@ -740,6 +740,14 @@ Frontend `readmymind-ui.js`: header 🧠 marker-hidden (`btn-readmymind--hidden`
 
 ⚠️ **A query turns that endpoint into a FLAT match list rather than a nested tree**, and the walk deliberately recurses past non-matching directories, since the whole point of searching is to reach a file whose ancestors do not match. An empty or whitespace-only query compiles to `null`, which is what keeps the default tree response byte-identical when no search is requested.
 
+⚠️ **Both walks are a per-directory round-robin (breadth-first) over ONE 5,000-node budget.** They used to recurse depth-first through name-sorted folders, so an early folder could spend the whole budget: 4 of 57 top-level folders were shown in one real workspace. Each round divides what is left equally among the directories queued so far, and a folder's own files count as one more share beside the subfolders it has just found (`walkTurnEnd()`). Those subfolders are read in a later round, so a folder that spent all of its share on its own files left them nothing, alone in its round or beside a sibling, and search hits inside them were lost. A turn that found no subfolder holds nothing back. That is fair per directory, not per subtree: a folder of thousands of one-file subfolders still crowds out its neighbours one level down (`a/b/c/d/needle.ts` beside such a `z/` is found depth-first and missed here), and that trade is accepted, since a file browser wants the shallow levels first. Search results therefore arrive shallow-first.
+
+⚠️ **The budget caps what a listing returns, never what it reads.** Taking turns means opening every folder found so far, where the depth-first walk stopped reading soon after its budget ran out, and a share of 16 entries costs the same read as a share of 5,000. So `readWalkFolder()` keeps only what the listing can use (the entries it lists, sorted, no more than the budget has left, with `more` recording that it dropped some: such a cursor stays queued and is flagged), and `MAX_WALK_ENTRIES_READ` (250,000) stops opening folders. The folder that crosses that line is read whole, as any folder always was, the ones not opened yet wait in `unread` and are flagged, and what was already read keeps its turns. Without the two, 100 folders of 50,000 files cost one GET 5 million entries and a 400 MB heap died (which ends the server, not the request). The price is that folders adding up to the ceiling leave later ones unread: flagged, not hidden, but not the fairness the budget gets, and a depth-first walk over the same tree showed less.
+
+⚠️ **A folder below the walk's root is probed before it is read** (rule #516), because the round-robin reaches folders the depth-first walk stopped short of, each with an unbounded `readdir`. It is a bulk scan, so default options and the stall cap apply. Anything but `directory` is skipped like an unreadable folder, with no flag, since the row says "file limit". With the cap engaged (two unrelated paths stalled, by default) that is every folder below the root, until one answers. The root is not probed: a refused probe there would blank a healthy workspace.
+
+⚠️ A directory gets its node, with `children: []`, when it is counted and is read in a later round, so every cursor still queued after the loop, and every one in `unread`, is a folder that is unread or half read: it must carry `incomplete: true` or it reads as an empty folder, the failure that went from 0 to 635 such folders before the flag existed. `renderFileBrowserTree` draws one dim, inert row under it while it is expanded, and the row is not a `.file-tree-item`, so the click wiring leaves it alone. The budget check inside the round runs BEFORE each `shift()`, so a directory the round skips stays queued and keeps its flag. Hidden and excluded entries are dropped at the read, so they never leave a folder looking unfinished, but a folder found as the budget ended is flagged unread even if it would have turned out to be empty: only reading it could tell. The walk's root has no node, so `truncated` is its only signal, and a folder dropped at the depth cap is not flagged, since the row says "file limit" and it looks the same as it always has.
+
 ⚠️ **Globs are never compiled into a RegExp**: `*a*a*a…` translated to `^.*a.*a.*a…$` is a classic backtracking blowup evaluated synchronously against every walked path, so one pathological query would freeze the event loop for the whole server (the same reason `search-service.ts` is regex-free). `globMatch()` is a two-pointer wildcard walk instead, O(text · pattern) with both operands short by construction, and an overlong query (`MAX_QUERY_LENGTH`, 256) also compiles to `null` rather than running.
 
 Matching semantics: a query containing `/` matches the path relative to the folder being walked (`workingDir`, or the worktree of a `root=live` listing; the returned `path`s stay relative to `workingDir` either way), otherwise the bare entry name; globs match anchored and case-insensitively (`*` spans any run, slashes included, `?` exactly one character), everything else is a plain case-insensitive substring.

docs/wiki/Working-With-Files.md

@@ -10,6 +10,15 @@ A panel that browses the active session's working directory, or the git worktree
 moved into ([Git Worktrees](Git-Worktrees#when-an-agent-moves-into-a-worktree)). Its header
 button is on by default; if it is missing, re-enable it in **App Settings → Header & Panels**.
 
+The panel and its search list up to 5,000 files and folders per load, shallow levels first and
+taking turns between folders, so one huge folder cannot use up that limit before the ones after it,
+or the folders inside it, get a turn. They also stop opening folders after reading 250,000 entries,
+so a workspace with several enormous folders can still leave later ones unread. When a workspace is
+bigger than either limit the status line says `(truncated)`, and an expanded folder the scan did
+not finish shows **Not fully scanned (file limit reached)** rather than looking empty. A folder that
+does not answer, such as a network mount that went away, is skipped rather than waited on, and
+looks empty.
+
 It renders what it can:
 
 | Kind                     | Behaviour                                                                 |