mirror of
https://github.com/Ark0N/Codeman.git
synced 2026-11-16 19:29:33 +01:00
fix(git): the interval fetches one run per folder, and the rest of the review (review)
The scheduler is the part that runs with nobody watching, so it is decided per folder rather than per session: sessions sharing a folder are the same clone and share one run that both see, `inFlight`/`lastRunAt` are keyed by the resolved folder, the schedule is stamped when a folder is picked instead of when its run starts, and a tick is skipped while the previous one is still going. The click route joins a run already going for the same folder rather than starting a second fetch in one clone. The tick keeps ONE bound for the git processes of the whole run and hands it to every folder, and bounds its folders with a bound of its own: the same semaphore cannot serve both levels, because a folder holding a slot would then wait for a slot its own fetch needs. Both bounds are `FETCH_CONCURRENCY`, so the ceiling is unchanged. The bound claim in CLAUDE.md, the `Limiter` jsdoc and docs/api-reference.md now describes it as it is. Before anything is fetched the folder is probed with the bounded path probe, so a case on a mount that stopped answering is skipped and never touched. The rest of the review: the hooks are named as they really are (`reference-transaction`, `pre-auto-gc` through the maintenance a fetch triggers, config commands such as `credential.helper`) and `FETCH_CONFIG_ARGS` turns maintenance and gc off for an unattended run; the claim that an agent in the same repository is not disturbed is replaced by the ref-lock collision it actually is; the click route publishes its own run as `trigger: 'click'`; `autoSaveGitFetch` uses `_apiJson` so a 400 shows the error toast and applies the returned settings; and the two wording fixes. Tests: the scheduler pins one run per folder, the two bounds, the probe before anything runs, and that closing the session holding the override keeps the folder while another session is still open on it; the route test pins the click publish and the join, asserting the join before the held run is released — releasing first lets the run finish and clears the lane, so the joiner would be entitled to its own fetch and the test would pass for the wrong reason.
This commit is contained in:
1 parent
9432d9f9e2
commit
ea4318947b
20 files changed
+774
-209
No files matched your search
@@ -236,7 +236,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph
|
||||
|
||||
**Git status indicator** (`showGitStatus`, per-device, default OFF; `src/git-workspace-status.ts`, `routes/git-status-routes.ts`, `git-status-ui.js`): the bottom-bar indicator and its panel read `GET /api/sessions/:id/git-status` and `/git-diff`, read-only and offline on its own (it never fetches; fetching is the separate path below). ⚠️ git never runs on a repository at or inside a Docker case workspace (walk-up, scan and diff alike, and a cached repository list is re-checked against the CURRENT Docker roots), since a container could plant a clean filter that runs on the host; remote and Docker sessions answer `unsupported`. ⚠️ `git-diff` takes `repo`/`path`/`kind` only as keys matched against the workspace's own repository list (`findWorkspaceRepo()`) and that repository's current status, never as paths. ⚠️ Keep `--no-optional-locks`, `core.fsmonitor=false` and `log.showSignature=false` on every call and `--no-ext-diff --no-textconv` on diffs; clean filters still run, which is why the Docker rule exists. ⚠️ An enclosing repository at `$HOME` or above is ignored (`isUnrelatedAncestor()`), and is identified by one cached `rev-parse` before any full status runs. Everything git supplies renders via `textContent`. Tests: `test/git-workspace-status.test.ts`, `test/routes/git-status-routes.test.ts`, and `test/git-status.browser.test.ts` (browser suite, not in the gate).
|
||||
|
||||
**Git fetch** (`src/git-fetch.ts`, `src/git-fetch-scheduler.ts`, `routes/git-status-routes.ts`, the panel's **Fetch** button): `POST /api/sessions/:id/git-fetch` is the ONLY git command Codeman runs that WRITES, and the only one that runs a repository's own `fetch-post` hook — which is why it is opt-in and never runs on a timer until the user turns it on. ⚠️ Fetch updates remote-tracking refs only; it must never commit, check out, merge or touch the working tree, since an agent is often working in the same repository. ⚠️ The same Docker rule as above applies BEFORE any git runs (a repository at or inside a Docker case workspace is refused), and remote sessions answer `unsupported`. ⚠️ Keep it non-interactive (`gitNonInteractiveEnv()`: `GIT_TERMINAL_PROMPT=0` + BatchMode ssh) or a fetch hangs on an invisible login prompt, and bounded: `MAX_REPOS`/`MAX_REPOS_LIMIT` (12/50), `FETCH_CONCURRENCY` (4), `MAX_FETCH_OUTPUT_BYTES` (2 MB) and the git timeout per spawn. ⚠️ `buildFetchArgs()` is `['fetch', '--quiet']` and must stay verb-invariant (`test/git-workspace-status.test.ts`). ⚠️ Failure text reaching a client is control-stripped and credential-redacted (`sanitizeGitOutput`) and the browser writes it with `textContent`. Scheduler semantics: settings are read fresh each tick (`GIT_FETCH_TICK_INTERVAL_MS`, 15 s), a per-case override wins even while the app-wide switch is off, the interval is timed from the run START, runs never overlap, and remote/Docker cases are re-checked every tick. ⚠️ The two settings (`gitFetchEnabled`, `gitFetchIntervalSeconds`, off/300 by default, clamped 30–3600) are **synced**, not per-device: they must stay out of `displayKeys` and out of the strip destructure in settings-ui.js, or the scheduler reads nobody's value. Per-case state rides `SessionUpdated` (no new SSE event) as `gitFetch`. Tests: `test/git-fetch.test.ts`, `test/git-fetch-scheduler.test.ts`, `test/routes/session-routes-git-fetch-settings.test.ts`, `test/git-fetch-i18n.test.ts`, `test/app-settings-structure.test.ts`, `test/session-options-structure.test.ts`, and the Fetch block of `test/git-status.browser.test.ts` (browser suite, not in the gate).
|
||||
**Git fetch** (`src/git-fetch.ts`, `src/git-fetch-scheduler.ts`, `routes/git-status-routes.ts`, the panel's **Fetch** button): `POST /api/sessions/:id/git-fetch` is the ONLY git command Codeman runs that WRITES, and the only one that can run a repository's own hooks (`reference-transaction`, `pre-auto-gc` through the automatic maintenance a fetch triggers, config commands such as `credential.helper`) — which is why it is opt-in and never runs on a timer until the user turns it on. ⚠️ Fetch updates remote-tracking refs only; it must never commit, check out, merge or touch the working tree. ⚠️ It is not as harmless as a read: a fetch holds git's ref lock, so an agent's own `git pull` in the same clone can hit the same `cannot lock ref` — hence one folder is fetched by at most one run at a time (`fetchWorkspaceOnce` joins a run already going), and `FETCH_CONFIG_ARGS` (`-c core.fsmonitor=false -c maintenance.auto=false -c gc.auto=0`) keeps an unattended fetch from starting a gc in a repository the agent is using. ⚠️ `buildFetchArgs()` is `['fetch', '--quiet']` and must stay verb-invariant (`test/git-workspace-status.test.ts`); those `-c` overrides are added by the runner, not by `buildFetchArgs()`. ⚠️ The same Docker rule as above applies BEFORE any git runs (a repository at or inside a Docker case workspace is refused), and remote sessions answer `unsupported`. ⚠️ Keep it non-interactive (`gitNonInteractiveEnv()`: `GIT_TERMINAL_PROMPT=0` + BatchMode ssh) or a fetch hangs on an invisible login prompt, and bounded: `MAX_REPOS`/`MAX_REPOS_LIMIT` (12/50), `MAX_FETCH_OUTPUT_BYTES` (2 MB), the git timeout per spawn, and `FETCH_CONCURRENCY` (4) as ONE `Limiter` shared by the fetches of the whole tick — `createLimiter` keeps its count in the closure, so a limiter that delegated to `mapLimited` per call bounds each call, and 4 folders × 4 repositories would start 16 git processes. ⚠️ One semaphore cannot bound both the folders and the fetches: a folder holding a slot would then wait for a slot its own fetch needs, so the tick bounds its folders with `mapLimited` and hands the shared bound to every run. ⚠️ Failure text reaching a client is control-stripped and credential-redacted (`sanitizeGitOutput`) and the browser writes it with `textContent`. Scheduler semantics: settings are read fresh each tick (`GIT_FETCH_TICK_INTERVAL_MS`, 15 s), a per-session override wins even while the app-wide switch is off, the interval is timed from the run START, state is keyed by FOLDER (sessions sharing one folder are the same clone, so one run serves them all, the schedule is stamped at selection, and every session on a folder is remembered — closing the one that holds the override must not stop the folder's fetching while another session is still open on it), a tick is skipped while the previous one is still going, and a user-chosen folder is probed BOUNDED with the default options before anything runs (rule #516 — `unknown` is never read as `absent`). ⚠️ The two settings (`gitFetchEnabled`, `gitFetchIntervalSeconds`, off/300 by default, clamped 30–3600) are **synced**, not per-device: they must stay out of `displayKeys` and out of the strip destructure in settings-ui.js, or the scheduler reads nobody's value. Per-session state rides `SessionUpdated` (no new SSE event) as `gitFetch`, and the click route publishes its own run with `trigger: 'click'` — the UI's "by click" text depends on it. Tests: `test/git-fetch.test.ts`, `test/git-fetch-scheduler.test.ts`, `test/routes/session-routes-git-fetch-settings.test.ts`, `test/git-fetch-i18n.test.ts`, `test/app-settings-structure.test.ts`, `test/session-options-structure.test.ts`, and the Fetch block of `test/git-status.browser.test.ts` (browser suite, not in the gate).
|
||||
|
||||
**Raw file bodies are streamed and range-aware**: `file-raw`, the attachments `/raw` route, `GET /api/download` and the rendered-HTML route `GET /html-view/<cap>/...` share `sendFileBody()`, advertise `Accept-Ranges: bytes` and answer `Range` with `206` + `Content-Range` (single-range, parser in `src/web/http-range.ts`); without it `<video>` cannot seek. The size cap (`MAX_FILE_DOWNLOAD_BYTES`, default 2GB, env `CODEMAN_MAX_DOWNLOAD_BYTES`, `0` = unlimited) is a sanity bound, not memory protection; never reintroduce a whole-file buffer. ⚠️ Bodies go out via `reply.hijack()`, so `sendRawStream` must copy the status onto `reply.raw` by hand or a partial body ships as `200`. ⚠️ Closing the preview must pause and unload media (`_stopFilePreviewMedia`), since a detached `HTMLMediaElement` keeps playing. → [architecture-invariants#raw-file-bodies-streamed-and-range-aware](docs/architecture-invariants.md#raw-file-bodies-streamed-and-range-aware)
|
||||
|
||||
|
||||
@@ -460,7 +460,7 @@ PTY Output → 16ms Server Batch → DEC 2026 Wrap → SSE → Client rAF → xt
|
||||
- **Background daemon & service install** — `codeman web -d` runs the server detached with a pidfile, `~/.codeman/web.log`, and verified startup (it polls the server until it answers, so a port clash never reads as success); `codeman service install` writes a systemd user unit (Linux) or LaunchAgent (macOS) with your shell's PATH baked in, so an nvm or Homebrew `node`, `tmux` and `claude` are actually found. Secrets are never written into unit files
|
||||
- **Diagnostics in Settings** — **App Settings → System → Diagnostics → Run checks** runs `codeman doctor` on the server and lists Node, tmux, every agent CLI and the optional office tools with versions, paths and install hints. Besides the `PATH`, it also looks in each CLI's usual install directories (`~/.local/bin`, `~/.npm-global/bin` and the like), so most installs are found under a service with a minimal `PATH`. Admin only in multi-user mode.
|
||||
- **Self-update** — git-clone installs under systemd/launchd update in place from **App Settings → System → Updates**: it detects the latest release, auto-stashes a dirty tree, and streams build progress across the service restart (npm installs report as non-updatable)
|
||||
- **Git status in the bottom bar** — off by default (**App Settings → Header & Panels → Bottom bar → Git status**, per device). A small indicator at the right of the bottom bar shows the active session's repository at a glance: `● 3` uncommitted files, `↑ 2` commits not pushed, `⚠` merge conflicts, `✓` all committed and pushed. Click it for a draggable window listing the staged, not-staged, untracked and conflicted files (grouped under collapsed folders, or as a flat list if you turn that setting off) and the unpushed commits; **click a file to see its diff** (new files as all additions, deleted files as all removals), with **Open file** to jump to the viewer. A folder that holds several projects gets one collapsible section per repository found up to two levels down, all collapsed until you open them. **Fetch** in that window's header runs `git fetch` for the repositories it lists, on your click or on an interval you turn on (**App Settings → Bottom bar**, off by default; **Session Options → Git** overrides both the switch and the interval per case). Apart from that fetch Codeman never fetches or changes the repo, so "behind" is as of the last fetch; not shown for Docker or remote sessions.
|
||||
- **Git status in the bottom bar** — off by default (**App Settings → Header & Panels → Bottom bar → Git status**, per device). A small indicator at the right of the bottom bar shows the active session's repository at a glance: `● 3` uncommitted files, `↑ 2` commits not pushed, `⚠` merge conflicts, `✓` all committed and pushed. Click it for a draggable window listing the staged, not-staged, untracked and conflicted files (grouped under collapsed folders, or as a flat list if you turn that setting off) and the unpushed commits; **click a file to see its diff** (new files as all additions, deleted files as all removals), with **Open file** to jump to the viewer. A folder that holds several projects gets one collapsible section per repository found up to two levels down, all collapsed until you open them. **Fetch** in that window's header runs `git fetch` for the repositories it lists, on your click or on an interval you turn on (**App Settings → Bottom bar**, off by default; **Session Options → Git** overrides both the switch and the interval per session). Sessions open on the same folder share one run and each sees its result. Apart from that fetch Codeman never fetches or changes the repo, so "behind" is as of the last fetch; not shown for Docker or remote sessions.
|
||||
- **Clone a GitHub repo as a case** — paste a repository URL into **Add Case → Clone Repo** and Codeman clones it into `~/codeman-cases/<name>` and registers it as a normal case, ready to run an agent in. It preflights the URL while you type (tells you whether it can be cloned anonymously and offers the repo's real branches and tags for the optional branch/tag field), fills the case name in from the URL, and lets you pick which CLI the Run button should use. Public repositories over `https://`; Codeman never collects or stores credentials
|
||||
- **Create a case in a custom folder** — tick **Create in a custom folder** in **Add Case → Create New**, pick a parent folder (Browse included) and a name, and Codeman scaffolds the new case there instead of `~/codeman-cases`. The target must be a new or empty folder; system directories, your home folder itself, credential trees such as `~/.ssh`, and Codeman's own data folder are refused. Admin only in multi-user mode.
|
||||
- **Multi-CLI** — run **Claude Code**, **OpenCode**, **Codex**, **Antigravity**, **Gemini**, **Pi**, **Grok**, **DeepSeek Harness**, **OMP**, or **GitHub Copilot CLI** per session; env-var prefixes auto-gate (`CLAUDE_CODE_*` vs `OPENCODE_*` vs `CODEX_*` vs `ANTIGRAVITY_*` vs `GEMINI_*`/`GOOGLE_*` vs `PI_*` vs `GROK_*`/`XAI_*` vs `DSH_*`/`DEEPSEEK_*` vs `OMP_*` vs `COPILOT_*`). See [`docs/opencode-integration.md`](docs/opencode-integration.md), [`docs/pi-integration.md`](docs/pi-integration.md), [`docs/grok-integration.md`](docs/grok-integration.md), [`docs/deepseek-integration.md`](docs/deepseek-integration.md) [`docs/omp-integration.md`](docs/omp-integration.md) and [`docs/copilot-integration.md`](docs/copilot-integration.md)
|
||||
|
||||
@@ -1175,11 +1175,11 @@ Both routes accept two optional query parameters, which the UI sends from its pe
|
||||
|
||||
Lists are capped (300 files and 50 commits per repository) while the counts stay exact. A branch with no upstream reports the commits no remote has (`HEAD --not --remotes`); a repository with no remote reports `unpushedCount: 0`, since there is nothing to push to. Concurrent polls of one folder share a single git invocation; `?fresh=1` (what the panel's Refresh button and opening the panel send) skips the short-lived caches, though it still joins a computation already running.
|
||||
|
||||
`POST /api/sessions/:id/git-fetch` is the panel's **Fetch** button: one `git fetch --quiet` per repository the panel would show, through the same resolver so the two cannot disagree about which repositories belong to a workspace, with the same `maxRepos` and `timeout` clamps. It is the only git command Codeman runs that writes, and it writes remote-tracking refs only — nothing commits, checks out, merges or touches the working tree, so an agent working in the same repository is not disturbed. It runs on that click, or on the interval below and never otherwise, and `git fetch` is exactly the command that runs a repository's own `fetch-post` hook, which is why nothing runs on a timer until you opt in, and why a repository at or inside a Docker case workspace is refused before any git runs. Remote and Docker sessions answer `state: 'unsupported'` like the status route. Bounded: 4 repositories at a time, one spawn per repository with the same per-command timeout, `GIT_TERMINAL_PROMPT=0` plus BatchMode ssh so a repository that needs a login fails fast instead of hanging the request on an invisible prompt — Codeman supplies no credentials, so a repository your own git is already signed in to works because that is your existing setup.
|
||||
`POST /api/sessions/:id/git-fetch` is the panel's **Fetch** button: one `git fetch --quiet` per repository the panel would show, through the same resolver so the two cannot disagree about which repositories belong to a workspace, with the same `maxRepos` and `timeout` clamps. It is the only git command Codeman runs that writes, and it writes remote-tracking refs only — nothing commits, checks out, merges or touches the working tree. It is not quite as harmless as a read: a fetch holds git's ref lock, so an agent's own `git pull` in the same clone can hit the same `cannot lock ref` while a fetch is running, which is why one folder is fetched by one run at a time. It runs on that click, or on the interval below and never otherwise, and `git fetch` is exactly the command that can run a repository's own hooks (`reference-transaction`, `pre-auto-gc` through the automatic maintenance a fetch triggers, and config commands such as `credential.helper`), which is why nothing runs on a timer until you opt in, and why a repository at or inside a Docker case workspace is refused before any git runs. Remote and Docker sessions answer `state: 'unsupported'` like the status route. Bounded: 4 repositories at a time across the whole run (one shared bound, not one per repository group — a scheduled tick keeps that one bound for all its folders, so 4 folders do not mean 4 × 4, and works at most 4 folders at a time), one spawn per repository with the same per-command timeout, `GIT_TERMINAL_PROMPT=0` plus BatchMode ssh so a repository that needs a login fails fast instead of hanging the request on an invisible prompt — Codeman supplies no credentials, so a repository your own git is already signed in to works because that is your existing setup. A click joins a run already going for the same folder, and the click publishes what it got as the session's `gitFetch` state with `trigger: 'click'` (below), so another tab open on the same folder sees the same run.
|
||||
|
||||
`data` is `{ state, repos, reposTruncated, repoLimit, fetchedAt }`, `repos[]` each `{ name, path, ok }` and, when it failed, `failure` (`{ code, message, stderr }`: the codes as in the clone failures, plus `NO_REMOTE`). A repository that could not be fetched keeps its row rather than being dropped. The panel shows the result under its list and then re-reads, so `behind` reflects what this fetch brought.
|
||||
|
||||
The interval is two **synced** settings in `PUT /api/settings` (`src/git-fetch-scheduler.ts`): `gitFetchEnabled` (default `false`) and `gitFetchIntervalSeconds` (30 to 3600, default 300, the gap between runs timed from the start of a run so a slow repository does not push the next one out). Synced, not per-device, because the server is what acts on them; the scheduler reads them fresh on each 15 s tick, so a change works without a restart. A case can override both, and the override wins even while the app-wide switch is off: `POST /api/sessions/:id/git-fetch-settings` takes `{ "enabled": true | false | null, "intervalSeconds": number | null }` — `null` on a field clears that half so the app value applies again, a field left out of the body leaves the other half as it was — is `400 INVALID_INPUT` for a remote or Docker session, and answers `200` with `{ gitFetchSettings: { enabled, intervalSeconds } }` where `null` means the case follows the app. What the last run did rides the session state as `gitFetch` (`{ ranAt, trigger, state, repos, failed, firstFailure? }`, `trigger` is `click` or `interval`), on the existing `SessionUpdated` event: no new event. Remote and Docker cases are never fetched on a timer, and the Docker rule is re-checked against the CURRENT Docker roots every tick.
|
||||
The interval is two **synced** settings in `PUT /api/settings` (`src/git-fetch-scheduler.ts`): `gitFetchEnabled` (default `false`) and `gitFetchIntervalSeconds` (30 to 3600, default 300, the gap between runs timed from the start of a run so a slow repository does not push the next one out). Synced, not per-device, because the server is what acts on them; the scheduler reads them fresh on each 15 s tick, so a change works without a restart. A session can override both, and the override wins even while the app-wide switch is off: `POST /api/sessions/:id/git-fetch-settings` takes `{ "enabled": true | false | null, "intervalSeconds": number | null }` — `null` on a field clears that half so the app value applies again, a field left out of the body leaves the other half as it was — is `400 INVALID_INPUT` for a remote or Docker session, and answers `200` with `{ gitFetchSettings: { enabled, intervalSeconds } }` where `null` means the session follows the app. What the last run did rides the session state as `gitFetch` (`{ ranAt, trigger, state, repos, failed, firstFailure? }`, `trigger` is `click` or `interval`), on the existing `SessionUpdated` event: no new event. Sessions on the same folder share one run and both see its result, the schedule is keyed by folder, and a folder is fetched by at most one run at a time; every session open on a folder is remembered, so closing the one that holds the override does not stop that folder's fetching while another session is still open on it. The tick skips anything its bounded probe does not confirm is a directory, stamps each folder's schedule when it picks it, and does not run a second tick over a selection still going. Remote and Docker sessions are never fetched on a timer, and the Docker rule is re-checked against the CURRENT Docker roots every tick.
|
||||
|
||||
## Git worktrees of a case
|
||||
|
||||
|
||||
@@ -93,11 +93,13 @@ Git window lists, so the `behind` number is current without a click, and **Git f
|
||||
the start of a run, so a slow repository does not push the next one out. These are the only Git settings
|
||||
that are synced rather than per device, because the server's scheduler is what acts on them: kept per
|
||||
device, the scheduler would read nobody's number. **Session Options → Git** overrides both the switch and
|
||||
the interval for one case, and that override wins even while the app switch is off; a case that follows the
|
||||
app shows the app's values and its rows go inert until you turn **Follow the app setting** off. Nothing is
|
||||
ever fetched for a Docker or remote (SSH) case, and a repository a container can write to is refused
|
||||
before any git runs. Both default to off because `git fetch` is the one git command Codeman runs that
|
||||
writes, and the one that runs a repository's own hooks.
|
||||
the interval for one session, and that override wins even while the app switch is off; a session that follows
|
||||
the app shows the app's values and its rows go inert until you turn **Follow the app setting** off. Nothing is
|
||||
ever fetched for a Docker or remote (SSH) session, and a repository a container can write to is refused
|
||||
before any git runs. Sessions open on the same folder share one run. Both default to off because `git fetch`
|
||||
is the one git command Codeman runs that writes, and the one that can run a repository's own hooks
|
||||
(`reference-transaction`, `pre-auto-gc` through the maintenance a fetch triggers, and config commands such as
|
||||
`credential.helper`).
|
||||
|
||||
Most default to off. The stock desktop header is system stats, File Viewer, Tiles, and the gear.
|
||||
**Header Stats Style** picks how the system stats and plan usage are drawn: *Compact*
|
||||
|
||||
@@ -248,14 +248,18 @@ Click it for a draggable window, in the style of the File Viewer:
|
||||
- **Fetch** in the header runs `git fetch` for the repositories the window lists, on your click; the
|
||||
window then re-reads, so "behind" is as of that fetch. A repository that could not be fetched keeps
|
||||
its row and shows why. **Git fetch on an interval** (Settings → Bottom bar, off by default) runs the
|
||||
same fetch on a timer for every case that is on, and **Session Options → Git** overrides both the
|
||||
switch and the interval for one case — a case that turned itself on runs even while the app-wide
|
||||
switch is off, and nothing is ever fetched for Docker or remote (SSH) cases.
|
||||
same fetch on a timer for every session that is on, and **Session Options → Git** overrides both the
|
||||
switch and the interval for one session — a session that turned itself on runs even while the app-wide
|
||||
switch is off, sessions open on the same folder share one run, and nothing is ever fetched for Docker or
|
||||
remote (SSH) sessions.
|
||||
|
||||
It is read-only apart from that fetch: Codeman never commits, checks out or changes the
|
||||
repository, and `git fetch` writes remote-tracking refs only — it is the one git command here that
|
||||
writes, and the one that runs a repository's own hooks, which is why nothing runs on a timer until you
|
||||
opt in. It is not shown for Docker or remote (SSH) sessions, and a repository at or inside a Docker case
|
||||
writes, and the one that can run a repository's own hooks (`reference-transaction`, `pre-auto-gc` through the
|
||||
maintenance a fetch triggers, and config commands such as `credential.helper`), which is why nothing runs on
|
||||
a timer until you opt in. A fetch holds git's ref lock, so an agent's own `git pull` in the same repository
|
||||
can hit the same `cannot lock ref`, and a folder is fetched by one run at a time. It is not shown for Docker
|
||||
or remote (SSH) sessions, and a repository at or inside a Docker case
|
||||
workspace is skipped even from a local session, before any git runs (a container can write there,
|
||||
and git would run that repository's own configuration on the host). Fetch asks nothing of you: it
|
||||
uses the sign-in your own git already has, and it gives up quickly rather than waiting on an
|
||||
|
||||
+134
-62
@@ -4,34 +4,50 @@
|
||||
*
|
||||
* The fetch module (src/git-fetch.ts) is the only place Codeman runs a git command that writes, and on a
|
||||
* timer it is the only one that can run without being asked — which is why the whole feature is off until
|
||||
* the user turns it on, in Settings → Bottom bar, per case in Session Options → Git.
|
||||
* the user turns it on, in Settings → Bottom bar, per session in Session Options → Git.
|
||||
*
|
||||
* The scheduler holds no timers of its own. The server ticks it (`GIT_FETCH_TICK_INTERVAL_MS`) and every
|
||||
* tick reads the app setting FRESH, so a live toggle takes effect on the next tick instead of at boot, and
|
||||
* turning the feature off actually stops the fetching. A case can override both the switch and the
|
||||
* interval; a case with no override follows the app.
|
||||
* turning the feature off actually stops the fetching. A session can override both the switch and the
|
||||
* interval; a session with no override follows the app.
|
||||
*
|
||||
* Rules the code keeps and the tests pin:
|
||||
* - Nothing runs when nothing is on: no repository set is read and nothing is spawned, so a quiet
|
||||
* install costs one settings read per tick. A case that turned itself on still runs even while the
|
||||
* app-wide switch is off — that is what a per-case switch has to mean.
|
||||
* - Remote and Docker cases are never fetched, exactly as the Git panel excludes them, and the Docker
|
||||
* workspace roots are checked again every tick, so a case that became a Docker case stops fetching at
|
||||
* the next tick.
|
||||
* - One run per case at a time: a tick skips a case still being fetched rather than stacking runs on it.
|
||||
* - At most `FETCH_CONCURRENCY` cases at a time across all of them, because each is a git process.
|
||||
* install costs one settings read per tick. A session that turned itself on still runs even while the
|
||||
* app-wide switch is off — that is what a per-session switch has to mean.
|
||||
* - Remote and Docker sessions are never fetched, exactly as the Git panel excludes them, and the Docker
|
||||
* workspace roots are checked again every tick, so a session whose case became a Docker case stops
|
||||
* fetching at the next tick.
|
||||
* - A folder is fetched by at most one run at a time, and the run belongs to the FOLDER, not to one
|
||||
* session: two sessions open on the same folder are the same clone, so they share one run and both get
|
||||
* its result. State is keyed by `resolve(workingDir)` — deliberately not `realpath`, because `realpath`
|
||||
* is itself one of the unbounded fs calls rule #516 bounds; the bounded probe below is the fs read.
|
||||
* Every session open on a folder is remembered, so closing the one holding the override does not stop
|
||||
* the folder's fetching while another session is still on it.
|
||||
* - Before anything is fetched, the folder is probed BOUNDED with the default options (rule #516): a case
|
||||
* on a mount that stopped answering is skipped, never touched, and a probe that answers `unknown` is not
|
||||
* read as "not a folder".
|
||||
* - A schedule is stamped at SELECTION, when a session goes into `due`, not when its run starts, so a
|
||||
* tick that is still running cannot pick the same folder again; and a tick is skipped while the previous
|
||||
* one is still going, so the bound below is the bound for the whole tick, not per tick.
|
||||
* - At most `FETCH_CONCURRENCY` git processes at a time across the whole tick, because each is a git
|
||||
* process: ONE bound is shared by every run of the tick, so `FETCH_CONCURRENCY` is the tick's ceiling
|
||||
* rather than each folder's own. The folders get their OWN bound (`mapLimited`), because one semaphore
|
||||
* cannot serve both levels: a folder that holds a slot and then waits for one for its own fetch is
|
||||
* waiting on itself, and the folders that fill the bound are exactly the ones blocking it.
|
||||
* - The interval is timed from the START of a run, so a run that takes three minutes does not push the
|
||||
* next one out by three minutes; it lands on the tick after its own interval elapses.
|
||||
* - The per-repository limit and the git timeout stay the module's defaults (`MAX_REPOS`,
|
||||
* `DEFAULT_GIT_TIMEOUT_MS`): both settings are per device and never reach the server, so the server
|
||||
* cannot know what a device chose.
|
||||
* - A failure is reported, not retried harder: the case keeps its reason and gets its next run at its
|
||||
* - A failure is reported, not retried harder: the folder keeps its schedule and gets its next run at its
|
||||
* own interval. A settings read that fails skips this tick only, because settings.json is rewritten in
|
||||
* place and a kept fallback would hold the feature off until the next settings change.
|
||||
*
|
||||
* @module git-fetch-scheduler
|
||||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { getErrorMessage } from './types.js';
|
||||
import {
|
||||
FETCH_CONCURRENCY,
|
||||
@@ -43,9 +59,10 @@ import {
|
||||
type GitFetchSettings,
|
||||
type GitFetchStatus,
|
||||
} from './git-fetch.js';
|
||||
import { clampInt, DEFAULT_GIT_TIMEOUT_MS, MAX_REPOS, mapLimited } from './git-workspace-status.js';
|
||||
import { clampInt, DEFAULT_GIT_TIMEOUT_MS, MAX_REPOS, createLimiter, mapLimited } from './git-workspace-status.js';
|
||||
import type { PathProbeKind } from './utils/bounded-path-probe.js';
|
||||
|
||||
/** What the scheduler needs from a case: the facts a decision needs, and nothing else. */
|
||||
/** What the scheduler needs from a session: the facts a decision needs, and nothing else. */
|
||||
export interface GitFetchSessionView {
|
||||
id: string;
|
||||
workingDir: string;
|
||||
@@ -62,13 +79,20 @@ export interface GitFetchPorts {
|
||||
/** Read FRESH every tick; that is the whole point of the tick-driven shape. */
|
||||
readSettings(): Promise<GitFetchSettings>;
|
||||
dockerWorkspaces(): Promise<string[]>;
|
||||
/**
|
||||
* Bounded probe (rule #516), default options: a bulk scan keeps the stall cap, so this must not be called
|
||||
* with `pastCap`. Anything that is not `directory` — including `unknown`, which is never `absent` — is
|
||||
* skipped, not fetched.
|
||||
*/
|
||||
probePath(path: string): Promise<PathProbeKind>;
|
||||
/** Called through `fetchWorkspaceOnce`, so a run already going for this folder is joined, not stacked. */
|
||||
fetch(cwd: string, opts: GitFetchOptions): Promise<GitFetchReport>;
|
||||
publish(id: string, status: GitFetchStatus): void;
|
||||
log?(message: string): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* What one case will actually do: its own override wins, otherwise the app-wide setting. The interval is
|
||||
* What one session will actually do: its own override wins, otherwise the app-wide setting. The interval is
|
||||
* clamped again here because a persisted value can predate the schema's bounds.
|
||||
*/
|
||||
export function effectiveGitFetch(
|
||||
@@ -88,78 +112,126 @@ export function effectiveGitFetch(
|
||||
|
||||
export class GitFetchScheduler {
|
||||
private readonly ports: GitFetchPorts;
|
||||
/** When each case last started a run; timed from the start, not the end. */
|
||||
/** When each FOLDER last started a run; timed from the start, not the end. */
|
||||
private readonly lastRunAt = new Map<string, number>();
|
||||
private readonly inFlight = new Set<string>();
|
||||
/** Session → folder, so `remove()` can keep the folder's schedule while other sessions are still on it. */
|
||||
private readonly folderOf = new Map<string, string>();
|
||||
private running = false;
|
||||
private stopped = false;
|
||||
|
||||
constructor(ports: GitFetchPorts) {
|
||||
this.ports = ports;
|
||||
}
|
||||
|
||||
/** One tick: fetch every case that is on and due. Returns how many ran. */
|
||||
/** One tick: fetch every folder whose sessions are on and due. Returns how many folders ran. */
|
||||
async tick(now = Date.now()): Promise<number> {
|
||||
if (this.stopped) return 0;
|
||||
|
||||
let settings: GitFetchSettings;
|
||||
if (this.stopped || this.running) return 0;
|
||||
this.running = true;
|
||||
try {
|
||||
settings = await this.ports.readSettings();
|
||||
} catch (err) {
|
||||
// A read can land mid-write. Skipping this tick costs one interval; deciding from a fallback would
|
||||
// hold the feature off until the next settings change.
|
||||
this.ports.log?.(`git fetch: settings unreadable, tick skipped: ${getErrorMessage(err)}`);
|
||||
return 0;
|
||||
}
|
||||
|
||||
const due: GitFetchSessionView[] = [];
|
||||
for (const id of this.ports.sessionIds()) {
|
||||
const view = this.ports.getSession(id);
|
||||
if (!view || view.remote || view.docker) continue;
|
||||
const effective = effectiveGitFetch(view, settings);
|
||||
if (!effective.enabled || this.inFlight.has(id)) continue;
|
||||
const started = this.lastRunAt.get(id);
|
||||
if (started !== undefined && now - started < effective.intervalSeconds * 1000) continue;
|
||||
due.push(view);
|
||||
}
|
||||
if (!due.length) return 0;
|
||||
|
||||
// Only now, because nothing above needs it: a quiet tick does not read the Docker case list.
|
||||
const dockerWorkspaces = await this.ports.dockerWorkspaces();
|
||||
await mapLimited(due, FETCH_CONCURRENCY, async (view) => {
|
||||
this.inFlight.add(view.id);
|
||||
// The clock starts at the run, so a slow repository does not shift the case's own schedule.
|
||||
this.lastRunAt.set(view.id, now);
|
||||
let settings: GitFetchSettings;
|
||||
try {
|
||||
const report = await this.ports.fetch(view.workingDir, {
|
||||
dockerWorkspaces,
|
||||
maxRepos: MAX_REPOS,
|
||||
timeoutMs: DEFAULT_GIT_TIMEOUT_MS,
|
||||
});
|
||||
this.ports.publish(view.id, gitFetchStatusFrom(report, 'interval'));
|
||||
settings = await this.ports.readSettings();
|
||||
} catch (err) {
|
||||
// The run that failed keeps its reason in the panel's own row; here only the case-level failure is
|
||||
// worth a line, because nothing else would say a scheduled fetch could not run at all.
|
||||
this.ports.log?.(`git fetch: ${view.id} could not run: ${getErrorMessage(err)}`);
|
||||
} finally {
|
||||
this.inFlight.delete(view.id);
|
||||
// A read can land mid-write. Skipping this tick costs one interval; deciding from a fallback would
|
||||
// hold the feature off until the next settings change.
|
||||
this.ports.log?.(`git fetch: settings unreadable, tick skipped: ${getErrorMessage(err)}`);
|
||||
return 0;
|
||||
}
|
||||
});
|
||||
return due.length;
|
||||
|
||||
// One entry per folder, holding every session open on it. Stamped here, at selection: a tick that is
|
||||
// still running must not pick the same folder again, and a folder already being fetched is not picked
|
||||
// at all.
|
||||
const due = new Map<string, GitFetchSessionView[]>();
|
||||
for (const id of this.ports.sessionIds()) {
|
||||
const view = this.ports.getSession(id);
|
||||
if (!view || view.remote || view.docker) continue;
|
||||
const effective = effectiveGitFetch(view, settings);
|
||||
if (!effective.enabled) continue;
|
||||
const folder = resolve(view.workingDir);
|
||||
// Recorded for EVERY session on the folder, not just the one that got picked, so `remove()` can still
|
||||
// see who is open on it when the session holding the override is the one that closes.
|
||||
this.folderOf.set(id, folder);
|
||||
// A session on a folder this tick already picked is the same clone: it joins that folder's run rather
|
||||
// than starting its own. Two fetches in one clone collide on git's ref lock, so one run has to serve
|
||||
// every session open on the folder.
|
||||
const group = due.get(folder);
|
||||
if (group) {
|
||||
group.push(view);
|
||||
continue;
|
||||
}
|
||||
if (this.inFlight.has(folder)) continue;
|
||||
const started = this.lastRunAt.get(folder);
|
||||
if (started !== undefined && now - started < effective.intervalSeconds * 1000) continue;
|
||||
const kind = await this.ports.probePath(view.workingDir);
|
||||
if (kind !== 'directory') {
|
||||
this.ports.log?.(`git fetch: ${view.id} skipped, folder answered ${kind}`);
|
||||
continue;
|
||||
}
|
||||
this.inFlight.add(folder);
|
||||
// The clock starts at the selection, so a slow folder does not shift its own schedule.
|
||||
this.lastRunAt.set(folder, now);
|
||||
due.set(folder, [view]);
|
||||
}
|
||||
if (!due.size) return 0;
|
||||
|
||||
// Only now, because nothing above needs it: a quiet tick does not read the Docker case list.
|
||||
const dockerWorkspaces = await this.ports.dockerWorkspaces();
|
||||
// ONE bound for the git processes of the whole tick, handed to every run so no folder gets its own:
|
||||
// each fetch is one git process, and a folder can hold up to `MAX_REPOS_LIMIT` of them, so a bound per
|
||||
// run would let `FETCH_CONCURRENCY` folders × `MAX_REPOS_LIMIT` processes run at once.
|
||||
const fetches = createLimiter(FETCH_CONCURRENCY);
|
||||
// The folders get their OWN bound, because the same semaphore cannot serve both levels: a folder that
|
||||
// holds a slot and then waits for a slot for its own fetch is waiting on itself — once
|
||||
// `FETCH_CONCURRENCY` folders fill the bound, the ones holding slots are exactly the ones that can
|
||||
// never get one back. Both bounds are `FETCH_CONCURRENCY`, so the tick still never runs more git
|
||||
// processes than the module's ceiling.
|
||||
await mapLimited([...due.entries()], FETCH_CONCURRENCY, async ([folder, group]) => {
|
||||
try {
|
||||
const report = await this.ports.fetch(folder, {
|
||||
dockerWorkspaces,
|
||||
maxRepos: MAX_REPOS,
|
||||
timeoutMs: DEFAULT_GIT_TIMEOUT_MS,
|
||||
limiter: fetches,
|
||||
});
|
||||
for (const view of group) this.ports.publish(view.id, gitFetchStatusFrom(report, 'interval'));
|
||||
} catch (err) {
|
||||
// The run that failed keeps its reason in the panel's own row; here only the folder-level failure is
|
||||
// worth a line, because nothing else would say a scheduled fetch could not run at all.
|
||||
this.ports.log?.(`git fetch: ${folder} could not run: ${getErrorMessage(err)}`);
|
||||
} finally {
|
||||
this.inFlight.delete(folder);
|
||||
}
|
||||
});
|
||||
return due.size;
|
||||
} finally {
|
||||
this.running = false;
|
||||
}
|
||||
}
|
||||
|
||||
/** A case that closed or was detached: its schedule and its in-flight flag go with it. */
|
||||
/**
|
||||
* A session that closed or was detached: its schedule goes with it, but the FOLDER's stays while another
|
||||
* session is still open on it — the run belongs to the clone, not to the session that asked for it.
|
||||
*/
|
||||
remove(id: string): void {
|
||||
this.lastRunAt.delete(id);
|
||||
this.inFlight.delete(id);
|
||||
const folder = this.folderOf.get(id);
|
||||
this.folderOf.delete(id);
|
||||
if (!folder) return;
|
||||
const stillOpen = [...this.folderOf.values()].includes(folder);
|
||||
if (!stillOpen) {
|
||||
this.lastRunAt.delete(folder);
|
||||
this.inFlight.delete(folder);
|
||||
}
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
this.stopped = true;
|
||||
this.lastRunAt.clear();
|
||||
this.inFlight.clear();
|
||||
this.folderOf.clear();
|
||||
}
|
||||
|
||||
/** Cases the scheduler still remembers a run for, so a test can see a `remove()` actually took. */
|
||||
/** Folders the scheduler still remembers a run for, so a test can see a `remove()` actually took. */
|
||||
tracked(): number {
|
||||
return new Set([...this.lastRunAt.keys(), ...this.inFlight]).size;
|
||||
}
|
||||
|
||||
+76
-10
@@ -7,7 +7,10 @@
|
||||
* fetch, so a number that is stale can only be made current by fetching. This is the one place Codeman
|
||||
* runs a git command that writes, and on a timer it is also the only one that can run without being asked.
|
||||
* Nothing here commits, checks out, merges or touches the working tree: `git fetch` updates remote-tracking
|
||||
* refs and nothing else, so an agent working in the same repository is not disturbed.
|
||||
* refs and nothing else, and `FETCH_CONFIG_ARGS` keeps it from starting a gc in a repository an agent is
|
||||
* already using. It is not harmless in the way a read is: a fetch holds git's ref lock, so an agent's own
|
||||
* `git pull` in the same clone can collide with it (`cannot lock ref`), which is why one folder is fetched
|
||||
* by at most one run at a time (`fetchWorkspaceOnce`).
|
||||
*
|
||||
* Same split as git-clone.ts: the pure half (`buildFetchArgs`, `describeFetchFailure`,
|
||||
* `gitFetchSettingsFrom`, `gitFetchStatusFrom`) holds every decision and tests without spawning, and the IO
|
||||
@@ -17,10 +20,15 @@
|
||||
* - The repository set is `resolveWorkspaceRepos`, the single source the panel and the diff route
|
||||
* already use, so Fetch cannot reach a repository the panel would not show. The Docker rule is
|
||||
* checked before any git runs: a repository a container can write to is refused outright, because a
|
||||
* planted config would run on the host — and `git fetch` is exactly the command that runs a
|
||||
* repository's own hooks (`fetch-post`), which is why a timer only runs it after the user opts in.
|
||||
* - Bounded: the git timeout from Settings → Bottom bar per repository, `FETCH_CONCURRENCY` at a time,
|
||||
* `MAX_REPOS_LIMIT` repositories at most, and a hard `timeout` on each spawn.
|
||||
* planted config would run on the host — and `git fetch` is exactly the command that can run a
|
||||
* repository's own hooks (`reference-transaction`, and `pre-auto-gc` through the automatic maintenance
|
||||
* that follows a fetch) and its config commands such as `credential.helper`, which is why a timer only
|
||||
* runs it after the user opts in.
|
||||
* - One run per folder at a time: `fetchWorkspaceOnce` joins a run already going for the same folder
|
||||
* instead of starting a second one, because two fetches in one clone collide on the ref lock.
|
||||
* - Bounded: the git timeout from Settings → Bottom bar per repository, `MAX_REPOS_LIMIT` repositories at
|
||||
* most, a hard `timeout` on each spawn, and `FETCH_CONCURRENCY` git processes at a time for the whole
|
||||
* run — one `Limiter`, shared by the scheduler's tick, not one bound per folder.
|
||||
* - Non-interactive: `GIT_TERMINAL_PROMPT=0` and BatchMode ssh, so a private repository that needs a
|
||||
* login fails fast instead of hanging the request on an invisible username prompt. Codeman collects
|
||||
* no credentials; a repository the user's own git is already signed in to works because that is the
|
||||
@@ -32,12 +40,13 @@
|
||||
*/
|
||||
|
||||
import { execFile } from 'node:child_process';
|
||||
import { basename, relative } from 'node:path';
|
||||
import { basename, relative, resolve } from 'node:path';
|
||||
import { promisify } from 'node:util';
|
||||
import { classifyGitFailure, gitNonInteractiveEnv, sanitizeGitOutput, type GitFailureCode } from './git-clone.js';
|
||||
import {
|
||||
createLimiter,
|
||||
DEFAULT_GIT_TIMEOUT_MS,
|
||||
mapLimited,
|
||||
type Limiter,
|
||||
resolveOverviewLimits,
|
||||
resolveWorkspaceRepos,
|
||||
type GitRunner,
|
||||
@@ -45,11 +54,25 @@ import {
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
/** Fetches in flight at once: each is one git process, and a folder can hold up to `MAX_REPOS_LIMIT`. */
|
||||
/**
|
||||
* Git processes in flight at once, for one run — and for a whole scheduler tick, because the tick hands ONE
|
||||
* bound to every run (see `createLimiter`): each fetch is one git process, and a folder can hold up to
|
||||
* `MAX_REPOS_LIMIT`. The tick bounds its folders separately, because the same semaphore cannot serve both
|
||||
* levels — a folder holding a slot would then wait for a slot its own fetch needs.
|
||||
*/
|
||||
export const FETCH_CONCURRENCY = 4;
|
||||
/** A fetch on a busy remote can print a lot; a bound on what we will hold. */
|
||||
const MAX_FETCH_OUTPUT_BYTES = 2 * 1024 * 1024;
|
||||
|
||||
/**
|
||||
* Config overrides set ahead of the argv, so an unattended fetch cannot do more than move refs: no
|
||||
* fsmonitor hook, and no automatic maintenance or gc. A fetch is the git command that can run a
|
||||
* repository's own hooks (`reference-transaction`, and `pre-auto-gc` through the maintenance that follows a
|
||||
* fetch), and a gc started by a timer in a repository an agent is working in is exactly the kind of extra
|
||||
* write this module must not do. Kept as a const so a test can pin it without spawning git.
|
||||
*/
|
||||
export const FETCH_CONFIG_ARGS = ['-c', 'core.fsmonitor=false', '-c', 'maintenance.auto=false', '-c', 'gc.auto=0'];
|
||||
|
||||
/** The interval the user can turn on (Settings → Bottom bar, and Session Options → Git): bounds and default. */
|
||||
export const DEFAULT_GIT_FETCH_INTERVAL_SECONDS = 300;
|
||||
export const MIN_GIT_FETCH_INTERVAL_SECONDS = 30;
|
||||
@@ -85,6 +108,10 @@ export function buildFetchArgs(): string[] {
|
||||
}
|
||||
|
||||
/**
|
||||
* The `git` process also gets the `FETCH_CONFIG_ARGS` overrides ahead of this argv; they are kept separate
|
||||
* so pinning the verb here does not have to say anything about maintenance, and so both can be tested
|
||||
* without spawning.
|
||||
*
|
||||
* A repository with no remote is a fact, not a fault. Git's wordings for it differ by version — 2.47
|
||||
* exits quietly and this code is never reached, older git says "no default remote repository" — so the
|
||||
* branch is pinned on literal fixtures and the code the panel shows is the code this writes.
|
||||
@@ -126,7 +153,7 @@ export function describeFetchFailure(err: unknown): FetchFailure {
|
||||
|
||||
/** Fetch is allowed to write refs, so this runner does not carry `--no-optional-locks`: fetch takes a required lock anyway. */
|
||||
export const runFetch: GitRunner = async (cwd, args, opts) => {
|
||||
const { stdout } = await execFileAsync('git', ['-c', 'core.fsmonitor=false', ...args], {
|
||||
const { stdout } = await execFileAsync('git', [...FETCH_CONFIG_ARGS, ...args], {
|
||||
cwd,
|
||||
timeout: opts?.timeoutMs ?? DEFAULT_GIT_TIMEOUT_MS,
|
||||
maxBuffer: MAX_FETCH_OUTPUT_BYTES,
|
||||
@@ -198,6 +225,12 @@ export interface GitFetchOptions {
|
||||
dockerWorkspaces?: string[];
|
||||
maxRepos?: number;
|
||||
timeoutMs?: number;
|
||||
/**
|
||||
* The shared slot count for a whole scheduler tick. Passed in so `FETCH_CONCURRENCY` bounds the tick
|
||||
* rather than each folder separately; without it, a tick over more than `FETCH_CONCURRENCY` folders could
|
||||
* have that many git processes going at once inside each one.
|
||||
*/
|
||||
limiter?: Limiter;
|
||||
}
|
||||
|
||||
export const emptyReport = (state: GitFetchReport['state'], extra: Partial<GitFetchReport> = {}): GitFetchReport => ({
|
||||
@@ -230,7 +263,8 @@ export async function fetchGitWorkspaceOverview(cwd: string, opts: GitFetchOptio
|
||||
if (!dirs.length) return emptyReport('not-a-repo');
|
||||
const { timeoutMs } = resolveOverviewLimits(opts);
|
||||
const runner = opts.git ?? runFetch;
|
||||
const failures = await mapLimited(dirs, FETCH_CONCURRENCY, (dir) => fetchOne(dir, runner, timeoutMs));
|
||||
const run = opts.limiter ?? createLimiter(FETCH_CONCURRENCY);
|
||||
const failures = await run(dirs, (dir) => fetchOne(dir, runner, timeoutMs));
|
||||
const repos: GitFetchEntry[] = dirs.map((dir, i) => {
|
||||
const base = { name: basename(dir), path: relative(cwd, dir) || '.' };
|
||||
const failure = failures[i];
|
||||
@@ -244,3 +278,35 @@ export async function fetchGitWorkspaceOverview(cwd: string, opts: GitFetchOptio
|
||||
fetchedAt: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs already going, keyed by the resolved folder. A request for a folder another run is already working on
|
||||
* JOINS that run instead of starting a second `git fetch` in the same clone: two fetches there collide on
|
||||
* git's ref lock (`cannot lock ref 'refs/remotes/origin/main'`), which is what a click during a tick would
|
||||
* otherwise do. A joiner gets the run already going's answer, including its limits — the point is not to
|
||||
* fetch twice, not to fetch with the freshest options.
|
||||
*
|
||||
* Keyed by `resolve`, not `realpath`, because `realpath` is itself one of the unbounded fs calls rule #516
|
||||
* bounds for user-chosen paths; the scheduler probes the folder bounded and calls this. Two paths that
|
||||
* resolve the same are the same folder here; a workspace reached through a symlink resolves to the link, so
|
||||
* a link and its target can be fetched by two runs — the accepted trade for keeping the probe bounded.
|
||||
*/
|
||||
const lane = new Map<string, Promise<GitFetchReport>>();
|
||||
|
||||
export async function fetchWorkspaceOnce(folder: string, opts: GitFetchOptions = {}): Promise<GitFetchReport> {
|
||||
const key = resolve(folder);
|
||||
const going = lane.get(key);
|
||||
if (going) return going;
|
||||
const run = fetchGitWorkspaceOverview(key, opts).finally(() => {
|
||||
// Only the run that is still the lane's own: a later run for the same folder must not be dropped by an
|
||||
// earlier one finishing late.
|
||||
if (lane.get(key) === run) lane.delete(key);
|
||||
});
|
||||
lane.set(key, run);
|
||||
return run;
|
||||
}
|
||||
|
||||
/** Test seam: an in-memory lane is not cleared by stopping the scheduler, so a test has to clear it itself. */
|
||||
export function clearGitFetchLane(): void {
|
||||
lane.clear();
|
||||
}
|
||||
@@ -597,7 +597,54 @@ export async function discoverChildRepos(
|
||||
|
||||
const discoveryCache = new Map<string, { at: number; value: { dirs: string[]; truncated: boolean } }>();
|
||||
|
||||
/** Run `fn` over `items` with at most `limit` in flight, keeping the input order. Shared with the fetch path. */
|
||||
/**
|
||||
* A slot count fixed ONCE, so callers share it instead of each getting its own bound (see the fetch path):
|
||||
* the count lives in the closure, so `limit` is the ceiling for everything going through THIS limiter at
|
||||
* once, not for each call to it — which is what makes a scheduler tick's bound honest.
|
||||
* ⚠️ One limiter cannot bound two levels of the same work: a caller that holds a slot and then waits for
|
||||
* another slot for the work IT starts cannot get one, because the holders are exactly the ones waiting. The
|
||||
* outer level needs its own bound (`mapLimited`), which is why the tick shares one for fetches and keeps a
|
||||
* separate one for folders.
|
||||
*/
|
||||
export type Limiter = <T, R>(items: T[], fn: (item: T) => Promise<R>) => Promise<R[]>;
|
||||
|
||||
export function createLimiter(limit: number): Limiter {
|
||||
let inUse = 0;
|
||||
const waiting: Array<() => void> = [];
|
||||
const acquire = (): Promise<void> => {
|
||||
if (inUse < limit) {
|
||||
inUse++;
|
||||
return Promise.resolve();
|
||||
}
|
||||
return new Promise<void>((done) => {
|
||||
waiting.push(() => {
|
||||
inUse++;
|
||||
done();
|
||||
});
|
||||
});
|
||||
};
|
||||
const release = () => {
|
||||
inUse--;
|
||||
const next = waiting.shift();
|
||||
if (next) next();
|
||||
};
|
||||
return async <T, R>(items: T[], fn: (item: T) => Promise<R>): Promise<R[]> => {
|
||||
const out: R[] = new Array(items.length);
|
||||
await Promise.all(
|
||||
items.map(async (item, i) => {
|
||||
await acquire();
|
||||
try {
|
||||
out[i] = await fn(item);
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
})
|
||||
);
|
||||
return out;
|
||||
};
|
||||
}
|
||||
|
||||
/** Run `fn` over `items` with at most `limit` in flight, keeping the input order. Per call; `createLimiter` shares one bound across calls. */
|
||||
export async function mapLimited<T, R>(items: T[], limit: number, fn: (item: T) => Promise<R>): Promise<R[]> {
|
||||
const out: R[] = new Array(items.length);
|
||||
let next = 0;
|
||||
|
||||
@@ -893,14 +893,18 @@ export interface SessionState {
|
||||
/** What the keep-warm is doing right now; ephemeral, never restored. */
|
||||
keepWarm?: KeepWarmStatus;
|
||||
/**
|
||||
* Git fetch on an interval for this case's repositories; absent = follow the app-wide
|
||||
* Git fetch on an interval for this session's repositories; absent = follow the app-wide
|
||||
* `gitFetchEnabled` default. Restored on a restart, like `keepWarmEnabled`, because the choice is the
|
||||
* user's and outlives the run.
|
||||
*/
|
||||
gitFetchEnabled?: boolean;
|
||||
/** Seconds between this case's runs; absent = follow the app-wide `gitFetchIntervalSeconds` default. */
|
||||
/** Seconds between this session's runs; absent = follow the app-wide `gitFetchIntervalSeconds` default. */
|
||||
gitFetchIntervalSeconds?: number;
|
||||
/** What this case's last fetch did; ephemeral, never restored. */
|
||||
/**
|
||||
* What this session's last fetch did. Written to disk with the rest of the record, but the restore copies
|
||||
* only the fields its own config lists and this is not one of them, so it is the panel's truth for the
|
||||
* life of the session, not across a restart.
|
||||
*/
|
||||
gitFetch?: GitFetchStatus;
|
||||
/** Pinned to the top of the session manager list (COD-139) */
|
||||
pinned?: boolean;
|
||||
|
||||
+10
-10
@@ -541,22 +541,22 @@
|
||||
// Git fetch on an interval (App Settings → Bottom bar, Session Options → Git, and the save toast in
|
||||
// session-ui.js). 'Status' and 'Git' are not added: the first already has a key, the second stays a
|
||||
// technical name, and the status row's text is written at runtime.
|
||||
"Whether Codeman fetches this case's repositories on a timer.": '{name} 是否按间隔拉取此案例的仓库。',
|
||||
"Whether Codeman fetches this session's repositories on a timer.": '{name} 是否按间隔拉取此会话的仓库。',
|
||||
'Git fetch on an interval': 'Git 定时拉取',
|
||||
'Runs `git fetch` for the repositories the Git window lists on a timer, so the "behind" number is current without a click. Off by default: this is the one git command Codeman runs that writes, and the one that runs a repository\'s own hooks, so nothing runs on a timer until you turn it on. A case can override both the switch and the interval in Session Options → Git.':
|
||||
'按间隔为 Git 窗口列出的仓库运行拉取,这样“落后”的数字无需点击就是最新的。默认关闭:这是 {name} 运行的唯一会写入的 git 命令,也是唯一会运行仓库自身钩子的命令,所以在您开启之前不会有任何定时运行。案例可以在“会话选项 → Git”中同时覆盖开关和间隔。',
|
||||
'Runs `git fetch` for the repositories the Git window lists on a timer, so the "behind" number is current without a click. Off by default: this is the one git command Codeman runs that writes, and the one that runs a repository\'s own hooks, so nothing runs on a timer until you turn it on. A session can override both the switch and the interval in Session Options → Git.':
|
||||
'按间隔为 Git 窗口列出的仓库运行拉取,这样“落后”的数字无需点击就是最新的。默认关闭:这是 {name} 运行的唯一会写入的 git 命令,也是唯一会运行仓库自身钩子的命令,所以在您开启之前不会有任何定时运行。会话可以在“会话选项 → Git”中同时覆盖开关和间隔。',
|
||||
'Git fetch: how often': 'Git 定时拉取:多久一次',
|
||||
'Seconds between runs for a case that has no override of its own (30 to 3600, default 300). Only a case that is on — here, or in Session Options → Git — is fetched.':
|
||||
'没有自己覆盖设置的案例,两次运行之间的秒数(30 到 3600,默认 300)。只有已开启的案例——这里,或在“会话选项 → Git”——才会被拉取。',
|
||||
'Seconds between runs for a session that has no override of its own (30 to 3600, default 300). Only a session that is on — here, or in Session Options → Git — is fetched, and sessions sharing one folder are fetched by one run.':
|
||||
'没有自己覆盖设置的会话,两次运行之间的秒数(30 到 3600,默认 300)。只有已开启的会话——这里,或在“会话选项 → Git”——才会被拉取,共用一个文件夹的会话由一次运行拉取。',
|
||||
'Fetch on an interval': '按间隔拉取',
|
||||
"Fetching is the only git command Codeman runs that writes, and the only one that runs a repository's own hooks, so nothing runs on a timer until you turn it on — here, or in App Settings → Bottom bar. The panel's Fetch button is always just a click.":
|
||||
'拉取是 {name} 运行的唯一会写入的 git 命令,也是唯一会运行仓库自身钩子的命令,所以在您开启之前(这里,或“应用设置 → 底部栏”)不会有任何定时运行。该窗口中的拉取按钮始终只是一次点击。',
|
||||
'Follow the app setting': '跟随应用设置',
|
||||
'Uses the app-wide switch and interval for this case. Switch it off to pick your own.':
|
||||
'为此案例使用应用级的开关和间隔。关闭它即可自行选择。',
|
||||
'Fetch this case': '拉取此案例',
|
||||
'Runs on its own even while the app-wide switch is off: a case that turned itself on means exactly that.':
|
||||
'即使应用级开关关闭也自行运行:一个自己开启的案例正是这个意思。',
|
||||
'Uses the app-wide switch and interval for this session. Switch it off to pick your own.':
|
||||
'为此会话使用应用级的开关和间隔。关闭它即可自行选择。',
|
||||
'Fetch this session': '拉取此会话',
|
||||
"Runs on its own even while the app-wide switch is off: a session that turned itself on means exactly that. Sessions sharing one folder share one run, and each sees that run's result.":
|
||||
'即使应用级开关关闭也自行运行:一个自己开启的会话正是这个意思。共用一个文件夹的会话共享一次运行,每个会话都会看到该运行的结果。',
|
||||
'How often': '多久一次',
|
||||
'30 sec to 1 hr, timed from the start of a run, so a slow repository does not push the next one out.':
|
||||
'30 秒到 1 小时,从一次运行的开始计时,所以较慢的仓库不会把下一次推远。',
|
||||
|
||||
@@ -1332,8 +1332,9 @@
|
||||
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M3 3v18h18"/><polyline points="7 14 11 10 14 13 20 7"/></svg>
|
||||
<span>Summary</span>
|
||||
</button>
|
||||
<!-- Git is last: a case that has not turned fetching on shows nothing here, so the section only
|
||||
earns its place once the feature is in use. -->
|
||||
<!-- Git is last: the section shows for every local session, but until the interval feature is
|
||||
in use it holds one toggle and a status row that has never run, so it earns its place at the
|
||||
end of the rail rather than among the tabs a session uses every day. -->
|
||||
<button type="button" class="set-rail-item" data-tab="git" onclick="app.switchOptionsTab('git')">
|
||||
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><circle cx="18" cy="6" r="3"/><circle cx="6" cy="18" r="3"/><path d="M6 3v12"/><path d="M18 9a9 9 0 0 1-9 9"/></svg>
|
||||
<span>Git</span>
|
||||
@@ -1733,13 +1734,14 @@
|
||||
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><circle cx="18" cy="6" r="3"/><circle cx="6" cy="18" r="3"/><path d="M6 3v12"/><path d="M18 9a9 9 0 0 1-9 9"/></svg>
|
||||
<h2>Git</h2>
|
||||
</div>
|
||||
<p class="set-section-blurb">Whether Codeman fetches this case's repositories on a timer.</p>
|
||||
<p class="set-section-blurb">Whether Codeman fetches this session's repositories on a timer.</p>
|
||||
|
||||
<div class="set-group">
|
||||
<div class="set-group-head"><h4>Fetch on an interval</h4></div>
|
||||
<p class="set-group-hint">Fetching is the only git command Codeman runs that writes, and the only one that runs a repository's own hooks, so nothing runs on a timer until you turn it on — here, or in App Settings → Bottom bar. The panel's Fetch button is always just a click.</p>
|
||||
<div class="set-group-body">
|
||||
<!-- Live from the case's own last run (session-ui.js: updateGitFetchStatus). -->
|
||||
<!-- Live from the session's own last run (session-ui.js: updateGitFetchStatus), which is the
|
||||
run its folder got: several sessions on one folder share one run and read the same row. -->
|
||||
<div class="set-row">
|
||||
<div class="set-row-text">
|
||||
<span class="set-row-label">Status</span>
|
||||
@@ -1749,14 +1751,14 @@
|
||||
<div class="set-row">
|
||||
<div class="set-row-text">
|
||||
<span class="set-row-label">Follow the app setting</span>
|
||||
<span class="set-row-desc">Uses the app-wide switch and interval for this case. Switch it off to pick your own.</span>
|
||||
<span class="set-row-desc">Uses the app-wide switch and interval for this session. Switch it off to pick your own.</span>
|
||||
</div>
|
||||
<label class="switch switch-sm"><input type="checkbox" id="modalGitFetchFollow" checked onchange="app.autoSaveGitFetch()"><span class="slider"></span></label>
|
||||
</div>
|
||||
<div class="set-row" id="gitFetchEnabledRow">
|
||||
<div class="set-row-text">
|
||||
<span class="set-row-label">Fetch this case</span>
|
||||
<span class="set-row-desc">Runs on its own even while the app-wide switch is off: a case that turned itself on means exactly that.</span>
|
||||
<span class="set-row-label">Fetch this session</span>
|
||||
<span class="set-row-desc">Runs on its own even while the app-wide switch is off: a session that turned itself on means exactly that. Sessions sharing one folder share one run, and each sees that run's result.</span>
|
||||
</div>
|
||||
<label class="switch switch-sm"><input type="checkbox" id="modalGitFetchEnabled" onchange="app.autoSaveGitFetch()"><span class="slider"></span></label>
|
||||
</div>
|
||||
@@ -2278,14 +2280,14 @@
|
||||
<div class="set-row" data-search="git fetch interval timer behind remote tracking refs poll run automatically">
|
||||
<div class="set-row-text">
|
||||
<span class="set-row-label">Git fetch on an interval <span class="set-scope">synced</span></span>
|
||||
<span class="set-row-desc">Runs `git fetch` for the repositories the Git window lists on a timer, so the "behind" number is current without a click. Off by default: this is the one git command Codeman runs that writes, and the one that runs a repository's own hooks, so nothing runs on a timer until you turn it on. A case can override both the switch and the interval in Session Options → Git.</span>
|
||||
<span class="set-row-desc">Runs `git fetch` for the repositories the Git window lists on a timer, so the "behind" number is current without a click. Off by default: this is the one git command Codeman runs that writes, and the one that runs a repository's own hooks, so nothing runs on a timer until you turn it on. A session can override both the switch and the interval in Session Options → Git.</span>
|
||||
</div>
|
||||
<label class="switch switch-sm"><input type="checkbox" id="appSettingsGitFetchEnabled"><span class="slider"></span></label>
|
||||
</div>
|
||||
<div class="set-row" data-search="git fetch how often seconds between runs interval case override session options">
|
||||
<div class="set-row-text">
|
||||
<span class="set-row-label">Git fetch: how often <span class="set-scope">synced</span></span>
|
||||
<span class="set-row-desc">Seconds between runs for a case that has no override of its own (30 to 3600, default 300). Only a case that is on — here, or in Session Options → Git — is fetched.</span>
|
||||
<span class="set-row-desc">Seconds between runs for a session that has no override of its own (30 to 3600, default 300). Only a session that is on — here, or in Session Options → Git — is fetched, and sessions sharing one folder are fetched by one run.</span>
|
||||
</div>
|
||||
<input type="number" id="appSettingsGitFetchIntervalSeconds" class="set-num" value="300" min="30" max="3600" step="1">
|
||||
</div>
|
||||
|
||||
@@ -3048,7 +3048,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
if (!session) return;
|
||||
|
||||
const app = this.loadAppSettingsFromStorage() || {};
|
||||
// No choice of its own on either field means the case follows the app. A partial one — its own switch,
|
||||
// No choice of its own on either field means the session follows the app. A partial one — its own switch,
|
||||
// the app's interval — means it does not, and the row that still follows shows the app's value.
|
||||
const follows = session.gitFetchEnabled === undefined && session.gitFetchIntervalSeconds === undefined;
|
||||
const enabled = follows ? app.gitFetchEnabled === true : session.gitFetchEnabled ?? false;
|
||||
@@ -3069,7 +3069,7 @@ Object.assign(CodemanApp.prototype, {
|
||||
run.state === 'ok' ? `${run.repos} repositories` : run.state,
|
||||
run.failed ? `${run.failed} failed` : '',
|
||||
]
|
||||
: [follows ? 'Nothing fetched on a timer yet, and this case follows the app setting.' : 'Nothing fetched on a timer yet for this case.'];
|
||||
: [follows ? 'Nothing fetched on a timer yet, and this session follows the app setting.' : 'Nothing fetched on a timer yet for this session.'];
|
||||
statusEl.textContent = parts.filter(Boolean).join(' · ');
|
||||
statusEl.hidden = false;
|
||||
},
|
||||
@@ -3096,9 +3096,18 @@ Object.assign(CodemanApp.prototype, {
|
||||
),
|
||||
};
|
||||
try {
|
||||
const data = await this._apiPost(`/api/sessions/${this.editingSessionId}/git-fetch-settings`, body);
|
||||
// `_apiPost` hands back the Response without checking it, so a refused body (a 400) still read as a
|
||||
// save. `_apiJson` is the one that says whether the call went through, and null here means it did not.
|
||||
const data = await this._apiJson(`/api/sessions/${this.editingSessionId}/git-fetch-settings`, {
|
||||
method: 'POST',
|
||||
body,
|
||||
});
|
||||
if (!data) {
|
||||
this.showToast('Failed to save Git fetch settings', 'error');
|
||||
return;
|
||||
}
|
||||
const session = this.sessions.get(this.editingSessionId);
|
||||
const saved = data?.gitFetchSettings;
|
||||
const saved = data.gitFetchSettings;
|
||||
if (session && saved) {
|
||||
session.gitFetchEnabled = saved.enabled ?? undefined;
|
||||
session.gitFetchIntervalSeconds = saved.intervalSeconds ?? undefined;
|
||||
|
||||
@@ -16,10 +16,10 @@
|
||||
import type { FastifyInstance } from 'fastify';
|
||||
import { ApiErrorCode, createErrorResponse, getErrorMessage, type ApiResponse } from '../../types.js';
|
||||
import { redactGitCredentials } from '../../git-clone.js';
|
||||
import { emptyReport, fetchGitWorkspaceOverview, type GitFetchReport } from '../../git-fetch.js';
|
||||
import { emptyReport, fetchWorkspaceOnce, gitFetchStatusFrom, type GitFetchReport } from '../../git-fetch.js';
|
||||
import { readDockerCases } from '../../docker-hosts.js';
|
||||
import { getDataDir } from '../../config/instance.js';
|
||||
import { findSessionOrFail } from '../route-helpers.js';
|
||||
import { findSessionOrFail, persistAndBroadcastSession } from '../route-helpers.js';
|
||||
import {
|
||||
emptyOverview,
|
||||
findWorkspaceRepo,
|
||||
@@ -32,7 +32,7 @@ import {
|
||||
type GitRunner,
|
||||
type GitWorkspaceOverview,
|
||||
} from '../../git-workspace-status.js';
|
||||
import type { SessionPort } from '../ports/index.js';
|
||||
import type { EventPort, SessionPort } from '../ports/index.js';
|
||||
|
||||
/**
|
||||
* Host paths of every Docker case workspace: repositories at or inside these are never inspected.
|
||||
@@ -58,7 +58,9 @@ function limitsFrom(query: { maxRepos?: unknown; timeout?: unknown }) {
|
||||
|
||||
export function registerGitStatusRoutes(
|
||||
app: FastifyInstance,
|
||||
ctx: SessionPort,
|
||||
// EventPort as well as SessionPort: the fetch route publishes the run it did, and a route that broadcasts
|
||||
// has to say so in its own signature.
|
||||
ctx: SessionPort & EventPort,
|
||||
git?: GitRunner,
|
||||
dockerWorkspaces: () => Promise<string[]> = defaultDockerWorkspaces
|
||||
): void {
|
||||
@@ -131,9 +133,14 @@ export function registerGitStatusRoutes(
|
||||
// Fetch, and only because the user clicked Fetch in the panel (src/git-fetch.ts): one
|
||||
// `git fetch --quiet` per repository the panel would show, through the same resolver so the two cannot
|
||||
// disagree about which repositories belong to the workspace. This is the only git command Codeman runs
|
||||
// that writes, and it writes remote-tracking refs only — nothing commits, checks out or merges, so an
|
||||
// agent working in the same repository is not disturbed. Remote and Docker sessions answer `unsupported`
|
||||
// like the status route, and a repository a container can write to is refused before any git runs.
|
||||
// that writes, and it writes remote-tracking refs only — nothing commits, checks out or merges. It is not
|
||||
// as harmless as a read though: a fetch holds git's ref lock, so an agent's own `git pull` in the same
|
||||
// clone can collide with it, which is why the run goes through `fetchWorkspaceOnce` — a click while a tick
|
||||
// is already fetching this folder joins that run instead of starting a second one. It can also run a
|
||||
// repository's own hooks (`reference-transaction`, `pre-auto-gc` through the maintenance git runs after a
|
||||
// fetch) and its config commands such as `credential.helper`, so the `FETCH_CONFIG_ARGS` overrides turn
|
||||
// maintenance and gc off, and a repository a container can write to is refused before any git runs. Remote
|
||||
// and Docker sessions answer `unsupported` like the status route.
|
||||
app.post('/api/sessions/:id/git-fetch', async (req): Promise<ApiResponse<GitFetchReport>> => {
|
||||
const { id } = req.params as { id: string };
|
||||
const query = req.query as { maxRepos?: unknown; timeout?: unknown };
|
||||
@@ -141,13 +148,16 @@ export function registerGitStatusRoutes(
|
||||
const session = findSessionOrFail(ctx, id, req);
|
||||
if (session.remote) return { success: true, data: emptyReport('unsupported', { reason: 'remote' }) };
|
||||
if (session.docker) return { success: true, data: emptyReport('unsupported', { reason: 'docker' }) };
|
||||
return {
|
||||
success: true,
|
||||
data: await fetchGitWorkspaceOverview(session.workingDir, {
|
||||
git,
|
||||
...limits,
|
||||
dockerWorkspaces: await dockerWorkspaces(),
|
||||
}),
|
||||
};
|
||||
const report = await fetchWorkspaceOnce(session.workingDir, {
|
||||
git,
|
||||
...limits,
|
||||
dockerWorkspaces: await dockerWorkspaces(),
|
||||
});
|
||||
// The response answers the click, and the run is also the session's current truth: what the panel's
|
||||
// "behind" number is based on. Publishing it is what the api docs say a click does, and what makes a
|
||||
// second tab open on the same folder see the same run — including a click that joined a tick already
|
||||
// going, where the row has to say who last fetched.
|
||||
if (session.setGitFetchStatus(gitFetchStatusFrom(report, 'click'))) persistAndBroadcastSession(ctx, session);
|
||||
return { success: true, data: report };
|
||||
});
|
||||
}
|
||||
@@ -3611,11 +3611,13 @@ export function registerSessionRoutes(
|
||||
|
||||
// ========== Git fetch on an interval ==========
|
||||
|
||||
// Per-case override for the fetch interval (src/git-fetch-scheduler.ts). `null` on either field clears
|
||||
// the case's choice so the app-wide setting applies again, and a field left out of the body leaves the
|
||||
// other half as it was: the Session Options rows save on their own. Refused for a remote or Docker
|
||||
// session, exactly as the Git panel excludes them — a container can write to the repository it mounts,
|
||||
// so git there would run that repository's own configuration on the host.
|
||||
// Per-session override for the fetch interval (src/git-fetch-scheduler.ts), which is what the Session
|
||||
// Options rows say it is: two sessions on one folder can hold different choices, and that folder is
|
||||
// fetched when the first of them is due. `null` on either field clears this session's choice so the
|
||||
// app-wide setting applies again, and a field left out of the body leaves the other half as it was: the
|
||||
// Session Options rows save on their own. Refused for a remote or Docker session, exactly as the Git
|
||||
// panel excludes them — a container can write to the repository it mounts, so git there would run that
|
||||
// repository's own configuration on the host.
|
||||
app.post('/api/sessions/:id/git-fetch-settings', async (req) => {
|
||||
const { id } = req.params as { id: string };
|
||||
const body = parseBody(GitFetchSettingsSchema, req.body, 'Invalid request body');
|
||||
|
||||
+5
-3
@@ -1492,9 +1492,11 @@ export const SettingsUpdateSchema = z
|
||||
* Git fetch on an interval (src/git-fetch-scheduler.ts): `git fetch` for the repositories the Git panel
|
||||
* lists, run on a timer instead of only on the panel's Fetch click. SYNCED, default OFF, because the
|
||||
* server is what runs it: `git fetch` is the only git command Codeman runs that writes, and the only
|
||||
* one that runs a repository's own hooks, so nothing runs on a timer until the user turns this on.
|
||||
* Cap in seconds between runs (30 to 3600, default 300). A case can override both in Session Options
|
||||
* → Git; `null` there clears the override.
|
||||
* one that can run a repository's own hooks (`reference-transaction`, `pre-auto-gc` through the
|
||||
* automatic maintenance a fetch triggers, config commands such as `credential.helper`), so nothing runs
|
||||
* on a timer until the user turns this on. Cap in seconds between runs (30 to 3600, default 300). A
|
||||
* session can override both in Session Options → Git; `null` there clears the override, and sessions
|
||||
* sharing one folder are fetched by one run.
|
||||
*/
|
||||
gitFetchEnabled: z.boolean().optional(),
|
||||
gitFetchIntervalSeconds: z.number().int().min(30).max(3600).optional(),
|
||||
|
||||
+11
-3
@@ -124,8 +124,9 @@ import { AI_CHECK_MODEL } from '../config/ai-defaults.js';
|
||||
import { approvalInbox } from './approval-inbox.js';
|
||||
import { SCREEN_BLOCKED_SOURCE } from './screen-blocked.js';
|
||||
import { PromptCacheKeepWarmService } from '../prompt-cache-keepwarm.js';
|
||||
import { fetchGitWorkspaceOverview } from '../git-fetch.js';
|
||||
import { fetchWorkspaceOnce } from '../git-fetch.js';
|
||||
import { GitFetchScheduler } from '../git-fetch-scheduler.js';
|
||||
import { probePathKind } from '../utils/bounded-path-probe.js';
|
||||
import { defaultDockerWorkspaces } from './routes/git-status-routes.js';
|
||||
import { composerIsEmpty, plainFrame } from '../session-submit-verifier.js';
|
||||
import { detectUsageLimitPause } from '../usage-limit-patterns.js';
|
||||
@@ -405,12 +406,19 @@ export class WebServer extends EventEmitter {
|
||||
readSettings: readGitFetchSettings,
|
||||
// The same Docker list the panel's routes use: the rule has to be one rule.
|
||||
dockerWorkspaces: defaultDockerWorkspaces,
|
||||
fetch: (cwd, opts) => fetchGitWorkspaceOverview(cwd, opts),
|
||||
// Bounded probe, DEFAULT options (rule #516): a tick is a bulk scan, so it keeps the stall cap and must
|
||||
// never be handed `pastCap`. A folder on a mount that stopped answering is skipped, not fetched.
|
||||
probePath: (path) => probePathKind(path),
|
||||
// Through the join, so the tick and the panel's Fetch button are the same run: a click on a folder the
|
||||
// tick is already fetching joins it rather than starting a second `git fetch` in the same clone.
|
||||
fetch: (cwd, opts) => fetchWorkspaceOnce(cwd, opts),
|
||||
publish: (id, status) => {
|
||||
const session = this.sessions.get(id);
|
||||
if (!session?.setGitFetchStatus(status)) return;
|
||||
// No new SSE event: the status rides the session state the page already listens for, and the Git
|
||||
// panel polls on its own clock. Persisted, because what a run found is the panel's current truth.
|
||||
// panel polls on its own clock. It is written to disk with the rest of the record, but the restore
|
||||
// copies only the fields its own config lists and this is not one of them (src/types/session.ts), so
|
||||
// what a run found is the panel's truth for the life of the session, not across a restart.
|
||||
this.persistSessionState(session);
|
||||
this.broadcast(SseEvent.SessionUpdated, this.getSessionStateWithRespawn(session));
|
||||
},
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
/**
|
||||
* @fileoverview The Git fetch interval scheduler (src/git-fetch-scheduler.ts): which cases a tick runs, and
|
||||
* which it must not. Every rule is checked by calling `tick(now)` with an explicit clock, so nothing here
|
||||
* @fileoverview The Git fetch interval scheduler (src/git-fetch-scheduler.ts): which sessions a tick runs,
|
||||
* and which it must not. Every rule is checked by calling `tick(now)` with an explicit clock, so nothing here
|
||||
* depends on fake timers or on how long a real tick takes — the scheduler owns no timers itself.
|
||||
*
|
||||
* The run belongs to the FOLDER, so these tests put two sessions on one folder and expect one fetch that
|
||||
* both see, and they check the tick's two bounds separately: one shared bound for the fetches of the whole
|
||||
* tick, and the folders bounded by a bound that cannot be the same semaphore.
|
||||
*
|
||||
* Port: N/A.
|
||||
*/
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
@@ -14,11 +18,13 @@ import {
|
||||
} from '../src/git-fetch-scheduler.js';
|
||||
import {
|
||||
FETCH_CONCURRENCY,
|
||||
type GitFetchOptions,
|
||||
type GitFetchReport,
|
||||
type GitFetchSettings,
|
||||
type GitFetchStatus,
|
||||
} from '../src/git-fetch.js';
|
||||
import { DEFAULT_GIT_TIMEOUT_MS, MAX_REPOS } from '../src/git-workspace-status.js';
|
||||
import type { PathProbeKind } from '../src/utils/bounded-path-probe.js';
|
||||
|
||||
const SETTINGS: GitFetchSettings = { enabled: true, intervalSeconds: 300 };
|
||||
const VIEW = (id: string, extra: Partial<GitFetchSessionView> = {}): GitFetchSessionView => ({
|
||||
@@ -38,10 +44,19 @@ const report = (cwd: string, name: string): GitFetchReport => ({
|
||||
type Harness = {
|
||||
scheduler: GitFetchScheduler;
|
||||
ports: GitFetchPorts;
|
||||
calls: Array<{ id: string; opts: unknown }>;
|
||||
/** The folders the tick actually ran, in the order they were picked. */
|
||||
calls: Array<{ cwd: string; opts: GitFetchOptions }>;
|
||||
published: Array<{ id: string; status: GitFetchStatus }>;
|
||||
/** Every path the tick probed, in order — the probe has to happen before anything is fetched. */
|
||||
probed: string[];
|
||||
settings: GitFetchSettings;
|
||||
dockerReads: number;
|
||||
/**
|
||||
* A session that closed: the server calls `remove()` when the session is gone, so this takes it out of the
|
||||
* ports as well as out of the scheduler. Otherwise a test that removes a session still sees it on the next
|
||||
* tick, and the folder it was on looks still open.
|
||||
*/
|
||||
remove(id: string): void;
|
||||
};
|
||||
|
||||
/** A run the test decides when to let finish, so a slow repository can be held open across ticks. */
|
||||
@@ -57,21 +72,32 @@ function deferred<T>(value: T) {
|
||||
*/
|
||||
function harness(
|
||||
sessions: GitFetchSessionView[],
|
||||
opts: { settings?: GitFetchSettings; fetch?: (id: string) => Promise<GitFetchReport> } = {}
|
||||
opts: {
|
||||
settings?: GitFetchSettings;
|
||||
fetch?: (cwd: string, portOpts: GitFetchOptions) => Promise<GitFetchReport>;
|
||||
/** Default: every path answers as a folder, so a test that does not care about the probe stays quiet. */
|
||||
probe?: (path: string) => Promise<PathProbeKind>;
|
||||
} = {}
|
||||
): Harness {
|
||||
const customFetch = opts.fetch;
|
||||
const settings = opts.settings ?? { ...SETTINGS };
|
||||
const gone = new Set<string>();
|
||||
const h: Harness = {
|
||||
scheduler: undefined as unknown as GitFetchScheduler,
|
||||
ports: undefined as unknown as GitFetchPorts,
|
||||
calls: [],
|
||||
published: [],
|
||||
probed: [],
|
||||
settings,
|
||||
dockerReads: 0,
|
||||
remove: (id) => {
|
||||
gone.add(id);
|
||||
h.scheduler.remove(id);
|
||||
},
|
||||
};
|
||||
h.ports = {
|
||||
sessionIds: () => sessions.map((s) => s.id),
|
||||
getSession: (id) => sessions.find((s) => s.id === id),
|
||||
sessionIds: () => sessions.filter((s) => !gone.has(s.id)).map((s) => s.id),
|
||||
getSession: (id) => (gone.has(id) ? undefined : sessions.find((s) => s.id === id)),
|
||||
readSettings: async () => {
|
||||
if (h.settings === null) throw new Error('settings.json is mid-write');
|
||||
return h.settings;
|
||||
@@ -80,10 +106,13 @@ function harness(
|
||||
h.dockerReads++;
|
||||
return [];
|
||||
},
|
||||
probePath: async (path) => {
|
||||
h.probed.push(path);
|
||||
return opts.probe ? await opts.probe(path) : 'directory';
|
||||
},
|
||||
fetch: async (cwd, portOpts) => {
|
||||
const id = cwd.slice(3); // the '/w/<id>' the view was built with
|
||||
h.calls.push({ id, opts: portOpts });
|
||||
return customFetch ? customFetch(id) : report(cwd, id);
|
||||
h.calls.push({ cwd, opts: portOpts });
|
||||
return customFetch ? customFetch(cwd, portOpts) : report(cwd, cwd.slice(3));
|
||||
},
|
||||
publish: (id, status) => h.published.push({ id, status }),
|
||||
};
|
||||
@@ -96,7 +125,8 @@ describe('a quiet tick', () => {
|
||||
const h = harness([VIEW('a'), VIEW('b')], { settings: { enabled: false, intervalSeconds: 300 } });
|
||||
expect(await h.scheduler.tick(1000)).toBe(0);
|
||||
expect(h.calls).toEqual([]);
|
||||
// The Docker case list is only worth reading once something is actually due.
|
||||
// The probe and the Docker case list are only worth reading once something is actually due.
|
||||
expect(h.probed).toEqual([]);
|
||||
expect(h.dockerReads).toBe(0);
|
||||
});
|
||||
|
||||
@@ -125,16 +155,38 @@ describe('a quiet tick', () => {
|
||||
expect(logged).toHaveBeenCalled();
|
||||
|
||||
h.settings = SETTINGS;
|
||||
// Nothing was recorded for the failed tick, so the case is still due at once.
|
||||
// Nothing was recorded for the failed tick, so the folder is still due at once.
|
||||
expect(await h.scheduler.tick(1000)).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the bounded probe', () => {
|
||||
it('probes the folder before fetching it, and skips anything that is not a directory', async () => {
|
||||
const kinds = new Map<string, PathProbeKind>([
|
||||
['/w/file', 'file'],
|
||||
['/w/unknown', 'unknown'],
|
||||
['/w/absent', 'absent'],
|
||||
]);
|
||||
const h = harness([VIEW('file'), VIEW('unknown'), VIEW('absent'), VIEW('ok')], {
|
||||
probe: async (path) => kinds.get(path) ?? 'directory',
|
||||
});
|
||||
const logged = vi.fn();
|
||||
h.ports.log = logged;
|
||||
expect(await h.scheduler.tick(0)).toBe(1);
|
||||
expect(h.calls.map((c) => c.cwd)).toEqual(['/w/ok']);
|
||||
// The folders that answered badly were probed and left alone, never fetched.
|
||||
expect(h.probed).toEqual(['/w/file', '/w/unknown', '/w/absent', '/w/ok']);
|
||||
expect(logged).toHaveBeenCalledTimes(3);
|
||||
// `unknown` is never read as `absent`: the log says what it answered rather than what it was assumed.
|
||||
expect(logged.mock.calls.join(' ')).toContain('unknown');
|
||||
});
|
||||
});
|
||||
|
||||
describe('who runs', () => {
|
||||
it('runs every case the app setting covers, once per interval', async () => {
|
||||
it('runs every session the app setting covers, once per interval', async () => {
|
||||
const h = harness([VIEW('a'), VIEW('b')], { settings: { enabled: true, intervalSeconds: 60 } });
|
||||
await h.scheduler.tick(0);
|
||||
expect(h.calls.map((c) => c.id).sort()).toEqual(['a', 'b']);
|
||||
expect(h.calls.map((c) => c.cwd).sort()).toEqual(['/w/a', '/w/b']);
|
||||
|
||||
await h.scheduler.tick(30_000); // not due yet
|
||||
expect(h.calls.length).toBe(2);
|
||||
@@ -143,29 +195,29 @@ describe('who runs', () => {
|
||||
expect(h.calls.length).toBe(4);
|
||||
});
|
||||
|
||||
it('runs a case that turned itself on even while the app-wide switch is off — that is what a per-case switch has to mean', async () => {
|
||||
it('runs a session that turned itself on even while the app-wide switch is off — that is what a per-session switch has to mean', async () => {
|
||||
const h = harness([VIEW('plain'), VIEW('own', { gitFetchEnabled: true, gitFetchIntervalSeconds: 30 })], {
|
||||
settings: { enabled: false, intervalSeconds: 300 },
|
||||
});
|
||||
await h.scheduler.tick(0);
|
||||
expect(h.calls.map((c) => c.id)).toEqual(['own']);
|
||||
expect(h.calls.map((c) => c.cwd)).toEqual(['/w/own']);
|
||||
|
||||
// Its own interval, not the app's: due at 30 s, not at 300.
|
||||
await h.scheduler.tick(30_000);
|
||||
expect(h.calls.length).toBe(2);
|
||||
});
|
||||
|
||||
it('never fetches a remote or Docker case, exactly as the Git panel excludes them', async () => {
|
||||
it('never fetches a remote or Docker session, exactly as the Git panel excludes them', async () => {
|
||||
const h = harness([
|
||||
VIEW('ssh', { remote: { host: 'x' } }),
|
||||
VIEW('docker', { docker: { container: 'c' } }),
|
||||
VIEW('local'),
|
||||
]);
|
||||
await h.scheduler.tick(0);
|
||||
expect(h.calls.map((c) => c.id)).toEqual(['local']);
|
||||
expect(h.calls.map((c) => c.cwd)).toEqual(['/w/local']);
|
||||
});
|
||||
|
||||
it('re-checks the Docker rule every tick, so a case that became a Docker case stops fetching', async () => {
|
||||
it('re-checks the Docker rule every tick, so a session whose case became a Docker case stops fetching', async () => {
|
||||
const sessions = [VIEW('a')];
|
||||
const h = harness(sessions);
|
||||
await h.scheduler.tick(0);
|
||||
@@ -183,16 +235,78 @@ describe('who runs', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('one run at a time', () => {
|
||||
it('skips a case still being fetched instead of stacking runs on it', async () => {
|
||||
const gate = deferred<GitFetchReport>(report('/w/a', 'a'));
|
||||
const h = harness([VIEW('a')], { settings: { enabled: true, intervalSeconds: 30 }, fetch: () => gate.promise });
|
||||
const running = h.scheduler.tick(0);
|
||||
await new Promise((resolve) => setImmediate(resolve)); // the run has started, and is still held open
|
||||
expect(await h.scheduler.tick(15_000)).toBe(0);
|
||||
describe('one folder, one run', () => {
|
||||
it('fetches a shared folder once and publishes the result to every session open on it', async () => {
|
||||
const h = harness([VIEW('a'), { ...VIEW('b'), workingDir: '/w/a' }]);
|
||||
expect(await h.scheduler.tick(0)).toBe(1); // one folder, not two runs
|
||||
expect(h.calls.length).toBe(1);
|
||||
gate.resolve(report('/w/a', 'a'));
|
||||
await running;
|
||||
expect(h.published.map((p) => p.id).sort()).toEqual(['a', 'b']);
|
||||
// Both see the same run, so the panel's row says the same thing in both tabs.
|
||||
expect(new Set(h.published.map((p) => p.status.failed)).size).toBe(1);
|
||||
});
|
||||
|
||||
it('does not pick a folder again while its run is going, and does not run a second tick over the same selection', async () => {
|
||||
const ids = Array.from({ length: FETCH_CONCURRENCY + 2 }, (_, i) => `s${i}`);
|
||||
const gate = deferred<GitFetchReport>(report('/w/first', 'first'));
|
||||
const h = harness(
|
||||
ids.map((id) => VIEW(id)),
|
||||
{
|
||||
settings: { enabled: true, intervalSeconds: 30 },
|
||||
fetch: () => gate.promise,
|
||||
}
|
||||
);
|
||||
|
||||
const first = h.scheduler.tick(0);
|
||||
await new Promise((resolve) => setImmediate(resolve)); // the first runs are going, the rest are queued
|
||||
const picked = h.calls.length;
|
||||
expect(picked).toBe(FETCH_CONCURRENCY);
|
||||
|
||||
// The server fires the next tick without waiting. The schedule was stamped at selection, so the folders
|
||||
// already picked are not picked again, and this tick is skipped while the previous one is still going.
|
||||
expect(await h.scheduler.tick(15_000)).toBe(0);
|
||||
expect(h.calls.length).toBe(picked);
|
||||
|
||||
gate.resolve(report('/w/first', 'first'));
|
||||
await first;
|
||||
// Every folder ran exactly once, and no folder ran twice.
|
||||
expect(h.calls.length).toBe(ids.length);
|
||||
expect(new Set(h.calls.map((c) => c.cwd)).size).toBe(ids.length);
|
||||
});
|
||||
|
||||
it('shares one bound across the whole tick instead of giving each folder its own, and bounds the folders with a bound of its own', async () => {
|
||||
const ids = Array.from({ length: FETCH_CONCURRENCY + 1 }, (_, i) => `s${i}`);
|
||||
let fetchesLive = 0;
|
||||
let fetchesPeak = 0;
|
||||
let foldersLive = 0;
|
||||
let foldersPeak = 0;
|
||||
const hold = () => new Promise((resolve) => setImmediate(resolve));
|
||||
const h = harness(
|
||||
ids.map((id) => VIEW(id)),
|
||||
{
|
||||
// Every folder is handed the SAME limiter for its work, so that work cannot get its own bound: a
|
||||
// per-call bound would let 5 folders × 6 items run at once. The folders themselves are bounded by a
|
||||
// DIFFERENT bound, because one semaphore cannot serve both levels — a folder holding a slot and then
|
||||
// waiting for one for its own fetch is waiting on itself, and the folders that fill the bound are
|
||||
// exactly the ones blocking it. If the tick used one semaphore for both, this test would hang and fail
|
||||
// as a timeout rather than as an assertion.
|
||||
fetch: async (cwd, opts) => {
|
||||
foldersLive++;
|
||||
foldersPeak = Math.max(foldersPeak, foldersLive);
|
||||
await opts.limiter?.([0, 1, 2, 3, 4, 5], async () => {
|
||||
fetchesLive++;
|
||||
fetchesPeak = Math.max(fetchesPeak, fetchesLive);
|
||||
await hold();
|
||||
fetchesLive--;
|
||||
});
|
||||
foldersLive--;
|
||||
return report(cwd, cwd.slice(3));
|
||||
},
|
||||
}
|
||||
);
|
||||
await h.scheduler.tick(0);
|
||||
expect(fetchesPeak).toBe(FETCH_CONCURRENCY); // the tick's ceiling, not each folder's own
|
||||
expect(foldersPeak).toBe(FETCH_CONCURRENCY);
|
||||
expect(h.calls.length).toBe(ids.length); // every folder ran, none was left waiting forever
|
||||
});
|
||||
|
||||
it('times the interval from the START of a run, so a slow repository does not push the next one out', async () => {
|
||||
@@ -203,34 +317,13 @@ describe('one run at a time', () => {
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
expect(h.calls.length).toBe(1);
|
||||
|
||||
// The run takes 200 s, longer than its own interval. Once it lands, the case is already past due: the
|
||||
// The run takes 200 s, longer than its own interval. Once it lands, the folder is already past due: the
|
||||
// next tick runs it again instead of waiting another 30 s from when the slow one finished.
|
||||
slow.resolve(report('/w/a', 'a'));
|
||||
await first;
|
||||
expect(await h.scheduler.tick(200_000)).toBe(1);
|
||||
expect(h.calls.length).toBe(2);
|
||||
});
|
||||
|
||||
it('runs at most FETCH_CONCURRENCY cases at a time, because each one is a git process', async () => {
|
||||
const ids = Array.from({ length: FETCH_CONCURRENCY + 3 }, (_, i) => `s${i}`);
|
||||
let live = 0;
|
||||
let peak = 0;
|
||||
const h = harness(
|
||||
ids.map((id) => VIEW(id)),
|
||||
{
|
||||
fetch: async () => {
|
||||
live++;
|
||||
peak = Math.max(peak, live);
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
live--;
|
||||
return report('/w/x', 'x');
|
||||
},
|
||||
}
|
||||
);
|
||||
await h.scheduler.tick(0);
|
||||
expect(peak).toBe(FETCH_CONCURRENCY);
|
||||
expect(h.calls.length).toBe(ids.length);
|
||||
});
|
||||
});
|
||||
|
||||
describe('what a run reports', () => {
|
||||
@@ -245,7 +338,7 @@ describe('what a run reports', () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("reports a failure instead of retrying harder, and keeps the case's own next run at its own interval", async () => {
|
||||
it("reports a failure instead of retrying harder, and keeps the folder's next run at its own interval", async () => {
|
||||
const h = harness([VIEW('a')], {
|
||||
settings: { enabled: true, intervalSeconds: 30 },
|
||||
fetch: () => Promise.reject(new Error('spawn git ENOENT')),
|
||||
@@ -261,7 +354,7 @@ describe('what a run reports', () => {
|
||||
});
|
||||
|
||||
describe('the effective choice', () => {
|
||||
it("a case's own override wins over the app setting", () => {
|
||||
it("a session's own override wins over the app setting", () => {
|
||||
const settings = { enabled: false, intervalSeconds: 600 };
|
||||
expect(effectiveGitFetch(VIEW('a', { gitFetchEnabled: true, gitFetchIntervalSeconds: 45 }), settings)).toEqual({
|
||||
enabled: true,
|
||||
@@ -286,19 +379,30 @@ describe('the effective choice', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('a case that goes away', () => {
|
||||
it('takes its schedule with it, so a reopened case starts on its own clock', async () => {
|
||||
const h = harness([VIEW('a')]);
|
||||
describe('a session that goes away', () => {
|
||||
it('takes its own choice with it, but keeps the folder while another session is still open on it', async () => {
|
||||
const sessions = [VIEW('a'), { ...VIEW('b'), workingDir: '/w/a' }];
|
||||
const h = harness(sessions);
|
||||
await h.scheduler.tick(0);
|
||||
expect(h.scheduler.tracked()).toBe(1);
|
||||
|
||||
h.scheduler.remove('a');
|
||||
h.remove('a');
|
||||
// The run belongs to the clone, so 'b' keeps the folder's schedule and its next run is still on time:
|
||||
// the choice that got the folder going was 'a's, but the folder is not 'a's.
|
||||
expect(h.scheduler.tracked()).toBe(1);
|
||||
await h.scheduler.tick(1000); // not due yet, and 'b' is still open on the folder
|
||||
expect(h.calls.length).toBe(1);
|
||||
|
||||
// The last session open on the folder goes, so the folder is forgotten: a session coming back on it is due
|
||||
// at once rather than on the schedule the closed one left behind.
|
||||
h.remove('b');
|
||||
expect(h.scheduler.tracked()).toBe(0);
|
||||
await h.scheduler.tick(1000);
|
||||
sessions.push({ ...VIEW('c'), workingDir: '/w/a' });
|
||||
await h.scheduler.tick(2000);
|
||||
expect(h.calls.length).toBe(2); // remembered again, from now
|
||||
});
|
||||
|
||||
it('stops everything after stop(), including a case that was already due', async () => {
|
||||
it('stops everything after stop(), including a folder that was already due', async () => {
|
||||
const h = harness([VIEW('a')]);
|
||||
h.scheduler.stop();
|
||||
expect(await h.scheduler.tick(0)).toBe(0);
|
||||
|
||||
+181
-6
@@ -1,16 +1,20 @@
|
||||
/**
|
||||
* @fileoverview Tests for the Git panel's Fetch button (src/git-fetch.ts): the one git command Codeman
|
||||
* runs that writes, and the one that runs a repository's own hooks — which is why it is a click, not a poll.
|
||||
* runs that writes, and the one that can run a repository's own hooks — which is why it is a click, not a
|
||||
* poll.
|
||||
*
|
||||
* Two halves, mirroring the module:
|
||||
* Three halves, mirroring the module:
|
||||
*
|
||||
* 1. The PURE half — the fixed argv, the failure classification, and the two reads the interval scheduler
|
||||
* depends on: what the settings say (`gitFetchSettingsFrom`) and what a run did
|
||||
* (`gitFetchStatusFrom`). No spawning.
|
||||
* 1. The PURE half — the fixed argv, the `-c` overrides the runner adds ahead of it, the failure
|
||||
* classification, and the two reads the interval scheduler depends on: what the settings say
|
||||
* (`gitFetchSettingsFrom`) and what a run did (`gitFetchStatusFrom`). No spawning.
|
||||
* 2. The IO half — a REAL git fetching a REAL local bare repo, so that "remote-tracking refs only" is
|
||||
* proven against git's actual behavior: HEAD, the index and the working tree must be exactly what they
|
||||
* were while the remote-tracking ref moves. The rest of the half uses an injected runner over real
|
||||
* repositories on disk, because what is being pinned is WHICH repositories git is allowed to touch.
|
||||
* 3. The join (`fetchWorkspaceOnce`) — one run per folder at a time, because two fetches in one clone
|
||||
* collide on git's ref lock, and the bound is ONE limiter shared with the scheduler's tick rather than a
|
||||
* bound each run gets for itself.
|
||||
*
|
||||
* Port: N/A (no server).
|
||||
*/
|
||||
@@ -23,14 +27,17 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||
import { classifyGitFailure, isGitAvailable } from '../src/git-clone.js';
|
||||
import {
|
||||
FETCH_CONCURRENCY,
|
||||
FETCH_CONFIG_ARGS,
|
||||
buildFetchArgs,
|
||||
clearGitFetchLane,
|
||||
describeFetchFailure,
|
||||
fetchGitWorkspaceOverview,
|
||||
fetchWorkspaceOnce,
|
||||
gitFetchSettingsFrom,
|
||||
gitFetchStatusFrom,
|
||||
type GitFetchReport,
|
||||
} from '../src/git-fetch.js';
|
||||
import { runGit, type GitRunner } from '../src/git-workspace-status.js';
|
||||
import { createLimiter, runGit, type GitRunner } from '../src/git-workspace-status.js';
|
||||
|
||||
const ENV = {
|
||||
...process.env,
|
||||
@@ -175,6 +182,36 @@ describe('gitFetchStatusFrom', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('FETCH_CONFIG_ARGS', () => {
|
||||
it('turns off the hooks and the maintenance a fetch can otherwise start, pinned without spawning git', () => {
|
||||
expect(FETCH_CONFIG_ARGS).toEqual([
|
||||
'-c',
|
||||
'core.fsmonitor=false',
|
||||
'-c',
|
||||
'maintenance.auto=false',
|
||||
'-c',
|
||||
'gc.auto=0',
|
||||
]);
|
||||
// Read as pairs: a `-c` and the setting it overrides, and every override is one of these three.
|
||||
for (let i = 0; i < FETCH_CONFIG_ARGS.length; i += 2) {
|
||||
expect(FETCH_CONFIG_ARGS[i]).toBe('-c');
|
||||
expect(FETCH_CONFIG_ARGS[i + 1]).toMatch(/^(core\.fsmonitor|maintenance\.auto|gc\.auto)=/);
|
||||
}
|
||||
expect(FETCH_CONFIG_ARGS.filter((a) => a !== '-c').sort()).toEqual([
|
||||
'core.fsmonitor=false',
|
||||
'gc.auto=0',
|
||||
'maintenance.auto=false',
|
||||
]);
|
||||
});
|
||||
|
||||
it('is kept apart from the pinned argv so both can be read without spawning, and the runner composes them in that order', () => {
|
||||
// The argv the recorded calls pin stays the verb alone: the overrides come from the runner, not from here.
|
||||
for (const arg of buildFetchArgs()) expect(arg).not.toBe('-c');
|
||||
// Overrides before the verb, which is where git will take them.
|
||||
expect([...FETCH_CONFIG_ARGS, ...buildFetchArgs()]).toEqual([...FETCH_CONFIG_ARGS, 'fetch', '--quiet']);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Real git, real local repository ─────────────────────────────────────────
|
||||
|
||||
const gitPresent = isGitAvailable();
|
||||
@@ -347,3 +384,141 @@ describe.skipIf(!gitPresent)('fetchGitWorkspaceOverview (real git)', () => {
|
||||
expect(calls.filter(([, args]) => args[0] === 'fetch')).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── One run per folder, and one bound for the whole tick ────────────────────
|
||||
|
||||
describe.skipIf(!gitPresent)('fetchWorkspaceOnce (the join)', () => {
|
||||
// The resolver still runs real git, so these need real repositories on disk; what is injected is only the
|
||||
// fetch, because what is being pinned here is the joining, not git's answer.
|
||||
beforeEach(() => {
|
||||
// The lane is module state: a run left in it by the previous test would be joined by the next one.
|
||||
clearGitFetchLane();
|
||||
});
|
||||
|
||||
it('joins a run already going for the same folder instead of starting a second git fetch in the same clone', async () => {
|
||||
const folder = join(dir, 'folder');
|
||||
initRepo(join(folder, 'api'));
|
||||
|
||||
let releaseRun = () => {};
|
||||
const held = new Promise<void>((resolve) => (releaseRun = resolve));
|
||||
let running = () => {};
|
||||
const runStarted = new Promise<void>((resolve) => (running = resolve));
|
||||
let firstRunFetches = 0;
|
||||
const firstRunner: GitRunner = async (cwd, args) => {
|
||||
if (args[0] === 'fetch') {
|
||||
firstRunFetches++;
|
||||
running();
|
||||
await held;
|
||||
}
|
||||
return '';
|
||||
};
|
||||
const first = fetchWorkspaceOnce(folder, { git: firstRunner });
|
||||
await runStarted; // the run is going, and still going
|
||||
|
||||
// A click while that run is working the folder: it joins, so it never touches git at all — not even the
|
||||
// resolver's own probe, because it does not resolve anything for itself.
|
||||
let joinerCalls = 0;
|
||||
const joiner = fetchWorkspaceOnce(folder, {
|
||||
git: async () => {
|
||||
joinerCalls++;
|
||||
return '';
|
||||
},
|
||||
});
|
||||
|
||||
releaseRun();
|
||||
const answer = await first;
|
||||
// The joiner gets that same run's answer, including its limits: the point is not to fetch twice.
|
||||
expect(await joiner).toBe(answer);
|
||||
expect(joinerCalls).toBe(0);
|
||||
expect(firstRunFetches).toBe(1);
|
||||
});
|
||||
|
||||
it('keys the join on the resolved folder, so the same folder written two ways is one run', async () => {
|
||||
const folder = join(dir, 'folder');
|
||||
initRepo(join(folder, 'api'));
|
||||
|
||||
let releaseRun = () => {};
|
||||
const held = new Promise<void>((resolve) => (releaseRun = resolve));
|
||||
let running = () => {};
|
||||
const runStarted = new Promise<void>((resolve) => (running = resolve));
|
||||
let fetches = 0;
|
||||
const firstRunner: GitRunner = async (cwd, args) => {
|
||||
if (args[0] === 'fetch') {
|
||||
fetches++;
|
||||
running();
|
||||
await held;
|
||||
}
|
||||
return '';
|
||||
};
|
||||
const first = fetchWorkspaceOnce(folder, { git: firstRunner });
|
||||
await runStarted;
|
||||
// Two sessions on one folder reach it by different spellings; the scheduler keys by `resolve`.
|
||||
let joinerCalls = 0;
|
||||
const second = fetchWorkspaceOnce(join(dir, 'folder', '.'), {
|
||||
git: async () => {
|
||||
joinerCalls++;
|
||||
return '';
|
||||
},
|
||||
});
|
||||
releaseRun();
|
||||
expect(await second).toBe(await first);
|
||||
expect(joinerCalls).toBe(0);
|
||||
expect(fetches).toBe(1);
|
||||
});
|
||||
|
||||
it('releases the folder when a run lands, so the next click is a new run and not an old answer', async () => {
|
||||
const folder = join(dir, 'folder');
|
||||
initRepo(join(folder, 'api'));
|
||||
let fetches = 0;
|
||||
const runner: GitRunner = async (cwd, args) => {
|
||||
if (args[0] === 'fetch') fetches++;
|
||||
return '';
|
||||
};
|
||||
await fetchWorkspaceOnce(folder, { git: runner });
|
||||
expect(fetches).toBe(1);
|
||||
await fetchWorkspaceOnce(folder, { git: runner });
|
||||
expect(fetches).toBe(2);
|
||||
});
|
||||
|
||||
it('bounds a whole tick with the limiter it is handed, because two runs given a bound each would get twice the bound', async () => {
|
||||
const left = join(dir, 'left');
|
||||
const right = join(dir, 'right');
|
||||
for (let i = 0; i < FETCH_CONCURRENCY + 1; i++) {
|
||||
initRepo(join(left, `l${i}`));
|
||||
initRepo(join(right, `r${i}`));
|
||||
}
|
||||
const held = () => new Promise((resolve) => setTimeout(resolve, 10));
|
||||
// Only the fetches go through the limiter, so only they are counted here: the resolver's own probe is
|
||||
// not a bounded git process.
|
||||
const counting =
|
||||
(state: { live: number; peak: number }): GitRunner =>
|
||||
async (cwd, args) => {
|
||||
if (args[0] !== 'fetch') return '';
|
||||
state.live++;
|
||||
state.peak = Math.max(state.peak, state.live);
|
||||
await held();
|
||||
state.live--;
|
||||
return '';
|
||||
};
|
||||
|
||||
// The scheduler hands every folder in the tick the SAME limiter, so the bound is the tick's, not the run's.
|
||||
const shared = { live: 0, peak: 0 };
|
||||
const sharedRunner = counting(shared);
|
||||
const limiter = createLimiter(FETCH_CONCURRENCY);
|
||||
await Promise.all([
|
||||
fetchGitWorkspaceOverview(left, { git: sharedRunner, limiter }),
|
||||
fetchGitWorkspaceOverview(right, { git: sharedRunner, limiter }),
|
||||
]);
|
||||
expect(shared.peak).toBe(FETCH_CONCURRENCY);
|
||||
|
||||
// Without a shared limiter each run bounds only its own call, which is the failure the scheduler is there
|
||||
// to prevent: two folders at FETCH_CONCURRENCY each, so twice as many git processes as the rule says.
|
||||
const alone = { live: 0, peak: 0 };
|
||||
const aloneRunner = counting(alone);
|
||||
await Promise.all([
|
||||
fetchGitWorkspaceOverview(left, { git: aloneRunner }),
|
||||
fetchGitWorkspaceOverview(right, { git: aloneRunner }),
|
||||
]);
|
||||
expect(alone.peak).toBe(2 * FETCH_CONCURRENCY);
|
||||
});
|
||||
});
|
||||
@@ -11,6 +11,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { createRouteTestHarness } from './_route-test-utils.js';
|
||||
import { registerGitStatusRoutes } from '../../src/web/routes/git-status-routes.js';
|
||||
import { clearGitStatusCache, runGit, type GitRunner } from '../../src/git-workspace-status.js';
|
||||
import { clearGitFetchLane } from '../../src/git-fetch.js';
|
||||
|
||||
const ENV = {
|
||||
...process.env,
|
||||
@@ -44,6 +45,8 @@ async function setup(
|
||||
|
||||
beforeEach(() => {
|
||||
clearGitStatusCache();
|
||||
// The fetch lane is module state: a run left in it by the previous test would be joined by the next one.
|
||||
clearGitFetchLane();
|
||||
dir = mkdtempSync(join(tmpdir(), 'git-status-route-'));
|
||||
});
|
||||
afterEach(() => {
|
||||
@@ -467,6 +470,54 @@ describe('POST /api/sessions/:id/git-fetch', () => {
|
||||
expect(fetches()).toEqual([]);
|
||||
});
|
||||
|
||||
it("publishes what the click got as the session's own fetch state, naming the trigger the user asked for", async () => {
|
||||
git(dir, 'init', '-q', '-b', 'main');
|
||||
const { runner } = recordingRunner();
|
||||
const { app, ctx } = await setup({ git: runner });
|
||||
const res = await app.inject({ method: 'POST', url: '/api/sessions/test-session-1/git-fetch' });
|
||||
expect(res.statusCode).toBe(200);
|
||||
// The response body is one tab's copy of the run; the state is what every other tab open on the same
|
||||
// folder sees, and a number the user did not ask for has to read differently from one they did.
|
||||
expect(session.gitFetch).toMatchObject({ trigger: 'click', state: 'ok', repos: 1, failed: 0 });
|
||||
expect(ctx.persistSessionState).toHaveBeenCalled();
|
||||
expect(ctx.broadcast).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('joins a run already going for the same folder instead of starting a second git process in the same clone', async () => {
|
||||
git(dir, 'init', '-q', '-b', 'main');
|
||||
let releaseRun = () => {};
|
||||
const held = new Promise<void>((resolve) => (releaseRun = resolve));
|
||||
let running = () => {};
|
||||
const runStarted = new Promise<void>((resolve) => (running = resolve));
|
||||
const fetches: string[] = [];
|
||||
const runner: GitRunner = async (cwd, args) => {
|
||||
if (args[0] === 'fetch') {
|
||||
fetches.push(cwd);
|
||||
running();
|
||||
await held;
|
||||
}
|
||||
return execFileSyncGit(cwd, args);
|
||||
};
|
||||
const { app } = await setup({ git: runner });
|
||||
|
||||
const first = app.inject({ method: 'POST', url: '/api/sessions/test-session-1/git-fetch' });
|
||||
await runStarted; // the run is going, and still going
|
||||
// A click from another tab open on the same clone. Two fetches there would collide on git's ref lock, so
|
||||
// this one joins the run already going rather than starting its own.
|
||||
const second = app.inject({ method: 'POST', url: '/api/sessions/test-session-1/git-fetch' });
|
||||
// Assert the join BEFORE releasing the run: the lane entry goes away when the run finishes, so releasing
|
||||
// first would let the run complete before this request reached the lane, and it would then be entitled to
|
||||
// start its own fetch — a test that passes for the wrong reason. One tick of the loop is enough for both
|
||||
// handlers to reach the lane while the run is still held open.
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
expect(fetches).toHaveLength(1); // one run, and the second request is waiting on it, not working
|
||||
|
||||
releaseRun();
|
||||
const a = (await first).json().data;
|
||||
const b = (await second).json().data;
|
||||
expect(b).toEqual(a); // both tabs get the same run's answer, not a partial one
|
||||
});
|
||||
|
||||
it('404s an unknown session, running no fetch', async () => {
|
||||
const { calls, fetches, runner } = recordingRunner();
|
||||
const { app } = await setup({ git: runner });
|
||||
|
||||
@@ -88,29 +88,36 @@ describe('Session Options modal structure', () => {
|
||||
expect(modal.match(/<button[^>]*data-tab="context"[^>]*>/)?.[0]).not.toContain('data-claude-only');
|
||||
});
|
||||
|
||||
it('keeps the Git rail entry for every local case, and hides it for the ones nothing is ever fetched for', () => {
|
||||
it('keeps the Git rail entry for every local session, and hides it for the ones nothing is ever fetched for', () => {
|
||||
const modal = optionsModal();
|
||||
// A shell case has repositories too, so Git must not ride the Claude-only flag.
|
||||
// A shell session has repositories too, so Git must not ride the Claude-only flag.
|
||||
expect(modal.match(/<button[^>]*data-tab="git"[^>]*>/)?.[0]).not.toContain('data-claude-only');
|
||||
const open = methodBody('openSessionOptions(sessionId)');
|
||||
expect(open).toContain('.set-rail-item[data-tab="git"]');
|
||||
// Remote and Docker cases answer 'unsupported' on both git routes and the scheduler never fetches them,
|
||||
// Remote and Docker sessions answer 'unsupported' on both git routes and the scheduler never fetches them,
|
||||
// so the entry goes the same way the Claude-only ones do.
|
||||
expect(open).toMatch(/gitTabBtn\.style\.display = session\.remote \|\| session\.docker \? 'none' : ''/);
|
||||
});
|
||||
|
||||
it('makes the Git rows inert while a case follows the app, the way the keep-warm rows go inert', () => {
|
||||
it('makes the Git rows inert while a session follows the app, the way the keep-warm rows go inert', () => {
|
||||
const modal = optionsModal();
|
||||
const update = methodBody('updateGitFetchStatus(sessionId)');
|
||||
expect(update).toContain('set-row-disabled');
|
||||
expect(update).toContain('el.disabled = follows');
|
||||
// Decided from BOTH fields: a half-choice means the case is not following, and the half that still follows
|
||||
// shows the app's value while the other shows its own — a switch-only check would gray out a live row.
|
||||
// Decided from BOTH fields: a half-choice means the session is not following, and the half that still
|
||||
// follows shows the app's value while the other shows its own — a switch-only check would gray out a live row.
|
||||
expect(update).toMatch(
|
||||
/follows = session\.gitFetchEnabled === undefined && session\.gitFetchIntervalSeconds === undefined/
|
||||
);
|
||||
// Following saves null on both halves, which is what clears the override on the route.
|
||||
expect(methodBody('async autoSaveGitFetch()')).toContain('{ enabled: null, intervalSeconds: null }');
|
||||
const save = methodBody('async autoSaveGitFetch()');
|
||||
expect(save).toContain('{ enabled: null, intervalSeconds: null }');
|
||||
// Saves through the helper that says whether the call went through: `_apiPost` hands back the Response
|
||||
// without checking it, so a body the route refused (a 400) read as a save, and the panel showed a choice
|
||||
// the server never took.
|
||||
expect(save).toContain('this._apiJson(`/api/sessions/${this.editingSessionId}/git-fetch-settings`');
|
||||
expect(save).toContain('if (!data)');
|
||||
expect(save).not.toMatch(/await this\._apiPost\(/);
|
||||
// The status row is runtime text this method writes, so translating it would rewrite the stamp.
|
||||
expect(modal).toContain('id="gitFetchStatus" data-i18n-skip hidden');
|
||||
});
|
||||
|
||||
Reference in new issue
Block a user