feat(local-model): types and module skeleton for the hardware-aware recommendation

The contract three builders code against in parallel. Nothing is recommended
yet: the route answers 501 and the catalogue is empty.

- src/types/local-model.ts: HardwareFacts, the catalogue rows, HarnessFact,
  Candidate, Recommendation, LocalModelStatus, and LOCAL_MODEL_LABELS, the one
  list of sentences the server may put into the payload (the zh-CN dictionary
  is keyed by them).
- src/local-model/format.ts: formatGb, the one place a byte count becomes a
  size a person reads (decimal GB), and memClassGb.
- src/config/local-model-catalog.ts: the Zod schema (model ids are name:tag,
  lowercase, no slash) and an empty stock catalogue. No override file.
- Skeletons with their final signatures: hardware.ts, recommend.ts,
  harness-facts.ts, routes/local-model-routes.ts (injectable dependencies),
  public/local-model-ui.js with its script tag.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Codeman maintainer committed 2026-11-06 14:56:13 +01:00
1 parent e9b2ecd552
commit 73ce6a1819
12 files changed
+824

No files matched your search

+42
View File
@@ -0,0 +1,42 @@
/**
* @fileoverview The formatters of the local model recommendation, and its byte units.
*
* `formatGb()` is the ONE place a byte count becomes a size a person reads: decimal GB (the
* unit the Finder and the model registry use). `hardware.ts` and `recommend.ts` call it to fill
* the `...Gb` text fields of the payload (`memGb`, `freeDiskGb`, `needGb`, `downloadGb`,
* `budgetGb`, and the size slots of a `LocalModelMessage`). Nothing else may turn bytes into a
* size for display, the browser included: it prints the text it was sent.
*
* GiB and MiB exist only for the budget math (reserves, the wired limit, the prompt cache) and
* never reach a person.
*
* Pure: no IO, no state.
*/
/** One decimal gigabyte, the unit a person reads. */
export const GB = 1_000_000_000;
/** One binary gigabyte. Budget math only. */
export const GIB = 1024 ** 3;
/** One binary megabyte. Budget math only. */
export const MIB = 1024 ** 2;
/**
* A byte count as decimal GB text, WITHOUT the unit: one decimal below 100 ('6.6', '17.7',
* '25.5'), a whole number from 100 ('212'). The caller's sentence supplies ' GB'.
* Anything that is not a positive finite number reads '0.0'.
*/
export function formatGb(bytes: number): string {
if (!Number.isFinite(bytes) || bytes <= 0) return '0.0';
const tenths = Math.round((bytes / GB) * 10) / 10;
return tenths >= 100 ? String(Math.round(bytes / GB)) : tenths.toFixed(1);
}
/**
* The memory size a Mac is sold as: whole binary gigabytes (17179869184 bytes is 16). This is
* `HardwareFacts.memClassGb`: the key of the tier rules ("below 36 GB") and of `TestedOn.memGb`.
* It names a class of Mac. A size for display is `formatGb()`.
*/
export function memClassGb(memBytes: number): number {
if (!Number.isFinite(memBytes) || memBytes <= 0) return 0;
return Math.round(memBytes / GIB);
}
+106
View File
@@ -0,0 +1,106 @@
/**
* @fileoverview The hardware probe of the local model recommendation: what this Mac is.
*
* SKELETON: the exported signatures are final, the bodies are not written yet. This module
* will hold:
* - `parseSysctl()`: a pure parser of `key: value` lines. The command runs WITHOUT `-n`, so a
* key that is missing on a machine (`sysctl.proc_translated` on Intel) cannot shift the
* others, and a line it does not know is ignored.
* - `readHardware()`: ONE bounded async spawn of `SYSCTL_PATH` with `SYSCTL_KEYS`, the free disk
* of the data dir through `fs.promises.statfs`, fallbacks from `node:os` when the spawn
* fails, and a 60 s cache.
*
* Rules:
* - Never throws and never rejects: a failed probe is an answer (`source: 'os-fallback'`), and
* off macOS nothing is spawned (`source: 'not-probed'`).
* - Never `system_profiler` (tens of milliseconds and a parse of free text), never a second
* spawn, never a synchronous exec on the server's event loop.
* - A non-zero exit is tolerated once `hw.memsize` parsed (a missing key makes sysctl exit 1).
* - `iogpu.wired_limit_mb` of 0 means "not set": it is reported as 0 and only a value above 0
* is ever used as a limit.
* - `memGb`, `memClassGb` and `freeDiskGb` are filled here through src/local-model/format.ts,
* the one place a size is formatted.
*/
import type { HardwareFacts } from '../types.js';
/** Absolute path, so the server's PATH (minimal under a service manager) cannot change what runs. */
export const SYSCTL_PATH = '/usr/sbin/sysctl';
/** The keys of the one spawn, in the order they are passed. */
export const SYSCTL_KEYS = [
'hw.memsize',
'machdep.cpu.brand_string',
'hw.optional.arm64',
'sysctl.proc_translated',
'iogpu.wired_limit_mb',
'kern.osproductversion',
'hw.model',
] as const;
/** Upper bound of the spawn (it answers in about 2 ms on a real Mac). */
export const SYSCTL_TIMEOUT_MS = 2000;
/** How long one probe result is reused. */
export const HARDWARE_CACHE_TTL_MS = 60_000;
/** What `parseSysctl()` read. Every field is absent when its key was missing or its value did not parse. */
export interface SysctlFacts {
/** `hw.memsize`. */
memBytes?: number;
/** `machdep.cpu.brand_string`. */
chip?: string;
/** `hw.optional.arm64` is 1. */
arm64?: boolean;
/** `sysctl.proc_translated` is 1. */
translated?: boolean;
/** `iogpu.wired_limit_mb`, as reported (0 stays 0). */
wiredLimitMb?: number;
/** `kern.osproductversion`. */
macosVersion?: string;
/** The leading number of `macosVersion`. */
macosMajor?: number;
/** `hw.model`. */
model?: string;
}
/** The IO of `readHardware()`, injectable so a test spawns nothing and reads no real disk. Each member defaults to the real thing. */
export interface HardwareProbeDeps {
/** Default `process.platform`. */
platform?: string;
/** Default `os.arch()`. */
arch?: string;
/** Runs the one spawn. Resolves its stdout, also after a non-zero exit, or `undefined` when it could not run or timed out. Never rejects. */
runSysctl?: () => Promise<string | undefined>;
/** Free bytes on the data dir's filesystem, or `undefined` when unknown. Never rejects. */
freeDiskBytes?: () => Promise<number | undefined>;
/** Default `os.totalmem()`. */
totalMemBytes?: () => number;
/** Default `os.cpus()[0]?.model`. */
cpuModel?: () => string | undefined;
/** Default `Date.now`. The clock of the cache. */
now?: () => number;
}
/**
* Parse the output of `sysctl <keys>` (no `-n`): one `key: value` line per key it knows.
* Pure. Unknown lines, blank lines and error lines are skipped.
*/
export function parseSysctl(text: string): SysctlFacts {
void text;
throw new Error('not implemented: parseSysctl');
}
/**
* What this machine is. Cached for `HARDWARE_CACHE_TTL_MS`; concurrent callers share one probe.
* Never rejects.
*/
export async function readHardware(deps?: HardwareProbeDeps): Promise<HardwareFacts> {
void deps;
throw new Error('not implemented: readHardware');
}
/** Drop the cached probe result (tests, and nothing else). */
export function resetHardwareCache(): void {
throw new Error('not implemented: resetHardwareCache');
}
+30
View File
@@ -0,0 +1,30 @@
/**
* @fileoverview What the server knows about each catalogue harness on this machine, for the
* local model recommendation. Server side: this is the one module of src/local-model/ that
* reads the CLI registry, so `recommend()` can stay pure and name no CLI.
*
* SKELETON: the exported signature is final, the body is not written yet. `buildHarnessFacts()`
* will, for every `LocalHarnessEntry` of the catalogue:
* - resolve its `cli` in the CLI registry AT CALL TIME (never in a module-level constant), and
* leave out a row whose CLI is unknown, disabled, or has no custom model mechanism
* (`capabilities.customModelInjection.kind` is `unsupported`);
* - report `label` from the registry entry and `installed` from the same probe the other
* routes use;
* - report `installable` only for a CLI that is not installed and whose install command for
* this platform needs no administrator rights (not one that starts with `brew` or `sudo`);
* - supply `acceptsModelId`, the same test a custom model launch applies to the model id
* before anything is written (the launch spec's model pattern).
*
* Rules:
* - No CLI id is compared with a literal here or anywhere: the id comes from the catalogue row
* and everything else from the registry entry it resolves to.
* - Read-only: nothing is installed, written or spawned beyond the existing availability probe.
*/
import type { HarnessFact, LocalModelCatalog } from '../types.js';
/** One fact per catalogue harness that resolves in the CLI registry, in catalogue order. */
export async function buildHarnessFacts(catalog: LocalModelCatalog): Promise<HarnessFact[]> {
void catalog;
throw new Error('not implemented: buildHarnessFacts');
}
+68
View File
@@ -0,0 +1,68 @@
/**
* @fileoverview The recommendation itself: which model, at which context, with which harness
* fits this Mac. Pure: no IO, no clock, no CLI registry, no data dir.
*
* SKELETON: the exported signatures are final, the bodies are not written yet. This module
* will hold:
* - `deviceBudget()`: L is the wired limit when it is above 0, else 2/3 of memory below 36 GB
* and 3/4 from 36 GB. B = min(L - 1 GiB, memory - reserve), with a reserve of 6 GiB up to
* 18 GB, 8 GiB up to 48 GB and 12 GiB from 64 GB.
* - `needBytes()`: F = weights + kvBytesPerToken * ctx + 1 GiB + the prompt cache tier of the
* catalogue for this Mac's memory.
* - `recommend()`: the candidates (model, context, harness) with `ctx >= harness.minContext`,
* the harness installed or installable, accepting the model id and not gated; the fit (F/B up
* to 0.85 good, up to 0.98 tight, above out); the speed (ok, slow or unknown: a chip that is
* in no chips row is unknown, a bandwidth is never guessed); the ranking (good before tight,
* then ok before unknown before slow, then model rank, harness rank, larger context); and the
* verdict.
*
* Rules:
* - Verdict `limited` when the best candidate's model has fewer than 12 billion parameters.
* - The word "Recommended" (`LOCAL_MODEL_LABELS.headlineTested`) only for a pick whose model
* has a `tested` entry matching this Mac. Otherwise `headlineOk` plus `noteUntested`.
* - Reasons for `none`: not-macos, intel, memory, disk (free disk minus the download is below
* 10 GiB for every fitting model), macos, no-harness.
* - Every label in the result comes from `LOCAL_MODEL_LABELS`, every size text from
* `formatGb()`. No number is spliced into a sentence here.
* - No CLI id is ever compared with a literal: a harness is a catalogue row plus a
* `HarnessFact`.
*/
import type {
DeviceBudget,
HardwareFacts,
HarnessFact,
LocalModelCatalog,
LocalModelEntry,
Recommendation,
} from '../types.js';
/** The memory a model may use on this Mac: the device limit L, the budget B and where L came from. */
export function deviceBudget(hw: HardwareFacts): DeviceBudget {
void hw;
throw new Error('not implemented: deviceBudget');
}
/** F: the bytes `model` needs in memory at a context of `ctx` tokens on this Mac. */
export function needBytes(model: LocalModelEntry, ctx: number, catalog: LocalModelCatalog, hw: HardwareFacts): number {
void model;
void ctx;
void catalog;
void hw;
throw new Error('not implemented: needBytes');
}
/**
* The recommendation for this Mac. `harnessFacts` holds one fact per catalogue harness the
* caller could resolve; a catalogue harness without a fact is not offered.
*/
export function recommend(
hw: HardwareFacts,
catalog: LocalModelCatalog,
harnessFacts: readonly HarnessFact[]
): Recommendation {
void hw;
void catalog;
void harnessFacts;
throw new Error('not implemented: recommend');
}