Codeman/docs/reliable-input-delivery.md

# Reliable input delivery (exactly-once, durable)

## The bug this fixes

With local echo on, pressing Enter cleared the overlay and then sent the prompt
over the WebSocket **fire-and-forget** (`ws.send({t:'i',d})`). On a flaky link
(e.g. a moving train) the socket is frequently *half-open*: `readyState === OPEN`
so `ws.send()` does **not** throw, but the underlying TCP is dead, so the frame is
silently discarded. Nothing was enqueued (the send "succeeded"), the on-screen
prompt was already wiped, and `navigator.onLine` stays `true` — so a long typed
prompt vanished with no trace and no resend.

## The guarantee

Every byte of user input is **recorded durably before delivery** and **only
dropped once the server ACKs it** — so a half-open socket, a reconnect, or a page
reload can never lose input. Redelivery is **exactly-once**: the server applies
each `(clientId, seq)` at most once, so a resend can't type the prompt twice.

## How it works

### Client (`app.js`)

- A stable **`clientId`** (`localStorage['codeman:clientId']`) identifies this
  browser to the server's dedup across reconnects and reloads.
- Each input frame gets a **monotonic per-session `seq`**. Frame records
  (`{seq,data,useMux,ts,tries,sentAt}`) live in `_pendingDeliveries`
  (`Map<sessionId, record[]>`), persisted (debounced, + flushed on `pagehide`/
  `visibilitychange`) to `localStorage['codeman:pendingInput']`. The seq counters
  persist too, so seqs stay monotonic across reloads (never reset — a reset would
  let the server treat fresh input as an already-applied duplicate).
- **Delivery** (`_drainSession`):
  - **WS path** — when the socket is `OPEN` for the session, send each not-yet-sent
    record (`sentAt === 0`) in seq order over the single ordered stream. Records
    stay pending until the server's `{t:'ia',seq}` ACK removes them.
  - **POST path** — when no WS, POST records in order, awaiting each (the HTTP 2xx
    *is* the ACK). A 404/410 (session gone) drops the record rather than retry
    forever.
- **Half-open recovery** (`_redeliverSweep`, every 2s): if the active WS session's
  oldest record is unacked past `_reliableAckTimeoutMs` (4s), the socket is assumed
  dead — `ws.close()` forces a fast reconnect; `onopen` (`_onWsReady`) resets
  `sentAt = 0` and re-sends everything pending. Also re-drains background sessions
  over POST, and fires on SSE-reconnect / `online`.
- The connection indicator shows pending count/bytes (`_pendingBytes`).

### Server

- **`Session.shouldApplyInput(clientId, seq)`** — returns `true` exactly once per
  `(clientId, seq)`: the first time a seq strictly greater than that client's
  last-applied is seen. A replayed/lower seq returns `false`. Bounded MRU map
  (`MAX_INPUT_DEDUP_CLIENTS = 256`).
- **WS route** (`ws-routes.ts`) — parses optional `cid`/`seq` on `{t:'i'}`; applies
  via `shouldApplyInput`. An applied frame is ACKed with `{t:'ia',seq}`; a duplicate is
  ACKed as `{t:'ia',seq,dup:true,last:<watermark>}`, where `last` is the server's
  highest applied seq for that `clientId` (`Session.lastInputSeq`). The client drops
  the record either way, and on `dup` it lifts its own counter to `last` first and
  re-sends a FIRST-attempt record (a retry being called a duplicate is the mechanism
  working: the original landed). Without `last`, a tab killed between a send and the
  persisted counter write came back counting BELOW the server's watermark, and every
  later keystroke was dropped-but-ACKed: a silently dead terminal a reload could not
  fix, since the stale counter was restored from localStorage too. The client now
  persists the counter synchronously on every send for the same reason. Untagged
  frames apply unconditionally (no behavior change).
- **POST route** (`/api/sessions/:id/input`) — optional `seq`/`clientId` in
  `SessionInputWithLimitSchema`; a deduped duplicate returns 200 without writing
  (the 200 is the client's ACK). `curl`/legacy callers omit the fields and always
  apply.

## Oversized input (issue #484)

Delivery has a third outcome besides "applied" and "retry": **refused for good**.
Both transports refuse a frame longer than `MAX_INPUT_LENGTH` (64 KiB,
`src/config/terminal-limits.ts`; the POST schema uses the same constant). Before
#484 the client treated that like a transient failure, so an oversized paste sat
at the head of the queue, was re-sent every 2 s forever, blocked every later
input for the session, and came back from localStorage on each reload.

- `_sendInputAsync()` splits a paste over the frame limit into in-limit frames
  (`CodemanInputLimit.split`, constants.js, never cutting a surrogate pair). They
  go out in seq order, so the PTY sees one contiguous stream. A paste over
  `PASTE_MAX_CHARS` (1 MiB), or an oversized `useMux` write (line-oriented, never
  split), is refused with a toast and never queued.
- The WebSocket answers an oversized sequenced frame with
  `{t:'ia', seq, err:'too_large', max}`; the client drops it with a toast. A
  client that predates `err` reads it as a plain ACK and drops it too.
- The POST drain drops a frame answered `400`/`413` (`401`/`403` stay transient:
  an expired login delivers once the user signs in again).
- `_loadReliableState()` prunes persisted frames over the limit, so a queue
  poisoned by an older build heals on the first load after upgrading.
- ⚠️ The frontend limit (`INPUT_FRAME_MAX_CHARS`) and the composer's
  `COMPOSER_INPUT_FRAME_LIMIT` must equal `MAX_INPUT_LENGTH`; pinned by
  `test/input-size-limit.test.ts`.

## Known limitation

Dedup state is in-memory on the server. A **server restart** between a write and
the client's redelivery of that same seq could re-apply it (a rare duplicate).
This is a deliberate trade-off: favor *never losing input* over a rare duplicate
across the narrow restart window.

## Tests

- `test/reliable-input-dedup.test.ts` — `Session.shouldApplyInput` exactly-once
  semantics (monotonic, per-client, gap-tolerant, eviction-safe).
- `test/routes/session-routes.test.ts` — POST `/input` applies a tagged
  `(clientId, seq)` once on redelivery; untagged input always applies.
- `test/input-size-limit.test.ts`: one input limit on both sides, frame
  splitting, and dropping (never retrying) a frame refused for good (#484).