fix(agent): hold input during a guarded submit instead of refusing it

The input lease is held across the awaited tmux Enter, so WebSocket frames
keep arriving while it is held. A tagged frame (seq N) was refused and its
seq rolled back, but the browser's WebSocket path does not wait for ACKs:
seq N+1 landed after the lease and moved the watermark past N, and the
redelivered N was then dropped as a duplicate. The keystroke was lost.

Input that arrives during the lease is now held and written in arrival order
when the lease is released, whatever the submit's outcome. A WebSocket frame
is applied and ACKed only once written; managed writes (writeViaMux) and
named keys (send-key) wait for their turn the same way, and a held write that
is still being sent holds everything behind it. A keystroke held before the
guard runs still moves the input revision, so it still invalidates the Enter.
The hold is capped per lease (256 inputs, 1 MiB): past it the WebSocket is
closed with 1013 and nothing later from it is applied, so the browser resends
every unACKed frame in order, and send-key answers 409 CONFLICT instead of
logging a tmux failure as INTERNAL_ERROR. POST .../input keeps its 409 for the
lease plus the held input's drain; its browser drainer is serialized.

Docs: correct the api-reference paragraph that said these keystrokes are
redelivered, and describe the capture fallback and the per-scope cursor cap
from the previous two commits in api-reference, architecture-invariants and
the changeset.
This commit is contained in:
Aamer Akhter committed 2026-10-26 09:19:05 -04:00
1 parent ecaddb40ae
commit 2aad93914e
9 files changed
+727 -100

No files matched your search

+144 -17
View File
@@ -465,6 +465,17 @@ export interface ClaudeMessage {
duration_ms?: number;
}
/**
* A write that a guarded submit's lease could not hold, because the per-lease hold
* queue (`Session.MAX_GUARDED_HELD_INPUTS`) was full. Nothing was written; retry it.
*/
export class GuardedInputBusyError extends Error {
constructor() {
super('A guarded submit is in progress on this session; nothing was written. Retry the input.');
this.name = 'GuardedInputBusyError';
}
}
/**
* Whether a programmatic write leaves the keep-alive turn running: the nudge itself, or a bare
* Enter (which submits nothing new). Anything else carrying an Enter starts a turn of its own.
@@ -4218,7 +4229,7 @@ export class Session extends EventEmitter {
prompt: string,
options?: { model?: string; onProgress?: (info: { tokens?: number; status?: string }) => void }
): Promise<{ result: string; cost: number }> {
if (this._guardedSubmitInFlight) {
if (this.guardedInputHeld) {
this._guardedInputRevision++;
throw new Error('Guarded submission in progress');
}
@@ -4738,6 +4749,19 @@ export class Session extends EventEmitter {
private _guardedInputRevision = 0;
private _guardedPendingWrites = 0;
private _guardedSubmitInFlight = false;
/**
* Input that arrived while a guarded submit held the lease, in arrival order. Each
* entry runs when the lease is released; an entry that returns a promise (a mux
* write, a named key) is waited for before the next runs, so nothing overtakes it.
*/
private _guardedHeld: Array<{ run: () => void | Promise<unknown>; bytes: number }> = [];
private _guardedHeldBytes = 0;
/** True while the drain waits for a held asynchronous write to finish. */
private _guardedHeldDraining = false;
/** Held entries per lease. A lease lasts milliseconds; past this the caller backs off. */
static readonly MAX_GUARDED_HELD_INPUTS = 256;
/** Held bytes per lease (16 browser frames at the 64 KiB frame cap). */
static readonly MAX_GUARDED_HELD_BYTES = 16 * 64 * 1024;
/** Ephemeral identity, never persisted or based on a potentially reused pid alone. */
get inspectionGeneration(): string {
return `${this._inspectionInstance}:${this._guardedPaneRevision}:${this.owner ?? ''}`;
@@ -4750,13 +4774,89 @@ export class Session extends EventEmitter {
return this._guardedPendingWrites;
}
/**
* True while a guarded submit holds the input lease. Every write path refuses for
* that span (a few milliseconds), so a route that can report a refusal checks this
* first instead of answering as if the bytes went out.
* True while a guarded submit holds the input lease (a few milliseconds). Input that
* arrives in that span is held and written after it (see {@link guardedInputHeld}).
*/
get guardedSubmitInFlight(): boolean {
return this._guardedSubmitInFlight;
}
/**
* True while input is being held behind a guarded submit: the lease itself, or held
* input not yet written after it. A write that is not routed through
* {@link holdForGuardedLease} must refuse in this window, or it overtakes held input.
*/
get guardedInputHeld(): boolean {
return this._guardedSubmitInFlight || this._guardedHeld.length > 0 || this._guardedHeldDraining;
}
/**
* Hold input behind a guarded submit's lease instead of refusing it.
*
* `free`: nothing is held, so the caller writes inline as it always did. `held`: `run`
* is queued and runs, in arrival order with every other held input, once the lease is
* released (whatever the submit's outcome). `full`: the per-lease cap is reached and
* nothing was queued; the caller must stop delivering from that source until it can
* resend in order. Refusing instead lost input: the browser's WebSocket path does not
* wait for ACKs, so a refused seq N was overtaken by N+1 and its redelivery was then
* rejected as a duplicate. `data`, when given, is observed as input on arrival, so a
* keystroke held during the lease still moves the input revision at once.
*/
holdForGuardedLease(run: () => void | Promise<unknown>, bytes = 0, data?: string): 'free' | 'held' | 'full' {
if (!this.guardedInputHeld) return 'free';
if (data !== undefined) this._observeGuardedKeyboardInput(data);
if (
this._guardedHeld.length >= Session.MAX_GUARDED_HELD_INPUTS ||
this._guardedHeldBytes + bytes > Session.MAX_GUARDED_HELD_BYTES
)
return 'full';
this._guardedHeld.push({ run, bytes });
this._guardedHeldBytes += bytes;
return 'held';
}
/**
* Run an asynchronous write now, or in its turn behind a guarded submit's lease. The
* turn lasts until the write settles, so later held input waits for it. Resolves
* `whenFull` without running anything when the hold queue is full.
*/
private _inGuardedTurn<T>(write: () => Promise<T>, whenFull: () => T): Promise<T> {
if (!this.guardedInputHeld) return write();
return new Promise<T>((resolve, reject) => {
const held = this.holdForGuardedLease(() => {
const running = write();
running.then(resolve, reject);
return running;
});
if (held === 'full') {
try {
resolve(whenFull());
} catch (err) {
reject(err);
}
}
});
}
/** Run held input in arrival order until it is all written or a new lease is taken. */
private _drainGuardedHeld(): void {
while (this._guardedHeld.length > 0 && !this._guardedSubmitInFlight && !this._guardedHeldDraining) {
const entry = this._guardedHeld.shift()!;
this._guardedHeldBytes -= entry.bytes;
let running: void | Promise<unknown>;
try {
running = entry.run();
} catch (err) {
console.error(`[Session ${this.id}] held input failed after a guarded submit:`, getErrorMessage(err));
continue;
}
if (running && typeof running.then === 'function') {
this._guardedHeldDraining = true;
void running
.catch(() => {})
.then(() => {
this._guardedHeldDraining = false;
this._drainGuardedHeld();
});
}
}
}
/** Expose only the path selection needed for attributed Codex evidence, never credentials. */
getCodexHomeOverride(): string | undefined {
return this._envOverrides?.CODEX_HOME || undefined;
@@ -4773,7 +4873,7 @@ export class Session extends EventEmitter {
this._closing ||
this.paneLifecycleInFlight ||
this._guardedPendingWrites ||
(!allowLease && this._guardedSubmitInFlight) ||
(!allowLease && this.guardedInputHeld) ||
this.remote ||
this.docker ||
this.paneExit ||
@@ -4808,7 +4908,7 @@ export class Session extends EventEmitter {
/** Stage through ordinary delivery, after cancelling any older Enter retry timer. */
async stageGuardedInput(message: string): Promise<boolean> {
if (/[\r\n]/.test(message)) return false;
if (this._guardedPendingWrites || this._guardedSubmitInFlight) {
if (this._guardedPendingWrites || this.guardedInputHeld) {
this._guardedInputRevision++;
return false;
}
@@ -4839,18 +4939,29 @@ export class Session extends EventEmitter {
return sent && dispatched;
} finally {
this._guardedSubmitInFlight = false;
this._drainGuardedHeld();
}
}
/** Lease/pending boundary for named keys whose bytes bypass the ordinary write methods. */
/**
* Lease/pending boundary for named keys whose bytes bypass the ordinary write methods.
* During a guarded submit the operation waits for the lease like any other write;
* only a full hold queue refuses it, with {@link GuardedInputBusyError}.
*/
async performAuxiliaryInput(operation: () => Promise<void>): Promise<void> {
this._guardedInputRevision++;
if (this._guardedSubmitInFlight) throw new Error('Guarded submission in progress');
this._guardedPendingWrites++;
try {
await operation();
} finally {
this._guardedPendingWrites--;
}
return this._inGuardedTurn(
async () => {
this._guardedPendingWrites++;
try {
await operation();
} finally {
this._guardedPendingWrites--;
}
},
() => {
throw new GuardedInputBusyError();
}
);
}
/**
* Sends input directly to the PTY process.
@@ -5040,7 +5151,17 @@ export class Session extends EventEmitter {
*/
async writeViaMux(data: string, options: SessionWriteOptions = {}): Promise<boolean> {
this._observeGuardedKeyboardInput(data);
if (this._guardedSubmitInFlight) return false;
// A guarded submit's lease lasts for its Enter only: wait for it, in arrival order,
// rather than refusing, so an approval answer or a programmatic prompt is not
// dropped. Without a lease the write starts synchronously, so the pending count
// rises before any await and no lease can be taken while it is in flight.
return this._inGuardedTurn(
() => this._writeViaMuxNow(data, options),
() => false
);
}
private async _writeViaMuxNow(data: string, options: SessionWriteOptions): Promise<boolean> {
this._guardedPendingWrites++;
try {
const submittedPrompt = this._trackSubmit(data, options);
@@ -5090,7 +5211,13 @@ export class Session extends EventEmitter {
? null
: this._mux.capturePaneText?.(this._muxSession.muxName),
sendEnter: () => {
if (this._guardedSubmitInFlight) this._submitVerifier?.cancel();
// A retry Enter is never held behind a guarded submit: once that submit's
// Enter went out, a re-press would submit whatever is on the composer next.
if (this.guardedInputHeld) {
this._guardedInputRevision++;
this._submitVerifier?.cancel();
return Promise.resolve(false);
}
return this.writeViaMux('\r');
},
// ⚠ NO fallback glyph here, unlike the screen-reading probe elsewhere in this file.
@@ -5261,7 +5388,7 @@ export class Session extends EventEmitter {
// Legacy method for sending input - wraps runPrompt
async sendInput(input: string): Promise<void> {
if (this._guardedSubmitInFlight) {
if (this.guardedInputHeld) {
this._guardedInputRevision++;
throw new Error('Guarded submission in progress');
}
+15 -7
View File
@@ -35,6 +35,7 @@ import {
type RemoteHost,
} from '../../types.js';
import {
GuardedInputBusyError,
Session,
cliTakesSessionModel,
isAltScreenStripMode,
@@ -2078,12 +2079,14 @@ export function registerSessionRoutes(
}
// A guarded submit (agent-staging.ts) holds the input lease for the few
// milliseconds of its conditional Enter, and every write path refuses while it
// does. Say so instead of answering 200 for bytes that were never written, and
// give the seq back so the browser's durable queue (which keeps a frame on any
// 409) delivers it once the lease is released. Nothing from here to the write
// below awaits, so the lease cannot be taken in between.
if (!duplicate && session.guardedSubmitInFlight === true) {
// milliseconds of its conditional Enter, and input that arrives meanwhile is held
// and written in arrival order after it. This route refuses for that span (the
// lease plus the held input's drain) rather than overtake held input. Say so
// instead of answering 200 for bytes that were never written, and give the seq
// back so the browser's durable queue (which keeps a frame on any 409, and whose
// POST drainer sends one frame at a time) delivers it on its next retry. Nothing
// from here to the write below awaits, so the lease cannot be taken in between.
if (!duplicate && session.guardedInputHeld === true) {
if (tagged) session.forgetInputSeq(clientId as string, seq as number);
return reply
.code(409)
@@ -2416,7 +2419,7 @@ export function registerSessionRoutes(
// own `capabilities.newline`. Claude Code's Ink input recognizes 0x0a as "insert newline" vs
// 0x0d (carriage return = submit).
app.post('/api/sessions/:id/send-key', async (req) => {
app.post('/api/sessions/:id/send-key', async (req, reply) => {
const { id } = req.params as { id: string };
const body = req.body as Record<string, unknown>;
const key = typeof body?.key === 'string' ? body.key : '';
@@ -2461,6 +2464,11 @@ export function registerSessionRoutes(
})
);
} catch (err) {
// A guarded submit's lease holds the key until its Enter is out; only a full
// hold queue refuses it, and that is a retryable conflict, not a tmux failure.
if (err instanceof GuardedInputBusyError) {
return reply.code(409).send(createErrorResponse(ApiErrorCode.CONFLICT, err.message));
}
console.error('[Server] send-key failed:', err);
return createErrorResponse(ApiErrorCode.INTERNAL_ERROR, 'tmux send-keys failed');
}
+83 -45
View File
@@ -29,6 +29,11 @@
* {"t":"i","d":"...","seq":N,"cid":"..."} — input (keystroke or paste). seq+cid are
* optional reliable-delivery tags: the server applies each
* (cid,seq) at-most-once and ACKs with {"t":"ia","seq":N}.
* A frame that arrives while a guarded submit holds the
* session's input lease is held, then applied and ACKed in
* arrival order once it is released; when that hold queue
* is full the socket is closed with 1013 and nothing later
* from it is applied, so the client resends in order.
* {"t":"z","c":N,"r":N,"f":bool} — resize terminal (f=true forces SIGWINCH even if dims unchanged)
*/
@@ -169,12 +174,69 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost
const sizingToken = Symbol('ws-desktop-sizing');
let holdsDesktopClaim = false;
// Set once an input frame could not be held behind a guarded submit's lease
// (the hold queue was full). Nothing later from this socket is applied: it is
// closed, and the browser reconnects and resends everything unACKed, in order.
let inputStalled = false;
// Apply one input frame and ACK it. Runs inline, or later and in arrival order
// when a guarded submit held the input lease as the frame arrived.
const applyInputFrame = (msg: { d: string; seq?: unknown; cid?: unknown }) => {
// Reliable delivery: when the frame carries a clientId + seq, apply it
// exactly once (skip a duplicate redelivery) but ACK it regardless so
// the client can drop it from its durable queue. Frames without seq
// (legacy/other tools) are applied as-is — no behavior change.
const cid = typeof msg.cid === 'string' ? msg.cid : null;
const seq = Number.isInteger(msg.seq) ? (msg.seq as number) : null;
const apply = cid && seq !== null ? session.shouldApplyInput(cid, seq) : true;
let delivered = true;
if (apply) {
// Typed input from a claim-holding desktop keeps the claim "hot"
// and re-asserts the desktop layout after a mobile override.
if (holdsDesktopClaim) session.noteDesktopActivity();
// Browser keystrokes are the user's own, so they may name the tab.
delivered = session.write(msg.d, { fromUser: true });
// A session whose PTY is gone swallows the write. ACKing anyway told
// the client to drop the frame from its durable queue and left the seq
// burnt, so the retry that reliable delivery exists for was rejected as
// a duplicate: the input was lost for good.
if (!delivered && cid && seq !== null) session.forgetInputSeq(cid, seq);
}
if (seq !== null && socket.readyState === 1) {
if (apply) {
if (delivered) socket.send(`{"t":"ia","seq":${seq}}`);
} else {
// REJECTED as a duplicate. ACK it — the client must still drop it
// from its durable queue — but say so, and hand back our watermark.
//
// A plain ACK here is indistinguishable from "applied", which is
// what made a client with a rolled-back counter unrecoverable: its
// seqs persist to localStorage on a DEBOUNCED write, so a tab killed
// between a send and that write comes back with a counter BELOW this
// watermark, every later keystroke lands at or under it, and each one
// is dropped-but-ACKed. The UI stays clean, nothing is delivered, and
// a reload restores the same stale counter. `last` is what lets the
// client lift itself out.
// ⚠️ Defensive: the session arrives through a structural port, and an
// implementation without this method must not take the whole input
// path down with it — a throw here aborts the message handler and the
// frame is never ACKed at all, which strands it in the client's queue.
const watermark =
typeof (session as { lastInputSeq?: (c: string) => number }).lastInputSeq === 'function'
? (session as { lastInputSeq: (c: string) => number }).lastInputSeq(cid as string)
: seq;
socket.send(`{"t":"ia","seq":${seq},"dup":true,"last":${watermark}}`);
}
}
};
// Attach message handler synchronously BEFORE any async work
// (@fastify/websocket requirement to avoid dropped messages).
socket.on('message', (raw) => {
try {
const msg = JSON.parse(String(raw));
if (msg.t === 'i' && typeof msg.d === 'string') {
if (inputStalled) return;
if (msg.d.length > MAX_INPUT_LENGTH) {
// Refused for good, so say so: a silent return left the frame
// unACKed and the client redelivered it every few seconds forever
@@ -185,51 +247,27 @@ export function registerWsRoutes(app: FastifyInstance, ctx: SessionPort, getHost
}
return;
}
// Reliable delivery: when the frame carries a clientId + seq, apply it
// exactly once (skip a duplicate redelivery) but ACK it regardless so
// the client can drop it from its durable queue. Frames without seq
// (legacy/other tools) are applied as-is — no behavior change.
const cid = typeof msg.cid === 'string' ? msg.cid : null;
const seq = Number.isInteger(msg.seq) ? (msg.seq as number) : null;
const apply = cid && seq !== null ? session.shouldApplyInput(cid, seq) : true;
let delivered = true;
if (apply) {
// Typed input from a claim-holding desktop keeps the claim "hot"
// and re-asserts the desktop layout after a mobile override.
if (holdsDesktopClaim) session.noteDesktopActivity();
// Browser keystrokes are the user's own, so they may name the tab.
delivered = session.write(msg.d, { fromUser: true });
// A session whose PTY is gone swallows the write. ACKing anyway told
// the client to drop the frame from its durable queue and left the seq
// burnt, so the retry that reliable delivery exists for was rejected as
// a duplicate: the input was lost for good.
if (!delivered && cid && seq !== null) session.forgetInputSeq(cid, seq);
}
if (seq !== null && socket.readyState === 1) {
if (apply) {
if (delivered) socket.send(`{"t":"ia","seq":${seq}}`);
} else {
// REJECTED as a duplicate. ACK it — the client must still drop it
// from its durable queue — but say so, and hand back our watermark.
//
// A plain ACK here is indistinguishable from "applied", which is
// what made a client with a rolled-back counter unrecoverable: its
// seqs persist to localStorage on a DEBOUNCED write, so a tab killed
// between a send and that write comes back with a counter BELOW this
// watermark, every later keystroke lands at or under it, and each one
// is dropped-but-ACKed. The UI stays clean, nothing is delivered, and
// a reload restores the same stale counter. `last` is what lets the
// client lift itself out.
// ⚠️ Defensive: the session arrives through a structural port, and an
// implementation without this method must not take the whole input
// path down with it — a throw here aborts the message handler and the
// frame is never ACKed at all, which strands it in the client's queue.
const watermark =
typeof (session as { lastInputSeq?: (c: string) => number }).lastInputSeq === 'function'
? (session as { lastInputSeq: (c: string) => number }).lastInputSeq(cid as string)
: seq;
socket.send(`{"t":"ia","seq":${seq},"dup":true,"last":${watermark}}`);
}
// A guarded submit (agent-staging.ts) holds the input lease across its
// awaited Enter, so frames keep arriving while it is held. Refusing one
// lost it: this path does not wait for ACKs, so the next seq landed after
// the lease, moved the watermark past the refused one, and its redelivery
// was then dropped as a duplicate. Hold the frame instead; the session runs
// it, ACK included, in arrival order the moment the lease is released.
// ⚠️ Defensive, like `lastInputSeq` below: the session arrives through a
// structural port, and one without this method must still deliver input
// (a throw here would drop the frame unACKed).
const held =
typeof session.holdForGuardedLease === 'function'
? session.holdForGuardedLease(() => applyInputFrame(msg), msg.d.length, msg.d)
: 'free';
if (held === 'free') {
applyInputFrame(msg);
} else if (held === 'full') {
// Not held, not applied, not ACKed, and no later frame from this socket
// is applied either, so nothing can overtake it.
inputStalled = true;
console.warn('[ws] input backlog during a guarded submit; closing for redelivery', { sessionId: id });
socket.close(1013, 'Input backlog during guarded submit');
}
} else if (
msg.t === 'z' &&