serve's try took its lane slot and waited for the minute (04704111)
before holdWriter.watch started, so for up to 2 minutes (or QueueWait,
unbounded by default) the agent got no status, no headers, no byte.
keepQueued runs beside lanes.take and rpms.wait: past keepHeldAfter it
sends the stream's 200 and ": keepalive" comments every keepaliveEvery
(keepAlive, the mechanism held tries already use; none for Gemini, #934).
Once they are out the try is held, so the vendor's error, or the queue
turning it away, reaches the agent as its protocol's stream error
(failTo: Chat's error chunk, Responses' response.failed, Messages'
event: error), and the next candidate still answers in the same stream.
A wait shorter than keepHeldAfter sends nothing: its 429 keeps its
Retry-After. Non-streamed requests are unchanged.
What the agents do with it (read from their app bundles, not tried live):
- WorkBuddy 5.5.6 / WorkBuddy AI 5.6.2 (codebuddy-headless.js): no
header timeout, first-token timer 20 min, so a 2-minute queue alone
doesn't time it out there.
- Qoder 0.4.3: 60s to headers, then 60s more after them; its SSE parser
drops comment lines, so keepalives don't extend that. Headers at 15s
give it 75s instead of 60s; longer waits need firstPayloadTimeoutMs.
- Trae: its agent is downloaded at runtime and wasn't read.
Verified (Linux, golang:1.26.8 in Docker):
- queue_keepalive_test.go through lanGuard(New().Handler()), with an
agent whose connection gives up after 400ms of silence: a request
queued 1.2s for the minute (chat, responses, messages) and one queued
for a Concurrency slot get 200, keepalives, then the reply; a vendor
500 after keepalives is each protocol's stream error; a QueueWait
waited out is the stream's error naming the limit; a short wait is
still a plain 429. With keepQueued taken out, every one but the 429
test fails with "i/o timeout". TestQueuedUnheldTryHeldOnceKeptAlive
fails without the hold (": keepalive\n{\"error\":...}" in the stream).
- go test -tags nogui ./internal/gateway: ok (122s).
- go test -race on internal/gateway the way CI runs it (go test -race -c,
build/race-shards.sh over 14 processes, -test.timeout=20m): 1258 tests,
every shard ok. A single serial -race process passes 10m's default
timeout, which is why CI shards it (#923).
- go vet -tags nogui ./internal/gateway for linux, windows and darwin;
gofmt -l clean. After rebasing on 5e17a183: TestQueued, TestBurst,
RPM, TestQuiet and TestSilentHeld under -race, and linux vet, pass.
yetone
24bef0f9be