Files
dsh-context/tests/host
Bowen Liang 635afceb83 feat(host): split the model generation window by decode block
The timing totals priced a model call as TTFT + one opaque generation
window, so thinking, answer text, and tool-argument decoding were
indistinguishable. Fold the stream's `block-start` framing into three new
additive-optional buckets — `reasoningMs`, `textMs`, `toolArgMs` — that
tile the window's marker span.

Both log generations feed the same split: a V0 log's `assistant/chunk`
markers open and close spans in the step slot; a V2+ log carries no chunk
events, so `decodeSpansOfStream` reads them off the embedded settlement
stream. The split is priced only when the window was (an unstamped call's
model time stays unattributed wholesale), and a zero span stays absent so
pre-split-shaped state gains no dead properties.

The fields are additive-optional, so `stateVersion` stays at 15 and
cached rows keep parsing — a bump would discard them and orphan the key
for idle sessions (the #37 regression).

Measured over 305 local sessions: reasoning 31.3% of active time, tool
args 15.3%, answer text 3.5% — the answer is 1/9th of what the model
spends decoding.
2026-09-10 02:21:45 +08:00
..