Bowen Liang
6819fa7fe6
The supported range (dsh 0.1.2-rc.1+) spans three durable-log generations, and the fold only read the V0 spellings — so on 0.1.5-alpha.1 sessions the system prompt priced 0, TTFT/generation stayed 0, and startSeq/endSeq replacements appended instead of replacing. Read SHAPES, never a detected version (the probe can be wrong: a healed profile mirror may name another release than the running harness): - system/message surface nodes feed systemTokens plus a bounded systems list (the harness's "last nonempty surviving system" rule), replacing the V0/V2 request/header.header.system envelope figure; the browser resolves the prompt in force per step and fetches its text from either event shape. - The first token comes from the embedded assistant stream (assistant/message.data.stream / assistant/attempt.data.stream) when the log carries no V0 assistant/chunk deltas, mirroring dsh-llm's assistantStreamFirstTokenTime. - Replacement endpoints accept both startSeq/endSeq and start/end. - tool/ptc-dispatch books nested file ops alongside tool/code-dispatch. Statistics were differentially verified against the harness's OWN projection values on real V0, V3-native, and V3-migrated logs: system/tools/message tokens, request and turn counts, TTFT, generation, tool time, and every billed cost bucket match exactly. The new state fields are additive-optional, so stateVersion stays 15: a bump would orphan the key for idle sessions (the #37 regression).
60 lines
2.8 KiB
TypeScript
60 lines
2.8 KiB
TypeScript
// The shared token heuristics (src/shared/estimate.ts): the fixed-density
|
|
// figures the host fold and the client boundary must agree on. The system
|
|
// prompt is priced from EITHER generation's payload — the V0/V2 envelope's
|
|
// rendered string or a V3 `system/message`'s content blocks — with the
|
|
// harness token-meter's own no-per-block-overhead rule.
|
|
|
|
import assert from 'node:assert/strict'
|
|
import { describe, test } from 'vitest'
|
|
import { estimateSystemContent, estimateSystemTokens } from '../../src/shared/estimate'
|
|
|
|
describe('estimateSystemTokens', () => {
|
|
test('prices text density plus role framing', () => {
|
|
assert.equal(estimateSystemTokens('abcdefgh'), 2 + 4)
|
|
assert.equal(estimateSystemTokens('abcde'), 2 + 4, 'partial tokens round up')
|
|
})
|
|
|
|
test('absent, empty, and non-string input prices 0', () => {
|
|
assert.equal(estimateSystemTokens(undefined), 0)
|
|
assert.equal(estimateSystemTokens(null), 0)
|
|
assert.equal(estimateSystemTokens(''), 0)
|
|
assert.equal(estimateSystemTokens(42), 0)
|
|
assert.equal(estimateSystemTokens({ text: 'x' }), 0)
|
|
})
|
|
})
|
|
|
|
describe('estimateSystemContent', () => {
|
|
test('prices every text block with no per-block overhead', () => {
|
|
assert.equal(estimateSystemContent([{ type: 'text', text: 'abcdefgh' }]), 2 + 4)
|
|
assert.equal(
|
|
estimateSystemContent([{ type: 'text', text: 'abcd' }, { type: 'text', text: 'efgh' }]),
|
|
2 + 4,
|
|
'blocks are summed before the density divide, exactly like the harness estimator',
|
|
)
|
|
})
|
|
|
|
test('empty content and non-array input price 0 (the harness reads "no system prompt")', () => {
|
|
assert.equal(estimateSystemContent([]), 0)
|
|
assert.equal(estimateSystemContent(undefined), 0)
|
|
assert.equal(estimateSystemContent('text'), 0)
|
|
})
|
|
|
|
test('a non-text block falls back to its JSON length', () => {
|
|
const block = { type: 'image', attachment: { width: 1, height: 1 } }
|
|
assert.equal(estimateSystemContent([block]), Math.ceil(JSON.stringify(block).length / 4) + 4)
|
|
})
|
|
|
|
test('hostile elements degrade instead of throwing', () => {
|
|
// A primitive element has no `text`; its JSON length is priced.
|
|
assert.equal(estimateSystemContent([7]), Math.ceil('7'.length / 4) + 4)
|
|
// `undefined` stringifies to undefined (not a string) and contributes nothing.
|
|
assert.equal(estimateSystemContent([undefined]), 0 + 4)
|
|
// A text-typed block whose text is not a string is priced by JSON, not trusted.
|
|
assert.equal(estimateSystemContent([{ type: 'text', text: 42 }]), Math.ceil('{"type":"text","text":42}'.length / 4) + 4)
|
|
// A cyclic block cannot be stringified: it contributes no characters.
|
|
const cyclic: Record<string, unknown> = { type: 'weird' }
|
|
cyclic.self = cyclic
|
|
assert.equal(estimateSystemContent([cyclic]), 4, 'role framing survives, content prices 0')
|
|
})
|
|
})
|