catalog, usage, gateway, gui: a price can have a long-context tier and a 1-hour cache write, and the price boxes take decimals (PAMI on Discord)
OpenAI bills gpt-6-astra, gpt-6.1-sol and gpt-6-luna over 272K input at 2× input and cache and 1.5× output, for the whole request. A price now has tiers: `magpie model price <m> 10,50,1,12.5 --tier 272k 20,75,2,25` (--tier can repeat). A tier applies when input + cache reads + cache writes is over its size, so 272,000 is not over and 272,001 is. The built-in prices take tiers only where models.dev lists one (cost.tiers of type "context"), and it does list one for those three. The ledger, each request and OTLP are costed at the tier the call reached. Session and day totals sum many calls, so they stay at the base price.
Anthropic bills a cache write kept for an hour at 2× input and one kept for 5 minutes at 1.25×. Its usage reports which is which in cache_creation.ephemeral_5m/1h_input_tokens. The gateway (API and Claude subscription) and Claude Code's transcripts now keep that split. A price's fifth part is the 1-hour write. When it isn't given, it is 2× input: that is Anthropic's rule, and only Anthropic reports 1-hour writes. Claude list prices carry the 2× too. cache_write_tokens is still the total, and the usage CSV adds cache_write_5m_tokens and cache_write_1h_tokens at the end. The app shows the split on a request's Cache and on the usage tooltips. Old prices and old records, which have no 1-hour part, read as before. The sessions cache (18) and the call cache (calls-v8) are re-read once.
In the GUI, the price boxes were number inputs and dropped what they didn't parse ("0,25"). They are now decimal text boxes that take both "0.25" and "0,25". A model's price row has a 1-hour box (empty means 2× input) and a quiet Long-context price row. That row shows the list's tier greyed in, or sits behind a link when the list has none. A bad size is refused by name. en/zh/ja/de.
Tests:
- New: TestPriceTiers, TestCostSplit, TestPriceJSON, TestModelPriceTiers, TestModelPriceTierCmd, TestAnthropicCacheWriteSplit and TestClaudeCacheWriteSplit. The ledger CSV test expects the new columns.
- model-price-tiers.test.cjs is new; model-price-edit and provider-price-rate are updated. All of them fail against the old assets and pass in WebKit. Chromium couldn't be launched here.
- go vet, go build (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui ./... under a temp HOME pass.
This commit is contained in:
yetone committed
1 parent
74c67785c8
commit
71d2da1f42
56 files changed
+1167
-242
No files matched your search
+36
-1
@@ -168,6 +168,11 @@ deleted keys keep their historical identity. Older records appear as
|
||||
**key not recorded**, never inferred from today's configured key.
|
||||
These are upstream credentials, not keys clients use to call Magpie.
|
||||
|
||||
The CSV's `cache_write_tokens` is still every cache write; the last two
|
||||
columns, `cache_write_5m_tokens` and `cache_write_1h_tokens`, split it by how
|
||||
long Anthropic keeps it (a record from before the split is all 5-minute).
|
||||
The app shows the split on a request's Cache and the usage tiles' tooltips.
|
||||
|
||||
It lists **accounts** too: each Codex, Claude or other subscription account's
|
||||
tokens and cost, by the account that actually answered — the one that took
|
||||
over after a failover, the one `X-Magpie-Account` pinned. The account is named
|
||||
@@ -443,7 +448,37 @@ magpie model prices # every model you pric
|
||||
|
||||
The four numbers are USD per million tokens. All four are asked for, because
|
||||
a price missing one would understate the rest of every call; `0` is a model
|
||||
served at no cost, which is a price, not the absence of one.
|
||||
served at no cost, which is a price, not the absence of one. Decimals take a
|
||||
point or, in the app's boxes, a comma (`0,25`).
|
||||
|
||||
A **fifth number** is a 1-hour cache write's price. Anthropic bills a cache
|
||||
write kept for 5 minutes at 1.25× input and one kept for an hour at 2× input,
|
||||
and its usage says which were which (`cache_creation.ephemeral_5m_input_tokens`
|
||||
/ `ephemeral_1h_input_tokens`, from the API and from Claude Code's
|
||||
transcripts). The fourth number is the 5-minute price; the fifth, when not
|
||||
given, is 2× input — Anthropic's rule, and only Anthropic reports 1-hour
|
||||
writes, so for any other model it never comes into play. A call recorded
|
||||
before magpie kept the split counts all its writes at the 5-minute price, as
|
||||
it did.
|
||||
|
||||
A **long-context price** is what the whole request costs once its prompt is
|
||||
over a size, as OpenAI bills gpt-6-astra, gpt-6.1-sol and gpt-6-luna over
|
||||
272K input (2× input and cache, 1.5× output):
|
||||
|
||||
```sh
|
||||
magpie model price openai/gpt-6-astra 10,50,1,12.5 --tier 272k 20,75,2,25
|
||||
```
|
||||
|
||||
The size counts the prompt the way OpenAI does: uncached input + cache reads
|
||||
+ cache writes, and the tier applies when that is **over** the size (272,000
|
||||
is not, 272,001 is). Then every part of the call — input, output, cache — is
|
||||
at the tier's price, not only the tokens past the size. `--tier` may be given
|
||||
again for another size. The built-in prices carry a tier only where
|
||||
models.dev lists one (`cost.tiers` of type `context`), as it does for those
|
||||
three. Session and day totals sum many calls, so they are counted at the base
|
||||
price; the ledger and each request are counted at the tier they reached. In
|
||||
the app, a model's price row has a quiet **Long-context price** row under it,
|
||||
shown with the list's tier greyed in, or behind a link where there is none.
|
||||
|
||||
The order a price is looked for in is: **the price for this model → the price
|
||||
for `<provider id>/*`, which covers every model of that provider → what the
|
||||
|
||||
@@ -139,7 +139,7 @@ func entryPrice(st settings.Settings, find func(string) (provider.Group, []provi
|
||||
var first catalog.Price
|
||||
for i, m := range ms {
|
||||
pr, ok := provider.EffectivePriceIn(st, m.Provider.ID, m.Model)
|
||||
if !ok || m.Fast || i > 0 && pr != first {
|
||||
if !ok || m.Fast || i > 0 && !pr.Same(first) {
|
||||
return catalog.Price{}, false
|
||||
}
|
||||
first = pr
|
||||
|
||||
+138
-5
@@ -102,18 +102,150 @@ type Price struct {
|
||||
Output float64 `json:"output"`
|
||||
CacheRead float64 `json:"cache_read"`
|
||||
CacheWrite float64 `json:"cache_write"`
|
||||
// CacheWrite1h is what a cache write kept for an hour costs, Anthropic's
|
||||
// 1-hour TTL, which it bills at 2× input where a 5-minute write (the
|
||||
// CacheWrite above) is 1.25×. 0 is not given: such a write is then
|
||||
// counted at 2× input, Anthropic's rule for every model, as only
|
||||
// Anthropic's usage says which writes were for an hour.
|
||||
CacheWrite1h float64 `json:"cache_write_1h,omitempty"`
|
||||
// Tiers are the prices a request is billed at, whole, once its input
|
||||
// goes over a size (OpenAI's gpt-6-astra: 2× input and cache, 1.5×
|
||||
// output above 272K), lowest first. models.dev lists them as "tiers" of
|
||||
// type "context".
|
||||
Tiers []Tier `json:"tiers,omitempty"`
|
||||
}
|
||||
|
||||
// Tier is a price from a request's input size on: what the whole request
|
||||
// costs once its input, cached and cache-written tokens included, is over
|
||||
// Above tokens.
|
||||
type Tier struct {
|
||||
Above int `json:"above"`
|
||||
Input float64 `json:"input"`
|
||||
Output float64 `json:"output"`
|
||||
CacheRead float64 `json:"cache_read"`
|
||||
CacheWrite float64 `json:"cache_write"`
|
||||
CacheWrite1h float64 `json:"cache_write_1h,omitempty"`
|
||||
}
|
||||
|
||||
// UnmarshalJSON reads a price as magpie writes it, and as models.dev does,
|
||||
// whose tiers name their size as {"tier": {"type": "context", "size": N}}
|
||||
// and may leave a part out, which is then the base price's. A tier of
|
||||
// another type, or of no size, is left out.
|
||||
func (p *Price) UnmarshalJSON(b []byte) error {
|
||||
type flat Price
|
||||
var raw struct {
|
||||
flat
|
||||
Tiers []struct {
|
||||
Above int `json:"above"`
|
||||
Input *float64 `json:"input"`
|
||||
Output *float64 `json:"output"`
|
||||
CacheRead *float64 `json:"cache_read"`
|
||||
CacheWrite *float64 `json:"cache_write"`
|
||||
CacheWrite1h *float64 `json:"cache_write_1h"`
|
||||
Tier *struct {
|
||||
Type string `json:"type"`
|
||||
Size int `json:"size"`
|
||||
} `json:"tier"`
|
||||
} `json:"tiers"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &raw); err != nil {
|
||||
return err
|
||||
}
|
||||
*p = Price(raw.flat)
|
||||
p.Tiers = nil
|
||||
or := func(v *float64, base float64) float64 {
|
||||
if v != nil {
|
||||
return *v
|
||||
}
|
||||
return base
|
||||
}
|
||||
for _, t := range raw.Tiers {
|
||||
above := t.Above
|
||||
if t.Tier != nil {
|
||||
if t.Tier.Type != "" && t.Tier.Type != "context" {
|
||||
continue
|
||||
}
|
||||
above = t.Tier.Size
|
||||
}
|
||||
if above <= 0 {
|
||||
continue
|
||||
}
|
||||
p.Tiers = append(p.Tiers, Tier{Above: above, Input: or(t.Input, p.Input), Output: or(t.Output, p.Output),
|
||||
CacheRead: or(t.CacheRead, p.CacheRead), CacheWrite: or(t.CacheWrite, p.CacheWrite), CacheWrite1h: or(t.CacheWrite1h, 0)})
|
||||
}
|
||||
slices.SortFunc(p.Tiers, func(a, b Tier) int { return a.Above - b.Above })
|
||||
return nil
|
||||
}
|
||||
|
||||
// claudeOneHour gives a Claude model's list price its 1-hour cache write,
|
||||
// which models.dev doesn't list: 2× input, as Anthropic bills it for every
|
||||
// model, wherever the model has a (5-minute) cache write price at all.
|
||||
func claudeOneHour(id string, p *Price) {
|
||||
if p == nil || p.CacheWrite <= 0 || p.CacheWrite1h > 0 || !strings.Contains(strings.ToLower(id), "claude") {
|
||||
return
|
||||
}
|
||||
p.CacheWrite1h = 2 * p.Input
|
||||
for i := range p.Tiers {
|
||||
if p.Tiers[i].CacheWrite1h <= 0 {
|
||||
p.Tiers[i].CacheWrite1h = 2 * p.Tiers[i].Input
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Times is the price at r times each of its parts.
|
||||
func (p Price) Times(r float64) Price {
|
||||
return Price{Input: p.Input * r, Output: p.Output * r, CacheRead: p.CacheRead * r, CacheWrite: p.CacheWrite * r}
|
||||
out := Price{Input: p.Input * r, Output: p.Output * r, CacheRead: p.CacheRead * r, CacheWrite: p.CacheWrite * r, CacheWrite1h: p.CacheWrite1h * r}
|
||||
for _, t := range p.Tiers {
|
||||
out.Tiers = append(out.Tiers, Tier{Above: t.Above, Input: t.Input * r, Output: t.Output * r, CacheRead: t.CacheRead * r, CacheWrite: t.CacheWrite * r, CacheWrite1h: t.CacheWrite1h * r})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Cost of a call at this price. Reasoning tokens are billed as output by
|
||||
// every vendor, and are already inside the output count.
|
||||
// Same is whether two prices are the same in every part and tier.
|
||||
func (p Price) Same(q Price) bool {
|
||||
return p.Input == q.Input && p.Output == q.Output && p.CacheRead == q.CacheRead && p.CacheWrite == q.CacheWrite &&
|
||||
p.CacheWrite1h == q.CacheWrite1h && slices.Equal(p.Tiers, q.Tiers)
|
||||
}
|
||||
|
||||
// At is the flat price a request whose input, cached and cache-written
|
||||
// tokens included, comes to prompt tokens is billed at: that of the highest
|
||||
// tier it is over, else the base price. OpenAI counts a request's input
|
||||
// with its cached tokens in it, and bills the whole request at the tier.
|
||||
func (p Price) At(prompt int) Price {
|
||||
flat := Price{Input: p.Input, Output: p.Output, CacheRead: p.CacheRead, CacheWrite: p.CacheWrite, CacheWrite1h: p.CacheWrite1h}
|
||||
for _, t := range p.Tiers {
|
||||
if prompt > t.Above {
|
||||
flat = Price{Input: t.Input, Output: t.Output, CacheRead: t.CacheRead, CacheWrite: t.CacheWrite, CacheWrite1h: t.CacheWrite1h}
|
||||
}
|
||||
}
|
||||
return flat
|
||||
}
|
||||
|
||||
// OneHour is what a 1-hour cache write costs at this flat price: the price
|
||||
// given, else 2× input, Anthropic's rule.
|
||||
func (p Price) OneHour() float64 {
|
||||
if p.CacheWrite1h > 0 {
|
||||
return p.CacheWrite1h
|
||||
}
|
||||
return 2 * p.Input
|
||||
}
|
||||
|
||||
// Cost of a call at this price, every cache write a 5-minute one.
|
||||
// Reasoning tokens are billed as output by every vendor, and are already
|
||||
// inside the output count.
|
||||
func (p Price) Cost(input, output, cacheRead, cacheWrite int) float64 {
|
||||
return (float64(input)*p.Input + float64(output)*p.Output +
|
||||
float64(cacheRead)*p.CacheRead + float64(cacheWrite)*p.CacheWrite) / 1e6
|
||||
return p.CostSplit(input, output, cacheRead, cacheWrite, 0)
|
||||
}
|
||||
|
||||
// CostSplit is Cost where cacheWrite1h of the cacheWrite tokens were
|
||||
// written for an hour, billed at that price, and the rest for 5 minutes.
|
||||
// The whole call is billed at the tier its input reaches.
|
||||
func (p Price) CostSplit(input, output, cacheRead, cacheWrite, cacheWrite1h int) float64 {
|
||||
cacheWrite1h = min(max(cacheWrite1h, 0), max(cacheWrite, 0))
|
||||
f := p.At(input + cacheRead + cacheWrite)
|
||||
return (float64(input)*f.Input + float64(output)*f.Output +
|
||||
float64(cacheRead)*f.CacheRead + float64(cacheWrite-cacheWrite1h)*f.CacheWrite +
|
||||
float64(cacheWrite1h)*f.OneHour()) / 1e6
|
||||
}
|
||||
|
||||
type mdProvider struct {
|
||||
@@ -238,6 +370,7 @@ func load() map[string]mdProvider {
|
||||
voted := map[string]bool{}
|
||||
for pid, p := range m {
|
||||
for id, x := range p.Models {
|
||||
claudeOneHour(id, x.Cost)
|
||||
if e := x.efforts(); len(e) > 0 {
|
||||
l := strings.Join(e, ",")
|
||||
if levels[bareID(id)] == nil {
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
package catalog
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"math"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func near(a, b float64) bool { return math.Abs(a-b) < 1e-9 }
|
||||
|
||||
// PAMI on Discord: OpenAI bills gpt-6-astra over 272K input at 2× input and
|
||||
// cache, 1.5× output, the whole request; models.dev lists it as a tier of
|
||||
// type "context". The tier is read, the whole call billed at it once its
|
||||
// input with the cached tokens in it is over the size, and not at it
|
||||
// otherwise.
|
||||
func TestPriceTiers(t *testing.T) {
|
||||
writeCatalog(t, `{"openai":{"id":"openai","models":{"gpt-6-astra":{"id":"gpt-6-astra","cost":{
|
||||
"input":10,"output":50,"cache_read":1,"cache_write":12.5,
|
||||
"tiers":[{"input":20,"output":75,"cache_read":2,"cache_write":25,"tier":{"type":"context","size":272000}},
|
||||
{"input":99,"tier":{"type":"time","size":5}}]}}}},
|
||||
"anthropic":{"id":"anthropic","models":{"claude-opus-5-5":{"id":"claude-opus-5-5","cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}}}}}`)
|
||||
p, ok := PriceOf("openai", "gpt-6-astra")
|
||||
if !ok {
|
||||
t.Fatal("no price")
|
||||
}
|
||||
if want := []Tier{{Above: 272000, Input: 20, Output: 75, CacheRead: 2, CacheWrite: 25}}; len(p.Tiers) != 1 || p.Tiers[0] != want[0] {
|
||||
t.Fatalf("tiers = %+v, want %+v (a tier of another type left out)", p.Tiers, want)
|
||||
}
|
||||
// 200K input, 72K of it cached: exactly 272K is not over it
|
||||
if got, want := p.Cost(200_000, 1000, 72_000, 0), (200_000*10.0+1000*50+72_000*1)/1e6; !near(got, want) {
|
||||
t.Errorf("at 272K: %v, want the base %v", got, want)
|
||||
}
|
||||
// one more cached token: the whole call at the tier
|
||||
if got, want := p.Cost(200_000, 1000, 72_001, 0), (200_000*20.0+1000*75+72_001*2)/1e6; !near(got, want) {
|
||||
t.Errorf("over 272K: %v, want the tier's %v", got, want)
|
||||
}
|
||||
// cache writes count toward the size too
|
||||
if got, want := p.Cost(100, 10, 0, 300_000), (100*20.0+10*75+300_000*25)/1e6; !near(got, want) {
|
||||
t.Errorf("written over 272K: %v, want %v", got, want)
|
||||
}
|
||||
if b := p.At(0); len(b.Tiers) != 0 || b.Input != 10 {
|
||||
t.Errorf("At(0) = %+v, want the base price without tiers", b)
|
||||
}
|
||||
if r := p.Times(0.5); r.Tiers[0].Input != 10 || r.Tiers[0].Above != 272000 {
|
||||
t.Errorf("Times = %+v, the tier halved too", r)
|
||||
}
|
||||
|
||||
// a Claude list price is given its 1-hour cache write, 2× input
|
||||
c, _ := PriceOf("anthropic", "claude-opus-5-5")
|
||||
if c.CacheWrite1h != 10 {
|
||||
t.Errorf("claude 1h = %v, want 10", c.CacheWrite1h)
|
||||
}
|
||||
if p.CacheWrite1h != 0 {
|
||||
t.Errorf("gpt 1h = %v, want none", p.CacheWrite1h)
|
||||
}
|
||||
}
|
||||
|
||||
// Anthropic bills a 1-hour cache write at 2× input and a 5-minute one at
|
||||
// 1.25×: a call's 1-hour writes at theirs, the price's own else 2× input,
|
||||
// and a call from before the split was kept (none for 1 hour) as before.
|
||||
func TestCostSplit(t *testing.T) {
|
||||
p := Price{Input: 5, Output: 25, CacheRead: 0.5, CacheWrite: 6.25}
|
||||
if got, want := p.CostSplit(1000, 100, 0, 3000, 1000), (1000*5.0+100*25+2000*6.25+1000*10)/1e6; !near(got, want) {
|
||||
t.Errorf("unset 1h: %v, want %v", got, want)
|
||||
}
|
||||
p.CacheWrite1h = 8
|
||||
if got, want := p.CostSplit(1000, 100, 0, 3000, 1000), (1000*5.0+100*25+2000*6.25+1000*8)/1e6; !near(got, want) {
|
||||
t.Errorf("1h given: %v, want %v", got, want)
|
||||
}
|
||||
if got, want := p.CostSplit(1000, 100, 0, 3000, 0), p.Cost(1000, 100, 0, 3000); got != want {
|
||||
t.Errorf("no 1h: %v, want Cost's %v", got, want)
|
||||
}
|
||||
// more 1h than written is held to what was written
|
||||
if got, want := p.CostSplit(0, 0, 0, 10, 50), 10*8/1e6; !near(got, want) {
|
||||
t.Errorf("clamped: %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A price magpie saved before tiers and the 1-hour write reads as it did,
|
||||
// and one with them reads back the same.
|
||||
func TestPriceJSON(t *testing.T) {
|
||||
var old Price
|
||||
if err := json.Unmarshal([]byte(`{"input":1,"output":2,"cache_read":0.1,"cache_write":0}`), &old); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !old.Same(Price{Input: 1, Output: 2, CacheRead: 0.1}) {
|
||||
t.Errorf("old = %+v", old)
|
||||
}
|
||||
p := Price{Input: 10, Output: 50, CacheRead: 1, CacheWrite: 12.5, CacheWrite1h: 20, Tiers: []Tier{{Above: 272000, Input: 20, Output: 75, CacheRead: 2, CacheWrite: 25}}}
|
||||
b, _ := json.Marshal(p)
|
||||
var back Price
|
||||
if err := json.Unmarshal(b, &back); err != nil || !back.Same(p) {
|
||||
t.Errorf("round trip %s = %+v, %v", b, back, err)
|
||||
}
|
||||
// a part a tier leaves out is the base price's
|
||||
var part Price
|
||||
if err := json.Unmarshal([]byte(`{"input":1,"output":2,"cache_read":0.1,"cache_write":0.5,"tiers":[{"above":1000,"input":3}]}`), &part); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want := (Tier{Above: 1000, Input: 3, Output: 2, CacheRead: 0.1, CacheWrite: 0.5}); len(part.Tiers) != 1 || part.Tiers[0] != want {
|
||||
t.Errorf("part = %+v, want %+v", part.Tiers, want)
|
||||
}
|
||||
}
|
||||
@@ -603,14 +603,34 @@ type aUsage struct {
|
||||
OutputTokens int `json:"output_tokens"`
|
||||
CacheReadInputTokens int `json:"cache_read_input_tokens"`
|
||||
CacheCreationInputTokens int `json:"cache_creation_input_tokens"`
|
||||
// CacheCreation splits the cache writes by how long they are kept,
|
||||
// which Anthropic bills apart: 1.25× input for 5 minutes, 2× for an
|
||||
// hour
|
||||
CacheCreation *aCacheCreation `json:"cache_creation,omitempty"`
|
||||
}
|
||||
|
||||
type aCacheCreation struct {
|
||||
Ephemeral5m int `json:"ephemeral_5m_input_tokens"`
|
||||
Ephemeral1h int `json:"ephemeral_1h_input_tokens"`
|
||||
}
|
||||
|
||||
func (u aUsage) usage() Usage {
|
||||
return Usage{Input: u.InputTokens, Output: u.OutputTokens, CacheRead: u.CacheReadInputTokens, CacheWrite: u.CacheCreationInputTokens}
|
||||
out := Usage{Input: u.InputTokens, Output: u.OutputTokens, CacheRead: u.CacheReadInputTokens, CacheWrite: u.CacheCreationInputTokens}
|
||||
if c := u.CacheCreation; c != nil {
|
||||
out.CacheWrite = max(out.CacheWrite, c.Ephemeral5m+c.Ephemeral1h)
|
||||
out.CacheWrite1h = c.Ephemeral1h
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// anthropic is the usage as Anthropic says it, the cache writes split by
|
||||
// how long they are kept where some were for an hour.
|
||||
func (u Usage) anthropic() aUsage {
|
||||
return aUsage{InputTokens: u.Input, OutputTokens: u.Output, CacheReadInputTokens: u.CacheRead, CacheCreationInputTokens: u.CacheWrite}
|
||||
out := aUsage{InputTokens: u.Input, OutputTokens: u.Output, CacheReadInputTokens: u.CacheRead, CacheCreationInputTokens: u.CacheWrite}
|
||||
if h := min(u.CacheWrite1h, u.CacheWrite); h > 0 {
|
||||
out.CacheCreation = &aCacheCreation{Ephemeral5m: u.CacheWrite - h, Ephemeral1h: h}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func stopFromAnthropic(s string) string {
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// PAMI on Discord: Anthropic bills a 1-hour cache write at 2× input and a
|
||||
// 5-minute one at 1.25×, and its usage says which were which
|
||||
// (cache_creation's ephemeral_5m/1h_input_tokens). Both the API's usage and
|
||||
// the Claude subscription's carry the split, which goes back out to an
|
||||
// Anthropic client as it came; a usage without it reads as before.
|
||||
func TestAnthropicCacheWriteSplit(t *testing.T) {
|
||||
var a aUsage
|
||||
if err := json.Unmarshal([]byte(`{"input_tokens":10,"output_tokens":5,"cache_read_input_tokens":100,"cache_creation_input_tokens":3000,
|
||||
"cache_creation":{"ephemeral_5m_input_tokens":1000,"ephemeral_1h_input_tokens":2000}}`), &a); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
u := a.usage()
|
||||
if u.CacheWrite != 3000 || u.CacheWrite1h != 2000 {
|
||||
t.Fatalf("usage = %+v, want 3000 written, 2000 of them for an hour", u)
|
||||
}
|
||||
b, _ := json.Marshal(u.anthropic())
|
||||
if !strings.Contains(string(b), `"ephemeral_5m_input_tokens":1000`) || !strings.Contains(string(b), `"ephemeral_1h_input_tokens":2000`) {
|
||||
t.Errorf("back out as %s, want the split", b)
|
||||
}
|
||||
|
||||
var old aUsage
|
||||
if err := json.Unmarshal([]byte(`{"input_tokens":10,"output_tokens":5,"cache_creation_input_tokens":300}`), &old); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u := old.usage(); u.CacheWrite != 300 || u.CacheWrite1h != 0 {
|
||||
t.Errorf("no split = %+v", u)
|
||||
}
|
||||
if b, _ := json.Marshal(old.usage().anthropic()); strings.Contains(string(b), "cache_creation\"") {
|
||||
t.Errorf("no split back out as %s, want no cache_creation", b)
|
||||
}
|
||||
|
||||
var c cliUsage
|
||||
if err := json.Unmarshal([]byte(`{"input_tokens":1,"output_tokens":2,"cache_creation_input_tokens":0,
|
||||
"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":4000}}`), &c); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u := c.gateway(); u.CacheWrite != 4000 || u.CacheWrite1h != 4000 {
|
||||
t.Errorf("subscription usage = %+v, want 4000 written for an hour", u)
|
||||
}
|
||||
// a stream's usage counted again keeps the split
|
||||
if s := (Usage{CacheWrite: 10, CacheWrite1h: 4}).plus(Usage{CacheWrite: 5, CacheWrite1h: 5}, false); s.CacheWrite1h != 9 {
|
||||
t.Errorf("plus = %+v", s)
|
||||
}
|
||||
}
|
||||
@@ -1871,10 +1871,11 @@ func (r *subscriptionRun) readOutput(rd io.Reader) {
|
||||
var unstreamedQuiet = 300 * time.Millisecond
|
||||
|
||||
type cliUsage struct {
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
CacheCreation *aCacheCreation `json:"cache_creation"`
|
||||
OutputDetails struct {
|
||||
Thinking int `json:"thinking_tokens"`
|
||||
} `json:"output_tokens_details"`
|
||||
@@ -1916,7 +1917,7 @@ func claudeLimits(raw json.RawMessage) []provider.ClaudeLimit {
|
||||
}
|
||||
|
||||
func (u Usage) plus(v Usage, only bool) Usage {
|
||||
for _, f := range []struct{ a, b *int }{{&u.Input, &v.Input}, {&u.Output, &v.Output}, {&u.CacheRead, &v.CacheRead}, {&u.CacheWrite, &v.CacheWrite}, {&u.Reasoning, &v.Reasoning}} {
|
||||
for _, f := range []struct{ a, b *int }{{&u.Input, &v.Input}, {&u.Output, &v.Output}, {&u.CacheRead, &v.CacheRead}, {&u.CacheWrite, &v.CacheWrite}, {&u.CacheWrite1h, &v.CacheWrite1h}, {&u.Reasoning, &v.Reasoning}} {
|
||||
if *f.a > 0 || !only {
|
||||
*f.a += *f.b
|
||||
}
|
||||
@@ -1925,7 +1926,12 @@ func (u Usage) plus(v Usage, only bool) Usage {
|
||||
}
|
||||
|
||||
func (u cliUsage) gateway() Usage {
|
||||
return Usage{Input: u.Input, Output: u.Output, CacheRead: u.CacheRead, CacheWrite: u.CacheWrite, Reasoning: u.OutputDetails.Thinking}
|
||||
out := Usage{Input: u.Input, Output: u.Output, CacheRead: u.CacheRead, CacheWrite: u.CacheWrite, Reasoning: u.OutputDetails.Thinking}
|
||||
if c := u.CacheCreation; c != nil {
|
||||
out.CacheWrite = max(out.CacheWrite, c.Ephemeral5m+c.Ephemeral1h)
|
||||
out.CacheWrite1h = c.Ephemeral1h
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func renderClaudePrompt(req *Request) ([]map[string]any, error) {
|
||||
|
||||
@@ -1836,7 +1836,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
|
||||
ProviderKeyID: providerKeyID, ProviderKeyName: providerKeyName, ProviderAccount: providerAccount,
|
||||
Requested: call.Model, Served: call.Usage.Served, Upstream: call.Usage.Upstream,
|
||||
Input: call.Usage.Input, Output: call.Usage.Output, CacheRead: call.Usage.CacheRead,
|
||||
CacheWrite: call.Usage.CacheWrite, Reasoning: call.Usage.Reasoning, Effort: sent, Millis: time.Since(began).Milliseconds(), Status: call.Status,
|
||||
CacheWrite: call.Usage.CacheWrite, CacheWrite1h: call.Usage.CacheWrite1h, Reasoning: call.Usage.Reasoning, Effort: sent, Millis: time.Since(began).Milliseconds(), Status: call.Status,
|
||||
TTFT: try.TTFT, FirstText: try.FirstText, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind,
|
||||
RequestID: call.Usage.RequestID, Endpoint: endpointOf(r, from, call.To), Archive: call.archiveName()}
|
||||
failedWith(&rec, call.Status, call.Error, call.Usage.ErrType)
|
||||
@@ -1873,7 +1873,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
|
||||
ProviderKeyID: providerKeyID, ProviderKeyName: providerKeyName, ProviderAccount: providerAccount,
|
||||
Requested: call.Model, Served: call.Usage.Served,
|
||||
Input: call.Usage.Input, Output: call.Usage.Output, CacheRead: call.Usage.CacheRead,
|
||||
CacheWrite: call.Usage.CacheWrite, Reasoning: call.Usage.Reasoning, Effort: sent, Millis: time.Since(began).Milliseconds(), Status: call.Status,
|
||||
CacheWrite: call.Usage.CacheWrite, CacheWrite1h: call.Usage.CacheWrite1h, Reasoning: call.Usage.Reasoning, Effort: sent, Millis: time.Since(began).Milliseconds(), Status: call.Status,
|
||||
TTFT: try.TTFT, FirstText: try.FirstText, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind,
|
||||
RequestID: call.Usage.RequestID, Endpoint: endpointOf(r, from, call.To), Archive: call.archiveName()}
|
||||
failedWith(&rec, call.Status, call.Error, call.Usage.ErrType)
|
||||
@@ -2146,7 +2146,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
|
||||
ProviderKeyID: providerKeyID, ProviderKeyName: providerKeyName, ProviderAccount: providerAccount,
|
||||
Requested: call.Model, Served: call.Usage.Served, Upstream: call.Usage.Upstream,
|
||||
Input: call.Usage.Input, Output: call.Usage.Output, CacheRead: call.Usage.CacheRead,
|
||||
CacheWrite: call.Usage.CacheWrite, Reasoning: call.Usage.Reasoning, Effort: sent, Millis: call.Millis, Status: call.Status,
|
||||
CacheWrite: call.Usage.CacheWrite, CacheWrite1h: call.Usage.CacheWrite1h, Reasoning: call.Usage.Reasoning, Effort: sent, Millis: call.Millis, Status: call.Status,
|
||||
TTFT: call.TTFT, FirstText: call.FirstText, Sent: sentMs, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind,
|
||||
RequestID: call.Usage.RequestID, ResponseID: call.Usage.ResponseID, Endpoint: endpointOf(r, from, call.To), Archive: call.archiveName()}
|
||||
failedWith(&rec, call.Status, call.Error, call.Usage.ErrType)
|
||||
|
||||
@@ -201,7 +201,12 @@ type Usage struct {
|
||||
Output int `json:"output"`
|
||||
CacheRead int `json:"cache_read"`
|
||||
CacheWrite int `json:"cache_write"`
|
||||
Reasoning int `json:"reasoning"`
|
||||
// CacheWrite1h is how many of the CacheWrite tokens were written to
|
||||
// be kept for an hour, which Anthropic bills at 2× input where a
|
||||
// 5-minute write is 1.25×, when its usage says so (cache_creation's
|
||||
// ephemeral_1h_input_tokens)
|
||||
CacheWrite1h int `json:"cache_write_1h,omitempty"`
|
||||
Reasoning int `json:"reasoning"`
|
||||
// Served: the model the vendor's reply says answered, when it named
|
||||
// one — which may not be the one it was asked for
|
||||
Served string `json:"served,omitempty"`
|
||||
@@ -237,6 +242,9 @@ func (u *Usage) add(v Usage) {
|
||||
if v.CacheWrite > 0 {
|
||||
u.CacheWrite = v.CacheWrite
|
||||
}
|
||||
if v.CacheWrite1h > 0 {
|
||||
u.CacheWrite1h = v.CacheWrite1h
|
||||
}
|
||||
if v.Reasoning > 0 {
|
||||
u.Reasoning = v.Reasoning
|
||||
}
|
||||
|
||||
@@ -2343,6 +2343,9 @@ body.window .ag-sk-icon { width: 22px; height: 22px; border-radius: 6px; }
|
||||
.mname .mprice > span { flex: none; }
|
||||
.mname .mpart { display: flex; align-items: center; gap: 4px; flex: none; }
|
||||
.mname .mpart input { width: 72px; height: 25px; font-size: 11.5px; }
|
||||
.mname .mtier { display: flex; flex-wrap: wrap; align-items: center; gap: 4px 8px; flex: 1 1 100%; min-width: 0; }
|
||||
.mname .mtier[hidden], .mname .mtier-add[hidden] { display: none; }
|
||||
.mname .mtier-add { font-size: 11.5px; padding: 0; }
|
||||
.mname .mlevels { display: flex; flex-wrap: wrap; gap: 2px 10px; font-size: 11.5px; }
|
||||
.editor .mname label.tick { gap: 4px; }
|
||||
.mname .text { flex: none; padding: 2px 7px; font-size: 11.5px; }
|
||||
|
||||
+123
-30
@@ -6230,6 +6230,33 @@ function field(label, control, hint) {
|
||||
if (hint) wrap.append(el("div", "hint", hint));
|
||||
return [l, wrap];
|
||||
}
|
||||
// decimalOf is a number as typed in a price box, a comma taken for the
|
||||
// decimal point as some keyboards give it (0,25); NaN for anything else.
|
||||
// The boxes are text ones: a number box drops what its locale doesn't
|
||||
// take, and "0,25" became 25.
|
||||
function decimalOf(s) {
|
||||
s = String(s ?? "").trim();
|
||||
if (/^\d*,\d+$/.test(s)) s = s.replace(",", ".");
|
||||
return /^(\d+\.?\d*|\.\d+)$/.test(s) ? Number(s) : NaN;
|
||||
}
|
||||
// tokensTyped is a number of tokens as typed, 272K, 1M or 200000; NaN for
|
||||
// anything else
|
||||
function tokensTyped(s) {
|
||||
const x = /^(\d+(?:[.,]\d+)?)\s*([km]?)$/i.exec(String(s ?? "").trim());
|
||||
if (!x) return NaN;
|
||||
const n = Math.round(Number(x[1].replace(",", ".")) * ({ k: 1e3, m: 1e6 }[x[2].toLowerCase()] || 1));
|
||||
return n > 0 ? n : NaN;
|
||||
}
|
||||
function tokenSize(n) {
|
||||
return n >= 1e6 && n % 1e6 === 0 ? n / 1e6 + "M" : n >= 1e3 && n % 1e3 === 0 ? n / 1e3 + "K" : String(n);
|
||||
}
|
||||
// samePrice: two prices alike in every part, their 1-hour cache writes and
|
||||
// long-context prices included
|
||||
function samePrice(a, b) {
|
||||
const norm = (p) => JSON.stringify([p.input, p.output, p.cache_read || 0, p.cache_write || 0, p.cache_write_1h || 0,
|
||||
(p.tiers || []).map((x) => [x.above, x.input, x.output, x.cache_read || 0, x.cache_write || 0, x.cache_write_1h || 0])]);
|
||||
return norm(a) === norm(b);
|
||||
}
|
||||
function input(value, placeholder, type = "text") {
|
||||
const i = el("input");
|
||||
i.type = type;
|
||||
@@ -6337,11 +6364,8 @@ function concurrencyField(p) {
|
||||
// costs are its list price, else its maker's, times this. Empty is the
|
||||
// price as listed; steps of 0.001.
|
||||
function priceRateField() {
|
||||
const box = input(draft.priceRate ?? "", t("1, the official price"), "number");
|
||||
const box = input(draft.priceRate ?? "", t("1, the official price"));
|
||||
box.classList.add("price-rate");
|
||||
box.min = "0";
|
||||
box.max = "1000";
|
||||
box.step = "0.001";
|
||||
box.inputMode = "decimal";
|
||||
box.oninput = () => { draft.priceRate = box.value; };
|
||||
return field(t("Price rate"), box, t("Costs are counted at the official price times this, as a relay bills, like 0.8 or 1.5; a price you set for a model stays as set"));
|
||||
@@ -6351,8 +6375,8 @@ function priceRateField() {
|
||||
function priceRateOfDraft() {
|
||||
const v = String(draft.priceRate ?? "").trim();
|
||||
if (!v) return null;
|
||||
const n = Number(v);
|
||||
if (!/^\d*\.?\d+$|^\d+\.$/.test(v) || !(n >= 0 && n <= 1000) || Math.abs(n * 1000 - Math.round(n * 1000)) > 1e-6) return undefined;
|
||||
const n = decimalOf(v);
|
||||
if (!(n >= 0 && n <= 1000) || Math.abs(n * 1000 - Math.round(n * 1000)) > 1e-6) return undefined;
|
||||
return n;
|
||||
}
|
||||
function priceRateError(ed) {
|
||||
@@ -8564,60 +8588,122 @@ function renderModels(p) {
|
||||
// Usage page counts it (#819: only `magpie model price` set it):
|
||||
// each box shows its list price until a price is given, a part left
|
||||
// empty is the list's, and every part empty is its list price again
|
||||
const parts = [["input", "Input"], ["output", "Output"], ["cache_read", "Cache read"], ["cache_write", "Cache write"]];
|
||||
// the 1-hour cache write, empty, is 2× input as Anthropic bills it;
|
||||
// a long-context price (OpenAI's over 272K) is what the whole request
|
||||
// costs when its input is over its size, empty parts its list's
|
||||
const parts = [["input", "Input"], ["output", "Output"], ["cache_read", "Cache read"], ["cache_write", "Cache write"], ["cache_write_1h", "Cache write 1h"]];
|
||||
const tierParts = parts.slice(0, 4);
|
||||
const priceNow = () => prefs[id]?.ownPrice ? null : prefs[id]?.price ?? m.price ?? null;
|
||||
const shown = (n) => String(Math.round(n * 1e6) / 1e6);
|
||||
const listTier = m.list?.tiers?.[0];
|
||||
const priceBox = el("div", "mprice");
|
||||
priceBox.title = t("What {id} costs, in US dollars per million tokens, as the Usage page counts it; empty: its list price, shown greyed. A price set here isn't multiplied by the provider's price rate", { id: m.id });
|
||||
priceBox.append(el("span", "", t("Price, $ / 1M tokens")));
|
||||
const cells = parts.map(([k, l]) => {
|
||||
const box = el("label", "mpart");
|
||||
const typed = draft.priceTyped?.[id];
|
||||
const i = input(typed ? typed[parts.findIndex(([x]) => x === k)] : priceNow() ? shown(priceNow()[k]) : "", m.list ? shown(m.list[k]) : "", "number");
|
||||
const typed = draft.priceTyped?.[id];
|
||||
const cell = (value, placeholder, k, label) => {
|
||||
const i = input(value, placeholder);
|
||||
i.inputMode = "decimal";
|
||||
i.min = "0";
|
||||
i.step = "any";
|
||||
i.dataset.part = k;
|
||||
i.setAttribute("aria-label", t(l));
|
||||
i.setAttribute("aria-label", label);
|
||||
i.onkeydown = (e) => { e.stopPropagation(); if (e.key === "Enter") i.blur(); };
|
||||
i.onchange = () => takePrice();
|
||||
return i;
|
||||
};
|
||||
const oneHour = (pr) => pr ? pr.cache_write_1h || 2 * pr.input : 0;
|
||||
const cells = parts.map(([k, l], n) => {
|
||||
const box = el("label", "mpart");
|
||||
const pr = priceNow();
|
||||
const value = typed ? typed[n] : !pr ? "" : k === "cache_write_1h" ? pr.cache_write_1h ? shown(pr.cache_write_1h) : "" : shown(pr[k]);
|
||||
const placeholder = !m.list ? "" : k === "cache_write_1h" ? shown(oneHour(m.list)) : shown(m.list[k]);
|
||||
const i = cell(value, placeholder, k, t(l));
|
||||
if (k === "cache_write_1h") box.title = t("A cache write kept for an hour, as Anthropic bills it; empty: 2× input");
|
||||
box.append(el("span", "", t(l)), i);
|
||||
priceBox.append(box);
|
||||
return i;
|
||||
});
|
||||
const showPrice = () => { const pr = priceNow(); cells.forEach((c, i) => { c.value = pr ? shown(pr[parts[i][0]]) : ""; }); };
|
||||
// the long-context price, a quiet row of its own: shown where there is
|
||||
// one, else behind a link
|
||||
const tierRow = el("div", "mtier");
|
||||
tierRow.title = t("What the whole request costs when its input — cached tokens and cache writes included — is over this many tokens, as OpenAI bills gpt-6-astra over 272K; empty: the list's");
|
||||
const tierNow = () => priceNow()?.tiers?.[0];
|
||||
const above = cell(typed ? typed[parts.length] : tierNow() ? tokenSize(tierNow().above) : "", listTier ? tokenSize(listTier.above) : "272K", "above", t("Over input tokens"));
|
||||
above.classList.add("mabove");
|
||||
const overBox = el("label", "mpart");
|
||||
overBox.append(el("span", "", t("Long-context price, over")), above);
|
||||
tierRow.append(overBox);
|
||||
const tierCells = tierParts.map(([k, l], n) => {
|
||||
const box = el("label", "mpart");
|
||||
const i = cell(typed ? typed[parts.length + 1 + n] : tierNow() ? shown(tierNow()[k]) : "", listTier ? shown(listTier[k]) : "", "tier_" + k, t("Long-context price") + " · " + t(l));
|
||||
box.append(el("span", "", t(l)), i);
|
||||
tierRow.append(box);
|
||||
return i;
|
||||
});
|
||||
const moreTier = el("button", "text action mtier-add", t("Long-context price"));
|
||||
moreTier.title = tierRow.title;
|
||||
let tierOpen = false;
|
||||
moreTier.onclick = (e) => { e.preventDefault(); tierOpen = true; drawTier(); above.focus({ preventScroll: true }); };
|
||||
const drawTier = () => {
|
||||
tierRow.hidden = !tierOpen && !tierNow() && !listTier;
|
||||
moreTier.hidden = !tierRow.hidden;
|
||||
};
|
||||
priceBox.append(moreTier, tierRow);
|
||||
drawTier();
|
||||
const showPrice = () => {
|
||||
const pr = priceNow();
|
||||
cells.forEach((c, i) => { const k = parts[i][0]; c.value = !pr ? "" : k === "cache_write_1h" ? pr.cache_write_1h ? shown(pr.cache_write_1h) : "" : shown(pr[k]); });
|
||||
const tr = tierNow();
|
||||
above.value = tr ? tokenSize(tr.above) : "";
|
||||
tierCells.forEach((c, i) => { c.value = tr ? shown(tr[tierParts[i][0]]) : ""; });
|
||||
drawTier();
|
||||
};
|
||||
const takePrice = () => {
|
||||
const v = cells.map((c) => c.value.trim());
|
||||
const tv = [above, ...tierCells].map((c) => c.value.trim());
|
||||
const x = pref();
|
||||
delete x.price; delete x.ownPrice;
|
||||
if (draft.priceTyped) delete draft.priceTyped[id];
|
||||
if (v.every((s) => s === "")) {
|
||||
if (v.every((s) => s === "") && tv.every((s) => s === "")) {
|
||||
if (m.price) x.ownPrice = true;
|
||||
drawReset();
|
||||
return;
|
||||
}
|
||||
const bad = (msg) => { status(msg, "err"); showPrice(); drawReset(); };
|
||||
const price = {};
|
||||
for (const [i, [k]] of parts.entries()) {
|
||||
const n = v[i] === "" ? m.list?.[k] ?? (k.startsWith("cache") ? 0 : NaN) : Number(v[i]);
|
||||
for (const [i, [k]] of tierParts.entries()) {
|
||||
const n = v[i] === "" ? m.list?.[k] ?? (k.startsWith("cache") ? 0 : NaN) : decimalOf(v[i]);
|
||||
// a model with no list price is given its input and output one
|
||||
// box at a time: what is typed stays, waiting for the other, and
|
||||
// the Save asks for it (PAMI on Discord: each box typed was
|
||||
// emptied again, so a price could never be set)
|
||||
if (Number.isNaN(n) && v[i] === "" && v.every((s) => s === "" || (Number.isFinite(Number(s)) && Number(s) >= 0))) {
|
||||
(draft.priceTyped = draft.priceTyped || {})[id] = v;
|
||||
if (Number.isNaN(n) && v[i] === "" && v.every((s) => s === "" || decimalOf(s) >= 0)) {
|
||||
(draft.priceTyped = draft.priceTyped || {})[id] = [...v, ...tv];
|
||||
status(t("{id} has no list price: give its input and output prices", { id: m.id }), "warn");
|
||||
drawReset();
|
||||
return;
|
||||
}
|
||||
if (!Number.isFinite(n) || n < 0) {
|
||||
status(Number.isNaN(n) && v[i] === "" ? t("{id} has no list price: give its input and output prices", { id: m.id }) : t("A price is a number of dollars, 0 or more"), "err");
|
||||
showPrice();
|
||||
drawReset();
|
||||
return;
|
||||
}
|
||||
if (!Number.isFinite(n) || n < 0) return bad(Number.isNaN(n) && v[i] === "" ? t("{id} has no list price: give its input and output prices", { id: m.id }) : t("A price is a number of dollars, 0 or more"));
|
||||
price[k] = n;
|
||||
}
|
||||
if (!m.price || parts.some(([k]) => m.price[k] !== price[k])) x.price = price;
|
||||
if (v[4] !== "") {
|
||||
const n = decimalOf(v[4]);
|
||||
if (!(n >= 0)) return bad(t("A price is a number of dollars, 0 or more"));
|
||||
if (n > 0) price.cache_write_1h = n;
|
||||
}
|
||||
// the long-context price: its parts typed, else the list's, else the
|
||||
// price's own; none typed, the list's as it is
|
||||
const from = tierNow() || listTier;
|
||||
if (tv.some((s) => s !== "")) {
|
||||
const size = tv[0] === "" ? from?.above : tokensTyped(tv[0]);
|
||||
if (!(size > 0)) return bad(t("A long-context price starts over a number of input tokens, like 272K"));
|
||||
const tier = { above: size };
|
||||
for (const [i, [k]] of tierParts.entries()) {
|
||||
const n = tv[i + 1] === "" ? from?.[k] ?? price[k] : decimalOf(tv[i + 1]);
|
||||
if (!(n >= 0)) return bad(t("A price is a number of dollars, 0 or more"));
|
||||
tier[k] = n;
|
||||
}
|
||||
price.tiers = [tier, ...((priceNow() || m.list)?.tiers || []).slice(1).filter((x) => x.above > size)];
|
||||
} else if (m.list?.tiers?.length) price.tiers = m.list.tiers;
|
||||
if (!m.price || !samePrice(m.price, price)) x.price = price;
|
||||
showPrice();
|
||||
drawReset();
|
||||
};
|
||||
@@ -13323,7 +13409,7 @@ function renderUsage() {
|
||||
// written count is secondary and only fits in the tooltip.
|
||||
const promptTokens = u.input + u.cache_read + u.cache_write;
|
||||
const hit = u.cache_read && promptTokens ? t("hit rate {p}", { p: Math.round(100 * u.cache_read / promptTokens) + "%" }) : "";
|
||||
tile(fmtN(u.cache_read), t("cache read"), hit, u.cache_write ? t("{n} written", { n: fmtN(u.cache_write) }) : "");
|
||||
tile(fmtN(u.cache_read), t("cache read"), hit, u.cache_write ? t("{n} written", { n: fmtN(u.cache_write) }) + cacheTTLs(u, fmtN) : "");
|
||||
tile(fmtN(u.reasoning), t("reasoning"), t("inside output"));
|
||||
tile(String(u.calls), t(u.calls === 1 ? "call" : "calls"), u.errors ? t("{n} failed", { n: u.errors }) : "");
|
||||
|
||||
@@ -13552,7 +13638,7 @@ function ledDetail(r, cols) {
|
||||
};
|
||||
opt("Sent", r.model);
|
||||
opt("Effort", r.effort);
|
||||
if (r.cache_write || r.cache_read) opt("Cache", t("{w} written · {r} read", { w: ledNum(r.cache_write || 0), r: ledNum(r.cache_read || 0) }));
|
||||
if (r.cache_write || r.cache_read) opt("Cache", t("{w} written · {r} read", { w: ledNum(r.cache_write || 0), r: ledNum(r.cache_read || 0) }) + cacheTTLs(r, ledNum));
|
||||
if (ledRowSpeed(r)) add("Speed", t("{n} tok/s", { n: ledNum(Math.round(ledRowSpeed(r))) }));
|
||||
add("Request ID", r.rid);
|
||||
add("Endpoint", r.ep);
|
||||
@@ -13716,6 +13802,13 @@ const LED_TREND = [...LED_METRICS, ["speed", "Speed"]];
|
||||
const LED_SPLITS = [["model", "Model"], ["modelAt", "Model · provider"], ["provider", "Provider"], ["agent", "Agent"]];
|
||||
const LED_SHOWN = 7; // told apart in a chart; the rest are "Other"
|
||||
const allTokens = (x) => x.input + x.output + x.cache_read + x.cache_write;
|
||||
// cacheTTLs: a call's cache writes split by how long they are kept, where
|
||||
// some were for an hour (Anthropic bills those at 2× input, 5 minutes'
|
||||
// at 1.25×); nothing when none were
|
||||
function cacheTTLs(x, fmt) {
|
||||
const h = Math.min(x.cache_write_1h || 0, x.cache_write || 0);
|
||||
return h ? " · " + t("{m} for 5 minutes, {h} for an hour", { m: fmt(x.cache_write - h), h: fmt(h) }) : "";
|
||||
}
|
||||
// how fast the timed replies of a point, a share or a part wrote, in tokens
|
||||
// a second after their first (usage.Totals.Speed): 0 for none timed
|
||||
const ledSpeed = (x) => x?.decode_ms > 0 ? (1000 * x.decode_out) / x.decode_ms : 0;
|
||||
@@ -14317,7 +14410,7 @@ function renderLedger() {
|
||||
td(r.effort || "—", r.effort ? "" : "faint");
|
||||
td(ledNum(r.in), "n");
|
||||
td(ledNum(r.out), "n", r.reasoning ? t("{n} reasoning, inside output", { n: ledNum(r.reasoning) }) : "");
|
||||
td(ledNum(r.cache_write), "n" + (r.cache_write ? "" : " faint"));
|
||||
td(ledNum(r.cache_write), "n" + (r.cache_write ? "" : " faint"), cacheTTLs(r, ledNum).replace(/^ · /, ""));
|
||||
td(ledNum(r.cache_read), "n" + (r.cache_read ? "" : " faint"));
|
||||
const cost = td(r.priced ? "≈" + fmtCost({ cost: r.cost, unpriced: 0 }) : "—", "n cost" + (r.priced ? "" : " faint"), r.priced ? (r.pricing_model ? t("API price reference") + ": " + r.pricing_model : "") : t("No known price for this model"));
|
||||
if (r.priced && r.pricing_model) {
|
||||
|
||||
@@ -1747,6 +1747,14 @@ const I18N = {
|
||||
"In": "输入",
|
||||
"Out": "输出",
|
||||
"Cache write": "缓存写入",
|
||||
"Cache write 1h": "1 小时缓存写入",
|
||||
"A cache write kept for an hour, as Anthropic bills it; empty: 2× input": "保留 1 小时的缓存写入，按 Anthropic 的计费方式；留空：输入价的 2 倍",
|
||||
"What the whole request costs when its input — cached tokens and cache writes included — is over this many tokens, as OpenAI bills gpt-6-astra over 272K; empty: the list's": "请求的输入（含缓存读取和缓存写入的 token）超过这个数时，整个请求按这个价格计，就像 OpenAI 对 gpt-6-astra 超过 272K 的计费；留空：用列表价",
|
||||
"Over input tokens": "超过的输入 token 数",
|
||||
"Long-context price, over": "长上下文价格，超过",
|
||||
"Long-context price": "长上下文价格",
|
||||
"A long-context price starts over a number of input tokens, like 272K": "长上下文价格要写从多少输入 token 起算，比如 272K",
|
||||
"{m} for 5 minutes, {h} for an hour": "{m} 保留 5 分钟，{h} 保留 1 小时",
|
||||
"Cache read": "缓存读取",
|
||||
"Duration": "耗时",
|
||||
"Status": "状态",
|
||||
@@ -5112,6 +5120,14 @@ const I18N = {
|
||||
"In": "入力",
|
||||
"Out": "出力",
|
||||
"Cache write": "キャッシュ書き込み",
|
||||
"Cache write 1h": "1 時間キャッシュ書き込み",
|
||||
"A cache write kept for an hour, as Anthropic bills it; empty: 2× input": "1 時間保持されるキャッシュ書き込み（Anthropic の課金どおり）。空欄：入力の 2 倍",
|
||||
"What the whole request costs when its input — cached tokens and cache writes included — is over this many tokens, as OpenAI bills gpt-6-astra over 272K; empty: the list's": "入力（キャッシュ読み取り・書き込みのトークンを含む）がこのトークン数を超えたとき、リクエスト全体にかかる価格。OpenAI が gpt-6-astra の 272K 超に課金するのと同じです。空欄：一覧の価格",
|
||||
"Over input tokens": "超える入力トークン数",
|
||||
"Long-context price, over": "長コンテキスト価格、超過",
|
||||
"Long-context price": "長コンテキスト価格",
|
||||
"A long-context price starts over a number of input tokens, like 272K": "長コンテキスト価格には、何トークンを超えたら適用するかを書いてください（例：272K）",
|
||||
"{m} for 5 minutes, {h} for an hour": "{m} は 5 分間、{h} は 1 時間保持",
|
||||
"Cache read": "キャッシュ読み取り",
|
||||
"Duration": "所要時間",
|
||||
"Status": "ステータス",
|
||||
@@ -8457,6 +8473,14 @@ const I18N = {
|
||||
"In": "Ein",
|
||||
"Out": "Aus",
|
||||
"Cache write": "Cache-Schreiben",
|
||||
"Cache write 1h": "Cache-Schreiben 1 h",
|
||||
"A cache write kept for an hour, as Anthropic bills it; empty: 2× input": "Ein Cache-Schreiben, das eine Stunde gehalten wird, wie Anthropic es abrechnet; leer: 2× Eingabe",
|
||||
"What the whole request costs when its input — cached tokens and cache writes included — is over this many tokens, as OpenAI bills gpt-6-astra over 272K; empty: the list's": "Was die ganze Anfrage kostet, wenn ihre Eingabe – gecachte und in den Cache geschriebene Tokens eingeschlossen – über so vielen Tokens liegt, wie OpenAI gpt-6-astra über 272K abrechnet; leer: der Listenpreis",
|
||||
"Over input tokens": "Über Eingabe-Tokens",
|
||||
"Long-context price, over": "Preis für langen Kontext, über",
|
||||
"Long-context price": "Preis für langen Kontext",
|
||||
"A long-context price starts over a number of input tokens, like 272K": "Ein Preis für langen Kontext gilt ab einer Zahl von Eingabe-Tokens, etwa 272K",
|
||||
"{m} for 5 minutes, {h} for an hour": "{m} für 5 Minuten, {h} für eine Stunde",
|
||||
"Cache read": "Cache-Lesen",
|
||||
"Duration": "Dauer",
|
||||
"Status": "Status",
|
||||
|
||||
@@ -64,38 +64,38 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
|
||||
await page.locator(".row.provider").click();
|
||||
if (!(await page.locator(".mnames:not([hidden])").count())) await page.getByRole("button", { name: L.names, exact: true }).click();
|
||||
const row = (id) => page.locator(".mname", { has: page.locator("code", { hasText: id }) });
|
||||
const boxes = (id) => row(id).locator(".mprice input");
|
||||
const boxes = (id) => row(id).locator(".mprice > .mpart input");
|
||||
|
||||
// the list price shows in the boxes, a price set in them
|
||||
assert.equal((await row("model-1").locator(".mprice > span").textContent()), L.price);
|
||||
assert.deepEqual(await boxes("model-1").evaluateAll((is) => is.map((i) => [i.value, i.placeholder])), [["", "1.25"], ["", "10"], ["", "0.125"], ["", "0"]]);
|
||||
assert.deepEqual(await boxes("model-2").evaluateAll((is) => is.map((i) => i.value)), ["0.5", "2", "0.05", "0"]);
|
||||
assert.equal(await row("model-1").getByRole("spinbutton", { name: L.input, exact: true }).count(), 1);
|
||||
assert.deepEqual(await boxes("model-1").evaluateAll((is) => is.map((i) => [i.value, i.placeholder])), [["", "1.25"], ["", "10"], ["", "0.125"], ["", "0"], ["", "2.5"]]);
|
||||
assert.deepEqual(await boxes("model-2").evaluateAll((is) => is.map((i) => i.value)), ["0.5", "2", "0.05", "0", ""]);
|
||||
assert.equal(await row("model-1").getByRole("textbox", { name: L.input, exact: true }).count(), 1);
|
||||
assert.equal(await row("model-1").locator("select").count(), 0);
|
||||
|
||||
// a part typed, the others the list's; Restore default drops a price
|
||||
const y = await page.evaluate(() => scrollY);
|
||||
await row("model-1").getByRole("spinbutton", { name: L.output, exact: true }).fill("8");
|
||||
await row("model-1").getByRole("spinbutton", { name: L.output, exact: true }).press("Enter");
|
||||
await row("model-1").getByRole("textbox", { name: L.output, exact: true }).fill("8");
|
||||
await row("model-1").getByRole("textbox", { name: L.output, exact: true }).press("Enter");
|
||||
assert(await row("model-1").getByText(L.unsaved, { exact: true }).isVisible());
|
||||
assert.deepEqual(await boxes("model-1").evaluateAll((is) => is.map((i) => i.value)), ["1.25", "8", "0.125", "0"]);
|
||||
assert.deepEqual(await boxes("model-1").evaluateAll((is) => is.map((i) => i.value)), ["1.25", "8", "0.125", "0", ""]);
|
||||
await row("model-2").getByRole("button", { name: L.reset }).click();
|
||||
assert.deepEqual(await boxes("model-2").evaluateAll((is) => is.map((i) => i.value)), ["", "", "", ""]);
|
||||
assert.deepEqual(await boxes("model-2").evaluateAll((is) => is.map((i) => i.value)), ["", "", "", "", ""]);
|
||||
// no list price: input and output must be given, one box at a time —
|
||||
// the part typed stays (PAMI on Discord: it was emptied, so no price
|
||||
// could be set), and a Save before the other part asks for it
|
||||
await row("model-3").getByRole("spinbutton", { name: L.input, exact: true }).fill("3");
|
||||
await row("model-3").getByRole("spinbutton", { name: L.input, exact: true }).press("Enter");
|
||||
await row("model-3").getByRole("textbox", { name: L.input, exact: true }).fill("3");
|
||||
await row("model-3").getByRole("textbox", { name: L.input, exact: true }).press("Enter");
|
||||
await page.locator("#status", { hasText: L.nolist }).waitFor();
|
||||
assert.deepEqual(await boxes("model-3").evaluateAll((is) => is.map((i) => i.value)), ["3", "", "", ""]);
|
||||
assert.deepEqual(await boxes("model-3").evaluateAll((is) => is.map((i) => i.value)), ["3", "", "", "", ""]);
|
||||
assert(await row("model-3").getByText(L.unsaved, { exact: true }).isVisible());
|
||||
await page.getByRole("button", { name: L.save, exact: true }).click();
|
||||
await page.locator(".editor-error", { hasText: L.nolist }).waitFor();
|
||||
assert.equal(await page.evaluate(() => document.activeElement?.getAttribute("aria-label")), L.output);
|
||||
assert.deepEqual(posts, [], "a price half given isn't sent");
|
||||
await row("model-3").getByRole("spinbutton", { name: L.output, exact: true }).fill("15");
|
||||
await row("model-3").getByRole("spinbutton", { name: L.output, exact: true }).press("Enter");
|
||||
assert.deepEqual(await boxes("model-3").evaluateAll((is) => is.map((i) => i.value)), ["3", "15", "0", "0"]);
|
||||
await row("model-3").getByRole("textbox", { name: L.output, exact: true }).fill("15");
|
||||
await row("model-3").getByRole("textbox", { name: L.output, exact: true }).press("Enter");
|
||||
assert.deepEqual(await boxes("model-3").evaluateAll((is) => is.map((i) => i.value)), ["3", "15", "0", "0", ""]);
|
||||
assert.equal(await page.evaluate(() => scrollY), y, "nothing scrolled the page");
|
||||
assert.deepEqual(posts, [], "nothing is sent before the Save");
|
||||
if (process.env.ARTIFACT_DIR) await page.locator(".mnames").screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-model-price-rows.png`) });
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
// Run with Node's test runner and Playwright on the module path; see README.md.
|
||||
// PAMI on Discord: a price box took no decimals as typed (a number box
|
||||
// drops what its locale doesn't take, "0,25" became 25), OpenAI bills
|
||||
// gpt-6-astra over 272K input at another price, and Anthropic a 1-hour cache
|
||||
// write at 2× input. The boxes are text ones that take "0.25" and "0,25"; a
|
||||
// model's 1-hour cache write has a box of its own, empty for 2× input; and a
|
||||
// quiet Long-context price row, shown where the list has one and behind a
|
||||
// link where it hasn't, gives what the whole request costs over a size.
|
||||
// English and Chinese.
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const { test } = require("node:test");
|
||||
const { chromium, webkit } = require("playwright");
|
||||
|
||||
const assets = path.resolve(__dirname, "../assets");
|
||||
const astra = { input: 10, output: 50, cache_read: 1, cache_write: 12.5, tiers: [{ above: 272000, input: 20, output: 75, cache_read: 2, cache_write: 25 }] };
|
||||
const opus = { input: 5, output: 25, cache_read: 0.5, cache_write: 6.25, cache_write_1h: 10 };
|
||||
const plain = { input: 1, output: 4, cache_read: 0.1, cache_write: 0 };
|
||||
|
||||
function serve(lang, posts) {
|
||||
const state = { agents: [], profiles: [], settings: { lang, theme: "light" } };
|
||||
const models = [
|
||||
{ id: "gpt-6-astra", name: "GPT-6 Astra", on: true, efforts: [], images: false, list: astra },
|
||||
{ id: "claude-opus-5-5", name: "Claude Opus 5.5", on: true, efforts: [], images: false, list: opus },
|
||||
{ id: "plain-1", name: "Plain 1", on: true, efforts: [], images: false, list: plain },
|
||||
];
|
||||
const provider = { id: "relay", name: "Relay", icon: "generic", chat: "https://relay.example/v1", responses: "", anthropic: "", models, agents: [], key: { set: true, masked: "sk-…1234" }, ready: true };
|
||||
const providers = { providers: [provider], presets: [], excluded: [], gateway: { running: true, window: true } };
|
||||
return async (route) => {
|
||||
const url = new URL(route.request().url());
|
||||
const json = (data) => route.fulfill({ json: data });
|
||||
if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:${JSON.stringify(lang)},theme:"light",web:true};` });
|
||||
if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
|
||||
if (url.pathname === "/api/state") return json(state);
|
||||
if (url.pathname === "/api/providers") return json(providers);
|
||||
if (url.pathname === "/api/groups") return json({ groups: [] });
|
||||
if (url.pathname.startsWith("/api/provider/")) {
|
||||
posts.push({ path: url.pathname, body: route.request().postDataJSON() });
|
||||
return json(providers);
|
||||
}
|
||||
if (url.pathname.startsWith("/api/")) return json({});
|
||||
const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);
|
||||
await route.fulfill({ body: await fs.readFile(file), contentType: { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)] });
|
||||
};
|
||||
}
|
||||
|
||||
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
|
||||
for (const lang of ["en", "zh"]) {
|
||||
test(`${engine} ${lang}: decimals, a 1-hour cache write and a long-context price`, async (t) => {
|
||||
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: process.env.PLAYWRIGHT_CHANNEL || "chromium" }));
|
||||
const page = await (await browser.newContext({ viewport: { width: 900, height: 900 }, reducedMotion: "reduce" })).newPage();
|
||||
t.after(async () => {
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-model-price-tiers.png`) });
|
||||
}
|
||||
await browser.close();
|
||||
});
|
||||
page.setDefaultTimeout(5000);
|
||||
const errors = [], posts = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await page.route("**/*", serve(lang, posts));
|
||||
await page.goto("http://magpie.test/?view=providers");
|
||||
const zh = lang === "zh";
|
||||
const L = { names: zh ? "名称与推理档位" : "Names & levels", input: zh ? "输入" : "Input", output: zh ? "输出" : "Output", cacheRead: zh ? "缓存读取" : "Cache read",
|
||||
oneHour: zh ? "1 小时缓存写入" : "Cache write 1h", long: zh ? "长上下文价格" : "Long-context price", over: zh ? "超过的输入 token 数" : "Over input tokens",
|
||||
save: zh ? "保存" : "Save", badSize: zh ? "长上下文价格要写从多少输入 token 起算，比如 272K" : "A long-context price starts over a number of input tokens, like 272K" };
|
||||
await page.locator(".row.provider").click();
|
||||
if (!(await page.locator(".mnames:not([hidden])").count())) await page.getByRole("button", { name: L.names, exact: true }).click();
|
||||
const row = (id) => page.locator(".mname", { has: page.locator("code", { hasText: id }) });
|
||||
const base = (id) => row(id).locator(".mprice > .mpart input");
|
||||
const tier = (id) => row(id).locator(".mtier input");
|
||||
const typeIn = async (box, v) => { await box.fill(v); await box.press("Enter"); };
|
||||
const y = await page.evaluate(() => scrollY);
|
||||
|
||||
// a list with a long-context price shows it in its quiet row, greyed
|
||||
assert(await row("gpt-6-astra").locator(".mtier").isVisible());
|
||||
assert(!(await row("gpt-6-astra").locator(".mtier-add").isVisible()));
|
||||
assert.deepEqual(await tier("gpt-6-astra").evaluateAll((is) => is.map((i) => [i.value, i.placeholder])), [["", "272K"], ["", "20"], ["", "75"], ["", "2"], ["", "25"]]);
|
||||
// the 1-hour box shows the list's, else 2× its input
|
||||
assert.deepEqual(await base("claude-opus-5-5").evaluateAll((is) => is.map((i) => i.placeholder)), ["5", "25", "0.5", "6.25", "10"]);
|
||||
assert.deepEqual(await base("gpt-6-astra").evaluateAll((is) => is.at(-1).placeholder), "20");
|
||||
// where there is none, the row waits behind a link
|
||||
assert(!(await row("plain-1").locator(".mtier").isVisible()));
|
||||
assert(await row("plain-1").locator(".mtier-add").isVisible());
|
||||
assert.equal(await row("plain-1").locator("input[type=number]").count(), 0, "no number boxes");
|
||||
|
||||
// decimals, with a point and with a comma, each kept as typed
|
||||
await typeIn(row("gpt-6-astra").getByRole("textbox", { name: L.cacheRead, exact: true }), "0,25");
|
||||
await typeIn(row("gpt-6-astra").getByRole("textbox", { name: L.output, exact: true }), "1.5");
|
||||
assert.deepEqual(await base("gpt-6-astra").evaluateAll((is) => is.map((i) => i.value)), ["10", "1.5", "0.25", "12.5", ""]);
|
||||
// the 1-hour cache write, its own price
|
||||
await typeIn(row("claude-opus-5-5").getByRole("textbox", { name: L.oneHour, exact: true }), "7,5");
|
||||
assert.equal(await base("claude-opus-5-5").evaluateAll((is) => is.at(-1).value), "7.5");
|
||||
|
||||
// a long-context price given where the list has none: a size that
|
||||
// isn't one is refused, then 200k and its input
|
||||
await row("plain-1").locator(".mtier-add").click();
|
||||
assert(await row("plain-1").locator(".mtier").isVisible());
|
||||
assert.equal(await page.evaluate(() => document.activeElement?.getAttribute("aria-label")), L.over);
|
||||
await typeIn(row("plain-1").getByRole("textbox", { name: L.over, exact: true }), "lots");
|
||||
await page.locator("#status", { hasText: L.badSize }).waitFor();
|
||||
await row("plain-1").getByRole("textbox", { name: L.over, exact: true }).fill("200k");
|
||||
await typeIn(row("plain-1").getByRole("textbox", { name: L.long + " · " + L.input, exact: true }), "2,5");
|
||||
assert.deepEqual(await tier("plain-1").evaluateAll((is) => is.map((i) => i.value)), ["200K", "2.5", "4", "0.1", "0"]);
|
||||
assert.equal(await page.evaluate(() => scrollY), y, "nothing scrolled the page");
|
||||
assert.equal(await page.locator(".mname select").count(), 0);
|
||||
if (process.env.ARTIFACT_DIR) await page.locator(".mnames").screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-model-price-tier-rows.png`) });
|
||||
|
||||
await page.getByRole("button", { name: L.save, exact: true }).click();
|
||||
await page.waitForFunction(() => !document.querySelector(".mnames"));
|
||||
assert.deepEqual(posts.map((p) => p.path), ["/api/provider/save"]);
|
||||
assert.deepEqual(posts[0].body.modelPrefs, {
|
||||
// its list's long-context price goes with the price set
|
||||
"gpt-6-astra": { price: { input: 10, output: 1.5, cache_read: 0.25, cache_write: 12.5, tiers: astra.tiers } },
|
||||
"claude-opus-5-5": { price: { input: 5, output: 25, cache_read: 0.5, cache_write: 6.25, cache_write_1h: 7.5 } },
|
||||
"plain-1": { price: { input: 1, output: 4, cache_read: 0.1, cache_write: 0, tiers: [{ above: 200000, input: 2.5, output: 4, cache_read: 0.1, cache_write: 0 }] } },
|
||||
});
|
||||
const missing = await page.evaluate(() => ["Cache write 1h", "Long-context price", "Long-context price, over", "Over input tokens",
|
||||
"A long-context price starts over a number of input tokens, like 272K", "A cache write kept for an hour, as Anthropic bills it; empty: 2× input",
|
||||
"What the whole request costs when its input — cached tokens and cache writes included — is over this many tokens, as OpenAI bills gpt-6-astra over 272K; empty: the list's",
|
||||
"{m} for 5 minutes, {h} for an hour"].filter((k) => !I18N.zh[k] || !I18N.ja[k] || !I18N.de[k]));
|
||||
assert.deepEqual(missing, []);
|
||||
// no left-border accents on the new row
|
||||
assert.deepEqual(await page.evaluate(() => [...document.querySelectorAll(".mtier, .mtier *")].filter((e) => parseFloat(getComputedStyle(e).borderLeftWidth) > 0 && getComputedStyle(e).borderLeftColor !== getComputedStyle(e).borderTopColor).length), 0);
|
||||
assert.deepEqual(errors, []);
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -114,7 +114,8 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
|
||||
test(`${engine} ${lang}: a signed-in account's rate opens as it is and is saved`, async (t) => {
|
||||
const { page, errors, posts } = await open(t, "Codex");
|
||||
assert.equal(await box(page).inputValue(), "0.8");
|
||||
await box(page).fill("1.5");
|
||||
// a comma for the decimal point, as some keyboards type it
|
||||
await box(page).fill("1,5");
|
||||
const saved = await save(page, posts);
|
||||
assert.equal(saved.body.id, "codex");
|
||||
assert.equal(saved.body.priceRate, 1.5);
|
||||
|
||||
@@ -38,7 +38,7 @@ func TestPlanModels(t *testing.T) {
|
||||
t.Fatalf("free: %v", free)
|
||||
}
|
||||
// a free model costs nothing, not its paid model's price
|
||||
if pr, ok := p.ListPrice("cline-free/deepseek-v4.1-flash"); !ok || pr != (catalog.Price{}) {
|
||||
if pr, ok := p.ListPrice("cline-free/deepseek-v4.1-flash"); !ok || !pr.Same(catalog.Price{}) {
|
||||
t.Fatalf("free price: %+v %v", pr, ok)
|
||||
}
|
||||
// another provider's list is left as it is
|
||||
@@ -100,7 +100,7 @@ func TestClineFeed(t *testing.T) {
|
||||
// a free model the feed named and no id gives away costs nothing too,
|
||||
// a plan's model is not called free
|
||||
for _, id := range []string{"stealth/new-free-one", "stealth/space-bunny-alpha"} {
|
||||
if pr, ok := p.ListPrice(id); !ok || pr != (catalog.Price{}) {
|
||||
if pr, ok := p.ListPrice(id); !ok || !pr.Same(catalog.Price{}) {
|
||||
t.Fatalf("%s: %+v %v", id, pr, ok)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -81,7 +81,7 @@ func TestKiloModels(t *testing.T) {
|
||||
t.Fatalf("model: %+v", m)
|
||||
}
|
||||
// a free model by its flag alone costs nothing, not its maker's price
|
||||
if pr, ok := p.ListPrice("stealth/space-bunny-alpha"); !ok || pr != (catalog.Price{}) {
|
||||
if pr, ok := p.ListPrice("stealth/space-bunny-alpha"); !ok || !pr.Same(catalog.Price{}) {
|
||||
t.Fatalf("free price: %+v %v", pr, ok)
|
||||
}
|
||||
|
||||
|
||||
@@ -144,10 +144,7 @@ func SetModelPrice(id string, p *catalog.Price) error {
|
||||
key = pr.ID + "/" + model
|
||||
}
|
||||
s := settings.Load()
|
||||
m := settings.ModelPrice{
|
||||
Input: new(p.Input), Output: new(p.Output),
|
||||
CacheRead: new(p.CacheRead), CacheWrite: new(p.CacheWrite),
|
||||
}
|
||||
m := settings.StatedPrice(*p)
|
||||
if err := settings.CheckModelPrice(key, m); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ func TestSetModelPrefsPrice(t *testing.T) {
|
||||
if err := SetModelPrefs("a", map[string]ModelPref{"sol": {Price: &want}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, ok := EffectivePrice("a", "sol"); !ok || got != want {
|
||||
if got, ok := EffectivePrice("a", "sol"); !ok || !got.Same(want) {
|
||||
t.Fatalf("a/sol costs %+v (%v), want %+v", got, ok, want)
|
||||
}
|
||||
if _, ok := settings.Load().ModelPrices["b/sol"]; ok {
|
||||
|
||||
@@ -352,11 +352,11 @@ func TestSetModelPriceKeysByIdNotByTheNameUsedToAsk(t *testing.T) {
|
||||
if !ok {
|
||||
t.Fatalf("the price is not keyed by the provider's id either: %v", s.ModelPrices)
|
||||
}
|
||||
if p, bad := got.Price(); bad != "" || p != pr {
|
||||
if p, bad := got.Price(); bad != "" || !p.Same(pr) {
|
||||
t.Errorf("the price stored under the id reads back as %+v (bad part %q), want %+v", p, bad, pr)
|
||||
}
|
||||
// and it is the price the model is counted at, which was the point
|
||||
if eff, ok := EffectivePrice("relay", "sol"); !ok || eff != pr {
|
||||
if eff, ok := EffectivePrice("relay", "sol"); !ok || !eff.Same(pr) {
|
||||
t.Errorf("the model is counted at %+v, %v; want the price just set %+v", eff, ok, pr)
|
||||
}
|
||||
// taking it away by either name clears the one entry
|
||||
|
||||
@@ -93,10 +93,10 @@ func TestZCode(t *testing.T) {
|
||||
t.Fatalf("zcode: %+v", z)
|
||||
}
|
||||
// input without the cache, output with its reasoning already
|
||||
if m := model(z, "glm-5.1"); m.Tokens != (Tokens{300, 40, 1000, 200}) {
|
||||
if m := model(z, "glm-5.1"); m.Tokens != (Tokens{300, 40, 1000, 200, 0}) {
|
||||
t.Fatalf("glm-5.1: %+v", m)
|
||||
}
|
||||
if m := model(z, "glm-5.1-air"); m.Tokens != (Tokens{50, 5, 0, 0}) {
|
||||
if m := model(z, "glm-5.1-air"); m.Tokens != (Tokens{50, 5, 0, 0, 0}) {
|
||||
t.Fatalf("the subagent's: %+v", m)
|
||||
}
|
||||
if !z.Start.Equal(at("2026-09-27T08:00:00Z")) || !z.Last.Equal(at("2026-09-27T08:00:30Z")) {
|
||||
@@ -121,10 +121,10 @@ func checkDsh(t *testing.T, ss []Session) {
|
||||
t.Fatalf("dsh: %+v", a)
|
||||
}
|
||||
// the format 3 file, not the one left behind
|
||||
if m := model(a, "deepseek-v4-pro"); m.Tokens != (Tokens{100, 20, 1000, 0}) {
|
||||
if m := model(a, "deepseek-v4-pro"); m.Tokens != (Tokens{100, 20, 1000, 0, 0}) {
|
||||
t.Fatalf("pro: %+v", m)
|
||||
}
|
||||
if m := model(a, "deepseek-v4-flash"); m.Tokens != (Tokens{17, 8, 200, 0}) {
|
||||
if m := model(a, "deepseek-v4-flash"); m.Tokens != (Tokens{17, 8, 200, 0, 0}) {
|
||||
t.Fatalf("flash, the subagent's in: %+v", m)
|
||||
}
|
||||
if !a.Start.Equal(at("2026-09-27T10:00:00Z")) || !a.Last.Equal(at("2026-09-27T10:01:00Z")) {
|
||||
@@ -132,17 +132,17 @@ func checkDsh(t *testing.T, ss []Session) {
|
||||
}
|
||||
// the forks: what they were seeded with counted where it came from
|
||||
c := find(t, ss, "dsh", "dsh-c")
|
||||
if c.Tokens != (Tokens{50, 10, 0, 0}) || c.Title != "Tidy the build script" {
|
||||
if c.Tokens != (Tokens{50, 10, 0, 0, 0}) || c.Title != "Tidy the build script" {
|
||||
t.Fatalf("format 3 fork: %+v", c)
|
||||
}
|
||||
if !c.Start.Equal(at("2026-09-27T11:00:00Z")) || !c.Last.Equal(at("2026-09-27T11:00:20Z")) {
|
||||
t.Fatalf("fork times %s %s", c.Start, c.Last)
|
||||
}
|
||||
if d := find(t, ss, "dsh", "dsh-d"); d.Tokens != (Tokens{1, 2, 0, 0}) || !d.Start.Equal(at("2026-09-27T12:00:00Z")) {
|
||||
if d := find(t, ss, "dsh", "dsh-d"); d.Tokens != (Tokens{1, 2, 0, 0, 0}) || !d.Start.Equal(at("2026-09-27T12:00:00Z")) {
|
||||
t.Fatalf("format 0 fork: %+v", d)
|
||||
}
|
||||
s := statsAt(0, statsNow)
|
||||
if got, _ := usageOn(s, "2026-09-27", "/work/dsh"); got != (Tokens{168, 40, 1200, 0}) {
|
||||
if got, _ := usageOn(s, "2026-09-27", "/work/dsh"); got != (Tokens{168, 40, 1200, 0, 0}) {
|
||||
t.Fatalf("dsh on the 27th: %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -158,7 +158,7 @@ func TestDsh(t *testing.T) {
|
||||
b, _ := os.ReadFile(p)
|
||||
os.WriteFile(p, append(b, more...), 0o644)
|
||||
a := find(t, List(0), "dsh", "dsh-a")
|
||||
if m := model(a, "deepseek-v4-flash"); m.Tokens != (Tokens{18, 9, 200, 0}) || !a.Last.Equal(at("2026-09-27T10:01:30Z")) {
|
||||
if m := model(a, "deepseek-v4-flash"); m.Tokens != (Tokens{18, 9, 200, 0, 0}) || !a.Last.Equal(at("2026-09-27T10:01:30Z")) {
|
||||
t.Fatalf("after a new reply: %+v %s", m, a.Last)
|
||||
}
|
||||
os.WriteFile(p, b, 0o644)
|
||||
@@ -203,10 +203,10 @@ func TestCline(t *testing.T) {
|
||||
if c.Cwd != "/work/cline" || c.Title != "Speed up the importer" {
|
||||
t.Fatalf("cline: %+v", c)
|
||||
}
|
||||
if m := model(c, "claude-opus-5-5"); m.Tokens != (Tokens{100, 50, 1000, 500}) || !m.Priced {
|
||||
if m := model(c, "claude-opus-5-5"); m.Tokens != (Tokens{100, 50, 1000, 500, 0}) || !m.Priced {
|
||||
t.Fatalf("opus: %+v", m)
|
||||
}
|
||||
if m := model(c, "claude-haiku-4-5"); m.Tokens != (Tokens{130, 30, 100, 0}) {
|
||||
if m := model(c, "claude-haiku-4-5"); m.Tokens != (Tokens{130, 30, 100, 0, 0}) {
|
||||
t.Fatalf("haiku, the subagent's in: %+v", m)
|
||||
}
|
||||
if !c.Start.Equal(at("2026-09-27T10:00:00Z")) || !c.Last.Equal(at("2026-09-27T10:02:00Z")) {
|
||||
@@ -219,7 +219,7 @@ func TestCline(t *testing.T) {
|
||||
}
|
||||
// the fork: what it copied counted in the session it came from
|
||||
f := find(t, ss, "cline", clineFork)
|
||||
if f.Tokens != (Tokens{100, 30, 300, 0}) || f.Title != "Speed up the importer" {
|
||||
if f.Tokens != (Tokens{100, 30, 300, 0, 0}) || f.Title != "Speed up the importer" {
|
||||
t.Fatalf("fork: %+v", f)
|
||||
}
|
||||
if !f.Start.Equal(at("2026-09-27T12:00:00Z")) || !f.Last.Equal(at("2026-09-27T12:01:00Z")) {
|
||||
@@ -253,7 +253,7 @@ func TestQoder(t *testing.T) {
|
||||
if q.Cwd != "/work/q" || q.Title != "Explain the cache" {
|
||||
t.Fatalf("%s: %+v", c.agent, q)
|
||||
}
|
||||
if m := model(q, c.model); m.Tokens != (Tokens{45, 30, 300, 0}) {
|
||||
if m := model(q, c.model); m.Tokens != (Tokens{45, 30, 300, 0, 0}) {
|
||||
t.Fatalf("%s, the subagent's in: %+v", c.agent, m)
|
||||
}
|
||||
if runtime.GOOS != "windows" {
|
||||
|
||||
@@ -24,7 +24,8 @@ import (
|
||||
// v5: an OpenCode call keeps the effort its prompt asked for (#680).
|
||||
// v6: Codex response records and compact disk-only parser state.
|
||||
// v7: reconcile recent Claude message revisions.
|
||||
const callCacheVersion = "calls-v7"
|
||||
// v8: a Claude Code call's 1-hour cache writes.
|
||||
const callCacheVersion = "calls-v8"
|
||||
const maxKeptCalls = 131072
|
||||
const maxKeptFiles = 64
|
||||
|
||||
|
||||
@@ -392,10 +392,11 @@ type ccCall struct {
|
||||
ID string `json:"id"`
|
||||
Model string `json:"model"`
|
||||
Usage *struct {
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Creation *ccCacheCreation `json:"cache_creation"`
|
||||
} `json:"usage"`
|
||||
} `json:"message"`
|
||||
}
|
||||
@@ -516,7 +517,7 @@ func claudeCallLine(st *callFile, b []byte) {
|
||||
return
|
||||
}
|
||||
c.Model = st.str(model)
|
||||
c.Tokens = Tokens{Input: u.Input, Output: u.Output, CacheRead: u.CacheRead, CacheWrite: u.CacheWrite}
|
||||
c.Tokens = ccTokens(u.Input, u.Output, u.CacheRead, u.CacheWrite, u.Creation)
|
||||
if c.Tokens.zero() && m.Usage == nil {
|
||||
return
|
||||
}
|
||||
|
||||
@@ -34,7 +34,7 @@ func checkOCCalls(t *testing.T, cs map[string]Call) {
|
||||
t.Fatalf("no call for OpenCode's own reply: %+v", cs)
|
||||
}
|
||||
if a.Model != "gpt-6-astra" || a.Upstream != "openai" || a.Session != ocMain || a.Cwd != "/work/oc" ||
|
||||
a.Tokens != (Tokens{1000, 120, 5000, 0}) || a.Reasoning != 20 || a.Millis != 24000 ||
|
||||
a.Tokens != (Tokens{1000, 120, 5000, 0, 0}) || a.Reasoning != 20 || a.Millis != 24000 ||
|
||||
!a.Time.Equal(at("2026-09-26T10:00:30Z")) {
|
||||
t.Fatalf("astra: %+v", a)
|
||||
}
|
||||
@@ -44,7 +44,7 @@ func checkOCCalls(t *testing.T, cs map[string]Call) {
|
||||
}
|
||||
o, ok := cs["msg_0004"]
|
||||
if !ok || o.Model != "claude-opus-5-5" || o.Upstream != "anthropic" || o.Session != ocMain ||
|
||||
o.Tokens != (Tokens{40, 60, 700, 300}) || o.Millis != 30000 || o.Effort != "" {
|
||||
o.Tokens != (Tokens{40, 60, 700, 300, 0}) || o.Millis != 30000 || o.Effort != "" {
|
||||
t.Fatalf("the subagent's opus: %+v", o)
|
||||
}
|
||||
if len(cs) != 2 {
|
||||
@@ -118,11 +118,11 @@ func TestOpenCodeCallsV2(t *testing.T) {
|
||||
if _, ok := cs["msg_0005"]; ok {
|
||||
t.Fatalf("a compaction through magpie counted: %+v", cs["msg_0005"])
|
||||
}
|
||||
if c := cs["msg_0007"]; c.Model != "big-pickle" || c.Upstream != "opencode" || c.Tokens != (Tokens{7, 3, 0, 0}) || c.Effort != "xhigh" {
|
||||
if c := cs["msg_0007"]; c.Model != "big-pickle" || c.Upstream != "opencode" || c.Tokens != (Tokens{7, 3, 0, 0, 0}) || c.Effort != "xhigh" {
|
||||
t.Fatalf("an OpenCode Zen reply: %+v", c)
|
||||
}
|
||||
// a compaction is made with the model and effort last replied with
|
||||
if c := cs["msg_0008"]; c.Model != "big-pickle" || c.Upstream != "opencode" || c.Tokens != (Tokens{11, 2, 0, 0}) || c.Effort != "xhigh" {
|
||||
if c := cs["msg_0008"]; c.Model != "big-pickle" || c.Upstream != "opencode" || c.Tokens != (Tokens{11, 2, 0, 0, 0}) || c.Effort != "xhigh" {
|
||||
t.Fatalf("its compaction: %+v", c)
|
||||
}
|
||||
}
|
||||
@@ -107,12 +107,12 @@ func TestCallsClaude(t *testing.T) {
|
||||
|
||||
cs := Calls(time.Time{})
|
||||
want := []Call{
|
||||
{Time: callT0.Add(11 * time.Second), Agent: "claude", Session: "sess1", Model: "claude-haiku-4-5-20251001", Tokens: Tokens{11, 12, 13, 14}, RequestID: "req_s1", Cwd: "/work/app"},
|
||||
{Time: callT0.Add(10 * time.Second), Agent: "claude-desktop", Session: "sess-d", Model: "claude-sonnet-5-5", Tokens: Tokens{1, 2, 3, 4}, RequestID: "req_m4", Cwd: "/work/app", Millis: 1000},
|
||||
{Time: callT0.Add(11 * time.Second), Agent: "claude", Session: "sess1", Model: "claude-haiku-4-5-20251001", Tokens: Tokens{11, 12, 13, 14, 0}, RequestID: "req_s1", Cwd: "/work/app"},
|
||||
{Time: callT0.Add(10 * time.Second), Agent: "claude-desktop", Session: "sess-d", Model: "claude-sonnet-5-5", Tokens: Tokens{1, 2, 3, 4, 0}, RequestID: "req_m4", Cwd: "/work/app", Millis: 1000},
|
||||
{Time: callT0.Add(9 * time.Second), Agent: "claude", Session: "sess1", Error: "unknown", ErrorText: "API Error: " + strings.Repeat("é", 389), Cwd: "/work/app"},
|
||||
{Time: callT0.Add(8 * time.Second), Agent: "claude", Session: "sess1", Error: "rate_limit", ErrorText: "You've hit your session limit · resets 3:40am", RequestID: "req_err", Cwd: "/work/app"},
|
||||
{Time: callT0.Add(5 * time.Second), Agent: "claude", Session: "sess1", Model: "claude-opus-5-5", Tokens: Tokens{100, 55, 5000, 1000}, RequestID: "req_m1", Cwd: "/work/app", Millis: 5000},
|
||||
{Time: callT0.Add(4 * time.Second), Agent: "claude", Session: "sess1", Model: "claude-haiku-4-5-20251001", Tokens: Tokens{7, 8, 0, 0}, RequestID: "req_m2", Cwd: "/work/app", Millis: 1000},
|
||||
{Time: callT0.Add(5 * time.Second), Agent: "claude", Session: "sess1", Model: "claude-opus-5-5", Tokens: Tokens{100, 55, 5000, 1000, 0}, RequestID: "req_m1", Cwd: "/work/app", Millis: 5000},
|
||||
{Time: callT0.Add(4 * time.Second), Agent: "claude", Session: "sess1", Model: "claude-haiku-4-5-20251001", Tokens: Tokens{7, 8, 0, 0, 0}, RequestID: "req_m2", Cwd: "/work/app", Millis: 1000},
|
||||
}
|
||||
if len(cs) != len(want) {
|
||||
t.Fatalf("want %d calls, got %d: %+v", len(want), len(cs), cs)
|
||||
@@ -140,7 +140,7 @@ func TestCallsCowork(t *testing.T) {
|
||||
t.Fatalf("want 3 calls, got %d: %+v", len(cs), cs)
|
||||
}
|
||||
if cs[0].Agent != "claude-desktop" || cs[0].Model != "claude-haiku-4-5-20251001" || cs[0].Session != "cw1" ||
|
||||
cs[1].Agent != "claude-desktop" || cs[1].Tokens != (Tokens{5, 6, 7, 8}) ||
|
||||
cs[1].Agent != "claude-desktop" || cs[1].Tokens != (Tokens{5, 6, 7, 8, 0}) ||
|
||||
cs[2].Agent != "claude" {
|
||||
t.Fatalf("calls: %+v", cs)
|
||||
}
|
||||
@@ -211,13 +211,13 @@ func TestCallsCodex(t *testing.T) {
|
||||
|
||||
cs := Calls(time.Time{})
|
||||
want := []Call{
|
||||
{Time: callT0.Add(200 * time.Second), Agent: "codex", Session: "0190cccc-1111-7222-8333-444455556666", Tokens: Tokens{6, 3, 4, 0}},
|
||||
{Time: callT0.Add(103 * time.Second), Agent: "codex", Session: "thread-2", Model: "gpt-6-astra", Tokens: Tokens{500, 20, 0, 0}, Reasoning: 5, Effort: "medium", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(102 * time.Second), Agent: "codex", Session: "thread-2", Model: "gpt-6-astra", Tokens: Tokens{100, 100, 3900, 0}, Reasoning: 30, Effort: "medium", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(200 * time.Second), Agent: "codex", Session: "0190cccc-1111-7222-8333-444455556666", Tokens: Tokens{6, 3, 4, 0, 0}},
|
||||
{Time: callT0.Add(103 * time.Second), Agent: "codex", Session: "thread-2", Model: "gpt-6-astra", Tokens: Tokens{500, 20, 0, 0, 0}, Reasoning: 5, Effort: "medium", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(102 * time.Second), Agent: "codex", Session: "thread-2", Model: "gpt-6-astra", Tokens: Tokens{100, 100, 3900, 0, 0}, Reasoning: 30, Effort: "medium", Cwd: "/work/it", Millis: 1000},
|
||||
// input_tokens holds the cache write as well as the read (#589): 2500 - 2000 - 300
|
||||
{Time: callT0.Add(12 * time.Second), Agent: "codex", Session: "sess-x", Model: "gpt-6-luna", Tokens: Tokens{200, 150, 2000, 300}, Reasoning: 40, Effort: "low", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(11 * time.Second), Agent: "codex", Session: "sess-x", Model: "gpt-6-luna", Tokens: Tokens{1000, 100, 1000, 0}, Reasoning: 40, Effort: "low", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(2 * time.Second), Agent: "codex", Session: "sess-x", Model: "gpt-6-astra", Tokens: Tokens{400, 50, 600, 0}, Reasoning: 20, Effort: "high", Cwd: "/work/it", TTFT: 700, Millis: 5000},
|
||||
{Time: callT0.Add(12 * time.Second), Agent: "codex", Session: "sess-x", Model: "gpt-6-luna", Tokens: Tokens{200, 150, 2000, 300, 0}, Reasoning: 40, Effort: "low", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(11 * time.Second), Agent: "codex", Session: "sess-x", Model: "gpt-6-luna", Tokens: Tokens{1000, 100, 1000, 0, 0}, Reasoning: 40, Effort: "low", Cwd: "/work/it", Millis: 1000},
|
||||
{Time: callT0.Add(2 * time.Second), Agent: "codex", Session: "sess-x", Model: "gpt-6-astra", Tokens: Tokens{400, 50, 600, 0, 0}, Reasoning: 20, Effort: "high", Cwd: "/work/it", TTFT: 700, Millis: 5000},
|
||||
}
|
||||
if len(cs) != len(want) {
|
||||
t.Fatalf("want %d calls, got %d: %+v", len(want), len(cs), cs)
|
||||
@@ -310,7 +310,7 @@ func TestCallsIncremental(t *testing.T) {
|
||||
if len(claude) != 3 || claude[0].Tokens.Input != 3 || claude[1].Tokens != (Tokens{Input: 2, Output: 9}) || claude[2].Tokens.Input != 1 {
|
||||
t.Fatalf("claude: %+v", claude)
|
||||
}
|
||||
if len(codex) != 2 || codex[0].Tokens != (Tokens{100, 30, 100, 0}) || codex[1].Tokens != (Tokens{50, 10, 50, 0}) {
|
||||
if len(codex) != 2 || codex[0].Tokens != (Tokens{100, 30, 100, 0, 0}) || codex[1].Tokens != (Tokens{50, 10, 50, 0, 0}) {
|
||||
t.Fatalf("codex: %+v", codex)
|
||||
}
|
||||
|
||||
|
||||
+12
-10
@@ -28,10 +28,11 @@ type ccLine struct {
|
||||
Model string `json:"model"`
|
||||
Content json.RawMessage `json:"content"`
|
||||
Usage *struct {
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Creation *ccCacheCreation `json:"cache_creation"`
|
||||
} `json:"usage"`
|
||||
} `json:"message"`
|
||||
}
|
||||
@@ -204,15 +205,16 @@ func ccReply(s *state, at time.Time, msg, b []byte, main bool) {
|
||||
return
|
||||
}
|
||||
var u struct {
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Input int `json:"input_tokens"`
|
||||
Output int `json:"output_tokens"`
|
||||
CacheRead int `json:"cache_read_input_tokens"`
|
||||
CacheWrite int `json:"cache_creation_input_tokens"`
|
||||
Creation *ccCacheCreation `json:"cache_creation"`
|
||||
}
|
||||
if json.NewDecoder(bytes.NewReader(b[i+len(ccUsage)-1:])).Decode(&u) != nil {
|
||||
return
|
||||
}
|
||||
ccCount(s, m, at, model, Tokens{Input: u.Input, Output: u.Output, CacheRead: u.CacheRead, CacheWrite: u.CacheWrite}, main)
|
||||
ccCount(s, m, at, model, ccTokens(u.Input, u.Output, u.CacheRead, u.CacheWrite, u.Creation), main)
|
||||
}
|
||||
|
||||
// claudeFull reads a line whole.
|
||||
@@ -280,7 +282,7 @@ func claudeFull(s *state, b []byte, main bool) {
|
||||
if u == nil || l.Message.Model == "<synthetic>" {
|
||||
return
|
||||
}
|
||||
ccCount(s, m, at, l.Message.Model, Tokens{Input: u.Input, Output: u.Output, CacheRead: u.CacheRead, CacheWrite: u.CacheWrite}, main)
|
||||
ccCount(s, m, at, l.Message.Model, ccTokens(u.Input, u.Output, u.CacheRead, u.CacheWrite, u.Creation), main)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
package sessions
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// PAMI on Discord: Claude Code's transcript says which cache writes were
|
||||
// kept for an hour (cache_creation's ephemeral_1h_input_tokens), which
|
||||
// Anthropic bills at 2× input, not 1.25×. Both readers keep them apart; a
|
||||
// line without the split reads as before.
|
||||
func TestClaudeCacheWriteSplit(t *testing.T) {
|
||||
for _, fast := range []bool{false, true} {
|
||||
t.Run(fmt.Sprint("fast=", fast), func(t *testing.T) {
|
||||
s := &state{}
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
id := `"id":"a","model":"m"`
|
||||
if fast {
|
||||
id = `"model":"m","id":"a"`
|
||||
}
|
||||
claudeLine(s, []byte(fmt.Sprintf(`{"type":"assistant","timestamp":%q,"requestId":"req-a","message":{%s,"role":"assistant","content":[],"usage":{"input_tokens":10,"output_tokens":2,"cache_read_input_tokens":30,"cache_creation_input_tokens":3000,"cache_creation":{"ephemeral_5m_input_tokens":1000,"ephemeral_1h_input_tokens":2000}}}}`,
|
||||
at.Format(time.RFC3339Nano), id)), true)
|
||||
claudeLine(s, claudeUsageLine(at.Add(time.Second), "b", "req-b", "m", Tokens{Input: 1, CacheWrite: 500}, "", fast), true)
|
||||
if want := (Tokens{Input: 11, Output: 2, CacheRead: 30, CacheWrite: 3500, CacheWrite1h: 2000}); s.Models["m"] != want {
|
||||
t.Fatalf("tokens = %+v, want %+v", s.Models["m"], want)
|
||||
}
|
||||
})
|
||||
}
|
||||
if got := ccTokens(1, 2, 3, 0, &ccCacheCreation{Ephemeral1h: 40}); got.CacheWrite != 40 || got.CacheWrite1h != 40 {
|
||||
t.Errorf("a split with no total = %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -17,7 +17,7 @@ func TestClaudeNativeReplayAcrossReadersAndRestart(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lines := strings.Split(strings.TrimSpace(string(data)), "\n")
|
||||
want := Tokens{10338, 37363, 281066, 63398}
|
||||
want := Tokens{10338, 37363, 281066, 63398, 0}
|
||||
for split := 0; split <= len(lines); split++ {
|
||||
t.Run(string(rune('0'+split)), func(t *testing.T) {
|
||||
d := setupCalls(t)
|
||||
@@ -114,7 +114,7 @@ func TestEvictedClaudeRevisionsReleasedAndRebuilt(t *testing.T) {
|
||||
t.Fatal("evicted Claude retained continuation")
|
||||
}
|
||||
appendText(t, path, lines[2]+"\n")
|
||||
want := Tokens{10338, 37363, 281066, 63398}
|
||||
want := Tokens{10338, 37363, 281066, 63398, 0}
|
||||
List(0)
|
||||
var got Tokens
|
||||
for _, v := range cache[path].Models {
|
||||
|
||||
@@ -34,7 +34,7 @@ func TestClaudeUsageNonAdjacentReplay(t *testing.T) {
|
||||
t.Run(fmt.Sprint("fast=", fast), func(t *testing.T) {
|
||||
s := &state{}
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
a, b := Tokens{10, 2, 30, 4}, Tokens{20, 4, 60, 8}
|
||||
a, b := Tokens{10, 2, 30, 4, 0}, Tokens{20, 4, 60, 8, 0}
|
||||
claudeLine(s, claudeUsageLine(at, "a", "req-a", "m", a, "", fast), true)
|
||||
claudeLine(s, claudeUsageLine(at.Add(time.Second), "b", "req-b", "m", b, "", fast), true)
|
||||
claudeLine(s, claudeUsageLine(at.Add(48*time.Hour), "a", "req-a", "m", a, "", fast), true)
|
||||
@@ -55,7 +55,7 @@ func TestClaudeUsageRevisionAndTools(t *testing.T) {
|
||||
t.Run(fmt.Sprint("fast=", fast), func(t *testing.T) {
|
||||
s := &state{}
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
old, updated := Tokens{10, 1, 30, 4}, Tokens{10, 7, 30, 4}
|
||||
old, updated := Tokens{10, 1, 30, 4, 0}, Tokens{10, 7, 30, 4, 0}
|
||||
tool := `[{"type":"tool_use","id":"tool-1","name":"Read","input":{"requestId":"not-the-response"}}]`
|
||||
newTool := `[{"type":"tool_use","id":"tool-2","name":"Skill","input":{"skill":"review"}}]`
|
||||
line := func(d time.Duration, model string, tok Tokens, tools string) []byte {
|
||||
@@ -64,7 +64,7 @@ func TestClaudeUsageRevisionAndTools(t *testing.T) {
|
||||
claudeLine(s, line(0, "m", old, tool), true)
|
||||
claudeLine(s, line(time.Second, "m", updated, newTool), true)
|
||||
claudeLine(s, line(2*time.Second, "m", old, tool), true)
|
||||
claudeLine(s, line(time.Millisecond, "m", Tokens{10, 99, 30, 4}, tool), true)
|
||||
claudeLine(s, line(time.Millisecond, "m", Tokens{10, 99, 30, 4, 0}, tool), true)
|
||||
if s.Models["m"] != updated || claudeUsageReplies(s) != 1 {
|
||||
t.Fatalf("old replay undid newer usage: %+v", s)
|
||||
}
|
||||
@@ -73,7 +73,7 @@ func TestClaudeUsageRevisionAndTools(t *testing.T) {
|
||||
t.Fatalf("tools=%v skills=%v", d.Tools, d.Skills)
|
||||
}
|
||||
// A genuinely newer correction can reduce tokens and change model.
|
||||
correction := Tokens{8, 5, 20, 0}
|
||||
correction := Tokens{8, 5, 20, 0, 0}
|
||||
claudeLine(s, line(48*time.Hour, "m2", correction, ""), true)
|
||||
if !s.Models["m"].zero() || s.Models["m2"] != correction || d.Replies != 1 {
|
||||
t.Fatalf("correction failed: %+v", s)
|
||||
@@ -91,7 +91,7 @@ func TestClaudeUsageRevisionAndTools(t *testing.T) {
|
||||
|
||||
func TestClaudeUsageIdentity(t *testing.T) {
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
tok := Tokens{3, 4, 5, 6}
|
||||
tok := Tokens{3, 4, 5, 6, 0}
|
||||
cases := []struct {
|
||||
name string
|
||||
identities [][2]string
|
||||
@@ -130,7 +130,7 @@ func TestClaudeUsageIdentity(t *testing.T) {
|
||||
func TestClaudeUsageMissingToolIdentityAndSubagent(t *testing.T) {
|
||||
s := &state{}
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
tok := Tokens{3, 4, 5, 6}
|
||||
tok := Tokens{3, 4, 5, 6, 0}
|
||||
content := `[{"type":"tool_use","name":"Read","input":{}}]`
|
||||
for i := range 2 {
|
||||
claudeLine(s, claudeUsageLine(at.Add(time.Duration(i)*time.Second), "a", "r", "m", tok, content, true), false)
|
||||
@@ -156,7 +156,7 @@ func TestClaudeToolRevisionRejectsReplayedName(t *testing.T) {
|
||||
func TestClaudeCallReplayKeepsCompletionWithoutUUID(t *testing.T) {
|
||||
st := &callFile{Agent: "claude"}
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
tok := Tokens{3, 4, 5, 6}
|
||||
tok := Tokens{3, 4, 5, 6, 0}
|
||||
content := `[{"type":"text","text":"original block"}]`
|
||||
claudeCallLine(st, claudeUsageLine(at, "a", "ra", "m", tok, content, true))
|
||||
claudeCallLine(st, claudeUsageLine(at.Add(time.Second), "b", "rb", "m", tok, "", true))
|
||||
@@ -171,7 +171,7 @@ func TestClaudeRepeatedBlockRejectsZeroedHistoricalSnapshot(t *testing.T) {
|
||||
t.Run(fmt.Sprint("fast=", fast), func(t *testing.T) {
|
||||
s, calls := &state{}, &callFile{Agent: "claude"}
|
||||
at := time.Date(2026, 9, 25, 3, 0, 0, 0, time.UTC)
|
||||
tokens := Tokens{69, 31852, 185103, 998}
|
||||
tokens := Tokens{69, 31852, 185103, 998, 0}
|
||||
// Two blocks report the same whole-message usage. Historical
|
||||
// copies later retain both original timestamps but zero usage.
|
||||
for i, usage := range []Tokens{tokens, tokens, {}, {}} {
|
||||
@@ -188,7 +188,7 @@ func TestClaudeRepeatedBlockRejectsZeroedHistoricalSnapshot(t *testing.T) {
|
||||
|
||||
func TestClaudeUsageResumeEquivalence(t *testing.T) {
|
||||
at := time.Date(2026, 9, 20, 12, 0, 0, 0, time.UTC)
|
||||
old, updated := Tokens{10, 1, 30, 4}, Tokens{10, 7, 30, 4}
|
||||
old, updated := Tokens{10, 1, 30, 4, 0}, Tokens{10, 7, 30, 4, 0}
|
||||
tool := `[{"type":"tool_use","id":"tool-1","name":"Read","input":{}}]`
|
||||
lines := [][]byte{
|
||||
claudeUsageLine(at, "a", "", "m", old, tool, true),
|
||||
|
||||
@@ -18,8 +18,8 @@ func TestCodexNativeReadersAgree(t *testing.T) {
|
||||
calls int
|
||||
tokens Tokens
|
||||
}{
|
||||
{"codex-native-paginated.jsonl", 3, Tokens{128290, 293, 254720, 0}},
|
||||
{"codex-native-compaction.jsonl", 4, Tokens{22857, 8001, 520320, 0}},
|
||||
{"codex-native-paginated.jsonl", 3, Tokens{128290, 293, 254720, 0, 0}},
|
||||
{"codex-native-compaction.jsonl", 4, Tokens{22857, 8001, 520320, 0, 0}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
|
||||
@@ -26,7 +26,7 @@ func TestCodexUsageBothCountersRestartAfterLegacyHistory(t *testing.T) {
|
||||
if cs := Calls(time.Time{}); len(cs) != 5 {
|
||||
t.Fatalf("calls=%+v", cs)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{400, 50, 100, 0}) {
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{400, 50, 100, 0, 0}) {
|
||||
t.Fatalf("summary=%+v", ss)
|
||||
}
|
||||
})
|
||||
@@ -43,7 +43,7 @@ func TestCodexUsageLargerFirstRequestAfterLegacyReset(t *testing.T) {
|
||||
if cs := Calls(time.Time{}); len(cs) != 2 {
|
||||
t.Fatalf("new runtime's first request lost or replay counted: %+v", cs)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{46055, 1281, 137984, 0}) {
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{46055, 1281, 137984, 0, 0}) {
|
||||
t.Fatalf("summary=%+v", ss)
|
||||
}
|
||||
}
|
||||
@@ -46,7 +46,7 @@ func TestCodexUsagePaginatedSnapshotBoundary(t *testing.T) {
|
||||
for _, c := range cs {
|
||||
total.add(c.Tokens)
|
||||
}
|
||||
if total != (Tokens{340, 52, 190, 0}) {
|
||||
if total != (Tokens{340, 52, 190, 0, 0}) {
|
||||
t.Fatalf("tokens=%+v", total)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != total {
|
||||
|
||||
@@ -33,7 +33,7 @@ func TestCodexUsageCompactionCounterDomains(t *testing.T) {
|
||||
}
|
||||
total.add(c.Tokens)
|
||||
}
|
||||
if len(cs) != 4 || total != (Tokens{11157, 6856, 496640, 0}) {
|
||||
if len(cs) != 4 || total != (Tokens{11157, 6856, 496640, 0, 0}) {
|
||||
t.Fatalf("calls=%d tokens=%+v", len(cs), total)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != total {
|
||||
@@ -90,7 +90,7 @@ func TestCodexUsageCounterDomainOffsetChangesAndReload(t *testing.T) {
|
||||
if epoch := callCache[path].CX.Usage.Epoch; epoch != 0 {
|
||||
t.Fatalf("cross-domain reset: %d", epoch)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{1080, 180, 720, 0}) {
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{1080, 180, 720, 0, 0}) {
|
||||
t.Fatalf("summary: %+v", ss)
|
||||
}
|
||||
resetCalls()
|
||||
@@ -152,7 +152,7 @@ func TestCodexUsageResumeCounterDomainsAndReload(t *testing.T) {
|
||||
if len(cs) != 9 {
|
||||
t.Fatalf("reset doubled/lost calls: %+v", cs)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{600, 100, 400, 0}) {
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{600, 100, 400, 0, 0}) {
|
||||
t.Fatalf("summary: %+v", ss)
|
||||
}
|
||||
Reset()
|
||||
@@ -199,7 +199,7 @@ func TestCodexUsageEmbeddedCompactionBeforeTopLevelRecord(t *testing.T) {
|
||||
if cs := Calls(time.Time{}); len(cs) != 2 {
|
||||
t.Fatalf("compaction replay duplicated usage: %+v", cs)
|
||||
}
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{120, 20, 80, 0}) {
|
||||
if ss := List(0); len(ss) != 1 || ss[0].Tokens != (Tokens{120, 20, 80, 0, 0}) {
|
||||
t.Fatalf("summary: %+v", ss)
|
||||
}
|
||||
}
|
||||
@@ -265,7 +265,7 @@ func TestCodexUsageRealCheckpointPairs(t *testing.T) {
|
||||
t.Fatalf("identity: %+v", c)
|
||||
}
|
||||
}
|
||||
if total != (Tokens{4515, 1084, 299776, 0}) {
|
||||
if total != (Tokens{4515, 1084, 299776, 0, 0}) {
|
||||
t.Fatal(total)
|
||||
}
|
||||
if cs[0].Time.Format(time.RFC3339Nano) != "2026-10-03T08:49:56.408Z" || cs[1].Time.Format(time.RFC3339Nano) != "2026-10-03T08:49:31.433Z" {
|
||||
|
||||
@@ -70,10 +70,10 @@ func TestDroid(t *testing.T) {
|
||||
t.Fatalf("droid: %+v", d)
|
||||
}
|
||||
// magpie's custom model by the model it asks for, the subagent's in
|
||||
if m := model(d, "anthropic/claude-sonnet-4-5"); m.Tokens != (Tokens{100, 50, 1000, 10}) {
|
||||
if m := model(d, "anthropic/claude-sonnet-4-5"); m.Tokens != (Tokens{100, 50, 1000, 10, 0}) {
|
||||
t.Fatalf("sonnet: %+v", d.Models)
|
||||
}
|
||||
if m := model(d, "claude-haiku-4-5"); m.Tokens != (Tokens{5, 2, 0, 0}) {
|
||||
if m := model(d, "claude-haiku-4-5"); m.Tokens != (Tokens{5, 2, 0, 0, 0}) {
|
||||
t.Fatalf("haiku, the subagent's in: %+v", d.Models)
|
||||
}
|
||||
if !d.Start.Equal(at("2026-09-27T08:00:00Z")) || !d.Last.Equal(at("2026-09-27T08:02:00Z")) {
|
||||
@@ -90,7 +90,7 @@ func TestDroid(t *testing.T) {
|
||||
if f.Title != "Try another way" || !f.Start.Equal(at("2026-09-27T09:00:00Z")) || !f.Last.Equal(at("2026-09-27T09:00:30Z")) {
|
||||
t.Fatalf("fork: %+v", f)
|
||||
}
|
||||
if m := model(f, "Kimi-K2"); m.Tokens != (Tokens{7, 3, 0, 0}) {
|
||||
if m := model(f, "Kimi-K2"); m.Tokens != (Tokens{7, 3, 0, 0, 0}) {
|
||||
t.Fatalf("fork models: %+v", f.Models)
|
||||
}
|
||||
if o := find(t, ss, "droid", droidOld); o.Title != "hi there" || !o.Tokens.zero() {
|
||||
|
||||
@@ -28,10 +28,10 @@ func TestGrok(t *testing.T) {
|
||||
t.Fatalf("grok: %+v", g)
|
||||
}
|
||||
// the cache out of the input, the subagent's turn in once
|
||||
if m := model(g, "grok-4.7-build"); m.Tokens != (Tokens{700, 100, 2800, 0}) {
|
||||
if m := model(g, "grok-4.7-build"); m.Tokens != (Tokens{700, 100, 2800, 0, 0}) {
|
||||
t.Fatalf("build: %+v", m)
|
||||
}
|
||||
if m := model(g, "grok-4.7-mini"); m.Tokens != (Tokens{300, 30, 100, 0}) {
|
||||
if m := model(g, "grok-4.7-mini"); m.Tokens != (Tokens{300, 30, 100, 0, 0}) {
|
||||
t.Fatalf("mini, the subagent's in: %+v", m)
|
||||
}
|
||||
// the hook that ran as it was closed, a day on, isn't when it was at work
|
||||
@@ -45,7 +45,7 @@ func TestGrok(t *testing.T) {
|
||||
}
|
||||
// the fork: what it copied counted in the session it came from
|
||||
f := find(t, ss, "grok", grokFork)
|
||||
if f.Tokens != (Tokens{100, 30, 300, 0}) || f.Title != "Try another way" {
|
||||
if f.Tokens != (Tokens{100, 30, 300, 0, 0}) || f.Title != "Try another way" {
|
||||
t.Fatalf("fork: %+v", f)
|
||||
}
|
||||
if !f.Start.Equal(at("2026-09-27T09:00:00.25Z")) || !f.Last.Equal(at("2026-09-27T09:00:30Z")) {
|
||||
@@ -85,7 +85,7 @@ func TestWorkBuddy(t *testing.T) {
|
||||
t.Fatalf("workbuddy: %+v", w)
|
||||
}
|
||||
// a reply's usage counted once, however many of its lines carry it
|
||||
if m := model(w, "hy3"); m.Tokens != (Tokens{400, 120, 2200, 0}) {
|
||||
if m := model(w, "hy3"); m.Tokens != (Tokens{400, 120, 2200, 0, 0}) {
|
||||
t.Fatalf("hy3: %+v", m)
|
||||
}
|
||||
if !w.Start.Equal(at("2026-09-27T10:00:00Z")) || !w.Last.Equal(at("2026-09-27T10:00:10Z")) {
|
||||
@@ -96,7 +96,7 @@ func TestWorkBuddy(t *testing.T) {
|
||||
if o.Cwd != "/work/wb-old" || o.Title != "Quarterly report" {
|
||||
t.Fatalf("old: %+v", o)
|
||||
}
|
||||
if m := model(o, "auto"); m.Tokens != (Tokens{5000, 100, 0, 0}) {
|
||||
if m := model(o, "auto"); m.Tokens != (Tokens{5000, 100, 0, 0, 0}) {
|
||||
t.Fatalf("auto: %+v", m)
|
||||
}
|
||||
|
||||
|
||||
@@ -117,7 +117,7 @@ func TestHermesListStatsCallsAndContent(t *testing.T) {
|
||||
if !s.ReadOnly || s.Resume != "" || s.Title != "A Hermes task" || s.Cwd != "/work/hermes" {
|
||||
t.Errorf("metadata/read-only state: %+v", s)
|
||||
}
|
||||
if m := model(s, "actual-model"); m.Tokens != (Tokens{100, 50, 25, 15}) {
|
||||
if m := model(s, "actual-model"); m.Tokens != (Tokens{100, 50, 25, 15, 0}) {
|
||||
t.Errorf("usage aggregate: %+v", m)
|
||||
}
|
||||
if model(s, "fallback-model").Model != "" {
|
||||
@@ -141,7 +141,7 @@ func TestHermesListStatsCallsAndContent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
if total != (Tokens{200, 100, 50, 30}) {
|
||||
if total != (Tokens{200, 100, 50, 30, 0}) {
|
||||
t.Errorf("Hermes stats total = %+v", total)
|
||||
}
|
||||
|
||||
@@ -162,7 +162,7 @@ func TestHermesUsageFallbackAndLiveRefresh(t *testing.T) {
|
||||
path := hermesFixture(t, root, "", "fallback", false)
|
||||
|
||||
first := hermesSessionByID(t, hermesID(path, "fallback"))
|
||||
if got := model(first, "fallback-model").Tokens; got != (Tokens{90, 30, 40, 10}) {
|
||||
if got := model(first, "fallback-model").Tokens; got != (Tokens{90, 30, 40, 10, 0}) {
|
||||
t.Fatalf("session aggregate fallback = %+v", got)
|
||||
}
|
||||
|
||||
@@ -192,7 +192,7 @@ func TestHermesUsageFallbackAndLiveRefresh(t *testing.T) {
|
||||
if updated.Title != "Updated title" {
|
||||
t.Errorf("title not refreshed: %q", updated.Title)
|
||||
}
|
||||
if got := model(updated, "new-model").Tokens; got != (Tokens{11, 7, 3, 2}) {
|
||||
if got := model(updated, "new-model").Tokens; got != (Tokens{11, 7, 3, 2, 0}) {
|
||||
t.Errorf("WAL usage not refreshed: %+v", got)
|
||||
}
|
||||
// The bucket timestamp intentionally stays fixed. Newly observed usage
|
||||
@@ -208,7 +208,7 @@ func TestHermesUsageFallbackAndLiveRefresh(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
updated = hermesSessionByID(t, hermesID(path, "fallback"))
|
||||
if updated.Title != "Changed again" || model(updated, "new-model").Tokens != (Tokens{13, 9, 4, 3}) {
|
||||
if updated.Title != "Changed again" || model(updated, "new-model").Tokens != (Tokens{13, 9, 4, 3, 0}) {
|
||||
t.Errorf("same-mtime usage/title refresh: %+v", updated)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -52,10 +52,10 @@ func TestHermesUsageAddsMainResidualWhenOnlyAuxiliaryRowsExist(t *testing.T) {
|
||||
setup(t)
|
||||
path := hermesTaskUsageFixture(t, "aux-only", nil)
|
||||
s := hermesSessionByID(t, hermesID(path, "aux-only"))
|
||||
if got := model(s, "fallback-model").Tokens; got != (Tokens{90, 30, 40, 10}) {
|
||||
if got := model(s, "fallback-model").Tokens; got != (Tokens{90, 30, 40, 10, 0}) {
|
||||
t.Fatalf("main-loop residual = %+v, want sessions aggregate", got)
|
||||
}
|
||||
if got := model(s, "actual-model").Tokens; got != (Tokens{100, 50, 25, 15}) {
|
||||
if got := model(s, "actual-model").Tokens; got != (Tokens{100, 50, 25, 15, 0}) {
|
||||
t.Fatalf("auxiliary usage = %+v", got)
|
||||
}
|
||||
// Hermes does not emit Calls; verify no leakage.
|
||||
@@ -114,13 +114,13 @@ func TestHermesUsageResidualSubtractsMainAttributionOnly(t *testing.T) {
|
||||
setup(t)
|
||||
path := hermesTaskUsageFixture(t, "mixed", &Tokens{Input: 60, Output: 20, CacheRead: 10, CacheWrite: 5})
|
||||
s := hermesSessionByID(t, hermesID(path, "mixed"))
|
||||
if got := model(s, "main-model").Tokens; got != (Tokens{60, 20, 10, 5}) {
|
||||
if got := model(s, "main-model").Tokens; got != (Tokens{60, 20, 10, 5, 0}) {
|
||||
t.Fatalf("main attribution = %+v", got)
|
||||
}
|
||||
if got := model(s, "fallback-model").Tokens; got != (Tokens{30, 10, 30, 5}) {
|
||||
if got := model(s, "fallback-model").Tokens; got != (Tokens{30, 10, 30, 5, 0}) {
|
||||
t.Fatalf("main residual = %+v, want remaining sessions aggregate", got)
|
||||
}
|
||||
if got := model(s, "actual-model").Tokens; got != (Tokens{100, 50, 25, 15}) {
|
||||
if got := model(s, "actual-model").Tokens; got != (Tokens{100, 50, 25, 15, 0}) {
|
||||
t.Fatalf("auxiliary usage changed = %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -31,10 +31,10 @@ func TestOmp(t *testing.T) {
|
||||
model string
|
||||
want Tokens
|
||||
}{
|
||||
{"claude-opus-5-5", Tokens{110, 55, 1500, 200}},
|
||||
{"claude-haiku-5", Tokens{20, 5, 0, 0}}, // a model_usage entry
|
||||
{"claude-sonnet-5", Tokens{340, 34, 100, 0}}, // the subagent and its own
|
||||
{"gpt-6-astra", Tokens{50, 10, 0, 0}}, // the advisor
|
||||
{"claude-opus-5-5", Tokens{110, 55, 1500, 200, 0}},
|
||||
{"claude-haiku-5", Tokens{20, 5, 0, 0, 0}}, // a model_usage entry
|
||||
{"claude-sonnet-5", Tokens{340, 34, 100, 0, 0}}, // the subagent and its own
|
||||
{"gpt-6-astra", Tokens{50, 10, 0, 0, 0}}, // the advisor
|
||||
} {
|
||||
if got := model(m, c.model); got.Tokens != c.want {
|
||||
t.Fatalf("%s: %+v, want %+v", c.model, got, c.want)
|
||||
@@ -54,7 +54,7 @@ func TestOmp(t *testing.T) {
|
||||
|
||||
// the fork: what it copied counted in the session it came from
|
||||
f := find(t, ss, "omp", ompFork)
|
||||
if f.Tokens != (Tokens{7, 3, 0, 0}) || f.Title != "Port the parser" {
|
||||
if f.Tokens != (Tokens{7, 3, 0, 0, 0}) || f.Title != "Port the parser" {
|
||||
t.Fatalf("fork: %+v", f)
|
||||
}
|
||||
if !f.Start.Equal(at("2026-09-29T09:00:00Z")) || !f.Last.Equal(at("2026-09-29T09:00:20Z")) {
|
||||
|
||||
@@ -64,13 +64,13 @@ func checkOpenCode(t *testing.T, ss []Session) {
|
||||
if oc.Cwd != "/work/oc" || oc.Title != "Why does the login test flake?" {
|
||||
t.Fatalf("opencode: %+v", oc)
|
||||
}
|
||||
if a := model(oc, "gpt-6-astra"); a.Tokens != (Tokens{1000, 120, 5000, 0}) || !a.Priced {
|
||||
if a := model(oc, "gpt-6-astra"); a.Tokens != (Tokens{1000, 120, 5000, 0, 0}) || !a.Priced {
|
||||
t.Fatalf("astra: %+v", a)
|
||||
}
|
||||
if l := model(oc, "codex/gpt-6-luna"); l.Tokens != (Tokens{300, 60, 2000, 0}) || l.Priced {
|
||||
if l := model(oc, "codex/gpt-6-luna"); l.Tokens != (Tokens{300, 60, 2000, 0, 0}) || l.Priced {
|
||||
t.Fatalf("luna: %+v", l)
|
||||
}
|
||||
if o := model(oc, "claude-opus-5-5"); o.Tokens != (Tokens{40, 60, 700, 300}) {
|
||||
if o := model(oc, "claude-opus-5-5"); o.Tokens != (Tokens{40, 60, 700, 300, 0}) {
|
||||
t.Fatalf("the subagent's opus: %+v", o)
|
||||
}
|
||||
if !oc.Start.Equal(at("2026-09-26T10:00:00Z")) || !oc.Last.Equal(at("2026-09-26T10:03:00Z")) {
|
||||
@@ -90,7 +90,7 @@ func TestOpenCodeFiles(t *testing.T) {
|
||||
|
||||
// the work of the main session only, 10:00:00–10:03:00, every pause short
|
||||
s := statsAt(0, statsNow)
|
||||
if got, sec := usageOn(s, "2026-09-26", "/work/oc"); got != (Tokens{1340, 240, 7700, 300}) || sec != 180 {
|
||||
if got, sec := usageOn(s, "2026-09-26", "/work/oc"); got != (Tokens{1340, 240, 7700, 300, 0}) || sec != 180 {
|
||||
t.Fatalf("opencode on the 26th: %+v, %ds active", got, sec)
|
||||
}
|
||||
}
|
||||
@@ -179,7 +179,7 @@ func TestOpenCodeDB(t *testing.T) {
|
||||
t.Fatalf("path %s", oc.Path)
|
||||
}
|
||||
s := statsAt(0, statsNow)
|
||||
if got, sec := usageOn(s, "2026-09-26", "/work/oc"); got != (Tokens{1340, 240, 7700, 300}) || sec != 180 {
|
||||
if got, sec := usageOn(s, "2026-09-26", "/work/oc"); got != (Tokens{1340, 240, 7700, 300, 0}) || sec != 180 {
|
||||
t.Fatalf("opencode on the 26th: %+v, %ds active", got, sec)
|
||||
}
|
||||
|
||||
@@ -187,7 +187,7 @@ func TestOpenCodeDB(t *testing.T) {
|
||||
ocInsert(t, db, ocChild, "msg_0005", 1790416990000, 1790417000000,
|
||||
`{"id":"msg_0005","role":"assistant","modelID":"claude-opus-5-5","tokens":{"input":1,"output":2,"reasoning":0,"cache":{"read":3,"write":4}},"time":{"created":1790416990000,"completed":1790417000000}}`)
|
||||
oc = find(t, List(0), "opencode", ocMain)
|
||||
if o := model(oc, "claude-opus-5-5"); o.Tokens != (Tokens{41, 62, 703, 304}) {
|
||||
if o := model(oc, "claude-opus-5-5"); o.Tokens != (Tokens{41, 62, 703, 304, 0}) {
|
||||
t.Fatalf("after a new reply: %+v", o)
|
||||
}
|
||||
if !oc.Last.Equal(at("2026-09-26T10:03:20Z")) {
|
||||
@@ -215,10 +215,10 @@ func TestPi(t *testing.T) {
|
||||
}
|
||||
// a tool's model work and a compaction on the model in use, a usage
|
||||
// entry on its own
|
||||
if o := model(p, "claude-opus-5-5"); o.Tokens != (Tokens{110, 55, 1300, 200}) || !o.Priced {
|
||||
if o := model(p, "claude-opus-5-5"); o.Tokens != (Tokens{110, 55, 1300, 200, 0}) || !o.Priced {
|
||||
t.Fatalf("opus: %+v", o)
|
||||
}
|
||||
if o := model(p, "claude/claude-opus-5-5"); o.Tokens != (Tokens{60, 40, 500, 0}) {
|
||||
if o := model(p, "claude/claude-opus-5-5"); o.Tokens != (Tokens{60, 40, 500, 0, 0}) {
|
||||
t.Fatalf("magpie's opus: %+v", o)
|
||||
}
|
||||
if !p.Start.Equal(at("2026-09-24T08:00:00Z")) || !p.Last.Equal(at("2026-09-24T08:04:00Z")) {
|
||||
@@ -232,7 +232,7 @@ func TestPi(t *testing.T) {
|
||||
|
||||
// the fork: what it copied counted in the session it came from
|
||||
f := find(t, ss, "pi", piFork)
|
||||
if f.Tokens != (Tokens{7, 3, 0, 0}) || f.Title != "Refactor the parser" {
|
||||
if f.Tokens != (Tokens{7, 3, 0, 0, 0}) || f.Title != "Refactor the parser" {
|
||||
t.Fatalf("fork: %+v", f)
|
||||
}
|
||||
if !f.Start.Equal(at("2026-09-25T09:00:00Z")) || !f.Last.Equal(at("2026-09-25T09:00:20Z")) {
|
||||
@@ -240,10 +240,10 @@ func TestPi(t *testing.T) {
|
||||
}
|
||||
|
||||
s := statsAt(0, statsNow)
|
||||
if got, sec := usageOn(s, "2026-09-24", "/work/pi"); got != (Tokens{170, 95, 1800, 200}) || sec != 240 {
|
||||
if got, sec := usageOn(s, "2026-09-24", "/work/pi"); got != (Tokens{170, 95, 1800, 200, 0}) || sec != 240 {
|
||||
t.Fatalf("pi on the 24th: %+v, %ds active", got, sec)
|
||||
}
|
||||
if got, sec := usageOn(s, "2026-09-25", "/work/pi"); got != (Tokens{7, 3, 0, 0}) || sec != 20 {
|
||||
if got, sec := usageOn(s, "2026-09-25", "/work/pi"); got != (Tokens{7, 3, 0, 0, 0}) || sec != 20 {
|
||||
t.Fatalf("pi on the 25th: %+v, %ds active", got, sec)
|
||||
}
|
||||
}
|
||||
@@ -350,7 +350,7 @@ func TestOpenCodeV2(t *testing.T) {
|
||||
t.Fatalf("v2 session: %+v", oc)
|
||||
}
|
||||
s := statsAt(0, statsNow)
|
||||
if got, sec := usageOn(s, "2026-09-26", "/work/oc"); got != (Tokens{1340, 240, 7700, 300}) || sec != 180 {
|
||||
if got, sec := usageOn(s, "2026-09-26", "/work/oc"); got != (Tokens{1340, 240, 7700, 300, 0}) || sec != 180 {
|
||||
t.Fatalf("opencode 2 on the 26th: %+v, %ds active", got, sec)
|
||||
}
|
||||
|
||||
@@ -360,7 +360,7 @@ func TestOpenCodeV2(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
oc = find(t, List(0), "opencode", ocMain)
|
||||
if l := model(oc, "codex/gpt-6-luna"); l.Tokens != (Tokens{305, 61, 2000, 0}) {
|
||||
if l := model(oc, "codex/gpt-6-luna"); l.Tokens != (Tokens{305, 61, 2000, 0, 0}) {
|
||||
t.Fatalf("luna after a compaction: %+v", l)
|
||||
}
|
||||
|
||||
|
||||
@@ -42,6 +42,9 @@ type Tokens struct {
|
||||
Output int `json:"output"`
|
||||
CacheRead int `json:"cache_read"`
|
||||
CacheWrite int `json:"cache_write"`
|
||||
// CacheWrite1h is how many of CacheWrite were written to be kept for an
|
||||
// hour, where the agent says (Claude Code's cache_creation)
|
||||
CacheWrite1h int `json:"cache_write_1h,omitempty"`
|
||||
}
|
||||
|
||||
func (t *Tokens) add(u Tokens) {
|
||||
@@ -49,6 +52,7 @@ func (t *Tokens) add(u Tokens) {
|
||||
t.Output += u.Output
|
||||
t.CacheRead += u.CacheRead
|
||||
t.CacheWrite += u.CacheWrite
|
||||
t.CacheWrite1h += u.CacheWrite1h
|
||||
}
|
||||
|
||||
func (t *Tokens) sub(u Tokens) {
|
||||
@@ -56,6 +60,24 @@ func (t *Tokens) sub(u Tokens) {
|
||||
t.Output -= u.Output
|
||||
t.CacheRead -= u.CacheRead
|
||||
t.CacheWrite -= u.CacheWrite
|
||||
t.CacheWrite1h -= u.CacheWrite1h
|
||||
}
|
||||
|
||||
// ccCacheCreation is Claude Code's cache writes split by how long they are
|
||||
// kept.
|
||||
type ccCacheCreation struct {
|
||||
Ephemeral5m int `json:"ephemeral_5m_input_tokens"`
|
||||
Ephemeral1h int `json:"ephemeral_1h_input_tokens"`
|
||||
}
|
||||
|
||||
// ccTokens is a Claude Code message's usage, its 1-hour cache writes apart.
|
||||
func ccTokens(in, out, read, write int, c *ccCacheCreation) Tokens {
|
||||
t := Tokens{Input: in, Output: out, CacheRead: read, CacheWrite: write}
|
||||
if c != nil {
|
||||
t.CacheWrite = max(t.CacheWrite, c.Ephemeral5m+c.Ephemeral1h)
|
||||
t.CacheWrite1h = c.Ephemeral1h
|
||||
}
|
||||
return t
|
||||
}
|
||||
|
||||
func (t Tokens) zero() bool { return t == Tokens{} }
|
||||
@@ -507,7 +529,8 @@ func CachePath() string { return filepath.Join(filepath.Dir(catalog.CachePath())
|
||||
// 11: Codex's input without what it wrote to the cache (#589).
|
||||
// 16: count Codex response records and compaction usage.
|
||||
// 17: reconcile recent Claude message revisions.
|
||||
const cacheVersion = 17
|
||||
// 18: Claude Code's 1-hour cache writes apart from its 5-minute ones.
|
||||
const cacheVersion = 18
|
||||
|
||||
type cacheFile struct {
|
||||
Version int `json:"version"`
|
||||
@@ -927,7 +950,9 @@ func assemble(fs []file, price func(string) *catalog.Price) (Session, bool) {
|
||||
}
|
||||
for _, m := range models {
|
||||
if p := price(m.Model); p != nil {
|
||||
m.Cost, m.Priced = p.Cost(m.Input, m.Output, m.CacheRead, m.CacheWrite), true
|
||||
// a sum of the session's calls of the model, at its base price:
|
||||
// which call went over a long-context tier isn't known here
|
||||
m.Cost, m.Priced = p.At(0).CostSplit(m.Input, m.Output, m.CacheRead, m.CacheWrite, m.CacheWrite1h), true
|
||||
s.Cost += m.Cost
|
||||
} else {
|
||||
s.Unpriced++
|
||||
|
||||
@@ -99,10 +99,10 @@ func TestList(t *testing.T) {
|
||||
if cc.ID != "11111111-2222-3333-4444-555555555555" || cc.Cwd != "/work/app" || cc.Title != "Fix the login bug in auth.go" {
|
||||
t.Fatalf("claude: %+v", cc)
|
||||
}
|
||||
if o := model(cc, "claude-opus-5-5"); o.Tokens != (Tokens{100, 50, 5000, 1000}) || !o.Priced {
|
||||
if o := model(cc, "claude-opus-5-5"); o.Tokens != (Tokens{100, 50, 5000, 1000, 0}) || !o.Priced {
|
||||
t.Fatalf("opus: %+v", o)
|
||||
}
|
||||
if h := model(cc, "claude-haiku-4-5-20251001"); h.Tokens != (Tokens{1010, 120, 200, 0}) || h.Priced {
|
||||
if h := model(cc, "claude-haiku-4-5-20251001"); h.Tokens != (Tokens{1010, 120, 200, 0, 0}) || h.Priced {
|
||||
t.Fatalf("haiku: %+v", h)
|
||||
}
|
||||
if want := (100*4 + 50*20 + 5000*0.2 + 1000*5) / 1e6; !near(cc.Cost, want) || cc.Unpriced != 1 {
|
||||
@@ -117,10 +117,10 @@ func TestList(t *testing.T) {
|
||||
if cx.ID != "01a0bdc9-b5fd-7e63-9658-68bc8dce5ecd" || cx.Cwd != "/work/it's" || cx.Title != "Add a README" {
|
||||
t.Fatalf("codex: %+v", cx)
|
||||
}
|
||||
if a := model(cx, "gpt-6-astra"); a.Tokens != (Tokens{8000, 400, 10000, 0}) {
|
||||
if a := model(cx, "gpt-6-astra"); a.Tokens != (Tokens{8000, 400, 10000, 0, 0}) {
|
||||
t.Fatalf("astra: %+v", a)
|
||||
}
|
||||
if l := model(cx, "gpt-6-luna"); l.Tokens != (Tokens{2000, 200, 10000, 0}) {
|
||||
if l := model(cx, "gpt-6-luna"); l.Tokens != (Tokens{2000, 200, 10000, 0, 0}) {
|
||||
t.Fatalf("luna: %+v", l)
|
||||
}
|
||||
if cx.Input != 10000 || cx.Output != 600 || cx.CacheRead != 20000 || cx.Unpriced != 1 {
|
||||
|
||||
@@ -236,7 +236,7 @@ func statsAt(days int, now time.Time) Stats {
|
||||
for k, t := range use {
|
||||
u := Usage{Agent: k.agent, Cwd: k.cwd, Model: k.model, Tokens: t}
|
||||
if p := price(k.model); p != nil {
|
||||
u.Cost, u.Priced = p.Cost(t.Input, t.Output, t.CacheRead, t.CacheWrite), true
|
||||
u.Cost, u.Priced = p.At(0).CostSplit(t.Input, t.Output, t.CacheRead, t.CacheWrite, t.CacheWrite1h), true
|
||||
}
|
||||
byDate[k.date].Usage = append(byDate[k.date].Usage, u)
|
||||
}
|
||||
@@ -309,7 +309,7 @@ func statsAt(days int, now time.Time) Stats {
|
||||
sum.Tokens.add(t)
|
||||
sum.Models = append(sum.Models, model)
|
||||
if p := price(model); p != nil {
|
||||
sum.Cost += p.Cost(t.Input, t.Output, t.CacheRead, t.CacheWrite)
|
||||
sum.Cost += p.At(0).CostSplit(t.Input, t.Output, t.CacheRead, t.CacheWrite, t.CacheWrite1h)
|
||||
} else {
|
||||
sum.Priced = false
|
||||
}
|
||||
|
||||
@@ -56,13 +56,13 @@ func TestStatsByDay(t *testing.T) {
|
||||
// Claude Code on the 20th: the session and its subagent, in its folder;
|
||||
// Codex on the 20th and, picked up again, on the 23rd. The session's own
|
||||
// file was at work 10:00:00–10:05:00, every pause under five minutes.
|
||||
if got, sec := usageOn(s, "2026-09-20", "/work/app"); got != (Tokens{1110, 170, 5200, 1000}) || sec != 300 {
|
||||
if got, sec := usageOn(s, "2026-09-20", "/work/app"); got != (Tokens{1110, 170, 5200, 1000, 0}) || sec != 300 {
|
||||
t.Fatalf("claude on the 20th: %+v, %ds active", got, sec)
|
||||
}
|
||||
if got, _ := usageOn(s, "2026-09-20", "/work/it's"); got != (Tokens{8000, 500, 14000, 0}) {
|
||||
if got, _ := usageOn(s, "2026-09-20", "/work/it's"); got != (Tokens{8000, 500, 14000, 0, 0}) {
|
||||
t.Fatalf("codex on the 20th: %+v", got)
|
||||
}
|
||||
if got, _ := usageOn(s, "2026-09-23", "/work/it's"); got != (Tokens{2000, 100, 6000, 0}) {
|
||||
if got, _ := usageOn(s, "2026-09-23", "/work/it's"); got != (Tokens{2000, 100, 6000, 0, 0}) {
|
||||
t.Fatalf("codex on the 23rd: %+v", got)
|
||||
}
|
||||
// the days add up to the sessions' totals
|
||||
@@ -114,13 +114,13 @@ func TestStatsMidnight(t *testing.T) {
|
||||
t.Fatalf("%s, the 21st: %+v %ds, want %+v %ds", when, got, sec, d21, a21)
|
||||
}
|
||||
}
|
||||
check("before midnight", Tokens{300, 30, 20, 0}, Tokens{}, 180, 0)
|
||||
check("before midnight", Tokens{300, 30, 20, 0, 0}, Tokens{}, 180, 0)
|
||||
|
||||
// read on from where it was left: m2 moves to the 21st, counted once
|
||||
f, _ := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0)
|
||||
f.WriteString(rest)
|
||||
f.Close()
|
||||
after20, after21 := Tokens{100, 10, 10, 0}, Tokens{1400, 145, 30, 0}
|
||||
after20, after21 := Tokens{100, 10, 10, 0, 0}, Tokens{1400, 145, 30, 0, 0}
|
||||
check("read on", after20, after21, 180, 240)
|
||||
|
||||
// the same from the start, and from the parse kept on disk
|
||||
@@ -145,7 +145,7 @@ func TestSum(t *testing.T) {
|
||||
t.Fatalf("days %+v", r.Days)
|
||||
}
|
||||
// only Codex's pick-up on the 23rd is in the last 7 days
|
||||
if r.Tokens != (Tokens{2000, 100, 6000, 0}) || r.DaysUsed != 1 || r.Days[1].Spent() != 2100 {
|
||||
if r.Tokens != (Tokens{2000, 100, 6000, 0, 0}) || r.DaysUsed != 1 || r.Days[1].Spent() != 2100 {
|
||||
t.Fatalf("%+v", r)
|
||||
}
|
||||
if len(r.Models) != 1 || r.Models[0].Name != "gpt-6-astra" || len(r.Folders) != 1 || r.Folders[0].Name != "/work/it's" {
|
||||
@@ -154,7 +154,7 @@ func TestSum(t *testing.T) {
|
||||
|
||||
all := statsAt(0, statsNow)
|
||||
app := all.Sum("", "/work/app")
|
||||
if app.Tokens != (Tokens{1110, 170, 5200, 1000}) || app.Active != 300 || len(app.Folders) != 2 {
|
||||
if app.Tokens != (Tokens{1110, 170, 5200, 1000, 0}) || app.Active != 300 || len(app.Folders) != 2 {
|
||||
t.Fatalf("app: %+v", app)
|
||||
}
|
||||
opus := all.Sum("claude-opus-5-5", "")
|
||||
@@ -212,7 +212,7 @@ func TestStatsHoursAndSessions(t *testing.T) {
|
||||
}
|
||||
x := find(s)
|
||||
if x == nil || x.Agent != "claude" || x.Cwd != "/work/night" || x.Active != 420 ||
|
||||
x.Tokens != (Tokens{700, 70, 30, 0}) || !x.Priced || x.Cost <= 0 ||
|
||||
x.Tokens != (Tokens{700, 70, 30, 0, 0}) || !x.Priced || x.Cost <= 0 ||
|
||||
len(x.Models) != 1 || len(x.Days) != 2 || x.Days[0] != 0 || x.Days[1] != 1 {
|
||||
t.Fatalf("summary %+v", x)
|
||||
}
|
||||
@@ -221,7 +221,7 @@ func TestStatsHoursAndSessions(t *testing.T) {
|
||||
}
|
||||
// a range counts only its own days
|
||||
day := statsAt(1, time.Date(2026, 9, 21, 12, 0, 0, 0, time.UTC))
|
||||
if x := find(day); x == nil || x.Tokens != (Tokens{400, 40, 10, 0}) || x.Active != 240 || len(x.Days) != 1 || x.Days[0] != 0 {
|
||||
if x := find(day); x == nil || x.Tokens != (Tokens{400, 40, 10, 0, 0}) || x.Active != 240 || len(x.Days) != 1 || x.Days[0] != 0 {
|
||||
t.Fatalf("the 21st alone: %+v", x)
|
||||
}
|
||||
if len(day.Sessions) != 1 {
|
||||
@@ -229,7 +229,7 @@ func TestStatsHoursAndSessions(t *testing.T) {
|
||||
}
|
||||
|
||||
// Get reads a session by its key, as Find does past the listed ones
|
||||
if g, ok := Get(key); !ok || g.Cwd != "/work/night" || g.Tokens != (Tokens{700, 70, 30, 0}) {
|
||||
if g, ok := Get(key); !ok || g.Cwd != "/work/night" || g.Tokens != (Tokens{700, 70, 30, 0, 0}) {
|
||||
t.Fatalf("get %+v %v", g, ok)
|
||||
}
|
||||
if _, ok := Get("claude:nope"); ok {
|
||||
|
||||
@@ -588,7 +588,7 @@ func (c *traceCursor) pi(line []byte, bodies bool) []TraceSpan {
|
||||
if json.Unmarshal(line, &usage) == nil {
|
||||
s := c.span(id, o.ID, "model "+usage.Model+" ("+usage.Purpose+")", "generation", o.Timestamp, o.Timestamp)
|
||||
s.Model, s.Provider, s.Inferred = usage.Model, usage.Provider, true
|
||||
s.Tokens = Tokens{usage.Usage.Input, usage.Usage.Output, usage.Usage.CacheRead, usage.Usage.CacheWrite}
|
||||
s.Tokens = Tokens{Input: usage.Usage.Input, Output: usage.Usage.Output, CacheRead: usage.Usage.CacheRead, CacheWrite: usage.Usage.CacheWrite}
|
||||
return []TraceSpan{s}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -189,7 +189,7 @@ func (c *traceCursor) claude(line []byte, bodies bool) []TraceSpan {
|
||||
}
|
||||
m := c.claudeMessage
|
||||
m.span.End = o.Timestamp
|
||||
m.span.Tokens = Tokens{o.Message.Usage.Input, o.Message.Usage.Output, o.Message.Usage.CacheRead, o.Message.Usage.CacheWrite}
|
||||
m.span.Tokens = Tokens{Input: o.Message.Usage.Input, Output: o.Message.Usage.Output, CacheRead: o.Message.Usage.CacheRead, CacheWrite: o.Message.Usage.CacheWrite}
|
||||
m.span.Error = o.IsApiErrorMessage
|
||||
m.stop = o.Message.StopReason
|
||||
if bodies {
|
||||
|
||||
@@ -211,7 +211,7 @@ func traceOpenCode(sid string, messages []ocTraceMessage, bodies bool) []TraceSp
|
||||
s := c.span(t.id, m.ID, "model "+m.ModelID, "generation", ms(m.Time.Created), end)
|
||||
s.Model, s.Provider, s.Error = m.ModelID, m.ProviderID, failed
|
||||
if u := m.Tokens; u != nil {
|
||||
s.Tokens = Tokens{u.Input, u.Output + u.Reasoning, u.Cache.Read, u.Cache.Write}
|
||||
s.Tokens = Tokens{Input: u.Input, Output: u.Output + u.Reasoning, CacheRead: u.Cache.Read, CacheWrite: u.Cache.Write}
|
||||
s.Reasoning = u.Reasoning
|
||||
}
|
||||
if bodies {
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"math"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/catalog"
|
||||
)
|
||||
|
||||
// A price with a long-context tier and a 1-hour cache write is kept and read
|
||||
// back whole; one written before either reads as before, its 1-hour write
|
||||
// unset (2× input); a tier not above the one before it, or with a bad part,
|
||||
// is refused by name.
|
||||
func TestModelPriceTiers(t *testing.T) {
|
||||
p := catalog.Price{Input: 10, Output: 50, CacheRead: 1, CacheWrite: 12.5, CacheWrite1h: 20,
|
||||
Tiers: []catalog.Tier{{Above: 272000, Input: 20, Output: 75, CacheRead: 2, CacheWrite: 25}}}
|
||||
b, err := json.Marshal(StatedPrice(p))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var back ModelPrice
|
||||
if err := json.Unmarshal(b, &back); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, bad := back.Price(); bad != "" || !got.Same(p) {
|
||||
t.Errorf("read back %s as %+v, %q; want %+v", b, got, bad, p)
|
||||
}
|
||||
|
||||
var old ModelPrice
|
||||
if err := json.Unmarshal([]byte(`{"input":1,"output":2,"cache_read":0.1,"cache_write":0}`), &old); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, bad := old.Price(); bad != "" || !got.Same(catalog.Price{Input: 1, Output: 2, CacheRead: 0.1}) || got.OneHour() != 2 {
|
||||
t.Errorf("an old price = %+v, %q", got, bad)
|
||||
}
|
||||
if b, _ := json.Marshal(StatedPrice(catalog.Price{Input: 1, Output: 2})); strings.Contains(string(b), "cache_write_1h") || strings.Contains(string(b), "tiers") {
|
||||
t.Errorf("a price without either is written %s, as before", b)
|
||||
}
|
||||
|
||||
for name, m := range map[string]ModelPrice{
|
||||
"tier 2": {Tiers: []catalog.Tier{{Above: 272000, Input: 1}, {Above: 272000, Input: 2}}},
|
||||
"tier 1": {Tiers: []catalog.Tier{{Above: 0, Input: 1}}},
|
||||
"tier 1 ": {Tiers: []catalog.Tier{{Above: 1000, Input: math.NaN()}}},
|
||||
"1-hour cache write": {CacheWrite1h: new(-1.0)},
|
||||
} {
|
||||
m.Input, m.Output, m.CacheRead, m.CacheWrite = new(1.0), new(2.0), new(0.0), new(0.0)
|
||||
if _, bad := m.Price(); bad != strings.TrimSpace(name) {
|
||||
t.Errorf("%s: bad = %q", name, bad)
|
||||
}
|
||||
if err := CheckModelPrice("relay/x", m); err == nil {
|
||||
t.Errorf("%s: no error", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -372,6 +372,12 @@ type ModelPrice struct {
|
||||
Output *float64 `json:"output,omitempty"`
|
||||
CacheRead *float64 `json:"cache_read,omitempty"`
|
||||
CacheWrite *float64 `json:"cache_write,omitempty"`
|
||||
// CacheWrite1h is a 1-hour cache write's price, absent where none is
|
||||
// given: such a write is then counted at 2× input (catalog.Price).
|
||||
CacheWrite1h *float64 `json:"cache_write_1h,omitempty"`
|
||||
// Tiers are the prices of a request whose input is over a size, each
|
||||
// whole: "magpie model price … --tier 272k …" (PAMI on Discord).
|
||||
Tiers []catalog.Tier `json:"tiers,omitempty"`
|
||||
}
|
||||
|
||||
// priceParts are the parts of a price, in the order they are asked for and
|
||||
@@ -391,10 +397,35 @@ func (m ModelPrice) Price() (catalog.Price, string) {
|
||||
return catalog.Price{}, priceParts[i]
|
||||
}
|
||||
}
|
||||
return catalog.Price{
|
||||
bad := func(v float64) bool { return math.IsNaN(v) || math.IsInf(v, 0) || v < 0 }
|
||||
if m.CacheWrite1h != nil && bad(*m.CacheWrite1h) {
|
||||
return catalog.Price{}, "1-hour cache write"
|
||||
}
|
||||
for i, t := range m.Tiers {
|
||||
if t.Above <= 0 || i > 0 && t.Above <= m.Tiers[i-1].Above ||
|
||||
bad(t.Input) || bad(t.Output) || bad(t.CacheRead) || bad(t.CacheWrite) || bad(t.CacheWrite1h) {
|
||||
return catalog.Price{}, fmt.Sprintf("tier %d", i+1)
|
||||
}
|
||||
}
|
||||
p := catalog.Price{
|
||||
Input: *m.Input, Output: *m.Output,
|
||||
CacheRead: *m.CacheRead, CacheWrite: *m.CacheWrite,
|
||||
}, ""
|
||||
Tiers: slices.Clone(m.Tiers),
|
||||
}
|
||||
if m.CacheWrite1h != nil {
|
||||
p.CacheWrite1h = *m.CacheWrite1h
|
||||
}
|
||||
return p, ""
|
||||
}
|
||||
|
||||
// StatedPrice is a price as ModelPrices keeps it: every part given, the
|
||||
// 1-hour cache write only when it is.
|
||||
func StatedPrice(p catalog.Price) ModelPrice {
|
||||
m := ModelPrice{Input: new(p.Input), Output: new(p.Output), CacheRead: new(p.CacheRead), CacheWrite: new(p.CacheWrite), Tiers: slices.Clone(p.Tiers)}
|
||||
if p.CacheWrite1h > 0 {
|
||||
m.CacheWrite1h = new(p.CacheWrite1h)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// CheckModelPrice is whether a price the user gives is one magpie will bill a
|
||||
@@ -414,7 +445,9 @@ func CheckModelPrice(key string, m ModelPrice) error {
|
||||
} else if err := CheckModelKey("price", key); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, bad := m.Price(); bad != "" {
|
||||
if _, bad := m.Price(); strings.HasPrefix(bad, "tier ") {
|
||||
return fmt.Errorf("the price of %q has a %s that is not over a size above 0 and above the tier before it, with every part finite and not negative", key, bad)
|
||||
} else if bad != "" {
|
||||
return fmt.Errorf("the price of %q needs %s %s price that is present, finite and not negative", key, anArticle(bad), bad)
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -233,7 +233,7 @@ const UnknownProvider = "session-unknown"
|
||||
// model it was sent to, never the one the response named) (#680).
|
||||
func logRecord(c sessions.Call) Record {
|
||||
r := Record{Time: c.Time, Agent: c.Agent, Provider: UnknownProvider, Model: c.Model, Served: c.Model, Requested: c.Requested,
|
||||
Input: c.Input, Output: c.Output, CacheRead: c.CacheRead, CacheWrite: c.CacheWrite, Reasoning: c.Reasoning, Effort: c.Effort,
|
||||
Input: c.Input, Output: c.Output, CacheRead: c.CacheRead, CacheWrite: c.CacheWrite, CacheWrite1h: c.CacheWrite1h, Reasoning: c.Reasoning, Effort: c.Effort,
|
||||
Millis: c.Millis, TTFT: c.TTFT, Session: c.Session, RequestID: c.RequestID, Error: c.ErrorText, ErrType: c.Error}
|
||||
if c.Agent == "codex" || c.Agent == "opencode" {
|
||||
r.Requested, r.Served = c.Model, ""
|
||||
@@ -302,7 +302,7 @@ func ledgerWithShared(since time.Time, f Filter, recs []Record, logs []sessions.
|
||||
}
|
||||
row := Row{Record: r, Swapped: r.Served != "" && Swapped(sent, r.Served), Routed: GroupRouted(sent, r.Served), Source: source}
|
||||
if pr != nil && r.Input+r.Output > 0 {
|
||||
row.Cost, row.Priced = pr.Cost(r.Input, r.Output, r.CacheRead, r.CacheWrite), true
|
||||
row.Cost, row.Priced = r.CostAt(*pr), true
|
||||
}
|
||||
row.Agent = AgentOf(r.Agent)
|
||||
rows = append(rows, row)
|
||||
@@ -390,9 +390,12 @@ func NewPricer() func([]Record) Totals {
|
||||
var CSVHeader = []string{"time", "agent", "requested_model", "provider", "host", "model", "served_model", "swapped",
|
||||
"effort", "input_tokens", "output_tokens", "cache_write_tokens", "cache_read_tokens", "reasoning_tokens",
|
||||
"cost_usd", "duration_ms", "ttft_ms", "status", "error", "session", "kind", "provider_key_id", "provider_key_name", "provider_account", "route_id",
|
||||
"request_id", "endpoint", "error_message", "error_type", "source", "rejected", "session_provider", "session_account", "session_official_login", "caller_key_id", "caller_key_name", "response_id", "upstream"}
|
||||
"request_id", "endpoint", "error_message", "error_type", "source", "rejected", "session_provider", "session_account", "session_official_login", "caller_key_id", "caller_key_name", "response_id", "upstream",
|
||||
"cache_write_5m_tokens", "cache_write_1h_tokens"}
|
||||
|
||||
// WriteCSV writes rows as CSV, a header first: times in RFC 3339 with
|
||||
// WriteCSV writes rows as CSV, a header first (cache_write_tokens is every
|
||||
// cache write, as it always was, and the last two columns split it into
|
||||
// 5-minute and 1-hour writes): times in RFC 3339 with
|
||||
// their offset, the cost in USD at the effective price (empty when unknown), error
|
||||
// "true" for a call that failed: answered with a status of 400 or more, or
|
||||
// ended by an error a session file tells.
|
||||
@@ -416,7 +419,8 @@ func WriteCSV(w io.Writer, rows []Row) error {
|
||||
cw.Write([]string{r.Time.Format(time.RFC3339), r.Agent, r.Requested, r.Provider, r.Host, r.Model, r.Served,
|
||||
strconv.FormatBool(r.Swapped), r.Effort, n(r.Input), n(r.Output), n(r.CacheWrite), n(r.CacheRead), n(r.Reasoning),
|
||||
cost, strconv.FormatInt(r.Millis, 10), ttft, n(r.Status), strconv.FormatBool(r.Failed()), r.Session, r.Kind, r.ProviderKeyID, r.ProviderKeyName, r.Account(), routeID,
|
||||
r.RequestID, r.Endpoint, r.Error, r.ErrType, r.Source, strconv.FormatBool(r.IsRejected()), r.SessionProvider, r.SessionAccount, strconv.FormatBool(r.SessionOfficialLogin), r.CallerKeyID, r.CallerKeyName, r.ResponseID, r.Upstream})
|
||||
r.RequestID, r.Endpoint, r.Error, r.ErrType, r.Source, strconv.FormatBool(r.IsRejected()), r.SessionProvider, r.SessionAccount, strconv.FormatBool(r.SessionOfficialLogin), r.CallerKeyID, r.CallerKeyName, r.ResponseID, r.Upstream,
|
||||
n(r.CacheWrite - min(r.CacheWrite1h, r.CacheWrite)), n(min(r.CacheWrite1h, r.CacheWrite))})
|
||||
}
|
||||
cw.Flush()
|
||||
return cw.Error()
|
||||
@@ -506,6 +510,7 @@ func (t *Totals) addRow(r Row) {
|
||||
t.Output += r.Output
|
||||
t.CacheRead += r.CacheRead
|
||||
t.CacheWrite += r.CacheWrite
|
||||
t.CacheWrite1h += r.CacheWrite1h
|
||||
t.Reasoning += r.Reasoning
|
||||
switch {
|
||||
case r.Priced:
|
||||
|
||||
@@ -90,8 +90,8 @@ func TestLedger(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := strings.Join(CSVHeader, ",") + "\n" +
|
||||
rows[1].Time.Format(time.RFC3339) + ",codex,relay/sol,relay,relay.example,sol,sol-2026-01-01,false,,10,1,0,0,0,0.000028,100,,200,false,,,,,,,,,,,,false,,,false,,,,\n" +
|
||||
rows[2].Time.Format(time.RFC3339) + ",codex,fast,relay,relay.example,sol,luna,true,high,2000,500,1000,4000,0,0.012500,3200,400,200,false,s1,,,,,123,,,,,,false,,,false,,,,\n"
|
||||
rows[1].Time.Format(time.RFC3339) + ",codex,relay/sol,relay,relay.example,sol,sol-2026-01-01,false,,10,1,0,0,0,0.000028,100,,200,false,,,,,,,,,,,,false,,,false,,,,,0,0\n" +
|
||||
rows[2].Time.Format(time.RFC3339) + ",codex,fast,relay,relay.example,sol,luna,true,high,2000,500,1000,4000,0,0.012500,3200,400,200,false,s1,,,,,123,,,,,,false,,,false,,,,,1000,0\n"
|
||||
if b.String() != want {
|
||||
t.Fatalf("csv:\n%s\nwant:\n%s", b.String(), want)
|
||||
}
|
||||
@@ -260,7 +260,7 @@ func TestLedgerWithSessionLogCalls(t *testing.T) {
|
||||
if err := WriteCSV(&b, rows[2:4]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{",req_log,,,,log,false,,,false,,,,\n", ",true,s2,,,,,,req_lim,,You've hit your limit,rate_limit,log,false,,,false,,,,\n"} {
|
||||
for _, want := range []string{",req_log,,,,log,false,,,false,,,,,500,0\n", ",true,s2,,,,,,req_lim,,You've hit your limit,rate_limit,log,false,,,false,,,,,0,0\n"} {
|
||||
if !strings.Contains(b.String(), want) {
|
||||
t.Fatalf("csv lacks %q:\n%s", want, b.String())
|
||||
}
|
||||
|
||||
@@ -91,7 +91,7 @@ func sessionRecord(s sessions.TraceSpan, config settings.OTel) Record {
|
||||
}
|
||||
typ := s.Kind
|
||||
return Record{Agent: s.Agent, Model: s.Model, Served: s.Served, Provider: s.Provider, Time: s.Start, Millis: s.End.Sub(s.Start).Milliseconds(), Status: status,
|
||||
Input: s.Tokens.Input, Output: s.Tokens.Output, CacheRead: s.Tokens.CacheRead, CacheWrite: s.Tokens.CacheWrite, Reasoning: s.Reasoning,
|
||||
Input: s.Tokens.Input, Output: s.Tokens.Output, CacheRead: s.Tokens.CacheRead, CacheWrite: s.Tokens.CacheWrite, CacheWrite1h: s.Tokens.CacheWrite1h, Reasoning: s.Reasoning,
|
||||
BodyIn: sessionBody(s.Input, config), BodyOut: sessionBody(s.Output, config),
|
||||
OTel: &OTelSpan{TraceID: sessions.TraceID(s.Agent, s.Session, s.Turn), SpanID: s.ID, ParentID: s.Parent, Root: s.Parent == "", End: s.End,
|
||||
Name: s.Name, Type: typ, SessionID: s.Session, TraceName: s.Agent + " interaction", Session: true, Inferred: s.Inferred, Update: s.Update}}
|
||||
|
||||
@@ -171,10 +171,12 @@ func (e *otelExporter) traces(records []Record) any {
|
||||
generation := r.OTel == nil || !r.OTel.Root && (!r.OTel.Session || r.OTel.Type == "generation")
|
||||
if generation && !r.IsRejected() {
|
||||
if pr := priceOf(r); pr != nil {
|
||||
// the call's tier, by its whole input, for both halves
|
||||
f := pr.At(r.Input + r.CacheRead + r.CacheWrite)
|
||||
cost, err := json.Marshal(map[string]float64{
|
||||
"input": pr.Cost(r.Input, 0, r.CacheRead, r.CacheWrite),
|
||||
"output": pr.Cost(0, r.Output, 0, 0),
|
||||
"total": pr.Cost(r.Input, r.Output, r.CacheRead, r.CacheWrite),
|
||||
"input": f.CostSplit(r.Input, 0, r.CacheRead, r.CacheWrite, r.CacheWrite1h),
|
||||
"output": f.Cost(0, r.Output, 0, 0),
|
||||
"total": r.CostAt(*pr),
|
||||
})
|
||||
if err == nil {
|
||||
a = append(a, otelString("langfuse.observation.cost_details", string(cost)))
|
||||
|
||||
@@ -52,7 +52,7 @@ type RequestPage struct {
|
||||
type packedRow struct {
|
||||
Time time.Time
|
||||
Text [28]uint32
|
||||
Tokens [5]int64
|
||||
Tokens [6]int64
|
||||
Millis, TTFT, FirstText, Order int64
|
||||
Sent int64
|
||||
RouteID int64
|
||||
@@ -96,7 +96,7 @@ func (c *rowChunk) add(r Row, msg string, order int64, failed bool) {
|
||||
c.Bytes += int64(len(s) + 48)
|
||||
return id
|
||||
}
|
||||
p := packedRow{Time: r.Time, Tokens: [5]int64{int64(r.Input), int64(r.Output), int64(r.CacheRead), int64(r.CacheWrite), int64(r.Reasoning)}, Millis: r.Millis, TTFT: r.TTFT, FirstText: r.FirstText, Sent: r.Sent, Order: order, RouteID: r.RouteID, Cost: r.Cost, Status: int32(r.Status)}
|
||||
p := packedRow{Time: r.Time, Tokens: [6]int64{int64(r.Input), int64(r.Output), int64(r.CacheRead), int64(r.CacheWrite), int64(r.Reasoning), int64(r.CacheWrite1h)}, Millis: r.Millis, TTFT: r.TTFT, FirstText: r.FirstText, Sent: r.Sent, Order: order, RouteID: r.RouteID, Cost: r.Cost, Status: int32(r.Status)}
|
||||
for i, s := range rowText(&r) {
|
||||
p.Text[i] = intern(*s)
|
||||
}
|
||||
@@ -128,7 +128,7 @@ func (c *rowChunk) add(r Row, msg string, order int64, failed bool) {
|
||||
}
|
||||
func (c *rowChunk) row(i int) Row {
|
||||
p := &c.Rows[i]
|
||||
r := Row{Record: Record{RouteID: p.RouteID, Time: p.Time, Input: int(p.Tokens[0]), Output: int(p.Tokens[1]), CacheRead: int(p.Tokens[2]), CacheWrite: int(p.Tokens[3]), Reasoning: int(p.Tokens[4]), Millis: p.Millis, TTFT: p.TTFT, FirstText: p.FirstText, Sent: p.Sent, Status: int(p.Status), Rejected: p.Flags&4 != 0, SessionOfficialLogin: p.Flags&8 != 0}, Cost: p.Cost, Priced: p.Flags&1 != 0, Swapped: p.Flags&2 != 0, Routed: p.Flags&32 != 0}
|
||||
r := Row{Record: Record{RouteID: p.RouteID, Time: p.Time, Input: int(p.Tokens[0]), Output: int(p.Tokens[1]), CacheRead: int(p.Tokens[2]), CacheWrite: int(p.Tokens[3]), CacheWrite1h: int(p.Tokens[5]), Reasoning: int(p.Tokens[4]), Millis: p.Millis, TTFT: p.TTFT, FirstText: p.FirstText, Sent: p.Sent, Status: int(p.Status), Rejected: p.Flags&4 != 0, SessionOfficialLogin: p.Flags&8 != 0}, Cost: p.Cost, Priced: p.Flags&1 != 0, Swapped: p.Flags&2 != 0, Routed: p.Flags&32 != 0}
|
||||
for i, s := range rowText(&r) {
|
||||
*s = c.Strings[p.Text[i]]
|
||||
}
|
||||
@@ -296,7 +296,7 @@ func queryPage(p Period, f Filter, offset, limit int, readSource func(sessions.C
|
||||
row := Row{Record: r, Source: source, Swapped: r.Served != "" && Swapped(sent, r.Served), Routed: GroupRouted(sent, r.Served)}
|
||||
if pr := price(r); pr != nil && r.Input+r.Output > 0 {
|
||||
row.Priced = true
|
||||
row.Cost = pr.Cost(r.Input, r.Output, r.CacheRead, r.CacheWrite)
|
||||
row.Cost = r.CostAt(*pr)
|
||||
}
|
||||
row.Agent = AgentOf(r.Agent)
|
||||
return row
|
||||
@@ -561,7 +561,7 @@ type matchKey struct {
|
||||
// Codex that names cache_write_input_tokens beside a gateway record that
|
||||
// didn't, an older Codex that doesn't beside one that does; the sum is the
|
||||
// same either way.
|
||||
func matchTokens(t [5]int64) [3]int64 { return [3]int64{t[0] + t[3], t[1], t[2]} }
|
||||
func matchTokens(t [6]int64) [3]int64 { return [3]int64{t[0] + t[3], t[1], t[2]} }
|
||||
|
||||
type matchEnd struct {
|
||||
at time.Time
|
||||
|
||||
+26
-11
@@ -60,7 +60,13 @@ type Record struct {
|
||||
Output int `json:"out"`
|
||||
CacheRead int `json:"cache_read,omitempty"`
|
||||
CacheWrite int `json:"cache_write,omitempty"`
|
||||
Reasoning int `json:"reasoning,omitempty"`
|
||||
// CacheWrite1h is how many of CacheWrite were written to be kept for
|
||||
// an hour (Anthropic's 1-hour TTL, billed at 2× input where 5 minutes'
|
||||
// is 1.25×); the rest were for 5 minutes. A record from before it was
|
||||
// kept has none: all its writes are counted as 5-minute ones, as they
|
||||
// were.
|
||||
CacheWrite1h int `json:"cache_write_1h,omitempty"`
|
||||
Reasoning int `json:"reasoning,omitempty"`
|
||||
// Effort is the reasoning the model was asked for — a routing group's
|
||||
// pick for the turn, or the agent's own — as it takes it; "" for none
|
||||
Effort string `json:"effort,omitempty"`
|
||||
@@ -245,17 +251,25 @@ const (
|
||||
// Periods lists the windows in order.
|
||||
var Periods = []Period{Today, Week, Month, All}
|
||||
|
||||
// CostAt is what the call costs at a price: at the tier its input reaches,
|
||||
// its 1-hour cache writes at theirs.
|
||||
func (r Record) CostAt(p catalog.Price) float64 {
|
||||
return p.CostSplit(r.Input, r.Output, r.CacheRead, r.CacheWrite, r.CacheWrite1h)
|
||||
}
|
||||
|
||||
// Totals is a sum of calls.
|
||||
type Totals struct {
|
||||
Calls int `json:"calls"`
|
||||
Errors int `json:"errors"`
|
||||
Input int `json:"input"`
|
||||
Output int `json:"output"`
|
||||
CacheRead int `json:"cache_read"`
|
||||
CacheWrite int `json:"cache_write"`
|
||||
Reasoning int `json:"reasoning"`
|
||||
Cost float64 `json:"cost"` // USD at the effective price, for the priced calls
|
||||
Unpriced int `json:"unpriced"` // calls with tokens but no known price
|
||||
Calls int `json:"calls"`
|
||||
Errors int `json:"errors"`
|
||||
Input int `json:"input"`
|
||||
Output int `json:"output"`
|
||||
CacheRead int `json:"cache_read"`
|
||||
CacheWrite int `json:"cache_write"`
|
||||
// CacheWrite1h is how many of CacheWrite were 1-hour writes
|
||||
CacheWrite1h int `json:"cache_write_1h,omitempty"`
|
||||
Reasoning int `json:"reasoning"`
|
||||
Cost float64 `json:"cost"` // USD at the effective price, for the priced calls
|
||||
Unpriced int `json:"unpriced"` // calls with tokens but no known price
|
||||
// Timed: the answered calls whose first token was timed (streamed),
|
||||
// TTFT the sum of their ttft_ms; DecodeMs the time from it to the end
|
||||
// of those that wrote any, over which DecodeOut tokens came: their
|
||||
@@ -346,6 +360,7 @@ func (t *Totals) add(r Record, price *catalog.Price) {
|
||||
t.Output += r.Output
|
||||
t.CacheRead += r.CacheRead
|
||||
t.CacheWrite += r.CacheWrite
|
||||
t.CacheWrite1h += r.CacheWrite1h
|
||||
t.Reasoning += r.Reasoning
|
||||
if r.TTFT > 0 && !r.Failed() {
|
||||
t.Timed++
|
||||
@@ -362,7 +377,7 @@ func (t *Totals) add(r Record, price *catalog.Price) {
|
||||
t.Unpriced++
|
||||
return
|
||||
}
|
||||
t.Cost += price.Cost(r.Input, r.Output, r.CacheRead, r.CacheWrite)
|
||||
t.Cost += r.CostAt(*price)
|
||||
}
|
||||
|
||||
// Group is the share of one agent or model.
|
||||
|
||||
+98
-15
@@ -26,7 +26,11 @@ const modelUsage = `usage:
|
||||
magpie model price <provider/model> <in>,<out>,<cache read>,<cache write>
|
||||
say what it costs, in USD per million tokens, all four parts as
|
||||
0.12,1.20,0.01,0.15; 0 is a model served for nothing, which is
|
||||
a price, not the absence of one
|
||||
a price, not the absence of one; a fifth part is a 1-hour cache
|
||||
write's (Anthropic's 2× input, what it is counted at if not given)
|
||||
magpie model price <provider/model> <in>,<out>,<cache read>,<cache write> --tier 272k <in>,<out>,<cr>,<cw>
|
||||
and what a request whose input, cached tokens included, is over
|
||||
272K costs, the whole request; --tier may be given again
|
||||
magpie model price <provider/model> --reset take your price off this model
|
||||
magpie model price <model> <in>,<out>,<cache read>,<cache write>
|
||||
what the model costs from any provider you have not priced it
|
||||
@@ -287,7 +291,9 @@ func modelPrice(args []string) error {
|
||||
return nil
|
||||
}
|
||||
fmt.Println(bold.Render(perMillion(pr)), muted.Render("· "+id))
|
||||
fmt.Println(faint.Render(" · cache read " + money(pr.CacheRead) + ", cache write " + money(pr.CacheWrite)))
|
||||
for _, l := range priceDetail(pr) {
|
||||
fmt.Println(faint.Render(l))
|
||||
}
|
||||
switch priceFrom(settings.Load(), p.ID, model) {
|
||||
case "model":
|
||||
fmt.Println(faint.Render(" · what you said this model costs · --reset takes that away"))
|
||||
@@ -313,26 +319,98 @@ func modelPrice(args []string) error {
|
||||
return err
|
||||
}
|
||||
fmt.Println(green.Render("✓"), id, muted.Render("costs"), bold.Render(perMillion(pr)))
|
||||
for _, t := range pr.Tiers {
|
||||
fmt.Println(faint.Render(fmt.Sprintf(" · and $%.4g/$%.4g in/out the whole request over %s input tokens", t.Input, t.Output, tokenSize(t.Above))))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// parsePrice is the four parts of a price as typed, in USD per million
|
||||
// tokens.
|
||||
// parsePrice is a price as typed, in USD per million tokens: its four
|
||||
// parts, a 1-hour cache write's after them if given, and then each
|
||||
// "--tier <size>" with the parts of what a request whose input is over that
|
||||
// size costs.
|
||||
func parsePrice(rest []string) (catalog.Price, error) {
|
||||
var nums []float64
|
||||
for _, a := range rest {
|
||||
for _, f := range strings.FieldsFunc(a, func(r rune) bool { return r == ',' || r == ' ' || r == '/' }) {
|
||||
v, err := strconv.ParseFloat(f, 64)
|
||||
if err != nil {
|
||||
return catalog.Price{}, fmt.Errorf("a price is numbers in USD per million tokens, like 0.12,1.20,0.01,0.15, not %q", f)
|
||||
var groups [][]string
|
||||
var sizes []string
|
||||
cur := []string{}
|
||||
for i := 0; i < len(rest); i++ {
|
||||
a := rest[i]
|
||||
size, isTier := strings.CutPrefix(a, "--tier=")
|
||||
if !isTier && a == "--tier" {
|
||||
if i+1 >= len(rest) {
|
||||
return catalog.Price{}, fmt.Errorf("--tier takes the input size it starts over, like --tier 272k 20,75,2,25")
|
||||
}
|
||||
nums = append(nums, v)
|
||||
isTier, size = true, rest[i+1]
|
||||
i++
|
||||
}
|
||||
if isTier {
|
||||
groups, sizes, cur = append(groups, cur), append(sizes, size), []string{}
|
||||
continue
|
||||
}
|
||||
cur = append(cur, a)
|
||||
}
|
||||
if len(nums) != 4 {
|
||||
return catalog.Price{}, fmt.Errorf("give all four parts, input,output,cache read,cache write — %d given", len(nums))
|
||||
groups = append(groups, cur)
|
||||
var out catalog.Price
|
||||
for g, args := range groups {
|
||||
var nums []float64
|
||||
for _, a := range args {
|
||||
for _, f := range strings.FieldsFunc(a, func(r rune) bool { return r == ',' || r == ' ' || r == '/' }) {
|
||||
v, err := strconv.ParseFloat(f, 64)
|
||||
if err != nil {
|
||||
return catalog.Price{}, fmt.Errorf("a price is numbers in USD per million tokens, like 0.12,1.20,0.01,0.15, not %q", f)
|
||||
}
|
||||
nums = append(nums, v)
|
||||
}
|
||||
}
|
||||
if len(nums) != 4 && len(nums) != 5 {
|
||||
what := "the price"
|
||||
if g > 0 {
|
||||
what = "the tier over " + sizes[g-1]
|
||||
}
|
||||
return catalog.Price{}, fmt.Errorf("give all four parts of %s, input,output,cache read,cache write (and a 1-hour cache write after them if you like) — %d given", what, len(nums))
|
||||
}
|
||||
var oneHour float64
|
||||
if len(nums) == 5 {
|
||||
oneHour = nums[4]
|
||||
}
|
||||
if g == 0 {
|
||||
out = catalog.Price{Input: nums[0], Output: nums[1], CacheRead: nums[2], CacheWrite: nums[3], CacheWrite1h: oneHour}
|
||||
continue
|
||||
}
|
||||
above, err := provider.ParseTokens(sizes[g-1])
|
||||
if err != nil || above <= 0 {
|
||||
return catalog.Price{}, fmt.Errorf("--tier takes the input size it starts over, like 272k or 200000, not %q", sizes[g-1])
|
||||
}
|
||||
out.Tiers = append(out.Tiers, catalog.Tier{Above: above, Input: nums[0], Output: nums[1], CacheRead: nums[2], CacheWrite: nums[3], CacheWrite1h: oneHour})
|
||||
}
|
||||
return catalog.Price{Input: nums[0], Output: nums[1], CacheRead: nums[2], CacheWrite: nums[3]}, nil
|
||||
slices.SortFunc(out.Tiers, func(a, b catalog.Tier) int { return a.Above - b.Above })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// priceDetail is the parts of a price after the two perMillion gives: its
|
||||
// cache's, and what a request over each of its sizes costs.
|
||||
func priceDetail(p catalog.Price) []string {
|
||||
cache := " · cache read " + money(p.CacheRead) + ", cache write " + money(p.CacheWrite) + " (5 min), " + money(p.OneHour()) + " (1 hour)"
|
||||
if p.CacheWrite1h <= 0 {
|
||||
cache += " at 2× input"
|
||||
}
|
||||
lines := []string{cache}
|
||||
for _, t := range p.Tiers {
|
||||
lines = append(lines, " · over "+tokenSize(t.Above)+" input tokens, the whole request: "+
|
||||
fmt.Sprintf("$%.4g/$%.4g in/out", t.Input, t.Output)+", cache "+money(t.CacheRead)+"/"+money(t.CacheWrite))
|
||||
}
|
||||
return lines
|
||||
}
|
||||
|
||||
// tokenSize is a number of tokens as --tier takes it: 272K, 1M.
|
||||
func tokenSize(n int) string {
|
||||
switch {
|
||||
case n >= 1e6 && n%1e6 == 0:
|
||||
return strconv.Itoa(n/1e6) + "M"
|
||||
case n >= 1e3 && n%1e3 == 0:
|
||||
return strconv.Itoa(n/1e3) + "K"
|
||||
}
|
||||
return strconv.Itoa(n)
|
||||
}
|
||||
|
||||
// anyModelPrice shows or sets what a model costs from any provider (*/model):
|
||||
@@ -347,7 +425,9 @@ func anyModelPrice(ref string, rest []string) error {
|
||||
if len(rest) == 0 {
|
||||
if pr, ok := statedPrice(settings.Load().ModelPrices, key); ok {
|
||||
fmt.Println(bold.Render(perMillion(pr)), muted.Render("· "+key))
|
||||
fmt.Println(faint.Render(" · cache read " + money(pr.CacheRead) + ", cache write " + money(pr.CacheWrite)))
|
||||
for _, l := range priceDetail(pr) {
|
||||
fmt.Println(faint.Render(l))
|
||||
}
|
||||
fmt.Println(faint.Render(" · what you said this model costs from any provider · --reset takes that away"))
|
||||
return nil
|
||||
}
|
||||
@@ -480,6 +560,9 @@ func modelPrices() error {
|
||||
}
|
||||
fmt.Println(bold.Render(id), muted.Render("· "+perMillion(p)+
|
||||
" · cache "+money(p.CacheRead)+"/"+money(p.CacheWrite)))
|
||||
for _, t := range p.Tiers {
|
||||
fmt.Println(faint.Render(fmt.Sprintf(" · over %s input tokens $%.4g/$%.4g in/out · cache %s/%s", tokenSize(t.Above), t.Input, t.Output, money(t.CacheRead), money(t.CacheWrite))))
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/catalog"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
)
|
||||
|
||||
// PAMI on Discord: magpie model price m 10,50,1,12.5 --tier 272k 20,75,2,25
|
||||
// gives what the whole request costs over 272K input, as OpenAI bills
|
||||
// gpt-6-astra, and a fifth part is the 1-hour cache write. Both are kept,
|
||||
// counted and shown; a size that isn't one, or a tier short of a part, is
|
||||
// refused.
|
||||
func TestModelPriceTierCmd(t *testing.T) {
|
||||
groupsHome(t)
|
||||
if err := modelCmd([]string{"price", "a/m", "10,50,1,12.5", "--tier", "272k", "20,75,2,25", "--tier=1M", "40", "100", "4", "50"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pr, ok := provider.EffectivePrice("a", "m")
|
||||
want := catalog.Price{Input: 10, Output: 50, CacheRead: 1, CacheWrite: 12.5, Tiers: []catalog.Tier{
|
||||
{Above: 272000, Input: 20, Output: 75, CacheRead: 2, CacheWrite: 25},
|
||||
{Above: 1000000, Input: 40, Output: 100, CacheRead: 4, CacheWrite: 50}}}
|
||||
if !ok || !pr.Same(want) {
|
||||
t.Fatalf("price = %+v, want %+v", pr, want)
|
||||
}
|
||||
out, err := said(t, func() error { return modelCmd([]string{"price", "a/m"}) })
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(out, "over 272K input tokens") || !strings.Contains(out, "over 1M input tokens") || !strings.Contains(out, "$20/$75") {
|
||||
t.Errorf("magpie model price a/m: %q; want its tiers", out)
|
||||
}
|
||||
out, _ = said(t, func() error { return modelCmd([]string{"prices"}) })
|
||||
if !strings.Contains(out, "over 272K") {
|
||||
t.Errorf("magpie model prices: %q; want its tier", out)
|
||||
}
|
||||
|
||||
if err := modelCmd([]string{"price", "a/only-a", "5,25,0.5,6.25,8"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if pr, _ := provider.EffectivePrice("a", "only-a"); pr.CacheWrite1h != 8 || pr.OneHour() != 8 {
|
||||
t.Fatalf("1h = %+v", pr)
|
||||
}
|
||||
out, _ = said(t, func() error { return modelCmd([]string{"price", "a/only-a"}) })
|
||||
if !strings.Contains(out, "(1 hour)") {
|
||||
t.Errorf("magpie model price a/only-a: %q; want its 1-hour cache write", out)
|
||||
}
|
||||
|
||||
for _, args := range [][]string{
|
||||
{"10,50,1,12.5", "--tier", "lots", "20,75,2,25"},
|
||||
{"10,50,1,12.5", "--tier", "272k", "20,75"},
|
||||
{"10,50,1,12.5", "--tier"},
|
||||
{"10,50,1,12.5", "--tier", "272k", "20,75,2,25", "--tier", "272k", "20,75,2,25"},
|
||||
} {
|
||||
if err := modelCmd(append([]string{"price", "a/m"}, args...)); err == nil {
|
||||
t.Errorf("%q was taken", args)
|
||||
}
|
||||
}
|
||||
if pr, _ := provider.EffectivePrice("a", "m"); !pr.Same(want) {
|
||||
t.Errorf("a refused price was written: %+v", pr)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user