Files
magpie/internal/usage
yetone 71d2da1f42 catalog, usage, gateway, gui: a price can have a long-context tier and a 1-hour cache write, and the price boxes take decimals (PAMI on Discord)
OpenAI bills gpt-6-astra, gpt-6.1-sol and gpt-6-luna over 272K input at 2× input and cache and 1.5× output, for the whole request. A price now has tiers: `magpie model price <m> 10,50,1,12.5 --tier 272k 20,75,2,25` (--tier can repeat). A tier applies when input + cache reads + cache writes is over its size, so 272,000 is not over and 272,001 is. The built-in prices take tiers only where models.dev lists one (cost.tiers of type "context"), and it does list one for those three. The ledger, each request and OTLP are costed at the tier the call reached. Session and day totals sum many calls, so they stay at the base price.

Anthropic bills a cache write kept for an hour at 2× input and one kept for 5 minutes at 1.25×. Its usage reports which is which in cache_creation.ephemeral_5m/1h_input_tokens. The gateway (API and Claude subscription) and Claude Code's transcripts now keep that split. A price's fifth part is the 1-hour write. When it isn't given, it is 2× input: that is Anthropic's rule, and only Anthropic reports 1-hour writes. Claude list prices carry the 2× too. cache_write_tokens is still the total, and the usage CSV adds cache_write_5m_tokens and cache_write_1h_tokens at the end. The app shows the split on a request's Cache and on the usage tooltips. Old prices and old records, which have no 1-hour part, read as before. The sessions cache (18) and the call cache (calls-v8) are re-read once.

In the GUI, the price boxes were number inputs and dropped what they didn't parse ("0,25"). They are now decimal text boxes that take both "0.25" and "0,25". A model's price row has a 1-hour box (empty means 2× input) and a quiet Long-context price row. That row shows the list's tier greyed in, or sits behind a link when the list has none. A bad size is refused by name. en/zh/ja/de.

Tests:
- New: TestPriceTiers, TestCostSplit, TestPriceJSON, TestModelPriceTiers, TestModelPriceTierCmd, TestAnthropicCacheWriteSplit and TestClaudeCacheWriteSplit. The ledger CSV test expects the new columns.
- model-price-tiers.test.cjs is new; model-price-edit and provider-price-rate are updated. All of them fail against the old assets and pass in WebKit. Chromium couldn't be launched here.
- go vet, go build (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui ./... under a temp HOME pass.
2026-10-05 21:56:59 +08:00
..
usage: each subscription account's tokens and cost, by the account that answered (#557: 希望用量账本能按订阅账号统计,每个 Codex / Claude 账号各用了多少 token、多少钱) — a call's record named the upstream key it went out on (ProviderKeyID/Name, #500) but never the subscription account, so with two Codex accounts signed in, or a Claude account that took over after a failover, the ledger could say what Codex cost and not what each account did. A record now carries providerAccount, the account that answered as the Routing trace names it (Account.User: its email or login, never a token), set by the gateway at the try that went out (the candidate loop's last try, so a failover's answer is the account that took over and the refused try stays the first account's; an X-Magpie-Account pin is the pinned one), and on the decide, draw and video paths too. Record.Account() gives it, or for a record written before the field the account its host named (Where() has long written "chatgpt.com as dee@example.com" for an account), and otherwise "" — shown as "account not recorded", never inferred from today's sign-in. The summary groups calls by provider and account (Summary.Accounts, the same email on Codex and on Claude being two accounts), priced as every other group; the ledger's Filter has Account, the search finds a call by its account, the CSV has a provider_account column beside provider_key_name (the provider's identity columns together, the caller's still last), and the Requests page's packed rows keep it and offer the period's accounts. magpie usage prints an accounts table after the provider keys, and magpie usage --csv --account <name> keeps one account's calls. The app's Usage page lists Accounts on the Overview (tokens and cost per account, the provider under its name, an unrecorded one in the faint colour, no stripe down a row's side) and has an Account filter on Requests, each account noting the providers it answered for, which asks for and exports that account's calls without moving the page. OTLP export is an allowlist and does not carry the account. TestUsageNamesPinnedAccount (both pinned accounts on magpie's endpoint and on Codex's own, the trace's Who and the record agreeing, no token in usage.jsonl) and TestUsageNamesAccountThatTookOver (Ordered routing, the first account refusing: the answer recorded as the spare, the refusal as the first) fail without the gateway change; TestSummaryByAccount, TestLedgerByAccount and TestRequestPageByAccount cover the grouping, the host fallback, "not recorded", keyed providers kept out, the filter, the search, the CSV column and the packed rows; TestLedger and TestLedgerWithSessionLogCalls take the new column. usage-accounts.test.cjs (WebKit and Chromium, en and zh: the Overview's accounts and costs, the Account picker's notes, its request and export, All accounts clearing it, scrollTop unchanged) fails without the asset change; api-key-usage, click-scroll, panel-usage, request-archive-usage, tray-usages, claude-usage-asked and every usage-* test pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed once on its TempDir cleanup, passes alone).
2026-10-02 20:15:33 +08:00
archive: a request the archive kept is opened and downloaded from the Usage page, with its bodies whole (jorben, #447: 请求存档 reachable from usage history, full bodies, download) — the archive's name was only on the Gateway page's recent calls, gone once a call left the list or the gateway restarted, and the archive kept the first 256 KB of each body, the recent list's cut. The usage ledger's row now names the call's archive (archive in usage.jsonl, kept through the Requests page's packed rows), and its details on Usage → Requests show the Request archive panel, Fetch from archive and Download; with the archive on, a request it has no copy of says "Not archived". The bodies are kept whole up to requestArchiveMaxMB in settings.json (32 MB unless set, at most 1024): the response is teed to a temp file as it is sent, the request body written to one when the call ends, so the upload queue holds file handles, not bodies, and one upload at a time reads them; a body past the limit is kept to it and marked cut, with its whole size. What goes upstream is unchanged, the archive stays off unless turned on, headers and bodies are scrubbed as before, and the temp files are removed after the upload or when the queue is full. The page is sent a body to show only when it is 256 KB or less (a longer one is said by its size), an archive over 80 MB is only said to be; Download streams the object whole, to Downloads in the app (archive/export, -2, -3… for a name taken) and as the browser's download in magpie web (archive/file). TestArchiveWholeBodies, TestArchiveLimit, TestRowKeepsArchive and TestArchivePreviewAndDownload fail without the change; TestArchiveOffUsage, TestArchiveTooLargeToPreview, TestArchiveOldPreview keep the rest. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; request-archive-usage (new), request-archive, click-scroll, old-webkit, usage-ledger, usage-ledger-detail, usage-ledger-content, web-page, sse-body pass.
2026-10-02 00:57:51 +08:00
usage: a Codex call through the gateway that wrote to its upstream's cache is no longer listed a second time as a local session call (laowangba233 on #589: 用量中的统计是不是重复了 … 为啥会一个是提供商的一个是本地会话的) — the Usage page drops a Codex rollout's call when it matches a gateway record by session, end time within 2s and equal tokens. Codex reads input_tokens_details.cache_write_tokens from a Responses reply and keeps it as cache_write_input_tokens; the gateway's rUsage read only cached_tokens, so for a call whose upstream told a write the record had CacheWrite 0 and the rollout didn't, the tokens never matched, and the call stayed listed under Local session beside its provider's row. Calls without a write matched, so only some calls were counted twice (Maru Code 92 calls plus 22 "本地会话" in the report). Also, sessions' spent took only the cache read out of Codex's input_tokens, which holds the write too, so such a call's write was counted twice in its own input. Now: the gateway reads cache_write_tokens (input = input_tokens - cached - written) and tells it to Codex when it translates a reply; spent takes the write out of the input as well; and the matcher compares a call's input plus cache writes, its output and its cache reads, so a record and a rollout that split the write differently — an older Codex without the field, or records already in usage.jsonl from before this change — still match, and the duplicates already on a user's page go too. The session caches' versions go up (sessions.json 11, calls-v4) so kept parses are read again, and older calls-v* folders are removed. Under a provider filter the provider ranking still lists the other providers, as it is a picker; the duplicates were what made it read as double counting. TestCodexCacheWriteMatchesGatewayCall (a Codex rollout and gateway records for two calls, one with a 9000-token cache write; with the rollout naming the write and the record not, both naming it, and an older Codex without it: two rows, no Local session row, the provider ranking maru-code alone, unfiltered and filtered to the provider, by QueryPage and LedgerOf; fails without the change with a third row from the log), TestCodexCacheWriteNotInInput and TestResponsesUsageCacheWrite (parse and round trip through what Codex is told) are new and fail without the change; TestCallsCodex's call with a write now expects input 200 (2500 - 2000 read - 300 written). go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./internal/... under a temp HOME pass (TestFXInState's known TempDir-cleanup flake aside, which passes alone).
2026-10-03 00:47:33 +08:00
usage: magpie usage and the TUI's Usage page show the agents' own calls, read from their session files, beside the gateway's, and a Codex-based client's calls aren't counted twice (Kumo31 on Discord: Codex used directly, not through magpie, was on the window's Usage tab but not in magpie usage) — magpie usage and the TUI only ever summed the gateway's log (usage.Summarize); the calls the agents' session files record that the gateway never saw (Codex on the user's own ChatGPT sign-in, Claude Code, Claude Desktop…) were merged only into the ledger the window's Requests tab and the tray panel read (#239). usage.Direct sums those rows of the ledger (Source "log", this computer's, rejected ones left out) by agent, model, account and session, priced as the ledger prices them, and magpie usage prints them after the gateway's tables under "not through magpie · the agents' own requests, read from their session files" (a model whose provider the file doesn't tell reads "local session/<model>"); the TUI's Usage page shows the same block under the gateway's, each with its totals and agents' and models' tables. The two parts are separate, so nothing is counted in both; a period with only the agents' own calls shows them under "no calls through magpie". All of magpie usage's output now goes to the writer it is given. Checking it against the owner's real data showed the ledger counted a Codex-based client's calls twice: miyi-helper's requests through the gateway are logged under its own name while its Codex session file names Codex, and the gateway/session-file matcher keyed on the agent, so they never matched; the matcher now keys on the session, tokens and time alone (a session id names one conversation), which also corrects the Requests tab. With read-only copies of the owner's ~/.config/magpie and four days of ~/.codex/sessions in a sandbox HOME: magpie usage week printed 177 calls not through magpie, in 1.3M, out 52.4K, cache read 3.9M, gpt-6-astra 154 and gpt-6-sol 23 — the same rows, to the token, as usage.LedgerOf (what /api/usage/requests serves) gave for the period; before the matcher fix, today's 7 miyi-helper calls showed in both parts. TestDirectCountsOnlySessionFileCalls (the miyi-helper case included), TestUsageShowsCallsNotThroughMagpie and TestUsagePageShowsCallsNotThroughMagpie fail on the old code. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass.
2026-10-04 23:03:15 +08:00
usage: magpie usage and the TUI's Usage page show the agents' own calls, read from their session files, beside the gateway's, and a Codex-based client's calls aren't counted twice (Kumo31 on Discord: Codex used directly, not through magpie, was on the window's Usage tab but not in magpie usage) — magpie usage and the TUI only ever summed the gateway's log (usage.Summarize); the calls the agents' session files record that the gateway never saw (Codex on the user's own ChatGPT sign-in, Claude Code, Claude Desktop…) were merged only into the ledger the window's Requests tab and the tray panel read (#239). usage.Direct sums those rows of the ledger (Source "log", this computer's, rejected ones left out) by agent, model, account and session, priced as the ledger prices them, and magpie usage prints them after the gateway's tables under "not through magpie · the agents' own requests, read from their session files" (a model whose provider the file doesn't tell reads "local session/<model>"); the TUI's Usage page shows the same block under the gateway's, each with its totals and agents' and models' tables. The two parts are separate, so nothing is counted in both; a period with only the agents' own calls shows them under "no calls through magpie". All of magpie usage's output now goes to the writer it is given. Checking it against the owner's real data showed the ledger counted a Codex-based client's calls twice: miyi-helper's requests through the gateway are logged under its own name while its Codex session file names Codex, and the gateway/session-file matcher keyed on the agent, so they never matched; the matcher now keys on the session, tokens and time alone (a session id names one conversation), which also corrects the Requests tab. With read-only copies of the owner's ~/.config/magpie and four days of ~/.codex/sessions in a sandbox HOME: magpie usage week printed 177 calls not through magpie, in 1.3M, out 52.4K, cache read 3.9M, gpt-6-astra 154 and gpt-6-sol 23 — the same rows, to the token, as usage.LedgerOf (what /api/usage/requests serves) gave for the period; before the matcher fix, today's 7 miyi-helper calls showed in both parts. TestDirectCountsOnlySessionFileCalls (the miyi-helper case included), TestUsageShowsCallsNotThroughMagpie and TestUsagePageShowsCallsNotThroughMagpie fail on the old code. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass.
2026-10-04 23:03:15 +08:00
routing: a group can let a member go that hasn't begun answering after a set time and ask the next, and a request's detail splits its first token into magpie's time and the vendor's (Chen on X: answers through magpie felt slow, with no way to tell whether magpie or the vendor was the wait) — a group's editor has a Slow to start row (Wait, 30 s, 1 min, 2 min; Wait, the default, keeps today's behaviour). With a time set, a streamed request to a member that has sent no text, reasoning or tool call by then is cancelled before any of its bytes reach the agent (the held start is dropped; keepalives go on meanwhile) and the next member is asked; the try is marked slow ("slow to start", narrated in the route), and the member doesn't rest, since nothing is wrong with it. The last member left is always waited for. The card says "Next after 1 min without a first token". Each request now records when it went to the vendor (httptrace WroteRequest, sent_ms), and its detail shows In magpie and Vendor's first token beside First token; after a failover In magpie includes the time spent on the members let go. With the owner's real DeepSeek key in a sandbox, a group [a local member that sends its role chunk then stalls 60 s, deepseek-flash] with 3 s: the answer came from DeepSeek in 4.4 s, none of the stalled member's stream reached the client, its try read slow / "no first token in 3s" with no rest, and the ledger had ttft 3622, sent 3003. TestGroupFirstTokenFailsOver (failover, the last one waited for, one in time kept; passes -race) and group-first-token.test.cjs and usage-ledger-detail.test.cjs (WebKit, en and zh) fail on the old code. group-from-model-add's narrow-window case fails the same on origin/main; Chromium doesn't launch in this sandbox. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass (TestProvidersAnswerWhileListsComeIn flaked once, passes alone).
2026-10-04 19:33:55 +08:00
usage, gateway: a model the vendor spells with other separators isn't marked as another one (vincentzhang on Discord: 这个说明有点奇怪。看不明白。) — Volcengine Ark answers deepseek-v4.1-flash as deepseek-v4-1-flash, the dots written as hyphens, and the Usage page's Requests and the Routing page marked every such row amber 「实际 deepseek-v4-1-flash」, saying another model had answered, not the one asked for under a dated name. Swapped now compares the bare names without their separators (. - _ and spaces) and case, so deepseek-v4-1-flash, deepseek_v4_1_flash, GLM-4-6 for glm-4.6 and gpt-4-1-2025-04-14 for gpt-4.1 are the model asked for, while deepseek-v4-flash or deepseek-v4.1-pro for deepseek-v4.1-flash still are another. Rows and routes kept marked swapped before are read back as not swapped when the names differ only so (usage.SameSpelled, in rowChunk.row and Route.routedAgain). The tag's explanation says it in plainer words: asked for {sent}, the reply says {served} answered, likely another model; the same model under a dated name or spelled otherwise isn't marked (zh, ja, de). TestSwappedIgnoresSeparators fails on the old code; TestKeptRowSameSpelledNotSwapped, TestRouteKeptSpelledOtherwiseNotSwapped. usage-ledger (its expected title updated), remote-group-served, panel-routing, gui-ja and gui-de pass on WebKit. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass (TestCodexVersion timed out once under load and passes alone).
2026-10-05 12:49:59 +08:00
usage, gateway: a model the vendor spells with other separators isn't marked as another one (vincentzhang on Discord: 这个说明有点奇怪。看不明白。) — Volcengine Ark answers deepseek-v4.1-flash as deepseek-v4-1-flash, the dots written as hyphens, and the Usage page's Requests and the Routing page marked every such row amber 「实际 deepseek-v4-1-flash」, saying another model had answered, not the one asked for under a dated name. Swapped now compares the bare names without their separators (. - _ and spaces) and case, so deepseek-v4-1-flash, deepseek_v4_1_flash, GLM-4-6 for glm-4.6 and gpt-4-1-2025-04-14 for gpt-4.1 are the model asked for, while deepseek-v4-flash or deepseek-v4.1-pro for deepseek-v4.1-flash still are another. Rows and routes kept marked swapped before are read back as not swapped when the names differ only so (usage.SameSpelled, in rowChunk.row and Route.routedAgain). The tag's explanation says it in plainer words: asked for {sent}, the reply says {served} answered, likely another model; the same model under a dated name or spelled otherwise isn't marked (zh, ja, de). TestSwappedIgnoresSeparators fails on the old code; TestKeptRowSameSpelledNotSwapped, TestRouteKeptSpelledOtherwiseNotSwapped. usage-ledger (its expected title updated), remote-group-served, panel-routing, gui-ja and gui-de pass on WebKit. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass (TestCodexVersion timed out once under load and passes alone).
2026-10-05 12:49:59 +08:00
usage: with WebDAV or S3 sync, each computer shares its usage and sees the others', the total and each computer's part (#542) — sync carried keys, agents and the library, and each computer's Usage page counted only the calls made on it. Sync now has a Usage tick (off unless picked, usage=yes in magpie webdav/magpie s3): a computer with it on puts its own calls up as one file a day, magpie/usage/<id>-<day>.magpie-usage beside the bundle (under the prefix's usage/ on S3), each a gzipped list of times, agents, providers, models, tokens, costs and statuses, sealed with the sync passphrase (backup.SealData, the key cached per passphrase so a sync derives it once); no request body, session, archive name or key goes, a call another computer passed on over the LAN (Via) isn't put up again, and an error is cut to 300 characters. The id is a random one kept in usage-computer.json, which no backup carries, with the host name beside it. A computer only ever writes and deletes its own names, so none overwrites another's; the files are not in the bundle and its hash, so sharing usage never makes a sync or a conflict of the bundle; the first sync puts up the last 90 days, after that only yesterday and today, at most every 15 minutes (a Save or Sync now at once), and a day past 90 is deleted by the computer it is of. The others' files, listed (PROPFIND Depth 1; ListObjectsV2) and fetched only when changed, are kept in usage-others and read with the ledger, as Record.Computer; a failure is said in the sync row ("Couldn't share usage") and never fails the sync. The Requests page and the panel's Usage tab get a Computer pick (this computer, other computers when more than one, each by name) sending computer=this|others|<id>; a row from another computer says "· on <name>" beside its agent, and its details name the computer and say what was said stays there, nothing asked of this computer's session files for it. With no other computer's usage there is no pick and nothing new on either page, and an older magpie never lists or reads usage/. TestUsageSharedBetweenComputers (computers a and b over a fake WebDAV under httptest: each sees 4 calls, the totals and its own and the other's part, nothing counted twice over repeated syncs, the bundle not put again for usage, a new call reaching the other, a day past the window deleted), TestUsageSharedThroughS3 (the same through a fake S3, signed list and delete, a prefix with a space and a plus) and TestUsageOffAsksNothing are new and don't build without the change; usage-computers.test.cjs (WebKit, en and zh: no pick without other computers, the pick and computer=this, another computer's row and details with no /content asked, the panel's pick, the sync form's tick posting usage: true, no click moving the page) fails without it on all but the no-other-computers case. s3-sync, sync-other-kind, panel-usage, request-archive-usage, click-scroll and usage-ledger-chart pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed once on its TempDir cleanup, passes alone). Turning the Usage tick off, or sync off, lets the other computers' usage kept here go (DropAllShared), so the Usage page counts this computer's alone again: TestUsageSharedBetweenComputers checks both, and fails without it.
2026-10-02 17:47:36 +08:00
usage: a reply that came in one burst at its end tells no tok/s, and is left out of the summed speed (#731, Arcadia822: a Gemini turn through Antigravity that was one write tool call, out 8264, ms 24360, ttft_ms 24359, read 8,264,000 tok/s and lifted the Usage page's average) — a reply's speed is out ÷ (ms − ttft_ms) since #196, and the sums are Σout ÷ Σ(ms − ttft_ms) (Totals.DecodeOut/DecodeMs). codeassist hands a native functionCall over whole, so a turn whose only content is one big tool call has its first content event a millisecond or two before the end: the window covers delivery, not writing, and its tokens over next to no time read millions a second and pull the summed speed up (the test's three calls: 100 tok/s each in truth, 616 with the burst counted). A decode window now counts only when it timed the writing: usage.DecodeWindow gives 0 when it is under MinDecodeMs (100 ms) or the tokens over it would be past MaxDecodeSpeed (10,000 tok/s), both far from any real stream (the fastest vendors write a few thousand a second); Totals.add (records) and Totals.addRow (ledger rows, which the Usage page sums) both go through it. Such a reply still counts as timed and its TTFT in the mean; only its speed is left out, so a model with only bursts shows no tok/s (the Usage page and the CLI already skip a speed with no decode window). The Routing page's story, which works out a try's speed itself (routing.js speedOf), applies the same bounds, so the burst says "first token in 24 s" and no tok/s. app.js's figure is decode_out ÷ decode_ms from the Go sums and needed no change. Tests: TestBurstTellsNoSpeed (the report's record with two 100 tok/s calls: 100 tok/s summed through summarize and addRow, TTFT mean keeps the burst; DecodeWindow's edges: 1 ms, 0 ms, 3 tokens in 50 ms, 8264 in 500 ms, exactly 10,000 tok/s over 100 ms, nothing written, not streamed) fails on the old rule with 616 tok/s; TestSummarizeTimesFirstTokens unchanged. routing-burst-speed.test.cjs (webkit, en and zh: the report's burst and a 500 ms one tell no speed, 100 and 50 tok/s replies keep theirs, picking a request doesn't move the page) fails on the old routing.js with "8264000 tok/s". routing-sessions, routing-served, routing-effort-change, usage-accounts, panel-usage and click-scroll GUI tests pass in webkit (routing-served failed once in a parallel run and passes on its own twice; its fixture has no TTFT). go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried with the reporter's ledger or a live Gemini stream; a short reply under 100 ms (a title, a few tokens) no longer counts toward the speed either, which moves a summed figure by next to nothing.
2026-10-04 11:28:50 +08:00