Files
magpie/internal/access/limit.go
T
yetone 0879894af3 gateway: each gateway key can have its own limit, a token total and/or an estimated cost per day, week or month, and a spent key is refused before any provider is asked (margbug01 on #585, v0.1.671: 每个网关密钥都能设置自己的用量上限) — a gateway key named who called and the Usage page grouped its calls, but nothing held a key to an amount, so one client shared on the network could spend a subscription's whole window. A key now carries limit {period day|week|month, tokens, cost, cacheReads} in caller-keys.json, set with the new "limit-key" change. Windows are calendar ones in local time (from midnight, from Monday, from the 1st) and a call counts in the window it started in. Tokens are a call's uncached input, output and cache writes, plus cache reads when the limit says so; cost is an estimate at the Usage page's effective prices (usage.NewCoster), a call with no known price adding none and counted as unpriced. What a key used is summed from usage.jsonl (usage.Visit), so it survives a restart; the new internal/budget keeps the sums in memory per window and appendUsage writes a key's record through budget.Append, which appends and counts under one lock so a window read meanwhile never counts a call twice. Before a keyed POST reaches the mux, keyLimited reserves for it: its body's size in tokens (4 bytes a token) plus the key's mean output per call this window, at the window's cost per token or the named model's input price; it is let in while used + reserved is under the cap, so requests sent at once overshoot by about one call, and the reservation is released when the request ends, a streamed reply being settled then with the usage its vendor reported. A spent key gets a 429 in its API's error shape (Anthropic, OpenAI chat/responses, Gemini) naming the key, the cap and the reset time, with Retry-After, X-Should-Retry: false and X-Magpie-Limit-Reset, recorded as a rejected call (gateway_key_limit); other keys go on. Requests from this computer without a gateway key are not limited; a key used from this computer is. count_tokens and bridge callbacks are never held. GET /v1/magpie/limit tells a key its own status. Gateway keys rows show a Limit badge (what is used of it, per what, Spent in red) and a line under a limited key (used of limit, left, resets / refused until, in flight; unpriced calls in its title); Limit opens an editor under the row (window, tokens with k/m/万/亿, cost in US$, Count cache reads too, what is counted) staged until Save, marked unsaved, Cancel/Escape dropping it, a bad amount disabling Save, No limit removing it; en and zh; no left-border accent; no click scrolls. The CLI gains magpie gateway-key limit <id> [off | day|week|month --tokens N --cost USD --cache-reads] and a LIMIT column in list. TestKeyLimitRefusesBeforeUpstream (two calls within 700 tokens, the third a 429 the fake upstream never sees, message/type/headers, the rejected record, Anthropic's shape on /v1/messages, another key and a keyless loopback request let in, still refused after budget.Forget as after a restart, a raised and a removed limit let in, /v1/magpie/limit), TestKeyLimitHoldsParallelRequests (20 at once against 2000 tokens: some refused, at most one call over, no reservation left; without the reservation all 20 got in and 7350 were used), TestKeyLimitWindowResets, TestLimitWindow, TestLimitValid and TestGatewayKeyLimitCLI are new; gateway-key-limit.test.cjs (WebKit, en and zh) is new and passes, as do gateway-caller-keys, gateway-connect-keys, api-key-theme and click-scroll (whose request-pick case flaked once under a parallel run and passes alone, as on main). go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState's known TempDir-cleanup flake aside).
2026-10-03 00:18:16 +08:00

70 lines
2.1 KiB
Go

package access
import (
"errors"
"math"
"time"
)
// Limit is a gateway key's budget for a calendar window in local time
// (#585): a day from midnight, a week from Monday's midnight, a month
// from the 1st's. Tokens counts each call's uncached input, output
// (reasoning included) and cache writes, and its cache reads too when
// CacheReads is set; Cost is an estimate in US dollars at the prices
// magpie shows on the Usage page, not a vendor's bill. A cap of 0 is no
// cap; a limit with neither is no limit.
type Limit struct {
Period string `json:"period"`
Tokens int64 `json:"tokens,omitempty"`
Cost float64 `json:"cost,omitempty"`
CacheReads bool `json:"cacheReads,omitempty"`
}
// Periods are the windows a limit can be set for.
var Periods = []string{"day", "week", "month"}
// Limited: l caps something.
func (l *Limit) Limited() bool { return l != nil && (l.Tokens > 0 || l.Cost > 0) }
// Valid is l as it is kept: nil for no limit, an error for one that
// can't be.
func (l *Limit) Valid() (*Limit, error) {
if l == nil {
return nil, nil
}
if l.Tokens < 0 || l.Cost < 0 || math.IsNaN(l.Cost) || math.IsInf(l.Cost, 0) {
return nil, errors.New("A limit can't be negative")
}
if !l.Limited() {
return nil, nil
}
v := *l
if v.Period == "" {
v.Period = "day"
}
switch v.Period {
case "day", "week", "month":
default:
return nil, errors.New("A limit's period is day, week or month")
}
v.Cost = math.Round(v.Cost*1e4) / 1e4
return &v, nil
}
// Window is the calendar window of period that now is in, in now's
// location: its start and when the next begins.
func Window(period string, now time.Time) (start, reset time.Time) {
y, m, d := now.Date()
loc := now.Location()
switch period {
case "week":
back := (int(now.Weekday()) + 6) % 7 // days since Monday
start = time.Date(y, m, d-back, 0, 0, 0, 0, loc)
return start, time.Date(y, m, d-back+7, 0, 0, 0, 0, loc)
case "month":
return time.Date(y, m, 1, 0, 0, 0, 0, loc), time.Date(y, m+1, 1, 0, 0, 0, 0, loc)
default:
return time.Date(y, m, d, 0, 0, 0, 0, loc), time.Date(y, m, d+1, 0, 0, 0, 0, loc)
}
}