provider, docs: a bare model id several providers serve goes by the Providers order, said and tested (MOMO on Discord)

MOMO's {"model":"glm-5.3"} was answered by opencode-go though a6api
serves glm-5.3 too, and they asked for a 409 or for the response to say
which provider was used. The pick was already deterministic, but
Resolve's comment said "exactly one provider" and no doc said what
happens.

Decision: keep serving bare ids, no 409. magpie writes only
provider/model ids into agents' configs (agent/route.go magpieModels),
so a bare id comes from a user's own config, and a 409 would break every
such client the day a second provider adds the model. The rule is now
written down and pinned:
- found groups on (the default): GroupFor takes the bare id to its found
  group auto-<slug>, members in the Providers order, smart routing and
  fail-over to the next provider; a conversation stays with the member
  that answered it while its cache is worth it (affinity).
- found groups off: Resolve takes the first provider in the Providers
  order that exposes it, alone.
- every reply already carries X-Magpie-Provider / X-Magpie-Model
  (member.go), and the usage log has the provider.

What changed: Resolve's comment; gateway-routing.md model resolution
row; reference.md, the ways a model is named and the headers;
bare_slug_test.go: two providers serving glm-5.3, both modes: the first
in the Providers order answers every new conversation, reordering moves
new ones, an answered one stays in the found group, and a 503 from the
first falls over only in the found group. It passes on the old code
too: it pins behaviour that was already there, now documented.

Run: gofmt -l; go vet -tags nogui on darwin, linux, windows for provider
and gateway; go test -tags nogui ./internal/gateway ./internal/provider
in a sandbox HOME.
This commit is contained in:
yetone committed 2026-10-10 05:56:07 +08:00
1 parent e50a958014
commit c1c887798b
4 files changed
+84 -4

No files matched your search

+11 -2
View File
@@ -1172,8 +1172,17 @@ request to which is translated anyway.
Requests pass straight through when the vendor speaks the agent's API and
are translated otherwise, streaming, tool calls and reasoning included. The
key is `magpie` (any value works; the gateway only listens on loopback), and
models are named `provider/model`. Anything with a base-URL setting can use
it:
models are named `provider/model`.
- A routing group is asked by its id (`group/<id>`) or its name.
- A bare model id (`glm-5.3`) works too. When several providers serve it,
it goes to the routing group magpie found for it, which asks them in the
order of the Providers page and falls over to the next. With found
groups off, it goes to the first of them in that order alone.
- Every reply names who answered in `X-Magpie-Provider` (the provider)
and `X-Magpie-Model` (`provider/model`), and so does the usage log.
Anything with a base-URL setting can use it:
| Tool speaks | Base URL | Environment |
| ----------- | -------------------------- | --------------------------------------------- |
+1 -1
View File
@@ -17,7 +17,7 @@ served by a plugin are in [Provider and plugin ownership](provider-plugins.md).
| --- | --- | --- |
| Address and caller | `DefaultAddr`. The local token `magpie` and the per-agent `magpie-<agent>` (`TokenFor`). `agentOf` tells which agent asked, first by token, then by User-Agent. | [`gateway.go`](../../internal/gateway/gateway.go) |
| HTTP surface | `Handler` routes `/v1/chat/completions`, `/v1/responses`, `/v1/messages` (each also without `/v1`), `/v1/models`, Gemini's `/v1beta/models/…`, the Codex backend path, `/v1/codex/models` (Codex's catalog shape for a Codex elsewhere with a gateway key, `codexCatalog`, kept to the key's models by `keyAllowed`; #1281), `/mcp/{name}` and the `/v1/magpie/*` status endpoints. A GET on `/v1/responses` (a client opening Responses over a WebSocket) is answered 426, a plain one 405: Responses is relayed over HTTP only. Every route is wrapped in `counted`, `corsGuard`, `callerGuard`, `withCaller` and `keyLimited`; `/v1/messages`, `/messages` and `/v1/messages/count_tokens` in `keepsSignIn` too. `corsGuard` (#1051) serves CORS only to the origins listed in Settings › Network › Web pages (`settings.CORSOrigins`, cleaned to `scheme://host[:port]` by `settings.CleanOrigins`, which refuses a wildcard). For a listed origin it answers the preflight (204) and adds `Access-Control-Allow-Origin`, and every other request from it needs an enabled gateway key (`access.Authenticate`), even on loopback, or gets a 401. It takes the key from the caller `lanGuard` recorded too, since on the running server `lanGuard` has already put magpie's own token in the key's place. Any other web page is refused 403 on every route (`foreignPage`): on loopback a call needs no key and the body's Content-Type isn't checked, so a page the user visits could otherwise send a "simple" `text/plain` POST and spend their subscriptions blind. Only an `http`/`https` Origin is judged, and one on this computer (`localhost`, `*.localhost`, `127.0.0.0/8`, `[::1]`, any port) or the request's own origin (its host and port equal to `Host`, such as a page on the gateway's LAN address) passes, as does usemagpie.ai on `/v1/magpie/omarchy`. `null` and other schemes (`app://`, `file://`, `tauri://`, `vscode-webview://`, `chrome-extension://`) come from desktop apps, webviews and extensions that don't enforce CORS, and pass. A request with no Origin (every agent's) passes through with no CORS headers, as before. There is no `Host` check, so a page whose own name resolves to 127.0.0.1 (DNS rebinding) counts as same-origin. | [`gateway.go`](../../internal/gateway/gateway.go), [`cors.go`](../../internal/gateway/cors.go), [`codex_catalog.go`](../../internal/gateway/codex_catalog.go), [`internal/settings/origins.go`](../../internal/settings/origins.go) |
| Model resolution | `serve` resolves the model the agent asked for: Claude tier stand-ins, a routing group (`provider.GroupFor`: its id, the model's id however a vendor spells it, else the one group whose name it is, unless a provider serves a model by that id), auto stand-ins, then `provider.Resolve`. A model id with `~` and no slash that resolves as nothing else is the one served id (a provider's model or a group) whose slashes it spells as `~` (`unflat`, `provider.Unflat`; MiniMax Code's spelling, #1387), taken back in `validateModel` before key limits, middleware and usage see it; two ids spelled alike are neither. An unknown model gets a 404 naming why: a switched-off provider, a disabled group or an empty group. Otherwise it names up to three ids nearest to what was asked (`provider.Closest`: words of a model's display name count before words of its id, and a provider's id or name only alongside them, so "Zen-Space-Bunny" is told `opencode-zen-free/space-bunny-free`, MOMO on Discord), then the ways a model can be named, then every id (`unknownModel`; chat and embeddings). | [`gateway.go`](../../internal/gateway/gateway.go), [`internal/provider/group.go`](../../internal/provider/group.go), [`internal/provider/flat.go`](../../internal/provider/flat.go) |
| Model resolution | `serve` resolves the model the agent asked for: Claude tier stand-ins, a routing group (`provider.GroupFor`: its id, the model's id however a vendor spells it, else the one group whose name it is, unless a provider serves a model by that id), auto stand-ins, then `provider.Resolve`. A bare model id several providers serve (MOMO on Discord: `glm-5.3` under opencode-go and a6api) is never refused: with found groups on (the default) `GroupFor` takes it to the group magpie found for it (`auto-<slug>`), its members in the Providers order, routed and failed over as any group, a conversation staying with the member that answered it while its cache is worth it (affinity); with them off, `Resolve` takes the first provider in the Providers order that exposes it, with no fallback. magpie writes only `provider/model` ids into agents' configs (`magpieModels`), so a bare id is one the user wrote. Every reply says who answered in `X-Magpie-Provider` and `X-Magpie-Model` (`noteMember`), and the usage log and the Routing page keep it. A model id with `~` and no slash that resolves as nothing else is the one served id (a provider's model or a group) whose slashes it spells as `~` (`unflat`, `provider.Unflat`; MiniMax Code's spelling, #1387), taken back in `validateModel` before key limits, middleware and usage see it; two ids spelled alike are neither. An unknown model gets a 404 naming why: a switched-off provider, a disabled group or an empty group. Otherwise it names up to three ids nearest to what was asked (`provider.Closest`: words of a model's display name count before words of its id, and a provider's id or name only alongside them, so "Zen-Space-Bunny" is told `opencode-zen-free/space-bunny-free`, MOMO on Discord), then the ways a model can be named, then every id (`unknownModel`; chat and embeddings). | [`gateway.go`](../../internal/gateway/gateway.go), [`internal/provider/group.go`](../../internal/provider/group.go), [`internal/provider/flat.go`](../../internal/provider/flat.go) |
| Candidates | `plan` lists one provider's candidates: one per key or account on, then each `Fallback` model, with resting ones last (`restLast`). `planGroup` and `planLevel` weigh a group's members together. A nested group plans by its own routing (#576). A member's own fallbacks are not the group's. | [`fallback.go`](../../internal/gateway/fallback.go) |
| Order of keys and accounts | `Routing`: smart (""), `order`, `rotate`, `usage` (least used, with an `usageHalfLife` of 1h), `pace` or `weight` (`weightedFirst`). Smart, `usage` and `pace` weigh a subscription by its allowance (`allowances`), and a key by its own usage windows when magpie reads them (`keyAllowance`: a sub2api key given a 5-hour, day or 7-day limit, or a key bought as a plan, such as Kimi Code's or a GLM Coding Plan's); any other key has none. A lone key or account is weighed against nobody, but what it has left is still read into the trace (`weighRouted`), so the Routing page shows it as it does for several (#1016); all but used up, it keeps its conversation rather than being said to move (`affine`). A failure rests a candidate (`failure`, `failureOf`). | [`routing.go`](../../internal/gateway/routing.go), [`weighted.go`](../../internal/gateway/weighted.go), [`internal/provider/routing.go`](../../internal/provider/routing.go) |
| Limits | Each key or account (a candidate's `who()`) has at most `MaxConcurrency` requests out at once (its own `AccountConcurrency` over it); one more waits in its lane's queue, bounded by `QueueLimit` and `QueueWait` and turned away with a 429 past either (`lanes.take`). `MaxRPM` (coeo91 on Discord: OpenRouter's free models take 20 a minute) bounds how many requests it sends the vendor in any rolling 60 seconds (`rpms`): one more waits for room in the minute, in the order they came, after it has its slot and right before it is sent. A wait that would be longer than 2 minutes (or the provider's `QueueWait`, when shorter: `rpmWait`) is turned away at once with a 429 whose `Retry-After` is when the minute has room; nothing rests for it. An agent gone while it waits gives its place back. Every request that reaches the vendor counts: each try of the candidate loop (a retry, the next key, a fallback) waits in `serve`, and the requests a try makes on the way and the gateway's own asks (count_tokens, System One's decisions, embeddings, drawing, Kimi's search) are counted by `rpmTransport` on `s.client` (`metered`, `paidFor`). count_tokens never waits: a candidate with no room is skipped, and the local estimate answers. A full lane or a minute with no room gives way to a ready key or account of the provider or the group (`laneMate`, `ready`); a fallback is never asked for want of either. A plugin's account sends through its own transport, so only its tries are counted, not a plugin's own extra requests. Set in the provider editor (Concurrency, Requests per minute, Queue size, Queue wait) or with `magpie provider account-concurrency`, `queue` and `rpm`; a subscription's are kept with its picks. | [`concurrency.go`](../../internal/gateway/concurrency.go), [`rpm.go`](../../internal/gateway/rpm.go), [`internal/provider/concurrency.go`](../../internal/provider/concurrency.go) |
+66
View File
@@ -0,0 +1,66 @@
package gateway
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/yetone/magpie/internal/provider"
)
// bareGLM asks for glm-5.3 by its bare id, as MOMO's client did, and says
// which provider the reply named and what it said. Each say is a
// conversation of its own: one answered stays where it was answered.
func bareGLM(t *testing.T, s *Server, say string) (code int, from, body string) {
t.Helper()
rec := httptest.NewRecorder()
req := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(`{"model":"glm-5.3","messages":[{"role":"user","content":"`+say+`"}]}`))
s.Handler().ServeHTTP(rec, req)
return rec.Code, rec.Header().Get(providerHeader), rec.Body.String()
}
// A bare model id two providers serve (MOMO on Discord: glm-5.3 under
// opencode-go and a6api) goes where the Providers order says, every
// time, and the reply says which provider it was. With found groups on
// (the default) the group magpie found for it is what's routed, so the
// next provider answers when the first can't; off, it is the first
// provider alone.
func TestBareSlugFollowsTheProvidersOrder(t *testing.T) {
for _, auto := range []bool{true, false} {
t.Run(map[bool]string{true: "found groups", false: "no found groups"}[auto], func(t *testing.T) {
fresh(t)
provider.SetAutoGroups(auto)
t.Cleanup(func() { provider.SetAutoGroups(true) })
goUp, a6 := &keyed{fail: map[string]int{}}, &keyed{fail: map[string]int{}}
serveOn(t, "opencode-go", "kg", []string{"glm-5.3"}, goUp)
serveOn(t, "a6api", "ka", []string{"glm-5.3"}, a6)
s := New()
for _, say := range []string{"a", "b", "c"} {
if code, from, body := bareGLM(t, s, say); code != http.StatusOK || from != "opencode-go" || !strings.Contains(body, "from kg") {
t.Fatalf("%d %q %s", code, from, body)
}
}
if err := provider.SetOrder([]string{"a6api", "opencode-go"}); err != nil {
t.Fatal(err)
}
if code, from, body := bareGLM(t, s, "d"); code != http.StatusOK || from != "a6api" || !strings.Contains(body, "from ka") {
t.Fatalf("reordered: %d %q %s", code, from, body)
}
// a conversation opencode-go answered, with its cache read,
// stays there in the found group; with none, the order decides
stays := map[bool]string{true: "opencode-go", false: "a6api"}[auto]
if code, from, _ := bareGLM(t, s, "a"); code != http.StatusOK || from != stays {
t.Fatalf("conversation a: %d %q, want %s", code, from, stays)
}
a6.fail["ka"] = http.StatusServiceUnavailable
code, from, _ := bareGLM(t, s, "e")
if auto && (code != http.StatusOK || from != "opencode-go") {
t.Fatalf("the found group's next provider wasn't asked: %d %q", code, from)
}
if !auto && code == http.StatusOK {
t.Fatalf("with no found group, %q answered", from)
}
})
}
}
+6 -1
View File
@@ -1364,7 +1364,12 @@ func entryIn(entries []Entry, id string) (Entry, bool) {
// Resolve maps an id an agent sent to a provider and the vendor's model id.
// It accepts catalog ids, "provider/model" for any model (exposed or not),
// and the bare model id when exactly one provider serves it.
// and a bare model id: the first provider in the Providers order (All)
// that exposes it, else the one provider that lists it unexposed. The
// gateway asks GroupFor first, so with found groups on a bare id several
// providers serve is that group, its members in the same order (MOMO on
// Discord: glm-5.3 under opencode-go and a6api). magpie writes only
// provider/model ids into agents' configs, so a bare one is the user's.
// A group's id resolves to its first member.
func Resolve(id string) (Provider, string, bool) {
// Claude Code's mark for a model with a 1M window; it drops it before