gateway: a browser that reaches loopback under a web site's name (DNS rebinding) is refused unless it sends an enabled gateway key
What changed
- lanGuard refuses (403) a loopback request that carries a browser's
marks (Origin, Sec-Fetch-Site/Mode, a Mozilla/ user agent) when its Host
isn't this computer's: an IP, localhost, *.localhost, a container
runtime's name for the host (host.docker.internal and kin),
MAGPIE_PUBLIC_URL's or MAGPIE_ADDR's host, or the machine's own name.
An enabled gateway key still lets it in.
- Why: a page whose DNS answers 127.0.0.1 after it loads is same-origin
with the gateway. Its GET sends no Origin and its POST one equal to the
Host, so corsGuard's foreignPage let it by, and on loopback no key is
asked. The nightly security review reproduced it on origin/main: Host
and Origin rebind.attacker.example:3425 from 127.0.0.1 got 200 on
GET /v1/models, POST /v1/chat/completions (spends subscriptions) and
GET /v1/magpie/quotas (account names, balances). The subsystem doc
listed it as a known gap.
- Agents send none of a browser's marks and pass under any name; browsers
and webviews calling localhost or an IP are unchanged.
- Tests: TestCORSKeyThroughTheServer and the Omarchy test now set a real
Host (httptest's default is example.com). TestForeignPagesRefused's
"own origin behind a port-443 name" row from loopback with no forwarding
header now sends a key: without one it is the rebound page.
- docs/subsystems/gateway-routing.md describes the check in place of the gap.
Ran
- TestReboundPageRefused: fails on origin/main ("a rebound page's GET:
200"), passes with the change.
- go test -count=1 ./internal/gateway/ ./internal/gui/ ./internal/access/
(macOS, go1.27): ok. gofmt -l clean; GOOS=linux and GOOS=windows go vet
./internal/gateway/ clean.
Not tried: a real browser against a real rebinding DNS name.
This commit is contained in:
yetone committed
1 parent
4225ba1efb
commit
c431225a08
5 files changed
+132
-2
No files matched your search
@@ -16,7 +16,7 @@ served by a plugin are in [Provider and plugin ownership](provider-plugins.md).
|
||||
| Part | Responsibility | Source |
|
||||
| --- | --- | --- |
|
||||
| Address and caller | `DefaultAddr`. The local token `magpie` and the per-agent `magpie-<agent>` (`TokenFor`). `agentOf` tells which agent asked, first by token, then by User-Agent. | [`gateway.go`](../../internal/gateway/gateway.go) |
|
||||
| HTTP surface | `Handler` routes `/v1/chat/completions`, `/v1/responses`, `/v1/messages` (each also without `/v1`), `/v1/models`, Gemini's `/v1beta/models/…`, the Codex backend path, `/v1/codex/models` (Codex's catalog shape for a Codex elsewhere with a gateway key, `codexCatalog`, kept to the key's models by `keyAllowed`; #1281), `/mcp/{name}` and the `/v1/magpie/*` status endpoints. A GET on `/v1/responses` (a client opening Responses over a WebSocket) is answered 426, a plain one 405: Responses is relayed over HTTP only. Every route is wrapped in `counted`, `corsGuard`, `callerGuard`, `withCaller` and `keyLimited`; `/v1/messages`, `/messages` and `/v1/messages/count_tokens` in `keepsSignIn` too. `corsGuard` (#1051) serves CORS only to the origins listed in Settings › Network › Web pages (`settings.CORSOrigins`, cleaned to `scheme://host[:port]` by `settings.CleanOrigins`, which refuses a wildcard). For a listed origin it answers the preflight (204) and adds `Access-Control-Allow-Origin`, and every other request from it needs an enabled gateway key (`access.Authenticate`), even on loopback, or gets a 401. It takes the key from the caller `lanGuard` recorded too, since on the running server `lanGuard` has already put magpie's own token in the key's place. Any other web page is refused 403 on every route (`foreignPage`): on loopback a call needs no key and the body's Content-Type isn't checked, so a page the user visits could otherwise send a "simple" `text/plain` POST and spend their subscriptions blind. Only an `http`/`https` Origin is judged, and one on this computer (`localhost`, `*.localhost`, `127.0.0.0/8`, `[::1]`, any port) or the request's own origin (its host and port equal to `Host`, such as a page on the gateway's LAN address) passes, as does usemagpie.ai on `/v1/magpie/omarchy`. `null` and other schemes (`app://`, `file://`, `tauri://`, `vscode-webview://`, `chrome-extension://`) come from desktop apps, webviews and extensions that don't enforce CORS, and pass. A request with no Origin (every agent's) passes through with no CORS headers, as before. There is no `Host` check, so a page whose own name resolves to 127.0.0.1 (DNS rebinding) counts as same-origin. | [`gateway.go`](../../internal/gateway/gateway.go), [`cors.go`](../../internal/gateway/cors.go), [`codex_catalog.go`](../../internal/gateway/codex_catalog.go), [`internal/settings/origins.go`](../../internal/settings/origins.go) |
|
||||
| HTTP surface | `Handler` routes `/v1/chat/completions`, `/v1/responses`, `/v1/messages` (each also without `/v1`), `/v1/models`, Gemini's `/v1beta/models/…`, the Codex backend path, `/v1/codex/models` (Codex's catalog shape for a Codex elsewhere with a gateway key, `codexCatalog`, kept to the key's models by `keyAllowed`; #1281), `/mcp/{name}` and the `/v1/magpie/*` status endpoints. A GET on `/v1/responses` (a client opening Responses over a WebSocket) is answered 426, a plain one 405: Responses is relayed over HTTP only. Every route is wrapped in `counted`, `corsGuard`, `callerGuard`, `withCaller` and `keyLimited`; `/v1/messages`, `/messages` and `/v1/messages/count_tokens` in `keepsSignIn` too. `corsGuard` (#1051) serves CORS only to the origins listed in Settings › Network › Web pages (`settings.CORSOrigins`, cleaned to `scheme://host[:port]` by `settings.CleanOrigins`, which refuses a wildcard). For a listed origin it answers the preflight (204) and adds `Access-Control-Allow-Origin`, and every other request from it needs an enabled gateway key (`access.Authenticate`), even on loopback, or gets a 401. It takes the key from the caller `lanGuard` recorded too, since on the running server `lanGuard` has already put magpie's own token in the key's place. Any other web page is refused 403 on every route (`foreignPage`): on loopback a call needs no key and the body's Content-Type isn't checked, so a page the user visits could otherwise send a "simple" `text/plain` POST and spend their subscriptions blind. Only an `http`/`https` Origin is judged, and one on this computer (`localhost`, `*.localhost`, `127.0.0.0/8`, `[::1]`, any port) or the request's own origin (its host and port equal to `Host`, such as a page on the gateway's LAN address) passes, as does usemagpie.ai on `/v1/magpie/omarchy`. `null` and other schemes (`app://`, `file://`, `tauri://`, `vscode-webview://`, `chrome-extension://`) come from desktop apps, webviews and extensions that don't enforce CORS, and pass. A request with no Origin (every agent's) passes through with no CORS headers, as before. A page whose own name resolves to 127.0.0.1 (DNS rebinding) would count as same-origin here, so `lanGuard` refuses it first (`rebound`, 403): a loopback request carrying a browser's marks (`Origin`, `Sec-Fetch-Site`/`Sec-Fetch-Mode`, a `Mozilla/` user agent) whose `Host` isn't this computer's (an IP, `localhost`, `*.localhost`, a container runtime's name for the host such as `host.docker.internal`, `MAGPIE_PUBLIC_URL`'s or `MAGPIE_ADDR`'s host, the machine's own name) needs an enabled gateway key. Agents send none of those marks and pass under any name. | [`gateway.go`](../../internal/gateway/gateway.go), [`cors.go`](../../internal/gateway/cors.go), [`codex_catalog.go`](../../internal/gateway/codex_catalog.go), [`internal/settings/origins.go`](../../internal/settings/origins.go) |
|
||||
| Model resolution | `serve` resolves the model the agent asked for: Claude tier stand-ins, a routing group (`provider.GroupFor`: its id, the model's id however a vendor spells it, else the one group whose name it is, unless a provider serves a model by that id), auto stand-ins, then `provider.Resolve`. A bare model id several providers serve (MOMO on Discord: `glm-5.3` under opencode-go and a6api) is never refused: with found groups on (the default) `GroupFor` takes it to the group magpie found for it (`auto-<slug>`), its members in the Providers order, routed and failed over as any group, a conversation staying with the member that answered it while its cache is worth it (affinity); with them off, `Resolve` takes the first provider in the Providers order that exposes it, with no fallback. magpie writes only `provider/model` ids into agents' configs (`magpieModels`), so a bare id is one the user wrote. Every reply says who answered in `X-Magpie-Provider` and `X-Magpie-Model` (`noteMember`), and the usage log and the Routing page keep it. A model id with `~` and no slash that resolves as nothing else is the one served id (a provider's model or a group) whose slashes it spells as `~` (`unflat`, `provider.Unflat`; MiniMax Code's spelling, #1387), taken back in `validateModel` before key limits, middleware and usage see it; two ids spelled alike are neither. An unknown model gets a 404 naming why: a switched-off provider, a disabled group or an empty group. Otherwise it names up to three ids nearest to what was asked (`provider.Closest`: words of a model's display name count before words of its id, and a provider's id or name only alongside them, so "Zen-Space-Bunny" is told `opencode-zen-free/space-bunny-free`, MOMO on Discord), then the ways a model can be named, then every id (`unknownModel`; chat and embeddings). | [`gateway.go`](../../internal/gateway/gateway.go), [`internal/provider/group.go`](../../internal/provider/group.go), [`internal/provider/flat.go`](../../internal/provider/flat.go) |
|
||||
| Candidates | `plan` lists one provider's candidates: one per key or account on, then each `Fallback` model, with resting ones last (`restLast`). `planGroup` and `planLevel` weigh a group's members together. A nested group plans by its own routing (#576). A member's own fallbacks are not the group's. | [`fallback.go`](../../internal/gateway/fallback.go) |
|
||||
| Order of keys and accounts | `Routing`: smart (""), `order`, `rotate`, `usage` (least used, with an `usageHalfLife` of 1h), `pace` or `weight` (`weightedFirst`). Smart, `usage` and `pace` weigh a subscription by its allowance (`allowances`), and a key by its own usage windows when magpie reads them (`keyAllowance`: a sub2api key given a 5-hour, day or 7-day limit, or a key bought as a plan, such as Kimi Code's or a GLM Coding Plan's); any other key has none. A lone key or account is weighed against nobody, but what it has left is still read into the trace (`weighRouted`), so the Routing page shows it as it does for several (#1016); all but used up, it keeps its conversation rather than being said to move (`affine`). A failure rests a candidate (`failure`, `failureOf`). | [`routing.go`](../../internal/gateway/routing.go), [`weighted.go`](../../internal/gateway/weighted.go), [`internal/provider/routing.go`](../../internal/provider/routing.go) |
|
||||
|
||||
@@ -149,6 +149,7 @@ func TestCORSKeyThroughTheServer(t *testing.T) {
|
||||
for _, key := range []string{secret, ""} {
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(chatReq))
|
||||
r.RemoteAddr = "127.0.0.1:50123"
|
||||
r.Host = "127.0.0.1:3425"
|
||||
r.Header.Set("Origin", "http://localhost:3000")
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set(SessionHeader, "browser-chat")
|
||||
@@ -271,7 +272,7 @@ func TestForeignPagesRefused(t *testing.T) {
|
||||
{"http://tauri.localhost", "127.0.0.1:3425", loop, ""}, // Tauri's webview on Windows
|
||||
{"https://app.example.com", "127.0.0.1:3425", loop, secret}, // listed, with its key
|
||||
{"http://192.168.1.5:3425", "192.168.1.5:3425", "192.168.1.9:50123", secret}, // the gateway's own origin, shared on the LAN
|
||||
{"https://magpie.example.com", "magpie.example.com", loop, ""}, // its own origin behind a port-443 name
|
||||
{"https://magpie.example.com", "magpie.example.com", loop, secret}, // its own origin behind a port-443 name, with a key (without one, a rebound page: TestReboundPageRefused)
|
||||
{"null", "127.0.0.1:3425", loop, ""}, // a sandboxed frame, a file:// page
|
||||
{"app://cherry", "127.0.0.1:3425", loop, ""}, // an Electron renderer
|
||||
{"file://", "127.0.0.1:3425", loop, ""},
|
||||
|
||||
@@ -234,6 +234,11 @@ func (s *Server) Relisten() error {
|
||||
func lanGuard(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
remote := !local(r)
|
||||
if !remote && rebound(r) {
|
||||
log.Printf("refused %s %s: a web page reached loopback under the name %q", r.Method, r.URL.Path, r.Host)
|
||||
writeError(w, provider.Chat, http.StatusForbidden, "magpie doesn't answer a browser that reached it as "+r.Host+" (a web site's name pointed at this computer): call it as localhost or 127.0.0.1, or send an enabled gateway key")
|
||||
return
|
||||
}
|
||||
if remote && !remoteKeyed() {
|
||||
if proxied(r) {
|
||||
log.Printf("refused %s %s through a proxy or tunnel: magpie isn't shared", r.Method, r.URL.Path)
|
||||
@@ -257,6 +262,59 @@ func lanGuard(next http.Handler) http.Handler {
|
||||
})
|
||||
}
|
||||
|
||||
// rebound: a browser sent this request to loopback under a hostname that
|
||||
// isn't this computer's — a web site whose DNS answered 127.0.0.1 after its
|
||||
// page loaded (DNS rebinding). The page is then same-origin with the
|
||||
// gateway: it sends no Origin on a GET, or one equal to the Host, so
|
||||
// corsGuard lets it by, and on loopback a call needs no key, so the page
|
||||
// could read the models and quotas and spend the user's subscriptions.
|
||||
// Agents call magpie as localhost or an IP, and send none of a browser's
|
||||
// marks; a hostname of this computer's own (MAGPIE_PUBLIC_URL's,
|
||||
// MAGPIE_ADDR's, its own name, a container runtime's name for the host)
|
||||
// isn't a web site's. An enabled gateway key, which such a page can't
|
||||
// know, still lets a call in.
|
||||
func rebound(r *http.Request) bool {
|
||||
if !fromBrowser(r) || ownHost(r.Host) {
|
||||
return false
|
||||
}
|
||||
return !slices.ContainsFunc(callerKeys(r), func(k string) bool { _, ok := access.Authenticate(k); return ok })
|
||||
}
|
||||
|
||||
// fromBrowser: the request carries what a browser puts on every request
|
||||
// and a page's script can't take off.
|
||||
func fromBrowser(r *http.Request) bool {
|
||||
return r.Header.Get("Origin") != "" || r.Header.Get("Sec-Fetch-Site") != "" ||
|
||||
r.Header.Get("Sec-Fetch-Mode") != "" || strings.HasPrefix(r.UserAgent(), "Mozilla/")
|
||||
}
|
||||
|
||||
// containerHostNames are the names container runtimes give the host.
|
||||
var containerHostNames = []string{"host.docker.internal", "gateway.docker.internal", "host.containers.internal", "host.orb.internal", "docker.for.mac.localhost"}
|
||||
|
||||
// ownHost: host (a Host header) names this computer — an IP, localhost,
|
||||
// *.localhost, or one of the names above.
|
||||
func ownHost(host string) bool {
|
||||
if h, _, err := net.SplitHostPort(host); err == nil {
|
||||
host = h
|
||||
}
|
||||
host = strings.TrimSuffix(strings.ToLower(strings.Trim(host, "[]")), ".")
|
||||
if host == "" || net.ParseIP(host) != nil || host == "localhost" || strings.HasSuffix(host, ".localhost") || slices.Contains(containerHostNames, host) {
|
||||
return true
|
||||
}
|
||||
if strings.EqualFold(host, PublicHost()) {
|
||||
return true
|
||||
}
|
||||
if h, _, err := net.SplitHostPort(Addr()); err == nil && strings.EqualFold(host, h) {
|
||||
return true
|
||||
}
|
||||
if name, err := os.Hostname(); err == nil {
|
||||
name = strings.TrimSuffix(strings.ToLower(name), ".local")
|
||||
if name != "" && (host == name || host == name+".local") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// callerGuard also covers embedded handlers used by the web app and tests.
|
||||
func callerGuard(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A web page whose site's DNS turns to 127.0.0.1 after it loads (DNS
|
||||
// rebinding) calls the gateway as its own origin: a GET carries no Origin,
|
||||
// a POST one equal to the Host, and on loopback no key is asked. Such a
|
||||
// browser request under a name that isn't this computer's is refused; an
|
||||
// agent, a browser on localhost or an IP, a container runtime's name for
|
||||
// the host, and a call with an enabled gateway key go through as before.
|
||||
func TestReboundPageRefused(t *testing.T) {
|
||||
fresh(t)
|
||||
t.Setenv("MAGPIE_ADDR", "")
|
||||
t.Setenv("MAGPIE_PUBLIC_URL", "")
|
||||
reached := false
|
||||
h := lanGuard(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { reached = true }))
|
||||
call := func(method, host string, hdr ...string) int {
|
||||
reached = false
|
||||
r := httptest.NewRequest(method, "/v1/models", nil)
|
||||
r.RemoteAddr = "127.0.0.1:51234"
|
||||
r.Host = host
|
||||
for i := 0; i+1 < len(hdr); i += 2 {
|
||||
r.Header.Set(hdr[i], hdr[i+1])
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
if w.Code == http.StatusForbidden && !strings.Contains(w.Body.String(), "localhost or 127.0.0.1") {
|
||||
t.Fatalf("%s %s: %q", method, host, w.Body.String())
|
||||
}
|
||||
return w.Code
|
||||
}
|
||||
chrome := []string{"User-Agent", "Mozilla/5.0 (Macintosh) Chrome/140", "Sec-Fetch-Site", "same-origin", "Sec-Fetch-Mode", "cors"}
|
||||
evil := "rebind.attacker.example:3425"
|
||||
|
||||
if c := call("GET", evil, chrome...); c != http.StatusForbidden || reached {
|
||||
t.Fatal("a rebound page's GET:", c)
|
||||
}
|
||||
if c := call("POST", evil, append([]string{"Origin", "http://" + evil}, chrome...)...); c != http.StatusForbidden || reached {
|
||||
t.Fatal("a rebound page's POST:", c)
|
||||
}
|
||||
// Safari before Sec-Fetch: the user agent alone
|
||||
if c := call("GET", evil, "User-Agent", "Mozilla/5.0 (Macintosh) Safari/605"); c != http.StatusForbidden || reached {
|
||||
t.Fatal("an older browser:", c)
|
||||
}
|
||||
for _, host := range []string{"127.0.0.1:3425", "localhost:3425", "[::1]:3425", "wails.localhost", "host.docker.internal:3425", "LOCALHOST.:3425"} {
|
||||
if c := call("GET", host, chrome...); c != 200 || !reached {
|
||||
t.Fatal("a browser at", host, c)
|
||||
}
|
||||
}
|
||||
// an agent sends no browser marks, whatever name it was given
|
||||
if c := call("POST", evil, "Authorization", "Bearer magpie", "User-Agent", "claude-cli/2.1"); c != 200 || !reached {
|
||||
t.Fatal("an agent:", c)
|
||||
}
|
||||
t.Setenv("MAGPIE_PUBLIC_URL", "https://magpie.example.net")
|
||||
if c := call("GET", "magpie.example.net", chrome...); c != 200 || !reached {
|
||||
t.Fatal("the public URL's host:", c)
|
||||
}
|
||||
_, secrets := newCaller(t, "Page")
|
||||
if c := call("GET", evil, append([]string{"Authorization", "Bearer " + secrets[0]}, chrome...)...); c != 200 || !reached {
|
||||
t.Fatal("with an enabled gateway key:", c)
|
||||
}
|
||||
if c := call("GET", evil, append([]string{"Authorization", "Bearer sk-magpie-wrong"}, chrome...)...); c != http.StatusForbidden || reached {
|
||||
t.Fatal("with a wrong key:", c)
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,7 @@ func TestOmarchyThemeForTheSite(t *testing.T) {
|
||||
call := func(method, from, origin string) *httptest.ResponseRecorder {
|
||||
r := httptest.NewRequest(method, "/v1/magpie/omarchy", nil)
|
||||
r.RemoteAddr = from
|
||||
r.Host = "127.0.0.1:3425"
|
||||
if origin != "" {
|
||||
r.Header.Set("Origin", origin)
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user