diff --git a/docs/reference.md b/docs/reference.md index 425e3f99..8dfebd55 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -230,6 +230,12 @@ separate Responses endpoint, `catalog=` to borrow a models.dev list, and `models=` to name the models to expose. Anything a preset does not know can be overridden the same way. +A model whose vendor answers that it is deprecated or retired (OpenCode +Zen's 410 "Model exo-free has been deprecated.") leaves that provider's +models in `magpie models`, `/v1/models` and every agent's list, even while +the vendor's own list still names it. It comes back once a request for it +as `provider/model` is answered, or after a week. + `magpie provider set header.=` sends a header of your own on every request to a key+URL provider (an empty value removes it; signed-in accounts ignore them). It replaces a header of the same name magpie would diff --git a/docs/subsystems/gateway-routing.md b/docs/subsystems/gateway-routing.md index ce4d6789..c0b52641 100644 --- a/docs/subsystems/gateway-routing.md +++ b/docs/subsystems/gateway-routing.md @@ -37,7 +37,7 @@ served by a plugin are in [Provider and plugin ownership](provider-plugins.md). - A model set for Codex's subagents (`settings.CodexSubagentModel`, the Agents page's Codex "subagent model", willz on Discord) is put on a Codex subagent's request (`x-openai-subagent` read as `kind:collab_spawn`) before this check, whatever model its lead's `spawn_agent` asked for (`codexSubagentPick` in [`codex_subagent.go`](../../internal/gateway/codex_subagent.go)): in `codexBackend` for one of Codex's own bare models, which then goes through `serveAgent`, and in `serve` for any other. Only a ChatGPT account's model can be set (`provider.SetCodexSubagentModel`), as nothing else reads the sealed task. When no ChatGPT account on in magpie serves it now, or a gateway key held to some models may not use it, the subagent goes on the model asked for. The route trace keeps both models and the reason it stayed (`Route.Subagent`), and the Routing page's `kindWhy` says it. The lead's turns, titles and every other kind are never moved. - Rules and image inputs are handled here too. 3. Candidates come from `planGroup` for a group or `plan` for a provider. They are filtered by `sealedReaders` and `allowedCandidates`. An account past its share is held out of the candidates (`capHeld`): a usage cap's share, each window at its own where the account gives it one (`AccountWindowCaps`, willz on Discord: 50% of the five hours, 40% of the week) and at the account's cap otherwise, or 100% of a window for a Codex account set not to spend its credits (`provider.HoldCaps`, `WindowCaps.Of`, `CodexNoCredits`). The account is held while any window is past its own share, and `cappedError` and the trace name the fullest such window's share and cap. When every candidate is held and one held for its credits spends its resets by itself, `autoReset` spends a reset on it and it is tried (`planned.held`); otherwise a capped or barred set returns `cappedError` or `barredError`. Then `affine` moves the conversation's last answerer first, unless it is resting or nearly used up. An `X-Magpie-Account` header (`AccountHeader`) pins the request to one account: only that account is tried, and when it can't answer the caller is told why. -4. The candidate loop calls `attempt` on each candidate in turn, through a `holdWriter`. The writer holds the reply until it is clearly passing. A candidate that fails before anything is sent rests (`failure`): out of credit for 30 minutes, out of quota until reset, rate limited until retry-after, otherwise a minute and longer on repeats. A 400, 403, 404 or 422 that says the model isn't served (`unservedWords`) and says nothing of quota, credit or a rate limit rests only that key's or account's model (`modelRefused`, keyed by `restID`, its backoff counted apart): its other models are still served first in order (#1235, SenseNova's 403 "model is not available in the current token plan"). A subscription account that fails any other way with a window full, by magpie's latest reading of its windows, rests until that window renews. Both quota rests and window rests are held to `longestQuota` (8 days), so a reset time read wrong (a year off) doesn't bench the account for that long. The next candidate is then tried. A failed provider waits `fallbackCooldown` (1 minute) at the back. The vendor's safety filter refusing a reply rests nobody, whether it refuses the turn before any of the reply is said or once some of it has been said (`refused`, `refusedAfter`). The ChatGPT backend's 502 "response protection is unavailable" is about the request's content, not the account: the same history fails the same on each of its accounts and models. It rests nobody, isn't tried again after a moment, and isn't asked of the provider's other accounts or models; another provider left in a group is asked once (`protectionRefused`, `elsewhere`). One cause is known (#1270): a history that replays the backend's own `web_search_call` in a request that doesn't declare the `web_search` tool is refused so, whatever the account, model or turn (a plain turn, a thread's description, a compaction, Codex's own text compaction for a custom provider with `tools: []`). Every request to the ChatGPT backend with such a history therefore declares cached-only search (`{"type":"web_search","external_web_access":false}`, `provider.DeclareSearch`): an account's in `codexBody`, Codex's own sign-in relayed in `codexUpstreamOn`. It goes in the tools, or for Responses Lite (its header, `reasoning.context` `all_turns`, or an `additional_tools` item already there), which refuses a top-level `web_search`, in the `additional_tools` item. A request that offered no tools gets `tool_choice` `none` so the model can't search; one with a forced choice keeps it. A history without a search, or a request that declares search already, is relayed unchanged. siwc, API-key requests to openai.com and the legacy `/responses/compact` are left as they were. +4. The candidate loop calls `attempt` on each candidate in turn, through a `holdWriter`. The writer holds the reply until it is clearly passing. A candidate that fails before anything is sent rests (`failure`): out of credit for 30 minutes, out of quota until reset, rate limited until retry-after, otherwise a minute and longer on repeats. A 400, 403, 404 or 422 that says the model isn't served (`unservedWords`) and says nothing of quota, credit or a rate limit rests only that key's or account's model (`modelRefused`, keyed by `restID`, its backoff counted apart): its other models are still served first in order (#1235, SenseNova's 403 "model is not available in the current token plan"). A 400, 404, 410 or 422 that says the model is retired (`retiredWords`: OpenCode Zen's 410 `ModelDeprecated` "Model exo-free has been deprecated.", whose own model list still names it; MOMO on Discord) rests only that model the same way, is retryable so a group's next member or another provider of it is asked, and is remembered (`provider.Retire`, `retired.json` beside `providers.json`): the model leaves that provider's entries in the catalog (`buildEntries`), so /v1/models, every agent's model list and the found groups go without it, until a request on it is answered again (`Unretire` in `servedCandidate`) or a week has passed. A refresh of the vendor's list doesn't bring it back early, since that list is where it still is. A `provider/model` request for it is still sent. A subscription account that fails any other way with a window full, by magpie's latest reading of its windows, rests until that window renews. Both quota rests and window rests are held to `longestQuota` (8 days), so a reset time read wrong (a year off) doesn't bench the account for that long. The next candidate is then tried. A failed provider waits `fallbackCooldown` (1 minute) at the back. The vendor's safety filter refusing a reply rests nobody, whether it refuses the turn before any of the reply is said or once some of it has been said (`refused`, `refusedAfter`). The ChatGPT backend's 502 "response protection is unavailable" is about the request's content, not the account: the same history fails the same on each of its accounts and models. It rests nobody, isn't tried again after a moment, and isn't asked of the provider's other accounts or models; another provider left in a group is asked once (`protectionRefused`, `elsewhere`). One cause is known (#1270): a history that replays the backend's own `web_search_call` in a request that doesn't declare the `web_search` tool is refused so, whatever the account, model or turn (a plain turn, a thread's description, a compaction, Codex's own text compaction for a custom provider with `tools: []`). Every request to the ChatGPT backend with such a history therefore declares cached-only search (`{"type":"web_search","external_web_access":false}`, `provider.DeclareSearch`): an account's in `codexBody`, Codex's own sign-in relayed in `codexUpstreamOn`. It goes in the tools, or for Responses Lite (its header, `reasoning.context` `all_turns`, or an `additional_tools` item already there), which refuses a top-level `web_search`, in the `additional_tools` item. A request that offered no tools gets `tool_choice` `none` so the model can't search; one with a forced choice keeps it. A history without a search, or a request that declares search already, is relayed unchanged. siwc, API-key requests to openai.com and the legacy `/responses/compact` are left as they were. 5. The last candidate's error reaches the agent. A try that answered is recorded (`served`, `answered`) for routing, affinity and usage. ## Constraints and failure behavior diff --git a/internal/gateway/fallback.go b/internal/gateway/fallback.go index 9003ccce..3dce590a 100644 --- a/internal/gateway/fallback.go +++ b/internal/gateway/fallback.go @@ -717,6 +717,9 @@ var unservedWords = regexp.MustCompile(`(?i)model.{0,80}(not (supported|accessib // and its other models are asked as before. A refusal that also says // quota, credit or a rate limit is about the account, and rests it. func modelRefused(status int, body []byte) bool { + if modelRetired(status, body) { + return true + } switch status { case 400, 403, 404, 422: default: @@ -725,6 +728,26 @@ func modelRefused(status int, body []byte) bool { return unservedWords.Match(body) && !quotaWords.Match(body) && !refusedWords.Match(body) } +// retiredWords are how a vendor says the model is retired, though its +// list may still name it: OpenCode Zen's 410 {"type":"error","error": +// {"type":"ModelDeprecated","message":"Model exo-free has been +// deprecated."}} (MOMO on Discord), OpenAI's "The model … has been +// deprecated". +var retiredWords = regexp.MustCompile(`(?i)"ModelDeprecated"|model_deprecated|model.{0,80}(has been|was|is now|is) (deprecated|retired|discontinued|decommissioned|sunset)`) + +// modelRetired says the vendor turned the request away because the model +// is retired there: it leaves that provider's place in the lists +// (provider.Retire), only it rests, and another provider of it, or the +// group's next member, is asked. Not when it also says quota or credit. +func modelRetired(status int, body []byte) bool { + switch status { + case 400, 404, 410, 422: + default: + return false + } + return retiredWords.Match(body) && !quotaWords.Match(body) +} + // refusedWords are how a vendor says it won't take requests from this // client at all — WorkBuddy's "Illegal API invocation from an unapproved // channel" to a chat opening with another agent's own system prompt — a @@ -790,6 +813,9 @@ func retryable(status int, body []byte) bool { return false case status == 401, status == 402, status == 403, status == 404, status == 408, status == 429, status >= 500: return true + case modelRetired(status, body): + // another provider of the model may still serve it + return true case status >= 400 && provider.EdgeBlocked(body): // the vendor's firewall blocked this address (Alibaba Cloud's 405 // in front of zcode.z.ai): another provider goes another way diff --git a/internal/gateway/gateway.go b/internal/gateway/gateway.go index 97a685f7..6dba37db 100644 --- a/internal/gateway/gateway.go +++ b/internal/gateway/gateway.go @@ -2040,6 +2040,11 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro if hw.passing { clientResponseID = call.Usage.ResponseID } + if !hw.passing && modelRetired(hw.code(), hw.errBody()) { + // the vendor's list may still name it (OpenCode Zen's + // exo-free): it leaves the lists of this provider's models + provider.Retire(c.p.ID, c.model) + } outgrew := false // a failure that didn't say so was the request's length if !hw.passing && !hw.refused && hw.code() >= 400 { if req, err := parse(from, attemptBody); err == nil { diff --git a/internal/gateway/retired_test.go b/internal/gateway/retired_test.go new file mode 100644 index 00000000..67206914 --- /dev/null +++ b/internal/gateway/retired_test.go @@ -0,0 +1,130 @@ +package gateway + +import ( + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "slices" + "strings" + "testing" + + "github.com/yetone/magpie/internal/provider" +) + +// zenDeprecated is OpenCode Zen's answer to a request for exo-free, as it +// was on 2026-10-10 (HTTP/2 410, content-type: application/json), while +// its /zen/v1/models still listed exo-free (MOMO on Discord). +const zenDeprecated = `{"type":"error","error":{"type":"ModelDeprecated","message":"Model exo-free has been deprecated."},"metadata":{"model":"exo-free"}}` + +// zen answers exo-free as OpenCode Zen does, and every other model. +type zen struct{ tried []string } + +func (z *zen) ServeHTTP(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + model := modelOf(body) + z.tried = append(z.tried, model) + w.Header().Set("Content-Type", "application/json") + if model == "exo-free" { + w.WriteHeader(http.StatusGone) + io.WriteString(w, zenDeprecated) + return + } + io.WriteString(w, `{"id":"x","choices":[{"index":0,"message":{"role":"assistant","content":"from zen `+model+`"},"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}`) +} + +func listedModels(t *testing.T, s *Server) []string { + t.Helper() + rec := httptest.NewRecorder() + s.Handler().ServeHTTP(rec, httptest.NewRequest("GET", "/v1/models", nil)) + var got struct { + Data []struct { + ID string `json:"id"` + } `json:"data"` + } + if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil { + t.Fatalf("/v1/models: %v %s", err, rec.Body.String()) + } + var ids []string + for _, m := range got.Data { + ids = append(ids, m.ID) + } + return ids +} + +func TestRetiredModelLeavesTheLists(t *testing.T) { + fresh(t) + z := &zen{} + serveOn(t, "opencode-zen-free", "k", []string{"exo-free", "space-bunny-free"}, z) + s := New() + if ids := listedModels(t, s); !slices.Contains(ids, "opencode-zen-free/exo-free") { + t.Fatalf("before: %v", ids) + } + code, body := postAs(t, s, "", `{"model":"opencode-zen-free/exo-free","messages":[{"role":"user","content":"hi"}]}`) + if code != http.StatusGone || !strings.Contains(body, "Model exo-free has been deprecated.") { + t.Fatalf("the vendor's answer is the agent's: %d %s", code, body) + } + if !provider.Retired("opencode-zen-free", "exo-free") { + t.Fatal("exo-free was not remembered as retired") + } + ids := listedModels(t, s) + if slices.Contains(ids, "opencode-zen-free/exo-free") { + t.Fatalf("a retired model still listed: %v", ids) + } + if !slices.Contains(ids, "opencode-zen-free/space-bunny-free") { + t.Fatalf("the provider's other models went too: %v", ids) + } + // the provider's other models are served as before + if code, body := postAs(t, s, "", `{"model":"opencode-zen-free/space-bunny-free","messages":[{"role":"user","content":"hi"}]}`); code != http.StatusOK { + t.Fatalf("space-bunny-free: %d %s", code, body) + } + // a provider/model id is still sent: the vendor says what it says, and + // an answer puts it back + provider.Unretire("opencode-zen-free", "exo-free") + if !slices.Contains(listedModels(t, s), "opencode-zen-free/exo-free") { + t.Fatal("unretired, exo-free is still out of the list") + } +} + +func TestRetiredMemberFallsOver(t *testing.T) { + fresh(t) + z := &zen{} + other := &keyed{} + serveOn(t, "opencode-zen-free", "k", []string{"exo-free"}, z) + serveOn(t, "b", "kb", []string{"exo-free"}, other) + if err := provider.SaveGroup(provider.Group{ + Name: "Exo", Members: []string{"opencode-zen-free/exo-free", "b/exo-free"}, Routing: provider.Ordered, + }); err != nil { + t.Fatal(err) + } + s := New() + code, body := postAs(t, s, "", `{"model":"group/exo","messages":[{"role":"user","content":"hi"}]}`) + if code != http.StatusOK || !strings.Contains(body, "from kb") { + t.Fatalf("the group's next member wasn't asked: %d %s", code, body) + } + if !provider.Retired("opencode-zen-free", "exo-free") || provider.Retired("b", "exo-free") { + t.Fatal("retired on the wrong provider") + } + ids := listedModels(t, s) + if slices.Contains(ids, "opencode-zen-free/exo-free") || !slices.Contains(ids, "b/exo-free") { + t.Fatalf("lists: %v", ids) + } +} + +func TestRetiredWords(t *testing.T) { + for _, c := range []struct { + status int + body string + want bool + }{ + {410, zenDeprecated, true}, + {404, `{"error":{"message":"The model text-davinci-003 has been deprecated","type":"invalid_request_error","code":"model_not_found"}}`, true}, + {400, `{"error":{"message":"max_tokens is deprecated, use max_completion_tokens"}}`, false}, + {410, `{"error":{"message":"gone"}}`, false}, + {429, zenDeprecated, false}, + } { + if got := modelRetired(c.status, []byte(c.body)); got != c.want { + t.Errorf("modelRetired(%d, %s) = %v", c.status, c.body, got) + } + } +} diff --git a/internal/gateway/routing.go b/internal/gateway/routing.go index 720ae3f2..a9971266 100644 --- a/internal/gateway/routing.go +++ b/internal/gateway/routing.go @@ -66,6 +66,7 @@ func served(rest, key string, tokens int) { func servedCandidate(c candidate, tokens int) { served(c.restKey(), c.restKey(), tokens) provider.NoteServed(c.p, time.Now()) + provider.Unretire(c.p.ID, c.model) // back in service, if it was said retired if id := c.restID(); id != c.restKey() { routed.Lock() delete(routed.failures, id) diff --git a/internal/provider/models.go b/internal/provider/models.go index cea86f1d..64ea3e67 100644 --- a/internal/provider/models.go +++ b/internal/provider/models.go @@ -1263,12 +1263,13 @@ func providerEntries() []Entry { func buildEntries() []Entry { var out []Entry s := settings.Load() + gone := retiredNow() // its vendor said it is retired (retired.go) for _, p := range All() { if !p.On() || p.DecideOnly() { // a dedicated decision API only routes continue } for _, m := range p.Exposed() { - if !p.isDecision(m) { + if !p.isDecision(m) && !gone[p.ID+"/"+m.ID] { out = append(out, entryFor(p, m, s)) } } diff --git a/internal/provider/retired.go b/internal/provider/retired.go new file mode 100644 index 00000000..7251aebb --- /dev/null +++ b/internal/provider/retired.go @@ -0,0 +1,150 @@ +package provider + +// A model its vendor has retired, though the vendor's own list still names +// it (MOMO on Discord: OpenCode Zen's /v1/models still lists exo-free, +// and a request for it gets 410 {"type":"error","error":{"type": +// "ModelDeprecated","message":"Model exo-free has been deprecated."}}). +// The gateway tells Retire when a provider answers so; the model is then +// left out of the catalog — /v1/models, every agent's model list, the +// routing groups magpie finds — for that provider only, until it answers +// a request again or retiredFor has passed. A request that names it as +// "provider/model" is still sent, so the vendor's answer is what the +// agent is told, and a model back in service is seen at once. + +import ( + "encoding/json" + "os" + "path/filepath" + "sync" + "time" + + "github.com/yetone/magpie/internal/catalog" +) + +// retiredFor is how long a model retired so stays out of the lists: the +// vendor's list is what brings it back, and the vendor's list is what +// still has it, so a refresh of the list can't be the end of it. A week +// later it is listed again, and the next request it gets refused so +// takes it out again. +const retiredFor = 7 * 24 * time.Hour + +var retired struct { + sync.Mutex + at map[string]time.Time // "provider/model" → when the vendor said it is retired + from string // the file read, and its time then + mod time.Time +} + +func retiredPath() string { return filepath.Join(filepath.Dir(Path()), "retired.json") } + +// readRetired brings retired.at up to the file, when the file changed since +// it was read: after a restart, or written by another magpie. Called with +// retired held. +func readRetired() { + path := retiredPath() + fi, err := os.Stat(path) + if err != nil { + if path != retired.from { + retired.at, retired.from, retired.mod = nil, path, time.Time{} + } + return + } + if path == retired.from && fi.ModTime().Equal(retired.mod) { + return + } + retired.from, retired.mod = path, fi.ModTime() + b, err := os.ReadFile(path) + if err != nil { + return + } + var saved map[string]time.Time + if json.Unmarshal(b, &saved) != nil { + return // unreadable: what was read before stands + } + retired.at = saved +} + +// writeRetired saves retired.at. Called with retired held. +func writeRetired() { + now := time.Now() + for k, at := range retired.at { + if now.Sub(at) >= retiredFor { + delete(retired.at, k) + } + } + b, err := json.MarshalIndent(retired.at, "", " ") + if err != nil { + return + } + path := retiredPath() + if os.MkdirAll(filepath.Dir(path), 0o700) != nil || writeFileAtomic(path, b) != nil { + return + } + if fi, err := os.Stat(path); err == nil { + retired.from, retired.mod = path, fi.ModTime() + } +} + +// Retire notes that provider pid's vendor said model is retired: it leaves +// the lists, and the agents' lists are written again. +func Retire(pid, model string) { + if pid == "" || model == "" { + return + } + k := pid + "/" + model + retired.Lock() + readRetired() + at, was := retired.at[k] + if was && time.Since(at) < retiredFor { + retired.Unlock() + return + } + if retired.at == nil { + retired.at = map[string]time.Time{} + } + retired.at[k] = time.Now() + writeRetired() + retired.Unlock() + catalog.Touched() +} + +// Unretire takes back a retirement of provider pid's model, when it has +// one: the vendor answered a request for it. +func Unretire(pid, model string) { + k := pid + "/" + model + retired.Lock() + readRetired() + if _, was := retired.at[k]; !was { + retired.Unlock() + return + } + delete(retired.at, k) + writeRetired() + retired.Unlock() + catalog.Touched() +} + +// Retired says whether provider pid's vendor said model is retired, within +// retiredFor. +func Retired(pid, model string) bool { + retired.Lock() + defer retired.Unlock() + readRetired() + at, ok := retired.at[pid+"/"+model] + return ok && time.Since(at) < retiredFor +} + +// retiredNow is the models retired now, by "provider/model", read once for +// a catalog built. +func retiredNow() map[string]bool { + retired.Lock() + defer retired.Unlock() + readRetired() + out := map[string]bool{} + for k, at := range retired.at { + if time.Since(at) < retiredFor { + out[k] = true + } + } + return out +}