okooooooo/magpie
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gateway, gui: Routing now says why an image was described for a model. Either nothing magpie knows says the model sees images, so it is counted as text only, or its provider's list or the user's own setting says text only. It names the fix for each: tick "Accepts images" in the provider's models, or pick the describing model in Settings › Models › Image recognition (#1287, Admin123456-check)

Browse Source 
The reporter set Codex to deepseek-v4.1-flash and saw a row "codex/gpt-6.1-sol · 图片描述" in Routing. The model was counted as unable to see images. With no Image recognition model set, magpie had a Codex account's model describe the image (AutoVision). The row said only "which can't see images", so it gave no reason and no way to change it.

CallFor gains Unknown. It is set when blindTo counted the model as text-only only because nothing says it sees (the request has no imageInput of its own). This holds on both the single-model path and the group candidate path. routing.js's kindWhy gives each case its own story. zh, zh-TW, ja and de are translated.

Verified:
- New TestDescriptionSaysWhyTheModelCantSee sends an image on the responses, codex, chat and anthropic routes. It covers an unknown model, a model whose list says text only, and a group of each. Without the gateway change it FAILs, e.g.
  vision_unknown_test.go:182: probe/text responses: the description's row is for {Agent:other Model:probe/text Unknown:false}, want unknown true
  with the same failure on codex, chat, anthropic and group/unknown.
- internal/gui/tests/routing-vision-why.test.cjs checks the vision row's tag and story in en and zh, in Chromium and WebKit. With routing.js from before this change it gives pass 0, fail 2.
- node --test routing-vision-why, routing-side-calls, gui-ja, gui-de, gui-zh-tw and i18n-keys: 18 pass, 0 fail.
- go vet -tags nogui ./internal/gateway ./internal/gui passes on darwin, GOOS=linux and GOOS=windows.
- HOME=<tmp> go test -tags nogui ./...: all ok except the root package and internal/plugin (TestHostRunsAsMagpieBun context deadline). Those fail only in the full run under load. Run alone, both pass (34.9s, 122.7s).

Not verified: the reporter's real provider. A throwaway test against the real models.dev data found no preset where deepseek-v4.1-flash counts as unable to see. So their case is one of: their provider's list says text only, a model setting, a stale models.dev cache, or a plugin. That is asked on the issue.
This commit is contained in:
yetone committed 2026-10-08 18:44:49 +08:00
1 parent 77f76ba203
commit e7de897f93
7 files changed
+204 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

internal/gateway/gateway.go
+8 -3
View File
@@ -1398,16 +1398,17 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
// a model its list says nothing of is text-only to a describer, as
// agents are told: while one describes images they are told every
// model takes them, and send the images on to be described
unknownSight := false
if imageInput == nil && !isGroup && hasImage(from, body) && blindTo(p.ID, model, nil) {
if _, ok := seeing(); ok {
no := false
imageInput = &no
imageInput, unknownSight = &no, true
}
}
if imageInput != nil && !*imageInput {
var currentImage bool
if see, ok := seeing(); ok && hasImage(from, body) {
seen, err := s.seenBody(withDescribeFor(r.Context(), call.Agent, call.Model, sessionOf(r.Header)), from, body, see)
seen, err := s.seenBody(withDescribeFor(r.Context(), call.Agent, call.Model, sessionOf(r.Header), unknownSight), from, body, see)
if err != nil {
call.Status, call.Error = 502, "image not described"
writeError(w, from, 502, fmt.Sprintf("model %q can't see images, and %s couldn't describe the image for it: %v", call.Model, see, err))
@@ -1584,10 +1585,13 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
// Described only when such a member is tried: a rule that sends the
// images to a model that sees asks for no description.
var seenGroup func() ([]byte, error)
// groupUnknown is whether the member the images are first described
// for is counted text-only for knowing nothing of it (CallFor.Unknown)
groupUnknown := false
if isGroup && hasImage(from, body) {
if see, ok := seeing(); ok {
seenGroup = sync.OnceValues(func() ([]byte, error) {
return s.seenBody(withDescribeFor(r.Context(), call.Agent, call.Model, sessionOf(r.Header)), from, body, see)
return s.seenBody(withDescribeFor(r.Context(), call.Agent, call.Model, sessionOf(r.Header), groupUnknown), from, body, see)
})
}
}
@@ -1717,6 +1721,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
if isGroup {
in := membersImageInput([]provider.Member{{Provider: c.p, Model: c.model}})
if seenGroup != nil && blindTo(c.p.ID, c.model, in) {
groupUnknown = in == nil
b, err := seenGroup()
if err != nil {
// only an image of the latest turn fails to be described
internal/gateway/search.go
+6
View File
@@ -54,6 +54,12 @@ type searchForKey struct{}
type CallFor struct {
Agent string `json:"agent"`
Model string `json:"model"`
// Unknown, on an image's description, is that the model was counted
// text-only because nothing magpie knows says whether it sees images
// (blindTo), not because its list or the user says it takes text only.
// The Routing view says which, so a user whose model does see knows
// to say so (#1287).
Unknown bool `json:"unknown,omitempty"`
}
func searchFor(ctx context.Context) *CallFor {
internal/gateway/vision.go
+2 -2
View File
@@ -71,8 +71,8 @@ type describeFor struct {
session string
}
func withDescribeFor(ctx context.Context, agent, model, session string) context.Context {
return context.WithValue(ctx, describeForKey{}, describeFor{&CallFor{Agent: agent, Model: model}, session})
func withDescribeFor(ctx context.Context, agent, model, session string, unknown bool) context.Context {
return context.WithValue(ctx, describeForKey{}, describeFor{&CallFor{Agent: agent, Model: model, Unknown: unknown}, session})
}
func describedFor(ctx context.Context) *CallFor {
internal/gateway/vision_unknown_test.go
+75
View File
@@ -1,6 +1,8 @@
package gateway
import (
"encoding/base64"
"fmt"
"net/http/httptest"
"strings"
"testing"
@@ -109,3 +111,76 @@ func TestDescriptionIsInRouting(t *testing.T) {
t.Fatalf("vision route: kind %q for %+v session %q; turn agent %q session %q", vision.Kind, vision.For, vision.Session, turn.Agent, turn.Session)
}
}
// The description's row says why the model was counted as unable to see:
// nothing magpie knows says it sees (Unknown), or its list says it takes
// text only — a DeepSeek model's images went to Codex's GPT with nothing
// saying which, or where to change it (#1287). A group's text-only member
// says it the same way, on every route an agent sends an image by.
func TestDescriptionSaysWhyTheModelCantSee(t *testing.T) {
s, _ := unknownEyed(t)
p, _ := provider.Find("probe")
if err := catalog.SaveLive("probe", p.Chat, []catalog.Model{
{ID: "text"},
{ID: "flat", ImageInput: imageInputBool(false)},
{ID: "eye", Images: true, ImageInput: imageInputBool(true)},
}); err != nil {
t.Fatal(err)
}
p.Models = append(p.Models, "flat")
if err := provider.Save(*p); err != nil {
t.Fatal(err)
}
for _, g := range []provider.Group{
{Name: "Unknown", Members: []string{"probe/text", "probe/eye"}, Routing: provider.Ordered},
{Name: "Flat", Members: []string{"probe/flat", "probe/eye"}, Routing: provider.Ordered},
} {
if err := provider.SaveGroup(g); err != nil {
t.Fatal(err)
}
}
img := 0
src := func() string { img++; return fmt.Sprintf("data:image/png;base64,%s", base64.StdEncoding.EncodeToString([]byte(fmt.Sprint("img", img)))) }
bodies := map[string]func(model string) (string, string){
"responses": func(m string) (string, string) {
return "/v1/responses", `{"model":"` + m + `","input":[{"role":"user","content":[{"type":"input_text","text":"read"},{"type":"input_image","image_url":"` + src() + `"}]}]}`
},
"codex": func(m string) (string, string) {
return CodexPath + "/responses", `{"model":"` + m + `","stream":true,"input":[{"role":"user","content":[{"type":"input_text","text":"read"},{"type":"input_image","image_url":"` + src() + `"}]}]}`
},
"chat": func(m string) (string, string) {
return "/v1/chat/completions", `{"model":"` + m + `","messages":[{"role":"user","content":[{"type":"text","text":"read"},{"type":"image_url","image_url":{"url":"` + src() + `"}}]}]}`
},
"anthropic": func(m string) (string, string) {
u := src()
_, data, _ := strings.Cut(u, ",")
return "/v1/messages", `{"model":"` + m + `","max_tokens":100,"messages":[{"role":"user","content":[{"type":"text","text":"read"},{"type":"image","source":{"type":"base64","media_type":"image/png","data":"` + data + `"}}]}]}`
},
}
for _, tc := range []struct {
model string
unknown bool
}{{"probe/text", true}, {"probe/flat", false}, {"group/unknown", true}, {"group/flat", false}} {
for name, body := range bodies {
path, b := body(tc.model)
before := len(s.trace.routes)
rec := httptest.NewRecorder()
s.Handler().ServeHTTP(rec, httptest.NewRequest("POST", path, strings.NewReader(b)))
if rec.Code != 200 {
t.Fatalf("%s %s: %d %s", tc.model, name, rec.Code, rec.Body.String())
}
var vision *Route
for _, r := range s.trace.routes[before:] {
if r.Kind == "vision" {
vision = r
}
}
if vision == nil || vision.For == nil {
t.Fatalf("%s %s: no description's row", tc.model, name)
}
if vision.For.Model != tc.model || vision.For.Unknown != tc.unknown {
t.Errorf("%s %s: the description's row is for %+v, want unknown %v", tc.model, name, *vision.For, tc.unknown)
}
}
}
}
internal/gui/assets/i18n.js
+8 -4
View File
@@ -2986,7 +2986,8 @@ const I18N = {
"magpie ran this web search for {agent}'s {model}, which can't search the web by itself: {searcher} searched, and {model} goes on answering once it has what was found. Not a turn of the conversation.": "magpie 代 {agent} 的 {model} 发起的联网搜索：{model} 无法自行搜索，由 {searcher} 搜索，{model} 拿到结果后继续回答。不属于对话轮次。",
"magpie ran this web search for a model that can't search the web by itself: {searcher} searched, and that model goes on answering once it has what was found. Not a turn of the conversation.": "magpie 代一个无法自行搜索的模型发起的联网搜索：由 {searcher} 搜索，该模型拿到结果后继续回答。不属于对话轮次。",
"Image description": "图片描述",
"magpie had {describer} describe an image for {agent}'s {model}, which can't see images: {model} is given the description in the image's place. Not a turn of the conversation.": "magpie 让 {describer} 为 {agent} 的 {model} 描述图片：{model} 无法看图，在图片位置收到这段描述。不属于对话轮次。",
"magpie had {describer} describe an image for {agent}'s {model}: nothing magpie knows says {model} can see images, so it is counted as text-only and given the description in the image's place. If it does see them, tick “Accepts images” for it in its provider's models. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie 让 {describer} 为 {agent} 的 {model} 描述图片：magpie 所知的信息都没有说 {model} 能看图，所以把它当作只收文本，在图片位置给它这段描述。如果它其实能看图，请在它的服务商的模型里勾选“支持图片输入”。描述图片的模型在 设置 › 模型 › 识图模型 里选。不属于对话轮次。",
"magpie had {describer} describe an image for {agent}'s {model}, which its provider's list or its own setting says takes text only: {model} is given the description in the image's place. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie 让 {describer} 为 {agent} 的 {model} 描述图片：它的服务商的模型列表或它自己的设置说它只收文本，{model} 在图片位置收到这段描述。描述图片的模型在 设置 › 模型 › 识图模型 里选。不属于对话轮次。",
"magpie had {describer} describe an image for a model that can't see images, which is given the description in the image's place. Not a turn of the conversation.": "magpie 让 {describer} 为一个无法看图的模型描述图片，该模型在图片位置收到这段描述。不属于对话轮次。",
"{agent} drafted the suggested prompts on its home page by itself, in the background, searching the project's files and connected apps, and checked them for safety. Not a turn of the conversation; Codex's Settings › Configuration › Suggested prompts turns it off.": "{agent} 在后台自行生成首页的提示词建议（会搜索项目文件和已连接的应用）并做安全检查，不属于对话轮次。可在 Codex 的 设置 › 配置 › 提示词建议 中关闭。",
"{agent} sent this turn on Luna Reserve, which it turns to once the plan's own allowance is used up; it picks the model itself.": "{agent} 用 Luna 储备发出了本轮（套餐额度用完后改用），模型由它自行选择。",
@@ -6729,7 +6730,8 @@ const I18N = {
"magpie ran this web search for {agent}'s {model}, which can't search the web by itself: {searcher} searched, and {model} goes on answering once it has what was found. Not a turn of the conversation.": "magpie 代 {agent} 的 {model} 發起的聯網搜尋：{model} 無法自行搜尋，由 {searcher} 搜尋，{model} 拿到結果後繼續回答。不屬於對話輪次。",
"magpie ran this web search for a model that can't search the web by itself: {searcher} searched, and that model goes on answering once it has what was found. Not a turn of the conversation.": "magpie 代一個無法自行搜尋的模型發起的聯網搜尋：由 {searcher} 搜尋，該模型拿到結果後繼續回答。不屬於對話輪次。",
"Image description": "圖片描述",
"magpie had {describer} describe an image for {agent}'s {model}, which can't see images: {model} is given the description in the image's place. Not a turn of the conversation.": "magpie 讓 {describer} 為 {agent} 的 {model} 描述圖片：{model} 無法看圖，在圖片位置收到這段描述。不屬於對話輪次。",
"magpie had {describer} describe an image for {agent}'s {model}: nothing magpie knows says {model} can see images, so it is counted as text-only and given the description in the image's place. If it does see them, tick “Accepts images” for it in its provider's models. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie 讓 {describer} 為 {agent} 的 {model} 描述圖片：magpie 所知的資訊都沒有說 {model} 能看圖，所以把它當作只收文字，在圖片位置給它這段描述。如果它其實能看圖，請在它的服務商的模型裡勾選「支援圖片輸入」。描述圖片的模型在 設定 › 模型 › 識圖模型 裡選。不屬於對話輪次。",
"magpie had {describer} describe an image for {agent}'s {model}, which its provider's list or its own setting says takes text only: {model} is given the description in the image's place. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie 讓 {describer} 為 {agent} 的 {model} 描述圖片：它的服務商的模型清單或它自己的設定說它只收文字，{model} 在圖片位置收到這段描述。描述圖片的模型在 設定 › 模型 › 識圖模型 裡選。不屬於對話輪次。",
"magpie had {describer} describe an image for a model that can't see images, which is given the description in the image's place. Not a turn of the conversation.": "magpie 讓 {describer} 為一個無法看圖的模型描述圖片，該模型在圖片位置收到這段描述。不屬於對話輪次。",
"{agent} drafted the suggested prompts on its home page by itself, in the background, searching the project's files and connected apps, and checked them for safety. Not a turn of the conversation; Codex's Settings › Configuration › Suggested prompts turns it off.": "{agent} 在後臺自行生成首頁的提示詞建議（會搜尋專案檔案和已連線的應用）並做安全檢查，不屬於對話輪次。可在 Codex 的 設定 › 設定 › 提示詞建議 中關閉。",
"{agent} sent this turn on Luna Reserve, which it turns to once the plan's own allowance is used up; it picks the model itself.": "{agent} 用 Luna 儲備發出了本輪（套餐額度用完後改用），模型由它自行選擇。",
@@ -10449,7 +10451,8 @@ const I18N = {
"magpie ran this web search for {agent}'s {model}, which can't search the web by itself: {searcher} searched, and {model} goes on answering once it has what was found. Not a turn of the conversation.": "magpie が、自分でウェブ検索できない {agent} の {model} のためにこのウェブ検索を実行しました。{searcher} が検索し、{model} は結果を受け取ってから応答を続けます。会話のターンではありません。",
"magpie ran this web search for a model that can't search the web by itself: {searcher} searched, and that model goes on answering once it has what was found. Not a turn of the conversation.": "magpie が自分でウェブ検索できないモデルのためにこのウェブ検索を実行しました。{searcher} が検索し、そのモデルは結果を受け取ってから応答を続けます。会話のターンではありません。",
"Image description": "画像の説明",
"magpie had {describer} describe an image for {agent}'s {model}, which can't see images: {model} is given the description in the image's place. Not a turn of the conversation.": "magpie が画像を見られない {agent} の {model} のために、{describer} に画像を説明させました。{model} には画像の代わりにその説明が渡されます。会話のターンではありません。",
"magpie had {describer} describe an image for {agent}'s {model}: nothing magpie knows says {model} can see images, so it is counted as text-only and given the description in the image's place. If it does see them, tick “Accepts images” for it in its provider's models. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie は {agent} の {model} のために {describer} に画像を説明させました。{model} が画像を見られると示す情報を magpie は持っていないため、テキストのみのモデルとして扱い、画像の代わりにその説明を渡しています。実際に画像を見られる場合は、プロバイダーのモデルで「画像入力に対応」をオンにしてください。説明するモデルは 設定 › モデル › 画像認識 で選べます。会話のターンではありません。",
"magpie had {describer} describe an image for {agent}'s {model}, which its provider's list or its own setting says takes text only: {model} is given the description in the image's place. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie は {agent} の {model} のために {describer} に画像を説明させました。プロバイダーのモデル一覧またはこのモデル自身の設定でテキストのみとされているため、{model} には画像の代わりにその説明が渡されます。説明するモデルは 設定 › モデル › 画像認識 で選べます。会話のターンではありません。",
"magpie had {describer} describe an image for a model that can't see images, which is given the description in the image's place. Not a turn of the conversation.": "magpie が画像を見られないモデルのために {describer} に画像を説明させ、そのモデルには画像の代わりにその説明が渡されます。会話のターンではありません。",
"{agent} drafted the suggested prompts on its home page by itself, in the background, searching the project's files and connected apps, and checked them for safety. Not a turn of the conversation; Codex's Settings › Configuration › Suggested prompts turns it off.": "{agent} がホーム画面のプロンプト候補をバックグラウンドで自ら作成し（プロジェクトのファイルや接続済みアプリを検索）、安全性をチェックしました。会話のターンではありません。Codex の 設定 › 構成 › Suggested prompts でオフにできます。",
"{agent} sent this turn on Luna Reserve, which it turns to once the plan's own allowance is used up; it picks the model itself.": "{agent} はこのターンを Luna Reserve で送信しました。プラン本来の利用枠を使い切ると Luna Reserve に切り替わり、モデルは自ら選びます。",
@@ -14186,7 +14189,8 @@ const I18N = {
"magpie ran this web search for {agent}'s {model}, which can't search the web by itself: {searcher} searched, and {model} goes on answering once it has what was found. Not a turn of the conversation.": "magpie hat diese Websuche für {model} von {agent} ausgeführt, das selbst nicht im Web suchen kann: {searcher} hat gesucht, und {model} antwortet weiter, sobald es die Ergebnisse hat. Keine Runde der Unterhaltung.",
"magpie ran this web search for a model that can't search the web by itself: {searcher} searched, and that model goes on answering once it has what was found. Not a turn of the conversation.": "magpie hat diese Websuche für ein Modell ausgeführt, das selbst nicht im Web suchen kann: {searcher} hat gesucht, und dieses Modell antwortet weiter, sobald es die Ergebnisse hat. Keine Runde der Unterhaltung.",
"Image description": "Bildbeschreibung",
"magpie had {describer} describe an image for {agent}'s {model}, which can't see images: {model} is given the description in the image's place. Not a turn of the conversation.": "magpie ließ {describer} ein Bild für {model} von {agent} beschreiben, das keine Bilder sehen kann: {model} erhält die Beschreibung anstelle des Bildes. Keine Runde der Unterhaltung.",
"magpie had {describer} describe an image for {agent}'s {model}: nothing magpie knows says {model} can see images, so it is counted as text-only and given the description in the image's place. If it does see them, tick “Accepts images” for it in its provider's models. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie ließ {describer} ein Bild für {model} von {agent} beschreiben: Nichts, was magpie weiß, sagt, dass {model} Bilder sehen kann, daher gilt es als reines Textmodell und erhält die Beschreibung anstelle des Bildes. Wenn es doch Bilder sieht, aktiviere „Akzeptiert Bilder“ dafür bei den Modellen seines Anbieters. Unter Einstellungen › Modelle › Bilderkennung wählst du das Modell, das beschreibt. Keine Runde der Unterhaltung.",
"magpie had {describer} describe an image for {agent}'s {model}, which its provider's list or its own setting says takes text only: {model} is given the description in the image's place. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.": "magpie ließ {describer} ein Bild für {model} von {agent} beschreiben, das laut der Modellliste seines Anbieters oder seiner eigenen Einstellung nur Text annimmt: {model} erhält die Beschreibung anstelle des Bildes. Unter Einstellungen › Modelle › Bilderkennung wählst du das Modell, das beschreibt. Keine Runde der Unterhaltung.",
"magpie had {describer} describe an image for a model that can't see images, which is given the description in the image's place. Not a turn of the conversation.": "magpie ließ {describer} ein Bild für ein Modell beschreiben, das keine Bilder sehen kann und die Beschreibung anstelle des Bildes erhält. Keine Runde der Unterhaltung.",
"{agent} drafted the suggested prompts on its home page by itself, in the background, searching the project's files and connected apps, and checked them for safety. Not a turn of the conversation; Codex's Settings › Configuration › Suggested prompts turns it off.": "{agent} hat die vorgeschlagenen Prompts auf seiner Startseite selbst im Hintergrund entworfen, dabei die Projektdateien und verbundene Apps durchsucht und sie auf Sicherheit geprüft. Keine Runde der Unterhaltung; abschalten lässt es sich in Codex unter Einstellungen › Konfiguration › Prompt-Vorschläge.",
"{agent} sent this turn on Luna Reserve, which it turns to once the plan's own allowance is used up; it picks the model itself.": "{agent} hat diese Runde über Luna Reserve gesendet, worauf es zurückgreift, sobald das eigene Kontingent des Plans aufgebraucht ist; das Modell wählt es selbst.",
internal/gui/assets/routing.js
+5 -1
View File
@@ -1498,8 +1498,12 @@
if (r.kind === "web_search") return r.for
? t("magpie ran this web search for {agent}'s {model}, which can't search the web by itself: {searcher} searched, and {model} goes on answering once it has what was found. Not a turn of the conversation.", { agent: agentName(r.for.agent), model: r.for.model, searcher: r.model })
: t("magpie ran this web search for a model that can't search the web by itself: {searcher} searched, and that model goes on answering once it has what was found. Not a turn of the conversation.", { searcher: r.model });
// why the model was counted as unable to see, and where the user says
// otherwise or picks the describer (#1287: a DeepSeek model's images
// went to Codex's GPT, and nothing said why or where to change it)
if (r.kind === "vision" && r.for?.unknown) return t("magpie had {describer} describe an image for {agent}'s {model}: nothing magpie knows says {model} can see images, so it is counted as text-only and given the description in the image's place. If it does see them, tick “Accepts images” for it in its provider's models. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.", { agent: agentName(r.for.agent), model: r.for.model, describer: r.model });
if (r.kind === "vision") return r.for
? t("magpie had {describer} describe an image for {agent}'s {model}, which can't see images: {model} is given the description in the image's place. Not a turn of the conversation.", { agent: agentName(r.for.agent), model: r.for.model, describer: r.model })
? t("magpie had {describer} describe an image for {agent}'s {model}, which its provider's list or its own setting says takes text only: {model} is given the description in the image's place. Settings › Models › Image recognition picks the model that describes. Not a turn of the conversation.", { agent: agentName(r.for.agent), model: r.for.model, describer: r.model })
: t("magpie had {describer} describe an image for a model that can't see images, which is given the description in the image's place. Not a turn of the conversation.", { describer: r.model });
return t("{agent} made this call itself ({kind}), not as a turn of the conversation, and picks its model itself.", { agent, kind: kindName(r.kind) });
}
internal/gui/tests/routing-vision-why.test.cjs
+100
View File
@@ -0,0 +1,100 @@
// Run with Node's test runner and Playwright on the module path; see README.md.
// A DeepSeek chat in Codex had its image described by Codex's GPT, and
// the row said only "Image description" (#1287: codex的图片识别还是调用默认的
// 模型): the row's story says why the model was counted as unable to see —
// nothing magpie knows says it sees, or its list says text only — and
// where that and the describer are changed.
const assert = require("node:assert/strict");
const fs = require("node:fs/promises");
const path = require("node:path");
const { test } = require("node:test");
const { chromium, webkit } = require("playwright");
const assets = path.resolve(__dirname, "../assets");
const now = new Date();
const day = [now.getFullYear(), now.getMonth() + 1, now.getDate()].map((n) => String(n).padStart(2, "0")).join("-");
const at = (i) => new Date(now.getTime() - (i + 1) * 60e3).toISOString();
const seat = (model, provider) => ({ id: provider, provider, name: provider === "codex" ? "Codex" : "DeepSeek", who: "a@b.c", kind: provider === "codex" ? "account" : "key", model });
const call = (id, i, agent, model, provider, extra) => ({
id, seq: id, time: at(i), agent, model: provider + "/" + model, provider, ...extra,
order: [seat(model, provider)], tries: [{ id: provider, model, start: at(i), done: true, status: 200, ms: 900 }],
done: true, status: 200, ms: 900, tokens: 1200,
});
const routes = [
call(202, 0, "magpie", "gpt-6.1-sol", "codex", { kind: "vision", for: { agent: "codex", model: "deepseek/deepseek-v4.1-flash", unknown: true } }),
call(201, 1, "magpie", "gpt-6.1-sol", "codex", { kind: "vision", for: { agent: "codex", model: "deepseek/deepseek-v4-pro" } }),
];
function serve(lang) {
const state = {
agents: [{ id: "codex", name: "Codex", path: "/test/config.toml", fields: [] }],
clients: [{ id: "codex", name: "Codex", icon: "codex-color" }, { id: "magpie", name: "magpie", icon: "magpie" }],
profiles: [], settings: { lang, theme: "light" },
};
return async (route) => {
const url = new URL(route.request().url());
const json = (data) => route.fulfill({ json: data });
if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:"${lang}",theme:"light",web:true};` });
if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
if (url.pathname === "/api/state") return json(state);
if (url.pathname === "/api/plugins") return json({ plugins: [] });
if (url.pathname === "/api/gateway/trace") {
if (url.searchParams.get("wait")) await new Promise((r) => setTimeout(r, 20e3));
return json({ mine: true, now: now.toISOString(), seq: 1, totals: { requests: 0, rerouted: 0, errors: 0 }, routes: [] });
}
if (url.pathname === "/api/gateway/history") {
const d = url.searchParams.get("day");
return json({ cut: false, days: [{ day, requests: routes.length }], routes: d ? routes : [] });
}
if (url.pathname === "/api/groups") return json({ groups: [] });
if (url.pathname === "/api/providers") return json({ providers: [], gateway: { running: true, window: true } });
if (url.pathname.startsWith("/api/")) return json({});
const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);
const contentType = { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)];
const body = await fs.readFile(file).catch(() => null);
await (body ? route.fulfill({ body, contentType }) : route.fulfill({ status: 404, body: "" }));
};
}
const want = {
en: { tag: "Image description", unknown: [/nothing magpie knows says deepseek\/deepseek-v4\.1-flash can see images/, /tick “Accepts images” for it in its provider's models/, /Settings › Models › Image recognition picks the model that describes/, /codex\/gpt-6\.1-sol describe an image for Codex's/],
listed: [/which its provider's list or its own setting says takes text only/, /Settings › Models › Image recognition/] },
zh: { tag: "图片描述", unknown: [/magpie 所知的信息都没有说 deepseek\/deepseek-v4\.1-flash 能看图/, /勾选“支持图片输入”/, /设置 › 模型 › 识图模型/, /让 codex\/gpt-6\.1-sol 为 Codex/],
listed: [/服务商的模型列表或它自己的设置说它只收文本/, /设置 › 模型 › 识图模型/] },
};
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
for (const lang of ["en", "zh"]) {
test(`${engine} ${lang}: an image's description says why the model can't see, and where to change it`, async (t) => {
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
const context = await browser.newContext({ viewport: { width: 1100, height: 760 }, reducedMotion: "reduce" });
const page = await context.newPage();
page.setDefaultTimeout(5000);
const errors = [];
page.on("pageerror", (e) => errors.push(e.message));
await page.route("**/*", serve(lang));
t.after(async () => {
if (process.env.ARTIFACT_DIR) {
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-vision-why.png`), fullPage: true });
}
await browser.close();
});
await page.goto("http://magpie.test/?view=routing");
await page.locator(".rt-days .rt-day").nth(1).click();
await page.locator(".rt-req").nth(routes.length - 1).waitFor();
const story = async (i) => {
await page.locator(".rt-req").nth(i).click();
await page.waitForTimeout(400);
return page.locator(".rt-steps li.kind");
};
let s = await story(0);
assert.equal(await s.locator(".kind").textContent(), want[lang].tag);
for (const re of want[lang].unknown) assert.match(await s.textContent(), re);
s = await story(1);
for (const re of want[lang].listed) assert.match(await s.textContent(), re);
assert.doesNotMatch(await s.textContent(), /nothing magpie knows|所知的信息/);
assert.deepEqual(errors, []);
});
}
}