providers: the add sheet lists Partners first, above Subscriptions, under a heading that says they sponsor magpie; who they are comes from usemagpie.ai/api/partners (site/partners.js), so a paid listing changes with a deploy, not a release (yetone)
The app fetches the list behind the call every 6 hours (10 minutes after a failure), never waiting, and keeps it in the cache folder. Each entry is checked: its own id, never a built-in preset's; a name and an endpoint; https only; at most 12; no decision API, no-key or user endpoint. Its logo goes through FetchIcon. A partner is a preset to the rest of magpie, and one that has left still is to the providers added from it, unsponsored. The window lists a partner in the languages it names only, with its tagline in the page's language; `magpie presets` and the TUI list them first too. MAGPIE_PARTNERS moves the list, off lists none. Run: go test -tags nogui -race ./internal/provider ./internal/gui ./internal/tui . (ok); TestPartners* -race -count=3; node --test internal/gui/tests/partners.test.cjs add-sheet gui-ja gui-de i18n-keys in Chromium and WebKit (en/zh/ja/de, 900 and 440px); node --test site/worker.test.mjs site/install.test.mjs; the built binary in a sandbox HOME against a local feed: listed, an http entry left out, listed from the cache with the site down, `provider add` from it.
This commit is contained in:
1 parent
5ba0601fda
commit
10578ce683
13 files changed
+857
-5
No files matched your search
@@ -30,6 +30,7 @@ keys and accounts per request is in [Gateway routing and fallback](gateway-routi
|
||||
| Allowance | `Quotas` asks every vendor at once: subscription windows (`SubscriptionUsage`), key-bought plans (`PlanQuotas`) and key balances (`KeyBalances`). The Usage page, `magpie quota --json` and `GET /v1/magpie/quotas` read it. A single card can be read again on its own (#840). A remote magpie's cards come from what it has kept (`CachedCards`, `GET /v1/magpie/quotas/cards`) and are never read there for this computer except by a card's refresh (`POST /v1/magpie/quotas/refresh`, throttled per card). What it has kept is the Usage page's cards with the readings made behind the requests since laid over them, a later one over an earlier by `readSeq` (`cards_kept.go`): each account's `LoginUsage` reading (the used-up switch, the Codex warm-up, routing's `Allowances`), and each key's `KeyAllowance` windows on its plan or balance card. An account or key read that way and never on the page gets a card from that reading, and after a restart, before either is read again, the cards come from `quotas.json`, dated (`AsOf`). A magpie serving only other magpies has nobody on its page (#1313). Cursor, Grok and a Kiro key keep the page's cards only. | [`cards_kept.go`](../../internal/provider/cards_kept.go), [`quotas.go`](../../internal/provider/quotas.go), [`subscription_usage.go`](../../internal/provider/subscription_usage.go), [`planquota.go`](../../internal/provider/planquota.go), [`balance.go`](../../internal/provider/balance.go), [`quota_refresh.go`](../../internal/provider/quota_refresh.go), [`remote_quotas.go`](../../internal/provider/remote_quotas.go) |
|
||||
| A key's own windows | A sub2api key given a spending limit per 5 hours, day or 7 days answers its `/v1/usage` with `rate_limits` and no `remaining` (`readSub2APIKeyLimits`). With the Balance URL set to it, the key's card shows them as the windows *5 hours*, *1 day*, *7 days*, in USD, beside a balance field the user wrote; *Check balance* and `magpie provider show` give what its fullest window has left (`Balance`). `KeyAllowance` gives routing the same windows: never waited for, read in the background at most once a minute, a failed read keeping the last, and seeded from the card's last reading (`quotas.json`) after a restart. A key bought as a plan (Kimi Code, GLM Coding Plan and Z.ai, MiniMax Coding Plan, OpenCode Go, Command Code: `planQuotaSourceOf`) gives routing its card's windows the same way (`planKeyWindows`, Zhipu's team quota behind a key with none of its own); the Usage page's read of its card is taken at once (`noteKeyAllowance`), and a key there that turns out pay-as-you-go, with no windows, is asked again only every 30 minutes (`keyNoWindowsAge`). StepFun's Step Plan is read through a sign-in, not the key, and isn't one of them. | [`sub2api_usage.go`](../../internal/provider/sub2api_usage.go), [`key_allowance.go`](../../internal/provider/key_allowance.go) |
|
||||
| Last served | `served.json` records when each account, plan or key last answered, shown as `lastServedAt` (#570). | [`served.go`](../../internal/provider/served.go) |
|
||||
| Partners | Services that pay to be listed first in the add sheet, under a heading of their own ("Partners · magpie's sponsors"). The list is `site/partners.js`, served at `usemagpie.ai/api/partners`: a change reaches users with a deploy, not a release. `Partners` never waits: it returns what was fetched last and fetches again behind the call every 6 hours (10 minutes after a failure), keeping the list in `partners.json` in the cache folder. Each entry is checked again on every read (`validPartners`): its own id, never a built-in preset's, a name, an endpoint, https only, at most 12; what only magpie's presets may set (a decision API, no key, a user endpoint) is cleared. Its logo is fetched through `FetchIcon` and kept as an uploaded icon. A partner is a preset to the rest of magpie (`Preset`, `FromPreset`, the editor), and one no longer listed still is to the providers added from it (`Seen`), without the sponsored mark. The window lists a partner in the languages it names (`langs`) only and shows its tagline in the page's language (`notes`, else `en`). `MAGPIE_PARTNERS` points it elsewhere, `off` lists none; tests list none unless they name a list. | [`partners.go`](../../internal/provider/partners.go), [`site/partners.js`](../../site/partners.js) |
|
||||
| Protocol detection | Works out which APIs a base URL speaks when a provider is added. Detection, endpoint tests and model tests share `probe`, which adds `anthropic-version` only for Anthropic Messages; OpenAI probes omit that default so a relay that refuses it can answer. User-supplied headers still apply through `Sign`. | [`detect.go`](../../internal/provider/detect.go), [`test.go`](../../internal/provider/test.go) |
|
||||
|
||||
## Runtime path
|
||||
@@ -62,6 +63,8 @@ keys and accounts per request is in [Gateway routing and fallback](gateway-routi
|
||||
go test -tags nogui ./internal/provider -run 'TestAccount|TestCodex|TestGroup|TestQuota|TestLogin|TestSignIn|TestSeveralKeys|TestKeyProtocol|TestSetKeyWeight|TestReadSub2APIKeyLimits|TestSub2APIKeyLimitsOnItsCard|TestKeyAllowance|TestPlanKeyAllowance|TestPayAsYouGoKeyAskedSeldom|TestClaudeAccountToldRenewed|TestPluginAccountToldRenewed'
|
||||
go test -tags nogui ./internal/provider -run 'TestVolcengine|TestQianfan|TestBedrock'
|
||||
go test -tags nogui ./internal/provider -run '^TestProviderProbesUseProtocolHeaders$'
|
||||
go test -tags nogui ./internal/provider -run '^TestPartners'
|
||||
node --test internal/gui/tests/partners.test.cjs site/worker.test.mjs
|
||||
go test -tags nogui ./internal/provider
|
||||
```
|
||||
|
||||
|
||||
@@ -6479,6 +6479,15 @@ function renderAdd() {
|
||||
const named = (x) => !f || x.name.toLowerCase().includes(f) || x.agent.includes(f) || "subscription".includes(f);
|
||||
const subs = SUBS.filter((x) => !(replacedSub(x.agent) || unusedSub(x.agent)) || signing?.agent === x.agent).map((x) => subOf(x.agent)).filter(named);
|
||||
const inPlugins = f && SUBS.some((x) => unusedSub(x.agent) && signing?.agent !== x.agent && named(subOf(x.agent)));
|
||||
// partners pay to be listed first, under a heading that says so; who
|
||||
// they are comes from usemagpie.ai (internal/provider/partners.go), and
|
||||
// each may name the languages it is shown in
|
||||
const partners = providers.presets.filter((p) => p.kind === "partner" && partnerShown(p) && (hit(p) || partnerNote(p).toLowerCase().includes(f)));
|
||||
if (partners.length) {
|
||||
any = true;
|
||||
const grid = section("Partners", "magpie's sponsors");
|
||||
for (const pr of partners) grid.append(partnerTile(pr));
|
||||
}
|
||||
if (subs.length || inPlugins) {
|
||||
any = true;
|
||||
const grid = section("Subscriptions", "sign in, no key");
|
||||
@@ -6688,6 +6697,32 @@ function tile(pr) {
|
||||
return b;
|
||||
}
|
||||
|
||||
// the page's language and its base ("zh-TW", "zh")
|
||||
function pageLangs() {
|
||||
const l = document.documentElement.lang || "en";
|
||||
return [l, l.split("-")[0]];
|
||||
}
|
||||
|
||||
// a partner listed in no language of its own is listed in every one
|
||||
function partnerShown(pr) {
|
||||
return !pr.langs?.length || pageLangs().some((l) => pr.langs.includes(l));
|
||||
}
|
||||
|
||||
// the partner's tagline in the page's language, else its English one; the
|
||||
// partner writes it, so it isn't translated here
|
||||
function partnerNote(pr) {
|
||||
const n = pr.notes || {};
|
||||
for (const l of pageLangs()) if (n[l]) return n[l];
|
||||
return n.en || "";
|
||||
}
|
||||
|
||||
// a partner's row: its own section says it is sponsored, so no badge
|
||||
function partnerTile(pr) {
|
||||
const b = tile({ ...pr, sponsored: false, note: "" });
|
||||
if (!pr.added) b.title = pr.name + (partnerNote(pr) ? " · " + partnerNote(pr) : "") + "\n" + hostOf(pr.chat || pr.responses || pr.anthropic);
|
||||
return b;
|
||||
}
|
||||
|
||||
let fieldIDs = 0;
|
||||
function field(label, control, hint) {
|
||||
const l = el("label", "", label);
|
||||
@@ -7642,7 +7677,7 @@ function drawEditor(p, presetID) {
|
||||
const h = el("div", "ehead");
|
||||
const copyOf = draft.copyOf && providers.providers.find((x) => x.id === draft.copyOf);
|
||||
h.append(icon(p?.icon || (copyOf && draft.icon) || pr?.icon || "generic"), el("b", "", p ? p.name : copyOf ? t("Copy of {name}", { name: copyOf.name }) : pr ? pr.name : t("Custom provider")));
|
||||
if (pr?.note) h.append(el("span", "note", t(pr.note)));
|
||||
if (pr?.kind === "partner") { if (partnerNote(pr)) h.append(el("span", "note", partnerNote(pr))); } else if (pr?.note) h.append(el("span", "note", t(pr.note)));
|
||||
h.append(el("span", "grow"));
|
||||
// a plugin's provider has plugin://<id> for its base, and its id is no
|
||||
// address to open: only a host with a dot or a port makes a link
|
||||
|
||||
@@ -1002,6 +1002,8 @@ const I18N = {
|
||||
"the makers' own APIs": "模型厂商的 API",
|
||||
"Subscriptions": "订阅",
|
||||
"sign in, no key": "登录即可,无需密钥",
|
||||
"Partners": "合作伙伴",
|
||||
"magpie's sponsors": "赞助 magpie 的服务",
|
||||
"Relays": "中转",
|
||||
"one key, many vendors": "一个密钥,多家模型",
|
||||
"Removed from magpie": "已从 magpie 移除",
|
||||
@@ -4773,6 +4775,8 @@ const I18N = {
|
||||
"the makers' own APIs": "模型廠商的 API",
|
||||
"Subscriptions": "訂閱",
|
||||
"sign in, no key": "登入即可,無需金鑰",
|
||||
"Partners": "合作夥伴",
|
||||
"magpie's sponsors": "贊助 magpie 的服務",
|
||||
"Relays": "中轉",
|
||||
"one key, many vendors": "一個金鑰,多家模型",
|
||||
"Removed from magpie": "已從 magpie 移除",
|
||||
@@ -8518,6 +8522,8 @@ const I18N = {
|
||||
"the makers' own APIs": "メーカー自身の API",
|
||||
"Subscriptions": "サブスクリプション",
|
||||
"sign in, no key": "サインインのみ、キー不要",
|
||||
"Partners": "パートナー",
|
||||
"magpie's sponsors": "magpie のスポンサー",
|
||||
"Relays": "中継サービス",
|
||||
"one key, many vendors": "1 つのキーで複数のベンダー",
|
||||
"Removed from magpie": "magpie から削除済み",
|
||||
@@ -12282,6 +12288,8 @@ const I18N = {
|
||||
"the makers' own APIs": "die eigenen APIs der Hersteller",
|
||||
"Subscriptions": "Abos",
|
||||
"sign in, no key": "anmelden, kein Schlüssel",
|
||||
"Partners": "Partner",
|
||||
"magpie's sponsors": "Sponsoren von magpie",
|
||||
"Relays": "Relays",
|
||||
"one key, many vendors": "ein Schlüssel, viele Anbieter",
|
||||
"Removed from magpie": "Aus magpie entfernt",
|
||||
|
||||
@@ -285,6 +285,9 @@ type presetJSON struct {
|
||||
// ZhipuTeam: a key of it may be on a team's GLM Coding Plan, whose
|
||||
// organization and project the editor offers to take
|
||||
ZhipuTeam bool `json:"zhipuTeam,omitempty"`
|
||||
// a partner's tagline by language, and the languages it is listed in
|
||||
Notes map[string]string `json:"notes,omitempty"`
|
||||
Langs []string `json:"langs,omitempty"`
|
||||
}
|
||||
|
||||
type gatewayJSON struct {
|
||||
@@ -639,6 +642,10 @@ func providersState() providersJSON {
|
||||
have[p.ID], have[p.Preset] = true, true
|
||||
s.Providers = append(s.Providers, providerInfo(p, uses))
|
||||
}
|
||||
// partners first, as the add sheet lists them
|
||||
for _, pa := range provider.Partners() {
|
||||
s.Presets = append(s.Presets, presetJSON{PresetDef: pa.PresetDef, Added: have[pa.ID], Notes: pa.Notes, Langs: pa.Langs})
|
||||
}
|
||||
for _, pr := range provider.Presets() {
|
||||
team := provider.TakesZhipuTeam(provider.Provider{Chat: pr.Chat, Responses: pr.Responses, Anthropic: pr.Anthropic})
|
||||
s.Presets = append(s.Presets, presetJSON{PresetDef: pr, Added: have[pr.ID], ZhipuTeam: team})
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
// Run with Node's test runner and Playwright on the module path; see README.md.
|
||||
// The add sheet's Partners (yetone: 在订阅上面加上一栏是「合作伙伴」): the
|
||||
// services that pay to be listed, first and under a heading that says so,
|
||||
// each one quiet row with no badge of its own. Who they are comes from
|
||||
// usemagpie.ai through /api/providers (internal/provider/partners.go); one
|
||||
// that names its languages is listed in those only, and its tagline, in
|
||||
// the row's title and the editor, is its own in the page's language, else
|
||||
// its English one. Searching finds one by its tagline. With no partner
|
||||
// there is no heading. English, Chinese, Japanese and German, at 900px and
|
||||
// 440px; no backend, the API is faked here.
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const { test } = require("node:test");
|
||||
const { chromium, webkit } = require("playwright");
|
||||
|
||||
const assets = path.resolve(__dirname, "../assets");
|
||||
|
||||
const partner = (id, name, extra = {}) => ({ id, name, icon: "generic", kind: "partner", sponsored: true, chat: `https://api.${id}.example.com/v1`, ...extra });
|
||||
const partners = [
|
||||
partner("acme", "Acme AI", { notes: { en: "Fast relay", zh: "快速中转", ja: "高速な中継" } }),
|
||||
partner("zhonly", "China Only", { langs: ["zh"], notes: { en: "Mainland relay", zh: "国内中转" } }),
|
||||
partner("taken", "Taken", { added: true }),
|
||||
];
|
||||
const presets = [
|
||||
{ id: "openai", name: "OpenAI", icon: "openai", kind: "vendor", chat: "https://api.openai.example.com/v1" },
|
||||
{ id: "openrouter", name: "OpenRouter", icon: "openai", kind: "relay", chat: "https://openrouter.example.com/v1", sponsored: true },
|
||||
];
|
||||
const providers = [{ id: "taken", name: "Taken", icon: "generic", preset: "taken", models: [], agents: [], key: { set: true, masked: "sk-…ab12" } }];
|
||||
|
||||
function server(lang, list) {
|
||||
return async (route) => {
|
||||
const url = new URL(route.request().url());
|
||||
const json = (data) => route.fulfill({ json: data });
|
||||
if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = {lang:"${lang}",theme:"light",web:false};` });
|
||||
if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
|
||||
if (url.pathname === "/api/state") return json({ agents: [], profiles: [], settings: { lang, theme: "light" } });
|
||||
if (url.pathname === "/api/providers") return json({ providers, presets: [...list, ...presets], excluded: [], gateway: { running: true, window: true } });
|
||||
if (url.pathname === "/api/groups") return json({ groups: [], pools: [] });
|
||||
if (url.pathname === "/api/gateway/trace") return json({ routes: [] });
|
||||
if (url.pathname.startsWith("/api/")) return json({});
|
||||
const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);
|
||||
const contentType = { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)];
|
||||
try { await route.fulfill({ body: await fs.readFile(file), contentType }); } catch { await route.fulfill({ status: 404, body: "" }); }
|
||||
};
|
||||
}
|
||||
|
||||
const L = {
|
||||
en: { head: ["Partners", "magpie's sponsors"], subs: "Subscriptions", rows: ["Acme AI", "Taken"], note: "Fast relay" },
|
||||
zh: { head: ["合作伙伴", "赞助 magpie 的服务"], subs: "订阅", rows: ["Acme AI", "China Only", "Taken"], note: "快速中转" },
|
||||
ja: { head: ["パートナー", "magpie のスポンサー"], subs: null, rows: ["Acme AI", "Taken"], note: "高速な中継" },
|
||||
de: { head: ["Partner", "Sponsoren von magpie"], subs: null, rows: ["Acme AI", "Taken"], note: "Fast relay" },
|
||||
};
|
||||
|
||||
async function openSheet(page) {
|
||||
await page.goto("http://magpie.test/?view=providers");
|
||||
await page.locator("#addProvider").click();
|
||||
const sheet = page.locator("#addSheet");
|
||||
await sheet.locator(".tile").first().waitFor();
|
||||
await sheet.evaluate((el) => Promise.all(el.getAnimations().map((a) => a.finished)));
|
||||
return sheet;
|
||||
}
|
||||
|
||||
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
|
||||
test(engine + ": partners come first in the add sheet", async (t) => {
|
||||
assert(["chromium", "webkit"].includes(engine), "BROWSER must be chromium or webkit");
|
||||
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
|
||||
t.after(() => browser.close());
|
||||
for (const lang of Object.keys(L)) for (const width of [900, 440]) {
|
||||
await t.test(lang + " " + width, async () => {
|
||||
const w = L[lang];
|
||||
const page = await (await browser.newContext({ viewport: { width, height: 800 } })).newPage();
|
||||
const errors = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await page.route("**/*", server(lang, partners));
|
||||
const sheet = await openSheet(page);
|
||||
|
||||
// first, under a heading that says they are sponsors
|
||||
const kinds = await sheet.locator(".kind").evaluateAll((ks) => ks.map((k) => [k.querySelector("b").textContent, k.querySelector("span")?.textContent || ""]));
|
||||
assert.deepEqual(kinds[0], w.head);
|
||||
if (w.subs) assert.equal(kinds[1][0], w.subs);
|
||||
|
||||
// its rows: listed in the page's language only, one quiet line each
|
||||
// with no badge, the tagline in the title
|
||||
const grid = sheet.locator(".kind").first().locator("xpath=following-sibling::div[1]");
|
||||
const rows = grid.locator(".tile");
|
||||
assert.deepEqual(await rows.locator(".n").allTextContents(), w.rows);
|
||||
const look = await rows.evaluateAll((rs) => rs.map((r) => ({ badge: r.querySelectorAll(".badge").length, over: r.scrollWidth > r.clientWidth + 1, h: r.getBoundingClientRect().height })));
|
||||
for (const l of look) {
|
||||
assert.equal(l.badge, 0);
|
||||
assert.equal(l.over, false);
|
||||
assert(l.h <= 40, "row " + l.h + "px tall");
|
||||
}
|
||||
const acme = rows.filter({ hasText: "Acme AI" });
|
||||
assert.match(await acme.getAttribute("title"), new RegExp("Acme AI · " + w.note));
|
||||
assert.match(await acme.getAttribute("title"), /api\.acme\.example\.com/);
|
||||
assert.equal(await rows.filter({ hasText: "Taken" }).locator(".have").count(), 1);
|
||||
// a sponsored built-in preset keeps its badge where it is
|
||||
assert.equal(await sheet.locator(".tile", { hasText: "OpenRouter" }).locator(".badge").count(), 1);
|
||||
|
||||
// the tagline finds it
|
||||
await page.locator("#addSheet input").first().fill(w.note.toLowerCase());
|
||||
await sheet.locator(".kind").first().waitFor();
|
||||
assert.deepEqual(await sheet.locator(".tile .n").allTextContents(), ["Acme AI"]);
|
||||
await page.locator("#addSheet input").first().fill("");
|
||||
|
||||
// its editor shows the tagline
|
||||
await sheet.locator(".tile", { hasText: "Acme AI" }).click();
|
||||
await page.locator("#modal .editor .ehead .note").waitFor();
|
||||
assert.equal(await page.locator("#modal .editor .ehead .note").textContent(), w.note);
|
||||
assert.deepEqual(errors, []);
|
||||
await page.context().close();
|
||||
});
|
||||
}
|
||||
await t.test("no partners, no heading", async () => {
|
||||
const page = await (await browser.newContext({ viewport: { width: 900, height: 800 } })).newPage();
|
||||
await page.route("**/*", server("en", []));
|
||||
const sheet = await openSheet(page);
|
||||
const heads = await sheet.locator(".kind b").allTextContents();
|
||||
assert(!heads.includes("Partners"), heads.join(","));
|
||||
assert.equal(heads[0], "Subscriptions");
|
||||
await page.context().close();
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,357 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/appdir"
|
||||
"github.com/yetone/magpie/internal/source"
|
||||
)
|
||||
|
||||
// Partners are services that pay to be listed first in the add sheet,
|
||||
// under a heading of their own that says so. Who they are changes without
|
||||
// a release: the list is fetched from PartnersURL, as the plugin market's
|
||||
// is, and kept on disk. A partner is a preset in all but where it comes
|
||||
// from, so adding one, its editor and its key page work as a preset's do.
|
||||
|
||||
// PartnersURL is where the list is kept; MAGPIE_PARTNERS moves it, and
|
||||
// MAGPIE_PARTNERS=off keeps to none.
|
||||
const PartnersURL = "https://usemagpie.ai/api/partners"
|
||||
|
||||
// KindPartner is a partner's Kind: the add sheet lists them apart.
|
||||
const KindPartner Kind = "partner"
|
||||
|
||||
// MaxPartners is the most partners listed; the rest of a longer list are
|
||||
// left out.
|
||||
const MaxPartners = 12
|
||||
|
||||
// Partner is one entry of the list: a preset, its tagline in each
|
||||
// language, the languages it is shown in, and when it is shown.
|
||||
type Partner struct {
|
||||
PresetDef
|
||||
// IconURL is the partner's logo, kept beside the user's own pictures
|
||||
// and shown as Icon.
|
||||
IconURL string `json:"iconUrl,omitempty"`
|
||||
// Notes is the tagline by language ("en", "zh", "zh-TW", "ja", ...);
|
||||
// "en" stands in for a language without one.
|
||||
Notes map[string]string `json:"notes,omitempty"`
|
||||
// Langs, when set, are the only languages the partner is listed in.
|
||||
Langs []string `json:"langs,omitempty"`
|
||||
// From and Until bound when it is listed; zero is unbounded.
|
||||
From time.Time `json:"from,omitzero"`
|
||||
Until time.Time `json:"until,omitzero"`
|
||||
}
|
||||
|
||||
// Live reports whether the partner is listed at now.
|
||||
func (p Partner) Live(now time.Time) bool {
|
||||
return (p.From.IsZero() || !now.Before(p.From)) && (p.Until.IsZero() || now.Before(p.Until))
|
||||
}
|
||||
|
||||
type partnerFeed struct {
|
||||
Partners []Partner `json:"partners"`
|
||||
}
|
||||
|
||||
// partnerCache is what is kept on disk: the list fetched last, and every
|
||||
// partner ever listed, so a provider added from one that has since left
|
||||
// still finds its preset (its endpoints, regions and key page).
|
||||
type partnerCache struct {
|
||||
At time.Time `json:"at"`
|
||||
Partners []Partner `json:"partners"`
|
||||
Seen map[string]Partner `json:"seen,omitempty"`
|
||||
}
|
||||
|
||||
var (
|
||||
partnerMu sync.Mutex
|
||||
partnerState *partnerCache // nil until read from disk
|
||||
partnerNext time.Time // the next fetch, when none is running
|
||||
partnerFetches bool
|
||||
)
|
||||
|
||||
const (
|
||||
partnerEvery = 6 * time.Hour
|
||||
partnerRetry = 10 * time.Minute
|
||||
)
|
||||
|
||||
var partnerID = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{1,40}$`)
|
||||
|
||||
func partnerCacheFile() string { return filepath.Join(appdir.Cache(), "partners.json") }
|
||||
|
||||
func partnersSource() string {
|
||||
src := os.Getenv("MAGPIE_PARTNERS")
|
||||
if src == "" {
|
||||
if testing.Testing() {
|
||||
// a test never asks the real site; one that wants a list
|
||||
// names its own
|
||||
return "off"
|
||||
}
|
||||
src = PartnersURL
|
||||
}
|
||||
return src
|
||||
}
|
||||
|
||||
// Partners are the partners listed now, in the list's order: what was
|
||||
// fetched last, never waited for. A list older than six hours is fetched
|
||||
// again behind the call, and the page asked again shows it.
|
||||
func Partners() []Partner {
|
||||
if partnersSource() == "off" {
|
||||
return nil
|
||||
}
|
||||
partnerMu.Lock()
|
||||
defer partnerMu.Unlock()
|
||||
loadPartners()
|
||||
if !partnerFetches && time.Now().After(partnerNext) {
|
||||
partnerFetches = true
|
||||
go refreshPartners()
|
||||
}
|
||||
now := time.Now()
|
||||
var out []Partner
|
||||
for _, p := range partnerState.Partners {
|
||||
if p.Live(now) {
|
||||
p.Sponsored = true
|
||||
out = append(out, p)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// PartnersNow is Partners for a command that ends at once, before a fetch
|
||||
// behind it could: a list due is fetched first, waited for at most wait.
|
||||
func PartnersNow(wait time.Duration) []Partner {
|
||||
if partnersSource() == "off" {
|
||||
return nil
|
||||
}
|
||||
partnerMu.Lock()
|
||||
loadPartners()
|
||||
due := !partnerFetches && time.Now().After(partnerNext)
|
||||
if due {
|
||||
partnerFetches = true
|
||||
}
|
||||
partnerMu.Unlock()
|
||||
if due {
|
||||
done := make(chan struct{})
|
||||
go func() { refreshPartners(); close(done) }()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(wait):
|
||||
}
|
||||
}
|
||||
return Partners()
|
||||
}
|
||||
|
||||
// loadPartners reads the list kept on disk the first time it's asked for;
|
||||
// partnerMu held.
|
||||
func loadPartners() {
|
||||
if partnerState != nil {
|
||||
return
|
||||
}
|
||||
partnerState = &partnerCache{Seen: map[string]Partner{}}
|
||||
b, err := os.ReadFile(partnerCacheFile())
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
var c partnerCache
|
||||
if json.Unmarshal(b, &c) != nil {
|
||||
return
|
||||
}
|
||||
if c.Seen == nil {
|
||||
c.Seen = map[string]Partner{}
|
||||
}
|
||||
// what was written is checked again: the rules may be stricter now
|
||||
c.Partners = validPartners(c.Partners)
|
||||
for id, p := range c.Seen {
|
||||
if v := validPartners([]Partner{p}); len(v) == 1 && v[0].ID == id {
|
||||
c.Seen[id] = v[0]
|
||||
} else {
|
||||
delete(c.Seen, id)
|
||||
}
|
||||
}
|
||||
partnerState = &c
|
||||
partnerNext = c.At.Add(partnerEvery)
|
||||
}
|
||||
|
||||
// refreshPartners fetches the list and keeps it; on a failure the list
|
||||
// held stays, and it is tried again in ten minutes.
|
||||
func refreshPartners() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
list, err := fetchPartners(ctx, partnersSource())
|
||||
if err == nil {
|
||||
for i := range list {
|
||||
list[i].Icon = partnerIcon(ctx, list[i])
|
||||
}
|
||||
}
|
||||
partnerMu.Lock()
|
||||
defer partnerMu.Unlock()
|
||||
partnerFetches = false
|
||||
loadPartners()
|
||||
if err != nil {
|
||||
partnerNext = time.Now().Add(partnerRetry)
|
||||
return
|
||||
}
|
||||
now := time.Now()
|
||||
partnerState.At, partnerState.Partners = now, list
|
||||
for _, p := range list {
|
||||
partnerState.Seen[p.ID] = p
|
||||
}
|
||||
partnerNext = now.Add(partnerEvery)
|
||||
if b, err := json.Marshal(partnerState); err == nil {
|
||||
_ = os.MkdirAll(filepath.Dir(partnerCacheFile()), 0o755)
|
||||
tmp := partnerCacheFile() + ".tmp"
|
||||
if os.WriteFile(tmp, b, 0o644) == nil {
|
||||
_ = os.Rename(tmp, partnerCacheFile())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func fetchPartners(ctx context.Context, src string) ([]Partner, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", src, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "magpie")
|
||||
req.Header.Set("Accept", "application/json")
|
||||
res, err := source.Do(http.DefaultClient, req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer res.Body.Close()
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("partners: %s", res.Status)
|
||||
}
|
||||
var f partnerFeed
|
||||
if err := json.NewDecoder(io.LimitReader(res.Body, 1<<20)).Decode(&f); err != nil {
|
||||
return nil, fmt.Errorf("partners: %w", err)
|
||||
}
|
||||
return validPartners(f.Partners), nil
|
||||
}
|
||||
|
||||
// validPartners keeps the entries magpie can list safely, at most
|
||||
// MaxPartners: an id of its own that no built-in preset has, a name, and
|
||||
// https addresses only. What a partner may not set (a decision API, no
|
||||
// key, an endpoint of the user's own) is cleared.
|
||||
func validPartners(in []Partner) []Partner {
|
||||
var out []Partner
|
||||
ids := map[string]bool{}
|
||||
for _, p := range in {
|
||||
if len(out) == MaxPartners {
|
||||
break
|
||||
}
|
||||
if err := checkPartner(p); err != nil || ids[p.ID] {
|
||||
continue
|
||||
}
|
||||
ids[p.ID] = true
|
||||
p.Kind, p.Sponsored = KindPartner, false
|
||||
p.Decide, p.NoKey, p.Hosts = "", false, false
|
||||
p.Endpoint, p.EndpointHint, p.EndpointNeeded = "", "", ""
|
||||
for i := range p.Regions {
|
||||
p.Regions[i].Decide = ""
|
||||
}
|
||||
if p.Chat == "" && p.Responses == "" && p.Anthropic == "" {
|
||||
// the first region is the default, as a built-in preset's is
|
||||
r := p.Regions[0]
|
||||
p.Chat, p.Responses, p.Anthropic = r.Chat, r.Responses, r.Anthropic
|
||||
}
|
||||
p.Note = p.Notes["en"]
|
||||
if !strings.HasPrefix(p.Icon, "file:") || IconFile(strings.TrimPrefix(p.Icon, "file:")) == "" {
|
||||
p.Icon = "generic"
|
||||
}
|
||||
out = append(out, p)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func checkPartner(p Partner) error {
|
||||
if !partnerID.MatchString(p.ID) {
|
||||
return errors.New("bad id")
|
||||
}
|
||||
if builtinPreset(p.ID) != nil {
|
||||
return errors.New("a built-in preset's id")
|
||||
}
|
||||
if strings.TrimSpace(p.Name) == "" {
|
||||
return errors.New("no name")
|
||||
}
|
||||
if p.Chat == "" && p.Responses == "" && p.Anthropic == "" && len(p.Regions) == 0 {
|
||||
return errors.New("no endpoint")
|
||||
}
|
||||
urls := []string{p.Chat, p.Responses, p.Anthropic, p.Website, p.KeysURL, p.IconURL}
|
||||
for _, r := range p.Regions {
|
||||
if r.Chat == "" && r.Responses == "" && r.Anthropic == "" {
|
||||
return errors.New("a region with no endpoint")
|
||||
}
|
||||
urls = append(urls, r.Chat, r.Responses, r.Anthropic, r.KeysURL, r.Website)
|
||||
}
|
||||
for _, u := range urls {
|
||||
if u != "" && !httpsURL(u) {
|
||||
return fmt.Errorf("not https: %s", u)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func httpsURL(s string) bool {
|
||||
u, err := url.Parse(s)
|
||||
return err == nil && u.Scheme == "https" && u.Host != "" && u.User == nil
|
||||
}
|
||||
|
||||
// partnerIcon keeps the partner's logo, through FetchIcon's checks, and
|
||||
// returns the Icon that shows it: the one kept before when it can't be
|
||||
// fetched, else the generic one.
|
||||
func partnerIcon(ctx context.Context, p Partner) string {
|
||||
if p.IconURL == "" {
|
||||
return "generic"
|
||||
}
|
||||
if icon, err := fetchPartnerIcon(ctx, p.IconURL); err == nil {
|
||||
return icon
|
||||
}
|
||||
partnerMu.Lock()
|
||||
defer partnerMu.Unlock()
|
||||
loadPartners()
|
||||
if s, ok := partnerState.Seen[p.ID]; ok && s.IconURL == p.IconURL {
|
||||
return s.Icon
|
||||
}
|
||||
return "generic"
|
||||
}
|
||||
|
||||
var fetchPartnerIcon = FetchIcon
|
||||
|
||||
// partnerPreset is the preset of a partner listed now or before, for a
|
||||
// provider added from it: one no longer listed isn't Sponsored.
|
||||
func partnerPreset(id string) *PresetDef {
|
||||
if partnersSource() == "off" {
|
||||
return nil
|
||||
}
|
||||
partnerMu.Lock()
|
||||
defer partnerMu.Unlock()
|
||||
loadPartners()
|
||||
p, ok := partnerState.Seen[id]
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
d := p.PresetDef
|
||||
d.Sponsored = false
|
||||
for _, l := range partnerState.Partners {
|
||||
if l.ID == id && l.Live(time.Now()) {
|
||||
d.Sponsored = true
|
||||
}
|
||||
}
|
||||
return &d
|
||||
}
|
||||
|
||||
// resetPartners forgets the list held in memory, for tests.
|
||||
func resetPartners() {
|
||||
partnerMu.Lock()
|
||||
partnerState, partnerNext, partnerFetches = nil, time.Time{}, false
|
||||
partnerMu.Unlock()
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fetchPartnersNow fetches the list as the background refresh does, and
|
||||
// waits for it.
|
||||
func fetchPartnersNow() {
|
||||
partnerMu.Lock()
|
||||
loadPartners()
|
||||
partnerMu.Unlock()
|
||||
refreshPartners()
|
||||
}
|
||||
|
||||
func partnerFeedServer(t *testing.T, body *atomic.Value) *httptest.Server {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("User-Agent") != "magpie" {
|
||||
t.Errorf("User-Agent %q", r.Header.Get("User-Agent"))
|
||||
}
|
||||
_, _ = rw.Write([]byte(body.Load().(string)))
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
return srv
|
||||
}
|
||||
|
||||
func partnerIDs(ps []Partner) string {
|
||||
var ids []string
|
||||
for _, p := range ps {
|
||||
ids = append(ids, p.ID)
|
||||
}
|
||||
return strings.Join(ids, ",")
|
||||
}
|
||||
|
||||
// The add sheet's Partners are what usemagpie.ai lists, checked: an entry
|
||||
// that would send a key over http, take a built-in preset's id or set
|
||||
// what only magpie's own presets may is left out, and one outside its
|
||||
// window isn't listed.
|
||||
func TestPartnersFromTheFeed(t *testing.T) {
|
||||
t.Setenv("XDG_CACHE_HOME", t.TempDir())
|
||||
resetPartners()
|
||||
t.Cleanup(resetPartners)
|
||||
was := fetchPartnerIcon
|
||||
t.Cleanup(func() { fetchPartnerIcon = was })
|
||||
fetchPartnerIcon = func(_ context.Context, u string) (string, error) {
|
||||
if u != "https://acme.example/logo.png" {
|
||||
t.Errorf("icon fetched from %q", u)
|
||||
}
|
||||
return StoreIcon([]byte("\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR"))
|
||||
}
|
||||
past := time.Now().Add(-time.Hour).UTC().Format(time.RFC3339)
|
||||
future := time.Now().Add(time.Hour).UTC().Format(time.RFC3339)
|
||||
var body atomic.Value
|
||||
body.Store(`{"partners":[
|
||||
{"id":"acme","name":"Acme AI","chat":"https://api.acme.example/v1","keysUrl":"https://acme.example/keys?ref=magpie",
|
||||
"iconUrl":"https://acme.example/logo.png","notes":{"en":"Fast relay","zh":"快速中转"},"langs":["zh"],
|
||||
"decide":"https://api.acme.example/decide","noKey":true},
|
||||
{"id":"plain","name":"Plain","chat":"http://api.plain.example/v1"},
|
||||
{"id":"openrouter","name":"Not OpenRouter","chat":"https://evil.example/v1"},
|
||||
{"id":"Bad_Id","name":"Bad","chat":"https://bad.example/v1"},
|
||||
{"id":"noname","chat":"https://x.example/v1"},
|
||||
{"id":"later","name":"Later","chat":"https://later.example/v1","from":"` + future + `"},
|
||||
{"id":"gone","name":"Gone","chat":"https://gone.example/v1","until":"` + past + `"},
|
||||
{"id":"regioned","name":"Regioned","regions":[{"id":"cn","name":"China","chat":"https://cn.regioned.example/v1"}]},
|
||||
{"id":"acme","name":"Acme twice","chat":"https://twice.example/v1"}
|
||||
]}`)
|
||||
t.Setenv("MAGPIE_PARTNERS", partnerFeedServer(t, &body).URL)
|
||||
|
||||
if got := Partners(); len(got) != 0 {
|
||||
t.Fatalf("listed before any list was fetched: %v", partnerIDs(got))
|
||||
}
|
||||
fetchPartnersNow()
|
||||
got := Partners()
|
||||
if ids := partnerIDs(got); ids != "acme,regioned" {
|
||||
t.Fatalf("partners = %s, want acme,regioned", ids)
|
||||
}
|
||||
a := got[0]
|
||||
if a.Kind != KindPartner || !a.Sponsored || a.Decide != "" || a.NoKey {
|
||||
t.Errorf("acme = kind %q sponsored %v decide %q noKey %v", a.Kind, a.Sponsored, a.Decide, a.NoKey)
|
||||
}
|
||||
if !strings.HasPrefix(a.Icon, "file:") || a.Note != "Fast relay" || a.Notes["zh"] != "快速中转" || len(a.Langs) != 1 {
|
||||
t.Errorf("acme = icon %q note %q notes %v langs %v", a.Icon, a.Note, a.Notes, a.Langs)
|
||||
}
|
||||
if got[1].Chat != "https://cn.regioned.example/v1" {
|
||||
t.Errorf("a partner with regions only is at %q, not its first region", got[1].Chat)
|
||||
}
|
||||
|
||||
// a provider is added from it as from a preset
|
||||
p, err := FromPreset("acme")
|
||||
if err != nil || p.Chat != "https://api.acme.example/v1" || p.KeysURL != "https://acme.example/keys?ref=magpie" || p.Decide != "" {
|
||||
t.Fatalf("FromPreset(acme) = %+v, %v", p, err)
|
||||
}
|
||||
if pr := Preset("acme"); pr == nil || !pr.Sponsored {
|
||||
t.Fatalf("Preset(acme) = %+v", pr)
|
||||
}
|
||||
// a built-in preset is never a partner's
|
||||
if pr := Preset("openrouter"); pr == nil || pr.Kind == KindPartner || strings.Contains(pr.Chat, "evil") {
|
||||
t.Fatalf("Preset(openrouter) = %+v", pr)
|
||||
}
|
||||
|
||||
// it leaves the list: no longer listed nor sponsored, but a provider
|
||||
// added from it still finds its preset, after a restart too
|
||||
body.Store(`{"partners":[]}`)
|
||||
fetchPartnersNow()
|
||||
if got := Partners(); len(got) != 0 {
|
||||
t.Fatalf("partners after the list emptied = %s", partnerIDs(got))
|
||||
}
|
||||
resetPartners()
|
||||
if pr := Preset("acme"); pr == nil || pr.Sponsored || pr.Chat != "https://api.acme.example/v1" {
|
||||
t.Fatalf("Preset(acme) after it left = %+v", pr)
|
||||
}
|
||||
|
||||
// a failed fetch keeps the list held, from disk after a restart
|
||||
body.Store(`{"partners":[{"id":"acme","name":"Acme AI","chat":"https://api.acme.example/v1"}]}`)
|
||||
fetchPartnersNow()
|
||||
body.Store(`not json`)
|
||||
resetPartners()
|
||||
fetchPartnersNow()
|
||||
if ids := partnerIDs(Partners()); ids != "acme" {
|
||||
t.Fatalf("partners after a failed fetch = %q, want acme", ids)
|
||||
}
|
||||
}
|
||||
|
||||
// MAGPIE_PARTNERS=off lists none and asks nobody, whatever was kept.
|
||||
func TestPartnersOff(t *testing.T) {
|
||||
t.Setenv("XDG_CACHE_HOME", t.TempDir())
|
||||
resetPartners()
|
||||
t.Cleanup(resetPartners)
|
||||
var asked atomic.Int32
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
|
||||
asked.Add(1)
|
||||
_, _ = rw.Write([]byte(`{"partners":[{"id":"acme","name":"Acme","chat":"https://a.example/v1"}]}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
t.Setenv("MAGPIE_PARTNERS", srv.URL)
|
||||
fetchPartnersNow()
|
||||
if _, err := os.Stat(partnerCacheFile()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MAGPIE_PARTNERS", "off")
|
||||
resetPartners()
|
||||
if got := Partners(); len(got) != 0 || Preset("acme") != nil {
|
||||
t.Fatalf("off listed %s", partnerIDs(got))
|
||||
}
|
||||
if asked.Load() != 1 {
|
||||
t.Fatalf("asked %d times", asked.Load())
|
||||
}
|
||||
}
|
||||
|
||||
// Partners never blocks on the site, and asks it at most once at a time.
|
||||
func TestPartnersDoesNotWait(t *testing.T) {
|
||||
t.Setenv("XDG_CACHE_HOME", t.TempDir())
|
||||
resetPartners()
|
||||
t.Cleanup(resetPartners)
|
||||
release := make(chan struct{})
|
||||
var asked atomic.Int32
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
|
||||
asked.Add(1)
|
||||
<-release
|
||||
_, _ = rw.Write([]byte(`{"partners":[{"id":"acme","name":"Acme","chat":"https://a.example/v1"}]}`))
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
t.Cleanup(func() {
|
||||
// the fetch ends with the test
|
||||
close(release)
|
||||
for {
|
||||
partnerMu.Lock()
|
||||
busy := partnerFetches
|
||||
partnerMu.Unlock()
|
||||
if !busy {
|
||||
return
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
t.Setenv("MAGPIE_PARTNERS", srv.URL)
|
||||
start := time.Now()
|
||||
for range 5 {
|
||||
Partners()
|
||||
}
|
||||
if d := time.Since(start); d > time.Second {
|
||||
t.Fatalf("Partners waited %v", d)
|
||||
}
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for asked.Load() == 0 && time.Now().Before(deadline) {
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
if n := asked.Load(); n != 1 {
|
||||
t.Fatalf("asked %d times, want 1", n)
|
||||
}
|
||||
}
|
||||
|
||||
// PartnersNow, for a command that ends at once, has the list fetched before
|
||||
// it answers, and doesn't wait past its bound.
|
||||
func TestPartnersNow(t *testing.T) {
|
||||
t.Setenv("XDG_CACHE_HOME", t.TempDir())
|
||||
resetPartners()
|
||||
t.Cleanup(resetPartners)
|
||||
var body atomic.Value
|
||||
body.Store(`{"partners":[{"id":"acme","name":"Acme","chat":"https://a.example/v1"}]}`)
|
||||
t.Setenv("MAGPIE_PARTNERS", partnerFeedServer(t, &body).URL)
|
||||
if ids := partnerIDs(PartnersNow(5 * time.Second)); ids != "acme" {
|
||||
t.Fatalf("PartnersNow = %q, want acme", ids)
|
||||
}
|
||||
|
||||
resetPartners()
|
||||
release := make(chan struct{})
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) { <-release }))
|
||||
t.Cleanup(srv.Close)
|
||||
t.Cleanup(func() {
|
||||
close(release)
|
||||
for {
|
||||
partnerMu.Lock()
|
||||
busy := partnerFetches
|
||||
partnerMu.Unlock()
|
||||
if !busy {
|
||||
return
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
t.Setenv("MAGPIE_PARTNERS", srv.URL)
|
||||
// the list kept is due again
|
||||
partnerMu.Lock()
|
||||
loadPartners()
|
||||
partnerNext = time.Time{}
|
||||
partnerMu.Unlock()
|
||||
start := time.Now()
|
||||
if ids := partnerIDs(PartnersNow(100 * time.Millisecond)); ids != "acme" {
|
||||
t.Fatalf("PartnersNow while the site hangs = %q, want the kept acme", ids)
|
||||
}
|
||||
if d := time.Since(start); d < 100*time.Millisecond || d > 2*time.Second {
|
||||
t.Fatalf("PartnersNow waited %v, want its bound", d)
|
||||
}
|
||||
}
|
||||
@@ -543,8 +543,17 @@ var presetAliases = map[string]struct{ preset, region string }{
|
||||
}
|
||||
|
||||
// Preset finds a preset by id, or by an id it carried before
|
||||
// (presetAliases).
|
||||
// (presetAliases), or a partner listed now or before (partners.go).
|
||||
func Preset(id string) *PresetDef {
|
||||
if pr := builtinPreset(id); pr != nil {
|
||||
return pr
|
||||
}
|
||||
return partnerPreset(id)
|
||||
}
|
||||
|
||||
// builtinPreset is the preset of that id built into magpie, partners'
|
||||
// aside.
|
||||
func builtinPreset(id string) *PresetDef {
|
||||
if a, ok := presetAliases[id]; ok {
|
||||
id = a.preset
|
||||
}
|
||||
|
||||
@@ -402,6 +402,9 @@ func (m *model) openProviderModels(id string) {
|
||||
// openPresets picks a vendor magpie knows, then asks for its key.
|
||||
func (m *model) openPresets() {
|
||||
var items []agent.Option
|
||||
for _, d := range provider.Partners() {
|
||||
items = append(items, agent.Option{Value: d.ID, Note: d.Name + " · partner (sponsor)"})
|
||||
}
|
||||
for _, d := range provider.Presets() {
|
||||
items = append(items, agent.Option{Value: d.ID, Note: d.Name + " · " + string(d.Kind)})
|
||||
}
|
||||
|
||||
+8
-3
@@ -190,13 +190,18 @@ func presets() error {
|
||||
have[p.ID], have[p.Preset] = true, true
|
||||
}
|
||||
kind := provider.Kind("")
|
||||
for _, pr := range provider.Presets() {
|
||||
// partners first, as the app lists them: their heading says they pay
|
||||
all := []provider.PresetDef{}
|
||||
for _, pa := range provider.PartnersNow(3 * time.Second) {
|
||||
all = append(all, pa.PresetDef)
|
||||
}
|
||||
for _, pr := range append(all, provider.Presets()...) {
|
||||
if pr.Kind != kind {
|
||||
kind = pr.Kind
|
||||
fmt.Println(faint.Render(" " + map[provider.Kind]string{provider.KindVendor: "vendors", provider.KindRelay: "relays", provider.KindLocal: "local"}[kind]))
|
||||
fmt.Println(faint.Render(" " + map[provider.Kind]string{provider.KindPartner: "partners (sponsors)", provider.KindVendor: "vendors", provider.KindRelay: "relays", provider.KindLocal: "local"}[kind]))
|
||||
}
|
||||
name := bold.Render(pr.Name)
|
||||
if pr.Sponsored {
|
||||
if pr.Sponsored && pr.Kind != provider.KindPartner {
|
||||
name += " " + faint.Render("sponsored")
|
||||
}
|
||||
state := muted.Render("magpie provider add " + pr.ID + " <key>")
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
// The add sheet's Partners: the services that pay to be listed first in
|
||||
// magpie's "Add provider" sheet, under a heading that says they sponsor
|
||||
// magpie. The app fetches this list from /api/partners every six hours and
|
||||
// keeps it (internal/provider/partners.go), so a change here reaches users
|
||||
// with `npx wrangler deploy`, no release. At most 12 are listed, in this
|
||||
// order.
|
||||
//
|
||||
// An entry is a preset (internal/provider/presets.go) and a few fields more:
|
||||
//
|
||||
// id a-z, 0-9 and "-", 2-41 characters; never a built-in preset's
|
||||
// id. It is kept in the providers users add from it: don't
|
||||
// change it, and don't give it to someone else later.
|
||||
// name the row's name; short a shorter one, when name is long
|
||||
// chat OpenAI chat completions base URL (…/v1)
|
||||
// responses OpenAI Responses base URL; anthropic Anthropic messages base URL
|
||||
// (at least one of the three, or regions)
|
||||
// regions [{id, name, chat, responses, anthropic, keysUrl, website}]: the
|
||||
// first is the default
|
||||
// keysUrl where the user makes a key (a referral link is fine)
|
||||
// website the service's home page
|
||||
// iconUrl its logo (png, svg, webp, jpeg; square), fetched and kept by
|
||||
// each app
|
||||
// notes the tagline by language: {en, zh, "zh-TW", ja, de}; en stands
|
||||
// in for a language without one
|
||||
// langs the app languages it is listed in (["zh"]); none is all
|
||||
// from, until RFC 3339 times bounding when it is listed (the paid term)
|
||||
//
|
||||
// Every URL is https. An entry that breaks a rule is left out by the app.
|
||||
export const PARTNERS = [];
|
||||
@@ -8,6 +8,7 @@
|
||||
// Both take ?lang=zh for the notes in Chinese, where a
|
||||
// release has them (below its <!-- lang:zh --> marker);
|
||||
// any other lang, or none, is the English alone.
|
||||
// /api/partners {partners: [...]}: the add sheet's Partners (partners.js)
|
||||
// /download the Apple Silicon dmg
|
||||
// /download/mac-arm64 the same; /download/mac-intel the Intel dmg
|
||||
// /download/windows the Windows app (x64); /download/windows-arm64
|
||||
@@ -22,6 +23,7 @@
|
||||
// Everything else is the static site in public/.
|
||||
|
||||
import { LANGS } from "./i18n.js";
|
||||
import { PARTNERS } from "./partners.js";
|
||||
|
||||
const REPO = "yetone/magpie-releases";
|
||||
const TTL = 300; // seconds the newest release is remembered
|
||||
@@ -56,6 +58,9 @@ export default {
|
||||
const lang = url.searchParams.get("lang");
|
||||
return json({ ...got.rel, notes: inLang(got.rel.notes, lang) }, 200, cacheFor(got));
|
||||
}
|
||||
if (url.pathname === "/api/partners") {
|
||||
return json({ partners: PARTNERS }, 200, { "Cache-Control": "public, max-age=600" });
|
||||
}
|
||||
if (url.pathname === "/api/notes") {
|
||||
const got = await releases(ctx, env);
|
||||
if (!got) return json({ error: "no releases" }, 503, NO_STORE);
|
||||
|
||||
@@ -270,3 +270,26 @@ test("docs: an English docs page sends a browser that prefers Chinese or Japanes
|
||||
const res = await worker.fetch(new Request("https://usemagpie.ai/docs/zh/start", { headers: { Cookie: "lang=en" } }), env, ctx);
|
||||
assert.equal(res.status, 200);
|
||||
});
|
||||
|
||||
// /api/partners is partners.js, cached ten minutes; every entry keeps the
|
||||
// rules the app checks (internal/provider/partners.go), so none is dropped
|
||||
// there unseen.
|
||||
test("partners", async () => {
|
||||
const { PARTNERS } = await import("./partners.js");
|
||||
const res = await ask("/api/partners");
|
||||
assert.equal(res.status, 200);
|
||||
assert.equal(res.headers.get("Cache-Control"), "public, max-age=600");
|
||||
assert.deepEqual(await res.json(), { partners: PARTNERS });
|
||||
assert(PARTNERS.length <= 12);
|
||||
const ids = new Set();
|
||||
for (const p of PARTNERS) {
|
||||
assert.match(p.id, /^[a-z0-9][a-z0-9-]{1,40}$/, p.id);
|
||||
assert(!ids.has(p.id), p.id + " twice");
|
||||
ids.add(p.id);
|
||||
assert(p.name?.trim(), p.id + " has no name");
|
||||
assert(p.chat || p.responses || p.anthropic || p.regions?.length, p.id + " has no endpoint");
|
||||
const urls = [p.chat, p.responses, p.anthropic, p.website, p.keysUrl, p.iconUrl, ...(p.regions || []).flatMap((r) => [r.chat, r.responses, r.anthropic, r.keysUrl, r.website])];
|
||||
for (const u of urls.filter(Boolean)) assert.equal(new URL(u).protocol, "https:", p.id + ": " + u);
|
||||
for (const t of [p.from, p.until].filter(Boolean)) assert(!isNaN(Date.parse(t)), p.id + ": " + t);
|
||||
}
|
||||
});
|
||||
Reference in new issue
Block a user