Files
magpie/internal/backup
yetone 4afb698638 library: checking skills for updates asks GitHub with a GitHub token, set in Settings → Network and sharing or taken from GITHUB_TOKEN / GH_TOKEN, and a check GitHub limited says until when and that a token raises it (Player on Discord: 资源库检查技能更新时提示 github is limiting request,希望能设置 GitHub token) — the check asks GitHub's API for the last commit to each skill's folder, which GitHub answers 60 times an hour for an address without a token, so a library of a few repositories, checked twice, or a network shared with others, used it up, and the check said only "GitHub is limiting requests from here until 15:04", with nothing to do about it. Now the API's requests carry Authorization: Bearer <token> (5,000 an hour) when there is one: the GitHub token set in Settings → Network and sharing (a new GitHub row: typed masked, saved on its own with POST /api/settings/github-token, then shown as ghp_…0123 with Remove; one with a space is refused in the row, what was typed kept), else GITHUB_TOKEN, else GH_TOKEN, the row saying when one of those is used. gh auth token is not run: it would borrow another app's credential without asking. The token goes to GitHub's API's host alone (withGitHubToken checks the request's host against githubAPI's), never to codeload, raw.githubusercontent or anything else, and Go's client drops it on a redirect to another host; rtk's latest-release lookup on api.github.com carries it too. It is kept in settings.json as the LAN key is: the page is told only the masked one and where it is from (githubTokenMask, githubTokenFrom), a save of the Settings page keeps it, a backup or sync without keys leaves it out, and restoring one without keys keeps this machine's. Each folder's answer is kept with its ETag and asked for again with If-None-Match, so an unchanged one is a 304, which GitHub doesn't count against the limit. A rate limit (403 with the limit spent, a secondary limit's Retry-After, or 429) now says it plainly: without a token, "GitHub's rate limit for requests without a token (60 an hour) is used up until 15:04. Add a GitHub token in Settings → Network and sharing to raise it to 5,000 an hour"; with one, that the token's limit is used up until then; and the check carries limited {until, token} for the page, which says the same in its own words, in English and Chinese, in the status and on each Not checked tag. A token GitHub refuses (401) says which: the one in Settings, or GITHUB_TOKEN / GH_TOKEN. The library's test sandbox clears GITHUB_TOKEN and GH_TOKEN, so no developer's token reaches a fake GitHub. TestGitHubTokenSent (no token: none sent; GH_TOKEN; GITHUB_TOKEN before it; Settings before both; codeload never gets it), TestGitHubTokenOnlyToTheAPI (api.github.com only, not codeload, raw, a look-alike host or the gateway; dropped on a redirect), TestGitHubRateLimitSaid (until when and the token advice without one, not with one, the token never in the error, a secondary limit, a refused token from Settings and from GH_TOKEN), TestGitHubConditionalRequests (If-None-Match, and the 304 answered as the 200 was), TestGitHubTokenBackup (left out without keys, fails without the change; kept on a keyless restore), TestGitHubTokenSetting (set, masked, kept by a Settings save, refused with a space, removed with the environment's then said) and github-token.test.cjs (WebKit, en and zh: the row, a refusal keeping what was typed, saved masked with the token nowhere on the page, removed with GH_TOKEN said, nothing scrolls; a limited check's status and tag say until when, and the token advice only without one) are new; TestCheckSkills, TestSettingsSaveKeepsWhatItDoesNotSend (now with the token) pass. GitHub was never called: httptest stands in for it. settings-sections, search-api, click-scroll, library-toggle-status, library-all, otel, lan-docker, every-skill pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed on its TempDir cleanup, the known flake).
2026-10-02 21:16:50 +08:00
..
library: checking skills for updates asks GitHub with a GitHub token, set in Settings → Network and sharing or taken from GITHUB_TOKEN / GH_TOKEN, and a check GitHub limited says until when and that a token raises it (Player on Discord: 资源库检查技能更新时提示 github is limiting request,希望能设置 GitHub token) — the check asks GitHub's API for the last commit to each skill's folder, which GitHub answers 60 times an hour for an address without a token, so a library of a few repositories, checked twice, or a network shared with others, used it up, and the check said only "GitHub is limiting requests from here until 15:04", with nothing to do about it. Now the API's requests carry Authorization: Bearer <token> (5,000 an hour) when there is one: the GitHub token set in Settings → Network and sharing (a new GitHub row: typed masked, saved on its own with POST /api/settings/github-token, then shown as ghp_…0123 with Remove; one with a space is refused in the row, what was typed kept), else GITHUB_TOKEN, else GH_TOKEN, the row saying when one of those is used. gh auth token is not run: it would borrow another app's credential without asking. The token goes to GitHub's API's host alone (withGitHubToken checks the request's host against githubAPI's), never to codeload, raw.githubusercontent or anything else, and Go's client drops it on a redirect to another host; rtk's latest-release lookup on api.github.com carries it too. It is kept in settings.json as the LAN key is: the page is told only the masked one and where it is from (githubTokenMask, githubTokenFrom), a save of the Settings page keeps it, a backup or sync without keys leaves it out, and restoring one without keys keeps this machine's. Each folder's answer is kept with its ETag and asked for again with If-None-Match, so an unchanged one is a 304, which GitHub doesn't count against the limit. A rate limit (403 with the limit spent, a secondary limit's Retry-After, or 429) now says it plainly: without a token, "GitHub's rate limit for requests without a token (60 an hour) is used up until 15:04. Add a GitHub token in Settings → Network and sharing to raise it to 5,000 an hour"; with one, that the token's limit is used up until then; and the check carries limited {until, token} for the page, which says the same in its own words, in English and Chinese, in the status and on each Not checked tag. A token GitHub refuses (401) says which: the one in Settings, or GITHUB_TOKEN / GH_TOKEN. The library's test sandbox clears GITHUB_TOKEN and GH_TOKEN, so no developer's token reaches a fake GitHub. TestGitHubTokenSent (no token: none sent; GH_TOKEN; GITHUB_TOKEN before it; Settings before both; codeload never gets it), TestGitHubTokenOnlyToTheAPI (api.github.com only, not codeload, raw, a look-alike host or the gateway; dropped on a redirect), TestGitHubRateLimitSaid (until when and the token advice without one, not with one, the token never in the error, a secondary limit, a refused token from Settings and from GH_TOKEN), TestGitHubConditionalRequests (If-None-Match, and the 304 answered as the 200 was), TestGitHubTokenBackup (left out without keys, fails without the change; kept on a keyless restore), TestGitHubTokenSetting (set, masked, kept by a Settings save, refused with a space, removed with the environment's then said) and github-token.test.cjs (WebKit, en and zh: the row, a refusal keeping what was typed, saved masked with the token nowhere on the page, removed with GH_TOKEN said, nothing scrolls; a limited check's status and tag say until when, and the token advice only without one) are new; TestCheckSkills, TestSettingsSaveKeepsWhatItDoesNotSend (now with the token) pass. GitHub was never called: httptest stands in for it. settings-sections, search-api, click-scroll, library-toggle-status, library-all, otel, lan-docker, every-skill pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed on its TempDir cleanup, the known flake).
2026-10-02 21:16:50 +08:00
library: checking skills for updates asks GitHub with a GitHub token, set in Settings → Network and sharing or taken from GITHUB_TOKEN / GH_TOKEN, and a check GitHub limited says until when and that a token raises it (Player on Discord: 资源库检查技能更新时提示 github is limiting request,希望能设置 GitHub token) — the check asks GitHub's API for the last commit to each skill's folder, which GitHub answers 60 times an hour for an address without a token, so a library of a few repositories, checked twice, or a network shared with others, used it up, and the check said only "GitHub is limiting requests from here until 15:04", with nothing to do about it. Now the API's requests carry Authorization: Bearer <token> (5,000 an hour) when there is one: the GitHub token set in Settings → Network and sharing (a new GitHub row: typed masked, saved on its own with POST /api/settings/github-token, then shown as ghp_…0123 with Remove; one with a space is refused in the row, what was typed kept), else GITHUB_TOKEN, else GH_TOKEN, the row saying when one of those is used. gh auth token is not run: it would borrow another app's credential without asking. The token goes to GitHub's API's host alone (withGitHubToken checks the request's host against githubAPI's), never to codeload, raw.githubusercontent or anything else, and Go's client drops it on a redirect to another host; rtk's latest-release lookup on api.github.com carries it too. It is kept in settings.json as the LAN key is: the page is told only the masked one and where it is from (githubTokenMask, githubTokenFrom), a save of the Settings page keeps it, a backup or sync without keys leaves it out, and restoring one without keys keeps this machine's. Each folder's answer is kept with its ETag and asked for again with If-None-Match, so an unchanged one is a 304, which GitHub doesn't count against the limit. A rate limit (403 with the limit spent, a secondary limit's Retry-After, or 429) now says it plainly: without a token, "GitHub's rate limit for requests without a token (60 an hour) is used up until 15:04. Add a GitHub token in Settings → Network and sharing to raise it to 5,000 an hour"; with one, that the token's limit is used up until then; and the check carries limited {until, token} for the page, which says the same in its own words, in English and Chinese, in the status and on each Not checked tag. A token GitHub refuses (401) says which: the one in Settings, or GITHUB_TOKEN / GH_TOKEN. The library's test sandbox clears GITHUB_TOKEN and GH_TOKEN, so no developer's token reaches a fake GitHub. TestGitHubTokenSent (no token: none sent; GH_TOKEN; GITHUB_TOKEN before it; Settings before both; codeload never gets it), TestGitHubTokenOnlyToTheAPI (api.github.com only, not codeload, raw, a look-alike host or the gateway; dropped on a redirect), TestGitHubRateLimitSaid (until when and the token advice without one, not with one, the token never in the error, a secondary limit, a refused token from Settings and from GH_TOKEN), TestGitHubConditionalRequests (If-None-Match, and the 304 answered as the 200 was), TestGitHubTokenBackup (left out without keys, fails without the change; kept on a keyless restore), TestGitHubTokenSetting (set, masked, kept by a Settings save, refused with a space, removed with the environment's then said) and github-token.test.cjs (WebKit, en and zh: the row, a refusal keeping what was typed, saved masked with the token nowhere on the page, removed with GH_TOKEN said, nothing scrolls; a limited check's status and tag say until when, and the token advice only without one) are new; TestCheckSkills, TestSettingsSaveKeepsWhatItDoesNotSend (now with the token) pass. GitHub was never called: httptest stands in for it. settings-sections, search-api, click-scroll, library-toggle-status, library-all, otel, lan-docker, every-skill pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed on its TempDir cleanup, the known flake).
2026-10-02 21:16:50 +08:00
test: every package's tests pass on Windows, and none writes outside its sandbox (the user: 把 Windows 上那几个失败的测试修掉) — on a Windows box the suite failed in most packages, and the root package hung for 25 minutes. Tests set HOME, which os.UserHomeDir ignores on Windows (it reads USERPROFILE), so they read and wrote the runner's own home; each now sets USERPROFILE beside HOME, and Devin's, Goose's, Crush's and Claude Desktop's tests set APPDATA/LOCALAPPDATA, where those keep their files there. said() read the command's output only once it returned, and a pipe holds a few KB on Windows, so magpie model --help blocked forever; it reads while the command writes. Unix permission bits (the gateway key store's 0600 too), exec bits and symlinks Windows lacks are checked only off Windows; TOML strings hold Windows paths as literal strings, JSON as marshalled ones; a URL's path is compared as one. The provider and usage packages run in a home of their own (TestMain, as the gateway's already does), since a CLI's identity kept by a refresh still going when its test ended landed in the runner's ~/.config/magpie — and on a Mac in the user's own. Two product fixes came out of it: imagemcp took a file:///C:/… root or image for the path /C:/…, which doesn't exist on Windows, and now makes a Windows path of it (a host for a UNC share); and session discovery could keep a new Cowork session hidden on Linux, where a folder's mtime is a kernel tick and its size doesn't change, so an entry added within the tick of the last listing left both as they were — a listing made within 2s of its folder's last change is read again, as git does for racy files. TestFilePath and TestRacyDirectoryListedAgain fail without the change. On Windows 11 every package passes (32 of 32) and a run of each package in a fresh home leaves nothing in home, APPDATA or LOCALAPPDATA; on Linux (WSL) and macOS go test -tags nogui ./... passes. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build pass.
2026-10-01 23:15:07 +08:00
usage: with WebDAV or S3 sync, each computer shares its usage and sees the others', the total and each computer's part (#542) — sync carried keys, agents and the library, and each computer's Usage page counted only the calls made on it. Sync now has a Usage tick (off unless picked, usage=yes in magpie webdav/magpie s3): a computer with it on puts its own calls up as one file a day, magpie/usage/<id>-<day>.magpie-usage beside the bundle (under the prefix's usage/ on S3), each a gzipped list of times, agents, providers, models, tokens, costs and statuses, sealed with the sync passphrase (backup.SealData, the key cached per passphrase so a sync derives it once); no request body, session, archive name or key goes, a call another computer passed on over the LAN (Via) isn't put up again, and an error is cut to 300 characters. The id is a random one kept in usage-computer.json, which no backup carries, with the host name beside it. A computer only ever writes and deletes its own names, so none overwrites another's; the files are not in the bundle and its hash, so sharing usage never makes a sync or a conflict of the bundle; the first sync puts up the last 90 days, after that only yesterday and today, at most every 15 minutes (a Save or Sync now at once), and a day past 90 is deleted by the computer it is of. The others' files, listed (PROPFIND Depth 1; ListObjectsV2) and fetched only when changed, are kept in usage-others and read with the ledger, as Record.Computer; a failure is said in the sync row ("Couldn't share usage") and never fails the sync. The Requests page and the panel's Usage tab get a Computer pick (this computer, other computers when more than one, each by name) sending computer=this|others|<id>; a row from another computer says "· on <name>" beside its agent, and its details name the computer and say what was said stays there, nothing asked of this computer's session files for it. With no other computer's usage there is no pick and nothing new on either page, and an older magpie never lists or reads usage/. TestUsageSharedBetweenComputers (computers a and b over a fake WebDAV under httptest: each sees 4 calls, the totals and its own and the other's part, nothing counted twice over repeated syncs, the bundle not put again for usage, a new call reaching the other, a day past the window deleted), TestUsageSharedThroughS3 (the same through a fake S3, signed list and delete, a prefix with a space and a plus) and TestUsageOffAsksNothing are new and don't build without the change; usage-computers.test.cjs (WebKit, en and zh: no pick without other computers, the pick and computer=this, another computer's row and details with no /content asked, the panel's pick, the sync form's tick posting usage: true, no click moving the page) fails without it on all but the no-other-computers case. s3-sync, sync-other-kind, panel-usage, request-archive-usage, click-scroll and usage-ledger-chart pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed once on its TempDir cleanup, passes alone). Turning the Usage tick off, or sync off, lets the other computers' usage kept here go (DropAllShared), so the Usage page counts this computer's alone again: TestUsageSharedBetweenComputers checks both, and fails without it.
2026-10-02 17:47:36 +08:00
gateway: a model's web search goes to a search API of the user's own when no provider can search (#419 by HwFee: let me set up Tavily, Brave or SearXNG with my key rather than lose the search tool) — with no provider that searches by itself and no provider magpie could search with, the gateway took the client's web search tool away. Tavily, Brave Search, Exa, Firecrawl and SearXNG (an address of one's own, a key only if it asks for one) can now be set up, in an order: the injected web_search tool is then offered, and a search goes to the searcher's model as before, or, with none or when it fails, to each search API in turn, called directly with no model, its results handed back as the same hits and text the searcher path gives, so clients see the same web_search_tool_result blocks. The order is provider searches itself, searcher provider, search APIs, strip. They are kept in providers.json (0600) like provider keys; a backup carries them, their keys only when it carries keys, and one restored without keys keeps the key here; sync does the same, and a backup or sync from a magpie before them leaves the ones here alone. magpie search lists them, magpie search add <api> [key] [url=…] adds or updates one, magpie search rm <api> removes one; Settings has a Web search section (联网搜索) to add and remove them. Prompts, identity lines and headers to model providers are unchanged. TestWebSearchBySearchAPI, TestNoSearchAPINoTool, TestBackupSearchAPIs, TestSyncSearchAPIs and search-api.test.cjs fail without the change; TestSearchAPIsEachAsked checks each vendor against a fake, TestSearchCLI the command. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; search-api, provider-search, click-scroll, old-webkit pass.
2026-10-01 19:44:44 +08:00