Commit Graph
22 Commits
Author SHA1 Message Date
yetone da1ee0f81a sync: the menu bar's usage is each computer's own, not synced (yoooo on Discord: usage in the menu bar turned off on a Mac came back on from a Windows box that shows it, over WebDAV) — a sync and a restored backup kept only the window's size, the proxy and the Dock from this computer, so which subscriptions show beside the icon, how often they refresh and their logos came from whichever computer synced last. They are now kept like the others (Settings.KeepOwn, used by both), and left out of what tells a change. TestSync fails without the change. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass. 2026-10-02 01:13:29 +08:00
yetone 3c04877cfc archive: a request the archive kept is opened and downloaded from the Usage page, with its bodies whole (jorben, #447: 请求存档 reachable from usage history, full bodies, download) — the archive's name was only on the Gateway page's recent calls, gone once a call left the list or the gateway restarted, and the archive kept the first 256 KB of each body, the recent list's cut. The usage ledger's row now names the call's archive (archive in usage.jsonl, kept through the Requests page's packed rows), and its details on Usage → Requests show the Request archive panel, Fetch from archive and Download; with the archive on, a request it has no copy of says "Not archived". The bodies are kept whole up to requestArchiveMaxMB in settings.json (32 MB unless set, at most 1024): the response is teed to a temp file as it is sent, the request body written to one when the call ends, so the upload queue holds file handles, not bodies, and one upload at a time reads them; a body past the limit is kept to it and marked cut, with its whole size. What goes upstream is unchanged, the archive stays off unless turned on, headers and bodies are scrubbed as before, and the temp files are removed after the upload or when the queue is full. The page is sent a body to show only when it is 256 KB or less (a longer one is said by its size), an archive over 80 MB is only said to be; Download streams the object whole, to Downloads in the app (archive/export, -2, -3… for a name taken) and as the browser's download in magpie web (archive/file). TestArchiveWholeBodies, TestArchiveLimit, TestRowKeepsArchive and TestArchivePreviewAndDownload fail without the change; TestArchiveOffUsage, TestArchiveTooLargeToPreview, TestArchiveOldPreview keep the rest. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; request-archive-usage (new), request-archive, click-scroll, old-webkit, usage-ledger, usage-ledger-detail, usage-ledger-content, web-page, sse-body pass. 2026-10-02 00:57:51 +08:00
yetone 0361189f9e test: every package's tests pass on Windows, and none writes outside its sandbox (the user: 把 Windows 上那几个失败的测试修掉) — on a Windows box the suite failed in most packages, and the root package hung for 25 minutes. Tests set HOME, which os.UserHomeDir ignores on Windows (it reads USERPROFILE), so they read and wrote the runner's own home; each now sets USERPROFILE beside HOME, and Devin's, Goose's, Crush's and Claude Desktop's tests set APPDATA/LOCALAPPDATA, where those keep their files there. said() read the command's output only once it returned, and a pipe holds a few KB on Windows, so magpie model --help blocked forever; it reads while the command writes. Unix permission bits (the gateway key store's 0600 too), exec bits and symlinks Windows lacks are checked only off Windows; TOML strings hold Windows paths as literal strings, JSON as marshalled ones; a URL's path is compared as one. The provider and usage packages run in a home of their own (TestMain, as the gateway's already does), since a CLI's identity kept by a refresh still going when its test ended landed in the runner's ~/.config/magpie — and on a Mac in the user's own. Two product fixes came out of it: imagemcp took a file:///C:/… root or image for the path /C:/…, which doesn't exist on Windows, and now makes a Windows path of it (a host for a UNC share); and session discovery could keep a new Cowork session hidden on Linux, where a folder's mtime is a kernel tick and its size doesn't change, so an entry added within the tick of the last listing left both as they were — a listing made within 2s of its folder's last change is read again, as git does for racy files. TestFilePath and TestRacyDirectoryListedAgain fail without the change. On Windows 11 every package passes (32 of 32) and a run of each package in a fresh home leaves nothing in home, APPDATA or LOCALAPPDATA; on Linux (WSL) and macOS go test -tags nogui ./... passes. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build pass. 2026-10-01 23:15:07 +08:00
yetone 9c54a560cc provider: a providers.json that can't be read is said where the providers are listed, never backed up or synced as none, and is replaced through a rename (PR #415's review) — #415 made edits refuse a file they couldn't read or decode, but every read for the user still took it as empty: magpie providers said "no providers yet", magpie models "no models yet", the Providers page opened the Add sheet as for a first use, and backup.Collect, so an export and a WebDAV/S3 sync, carried it as no providers, which another computer's Mirror would then take as every provider removed. magpie providers and magpie models now list what is left (the signed-in accounts) and end in the error, the TUI says it over its list, and the Providers page shows a "Your providers file can't be read" box (无法读取供应商配置文件) over the list with magpie's reason and no Add sheet; start-up, the gateway and the catalog still go on with it as empty. Stored and StoredGroups return the read error, so a backup and a sync stop on it with nothing pushed. The error, for reads and refused writes alike, names the file, says magpie left it unchanged and to fix it or move it aside (errors.Is ErrUnreadable). store writes through writePrivate, a 0600 temp file renamed over it (steady.Rename waits out a Windows reader holding it, a symlink keeps its target), instead of os.WriteFile's truncate-then-write, and the read goes through steady.ReadFile. TestFileErrorSaysUnreadable, TestWriteErrorsSayLeftUnchanged, TestStoredRefusesUnreadable, TestStoreReplacesAtomically, TestSyncStopsOnUnreadableProviders, TestProvidersCmdSaysUnreadable and provider-file-error.test.cjs fail without the change. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; provider-file-error, click-scroll, old-webkit, providers-off-fold, add-sheet pass. 2026-10-01 19:45:24 +08:00
yetone f654bec468 gateway: a model's web search goes to a search API of the user's own when no provider can search (#419 by HwFee: let me set up Tavily, Brave or SearXNG with my key rather than lose the search tool) — with no provider that searches by itself and no provider magpie could search with, the gateway took the client's web search tool away. Tavily, Brave Search, Exa, Firecrawl and SearXNG (an address of one's own, a key only if it asks for one) can now be set up, in an order: the injected web_search tool is then offered, and a search goes to the searcher's model as before, or, with none or when it fails, to each search API in turn, called directly with no model, its results handed back as the same hits and text the searcher path gives, so clients see the same web_search_tool_result blocks. The order is provider searches itself, searcher provider, search APIs, strip. They are kept in providers.json (0600) like provider keys; a backup carries them, their keys only when it carries keys, and one restored without keys keeps the key here; sync does the same, and a backup or sync from a magpie before them leaves the ones here alone. magpie search lists them, magpie search add <api> [key] [url=…] adds or updates one, magpie search rm <api> removes one; Settings has a Web search section (联网搜索) to add and remove them. Prompts, identity lines and headers to model providers are unchanged. TestWebSearchBySearchAPI, TestNoSearchAPINoTool, TestBackupSearchAPIs, TestSyncSearchAPIs and search-api.test.cjs fail without the change; TestSearchAPIsEachAsked checks each vendor against a fake, TestSearchCLI the command. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; search-api, provider-search, click-scroll, old-webkit pass. 2026-10-01 19:44:44 +08:00
yetone 2e94643c7b sync: moving between WebDAV and S3 keeps the other's server, and moving back finds it (ARNO on Discord: 我体验了一下s3同步,结果它把我之前配置好的webdav同步设置给删了) — sync.json held one setup, and saving S3 in Settings › Sync and backup (or magpie s3 on) wrote the bucket over it, so the WebDAV address, user and password were gone. sync.json now keeps the other kind's server beside the one synced to (other: its address, user, password or secret, and S3's endpoint, region and path-style), in the same file, readable by the user alone, and never in a backup, as before. Only one is synced to. Moving back with the password left empty uses the one kept, for the same server and user alone; magpie webdav on / magpie s3 on with nothing else given moves back to the kept one, and magpie webdav / s3 say which is kept. In Settings the row says "WebDAV settings kept" (or S3's), the form marks the kind synced to "· on", says whether saving keeps it or moves sync, its button reads "Move sync to WebDAV" when it does, and the other kind's fields show the kept server. A sync.json from before reads as it did; turning sync off still removes it all. TestConfigureKeepsTheOtherKind, TestS3Cmd and sync-other-kind.test.cjs fail without the change. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; sync-other-kind, s3-sync, webdav-settings-refresh, click-scroll, old-webkit pass. 2026-10-01 19:44:29 +08:00
TryWorld a32186a6ba fix(profile): preserve default values in snapshots (#421) 2026-10-01 19:43:32 +08:00
TryWorld e4bf875b85 fix(backup): exclude balance tokens from keyless exports (#418) 2026-10-01 19:43:27 +08:00
yetone 258d04f569 gateway: a request archive, off unless turned on, keeps each call's headers and bodies in the user's own S3 bucket and a recent call reads its own back (Jorben on Discord: 请求存档) — the Gateway page's recent calls only ever held the last bodies in memory, the first 256 KB of each, with no headers, gone at a restart. A switch over Recent calls ("Request archive", POST /api/settings/archive, settings' requestArchive) turns it on, refused with a reason while sync isn't set up with an s3:// bucket; with it on, each call the gateway serves gets an id (UTC HHMMSS-<16 hex>, said back in X-Magpie-Archive-Id) and, once it is recorded, its method, path and query, request headers and body and response status, headers and body go as one JSON file to <prefix>/magpie/archive/<YYYY-MM-DD>/<id>.json in the bucket sync keeps its backup in, signed with the same client (davsync.Bucket, no new SDK). Before anything leaves, Authorization, x-api-key, Cookie, Set-Cookie, any header or JSON field or query parameter named for a key, token, secret or password, and anything that looks like a key or token anywhere in the text (the redaction rules' patterns, in a stream's data lines too) become [REDACTED] — gone, no placeholder kept; the archive is plain JSON, readable in the bucket, not sealed with the passphrase. Uploads go one at a time on their own goroutine from a queue of 16 calls, each body capped at the 256 KB the call already keeps and marked truncated; a full queue drops the call with a log line, a failed upload is logged and shown under the switch until the next one goes, and no request waits on any of it. A recent call the archive kept shows its date and id under its bodies, and "Fetch from archive" (GET /api/archive?date=&id=, only a date and an id of that shape) shows the request's and response's headers and bodies as the bucket has them, or says it isn't there yet. The hook is in record() and the Call literal, so PR #239's request log lines are not touched; the archived call carries its usage, and with #239 its vendor request id. Only calls through serve() are archived, not Codex's backend, image or video calls. TestArchiveOn, TestArchiveOff, TestArchiveName, TestRequestArchiveToS3 (a real gateway request through the signature-checking fake S3), TestScrub and request-archive.test.cjs fail without the change; TestSettingsSaveKeepsWhatItDoesNotSend now sets requestArchive. A sandbox magpie web with a fake vendor and a fake S3 server uploaded nothing while off; on, it uploaded a signed archive with none of eight planted secrets in it, read it back through /api/archive and in the page, and with the S3 server down served the call in 0.1 s and said why the upload failed. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass (Claude account tests and TestClaudeRunEndsWhenTheClientLeaves pass alone); request-archive, click-scroll, old-webkit, sse-body, gateway-fold pass. 2026-10-01 17:13:34 +08:00
yetone 8931a4b3ac davsync: sync can keep its file in an S3-compatible bucket as well as a WebDAV folder (#296) — AWS S3, Cloudflare R2, Backblaze B2, MinIO, Garage or a NAS. Settings › Sync and backup picks WebDAV or S3: endpoint (empty is AWS), bucket, prefix, region (empty is us-east-1, auto for R2), access key ID and secret, path-style; CLI magpie s3 on s3://bucket/prefix access-key-id=… endpoint=… region=… path-style. The bucket holds the same passphrase-sealed file at <prefix>/magpie/magpie.magpie-backup, signed with SigV4 (no SDK). It is read with its ETag and written with If-Match (If-None-Match: * for the first), so two computers syncing at once don't overwrite each other; a server that refuses conditional writes (501, or 400 naming the header) is remembered and gets a HEAD-then-PUT instead. An IP, localhost or a dotted bucket on https goes path-style on its own. Errors say what to do: wrong key/secret, clock skew, bucket in another region (names it), no such bucket, 403 on the first read on AWS without s3:ListBucket. The secret is kept as the WebDAV password is, sent only to the endpoint and key it was typed for, and may not be the passphrase. sigv4_test (AWS's published vectors), s3_test (fake server: conditions, conflict, fallback, regions, errors), s3_cli_test and s3-sync.test.cjs (Chromium, WebKit, en/zh) fail without the change; tested against MinIO. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass (TestGroupAddSetRemove, a TempDir-cleanup flake under load, passes alone); click-scroll, old-webkit pass. 2026-09-30 17:07:47 +08:00
yetone 04f374fa8c agent: OmO (oh-my-openagent, omo-ai), a fork of Pi that keeps its own ~/.omo/agent, gets its own row: a magpie model sets settings.json's defaultProvider/defaultModel and magpie's provider in models.json there as Pi's does, $OMO_CODING_AGENT_DIR / $SENPI_CODING_AGENT_DIR are honoured, the Library writes its AGENTS.md, mcp.json and skills, and omo update updates it (#264 rhyme227). Pi's own ~/.pi is left alone. piIn becomes piLike(at, id, name, dir) so both share one wiring. Tested with omo_test.go in agent and library (fail without the change); go vet, go build -tags nogui (darwin, GOOS=linux, GOOS=windows), go test -tags nogui ./... pass. 2026-09-30 11:38:07 +08:00
AronnaxandClaude Opus 5.5 7931723d79 cli: 新增 magpie webdav,在终端里开启、修改和触发 WebDAV 同步 / add magpie webdav to turn on, change and run WebDAV sync from the terminal (#127)
* cli: magpie webdav — WebDAV sync from the terminal, what Settings › WebDAV sync does, for a machine with no window (WSL, a server over SSH)

magpie webdav shows it; on <address> [user=…] [password=…] [keys=no] [agents=no] [library=no] turns it on and syncs at once, asking for the passphrase (and the password, when a user has none); set k=v… changes what is given and keeps the rest; now, dismiss and off are the page's Sync now, OK and Turn off. Through the same davsync.Configure, Now, Dismiss and Off.

A sync, Off and Dismiss hold a lock the system keeps on sync.lock (flock; LockFileEx on Windows): the gateway's sync every 3 minutes and one from a terminal — or from the app's window while magpie serve has the gateway — would each save sync-state.json over the other's. The system lets it go when the magpie holding it ends, however it ends.

Configure keeps the saved password only for the same server (scheme and host) and user: an address moved to another server, or another user, no longer takes it there — in the app too, where a new address with the password left empty was sent the old one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* davsync, cli: from the review of #127 — the setup read again under the lock, Configure locked too, the password never an argument, one left behind asked for, a wait that runs out said in the status

Now reads the setup again once it has the lock: one that waited while another magpie turned sync off went on with the setup it had read, and saved sync-state.json back after the Off. Configure takes mu and the lock too, after a sync in progress.

magpie webdav takes neither the password nor the passphrase as an argument: password= asks for it, as passphrase= does.

Configure with the saved password left behind — another server or user, the password empty — says to type it and saves nothing, where the Settings page dropped it and the sync failed with 401 under a field that still said saved.

A wait for the lock that runs out is davsync.ErrBusy. Dismiss returns it, and the davsync routes put ErrBusy from a sync in the status they answer with, where the page shows a sync's failures; Save's own errors stay in its form, what was typed in it kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* davsync, cli: the lock before mu, a wait bounded under the page's, one called off not called busy, the password rule in one place, a bare address with = in it

The lock comes before mu everywhere, so a wait for another magpie never holds mu, and Configure, Off and Dismiss wait for it 30 seconds at most, less than the Settings page gives a request: before, a request could hang on mu and then on the lock for minutes.

A wait for the lock called off, as when the page is closed or the gateway stops, is its context's error, not ErrBusy: nothing said another magpie was syncing.

What becomes of the saved password is decided in one place, davsync.SavedPassword, which Configure and magpie webdav both go by, where the two had rules of their own that had come apart: an account with a password and no user name, moved to another server, had its password dropped with nothing asked, and the sync failed with 401. It is asked for now, as for an account with a user.

magpie webdav takes a bare argument with :// in it as the address, = in it or not: https://host/dav?token=… was read as a field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 14:54:04 +08:00
yetone 2b2a7bf83f sync: a WebDAV server that answers 403 for what isn't there (a Synology, PROPFIND too) no longer fails the first sync as if the account couldn't write in the folder — magpie's own folder in it is looked for, and when it isn't there a 403 to reading the backup is no backup yet and one to writing it is made as a 404 is 2026-09-28 10:51:05 +08:00
yetone c6dc08e105 sync: an address at OpenList's or Alist's /dav/ says to go into one of its storages, naming them — the list of storages there takes no folder (MKCOL 405, as if it were there) and no file (PUT 404), so a first sync failed with a bare HTTP 404; a folder in the address that isn't there says so. Tried on OpenList 4.2.6 2026-09-28 10:10:03 +08:00
yetone 920bc04d48 sync: a WebDAV 403 says why rather than blaming the password — a folder in the address the server hasn't got (a Synology answers 403 for a shared folder not made), one the account may only read, or one it may not use, told apart by looking at that folder; only a 401 is the user name or password 2026-09-28 09:31:10 +08:00
yetone 6c9a0aa4d7 agents: OpenHanako — magpie is a provider there, chat completions at the gateway's /v1 under the key magpie-hanako with every model it serves, their windows, output, images and reasoning said; picking one makes it the primary agent's chat model, and default puts back the one it had and removes the provider, marked deleted as OpenHanako marks one it removes. While OpenHanako runs it goes through its local API (server-info.json's port and token), which reloads at once; otherwise into provider-catalog.json and agents/<id>/config.yaml for its next start. Asked on Discord; checked with OpenHanako's own provider registry and model manager, which list magpie's models and resolve the chat model to the gateway 2026-09-28 01:12:46 +08:00
yetone 115f0cbd9d sync: 坚果云 (Nutstore) syncs from the first time — it answers a read in a folder not made yet with a 409 where other WebDAV servers say 404, which magpie took for a failure (#114); a 409 is now no backup yet, and the first sync makes the folder and puts it 2026-09-27 21:01:18 +08:00
yetone e173710982 backup: the library goes with a backup and a WebDAV sync — the instructions, MCP servers and skills with their files, a fifth part, ticked by default (magpie backup/restore --no-library). A backup without keys leaves the servers' secret env and headers empty and a restore fills them from the servers kept here; a skill's file over 2 MB, or past 32 MB in all, stays behind; a path that climbs out of its skill is refused. What it replaces is kept with the library's backups. Tested from a sandbox: exported without keys, changed and removed things, imported — the server back with the local key kept, the skill back as files, run.sh still executable 2026-09-27 17:24:53 +08:00
yetone dad2787f55 settings: the Dock can follow the window — Hide, With window, Show. With window, magpie is in the Dock (and Cmd-Tab) while its window is open and leaves it when the window is closed, so it can be switched to without an icon kept there the rest of the time. Tested on a sandboxed bundle: shown at start, gone on close, back on reopen; Show and Hide behave as before. Issue #101 2026-09-27 16:14:54 +08:00
yetone 99fc679e38 sync: a passphrase that is the WebDAV password is refused — the server is sent the password, so with it the server could open the file it is only meant to hold sealed; pick a passphrase of its own 2026-09-26 15:54:16 +08:00
yetone 0423553454 gui: magpie can stay in the Mac's Dock — Settings → Dock → Show keeps it there as well as in the menu bar, for a menu bar too full to show its icon; it takes effect at once and at every start, the Dock icon opens the window (only the window: Wails would have shown the panel too), and it stays each computer's own in sync and backups 2026-09-25 21:13:30 +08:00
yetone 4c67d069d0 sync: magpie's setup kept the same on every computer through WebDAV (坚果云, Nextcloud, a NAS…) — Settings has Sync and backup: the providers with their keys, groups and icons, the settings, the profiles and each agent's models go to the server as one file sealed with a passphrase, so the server never sees them; the magpie serving the gateway looks every three minutes, sends what changed here and brings in what changed elsewhere, a removal too; a part changed on both sides goes to the newer one, and the copy it replaced is kept in the sync folder with a note saying what was replaced; the proxy and the window size stay each computer's own, and a computer set not to send keys never strips them from the server; Export and Import carry the same file by hand (#28) 2026-09-25 19:52:37 +08:00