Files
yetone
76f3b97c09
sync: it can sync by itself less often or only when asked, and the server's setup can be restored over this computer's, undoably (Sun1090, #847: 同步太快了，需要设置个开关控制自动同步，还有恢复按钮) — the gateway's magpie synced every 3 minutes with no way to change it, and a free 坚果云 account is allowed only so many WebDAV requests (600 per 30 minutes, 1 GB up and 3 GB down a month); and there was no way to take the server's setup back over a computer's own, which the next sync would otherwise push up. Settings › Sync now has Sync by itself, the app's segmented control (no <select>): Off, 3 min (the default, as before), 15 min, 30 min, 1 hour; Off syncs only on Sync now, and the sync row says only when asked. The loop reads the setting each minute (no request), so a change takes effect without a restart, and a server limiting requests is still waited for as backoff says. Restore from the server asks first in the app's own dialog, naming the parts replaced; it reads the server's file as it is now, seals this computer's setup (keys and all) into the sync folder as <time>-before-restore (newest 5 kept), brings in each part that differs as a sync brings one changed only there, and leaves the state as such a sync would, so the next sync pushes nothing back. Its notice says what was restored, with Undo, which brings the kept setup back (the next sync then takes it to the others); OK, or a later sync's own notice, drops the undo. The form's Save keeps the setting (SetAuto alone sets it). CLI: magpie webdav|s3 auto off|3|15|30|60, restore, undo; the status says how often it syncs by itself. zh, ja, de strings. TestAutoSync (off: no request by itself; back to 15 min: one sync, then none; Save keeps it; 7 refused) and TestRestore (b's own changes replaced by the server's with no push, the kept copy holds b's provider and key, Undo puts it back and a gets it, nothing differing restores nothing) pass -race; sync-auto-restore.test.cjs (WebKit, en and zh: Off posts -1 and the rows say so, Cancel posts nothing, Restore and Undo post, no click moves the page) fails on the old code. sync-other-kind, sync-spin, s3-sync, webdav-settings-refresh, usage-computers, click-scroll, gui-ja, gui-de and settings-sections pass on WebKit. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass.
gateway: a request archive, off unless turned on, keeps each call's headers and bodies in the user's own S3 bucket and a recent call reads its own back (Jorben on Discord: 请求存档) — the Gateway page's recent calls only ever held the last bodies in memory, the first 256 KB of each, with no headers, gone at a restart. A switch over Recent calls ("Request archive", POST /api/settings/archive, settings' requestArchive) turns it on, refused with a reason while sync isn't set up with an s3:// bucket; with it on, each call the gateway serves gets an id (UTC HHMMSS-<16 hex>, said back in X-Magpie-Archive-Id) and, once it is recorded, its method, path and query, request headers and body and response status, headers and body go as one JSON file to <prefix>/magpie/archive/<YYYY-MM-DD>/<id>.json in the bucket sync keeps its backup in, signed with the same client (davsync.Bucket, no new SDK). Before anything leaves, Authorization, x-api-key, Cookie, Set-Cookie, any header or JSON field or query parameter named for a key, token, secret or password, and anything that looks like a key or token anywhere in the text (the redaction rules' patterns, in a stream's data lines too) become [REDACTED] — gone, no placeholder kept; the archive is plain JSON, readable in the bucket, not sealed with the passphrase. Uploads go one at a time on their own goroutine from a queue of 16 calls, each body capped at the 256 KB the call already keeps and marked truncated; a full queue drops the call with a log line, a failed upload is logged and shown under the switch until the next one goes, and no request waits on any of it. A recent call the archive kept shows its date and id under its bodies, and "Fetch from archive" (GET /api/archive?date=&id=, only a date and an id of that shape) shows the request's and response's headers and bodies as the bucket has them, or says it isn't there yet. The hook is in record() and the Call literal, so PR #239's request log lines are not touched; the archived call carries its usage, and with #239 its vendor request id. Only calls through serve() are archived, not Codex's backend, image or video calls. TestArchiveOn, TestArchiveOff, TestArchiveName, TestRequestArchiveToS3 (a real gateway request through the signature-checking fake S3), TestScrub and request-archive.test.cjs fail without the change; TestSettingsSaveKeepsWhatItDoesNotSend now sets requestArchive. A sandbox magpie web with a fake vendor and a fake S3 server uploaded nothing while off; on, it uploaded a signed archive with none of eight planted secrets in it, read it back through /api/archive and in the page, and with the S3 server down served the call in 0.1 s and said why the upload failed. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass (Claude account tests and TestClaudeRunEndsWhenTheClientLeaves pass alone); request-archive, click-scroll, old-webkit, sse-body, gateway-fold pass.
archive: a request the archive kept is opened and downloaded from the Usage page, with its bodies whole (jorben, #447: 请求存档 reachable from usage history, full bodies, download) — the archive's name was only on the Gateway page's recent calls, gone once a call left the list or the gateway restarted, and the archive kept the first 256 KB of each body, the recent list's cut. The usage ledger's row now names the call's archive (archive in usage.jsonl, kept through the Requests page's packed rows), and its details on Usage → Requests show the Request archive panel, Fetch from archive and Download; with the archive on, a request it has no copy of says "Not archived". The bodies are kept whole up to requestArchiveMaxMB in settings.json (32 MB unless set, at most 1024): the response is teed to a temp file as it is sent, the request body written to one when the call ends, so the upload queue holds file handles, not bodies, and one upload at a time reads them; a body past the limit is kept to it and marked cut, with its whole size. What goes upstream is unchanged, the archive stays off unless turned on, headers and bodies are scrubbed as before, and the temp files are removed after the upload or when the queue is full. The page is sent a body to show only when it is 256 KB or less (a longer one is said by its size), an archive over 80 MB is only said to be; Download streams the object whole, to Downloads in the app (archive/export, -2, -3… for a name taken) and as the browser's download in magpie web (archive/file). TestArchiveWholeBodies, TestArchiveLimit, TestRowKeepsArchive and TestArchivePreviewAndDownload fail without the change; TestArchiveOffUsage, TestArchiveTooLargeToPreview, TestArchiveOldPreview keep the rest. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; request-archive-usage (new), request-archive, click-scroll, old-webkit, usage-ledger, usage-ledger-detail, usage-ledger-content, web-page, sse-body pass.
sync: a backup on a slow line is no longer cut off at 2 minutes; a server that stops moving is (#657, irx999: a 43 MB backup with many skills to a WebDAV server on a 2.4 Mbps VPS, 305 KB/s, needs 2.4 minutes and every sync failed) — a sync had a fixed deadline in all (2 minutes from the gateway's loop and magpie webdav/s3 sync, 90 s from Settings › Sync now), so a large backup on a slow line never got through, however steadily it moved. WebDAV and S3 requests now go through a watchdog instead (stall.go): the backup's GET and PUT fail once nothing has moved either way for 60 s, each byte of the body sent or read putting it back, and after a PUT's body is all handed over the answer gets 60 s on top of as long as the write took (at least 4 minutes), for what still sits in the system's buffers to leave; the other requests (PROPFIND, MKCOL, HEAD, MOVE, DELETE) have 30 s in all. A stall says so ("the server sent and took nothing for 1m0s (PUT)"). Sync now and the CLI have no deadline; the gateway's loop keeps an hour as a backstop; a wait for another magpie's sync stays 30 s. TestSlowTransferIsNotCutShort (20 MB up and down at a steady trickle, over 5 times the watchdog each way), TestStalledTransferFailsPromptly (a server stopping mid-PUT and mid-GET fails within a second) and TestMetadataRequestHasALimit fail on the old code (no watchdog: the stalled ones hang) and pass. go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried against a real slow WebDAV server.
sync: it can sync by itself less often or only when asked, and the server's setup can be restored over this computer's, undoably (Sun1090, #847: 同步太快了，需要设置个开关控制自动同步，还有恢复按钮) — the gateway's magpie synced every 3 minutes with no way to change it, and a free 坚果云 account is allowed only so many WebDAV requests (600 per 30 minutes, 1 GB up and 3 GB down a month); and there was no way to take the server's setup back over a computer's own, which the next sync would otherwise push up. Settings › Sync now has Sync by itself, the app's segmented control (no <select>): Off, 3 min (the default, as before), 15 min, 30 min, 1 hour; Off syncs only on Sync now, and the sync row says only when asked. The loop reads the setting each minute (no request), so a change takes effect without a restart, and a server limiting requests is still waited for as backoff says. Restore from the server asks first in the app's own dialog, naming the parts replaced; it reads the server's file as it is now, seals this computer's setup (keys and all) into the sync folder as <time>-before-restore (newest 5 kept), brings in each part that differs as a sync brings one changed only there, and leaves the state as such a sync would, so the next sync pushes nothing back. Its notice says what was restored, with Undo, which brings the kept setup back (the next sync then takes it to the others); OK, or a later sync's own notice, drops the undo. The form's Save keeps the setting (SetAuto alone sets it). CLI: magpie webdav|s3 auto off|3|15|30|60, restore, undo; the status says how often it syncs by itself. zh, ja, de strings. TestAutoSync (off: no request by itself; back to 15 min: one sync, then none; Save keeps it; 7 refused) and TestRestore (b's own changes replaced by the server's with no push, the kept copy holds b's provider and key, Undo puts it back and a gets it, nothing differing restores nothing) pass -race; sync-auto-restore.test.cjs (WebKit, en and zh: Off posts -1 and the rows say so, Cancel posts nothing, Restore and Undo post, no click moves the page) fails on the old code. sync-other-kind, sync-spin, s3-sync, webdav-settings-refresh, usage-computers, click-scroll, gui-ja, gui-de and settings-sections pass on WebKit. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass.
cli: 新增 magpie webdav，在终端里开启、修改和触发 WebDAV 同步 / add magpie webdav to turn on, change and run WebDAV sync from the terminal (#127)
cli: 新增 magpie webdav，在终端里开启、修改和触发 WebDAV 同步 / add magpie webdav to turn on, change and run WebDAV sync from the terminal (#127)
cli: 新增 magpie webdav，在终端里开启、修改和触发 WebDAV 同步 / add magpie webdav to turn on, change and run WebDAV sync from the terminal (#127)
sync: a backup on a slow line is no longer cut off at 2 minutes; a server that stops moving is (#657, irx999: a 43 MB backup with many skills to a WebDAV server on a 2.4 Mbps VPS, 305 KB/s, needs 2.4 minutes and every sync failed) — a sync had a fixed deadline in all (2 minutes from the gateway's loop and magpie webdav/s3 sync, 90 s from Settings › Sync now), so a large backup on a slow line never got through, however steadily it moved. WebDAV and S3 requests now go through a watchdog instead (stall.go): the backup's GET and PUT fail once nothing has moved either way for 60 s, each byte of the body sent or read putting it back, and after a PUT's body is all handed over the answer gets 60 s on top of as long as the write took (at least 4 minutes), for what still sits in the system's buffers to leave; the other requests (PROPFIND, MKCOL, HEAD, MOVE, DELETE) have 30 s in all. A stall says so ("the server sent and took nothing for 1m0s (PUT)"). Sync now and the CLI have no deadline; the gateway's loop keeps an hour as a backstop; a wait for another magpie's sync stays 30 s. TestSlowTransferIsNotCutShort (20 MB up and down at a steady trickle, over 5 times the watchdog each way), TestStalledTransferFailsPromptly (a server stopping mid-PUT and mid-GET fails within a second) and TestMetadataRequestHasALimit fail on the old code (no watchdog: the stalled ones hang) and pass. go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried against a real slow WebDAV server.
usage: quota cards draw what was left over time, against an even burn, and the tray card its cycle's line (#651, songlairui) — each reading magpie makes anyway of a subscription's windows (and of a plan bought with a key) is kept as a point in quota-history.json beside the config: provider, account, window, when read, percent left, the window's start and reset — no token, no prompt. A failed reading, one kept from before (As of), an unlimited window and a model family's aren't kept; Antigravity's pools are kept by pool. Within a cycle points are 5 minutes apart at least and a run of one figure is its two ends; a cycle ends where the window starts again (its reset passed, or moved further than the time between two readings, so a rolling window's isn't one). 45 days and 6000 points a window are kept. The Usage page's card has, under its meters, a line a window (several tiers of one account on one chart), broken at each reset, a dashed even burn from each window's start to its reset and an upright now line, with 2 days / Cycle (the account's longest current window, start to reset) in the app's segmented control, remembered, turning every card without moving the page. The tray's card has a thin line of its first window's current cycle. GET /v1/magpie/quotas/history (?days=, ?provider=, ?user=; local or LAN with the key, as /v1/magpie/quotas) and /api/usage/quotas/history serve it; /v1/magpie/quotas and
magpie quota stay snapshots. Computers sharing usage (WebDAV/S3, Settings › Sync › usage) put their history beside their days, sealed, as <id>.magpie-quotas, and merge the others': the union by account, window and time, two points within 30 s the newer, nothing twice however often they sync; an older magpie never reads the file. TestQuotaHistoryRecords, TestQuotaHistoryCycles, TestMergeQuotaHistory, TestQuotasHistory and TestQuotaHistorySharedBetweenComputers cover recording, cycles, pruning, merge, the API and two computers on one WebDAV folder; quota-curve.test.cjs (WebKit, en/zh, light and dark) the curves, the toggle, no curve without readings, and the tray's line (Chromium doesn't start in this sandbox). go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried with real accounts; pay-as-you-go balances aren't drawn.
sync: it can sync by itself less often or only when asked, and the server's setup can be restored over this computer's, undoably (Sun1090, #847: 同步太快了，需要设置个开关控制自动同步，还有恢复按钮) — the gateway's magpie synced every 3 minutes with no way to change it, and a free 坚果云 account is allowed only so many WebDAV requests (600 per 30 minutes, 1 GB up and 3 GB down a month); and there was no way to take the server's setup back over a computer's own, which the next sync would otherwise push up. Settings › Sync now has Sync by itself, the app's segmented control (no <select>): Off, 3 min (the default, as before), 15 min, 30 min, 1 hour; Off syncs only on Sync now, and the sync row says only when asked. The loop reads the setting each minute (no request), so a change takes effect without a restart, and a server limiting requests is still waited for as backoff says. Restore from the server asks first in the app's own dialog, naming the parts replaced; it reads the server's file as it is now, seals this computer's setup (keys and all) into the sync folder as <time>-before-restore (newest 5 kept), brings in each part that differs as a sync brings one changed only there, and leaves the state as such a sync would, so the next sync pushes nothing back. Its notice says what was restored, with Undo, which brings the kept setup back (the next sync then takes it to the others); OK, or a later sync's own notice, drops the undo. The form's Save keeps the setting (SetAuto alone sets it). CLI: magpie webdav|s3 auto off|3|15|30|60, restore, undo; the status says how often it syncs by itself. zh, ja, de strings. TestAutoSync (off: no request by itself; back to 15 min: one sync, then none; Save keeps it; 7 refused) and TestRestore (b's own changes replaced by the server's with no push, the kept copy holds b's provider and key, Undo puts it back and a gets it, nothing differing restores nothing) pass -race; sync-auto-restore.test.cjs (WebKit, en and zh: Off posts -1 and the rows say so, Cancel posts nothing, Restore and Undo post, no click moves the page) fails on the old code. sync-other-kind, sync-spin, s3-sync, webdav-settings-refresh, usage-computers, click-scroll, gui-ja, gui-de and settings-sections pass on WebKit. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass.
sync: it can sync by itself less often or only when asked, and the server's setup can be restored over this computer's, undoably (Sun1090, #847: 同步太快了，需要设置个开关控制自动同步，还有恢复按钮) — the gateway's magpie synced every 3 minutes with no way to change it, and a free 坚果云 account is allowed only so many WebDAV requests (600 per 30 minutes, 1 GB up and 3 GB down a month); and there was no way to take the server's setup back over a computer's own, which the next sync would otherwise push up. Settings › Sync now has Sync by itself, the app's segmented control (no <select>): Off, 3 min (the default, as before), 15 min, 30 min, 1 hour; Off syncs only on Sync now, and the sync row says only when asked. The loop reads the setting each minute (no request), so a change takes effect without a restart, and a server limiting requests is still waited for as backoff says. Restore from the server asks first in the app's own dialog, naming the parts replaced; it reads the server's file as it is now, seals this computer's setup (keys and all) into the sync folder as <time>-before-restore (newest 5 kept), brings in each part that differs as a sync brings one changed only there, and leaves the state as such a sync would, so the next sync pushes nothing back. Its notice says what was restored, with Undo, which brings the kept setup back (the next sync then takes it to the others); OK, or a later sync's own notice, drops the undo. The form's Save keeps the setting (SetAuto alone sets it). CLI: magpie webdav|s3 auto off|3|15|30|60, restore, undo; the status says how often it syncs by itself. zh, ja, de strings. TestAutoSync (off: no request by itself; back to 15 min: one sync, then none; Save keeps it; 7 refused) and TestRestore (b's own changes replaced by the server's with no push, the kept copy holds b's provider and key, Undo puts it back and a gets it, nothing differing restores nothing) pass -race; sync-auto-restore.test.cjs (WebKit, en and zh: Off posts -1 and the rows say so, Cancel posts nothing, Restore and Undo post, no click moves the page) fails on the old code. sync-other-kind, sync-spin, s3-sync, webdav-settings-refresh, usage-computers, click-scroll, gui-ja, gui-de and settings-sections pass on WebKit. go vet, builds (darwin, GOOS=linux nogui, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass.
sync: a backup on a slow line is no longer cut off at 2 minutes; a server that stops moving is (#657, irx999: a 43 MB backup with many skills to a WebDAV server on a 2.4 Mbps VPS, 305 KB/s, needs 2.4 minutes and every sync failed) — a sync had a fixed deadline in all (2 minutes from the gateway's loop and magpie webdav/s3 sync, 90 s from Settings › Sync now), so a large backup on a slow line never got through, however steadily it moved. WebDAV and S3 requests now go through a watchdog instead (stall.go): the backup's GET and PUT fail once nothing has moved either way for 60 s, each byte of the body sent or read putting it back, and after a PUT's body is all handed over the answer gets 60 s on top of as long as the write took (at least 4 minutes), for what still sits in the system's buffers to leave; the other requests (PROPFIND, MKCOL, HEAD, MOVE, DELETE) have 30 s in all. A stall says so ("the server sent and took nothing for 1m0s (PUT)"). Sync now and the CLI have no deadline; the gateway's loop keeps an hour as a backstop; a wait for another magpie's sync stays 30 s. TestSlowTransferIsNotCutShort (20 MB up and down at a steady trickle, over 5 times the watchdog each way), TestStalledTransferFailsPromptly (a server stopping mid-PUT and mid-GET fails within a second) and TestMetadataRequestHasALimit fail on the old code (no watchdog: the stalled ones hang) and pass. go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried against a real slow WebDAV server.
gateway: a model's web search goes to a search API of the user's own when no provider can search (#419 by HwFee: let me set up Tavily, Brave or SearXNG with my key rather than lose the search tool) — with no provider that searches by itself and no provider magpie could search with, the gateway took the client's web search tool away. Tavily, Brave Search, Exa, Firecrawl and SearXNG (an address of one's own, a key only if it asks for one) can now be set up, in an order: the injected web_search tool is then offered, and a search goes to the searcher's model as before, or, with none or when it fails, to each search API in turn, called directly with no model, its results handed back as the same hits and text the searcher path gives, so clients see the same web_search_tool_result blocks. The order is provider searches itself, searcher provider, search APIs, strip. They are kept in providers.json (0600) like provider keys; a backup carries them, their keys only when it carries keys, and one restored without keys keeps the key here; sync does the same, and a backup or sync from a magpie before them leaves the ones here alone. magpie search lists them, magpie search add <api> [key] [url=…] adds or updates one, magpie search rm <api> removes one; Settings has a Web search section (联网搜索) to add and remove them. Prompts, identity lines and headers to model providers are unchanged. TestWebSearchBySearchAPI, TestNoSearchAPINoTool, TestBackupSearchAPIs, TestSyncSearchAPIs and search-api.test.cjs fail without the change; TestSearchAPIsEachAsked checks each vendor against a fake, TestSearchCLI the command. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; search-api, provider-search, click-scroll, old-webkit pass.
davsync: sync can keep its file in an S3-compatible bucket as well as a WebDAV folder (#296) — AWS S3, Cloudflare R2, Backblaze B2, MinIO, Garage or a NAS. Settings › Sync and backup picks WebDAV or S3: endpoint (empty is AWS), bucket, prefix, region (empty is us-east-1, auto for R2), access key ID and secret, path-style; CLI
magpie s3 on s3://bucket/prefix access-key-id=… endpoint=… region=… path-style. The bucket holds the same passphrase-sealed file at <prefix>/magpie/magpie.magpie-backup, signed with SigV4 (no SDK). It is read with its ETag and written with If-Match (If-None-Match: * for the first), so two computers syncing at once don't overwrite each other; a server that refuses conditional writes (501, or 400 naming the header) is remembered and gets a HEAD-then-PUT instead. An IP, localhost or a dotted bucket on https goes path-style on its own. Errors say what to do: wrong key/secret, clock skew, bucket in another region (names it), no such bucket, 403 on the first read on AWS without s3:ListBucket. The secret is kept as the WebDAV password is, sent only to the endpoint and key it was typed for, and may not be the passphrase. sigv4_test (AWS's published vectors), s3_test (fake server: conditions, conflict, fallback, regions, errors), s3_cli_test and s3-sync.test.cjs (Chromium, WebKit, en/zh) fail without the change; tested against MinIO. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass (TestGroupAddSetRemove, a TempDir-cleanup flake under load, passes alone); click-scroll, old-webkit pass.
davsync: sync can keep its file in an S3-compatible bucket as well as a WebDAV folder (#296) — AWS S3, Cloudflare R2, Backblaze B2, MinIO, Garage or a NAS. Settings › Sync and backup picks WebDAV or S3: endpoint (empty is AWS), bucket, prefix, region (empty is us-east-1, auto for R2), access key ID and secret, path-style; CLI
magpie s3 on s3://bucket/prefix access-key-id=… endpoint=… region=… path-style. The bucket holds the same passphrase-sealed file at <prefix>/magpie/magpie.magpie-backup, signed with SigV4 (no SDK). It is read with its ETag and written with If-Match (If-None-Match: * for the first), so two computers syncing at once don't overwrite each other; a server that refuses conditional writes (501, or 400 naming the header) is remembered and gets a HEAD-then-PUT instead. An IP, localhost or a dotted bucket on https goes path-style on its own. Errors say what to do: wrong key/secret, clock skew, bucket in another region (names it), no such bucket, 403 on the first read on AWS without s3:ListBucket. The secret is kept as the WebDAV password is, sent only to the endpoint and key it was typed for, and may not be the passphrase. sigv4_test (AWS's published vectors), s3_test (fake server: conditions, conflict, fallback, regions, errors), s3_cli_test and s3-sync.test.cjs (Chromium, WebKit, en/zh) fail without the change; tested against MinIO. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass (TestGroupAddSetRemove, a TempDir-cleanup flake under load, passes alone); click-scroll, old-webkit pass.
sync: a backup on a slow line is no longer cut off at 2 minutes; a server that stops moving is (#657, irx999: a 43 MB backup with many skills to a WebDAV server on a 2.4 Mbps VPS, 305 KB/s, needs 2.4 minutes and every sync failed) — a sync had a fixed deadline in all (2 minutes from the gateway's loop and magpie webdav/s3 sync, 90 s from Settings › Sync now), so a large backup on a slow line never got through, however steadily it moved. WebDAV and S3 requests now go through a watchdog instead (stall.go): the backup's GET and PUT fail once nothing has moved either way for 60 s, each byte of the body sent or read putting it back, and after a PUT's body is all handed over the answer gets 60 s on top of as long as the write took (at least 4 minutes), for what still sits in the system's buffers to leave; the other requests (PROPFIND, MKCOL, HEAD, MOVE, DELETE) have 30 s in all. A stall says so ("the server sent and took nothing for 1m0s (PUT)"). Sync now and the CLI have no deadline; the gateway's loop keeps an hour as a backstop; a wait for another magpie's sync stays 30 s. TestSlowTransferIsNotCutShort (20 MB up and down at a steady trickle, over 5 times the watchdog each way), TestStalledTransferFailsPromptly (a server stopping mid-PUT and mid-GET fails within a second) and TestMetadataRequestHasALimit fail on the old code (no watchdog: the stalled ones hang) and pass. go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried against a real slow WebDAV server.
sync: a backup on a slow line is no longer cut off at 2 minutes; a server that stops moving is (#657, irx999: a 43 MB backup with many skills to a WebDAV server on a 2.4 Mbps VPS, 305 KB/s, needs 2.4 minutes and every sync failed) — a sync had a fixed deadline in all (2 minutes from the gateway's loop and magpie webdav/s3 sync, 90 s from Settings › Sync now), so a large backup on a slow line never got through, however steadily it moved. WebDAV and S3 requests now go through a watchdog instead (stall.go): the backup's GET and PUT fail once nothing has moved either way for 60 s, each byte of the body sent or read putting it back, and after a PUT's body is all handed over the answer gets 60 s on top of as long as the write took (at least 4 minutes), for what still sits in the system's buffers to leave; the other requests (PROPFIND, MKCOL, HEAD, MOVE, DELETE) have 30 s in all. A stall says so ("the server sent and took nothing for 1m0s (PUT)"). Sync now and the CLI have no deadline; the gateway's loop keeps an hour as a backstop; a wait for another magpie's sync stays 30 s. TestSlowTransferIsNotCutShort (20 MB up and down at a steady trickle, over 5 times the watchdog each way), TestStalledTransferFailsPromptly (a server stopping mid-PUT and mid-GET fails within a second) and TestMetadataRequestHasALimit fail on the old code (no watchdog: the stalled ones hang) and pass. go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried against a real slow WebDAV server.
provider: a providers.json that can't be read is said where the providers are listed, never backed up or synced as none, and is replaced through a rename (PR #415's review) — #415 made edits refuse a file they couldn't read or decode, but every read for the user still took it as empty: magpie providers said "no providers yet", magpie models "no models yet", the Providers page opened the Add sheet as for a first use, and backup.Collect, so an export and a WebDAV/S3 sync, carried it as no providers, which another computer's Mirror would then take as every provider removed. magpie providers and magpie models now list what is left (the signed-in accounts) and end in the error, the TUI says it over its list, and the Providers page shows a "Your providers file can't be read" box (无法读取供应商配置文件) over the list with magpie's reason and no Add sheet; start-up, the gateway and the catalog still go on with it as empty. Stored and StoredGroups return the read error, so a backup and a sync stop on it with nothing pushed. The error, for reads and refused writes alike, names the file, says magpie left it unchanged and to fix it or move it aside (errors.Is ErrUnreadable). store writes through writePrivate, a 0600 temp file renamed over it (steady.Rename waits out a Windows reader holding it, a symlink keeps its target), instead of os.WriteFile's truncate-then-write, and the read goes through steady.ReadFile. TestFileErrorSaysUnreadable, TestWriteErrorsSayLeftUnchanged, TestStoredRefusesUnreadable, TestStoreReplacesAtomically, TestSyncStopsOnUnreadableProviders, TestProvidersCmdSaysUnreadable and provider-file-error.test.cjs fail without the change. go vet, go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui ./... pass; provider-file-error, click-scroll, old-webkit, providers-off-fold, add-sheet pass.
usage: with WebDAV or S3 sync, each computer shares its usage and sees the others', the total and each computer's part (#542) — sync carried keys, agents and the library, and each computer's Usage page counted only the calls made on it. Sync now has a Usage tick (off unless picked, usage=yes in
magpie webdav/magpie s3): a computer with it on puts its own calls up as one file a day, magpie/usage/<id>-<day>.magpie-usage beside the bundle (under the prefix's usage/ on S3), each a gzipped list of times, agents, providers, models, tokens, costs and statuses, sealed with the sync passphrase (backup.SealData, the key cached per passphrase so a sync derives it once); no request body, session, archive name or key goes, a call another computer passed on over the LAN (Via) isn't put up again, and an error is cut to 300 characters. The id is a random one kept in usage-computer.json, which no backup carries, with the host name beside it. A computer only ever writes and deletes its own names, so none overwrites another's; the files are not in the bundle and its hash, so sharing usage never makes a sync or a conflict of the bundle; the first sync puts up the last 90 days, after that only yesterday and today, at most every 15 minutes (a Save or Sync now at once), and a day past 90 is deleted by the computer it is of. The others' files, listed (PROPFIND Depth 1; ListObjectsV2) and fetched only when changed, are kept in usage-others and read with the ledger, as Record.Computer; a failure is said in the sync row ("Couldn't share usage") and never fails the sync. The Requests page and the panel's Usage tab get a Computer pick (this computer, other computers when more than one, each by name) sending computer=this|others|<id>; a row from another computer says "· on <name>" beside its agent, and its details name the computer and say what was said stays there, nothing asked of this computer's session files for it. With no other computer's usage there is no pick and nothing new on either page, and an older magpie never lists or reads usage/. TestUsageSharedBetweenComputers (computers a and b over a fake WebDAV under httptest: each sees 4 calls, the totals and its own and the other's part, nothing counted twice over repeated syncs, the bundle not put again for usage, a new call reaching the other, a day past the window deleted), TestUsageSharedThroughS3 (the same through a fake S3, signed list and delete, a prefix with a space and a plus) and TestUsageOffAsksNothing are new and don't build without the change; usage-computers.test.cjs (WebKit, en and zh: no pick without other computers, the pick and computer=this, another computer's row and details with no /content asked, the panel's pick, the sync form's tick posting usage: true, no click moving the page) fails without it on all but the no-other-computers case. s3-sync, sync-other-kind, panel-usage, request-archive-usage, click-scroll and usage-ledger-chart pass in WebKit. go vet ./internal/..., go build -tags nogui (darwin, GOOS=linux), GOOS=windows go build, go test -tags nogui -count=1 ./... under a temp HOME pass (TestFXInState failed once on its TempDir cleanup, passes alone). Turning the Usage tick off, or sync off, lets the other computers' usage kept here go (DropAllShared), so the Usage page counts this computer's alone again: TestUsageSharedBetweenComputers checks both, and fails without it.
usage: quota cards draw what was left over time, against an even burn, and the tray card its cycle's line (#651, songlairui) — each reading magpie makes anyway of a subscription's windows (and of a plan bought with a key) is kept as a point in quota-history.json beside the config: provider, account, window, when read, percent left, the window's start and reset — no token, no prompt. A failed reading, one kept from before (As of), an unlimited window and a model family's aren't kept; Antigravity's pools are kept by pool. Within a cycle points are 5 minutes apart at least and a run of one figure is its two ends; a cycle ends where the window starts again (its reset passed, or moved further than the time between two readings, so a rolling window's isn't one). 45 days and 6000 points a window are kept. The Usage page's card has, under its meters, a line a window (several tiers of one account on one chart), broken at each reset, a dashed even burn from each window's start to its reset and an upright now line, with 2 days / Cycle (the account's longest current window, start to reset) in the app's segmented control, remembered, turning every card without moving the page. The tray's card has a thin line of its first window's current cycle. GET /v1/magpie/quotas/history (?days=, ?provider=, ?user=; local or LAN with the key, as /v1/magpie/quotas) and /api/usage/quotas/history serve it; /v1/magpie/quotas and
magpie quota stay snapshots. Computers sharing usage (WebDAV/S3, Settings › Sync › usage) put their history beside their days, sealed, as <id>.magpie-quotas, and merge the others': the union by account, window and time, two points within 30 s the newer, nothing twice however often they sync; an older magpie never reads the file. TestQuotaHistoryRecords, TestQuotaHistoryCycles, TestMergeQuotaHistory, TestQuotasHistory and TestQuotaHistorySharedBetweenComputers cover recording, cycles, pruning, merge, the API and two computers on one WebDAV folder; quota-curve.test.cjs (WebKit, en/zh, light and dark) the curves, the toggle, no curve without readings, and the tray's line (Chromium doesn't start in this sandbox). go vet, builds (darwin, GOOS=linux, GOOS=windows) and go test -tags nogui -count=1 ./... under a temp HOME pass. Not tried with real accounts; pay-as-you-go balances aren't drawn.