The copy cache trusted a change time it had seen as proof the file had not
changed. NTFS stamps ChangeTime from Windows' clock tick, not the precise
clock: on the Windows box, 300 ms of back-to-back same-size writes got a
new ChangeTime only every ~0.5 ms (2147 of 2691 writes kept the previous
stamp; diffs 0.5/1.0/1.5 ms). A rewrite in the tick of the comparison kept
the stamp the cache saw, so the cached relation was reused. That, not write
merging, is why TestCodexArchivedSameSizeRewriteInvalidatesDiscovery failed
19 of 300 runs on Windows, alone as well as in the full package.
A comparison is now cached only when both files' change times are more than
a second older than the moment it started (codexStampSettle); a file changed
more recently is compared again next time. A later rewrite then lands in a
later tick and changes the stamp. Change times are compared as times
(Windows FILETIME converted).
TestCodexSameTickRewriteIsNotCached gives every write one change time and a
clock 10 ms into that tick. With the settle rule taken out (cacheable = both
stamps present), it fails: "a rewrite in the tick of the comparison reused
its stale relation". With it, it passes, and the second half checks settled
copies are still compared once, not every time.
TestCodexArchivedDiscoveryCacheAndAppend moves its clock a minute on so the
just-written copies count as settled.
The Windows CI step now runs the Codex copy tests (CodexArchived,
CodexSameTick, CodexPlainTwin), which it had never run.
Run: Windows 11 box (cross-built sessions.test.exe, sandbox USERPROFILE/
APPDATA/TEMP): old binary 19/300 failures of the reported test; new binary
0/1000 for the three Codex copy tests, and the Windows CI regex x5 with
testdata passes. macOS: go test -race ./internal/sessions, -race -count=30 of
the Codex copy tests, go vet (darwin, GOOS=linux, GOOS=windows), gofmt -l.
Not run: Linux with a coarse ctime clock (kernels before 6.13); the rule
covers it the same way.
seed.mjs write fills the fresh home before magpie starts:
- a Sub2API relay (three keys, rotate, 5h/1d/7d windows, a month of balance history), Acme Cloud and a routing group with a rule;
- 30 days of requests from Claude Code, Codex, OpenCode and Gemini CLI over models priced on models.dev, at different speeds, with TTFT, cache reads, reasoning, sessions, accounts and a few 429/502s;
- 308 Claude Code and 94 Codex session files, more than a page, many never sent through magpie;
- two ChatGPT accounts signed in to Codex (Pro with credits, Plus near its limit) with five weeks of quota history;
- library skills from GitHub (one author with two repositories) and a folder, and some only in the agents' folders.
seed.mjs serve stays up as the mock: the fake providers in the Chat, Responses and Anthropic APIs, the relay's /v1/usage, and, as magpie's HTTPS_PROXY, chatgpt.com's usage and Codex models under a certificate from a CA that run.sh makes. Every other host is tunnelled as it is, so DeepSeek still answers for real. The warm-up also goes through the relay, Acme and the group, and record.mjs's guide says what the sandbox now has.
Checked by running run.sh in a Linux container:
- the Codex cards are read live, not stale, and their history draws;
- the relay's windows and balance trend show;
- every model is priced;
- sessions list 200 of 402;
- the warm-ups all pass.
Then record.mjs ran for real (DeepSeek planning, Playwright) on 6e53ee1f's diff (#1031). It planned and shot the skills page's author groups, with GitHub skills above and local below.
On 10-05 and 10-06 nearly every Test run on main was cancelled by the
next push, so a release had no finished run to wait for and Linux-only
failures shipped (LESSONS.md: "Don't tag until a CI Test run ... has
finished green"). cancel-in-progress is now false on main only: a running
run completes, and GitHub still replaces a queued one with the newest.
PRs keep cancelling their older runs. The workflow had no permissions
block, so it got the repository default token; it needs only to read.
The Race step took 690s on ubuntu and 908s on macOS, almost all of it
internal/gateway: its 1046 tests can't call t.Parallel (t.Setenv rules
it out), so the package ran on one core under the race detector.
build/race-shards.sh compiles nothing itself: it takes the package's race
test binary (go test -race -c), splits its tests round-robin by name over
N processes run at once, prints each process's log in a folded group and
fails when any does. The gateway's TestMain already isolates every
process in its own temporary home, keychain stand-ins and PATH, so the
processes share nothing. The step runs it with one process per core
(getconf _NPROCESSORS_ONLN: 4 on ubuntu, 3 on macOS), the same -race
binary and the same 20-minute timeout; redact keeps its own go test -race.
Every test still runs under the race detector, once.
Locally (Apple silicon, 4 processes) the gateway's race suite passed in
4:39 against 567s of test time run one after another; one test,
TestLargeRequestEndpoints (seven 64 MB bodies, 198s), now bounds it.
The image is distroless, which has no shell at all, so a NAS panel's
terminal (Synology, 1Panel, Portainer open /bin/bash or /bin/sh) and
`docker exec -it magpie bash` failed, and a sign-in that needs a terminal
(magpie accounts add devin) couldn't be made there. The image now copies
in Debian 12's bash with the libtinfo it links (distroless cc is Debian 12,
so its glibc runs it) and busybox's commands, and puts magpie on PATH
(/usr/local/bin/magpie). The image still runs as nonroot and starts
/magpie serve as before; it grows from about 37 MB of extras to 66.6 MB.
Built locally (docker 29.4, arm64): bash 5.2.15 and sh run as uid 65532,
`magpie version` and `magpie healthcheck` work from bash, the gateway
answers 200 on /v1/models, /config keeps its folders. The Docker workflow's
smoke test now runs bash and sh in the container. README says so.
* plugin: a slow reader pauses its stream instead of racing a watchdog
#664 gave a plugin reply a soft watermark (softQueuedBytes 4 MiB, softQueuedEvents 4096) and a 10-second idle watchdog: past the watermark, a reader that made no progress for stallIdle had its call failed with errStalledReader, and hardQueuedBytes/hardQueuedEvents (64 MiB / 65536) cut the reply outright. Both are guesses at how long a reader may take, and both end a reply that reader was still consuming.
The host now hands each fetch a credit window instead. Go grants streamWindow (512 KiB) up front and returns credit as the reader takes bytes, so the child stops producing at the window instead of being timed out; the window is charged per delivered byte plus frameOverhead (2048), so a run of tiny frames cannot fill it with metadata. streamBudget (64 MiB) and maxFrame (64 KiB) bound a whole reply and a single frame, and a child that sends past its window is a protocol violation (errProtocolOverrun) rather than a slow reader.
The plugin side grew the matching gate (makeGate, gate.take/spend/add, waitDrain): a fetch holds at the window until Go credits it, and its pending writes are cancellable. Cancellation and shutdown now have one owner per call - a five-state lifecycle (sending, active, draining, aborting, released) whose single transition helper disposes the queue, detaches the watcher, commits the terminal answer, creates the abort and returns capacity exactly once - so a host failure, a Close, a cancelled context and an already-delivered reply can no longer race each other over who cleans up. A reply the host already delivered keeps its bytes and its vendor outcome when the host dies.
Tests: the JS cancellation cases run the production sections with controlled stdout backpressure (stopped gate, split chunk, terminal suppression, acquired-body cancel, 30,000 cancelled bootstrap waits with no retained waiter or ready reaction); the Go cases drive lifecycle orderings directly (terminal-before-failure, failure-before-terminal, abort acknowledged before capacity returns, blocked enqueue), and the shutdown, body-close and mid-setup-cancel cases block on explicit channels instead of sleeping. TestFlowBigReplyProd reads 100 MiB non-SSE and SSE through the real host and checks lengths and SHA-256, including a 12 s reader pause. CI installs Bun 1.4.2 on Linux and runs that case in its own step, since the plugin sandbox skips when bun is not on PATH.
* plugin: the stream-head credit test waits for the credits before the reply ends (CI, macos-latest: TestHostStreamHeadChunksFinal failed on the macOS runner with "the credits were given back didn't happen", while the same package passed on ubuntu-latest and locally) — the test sent the child's final answer together with its 50 chunks, so the call could be marked stopped and its pending credits dropped before waitFor saw them; a completed fetch needs no credit, so waiting longer could not recover them and the expectation was a race, not a real accounting bug. The final answer is now withheld until io.ReadFull has taken the 140 payload bytes in order and exactly 140 + 49*frameOverhead credits have been observed with frameOverhead still owed, and only then does the child answer and the test assert a zero-byte EOF read. No production code changed.
Tests: go test -run '^TestHostStreamHeadChunksFinal$' -count=500 ./internal/plugin/ and -count=200 -race pass; go test -count=1 and -race -count=1 ./internal/plugin/... pass; gofmt -l and git diff --check clean.
* Add German translation and harden paths against unset env vars
Add a German locale to the GUI, tray menu, usage alerts and website home page, and refuse to start when the home folder is unset or relative so nothing is written under the working folder; the pi plugin no longer adopts the user's PI_CODING_AGENT_DIR, and an inherited APPIMAGE no longer selects portable mode. Scripts refuse to run from the wrong folder or empty a shared out folder, and a new internal/testenv sandboxes every test package from the user's real files, which also fixes the Claude picker and Termux installer tests on Windows.
* Makefile: know magpie's folder by its go.mod, so paths with spaces work
The check compared $(CURDIR) with $(dir $(abspath …)), which make splits on spaces, so a checkout under 'My Projects' or 'John Doe' refused every target. It now reads go.mod relative to the working folder and wants magpie's module there: make -C and a symlinked Makefile work, make -f from another folder still stops before any recipe runs.
* i18n: German for the strings main added since, with a parity test
75 strings main added (group order, patterns, slow-start wait, usage caps, RTK savings …) get their German, and two that main dropped go. gui-de.test.cjs checks, as gui-ja does, that de has every zh string with its placeholders and tags, and that the window reads in German when picked or on a German system.
* appdir: ignore relative folder variables when reading, keep them for children
CheckEnv no longer unsets anything. magpie reads the XDG folders, APPDATA/LOCALAPPDATA and every agent's folder variable through appdir.Getenv/LookupEnv, which pass over a relative value; the environment stays as it is, so the agents magpie starts get the variable as the user set it. That covers what proc.UserPath lends from the login shell too, which is set after main's check and reached the desktop app unchecked. On Windows a path rooted on the current drive (\Users\x, Git Bash's /c/x) counts as usable, and magpie version and help answer without a home, before run makes any folder.
* Sandbox the tests main added since
internal/wslrun gets testenv's TestMain, and four tests main added that set HOME alone set their home with testenv.SetHome, so on Windows they don't read the runner's profile through USERPROFILE.
* agentenv: PI_CONFIG_DIR and OPENCODE_DB take relative names on purpose
omp puts $PI_CONFIG_DIR under the home and OpenCode a relative $OPENCODE_DB in its data folder, so neither depends on the working folder: appdir no longer passes them over.
* i18n: German for the Library's multi-select and System One's decision API
The 19 strings main added since (#791's skill multi-select, the decision API snippets) get their German; the one main replaced (four APIs → five APIs) goes.
* appdir: tidy the import into magpie's group; say what Go makes of /c/x
The appdir import joins the group of magpie's own imports where a file has one. rooted's comment now says that Go takes a Git Bash path like /c/x as \c\x on the current drive, not C:\x.
Return the original rename error unless the move crosses filesystems.
Permission and sharing errors must not start a copy-and-delete operation
that can leave the destination behind or remove source contents.
Preserve the source Windows symlink's file or directory kind rather than
inferring it from a target that may not exist at the destination yet.
Keep relative targets, Developer Mode handling and long-path support.
Cover rename failures, Linux cross-device moves and Windows link kinds,
and run the focused session tests natively in Windows CI.
* termux: build natively and run plugins on Bun 1.4.2
Select nogui for Android builds, open browser and OAuth links with
termux-open-url, and write an executable Termux:Boot script that starts
the gateway with shell-quoted paths.
Bun 1.3 has no official Android build. Upgrade the default plugin runtime
to 1.4.2, add the official Android arm64 and amd64 download targets, and
update SHA-256 checksums for every supported platform.
Document Termux installation, startup and source updates.
Validation on Termux arm64:
- Native build and go vet pass.
- A fresh config downloads and verifies Android Bun 1.4.2 and loads a
test plugin.
- Boot startup can be enabled and disabled.
- The Web page, authenticated state API, unauthenticated API rejection,
gateway healthcheck and Web quit action work.
- Browser argument preservation, boot path quoting and permissions, and
plugin tests pass.
Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a10014-56c1-73d3-b3ee-2ea569713989
* site: install Android builds in Termux
Detect Termux through TERMUX_VERSION or its PREFIX and select the Android
CLI asset instead of a Linux binary. Install into PREFIX/bin by default,
keep MAGPIE_BIN_DIR overrides, and suggest the Web or terminal interface
without Linux desktop setup instructions.
When the release has no Android asset, explain how to build from source.
Add android/arm64 to release-cli and stop the build loop on failure.
Document the installer and its release-asset requirement.
Shell syntax and all 11 site tests pass, covering platform selection,
custom paths, missing assets, checksum rejection and existing Linux
installation paths. The Android arm64 release build succeeds on Termux;
with a simulated release feed, the installer verifies and installs that
binary into PREFIX/bin and the installed command runs successfully.
Amp-Thread-ID: https://ampcode.com/threads/T-01a10014-56c1-73d3-b3ee-2ea569713989
Co-authored-by: Amp <amp@ampcode.com>
* ci: check Android releases and the installer
Build both linux/amd64 and android/arm64 through make release-cli in the
CLI test matrix. Run the site and installer tests in the Linux job with
Node 22, and check the installer's shell syntax.
The release workflow in yetone/magpie-releases already calls the source
Makefile's release-cli target, uploads dist/magpie-cli-*, and generates
checksums for every artifact. The Android target added to TARGETS is
therefore included without changing that workflow.
Replace the full Termux README section with a short platform note beside
the Linux and Windows installation requirements, removing duplicate
commands, the fork URL and temporary release availability details.
Both CLI release builds, all 11 site tests, shell syntax checks and
actionlint validation pass locally.
Amp-Thread-ID: https://ampcode.com/threads/T-01a10014-56c1-73d3-b3ee-2ea569713989
Co-authored-by: Amp <amp@ampcode.com>
---------
Co-authored-by: Amp <amp@ampcode.com>
.github/workflows/ui-preview.yml, on pull_request_target or by hand (-f pr=N): detect (internal/gui/assets/, the GUI's non-test Go, or the ui-preview label) puts a "recording" note in the block; record, behind the ui-preview environment's approval since PRs come from forks, builds the PR's magpie (nogui), runs `magpie web` in a fresh home with DeepSeek added and a few requests sent through the gateway, and has DeepSeek plan from the diff and the pages' real outlines what to click and capture (record.mjs, Chromium at 2x, cursor.js drawing the mouse); publish, which never runs the PR's code, puts the files on the ui-previews branch served by Pages and rewrites only the block between <!-- magpie-ui-preview:begin/end --> — the video's poster links to its player page, since GitHub embeds no video it didn't host. Captions say what is being done, never what should come of it. Anything on screen matching the key or the web link's key throws the whole recording away.
Tested locally with run.sh on 033a151's diff in a sandbox home: 3 screenshots and a 16 s mp4 of the Codex model list's 全部隐藏/全部显示, pointer, trail and clicks visible; actionlint passes. Not yet run on GitHub.
* docker: a server image and a multi-arch build workflow
The image is the terminal-only binary built with -tags nogui on
distroless static, run as nonroot with magpie's files on a /config
volume, created and chowned at build time so a fresh named volume
is writable by uid 65532.
The workflow builds amd64 and arm64 on native runners, smoke-tests
each image before anything is pushed (ready probe, unknown-model
4xx JSON, /config write), pushes by digest, merges a multi-arch
manifest (semver trio plus latest on v* tags, edge otherwise) and
attests build provenance. PR runs build and smoke-test only.
The README documents running the image, the web UI mode, and why
subscription sign-ins must start where a browser shares loopback
with magpie.
Co-Authored-By: glm-5.3-flash <hakase@ooooo.space>
Generated-By: omp 18.4.3
* docker: README publishes the ports on the host's loopback and says how to require a key — until the gateway is shared from Settings it takes any key from anyone who reaches it, and Docker's -p 3425:3425 puts it on every interface of the host past its firewall. Checked in the built image: with settings lan+lanKey a request from the host through the published port without the key gets 401 and with it 200; without them any key gets 200.
---------
Co-authored-by: glm-5.3-flash <hakase@ooooo.space>
Co-authored-by: yetone <yetoneful@gmail.com>
The site leads with what magpie is for: Codex on DeepSeek, Claude Code on
Kimi. New screenshots (Codex's picker, the panel, Providers), a diagram of
the gateway, shared subscriptions, a light/dark/system theme switch, a new
OG image, and links to the source.
yetone