gui: the sessions warm-up hands its garbage back to the system when it ends, so a magpie just started doesn't read ~470 MB at rest for its first two minutes (H20 on Discord)

The warm-up reads every session file for the Sessions page and Usage › Requests. On a 13.5k-session history it allocates ~830 MB (QueryPage ~570, StatsFor ~260) to keep ~80 MB of indexes, and Go's scavenger returns the freed pages only over the next minutes. warmUp now calls debug.FreeOSMemory once its reads are done: one forced GC and scavenge, once, at start.

RSS of a linux nogui `magpie web` in Docker on a copy of a real config with those sessions mounted read-only, sampled every 15 s from start (with the dsh round change before it): 15 s 472 -> 197 MB, 30 s 475 -> 285, 60 s 417 -> 281, 2 min 337 -> 281, 3.5 min 266 -> 218, 12 min ~250 -> ~204. TestWarmUpReturnsItsGarbage fails without the FreeOSMemory call (10 of 10: 46-334 MB of freed heap still held, or the heap not yet collected) and passes 10 of 10 with it. go test -tags nogui ./internal/gui ./internal/sessions (Docker golang:1.26.8, temp HOME), go vet -tags nogui ./internal/gui for linux, windows and darwin, and gofmt -l pass. Not measured on macOS: a fresh build there hangs at exec behind syspolicyd today.
This commit is contained in:
yetone committed 2026-10-08 21:21:01 +08:00
1 parent 1d1c9e07d2
commit ddb6f806ac
2 files changed
+47 -3

No files matched your search

+19 -3
View File
@@ -8,6 +8,7 @@ import (
"net/http"
"os"
"runtime"
"runtime/debug"
"strconv"
"strings"
"sync"
@@ -320,12 +321,27 @@ func warmSessions() {
}
go func() {
time.Sleep(3 * time.Second)
statsFor(0)
statsFor(30)
usage.QueryPage(usage.All, usage.Filter{}, 0, 50)
warmUp(
func() { statsFor(0) },
func() { statsFor(30) },
func() { usage.QueryPage(usage.All, usage.Filter{}, 0, 50) },
)
}()
}
// warmUp runs the warm-up's reads, then hands what they threw away back to
// the system at once. Reading a long history allocates many times what it
// keeps (a 13k-session Codex and Claude Code history: ~830 MB allocated to
// keep ~80 MB of indexes), and Go's scavenger returns those pages only over
// the next minutes: a magpie that had just started read ~470 MB for its
// first two minutes at rest.
func warmUp(steps ...func()) {
for _, step := range steps {
step()
}
debug.FreeOSMemory()
}
// openTerminal runs a command in the chosen Mac terminal through a .command
// file, as it would open from Finder: no Automation consent.
// The shell is left open when the agent quits.
+28
View File
@@ -0,0 +1,28 @@
package gui
import (
"runtime"
"testing"
)
// The warm-up's garbage goes back to the system when it ends, not over the
// scavenger's next minutes: an idle magpie that just started read ~470 MB.
func TestWarmUpReturnsItsGarbage(t *testing.T) {
var keep [][]byte
warmUp(func() {
// a read that allocates far more than it keeps
for range 256 {
keep = append(keep, make([]byte, 1<<20))
}
keep = [][]byte{keep[0]}
})
var m runtime.MemStats
runtime.ReadMemStats(&m)
if held := m.HeapIdle - m.HeapReleased; held > 32<<20 {
t.Fatalf("after the warm-up %d MB of freed heap is still held from the system", held>>20)
}
if m.HeapAlloc > 128<<20 {
t.Fatalf("after the warm-up %d MB of heap is still in use", m.HeapAlloc>>20)
}
runtime.KeepAlive(keep)
}