yetone
1ef6c92670
TestWSLProbeFindsBunOmp, TestReasonixCLIRequiresSupportedNativeVersion,
TestReasonixCLIRecoversAfterProbeTimeout and TestGrokInstallsWithoutShell
failed in loaded full runs and passed alone. None of the causes was a
serialized probe, a lock held across the exec or CPU starving the fake.
Each test exec'd a newly written file under a fixed wait. On macOS a new
executable waits for the system's first-run check before its first
exec. That check is a queue shared with every other new program starting
(each package's test binary and its fakes), so under a go test ./... it
outlasts the wait:
- WSL omp: the fake `omp` was written with os.WriteFile (its `bun` was
already a testenv.Program). The probe script's own `timeout 10 "$p"
--version` killed it, so the version came back empty. It is a
testenv.Program now.
- Reasonix: warmReasonixCLIFixture gave the freshly built Go binary's
first launch 10s ("start native CLI fixture: signal: killed"). It now
waits as long as the test runs (t.Context()), which is what the warm-up
is for. The probe assertions after it still run under reasonixCLI's 2s.
- Grok: the installer runs the downloaded file with `--version` under its
own 30s, and that file was always new. The test puts an already-run
file at the download's temp path. The installer writes over it in place
(O_TRUNC) and renames it, which keeps the pass.
- quota_served_cli_test's inert stand-ins (`exit 1`, each asked under a
3s wait) are testenv.Programs too. They gave the same answer when killed,
so this one cost time (5s at load), not a failure.
Production timeouts are unchanged. The WSL probe's 10s runs inside a
Linux distro, which has no such check. Grok's 30s verify and Reasonix's
2s probe of a binary that has already run passed in every loaded run
here. Reasonix doesn't cache a failed probe, so a first run that is slow
after an update is tried again on the next scan.
Evidence (macOS, load2.sh = the hjc load.sh recipe with N fresh-exec
loops, tests under `taskpolicy -c background`):
- A fresh 2-line script's first exec took 2.1-7.0s at 8 loops and
10.7-14.2s at 32. Its second exec took 0.02s. A file written over in
place and renamed ran in 0.02s.
- An instrumented copy of the omp test at 8 loops: a fresh omp hit
`timeout 10` in 6 of 6 runs (version empty). The same test with omp as
a testenv.Program took 0-1s in 6 of 6.
- TestWSLProbeFindsBunOmp at 32 loops: before 21.0s FAIL (version empty),
9.3s, 9.6s. After 0.63s and 0.75s, plus 21.6s for the first run, which
is the pool's one-time warm-up and has no timeout on it.
- TestGrokInstallsWithoutShell at 32 loops, before: "the downloaded Grok
Build 1.2.3 doesn't run here: signal: killed" after 78s.
- TestReasonixCLIRequiresSupportedNativeVersion at 8 loops, before:
"reasonix_test.go:178: start native CLI fixture: signal: killed" (12.0s).
- After the change, the four tests passed in every loaded A/B round
(8/24/32 loops).
84 lines
2.2 KiB
Go
84 lines
2.2 KiB
Go
package main
|
|
|
|
import (
|
|
"encoding/json"
|
|
"io"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"os"
|
|
"path/filepath"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/yetone/magpie/internal/agentenv"
|
|
"github.com/yetone/magpie/internal/provider"
|
|
"github.com/yetone/magpie/internal/testenv"
|
|
)
|
|
|
|
// magpie quota --json tells when each key last answered through the gateway
|
|
// — a process of its own, which wrote it down — and marks the latest last,
|
|
// as GET /v1/magpie/quotas does (#570).
|
|
func TestQuotaJSONLastServedAt(t *testing.T) {
|
|
groupsHome(t)
|
|
for _, k := range agentenv.Vars {
|
|
t.Setenv(k, "")
|
|
os.Unsetenv(k)
|
|
}
|
|
// no agent's CLI nor the keychain is asked: what answers is inert
|
|
bin := t.TempDir()
|
|
for _, name := range []string{"security", "secret-tool", "claude", "codex", "cursor-agent", "devin", "grok", "kiro-cli"} {
|
|
testenv.Program(t, filepath.Join(bin, name), "#!/bin/sh\nexit 1\n")
|
|
}
|
|
t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH"))
|
|
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, `{"balance":"7"}`) }))
|
|
t.Cleanup(up.Close)
|
|
for _, id := range []string{"a", "b"} {
|
|
p, err := provider.Find(id)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
p.BalanceURL, p.BalancePath = up.URL, "balance"
|
|
if err := provider.Save(*p); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
provider.ForgetBalances()
|
|
at := time.Now().Add(-time.Minute).Truncate(time.Second)
|
|
a, _ := provider.Find("a")
|
|
provider.NoteServed(*a, at)
|
|
r, w, err := os.Pipe()
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
old := os.Stdout
|
|
os.Stdout = w
|
|
done := make(chan []byte)
|
|
go func() { b, _ := io.ReadAll(r); done <- b }()
|
|
err = quotaCmd([]string{"quota", "a", "b", "--json"})
|
|
w.Close()
|
|
os.Stdout = old
|
|
out := <-done
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
var qs []map[string]any
|
|
if err := json.Unmarshal(out, &qs); err != nil {
|
|
t.Fatal(err, string(out))
|
|
}
|
|
if len(qs) != 2 {
|
|
t.Fatalf("%s", out)
|
|
}
|
|
for _, q := range qs {
|
|
switch q["provider"] {
|
|
case "a":
|
|
if q["lastServedAt"] != at.UTC().Format(time.RFC3339) || q["last"] != true {
|
|
t.Errorf("served: %v", q)
|
|
}
|
|
case "b":
|
|
if _, ok := q["lastServedAt"]; ok || q["last"] != nil {
|
|
t.Errorf("never served: %v", q)
|
|
}
|
|
}
|
|
}
|
|
}
|