feat(gateway): named gateway keys with per-client usage (#344)
* feat(gui): show and filter usage for each API key * feat(gateway): issue per-user caller keys and attribute usage * feat(gui): manage gateway users and filter caller-key usage * test: verify caller-key failover and document user management * refactor(gateway): use named caller keys without user accounts * feat(gui): manage named caller keys with existing account styles * fix: remove user breakdown from CLI and leftover gateway styling * fix(usage): drop stale user field from request response * feat(gateway): authenticate LAN calls with named caller keys * fix(gateway): preserve LAN key rotation in the named caller store * fix(gui): retain native LAN key copy and rotation controls * fix(gui): inherit the global theme for API key inputs * fix(usage): show client API keys using the existing terminology * fix(gateway): rotate named keys without changing usage identity * fix(gui): unify gateway keys and selectable connection examples Keep sharing controls in Settings and manage credentials only in Gateway. Refresh the named-key list on navigation, ignore stale reads after key actions, and generate all API examples from the selected address and enabled key. * fix(gui): keep caller key names literal in connection menus * fix(access): use Magpie as default key and drop user migration * refactor(gui): name caller key source and tests consistently * refactor(usage): split provider-key accounting out of caller-key PR * fix(gateway): preserve LAN compatibility and add headless key management * fix(gui): clarify gateway keys and confirm destructive actions * test(gateway): guard caller attribution under 120 concurrent requests * fix(gateway): keep open access, tolerate migration errors and revoke legacy keys * fix(backup): preserve named gateway keys and honor no-keys restores * fix(gui): keep local API-key semantics and disambiguate shared credentials * docs(gateway): keep review scope focused and append browser test documentation * fix(backup): retain existing default keys when old backups lack their store * fix(gateway): attribute video creation usage to the caller key * fix(access): explain default key revocation blocked by read-only settings * test(usage): preserve provider and caller identities after rebase * test(gui): reconcile caller key fixtures with merged main * docs(usage): clarify caller coverage and independent provider accounting * fix(gui): initialize fast members for model-shortcut groups Latest main introduced fast-member arrays but omitted the model shortcut's draft. The existing unlisted-models browser regression fails unchanged on main. Initialize fast to an empty list and assert a standard-speed group is saved, without relaxing existing assertions. * fix(access): retain legacy LAN access when the key store cannot be created Allow only the configured legacy credential before either migration marker exists. Refuse revoked mirrors, disabled matches and unreadable stores; keep the fallback's caller identity stable when migration later succeeds. Cover read-only configuration directories, remote credential transports and revocation. * fix(backup): save settings before replacing gateway credentials Leave the key store untouched when settings validation or writes fail. Restore the previous settings association when the key store refuses its write, and report a rollback failure separately. Cover invalid settings and read-only settings/store failures. * fix(gui): reset missing caller filters and focus key controls without scrolling * test(provider): synchronize Zhipu quota fixture observations
This commit is contained in:
1 parent
820413b87f
commit
bb1aa3acbd
56 files changed
+4080
-248
No files matched your search
@@ -156,6 +156,54 @@ Its list is the models the shared magpie's agents are shown, each named with
|
||||
its provider there (`Claude Sonnet 5 · Relay A · office`), and its image
|
||||
models are listed under Settings → Images and draw through it.
|
||||
|
||||
The gateway issues **gateway keys** for clients, separate from a provider's
|
||||
upstream API keys. Turn on **Settings → Share on local network**, then open
|
||||
**Gateway → Gateway keys → Add gateway key**. This block appears only while
|
||||
sharing is on. Create a named key for each client and copy it from its row.
|
||||
Rename, disable, rotate or remove keys independently; rotation and removal
|
||||
ask for confirmation. Rotation keeps the name, enabled state and usage
|
||||
history; other keys are unchanged. While sharing is on, **Gateway → Connect**
|
||||
offers loopback and shared addresses, plus enabled gateway keys, for all
|
||||
connection examples. With sharing off, Connect keeps the original **API key**
|
||||
field and local `magpie` token, without a gateway-key picker. Its arbitrary
|
||||
local option is **This computer**, distinct from the named **Magpie** key.
|
||||
|
||||
For a headless gateway, use the CLI before exposing the port:
|
||||
|
||||
```sh
|
||||
magpie gateway-key add "Remote laptop" # prints the new credential once
|
||||
magpie gateway-key list # ids, names, enabled state and masked keys
|
||||
magpie gateway-key rotate <id> # prints the replacement; identity stays the same
|
||||
magpie gateway-key remove <id> # revokes remote access
|
||||
```
|
||||
|
||||
While LAN sharing is enabled, remote requests require an enabled gateway key
|
||||
sent as Bearer, `x-api-key`, `x-goog-api-key` or `?key=`. Loopback remains
|
||||
permissive: any token works, including a stale or disabled gateway key.
|
||||
Only a valid, enabled key is attributed to its named identity.
|
||||
Without sharing, an explicitly exposed `MAGPIE_ADDR` keeps its original open
|
||||
access, including old `sk-magpie-…` tokens, without key authentication.
|
||||
|
||||
**Usage → Overview → Gateway keys** groups calls by the client's key, never
|
||||
the provider's credential. **Usage → Requests** offers the same filter;
|
||||
CSV includes `caller_key_id` and `caller_key_name`. Deleted keys keep their
|
||||
history. The usual local `magpie` token and older records stay unattributed.
|
||||
Caller attribution includes chat, images, video creation and System One.
|
||||
Provider attribution stays independent; CSV puts `provider_key_*` before
|
||||
`caller_key_*`.
|
||||
An existing LAN key becomes **Magpie** without changing the credential.
|
||||
`lanKey` remains in settings for older Magpie versions. Disabling or removing
|
||||
the default key replaces that mirror with a random non-empty revoked value;
|
||||
rotation does not re-enable it. The key store records migration completion,
|
||||
even if `lanKeyId` cannot be saved, so reads do not keep retrying that write.
|
||||
A migration write failure is logged without preventing gateway startup,
|
||||
CLI key management, or the Settings and key-list pages from opening.
|
||||
If settings are read-only, changing the default key fails without changing
|
||||
it: make `settings.json` writable and retry so older versions cannot keep
|
||||
accepting its old credential. Independent named keys remain manageable.
|
||||
Gateway credentials stay in `~/.config/magpie/caller-keys.json` (XDG-aware,
|
||||
mode `0600`), never in usage records or list responses.
|
||||
|
||||
Baidu Qianfan's [Token Plans](https://cloud.baidu.com/doc/qianfan/s/Dmrabu8b6)
|
||||
are available as `baidu-qianfan`: a personal (个人版) and an enterprise (企业版)
|
||||
plan and pay as you go, each with its own Chat Completions, Responses and
|
||||
@@ -631,9 +679,12 @@ docker run -d --name magpie -p 127.0.0.1:3425:3425 -p 127.0.0.1:3430:3430 -v mag
|
||||
published on the host's loopback only; Docker's `-p 3425:3425` would put it
|
||||
on every interface of the host, past its firewall. To reach it from other
|
||||
machines, turn on Settings → Share on local network in the browser UI (or
|
||||
put `"lan": true, "lanKey": "sk-magpie-…"` in `/config/magpie/settings.json`):
|
||||
from then on a request from outside the container must carry that key as its
|
||||
API key, and only then publish the port beyond 127.0.0.1. Inside the container
|
||||
put `"lan": true` in `/config/magpie/settings.json`). Turning it on in Settings
|
||||
creates a named **Magpie** key. With settings edited by hand, run
|
||||
`magpie gateway-key add "Docker client"` in the container to create a key
|
||||
without the browser UI. A request from outside the container must carry one of
|
||||
those keys as its API key. Only then publish the port beyond 127.0.0.1.
|
||||
Inside the container
|
||||
magpie only sees the container's own address (Docker's 172.17.x), so set
|
||||
`-e MAGPIE_PUBLIC_URL=http://<the host's or NAS's address>:3425` (the port
|
||||
published on the host) for the address it shows and prints to be the one
|
||||
@@ -755,6 +806,10 @@ pictures picked for them, the settings, the profiles, every agent's model and
|
||||
the library (unless `--no-library`): the instruction sets, the MCP servers and
|
||||
the skills with their files (a file over 2 MB is left out). Without keys, a
|
||||
server's environment variables and headers that look like a key go empty.
|
||||
Gateway credentials, their names, ids and disabled state travel encrypted
|
||||
with Settings too. Restoring Settings replaces the gateway-key store with
|
||||
the backed-up one. `--no-keys` leaves gateway credentials and their legacy
|
||||
mirror out; restoring it preserves the destination's existing keys instead.
|
||||
Restoring the library replaces the one there — what it replaces is kept with
|
||||
the library's backups — and writes it into the agents on that machine.
|
||||
It is encrypted on your machine (AES-256-GCM, the key derived from the
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strings"
|
||||
"text/tabwriter"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
)
|
||||
|
||||
func gatewayKeys(args []string) error { return gatewayKeysTo(os.Stdout, args) }
|
||||
|
||||
func gatewayKeysTo(out io.Writer, args []string) error {
|
||||
args = args[1:]
|
||||
if len(args) == 0 {
|
||||
args = []string{"list"}
|
||||
}
|
||||
action := args[0]
|
||||
if (action == "list" && len(args) != 1) || (action != "list" && len(args) != 2) {
|
||||
return fmt.Errorf("usage: magpie gateway-key list | add <name> | rotate <id> | remove <id>")
|
||||
}
|
||||
switch action {
|
||||
case "list", "add", "rotate", "remove":
|
||||
default:
|
||||
return fmt.Errorf("unknown gateway-key command %q", action)
|
||||
}
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
if action == "list" {
|
||||
keys, err := access.List()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
w := tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
|
||||
fmt.Fprintln(w, "ID\tNAME\tSTATE\tGATEWAY KEY")
|
||||
for _, k := range keys {
|
||||
state := "enabled"
|
||||
if k.Off {
|
||||
state = "disabled"
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\n", k.ID, strings.Map(func(r rune) rune {
|
||||
if r < 32 || r == 127 {
|
||||
return ' '
|
||||
}
|
||||
return r
|
||||
}, k.Name), state, k.Masked)
|
||||
}
|
||||
return w.Flush()
|
||||
}
|
||||
in := access.Change{Key: args[1]}
|
||||
if action == "add" {
|
||||
in = access.Change{Name: args[1]}
|
||||
}
|
||||
secret, err := access.Update(action+"-key", in)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if secret != "" {
|
||||
_, err = fmt.Fprintln(out, secret)
|
||||
} else {
|
||||
_, err = fmt.Fprintln(out, "Gateway key removed")
|
||||
}
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
func TestGatewayKeyCLI(t *testing.T) {
|
||||
t.Setenv("HOME", t.TempDir())
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
if err := settings.Save(settings.Settings{LAN: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
call := func(args ...string) string {
|
||||
t.Helper()
|
||||
var out bytes.Buffer
|
||||
if err := gatewayKeysTo(&out, append([]string{"gateway-key"}, args...)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return strings.TrimSpace(out.String())
|
||||
}
|
||||
if got := call("list"); !strings.Contains(got, "GATEWAY KEY") {
|
||||
t.Fatal(got)
|
||||
}
|
||||
secret := call("add", "Remote laptop")
|
||||
keys, _ := access.List()
|
||||
if len(keys) != 1 {
|
||||
t.Fatal(keys)
|
||||
}
|
||||
id := keys[0].ID
|
||||
if who, ok := access.Authenticate(secret); !ok || who.KeyID != id || who.KeyName != "Remote laptop" {
|
||||
t.Fatal(who, ok)
|
||||
}
|
||||
if got := call(); strings.Contains(got, secret) || !strings.Contains(got, id) || !strings.Contains(got, "Remote laptop") {
|
||||
t.Fatal("list must mask credentials", got)
|
||||
}
|
||||
next := call("rotate", id)
|
||||
if next == secret {
|
||||
t.Fatal("rotation reused the secret")
|
||||
}
|
||||
if _, ok := access.Authenticate(secret); ok {
|
||||
t.Fatal("old credential works after rotation")
|
||||
}
|
||||
if who, ok := access.Authenticate(next); !ok || who.KeyID != id {
|
||||
t.Fatal("rotation changed identity", who, ok)
|
||||
}
|
||||
call("remove", id)
|
||||
if _, ok := access.Authenticate(next); ok {
|
||||
t.Fatal("removed credential still works")
|
||||
}
|
||||
for _, args := range [][]string{{"add"}, {"add", ""}, {"list", "extra"}, {"rotate", "missing"}, {"remove", "missing"}, {"unknown", "x"}} {
|
||||
if err := gatewayKeysTo(&bytes.Buffer{}, append([]string{"gateway-key"}, args...)); err == nil {
|
||||
t.Error("accepted", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayKeyListWithReadOnlyMigration(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Settings{LAN: true, LANKey: "sk-magpie-fixture-cli"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Path(), 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Path(), 0o600) })
|
||||
if err := settings.Save(s); err == nil {
|
||||
t.Skip("settings.json remains writable")
|
||||
}
|
||||
var out bytes.Buffer
|
||||
if err := gatewayKeysTo(&out, []string{"gateway-key", "list"}); err != nil || !strings.Contains(out.String(), "Magpie") {
|
||||
t.Fatal("read-only migration blocked CLI list", out.String(), err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,345 @@
|
||||
// Package access keeps the named API keys issued by the gateway.
|
||||
package access
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"crypto/subtle"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/yetone/magpie/internal/edit"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
const Prefix = "sk-magpie-key-"
|
||||
const legacyLANPrefix = "sk-magpie-"
|
||||
const revokedLANPrefix = "sk-magpie-revoked-"
|
||||
|
||||
func Managed(secret string) bool {
|
||||
return strings.HasPrefix(secret, legacyLANPrefix)
|
||||
}
|
||||
|
||||
type Key struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Off bool `json:"off,omitempty"`
|
||||
LAN bool `json:"lan,omitempty"` // default key and durable migration marker
|
||||
Secret string `json:"secret,omitempty"`
|
||||
Masked string `json:"masked,omitempty"`
|
||||
}
|
||||
|
||||
type Identity struct{ KeyID, KeyName string }
|
||||
type contextKey struct{}
|
||||
|
||||
func WithIdentity(ctx context.Context, who Identity) context.Context {
|
||||
return context.WithValue(ctx, contextKey{}, who)
|
||||
}
|
||||
func Caller(ctx context.Context) Identity { who, _ := ctx.Value(contextKey{}).(Identity); return who }
|
||||
|
||||
var mu sync.Mutex
|
||||
|
||||
func Path() string { return filepath.Join(settings.Dir(), "caller-keys.json") }
|
||||
|
||||
func load() ([]Key, error) {
|
||||
b, err := os.ReadFile(Path())
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return []Key{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var keys []Key
|
||||
err = json.Unmarshal(b, &keys)
|
||||
return keys, err
|
||||
}
|
||||
|
||||
// List never returns credentials; only the administrator's copy action does.
|
||||
func List() ([]Key, error) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
keys, err := load()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for i := range keys {
|
||||
k := &keys[i]
|
||||
if len(k.Secret) > 8 {
|
||||
prefix := legacyLANPrefix
|
||||
if strings.HasPrefix(k.Secret, Prefix) {
|
||||
prefix = Prefix
|
||||
}
|
||||
k.Masked = prefix + "…" + k.Secret[len(k.Secret)-6:]
|
||||
}
|
||||
k.Secret = ""
|
||||
}
|
||||
return keys, nil
|
||||
}
|
||||
|
||||
// Export returns credentials only for the encrypted backup bundle.
|
||||
func Export() ([]Key, error) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return load()
|
||||
}
|
||||
|
||||
// Restore replaces the gateway credentials when restoring their settings.
|
||||
func Restore(keys []Key) error {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return save(slices.Clone(keys))
|
||||
}
|
||||
|
||||
func random(n int) (string, error) {
|
||||
b := make([]byte, n)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(b), nil
|
||||
}
|
||||
|
||||
type Change struct {
|
||||
Key string `json:"key"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
// Update writes the named key store atomically.
|
||||
func Update(action string, in Change) (string, error) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
keys, err := load()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
name := strings.TrimSpace(in.Name)
|
||||
if action == "add-key" || action == "rename-key" {
|
||||
if name == "" || utf8.RuneCountInString(name) > 120 {
|
||||
return "", errors.New("Use a name between 1 and 120 characters")
|
||||
}
|
||||
}
|
||||
var secret string
|
||||
var mirror *Key
|
||||
if action == "add-key" {
|
||||
id, err := random(12)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
token, err := random(24)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
secret = Prefix + token
|
||||
keys = append(keys, Key{ID: id, Name: name, Secret: secret})
|
||||
} else {
|
||||
i := slices.IndexFunc(keys, func(k Key) bool { return k.ID == in.Key })
|
||||
if i < 0 {
|
||||
return "", errors.New("Key not found")
|
||||
}
|
||||
defaultKey := keys[i].LAN || keys[i].ID == settings.Load().LANKeyID
|
||||
if defaultKey {
|
||||
keys[i].LAN = true
|
||||
}
|
||||
switch action {
|
||||
case "rotate-key":
|
||||
token, err := random(24)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
secret = Prefix + token
|
||||
keys[i].Secret = secret
|
||||
case "rename-key":
|
||||
keys[i].Name = name
|
||||
case "on-key", "off-key":
|
||||
keys[i].Off = action == "off-key"
|
||||
case "remove-key":
|
||||
if defaultKey {
|
||||
k := keys[i]
|
||||
k.Off = true
|
||||
mirror = &k
|
||||
}
|
||||
keys = slices.Delete(keys, i, i+1)
|
||||
case "copy-key":
|
||||
return keys[i].Secret, nil
|
||||
default:
|
||||
return "", fmt.Errorf("unknown key action %q", action)
|
||||
}
|
||||
if defaultKey && action != "remove-key" && action != "rename-key" {
|
||||
mirror = &keys[i]
|
||||
}
|
||||
}
|
||||
if mirror != nil {
|
||||
s := settings.Load()
|
||||
if err := setLegacyMirror(&s, *mirror); err != nil {
|
||||
return "", err
|
||||
}
|
||||
// Revoke the old-version mirror before changing the active key store.
|
||||
if err := settings.Save(s); err != nil {
|
||||
return "", fmt.Errorf("Cannot change the default gateway key without updating settings.json: older Magpie versions may still accept its old credential. Make the settings file writable and retry; the key is unchanged: %w", err)
|
||||
}
|
||||
}
|
||||
if err := save(keys); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return secret, nil
|
||||
}
|
||||
|
||||
func setLegacyMirror(s *settings.Settings, k Key) error {
|
||||
s.LANKeyID = k.ID
|
||||
if !k.Off {
|
||||
s.LANKey = k.Secret
|
||||
return nil
|
||||
}
|
||||
token, err := random(24)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.LANKey = revokedLANPrefix + token
|
||||
return nil
|
||||
}
|
||||
|
||||
// Authenticate reloads the store so revocation takes effect in running gateways.
|
||||
func Authenticate(secret string) (Identity, bool) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
keys, err := load()
|
||||
if err != nil || secret == "" {
|
||||
return Identity{}, false
|
||||
}
|
||||
for _, k := range keys {
|
||||
if subtle.ConstantTimeCompare([]byte(secret), []byte(k.Secret)) == 1 {
|
||||
if k.Off {
|
||||
return Identity{}, false
|
||||
}
|
||||
return Identity{k.ID, k.Name}, true
|
||||
}
|
||||
}
|
||||
s := settings.Load()
|
||||
// A read-only config directory may prevent the first store write. Never
|
||||
// fall back after either durable marker exists, or to a revoked mirror.
|
||||
if s.LAN && s.LANKeyID == "" && s.LANKey != "" && !strings.HasPrefix(s.LANKey, revokedLANPrefix) && !slices.ContainsFunc(keys, func(k Key) bool { return k.LAN }) && subtle.ConstantTimeCompare([]byte(secret), []byte(s.LANKey)) == 1 {
|
||||
k := legacyLANKey(s.LANKey)
|
||||
return Identity{k.ID, k.Name}, true
|
||||
}
|
||||
return Identity{}, false
|
||||
}
|
||||
|
||||
func legacyLANKey(secret string) Key {
|
||||
// The fallback and eventual persisted key must share their usage identity.
|
||||
sum := sha256.Sum256([]byte(secret))
|
||||
return Key{ID: "lan-" + hex.EncodeToString(sum[:12]), Name: "Magpie", LAN: true, Secret: secret}
|
||||
}
|
||||
|
||||
// MigrateLegacyLANKey makes the old Settings key a normal, revocable caller
|
||||
// key. Keep LANKey for older Magpie versions; LANKeyID marks a completed
|
||||
// migration so the retained credential cannot resurrect a removed key.
|
||||
func MigrateLegacyLANKey() error {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
return migrateLegacyLANKey()
|
||||
}
|
||||
|
||||
func MigrateLegacyLANKeyBestEffort() {
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
log.Printf("magpie: could not migrate LAN key: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func migrateLegacyLANKey() error {
|
||||
s := settings.Load()
|
||||
if s.LANKey == "" || s.LANKeyID != "" || strings.HasPrefix(s.LANKey, revokedLANPrefix) {
|
||||
return nil
|
||||
}
|
||||
keys, err := load()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if slices.ContainsFunc(keys, func(k Key) bool { return k.LAN }) {
|
||||
return nil // the key-store write succeeded, even if settings were read-only
|
||||
}
|
||||
i := slices.IndexFunc(keys, func(k Key) bool {
|
||||
return subtle.ConstantTimeCompare([]byte(k.Secret), []byte(s.LANKey)) == 1
|
||||
})
|
||||
if i < 0 {
|
||||
keys = append(keys, legacyLANKey(s.LANKey))
|
||||
i = len(keys) - 1
|
||||
}
|
||||
keys[i].LAN = true
|
||||
if err := save(keys); err != nil {
|
||||
return err
|
||||
}
|
||||
s.LANKeyID = keys[i].ID
|
||||
return settings.Save(s)
|
||||
}
|
||||
|
||||
// ConfigureLAN starts sharing with a named key. Rotation preserves the key's
|
||||
// identity, name and enabled state.
|
||||
func ConfigureLAN(on, rotate bool) error {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if err := migrateLegacyLANKey(); err != nil {
|
||||
return err
|
||||
}
|
||||
s := settings.Load()
|
||||
if on {
|
||||
keys, err := load()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
i := slices.IndexFunc(keys, func(k Key) bool { return k.ID == s.LANKeyID || k.LAN })
|
||||
if i < 0 || rotate {
|
||||
token, err := random(24)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if i < 0 {
|
||||
id, err := random(12)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
keys = append(keys, Key{ID: id, Name: "Magpie", LAN: true})
|
||||
i = len(keys) - 1
|
||||
}
|
||||
keys[i].Secret = Prefix + token
|
||||
if err := save(keys); err != nil {
|
||||
return err
|
||||
}
|
||||
s.LANKeyID = keys[i].ID
|
||||
}
|
||||
if err := setLegacyMirror(&s, keys[i]); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
s.LAN = on
|
||||
return settings.Save(s)
|
||||
}
|
||||
|
||||
func save(keys []Key) error {
|
||||
b, err := json.MarshalIndent(keys, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(Path()), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
f, err := os.OpenFile(Path(), os.O_CREATE|os.O_WRONLY, 0o600)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.Chmod(Path(), 0o600); err != nil {
|
||||
return err
|
||||
}
|
||||
return edit.WriteAtomic(Path(), append(b, '\n'))
|
||||
}
|
||||
@@ -0,0 +1,302 @@
|
||||
package access
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
func TestNamedKeys(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
if _, err := Update("add-key", Change{Name: " "}); err == nil {
|
||||
t.Fatal("accepted empty name")
|
||||
}
|
||||
keys, err := List()
|
||||
if err != nil || len(keys) != 0 {
|
||||
t.Fatal(keys, err)
|
||||
}
|
||||
secret, err := Update("add-key", Change{Name: "Laptop"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, err = List()
|
||||
if err != nil || len(keys) != 1 || keys[0].Name != "Laptop" || keys[0].Secret != "" || !strings.HasPrefix(keys[0].Masked, Prefix+"…") {
|
||||
t.Fatal(keys, err)
|
||||
}
|
||||
id := keys[0].ID
|
||||
if who, ok := Authenticate(secret); !ok || who.KeyID != id || who.KeyName != "Laptop" {
|
||||
t.Fatal(who, ok)
|
||||
}
|
||||
if _, ok := Authenticate(secret + "x"); ok {
|
||||
t.Fatal("accepted invalid key")
|
||||
}
|
||||
for _, step := range []struct {
|
||||
action, name string
|
||||
valid bool
|
||||
}{
|
||||
{"rename-key", "Travel", true}, {"off-key", "", false}, {"on-key", "", true},
|
||||
} {
|
||||
if _, err := Update(step.action, Change{Key: id, Name: step.name}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if who, ok := Authenticate(secret); ok != step.valid || (ok && who.KeyName != "Travel") {
|
||||
t.Fatal(step.action, who, ok)
|
||||
}
|
||||
}
|
||||
if got, err := Update("copy-key", Change{Key: id}); err != nil || got != secret {
|
||||
t.Fatal("copy", err)
|
||||
}
|
||||
st, err := os.Stat(Path())
|
||||
if err != nil || st.Mode().Perm() != 0o600 {
|
||||
t.Fatal("permissions", err)
|
||||
}
|
||||
var persisted []Key
|
||||
b, _ := os.ReadFile(Path())
|
||||
if err := json.Unmarshal(b, &persisted); err != nil || len(persisted) != 1 || persisted[0].ID != id || persisted[0].Secret != secret {
|
||||
t.Fatal("key was not persisted", err)
|
||||
}
|
||||
if _, err := Update("remove-key", Change{Key: id}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := Authenticate(secret); ok {
|
||||
t.Fatal("removed key works")
|
||||
}
|
||||
}
|
||||
|
||||
func TestConcurrentKeyCreationAndCorruptStore(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 12; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if _, err := Update("add-key", Change{Name: "Key"}); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
keys, _ := List()
|
||||
if len(keys) != 12 {
|
||||
t.Fatal("lost concurrent changes")
|
||||
}
|
||||
secret, _ := Update("copy-key", Change{Key: keys[0].ID})
|
||||
os.WriteFile(Path(), []byte("broken"), 0o600)
|
||||
if _, ok := Authenticate(secret); ok {
|
||||
t.Fatal("corrupt store accepted a key")
|
||||
}
|
||||
if _, err := Update("add-key", Change{Name: "New"}); err == nil {
|
||||
t.Fatal("overwrote corrupt store")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyLANKeyMigrationIsIdempotentAndRevocable(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Load()
|
||||
s.LAN, s.LANKey = true, "sk-magpie-legacy-lan-secret"
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, err := List()
|
||||
if err != nil || len(keys) != 1 || keys[0].Name != "Magpie" || !strings.HasPrefix(keys[0].Masked, "sk-magpie-…") {
|
||||
t.Fatal(keys, err)
|
||||
}
|
||||
if settings.Load().LANKey != s.LANKey || !settings.Load().LAN {
|
||||
t.Fatal("migration broke older Magpie's sharing settings")
|
||||
}
|
||||
if who, ok := Authenticate(s.LANKey); !ok || who.KeyID != keys[0].ID {
|
||||
t.Fatal(who, ok)
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ = List()
|
||||
if len(keys) != 1 {
|
||||
t.Fatal("duplicate migration", keys)
|
||||
}
|
||||
if _, err := Update("off-key", Change{Key: keys[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := Authenticate(s.LANKey); ok {
|
||||
t.Fatal("disabled LAN key still works")
|
||||
}
|
||||
if _, err := Update("remove-key", Change{Key: keys[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := Authenticate(s.LANKey); ok {
|
||||
t.Fatal("removed LAN key still works")
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if keys, _ := List(); len(keys) != 0 {
|
||||
t.Fatal("retained legacy credential resurrected a removed key", keys)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyLANKeyMigrationPreservesNamedKeys(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
secret, err := Update("add-key", Change{Name: "Client"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ := List()
|
||||
id := keys[0].ID
|
||||
s := settings.Load()
|
||||
s.LANKey = "sk-magpie-legacy"
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ = List()
|
||||
if len(keys) != 2 || keys[0].ID != id || keys[1].Name != "Magpie" {
|
||||
t.Fatal(keys)
|
||||
}
|
||||
if _, ok := Authenticate(secret); !ok {
|
||||
t.Fatal("lost named key")
|
||||
}
|
||||
if _, ok := Authenticate(s.LANKey); !ok {
|
||||
t.Fatal("lost LAN key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyLANKeyMigrationRetryAndFailure(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Load()
|
||||
s.LANKey = "sk-magpie-test-legacy-lan"
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The key write succeeded but saving the migration marker was interrupted.
|
||||
mu.Lock()
|
||||
err := save([]Key{{ID: "migrated", Name: "Remote", Off: true, Secret: s.LANKey}})
|
||||
mu.Unlock()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, err := List()
|
||||
if err != nil || len(keys) != 1 || keys[0].ID != "migrated" || !keys[0].Off || settings.Load().LANKeyID != "migrated" {
|
||||
t.Fatal("retry duplicated or re-enabled a key", keys, err)
|
||||
}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(Path(), []byte("broken"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err == nil {
|
||||
t.Fatal("migration accepted a corrupt store")
|
||||
}
|
||||
if settings.Load().LANKey != s.LANKey {
|
||||
t.Fatal("failed migration cleared the old credential")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNamedLANKeyKeepsItsName(t *testing.T) {
|
||||
for _, name := range []string{"Local network", "Magpie", "Desk"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Load()
|
||||
s.LANKeyID = "lan"
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := []Key{{ID: "lan", Name: name, Off: true, Secret: Prefix + "original"},
|
||||
{ID: "other", Name: "Other", Secret: Prefix + "other"},
|
||||
{ID: "named", Name: "Local network", Secret: Prefix + "named"}}
|
||||
if err := save(original); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for range 2 {
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
keys, err := load()
|
||||
if err != nil || len(keys) != 3 || keys[0] != original[0] || keys[1] != original[1] || keys[2] != original[2] {
|
||||
t.Fatal("migration changed an existing named key", keys, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallerKeyRotationPreservesIdentityAndState(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
old, err := Update("add-key", Change{Name: "Desk"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other, err := Update("add-key", Change{Name: "Other"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ := List()
|
||||
id := keys[0].ID
|
||||
for _, off := range []bool{false, true} {
|
||||
if off {
|
||||
if _, err := Update("off-key", Change{Key: id}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
secret, err := Update("rotate-key", Change{Key: id})
|
||||
if err != nil || secret == old || !strings.HasPrefix(secret, Prefix) {
|
||||
t.Fatal("key was not rotated", err)
|
||||
}
|
||||
if _, ok := Authenticate(old); ok {
|
||||
t.Fatal("old key still authenticates")
|
||||
}
|
||||
if who, ok := Authenticate(secret); ok == off || (!off && (who.KeyID != id || who.KeyName != "Desk")) {
|
||||
t.Fatal("rotation changed identity or enabled state", who, ok)
|
||||
}
|
||||
keys, _ = List()
|
||||
if keys[0].ID != id || keys[0].Name != "Desk" || keys[0].Off != off || keys[0].Secret != "" {
|
||||
t.Fatal(keys)
|
||||
}
|
||||
if _, ok := Authenticate(other); !ok {
|
||||
t.Fatal("rotation revoked another key")
|
||||
}
|
||||
old = secret
|
||||
}
|
||||
if _, err := Update("rotate-key", Change{Key: "missing"}); err == nil {
|
||||
t.Fatal("rotated a missing key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLANRotationKeepsOlderMagpieWorking(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
if err := ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := settings.Load()
|
||||
if before.LANKey == "" {
|
||||
t.Fatal("older Magpie has no sharing key")
|
||||
}
|
||||
next, err := Update("rotate-key", Change{Key: before.LANKeyID})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if settings.Load().LANKey != next {
|
||||
t.Fatal("rotation left the old version with a stale key")
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if keys, _ := List(); len(keys) != 1 || keys[0].ID != before.LANKeyID {
|
||||
t.Fatal("rotation duplicated the default key", keys)
|
||||
}
|
||||
if _, ok := Authenticate(before.LANKey); ok {
|
||||
t.Fatal("old key resurrected")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
package access
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
func TestLegacyLANFallbackBeforeMigration(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Settings{LAN: true, LANKey: "sk-magpie-old-client"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before, ok := Authenticate(s.LANKey)
|
||||
if !ok || before.KeyID == "" || before.KeyName != "Magpie" {
|
||||
t.Fatal("unmigrated legacy key lost access", before, ok)
|
||||
}
|
||||
if _, err := os.Stat(Path()); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatal("authentication must not create a key store", err)
|
||||
}
|
||||
for _, secret := range []string{"", "wrong", "magpie", s.LANKey + "x"} {
|
||||
if _, ok := Authenticate(secret); ok {
|
||||
t.Fatal("legacy fallback accepted another token")
|
||||
}
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after, ok := Authenticate(s.LANKey); !ok || after != before {
|
||||
t.Fatal("migration split the legacy caller's usage identity", before, after, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyLANFallbackDoesNotBypassRevocation(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
cfg settings.Settings
|
||||
keys []Key
|
||||
}{
|
||||
{"not-shared", settings.Settings{LANKey: "legacy"}, nil},
|
||||
{"completed", settings.Settings{LAN: true, LANKey: "legacy", LANKeyID: "removed"}, nil},
|
||||
{"revoked-mirror", settings.Settings{LAN: true, LANKey: revokedLANPrefix + "old"}, nil},
|
||||
{"disabled", settings.Settings{LAN: true, LANKey: "legacy"}, []Key{{ID: "default", LAN: true, Secret: "legacy", Off: true}}},
|
||||
{"rotated", settings.Settings{LAN: true, LANKey: "legacy"}, []Key{{ID: "default", LAN: true, Secret: "new"}}},
|
||||
{"disabled-matching-key", settings.Settings{LAN: true, LANKey: "legacy"}, []Key{{ID: "other", Secret: "legacy", Off: true}}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
if err := settings.Save(tc.cfg); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tc.keys != nil {
|
||||
if err := Restore(tc.keys); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if _, ok := Authenticate(tc.cfg.LANKey); ok {
|
||||
t.Fatal("fallback accepted a revoked or inactive legacy credential")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyLANFallbackWithReadOnlyDirectory(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Settings{LAN: true, LANKey: "sk-magpie-readonly"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Dir(), 0o500); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Dir(), 0o700) })
|
||||
if err := MigrateLegacyLANKey(); err == nil {
|
||||
t.Skip("configuration directory remains writable")
|
||||
} else if !errors.Is(err, os.ErrPermission) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for range 3 {
|
||||
if who, ok := Authenticate(s.LANKey); !ok || who.KeyID == "" || who.KeyName != "Magpie" {
|
||||
t.Fatal("failed key-store creation broke legacy access", who, ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyLANFallbackRefusesCorruptStore(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Settings{LAN: true, LANKey: "legacy"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(Path(), []byte("broken"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := Authenticate(s.LANKey); ok {
|
||||
t.Fatal("an unreadable revocation state must not enable legacy fallback")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package access
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
func TestDefaultKeyRevocationSurvivesDowngrade(t *testing.T) {
|
||||
for _, action := range []string{"off-key", "remove-key"} {
|
||||
t.Run(action, func(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
if err := ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := settings.Load()
|
||||
if _, err := Update(action, Change{Key: before.LANKeyID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after := settings.Load()
|
||||
if after.LANKey == "" || after.LANKey == before.LANKey {
|
||||
t.Fatal("older Magpie would accept the revoked credential")
|
||||
}
|
||||
if _, ok := Authenticate(after.LANKey); ok {
|
||||
t.Fatal("legacy replacement is an active gateway key")
|
||||
}
|
||||
// An older binary saves settings without the new migration marker.
|
||||
after.LANKeyID = ""
|
||||
if err := settings.Save(after); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, err := List()
|
||||
if err != nil || (action == "remove-key" && len(keys) != 0) || (action == "off-key" && (len(keys) != 1 || !keys[0].Off)) {
|
||||
t.Fatal("downgrade resurrected a key", keys, err)
|
||||
}
|
||||
if action == "off-key" {
|
||||
if _, err := Update("rotate-key", Change{Key: before.LANKeyID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
current, _ := Update("copy-key", Change{Key: before.LANKeyID})
|
||||
if settings.Load().LANKey == current {
|
||||
t.Fatal("rotation re-enabled the disabled legacy credential")
|
||||
}
|
||||
if _, err := Update("on-key", Change{Key: before.LANKeyID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if settings.Load().LANKey != current {
|
||||
t.Fatal("re-enabling did not restore the legacy mirror")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadOnlyMigrationDoesNotRetryCompletedKeyWrite(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
s := settings.Settings{LAN: true, LANKey: "sk-magpie-fixture-legacy"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Path(), 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Path(), 0o600) })
|
||||
if err := settings.Save(s); err == nil {
|
||||
t.Skip("settings.json remains writable")
|
||||
}
|
||||
if err := MigrateLegacyLANKey(); err == nil {
|
||||
t.Fatal("initial marker write unexpectedly succeeded")
|
||||
}
|
||||
for range 3 {
|
||||
if err := MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal("completed key migration retried read-only settings", err)
|
||||
}
|
||||
}
|
||||
if who, ok := Authenticate(s.LANKey); !ok || who.KeyName != "Magpie" {
|
||||
t.Fatal("legacy key was not usable after partial migration", who, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadOnlyDefaultKeyMutationExplainsLegacyMirror(t *testing.T) {
|
||||
for _, action := range []string{"rotate-key", "off-key", "remove-key"} {
|
||||
t.Run(action, func(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
if err := ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := settings.Load()
|
||||
independent, err := Update("add-key", Change{Name: "Laptop"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Path(), 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Path(), 0o600) })
|
||||
if err := settings.Save(s); err == nil {
|
||||
t.Skip("settings.json remains writable")
|
||||
}
|
||||
before, err := os.ReadFile(Path())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, err = Update(action, Change{Key: s.LANKeyID})
|
||||
if !errors.Is(err, os.ErrPermission) || !strings.Contains(err.Error(), "older Magpie") || !strings.Contains(err.Error(), "writable") || !strings.Contains(err.Error(), "unchanged") {
|
||||
t.Fatal("default-key error does not explain the recovery or downgrade risk", err)
|
||||
}
|
||||
after, err := os.ReadFile(Path())
|
||||
if err != nil || string(after) != string(before) {
|
||||
t.Fatal("failed legacy mirror write changed active keys", err)
|
||||
}
|
||||
if _, ok := Authenticate(s.LANKey); !ok {
|
||||
t.Fatal("operation reported failure but revoked the default key")
|
||||
}
|
||||
who, ok := Authenticate(independent)
|
||||
if !ok {
|
||||
t.Fatal("independent key is missing")
|
||||
}
|
||||
if _, err := Update(action, Change{Key: who.KeyID}); err != nil {
|
||||
t.Fatal("read-only settings blocked an independent key", err)
|
||||
}
|
||||
if _, ok := Authenticate(independent); ok {
|
||||
t.Fatal("independent key kept its old credential")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
+61
-13
@@ -28,6 +28,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/agent"
|
||||
"github.com/yetone/magpie/internal/edit"
|
||||
"github.com/yetone/magpie/internal/library"
|
||||
@@ -46,17 +47,18 @@ const (
|
||||
|
||||
// Bundle is what a backup holds.
|
||||
type Bundle struct {
|
||||
Version int `json:"version"`
|
||||
Created time.Time `json:"created"`
|
||||
App string `json:"app,omitempty"` // the magpie that made it
|
||||
Keys bool `json:"keys"` // whether the providers carry their keys
|
||||
Providers []provider.Provider `json:"providers"`
|
||||
Icons map[string][]byte `json:"icons,omitempty"` // pictures picked for providers, by file name
|
||||
Groups []provider.Group `json:"groups,omitempty"` // the user's model groups
|
||||
Settings *settings.Settings `json:"settings,omitempty"`
|
||||
Profiles map[string]profile.Profile `json:"profiles,omitempty"`
|
||||
Agents map[string]string `json:"agents,omitempty"` // every agent's fields as they are now
|
||||
Library *library.Bundle `json:"library,omitempty"` // nil from a magpie before it, or with none
|
||||
Version int `json:"version"`
|
||||
Created time.Time `json:"created"`
|
||||
App string `json:"app,omitempty"` // the magpie that made it
|
||||
Keys bool `json:"keys"` // whether credentials are included
|
||||
Providers []provider.Provider `json:"providers"`
|
||||
Icons map[string][]byte `json:"icons,omitempty"` // pictures picked for providers, by file name
|
||||
Groups []provider.Group `json:"groups,omitempty"` // the user's model groups
|
||||
Settings *settings.Settings `json:"settings,omitempty"`
|
||||
GatewayKeys *[]access.Key `json:"gatewayKeys,omitempty"`
|
||||
Profiles map[string]profile.Profile `json:"profiles,omitempty"`
|
||||
Agents map[string]string `json:"agents,omitempty"` // every agent's fields as they are now
|
||||
Library *library.Bundle `json:"library,omitempty"` // nil from a magpie before it, or with none
|
||||
// Searches are the web search APIs (#419), their keys with the
|
||||
// providers'; nil from a magpie before them.
|
||||
Searches *[]provider.SearchAPI `json:"searches,omitempty"`
|
||||
@@ -90,6 +92,18 @@ func Collect(keys bool, app string) (Bundle, error) {
|
||||
if b.Groups, err = provider.StoredGroups(); err != nil {
|
||||
return b, err
|
||||
}
|
||||
if keys {
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
if _, err := os.Stat(access.Path()); err == nil {
|
||||
gatewayKeys, err := access.Export()
|
||||
if err != nil {
|
||||
return b, err
|
||||
}
|
||||
b.GatewayKeys = &gatewayKeys
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return b, err
|
||||
}
|
||||
}
|
||||
for _, p := range stored {
|
||||
if !keys {
|
||||
p = withoutKeys(p)
|
||||
@@ -116,6 +130,9 @@ func Collect(keys bool, app string) (Bundle, error) {
|
||||
b.Searches = &searches
|
||||
if _, err := os.Stat(settings.Path()); err == nil {
|
||||
s := settings.Load()
|
||||
if !keys {
|
||||
s.LANKey, s.LANKeyID = "", ""
|
||||
}
|
||||
b.Settings = &s
|
||||
}
|
||||
ps, err := profile.Load()
|
||||
@@ -285,13 +302,44 @@ func Restore(b Bundle, parts Parts) (Result, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
if parts.Settings && b.Settings != nil {
|
||||
if parts.Settings && (b.Settings != nil || b.GatewayKeys != nil) {
|
||||
// the window's size and the proxy are this machine's own
|
||||
s, cur := *b.Settings, settings.Load()
|
||||
cur := settings.Load()
|
||||
s := cur
|
||||
if b.Settings != nil {
|
||||
s = *b.Settings
|
||||
}
|
||||
s.Window, s.Proxy, s.Dock, s.DockWindow = cur.Window, cur.Proxy, cur.Dock, cur.DockWindow
|
||||
if !b.Keys {
|
||||
s.LANKey, s.LANKeyID = cur.LANKey, cur.LANKeyID
|
||||
} else if b.GatewayKeys == nil {
|
||||
// An older backup may carry a marker without its named-key store.
|
||||
s.LANKeyID = ""
|
||||
keys, err := access.List()
|
||||
if err != nil {
|
||||
return r, err
|
||||
}
|
||||
if slices.ContainsFunc(keys, func(k access.Key) bool { return k.LAN }) {
|
||||
// Its missing store must not detach this machine's default key.
|
||||
s.LANKey, s.LANKeyID = cur.LANKey, cur.LANKeyID
|
||||
}
|
||||
}
|
||||
if err := settings.Save(s); err != nil {
|
||||
return r, err
|
||||
}
|
||||
if b.Keys && b.GatewayKeys != nil {
|
||||
if err := access.Restore(*b.GatewayKeys); err != nil {
|
||||
// Settings must be writable before replacing credentials. If the
|
||||
// store refuses the write, restore their original association.
|
||||
if rollback := settings.Save(cur); rollback != nil {
|
||||
return r, errors.Join(err, fmt.Errorf("could not restore original settings after gateway-key restore failed: %w", rollback))
|
||||
}
|
||||
return r, err
|
||||
}
|
||||
}
|
||||
if b.Keys && b.GatewayKeys == nil {
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
}
|
||||
r.Settings = true
|
||||
}
|
||||
if parts.Profiles && len(b.Profiles) > 0 {
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
func TestGatewayKeysBackupRoundTrip(t *testing.T) {
|
||||
home(t)
|
||||
if err := access.ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defaultKey := settings.Load()
|
||||
client, err := access.Update("add-key", access.Change{Name: "Laptop"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ := access.List()
|
||||
if _, err := access.Update("off-key", access.Change{Key: defaultKey.LANKeyID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
b, err := Collect(true, "test")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := Seal(b, "gateway passphrase")
|
||||
if err != nil || strings.Contains(string(data), client) || strings.Contains(string(data), defaultKey.LANKey) {
|
||||
t.Fatal("backup did not seal gateway credentials", err)
|
||||
}
|
||||
home(t)
|
||||
b, err = Open(data, "gateway passphrase")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Restore(b, Parts{Settings: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := access.List()
|
||||
if err != nil || len(got) != len(keys) || got[0].ID != defaultKey.LANKeyID || !got[0].Off || got[1].ID != keys[1].ID || got[1].Name != "Laptop" {
|
||||
t.Fatal("named key identity/state lost during restore", got, err)
|
||||
}
|
||||
if who, ok := access.Authenticate(client); !ok || who.KeyID != keys[1].ID {
|
||||
t.Fatal("restored client cannot authenticate", who, ok)
|
||||
}
|
||||
if _, ok := access.Authenticate(defaultKey.LANKey); ok {
|
||||
t.Fatal("restore resurrected the disabled default key")
|
||||
}
|
||||
if err := access.MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got, _ := access.List(); len(got) != len(keys) {
|
||||
t.Fatal("restore imported a duplicate default key", got)
|
||||
}
|
||||
if _, err := access.Update("on-key", access.Change{Key: defaultKey.LANKeyID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if settings.Load().LANKey != defaultKey.LANKey {
|
||||
t.Fatal("restored default key lost its legacy link")
|
||||
}
|
||||
info, err := os.Stat(access.Path())
|
||||
if err != nil || info.Mode().Perm() != 0o600 {
|
||||
t.Fatal("restored key-store permissions", info, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayKeysWithoutSecretsOrSettingsRestore(t *testing.T) {
|
||||
home(t)
|
||||
if err := access.ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
secret := settings.Load().LANKey
|
||||
full, err := Collect(true, "test")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
without, err := Collect(false, "test")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
plain, _ := json.Marshal(without)
|
||||
if strings.Contains(string(plain), secret) || without.Settings.LANKey != "" || without.Settings.LANKeyID != "" {
|
||||
t.Fatal("--no-keys backup contains a gateway credential")
|
||||
}
|
||||
home(t)
|
||||
if err := access.ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
current := settings.Load()
|
||||
if _, err := Restore(full, Parts{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Restore(without, Parts{Settings: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := settings.Load(); got.LANKey != current.LANKey || got.LANKeyID != current.LANKeyID {
|
||||
t.Fatal("keyless restore replaced this machine's LAN credential")
|
||||
}
|
||||
if _, ok := access.Authenticate(current.LANKey); !ok {
|
||||
t.Fatal("restore without gateway keys revoked this machine's key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOlderBackupRecoversLegacyGatewayCredential(t *testing.T) {
|
||||
home(t)
|
||||
b := Bundle{Version: 1, Keys: true, Settings: &settings.Settings{LAN: true, LANKey: "{{API_KEY_88kqppzx}}", LANKeyID: "id-from-incomplete-backup"}}
|
||||
if _, err := Restore(b, Parts{Settings: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if who, ok := access.Authenticate(b.Settings.LANKey); !ok || who.KeyName != "Magpie" {
|
||||
t.Fatal("old backup's migration marker lost its credential", who, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOlderBackupKeepsExistingGatewayKeyStore(t *testing.T) {
|
||||
home(t)
|
||||
if err := access.ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cur := settings.Load()
|
||||
b := Bundle{Version: 1, Keys: true, Settings: &settings.Settings{LAN: true, LANKey: "fixture-older-backup-key", LANKeyID: "old-marker"}}
|
||||
if _, err := Restore(b, Parts{Settings: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := settings.Load(); got.LANKey != cur.LANKey || got.LANKeyID != cur.LANKeyID {
|
||||
t.Fatal("backup without a key store detached the existing default key")
|
||||
}
|
||||
if _, ok := access.Authenticate(cur.LANKey); !ok {
|
||||
t.Fatal("backup without a key store revoked the existing gateway key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayKeysBackupWithoutSettingsFile(t *testing.T) {
|
||||
home(t)
|
||||
secret, err := access.Update("add-key", access.Change{Name: "CLI client"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
b, err := Collect(true, "test")
|
||||
if err != nil || b.Settings != nil {
|
||||
t.Fatal("CLI-only key fixture unexpectedly has settings", err)
|
||||
}
|
||||
home(t)
|
||||
if _, err := Restore(b, Parts{Settings: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if who, ok := access.Authenticate(secret); !ok || who.KeyName != "CLI client" {
|
||||
t.Fatal("standalone CLI key was not restored", who, ok)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayKeyRestoreFailureKeepsSettingsAndKeys(t *testing.T) {
|
||||
for _, failure := range []string{"invalid-settings", "readonly-settings", "readonly-store"} {
|
||||
t.Run(failure, func(t *testing.T) {
|
||||
home(t)
|
||||
if err := access.ConfigureLAN(true, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cur := settings.Load()
|
||||
before, err := os.ReadFile(access.Path())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
incoming := cur
|
||||
incoming.Theme, incoming.LANKeyID, incoming.LANKey = "dark", "restored", "sk-magpie-restored"
|
||||
keys := []access.Key{{ID: incoming.LANKeyID, Name: "Restored", Secret: incoming.LANKey, LAN: true}}
|
||||
if failure == "invalid-settings" {
|
||||
incoming.Theme = "invalid-theme"
|
||||
} else {
|
||||
path := settings.Path()
|
||||
if failure == "readonly-store" {
|
||||
path = access.Path()
|
||||
}
|
||||
if err := os.Chmod(path, 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(path, 0o600) })
|
||||
f, err := os.OpenFile(path, os.O_WRONLY, 0)
|
||||
if err == nil {
|
||||
f.Close()
|
||||
t.Skip("file remains writable")
|
||||
}
|
||||
if !errors.Is(err, os.ErrPermission) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
r, err := Restore(Bundle{Version: 1, Keys: true, Settings: &incoming, GatewayKeys: &keys}, Parts{Settings: true})
|
||||
if err == nil || r.Settings {
|
||||
t.Fatal("failed restore reported success", r, err)
|
||||
}
|
||||
if failure != "invalid-settings" && !errors.Is(err, os.ErrPermission) {
|
||||
t.Fatal("restore lost permission error", err)
|
||||
}
|
||||
after, err := os.ReadFile(access.Path())
|
||||
if err != nil || string(after) != string(before) || !reflect.DeepEqual(settings.Load(), cur) {
|
||||
t.Fatal("failed restore left settings and keys inconsistent", err)
|
||||
}
|
||||
if _, ok := access.Authenticate(cur.LANKey); !ok {
|
||||
t.Fatal("failed restore revoked the current key")
|
||||
}
|
||||
if _, ok := access.Authenticate(incoming.LANKey); ok {
|
||||
t.Fatal("failed restore activated the incoming key")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,306 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
"github.com/yetone/magpie/internal/usage"
|
||||
)
|
||||
|
||||
func TestLoopbackRemainsPermissive(t *testing.T) {
|
||||
fresh(t)
|
||||
keys, secrets := newCaller(t, "Desk")
|
||||
access.Update("off-key", access.Change{Key: keys[0].ID})
|
||||
for _, shared := range []bool{false, true} {
|
||||
if err := settings.Save(settings.Settings{LAN: shared}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, token := range []string{"", "anything", Token, TokenFor("alma"), "sk-magpie-stale", secrets[0]} {
|
||||
for _, addr := range []string{"127.0.0.1:5000", "[::1]:5000"} {
|
||||
r := httptest.NewRequest("GET", "/v1/models", nil)
|
||||
r.RemoteAddr = addr
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
w := httptest.NewRecorder()
|
||||
lanGuard(New().Handler()).ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Fatalf("shared=%v %s %q: %d", shared, addr, token, w.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExplicitOpenGatewayAcceptsAllRemoteTokens(t *testing.T) {
|
||||
fresh(t)
|
||||
t.Setenv("MAGPIE_ADDR", "0.0.0.0:3425")
|
||||
if _, err := access.Update("add-key", access.Change{Name: "Unused"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ := access.List()
|
||||
secret, _ := access.Update("copy-key", access.Change{Key: keys[0].ID})
|
||||
for _, token := range []string{"", "anything", "sk-magpie-stale", secret} {
|
||||
for _, header := range []string{"Authorization", "x-api-key", "query"} {
|
||||
r := httptest.NewRequest("GET", "/v1/models", nil)
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
if header == "query" {
|
||||
r.URL.RawQuery = "key=" + token
|
||||
} else if header == "Authorization" {
|
||||
r.Header.Set(header, "Bearer "+token)
|
||||
} else {
|
||||
r.Header.Set(header, token)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
lanGuard(New().Handler()).ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Fatalf("explicit open gateway rejected %s %q: %d", header, token, w.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationFailureDoesNotStopGateway(t *testing.T) {
|
||||
fresh(t)
|
||||
t.Setenv("MAGPIE_ADDR", "127.0.0.1:0")
|
||||
if err := settings.Save(settings.Settings{LAN: true, LANKey: "sk-magpie-old"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Path(), 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Path(), 0o600) })
|
||||
if err := settings.Save(settings.Load()); err == nil {
|
||||
t.Skip("settings.json remains writable on this platform")
|
||||
}
|
||||
var logs bytes.Buffer
|
||||
old := log.Writer()
|
||||
log.SetOutput(&logs)
|
||||
defer log.SetOutput(old)
|
||||
s := New()
|
||||
if err := s.Relisten(); err != nil {
|
||||
t.Fatal("migration stopped Relisten", err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(t.Context())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- s.ListenAndServe(ctx) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
if err := <-done; err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
if !strings.Contains(logs.String(), "migrat") {
|
||||
t.Error("migration failure was not logged")
|
||||
}
|
||||
}()
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for {
|
||||
s.lnMu.Lock()
|
||||
ln := s.ln
|
||||
s.lnMu.Unlock()
|
||||
if ln != nil {
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("gateway did not bind after failed migration")
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyCallerWithReadOnlyConfiguration(t *testing.T) {
|
||||
fresh(t)
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer upstream" || r.URL.Query().Get("key") == "sk-magpie-readonly" {
|
||||
t.Error("legacy caller credential leaked upstream")
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
io.WriteString(w, `{"choices":[{"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":30,"completion_tokens":5}}`)
|
||||
}))
|
||||
t.Cleanup(up.Close)
|
||||
if err := provider.Save(provider.Provider{ID: "plan", Name: "Plan", Key: "upstream", Chat: up.URL + "/v1", Models: []string{"m1"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := settings.Settings{LAN: true, LANKey: "sk-magpie-readonly"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Existing logs remain appendable even if no new store can be created.
|
||||
if err := os.WriteFile(usage.Path(), nil, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Dir(), 0o500); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Dir(), 0o700) })
|
||||
if err := migrateLANKey(); err == nil {
|
||||
t.Skip("configuration directory remains writable")
|
||||
} else if !errors.Is(err, os.ErrPermission) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h := lanGuard(New().Handler())
|
||||
call := func(secret, header string) int {
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(chatReq))
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
switch header {
|
||||
case "Authorization":
|
||||
r.Header.Set(header, "Bearer "+secret)
|
||||
case "query":
|
||||
r.URL.RawQuery = "key=" + secret
|
||||
default:
|
||||
r.Header.Set(header, secret)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
return w.Code
|
||||
}
|
||||
for _, header := range []string{"Authorization", "x-api-key", "query"} {
|
||||
if code := call(s.LANKey, header); code != http.StatusOK {
|
||||
t.Fatal("unmigrated legacy client lost access", header, code)
|
||||
}
|
||||
for _, secret := range []string{"", "wrong", Token} {
|
||||
if code := call(secret, header); code != http.StatusUnauthorized {
|
||||
t.Fatal("fallback accepted another credential", header, code)
|
||||
}
|
||||
}
|
||||
}
|
||||
recs := usage.Load(time.Time{})
|
||||
if len(recs) != 3 || recs[0].CallerKeyID == "" {
|
||||
t.Fatal("fallback attribution missing", recs)
|
||||
}
|
||||
for _, rec := range recs {
|
||||
if rec.CallerKeyID != recs[0].CallerKeyID || rec.CallerKeyName != "Magpie" || rec.ProviderKeyID != provider.KeyID("upstream") {
|
||||
t.Fatal("fallback attribution changed", rec)
|
||||
}
|
||||
}
|
||||
if _, err := os.Stat(access.Path()); !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatal("fallback wrote a key store", err)
|
||||
}
|
||||
log, err := os.ReadFile(usage.Path())
|
||||
if err != nil || strings.Contains(string(log), s.LANKey) {
|
||||
t.Fatal("legacy credential in usage log", err)
|
||||
}
|
||||
if err := os.Chmod(settings.Dir(), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := migrateLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code := call(s.LANKey, "Authorization"); code != 200 || lastUsage(t).CallerKeyID != recs[0].CallerKeyID {
|
||||
t.Fatal("migration changed the fallback identity", code)
|
||||
}
|
||||
if _, err := access.Update("remove-key", access.Change{Key: recs[0].CallerKeyID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code := call(s.LANKey, "Authorization"); code != 401 {
|
||||
t.Fatal("fallback resurrected a removed legacy key", code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallerUsageSystemOne(t *testing.T) {
|
||||
fresh(t)
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer upstream" {
|
||||
t.Error("caller key reached System One upstream")
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
io.WriteString(w, `{"model":"jev","usage":{"input_tokens":120,"output_tokens":2}}`)
|
||||
}))
|
||||
defer up.Close()
|
||||
if err := provider.Save(provider.Provider{ID: "jev", Name: "Jev", Key: "upstream", Decide: up.URL + "/v1"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, secrets := newCaller(t, "System One client")
|
||||
r := httptest.NewRequest("POST", "/v1/systemone", strings.NewReader(`{"model":"jev/jev-latest","state":{"message":"hi"},"questions":{}}`))
|
||||
r.Header.Set("Authorization", "Bearer "+secrets[0])
|
||||
w := httptest.NewRecorder()
|
||||
s := New()
|
||||
s.Handler().ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Fatal(w.Code, w.Body)
|
||||
}
|
||||
if rec := lastUsage(t); rec.RouteID == 0 || rec.RouteID != lastRoute(s).ID || rec.CallerKeyID != keys[0].ID || rec.CallerKeyName != "System One client" || rec.ProviderKeyID != provider.KeyID("upstream") || rec.Input != 120 {
|
||||
t.Fatal(rec)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallerUsageCodexOwnModel(t *testing.T) {
|
||||
setup(t, provider.Chat, &fake{t: t})
|
||||
chatgpt(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer chatgpt-token" {
|
||||
t.Error("Codex's own sign-in was changed")
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
io.WriteString(w, sse(`data: {"type":"response.completed","response":{"id":"r1","usage":{"input_tokens":9,"output_tokens":2}}}`))
|
||||
})
|
||||
r := httptest.NewRequest("POST", CodexPath+"/responses", strings.NewReader(`{"model":"gpt-5.5","stream":true,"input":"hi"}`))
|
||||
r.Header.Set("Authorization", "Bearer chatgpt-token")
|
||||
r = r.WithContext(access.WithIdentity(r.Context(), access.Identity{KeyID: "desk", KeyName: "Desk"}))
|
||||
w := httptest.NewRecorder()
|
||||
s := New()
|
||||
s.Handler().ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Fatal(w.Code, w.Body)
|
||||
}
|
||||
if rec := lastUsage(t); rec.RouteID == 0 || rec.RouteID != lastRoute(s).ID || rec.CallerKeyID != "desk" || rec.CallerKeyName != "Desk" || rec.Input != 9 || rec.Output != 2 {
|
||||
t.Fatal(rec)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConcurrentCallerAttribution(t *testing.T) {
|
||||
fresh(t)
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer upstream" {
|
||||
t.Error("caller credential leaked upstream")
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
io.WriteString(w, `{"choices":[{"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":30,"completion_tokens":5}}`)
|
||||
}))
|
||||
defer up.Close()
|
||||
if err := provider.Save(provider.Provider{ID: "plan", Name: "Plan", Key: "upstream", Chat: up.URL + "/v1", Models: []string{"m1"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, secrets := newCaller(t, "Desk", "Server")
|
||||
if err := settings.Save(settings.Settings{LAN: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h := lanGuard(New().Handler())
|
||||
var wg sync.WaitGroup
|
||||
for i := range 120 {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(chatReq))
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
r.Header.Set("Authorization", "Bearer "+secrets[i%len(secrets)])
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Error(w.Code, w.Body)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
counts := map[string]int{}
|
||||
for _, rec := range usage.Load(time.Time{}) {
|
||||
counts[rec.CallerKeyID]++
|
||||
if rec.CallerKeyName != keys[0].Name && rec.CallerKeyName != keys[1].Name {
|
||||
t.Error("lost caller name", rec)
|
||||
}
|
||||
}
|
||||
if len(counts) != 2 || counts[keys[0].ID] != 60 || counts[keys[1].ID] != 60 {
|
||||
t.Fatal("concurrent calls mixed identities", counts)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,323 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
"github.com/yetone/magpie/internal/usage"
|
||||
)
|
||||
|
||||
func newCaller(t *testing.T, names ...string) ([]access.Key, []string) {
|
||||
t.Helper()
|
||||
s := settings.Load()
|
||||
s.LAN = true
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var secrets []string
|
||||
for _, name := range names {
|
||||
secret, err := access.Update("add-key", access.Change{Name: name})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
secrets = append(secrets, secret)
|
||||
}
|
||||
keys, err := access.List()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return keys[len(keys)-len(names):], secrets
|
||||
}
|
||||
|
||||
func TestCallerUsageAcrossKeys(t *testing.T) {
|
||||
fresh(t)
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer upstream-secret" {
|
||||
t.Error("caller credential reached provider")
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("X-Request-Id", "req-caller")
|
||||
io.WriteString(w, `{"choices":[{"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":30,"completion_tokens":5}}`)
|
||||
}))
|
||||
defer up.Close()
|
||||
if err := provider.Save(provider.Provider{ID: "plan", Name: "Plan", Key: "upstream-secret", Chat: up.URL + "/v1", Models: []string{"m1"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, secrets := newCaller(t, "Laptop", "Server", "Work")
|
||||
h := New().Handler()
|
||||
call := func(secret, header string) int {
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(chatReq))
|
||||
if header == "Authorization" {
|
||||
secret = "Bearer " + secret
|
||||
}
|
||||
r.Header.Set(header, secret)
|
||||
r.Header.Set(SessionHeader, "override-session")
|
||||
r.Header.Set("X-Session-Id", "native-session")
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
return w.Code
|
||||
}
|
||||
for i, secret := range secrets {
|
||||
header := []string{"Authorization", "x-api-key", "x-goog-api-key"}[i]
|
||||
if code := call(secret, header); code != 200 {
|
||||
t.Fatal("request", code)
|
||||
}
|
||||
}
|
||||
recs := usage.Load(time.Time{})
|
||||
if len(recs) != 3 || recs[0].CallerKeyID != keys[0].ID || recs[1].CallerKeyID != keys[1].ID || recs[2].CallerKeyName != "Work" {
|
||||
t.Fatalf("attribution: %+v", recs)
|
||||
}
|
||||
for _, rec := range recs {
|
||||
if rec.Input != 30 || rec.Output != 5 || rec.RequestID != "req-caller" || rec.Endpoint != "/v1/chat/completions" || rec.Session != "override-session" || rec.NativeSession != "native-session" {
|
||||
t.Fatal(rec)
|
||||
}
|
||||
}
|
||||
s := usage.Summarize(usage.All)
|
||||
if len(s.CallerKeys) != 3 || s.Calls != 3 {
|
||||
t.Fatalf("summary: %+v", s)
|
||||
}
|
||||
rows, totals, _ := usage.Ledger(usage.All, usage.Filter{CallerKey: keys[1].ID})
|
||||
if len(rows) != 1 || totals.Input != 30 || rows[0].CallerKeyName != "Server" {
|
||||
t.Fatal(rows, totals)
|
||||
}
|
||||
access.Update("off-key", access.Change{Key: keys[0].ID})
|
||||
if code := call(secrets[0], "Authorization"); code != 401 {
|
||||
t.Fatal("disabled key", code)
|
||||
}
|
||||
if code := call(secrets[2], "Authorization"); code != 200 {
|
||||
t.Fatal("other key", code)
|
||||
}
|
||||
access.Update("remove-key", access.Change{Key: keys[2].ID})
|
||||
if code := call(secrets[2], "Authorization"); code != 401 {
|
||||
t.Fatal("deleted key", code)
|
||||
}
|
||||
log, _ := os.ReadFile(usage.Path())
|
||||
for _, secret := range secrets {
|
||||
if strings.Contains(string(log), secret) {
|
||||
t.Fatal("caller credential in ledger")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectedRequestRetainsCallerIdentity(t *testing.T) {
|
||||
fresh(t)
|
||||
keys, secrets := newCaller(t, "Rejected client")
|
||||
r := httptest.NewRequest("POST", "/v1/messages", strings.NewReader(`{"model":"nothing/here","max_tokens":10,"messages":[{"role":"user","content":"hi"}]}`))
|
||||
r.Header.Set("Authorization", "Bearer "+secrets[0])
|
||||
w := httptest.NewRecorder()
|
||||
New().Handler().ServeHTTP(w, r)
|
||||
if w.Code != http.StatusNotFound {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
rec := lastUsage(t)
|
||||
if !rec.Rejected || rec.CallerKeyID != keys[0].ID || rec.CallerKeyName != "Rejected client" || rec.Endpoint != "/v1/messages" || rec.Error != "unknown model" || rec.ProviderKeyID != "" {
|
||||
t.Fatalf("rejected request lost caller or failure metadata: %+v", rec)
|
||||
}
|
||||
page := usage.QueryPage(usage.All, usage.Filter{CallerKey: keys[0].ID}, 0, 100)
|
||||
if page.Total != 1 || page.Sum.Calls != 0 || len(page.CallerKeys) != 0 || usage.Summarize(usage.All).Calls != 0 {
|
||||
t.Fatalf("local rejection must stay visible without inflating usage: %+v", page)
|
||||
}
|
||||
log, err := os.ReadFile(usage.Path())
|
||||
if err != nil || strings.Contains(string(log), secrets[0]) {
|
||||
t.Fatal("caller credential in rejected request log", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestManagedLANGuard(t *testing.T) {
|
||||
fresh(t)
|
||||
keys, secrets := newCaller(t, "Remote")
|
||||
shared := settings.Load()
|
||||
shared.LAN = true
|
||||
if err := settings.Save(shared); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h := lanGuard(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
who := access.Caller(r.Context())
|
||||
if who.KeyID != keys[0].ID || r.URL.Query().Get("key") != Token {
|
||||
t.Fatal("query key or user not normalized", who)
|
||||
}
|
||||
}))
|
||||
r := httptest.NewRequest("POST", "/v1beta/models/m:generateContent?key="+secrets[0], nil)
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
access.Update("off-key", access.Change{Key: keys[0].ID})
|
||||
w = httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
if w.Code != 401 {
|
||||
t.Fatal("disabled remote key", w.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigratedLANKeyUsageAndRevocation(t *testing.T) {
|
||||
fresh(t)
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer upstream-secret" {
|
||||
t.Error("caller credential reached provider")
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
io.WriteString(w, `{"choices":[{"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":30,"completion_tokens":5}}`)
|
||||
}))
|
||||
defer up.Close()
|
||||
if err := provider.Save(provider.Provider{ID: "plan", Name: "Plan", Key: "upstream-secret", Chat: up.URL + "/v1", Models: []string{"m1"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := settings.Load()
|
||||
s.LAN, s.LANKey = true, "sk-magpie-test-legacy-lan"
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := access.MigrateLegacyLANKey(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, err := access.List()
|
||||
if err != nil || len(keys) != 1 {
|
||||
t.Fatal(keys, err)
|
||||
}
|
||||
h := lanGuard(New().Handler())
|
||||
call := func() int {
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(chatReq))
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
r.Header.Set("Authorization", "Bearer "+s.LANKey)
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
return w.Code
|
||||
}
|
||||
if code := call(); code != http.StatusOK {
|
||||
t.Fatal("migrated remote caller", code)
|
||||
}
|
||||
rec := lastUsage(t)
|
||||
if rec.CallerKeyID != keys[0].ID || rec.CallerKeyName != keys[0].Name || rec.Input != 30 || rec.Output != 5 {
|
||||
t.Fatal("remote usage attribution", rec)
|
||||
}
|
||||
if err := access.ConfigureLAN(true, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code := call(); code != http.StatusUnauthorized {
|
||||
t.Fatal("rotated LAN credential still works", code)
|
||||
}
|
||||
s.LANKey, err = access.Update("copy-key", access.Change{Key: keys[0].ID})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code := call(); code != http.StatusOK || lastUsage(t).CallerKeyID != keys[0].ID {
|
||||
t.Fatal("rotation changed remote usage identity", code)
|
||||
}
|
||||
if _, err := access.Update("remove-key", access.Change{Key: keys[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code := call(); code != http.StatusUnauthorized {
|
||||
t.Fatal("revoked old LAN credential still works", code)
|
||||
}
|
||||
if recs := usage.Load(time.Time{}); len(recs) != 2 {
|
||||
t.Fatal("unauthorized request counted as usage", recs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestImageUsageIncludesCaller(t *testing.T) {
|
||||
s, _ := easeled(t)
|
||||
keys, secrets := newCaller(t, "Drawing")
|
||||
r := httptest.NewRequest("POST", "/v1/images/generations", strings.NewReader(`{"model":"art/gpt-image-1","prompt":"a bird"}`))
|
||||
r.Header.Set("Authorization", "Bearer "+secrets[0])
|
||||
w := httptest.NewRecorder()
|
||||
s.Handler().ServeHTTP(w, r)
|
||||
if w.Code != 200 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
rec := lastUsage(t)
|
||||
if rec.CallerKeyID != keys[0].ID || rec.ProviderKeyID != provider.KeyID("key") || rec.Input != 7 {
|
||||
t.Fatal(rec)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallerIdentitySurvivesStreamingFailover(t *testing.T) {
|
||||
fresh(t)
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") == "Bearer first" {
|
||||
w.WriteHeader(429)
|
||||
io.WriteString(w, `{"error":{"message":"rate limited"}}`)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
io.WriteString(w, "data: "+`{"choices":[{"index":0,"delta":{"content":"ok"},"finish_reason":null}]}`+"\n\n")
|
||||
io.WriteString(w, "data: "+`{"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":100,"completion_tokens":20}}`+"\n\ndata: [DONE]\n\n")
|
||||
}))
|
||||
defer up.Close()
|
||||
p := provider.Provider{ID: "plan", Name: "Plan", Key: "first", Chat: up.URL + "/v1", Models: []string{"m1"},
|
||||
Keys: []provider.KeyAccount{{Key: "backup", Name: "Backup"}}}
|
||||
if err := provider.Save(p); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, secrets := newCaller(t, "Stream")
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(`{"model":"plan/m1","messages":[{"role":"user","content":"hi"}],"stream":true}`))
|
||||
r.Header.Set("Authorization", "Bearer "+secrets[0])
|
||||
w := httptest.NewRecorder()
|
||||
s := New()
|
||||
s.Handler().ServeHTTP(w, r)
|
||||
if w.Code != 200 || !strings.Contains(w.Body.String(), "[DONE]") {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
rec := lastUsage(t)
|
||||
if rec.RouteID == 0 || rec.RouteID != lastRoute(s).ID || rec.CallerKeyID != keys[0].ID || rec.CallerKeyName != "Stream" || rec.ProviderKeyID != provider.KeyID("backup") || rec.ProviderKeyName != "Backup" || rec.Input != 100 || rec.Output != 20 {
|
||||
t.Fatal(rec)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallerUsageWithRequestArchive(t *testing.T) {
|
||||
for _, header := range []string{"Authorization", "x-api-key", "query"} {
|
||||
t.Run(header, func(t *testing.T) {
|
||||
fresh(t)
|
||||
serveOn(t, "fake", "provider-secret", []string{"m1"}, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer provider-secret" {
|
||||
t.Error("caller credential reached archived request upstream")
|
||||
}
|
||||
archiveVendor{}.ServeHTTP(w, r)
|
||||
}))
|
||||
keys, secrets := newCaller(t, "Archived client")
|
||||
shared := settings.Load()
|
||||
shared.RequestArchive = true
|
||||
if err := settings.Save(shared); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
bucket := &memBucket{objs: map[string][]byte{}}
|
||||
archiveTo(t, bucket)
|
||||
r := httptest.NewRequest("POST", "/v1/chat/completions", strings.NewReader(`{"model":"fake/m1","messages":[{"role":"user","content":"hi"}]}`))
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
switch header {
|
||||
case "Authorization":
|
||||
r.Header.Set(header, "Bearer "+secrets[0])
|
||||
case "query":
|
||||
r.URL.RawQuery = "key=" + secrets[0]
|
||||
default:
|
||||
r.Header.Set(header, secrets[0])
|
||||
}
|
||||
s := New()
|
||||
w := httptest.NewRecorder()
|
||||
lanGuard(s.Handler()).ServeHTTP(w, r)
|
||||
archivePending.Wait()
|
||||
if w.Code != 200 || w.Header().Get(ArchiveHeader) == "" || len(bucket.objs) != 1 {
|
||||
t.Fatal("named caller request was not archived", w.Code, w.Body)
|
||||
}
|
||||
for _, data := range bucket.objs {
|
||||
if strings.Contains(string(data), secrets[0]) || strings.Contains(string(data), "provider-secret") {
|
||||
t.Fatal("credential leaked into request archive")
|
||||
}
|
||||
}
|
||||
rec := lastUsage(t)
|
||||
if rec.RouteID == 0 || rec.RouteID != lastRoute(s).ID || rec.CallerKeyID != keys[0].ID || rec.CallerKeyName != "Archived client" || rec.ProviderKeyID != provider.KeyID("provider-secret") || rec.Input != 3 {
|
||||
t.Fatal("archive lost usage attribution", rec)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/usage"
|
||||
)
|
||||
|
||||
func TestCallerUsageVideos(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
prompt string
|
||||
status int
|
||||
}{
|
||||
{"a fox", 200},
|
||||
{"TOOLONG", 400},
|
||||
{"NOID", 502},
|
||||
} {
|
||||
for _, header := range []string{"Authorization", "x-api-key", "query"} {
|
||||
t.Run(tc.prompt+"/"+header, func(t *testing.T) {
|
||||
grokSignedIn(t)
|
||||
newGrokMedia(t)
|
||||
keys, secrets := newCaller(t, "Video client")
|
||||
r := httptest.NewRequest("POST", "/v1/videos", strings.NewReader(`{"model":"grok/grok-imagine-video","prompt":"`+tc.prompt+`"}`))
|
||||
r.RemoteAddr = "192.168.1.9:5000"
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Magpie-Session", "video-session")
|
||||
if header == "query" {
|
||||
r.URL.RawQuery = "key=" + secrets[0]
|
||||
} else if header == "Authorization" {
|
||||
r.Header.Set(header, "Bearer "+secrets[0])
|
||||
} else {
|
||||
r.Header.Set(header, secrets[0])
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
lanGuard(New().Handler()).ServeHTTP(w, r)
|
||||
if w.Code != tc.status {
|
||||
t.Fatal(w.Code, w.Body)
|
||||
}
|
||||
rec := lastUsage(t)
|
||||
if rec.CallerKeyID != keys[0].ID || rec.CallerKeyName != "Video client" || rec.Provider != "grok" || rec.Model != "grok-imagine-video" || rec.Status != tc.status || rec.Session != "video-session" {
|
||||
t.Fatalf("video caller attribution: %+v", rec)
|
||||
}
|
||||
rows, _, _ := usage.Ledger(usage.All, usage.Filter{CallerKey: keys[0].ID})
|
||||
if len(rows) != 1 {
|
||||
t.Fatal("video missing from caller-key ledger", rows)
|
||||
}
|
||||
log, err := os.ReadFile(usage.Path())
|
||||
if err != nil || strings.Contains(string(log), secrets[0]) {
|
||||
t.Fatal("video usage leaked the caller credential", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -403,7 +403,7 @@ func (s *Server) codexUpstream(w http.ResponseWriter, r *http.Request, rest stri
|
||||
Reasoning: uu.Reasoning, Millis: call.Millis, TTFT: call.TTFT, FirstText: call.FirstText, Status: call.Status, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind,
|
||||
RequestID: requestID(res.Header), Endpoint: r.URL.Path}
|
||||
failedWith(&rec, call.Status, call.Error, errType)
|
||||
usage.Append(rec)
|
||||
appendUsage(r, rec)
|
||||
}
|
||||
|
||||
// unreadableItem is the item OpenAI's refusal names: sealed content it
|
||||
|
||||
@@ -465,7 +465,7 @@ func (s *Server) serveSystemOne(w http.ResponseWriter, r *http.Request) {
|
||||
if p.Account == nil && p.Key != "" {
|
||||
keyID, keyName = provider.KeyID(p.Key), p.KeyName
|
||||
}
|
||||
usage.Append(usage.Record{RouteID: tr.ID, Time: start, Agent: agentOf(r), Provider: p.ID, Host: p.Where(), Model: model, Requested: asked, Served: use.Model,
|
||||
appendUsage(r, usage.Record{RouteID: tr.ID, Time: start, Agent: agentOf(r), Provider: p.ID, Host: p.Where(), Model: model, Requested: asked, Served: use.Model,
|
||||
ProviderKeyID: keyID, ProviderKeyName: keyName,
|
||||
Input: use.Usage.Input, Output: use.Usage.Output, Millis: time.Since(start).Milliseconds(), Status: status})
|
||||
end(status, errMsg, tokens)
|
||||
|
||||
@@ -243,7 +243,7 @@ func (s *Server) images(edit bool) http.HandlerFunc {
|
||||
if p.Account == nil && p.Key != "" {
|
||||
providerKeyID, providerKeyName = provider.KeyID(p.Key), p.KeyName
|
||||
}
|
||||
usage.Append(usage.Record{Time: start, Agent: call.Agent, Via: call.Via, Provider: p.ID, Host: p.Where(), Model: model, Requested: call.Model, ProviderKeyID: providerKeyID, ProviderKeyName: providerKeyName,
|
||||
appendUsage(r, usage.Record{Time: start, Agent: call.Agent, Via: call.Via, Provider: p.ID, Host: p.Where(), Model: model, Requested: call.Model, ProviderKeyID: providerKeyID, ProviderKeyName: providerKeyName,
|
||||
Input: out.Input, Output: out.Output, Millis: call.Millis, Status: call.Status, Session: sessionOf(r.Header)})
|
||||
if err != nil {
|
||||
call.Error = err.Error()
|
||||
|
||||
@@ -294,7 +294,7 @@ var WhileServing []func(context.Context)
|
||||
// long as a stream takes. A bind error means another magpie is already
|
||||
// serving, which is fine for the caller to ignore.
|
||||
func (s *Server) ListenAndServe(ctx context.Context) error {
|
||||
loadLANKey()
|
||||
migrateLANKeyBestEffort()
|
||||
ln, err := Listen(listenAddr())
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -392,7 +392,7 @@ func (s *Server) Handler() http.Handler {
|
||||
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, provider.Chat, http.StatusNotFound, "magpie serves /v1/chat/completions, /v1/responses, /v1/messages, /v1/systemone, /v1/images/generations, /v1/images/edits, /v1/videos and /v1beta/models/*")
|
||||
})
|
||||
return withCaller(mux)
|
||||
return callerGuard(withCaller(mux))
|
||||
}
|
||||
|
||||
func (s *Server) info(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -818,7 +818,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
|
||||
rec := usage.Record{Time: start, Agent: call.Agent, Via: call.Via, Provider: call.Provider, Model: call.Model, Requested: call.Model,
|
||||
Millis: call.Millis, Status: call.Status, Rejected: true, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind, Endpoint: endpointOf(r, from, "")}
|
||||
failedWith(&rec, call.Status, call.Error, "")
|
||||
usage.Append(rec)
|
||||
appendUsage(r, rec)
|
||||
}
|
||||
// a model's id without a provider in it that names a routing group is
|
||||
// the group's, as "group/<id>" is, rather than one provider's that
|
||||
@@ -1225,7 +1225,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
|
||||
TTFT: try.TTFT, FirstText: try.FirstText, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind,
|
||||
RequestID: call.Usage.RequestID, Endpoint: endpointOf(r, from, call.To)}
|
||||
failedWith(&rec, call.Status, call.Error, call.Usage.ErrType)
|
||||
usage.Append(rec)
|
||||
appendUsage(r, rec)
|
||||
}
|
||||
continue
|
||||
}
|
||||
@@ -1383,7 +1383,7 @@ func (s *Server) serve(w http.ResponseWriter, r *http.Request, from provider.Pro
|
||||
TTFT: call.TTFT, FirstText: call.FirstText, Session: sessionOf(r.Header), NativeSession: nativeSessionOf(r.Header), Kind: call.Kind,
|
||||
RequestID: call.Usage.RequestID, Endpoint: endpointOf(r, from, call.To)}
|
||||
failedWith(&rec, call.Status, call.Error, call.Usage.ErrType)
|
||||
usage.Append(rec)
|
||||
appendUsage(r, rec)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+70
-63
@@ -2,34 +2,26 @@ package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/subtle"
|
||||
"encoding/hex"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
"github.com/yetone/magpie/internal/usage"
|
||||
)
|
||||
|
||||
// The gateway listens on loopback and takes any token, which is safe only
|
||||
// because nothing but this computer reaches it. Shared on the local network
|
||||
// (settings' LAN), it listens on every interface, and a request from
|
||||
// another machine must carry the key magpie made for it, as its API key;
|
||||
// this computer's agents go on as before.
|
||||
|
||||
// lanKey is the key a request from another machine must carry, "" while
|
||||
// the gateway isn't shared.
|
||||
var lanKey atomic.Pointer[string]
|
||||
// The gateway accepts named caller keys locally and, while shared, remotely.
|
||||
// Loopback clients may still use any token, including a stale named key.
|
||||
|
||||
// listenAddr is where the gateway listens: every interface while it is
|
||||
// shared, on its port, else its address.
|
||||
func listenAddr() string {
|
||||
if s := settings.Load(); s.LAN && s.LANKey != "" {
|
||||
if s := settings.Load(); s.LAN {
|
||||
return "0.0.0.0:" + Port()
|
||||
}
|
||||
return Addr()
|
||||
@@ -44,19 +36,10 @@ func Port() string {
|
||||
return p
|
||||
}
|
||||
|
||||
func loadLANKey() {
|
||||
k := ""
|
||||
if s := settings.Load(); s.LAN {
|
||||
k = s.LANKey
|
||||
}
|
||||
lanKey.Store(&k)
|
||||
}
|
||||
func migrateLANKey() error { return access.MigrateLegacyLANKey() }
|
||||
|
||||
// NewLANKey makes a key for sharing the gateway.
|
||||
func NewLANKey() string {
|
||||
b := make([]byte, 20)
|
||||
rand.Read(b)
|
||||
return "sk-magpie-" + hex.EncodeToString(b)
|
||||
func migrateLANKeyBestEffort() {
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
}
|
||||
|
||||
// publicURL is MAGPIE_PUBLIC_URL, the base URL other machines are told to
|
||||
@@ -130,10 +113,9 @@ func LANURLs() []string {
|
||||
}
|
||||
|
||||
// Relisten moves the gateway to where settings now say it listens — onto
|
||||
// the network or back to loopback — and takes up the key; requests in
|
||||
// flight finish.
|
||||
// the network or back to loopback. Requests in flight finish.
|
||||
func (s *Server) Relisten() error {
|
||||
loadLANKey()
|
||||
migrateLANKeyBestEffort()
|
||||
s.lnMu.Lock()
|
||||
defer s.lnMu.Unlock()
|
||||
if s.ln == nil {
|
||||
@@ -158,50 +140,75 @@ func (s *Server) Relisten() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// lanGuard lets a request from another machine through only with the key,
|
||||
// and hands it on as though it came from here, with the gateway's token.
|
||||
// A gateway put on the network with MAGPIE_ADDR and not shared from the
|
||||
// Settings page is open, as it has always been.
|
||||
// lanGuard requires a named caller key on the local network. An explicit
|
||||
// MAGPIE_ADDR without LAN sharing retains its existing open-gateway behavior.
|
||||
func lanGuard(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if local(r) {
|
||||
next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
key := ""
|
||||
if k := lanKey.Load(); k != nil {
|
||||
key = *k
|
||||
}
|
||||
if key == "" {
|
||||
if os.Getenv("MAGPIE_ADDR") != "" {
|
||||
next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
remote := !local(r)
|
||||
shared := settings.Load().LAN
|
||||
if remote && !shared && os.Getenv("MAGPIE_ADDR") == "" {
|
||||
http.Error(w, "magpie isn't shared on the local network", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
if subtle.ConstantTimeCompare([]byte(callerKey(r)), []byte(key)) != 1 {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
w.Write([]byte(`{"error":{"type":"authentication_error","message":"use the API key shown in magpie's Settings, under Share on local network"}}`))
|
||||
return
|
||||
}
|
||||
r = r.WithContext(context.WithValue(r.Context(), lanKeyed{}, true))
|
||||
r.Header.Set("Authorization", "Bearer "+Token)
|
||||
if r.Header.Get("x-api-key") != "" {
|
||||
r.Header.Set("x-api-key", Token)
|
||||
}
|
||||
if r.Header.Get("x-goog-api-key") != "" {
|
||||
r.Header.Set("x-goog-api-key", Token)
|
||||
}
|
||||
if q := r.URL.Query(); q.Get("key") != "" {
|
||||
q.Set("key", Token)
|
||||
r.URL.RawQuery = q.Encode()
|
||||
if (remote && shared) || (!remote && access.Managed(callerKey(r))) {
|
||||
var ok bool
|
||||
r, ok = identifyCaller(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if remote && shared {
|
||||
r = r.WithContext(context.WithValue(r.Context(), lanKeyed{}, true))
|
||||
}
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// callerGuard also covers embedded handlers used by the web app and tests.
|
||||
func callerGuard(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if access.Caller(r.Context()).KeyID == "" && access.Managed(callerKey(r)) && (local(r) || settings.Load().LAN) {
|
||||
var ok bool
|
||||
r, ok = identifyCaller(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
func identifyCaller(w http.ResponseWriter, r *http.Request) (*http.Request, bool) {
|
||||
who, ok := access.Authenticate(callerKey(r))
|
||||
if !ok && !local(r) {
|
||||
writeError(w, provider.Chat, http.StatusUnauthorized, "API key is disabled, removed or invalid")
|
||||
return r, false
|
||||
}
|
||||
if ok {
|
||||
r = r.WithContext(access.WithIdentity(r.Context(), who))
|
||||
}
|
||||
r.Header = r.Header.Clone()
|
||||
r.Header.Set("Authorization", "Bearer "+Token)
|
||||
for _, h := range []string{"x-api-key", "x-goog-api-key"} {
|
||||
if r.Header.Get(h) != "" {
|
||||
r.Header.Set(h, Token)
|
||||
}
|
||||
}
|
||||
if q := r.URL.Query(); q.Get("key") != "" {
|
||||
q.Set("key", Token)
|
||||
u := *r.URL
|
||||
u.RawQuery = q.Encode()
|
||||
r.URL = &u
|
||||
}
|
||||
return r, true
|
||||
}
|
||||
|
||||
func appendUsage(r *http.Request, rec usage.Record) {
|
||||
who := access.Caller(r.Context())
|
||||
rec.CallerKeyID, rec.CallerKeyName = who.KeyID, who.KeyName
|
||||
usage.Append(rec)
|
||||
}
|
||||
|
||||
// lanKeyed marks a request from another machine that carried the key.
|
||||
type lanKeyed struct{}
|
||||
|
||||
|
||||
@@ -6,17 +6,22 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
// Shared on the network, a request from another machine needs the key, and
|
||||
// goes on with the gateway's own token; this computer's need none.
|
||||
// Sharing gates remote access; each authorized remote call carries the same
|
||||
// named identity as its local counterpart, so usage is attributed by key.
|
||||
func TestLANGuard(t *testing.T) {
|
||||
var got string
|
||||
fresh(t)
|
||||
var got, id string
|
||||
h := lanGuard(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
got = r.Header.Get("Authorization") + "|" + r.Header.Get("x-api-key")
|
||||
id = access.Caller(r.Context()).KeyID
|
||||
}))
|
||||
call := func(from string, hdr ...string) int {
|
||||
got = ""
|
||||
got, id = "", ""
|
||||
r := httptest.NewRequest("POST", "/v1/messages", nil)
|
||||
r.RemoteAddr = from
|
||||
for i := 0; i+1 < len(hdr); i += 2 {
|
||||
@@ -26,35 +31,52 @@ func TestLANGuard(t *testing.T) {
|
||||
h.ServeHTTP(w, r)
|
||||
return w.Code
|
||||
}
|
||||
key := "sk-magpie-k"
|
||||
none := ""
|
||||
t.Cleanup(func() { lanKey.Store(&none) })
|
||||
|
||||
lanKey.Store(&none)
|
||||
secret, err := access.Update("add-key", access.Change{Name: "Laptop"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keys, _ := access.List()
|
||||
keyID := keys[0].ID
|
||||
t.Setenv("MAGPIE_ADDR", "")
|
||||
if c := call("192.168.1.9:5000"); c != http.StatusForbidden {
|
||||
t.Fatal("unshared, another machine got", c)
|
||||
if c := call("192.168.1.9:5000", "x-api-key", secret); c != http.StatusForbidden {
|
||||
t.Fatal("unshared remote", c)
|
||||
}
|
||||
if c := call("127.0.0.1:5000"); c != 200 {
|
||||
t.Fatal("loopback got", c)
|
||||
if c := call("127.0.0.1:5000", "Authorization", "Bearer anything"); c != 200 {
|
||||
t.Fatal("loopback", c)
|
||||
}
|
||||
t.Setenv("MAGPIE_ADDR", "0.0.0.0:3425")
|
||||
s := settings.Load()
|
||||
s.LAN = true
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if addr := listenAddr(); addr != "0.0.0.0:"+Port() {
|
||||
t.Fatal(addr)
|
||||
}
|
||||
for _, hdr := range [][]string{nil, {"Authorization", "Bearer magpie"}, {"x-api-key", "wrong"}} {
|
||||
if c := call("192.168.1.9:5000", hdr...); c != http.StatusUnauthorized {
|
||||
t.Fatal("unauthorized remote", hdr, c)
|
||||
}
|
||||
}
|
||||
if c := call("192.168.1.9:5000", "x-api-key", secret); c != 200 || id != keyID || got != "Bearer magpie|magpie" {
|
||||
t.Fatal(c, id, got)
|
||||
}
|
||||
if c := call("127.0.0.1:5000", "Authorization", "Bearer "+secret); c != 200 || id != keyID {
|
||||
t.Fatal("local named key", c, id)
|
||||
}
|
||||
access.Update("off-key", access.Change{Key: keyID})
|
||||
if c := call("192.168.1.9:5000", "x-api-key", secret); c != http.StatusUnauthorized {
|
||||
t.Fatal("disabled key", c)
|
||||
}
|
||||
access.Update("on-key", access.Change{Key: keyID})
|
||||
access.Update("remove-key", access.Change{Key: keyID})
|
||||
if c := call("192.168.1.9:5000", "x-api-key", secret); c != http.StatusUnauthorized {
|
||||
t.Fatal("removed key", c)
|
||||
}
|
||||
s.LAN = false
|
||||
settings.Save(s)
|
||||
t.Setenv("MAGPIE_ADDR", "0.0.0.0:3499")
|
||||
if c := call("192.168.1.9:5000"); c != 200 {
|
||||
t.Fatal("MAGPIE_ADDR's open gateway got", c)
|
||||
}
|
||||
|
||||
lanKey.Store(&key)
|
||||
if c := call("192.168.1.9:5000"); c != http.StatusUnauthorized {
|
||||
t.Fatal("no key got", c)
|
||||
}
|
||||
if c := call("192.168.1.9:5000", "Authorization", "Bearer wrong"); c != http.StatusUnauthorized {
|
||||
t.Fatal("a wrong key got", c)
|
||||
}
|
||||
if c := call("192.168.1.9:5000", "x-api-key", key); c != 200 || got != "Bearer magpie|magpie" {
|
||||
t.Fatal(c, got)
|
||||
}
|
||||
if c := call("[::1]:5000", "Authorization", "Bearer anything"); c != 200 || got != "Bearer anything|" {
|
||||
t.Fatal("loopback", c, got)
|
||||
t.Fatal("explicit open gateway", c)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -70,8 +92,8 @@ func TestURLOfWildcard(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// An agent whose User-Agent is only the AI SDK's (Alma) is known by the
|
||||
// token it was given; the others still by their User-Agent.
|
||||
// An agent whose User-Agent is only the AI SDK's (Alma) is known by
|
||||
// the token it was given; the others still by their User-Agent.
|
||||
func TestAgentOf(t *testing.T) {
|
||||
for _, c := range []struct{ auth, key, ua, want string }{
|
||||
{"Bearer " + TokenFor("alma"), "", "ai-sdk/openai/2.0.52 ai-sdk/provider-utils/3.0.12 runtime/node.js/v22", "alma"},
|
||||
|
||||
@@ -5,6 +5,9 @@ import (
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
// GET /v1/magpie/quotas answers another machine with the key of the
|
||||
@@ -12,7 +15,7 @@ import (
|
||||
// a wrong key, none, or a gateway not shared still get nothing, and the
|
||||
// refusal says how.
|
||||
func TestQuotasOverLAN(t *testing.T) {
|
||||
setHome(t, t.TempDir())
|
||||
fresh(t)
|
||||
t.Setenv("MAGPIE_ADDR", "")
|
||||
h := lanGuard(New().Handler())
|
||||
call := func(from string, hdr ...string) (int, string) {
|
||||
@@ -25,11 +28,8 @@ func TestQuotasOverLAN(t *testing.T) {
|
||||
h.ServeHTTP(w, r)
|
||||
return w.Code, w.Body.String()
|
||||
}
|
||||
key := "sk-magpie-k"
|
||||
none := ""
|
||||
t.Cleanup(func() { lanKey.Store(&none) })
|
||||
|
||||
lanKey.Store(&key)
|
||||
keys, secrets := newCaller(t, "Quota client")
|
||||
key := secrets[0]
|
||||
if c, b := call("100.64.0.7:5000", "Authorization", "Bearer "+key); c != 200 || !strings.Contains(b, `"object":"list"`) {
|
||||
t.Fatal("shared, with the key:", c, b)
|
||||
}
|
||||
@@ -46,7 +46,17 @@ func TestQuotasOverLAN(t *testing.T) {
|
||||
t.Fatal("loopback got", c)
|
||||
}
|
||||
|
||||
lanKey.Store(&none)
|
||||
if _, err := access.Update("off-key", access.Change{Key: keys[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if c, _ := call("192.168.1.9:5000", "Authorization", "Bearer "+key); c != http.StatusUnauthorized {
|
||||
t.Fatal("disabled key got", c)
|
||||
}
|
||||
shared := settings.Load()
|
||||
shared.LAN = false
|
||||
if err := settings.Save(shared); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if c, _ := call("192.168.1.9:5000", "Authorization", "Bearer "+key); c != http.StatusForbidden {
|
||||
t.Fatal("not shared got", c)
|
||||
}
|
||||
|
||||
@@ -11,7 +11,9 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
// GET /v1/magpie/route tells an agent's UI where its session's turn went
|
||||
@@ -132,16 +134,34 @@ func TestSessionRoute(t *testing.T) {
|
||||
t.Fatal("no session got", code)
|
||||
}
|
||||
// another machine needs the shared gateway's key, as the quotas do
|
||||
key, none := "sk-magpie-k", ""
|
||||
t.Cleanup(func() { lanKey.Store(&none) })
|
||||
lanKey.Store(&key)
|
||||
callerKeys, secrets := newCaller(t, "Session status")
|
||||
key := secrets[0]
|
||||
if code, _ := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1"); code != http.StatusUnauthorized {
|
||||
t.Fatal("no key got", code)
|
||||
}
|
||||
if code, out := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1", "Authorization", "Bearer "+key); code != 200 || out["route"] == nil {
|
||||
t.Fatalf("with the key: %d %v", code, out)
|
||||
}
|
||||
lanKey.Store(&none)
|
||||
if code, out := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1", "x-api-key", key); code != 200 || out["route"] == nil {
|
||||
t.Fatalf("with x-api-key: %d %v", code, out)
|
||||
}
|
||||
if code, out := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1&key="+key); code != 200 || out["route"] == nil {
|
||||
t.Fatalf("with query key: %d %v", code, out)
|
||||
}
|
||||
if code, _ := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1", "Authorization", "Bearer wrong"); code != http.StatusUnauthorized {
|
||||
t.Fatal("wrong key got", code)
|
||||
}
|
||||
if _, err := access.Update("off-key", access.Change{Key: callerKeys[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, _ := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1", "Authorization", "Bearer "+key); code != http.StatusUnauthorized {
|
||||
t.Fatal("disabled key got", code)
|
||||
}
|
||||
shared := settings.Load()
|
||||
shared.LAN = false
|
||||
if err := settings.Save(shared); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MAGPIE_ADDR", "0.0.0.0:3425")
|
||||
if code, _ := get("192.168.1.9:5000", "/v1/magpie/route?session=pi-1"); code != http.StatusForbidden {
|
||||
t.Fatal("MAGPIE_ADDR, not shared, got", code)
|
||||
|
||||
@@ -461,7 +461,7 @@ func (s *Server) videosCreate(w http.ResponseWriter, r *http.Request) {
|
||||
call.Status = code
|
||||
}
|
||||
}
|
||||
usage.Append(usage.Record{Time: start, Agent: call.Agent, Via: call.Via, Provider: p.ID, Host: p.Where(), Model: model, Requested: call.Model,
|
||||
appendUsage(r, usage.Record{Time: start, Agent: call.Agent, Via: call.Via, Provider: p.ID, Host: p.Where(), Model: model, Requested: call.Model,
|
||||
Millis: call.Millis, Status: call.Status, Session: sessionOf(r.Header)})
|
||||
if err != nil {
|
||||
call.Error = err.Error()
|
||||
|
||||
+8
-8
@@ -20,6 +20,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/agent"
|
||||
"github.com/yetone/magpie/internal/autostart"
|
||||
"github.com/yetone/magpie/internal/catalog"
|
||||
@@ -269,6 +270,7 @@ func searchState(s *settingsJSON) {
|
||||
|
||||
func settingsState() settingsJSON {
|
||||
s := settingsJSON{Settings: settings.Load(), Version: Version, Dir: tilde(settings.Dir()), Gateway: gateway.URL()}
|
||||
s.LANKey = "" // the retained credential belongs on disk, not in UI state
|
||||
if found, err := discoverTerminals(); err == nil {
|
||||
for _, app := range found.Apps {
|
||||
s.TerminalApps = append(s.TerminalApps, terminalChoice{ID: app.ID, Name: app.Name})
|
||||
@@ -546,12 +548,14 @@ func Handler(w Windows, gw *gateway.Server) http.Handler {
|
||||
providerRoutes(mux, w)
|
||||
importRoutes(mux)
|
||||
usageRoutes(mux, w)
|
||||
callerKeyRoutes(mux)
|
||||
sessionRoutes(mux, w)
|
||||
backupRoutes(mux, w)
|
||||
archiveRoutes(mux)
|
||||
libraryRoutes(mux, w)
|
||||
updateRoutes(mux, w)
|
||||
mux.HandleFunc("GET /api/settings", func(rw http.ResponseWriter, r *http.Request) {
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
writeJSON(rw, settingsState())
|
||||
})
|
||||
mux.HandleFunc("POST /api/settings", func(rw http.ResponseWriter, r *http.Request) {
|
||||
@@ -576,6 +580,7 @@ func Handler(w Windows, gw *gateway.Server) http.Handler {
|
||||
in.Visible, in.HiddenModels = cur.Visible, cur.HiddenModels
|
||||
settings.CarryPerModel(&in, &cur)
|
||||
in.LAN, in.LANKey = cur.LAN, cur.LANKey
|
||||
in.LANKeyID = cur.LANKeyID
|
||||
in.RequestArchive = cur.RequestArchive // the Gateway page's, set on its own
|
||||
in.RedactRules = cur.RedactRules // the masking rules, set on their own
|
||||
// used or left is the Usage page's toggle as much as Settings', set on its own
|
||||
@@ -750,20 +755,15 @@ func Handler(w Windows, gw *gateway.Server) http.Handler {
|
||||
}
|
||||
writeJSON(rw, settingsState())
|
||||
})
|
||||
// sharing the gateway on the local network: on makes its key the first
|
||||
// time, new asks for another (the old one stops working)
|
||||
// Sharing controls exposure; the gateway's named caller keys authenticate
|
||||
// remote clients just as they do local ones.
|
||||
mux.HandleFunc("POST /api/settings/lan", func(rw http.ResponseWriter, r *http.Request) {
|
||||
var in struct{ On, NewKey bool }
|
||||
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
|
||||
fail(rw, err)
|
||||
return
|
||||
}
|
||||
s := settings.Load()
|
||||
s.LAN = in.On
|
||||
if s.LANKey == "" || in.NewKey {
|
||||
s.LANKey = gateway.NewLANKey()
|
||||
}
|
||||
if err := settings.Save(s); err != nil {
|
||||
if err := access.ConfigureLAN(in.On, in.NewKey); err != nil {
|
||||
fail(rw, err)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -1334,8 +1334,12 @@ body.window .ag-sk-icon { width: 22px; height: 22px; border-radius: 6px; }
|
||||
.connect .hint { font-size: 11px; color: var(--faint); margin-top: 4px; line-height: 1.4; }
|
||||
.connect .val { display: inline-flex; align-items: center; gap: 4px; height: 26px; padding: 0 4px 0 9px; border-radius: 7px; background: var(--pill); box-shadow: inset 0 0 0 1px var(--line-2); max-width: 100%; }
|
||||
.connect .val code { font-size: 12px; color: var(--fg); user-select: text; -webkit-user-select: text; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.connect .val > .val { min-width: 0; height: auto; padding: 0; background: none; box-shadow: none; }
|
||||
.connect .segs { margin-top: -1px; }
|
||||
.connect .stack { display: flex; flex-direction: column; gap: 8px; min-width: 0; }
|
||||
@media (max-width: 520px) {
|
||||
.connect { grid-template-columns: 48px minmax(0, 1fr); column-gap: 8px; padding-left: 10px; padding-right: 10px; }
|
||||
}
|
||||
.copy { width: 22px; height: 22px; border-radius: 5px; display: inline-flex; align-items: center; justify-content: center; color: var(--muted); flex: none; transition: background .12s, color .12s; }
|
||||
.copy:hover { background: var(--pill-hover); color: var(--fg); }
|
||||
.copy.done { color: var(--green); background: var(--green-soft); animation: copy-pop .36s var(--ease); }
|
||||
@@ -1494,6 +1498,8 @@ body.window .ag-sk-icon { width: 22px; height: 22px; border-radius: 6px; }
|
||||
.acc .proto:hover, .acc .proto.open { color: var(--fg); background: var(--pill-hover); }
|
||||
.acc .n, .acc .rename { flex: 0 1 auto; }
|
||||
.acc .plan { flex: none; }
|
||||
#gatewayKeys .acc { min-width: 0; }
|
||||
#gatewayKeys .acc .plan { flex: 0 1 auto; min-width: 0; overflow: hidden; text-overflow: ellipsis; }
|
||||
/* the Settings row's picker: the key row's badge, always shown */
|
||||
.row.pref .proto.pick { flex: none; margin-left: auto; display: inline-flex; align-items: center; gap: 3px; height: 24px; padding: 0 6px 0 9px; font: inherit; font-size: 12px; color: var(--fg-2); border: 0; border-radius: 7px; background: var(--pill); white-space: nowrap; cursor: default; transition: background .12s, color .12s; }
|
||||
.row.pref .proto.pick svg { opacity: .7; transition: transform .15s var(--ease); }
|
||||
@@ -1656,14 +1662,14 @@ body.window .ag-sk-icon { width: 22px; height: 22px; border-radius: 6px; }
|
||||
.stepplan .steps { margin: 0; padding-left: 18px; display: flex; flex-direction: column; gap: 8px; font-size: 12px; color: var(--muted); line-height: 1.5; }
|
||||
.stepplan .steps .pair { margin-top: 2px; }
|
||||
.stepplan .bookmarklet { display: inline-block; padding: 1px 8px; border: 1px solid var(--line); border-radius: 6px; color: var(--fg); background: var(--pill); text-decoration: none; cursor: grab; white-space: nowrap; }
|
||||
.editor input[type="text"], .editor input[type="password"], .editor input[type="url"] {
|
||||
.editor input[type="text"], .editor input[type="password"], .editor input[type="url"], .acc .kf input, .acc input.rename-in {
|
||||
height: 27px; padding: 0 9px; border-radius: 7px;
|
||||
border: 1px solid var(--line); background: var(--card);
|
||||
font-size: 12.5px; outline: none; width: 100%; min-width: 0;
|
||||
transition: border-color .12s, box-shadow .12s;
|
||||
}
|
||||
.editor input:focus, .editor .headers textarea.json:focus { border-color: var(--accent); box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 22%, transparent); outline: none; }
|
||||
.editor input::placeholder { color: var(--faint); }
|
||||
.editor input:focus, .editor .headers textarea.json:focus, .acc input:focus { border-color: var(--accent); box-shadow: 0 0 0 3px color-mix(in srgb, var(--accent) 22%, transparent); outline: none; }
|
||||
.editor input::placeholder, .acc input::placeholder { color: var(--faint); }
|
||||
.editor .pair { display: flex; gap: 8px; align-items: center; }
|
||||
.editor .pair input:first-child { flex: 1; }
|
||||
.editor .side { display: flex; gap: 4px; align-items: center; flex: none; min-width: 52px; justify-content: flex-end; }
|
||||
|
||||
+100
-57
@@ -2493,13 +2493,13 @@ function askForgetSaved(x) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
closeResetAsk();
|
||||
closeConfirmAsk();
|
||||
};
|
||||
const cancel = el("button", "text", t("Cancel"));
|
||||
cancel.onclick = (e) => { e.stopPropagation(); closeResetAsk(); };
|
||||
cancel.onclick = (e) => { e.stopPropagation(); closeConfirmAsk(); };
|
||||
bar.append(el("span", "grow"), cancel, go);
|
||||
ed.append(bar);
|
||||
resetAsk = ed;
|
||||
confirmAsk = ed;
|
||||
openModal(ed);
|
||||
$("#modal").classList.add("lib");
|
||||
cancel.focus();
|
||||
@@ -2628,6 +2628,9 @@ function renderGatewayView() {
|
||||
page.classList.remove("loading");
|
||||
page.removeAttribute("aria-busy");
|
||||
renderGateway();
|
||||
$("#gatewayKeysBlock").hidden = !providers.gateway.lan;
|
||||
if (!providers.gateway.lan) gatewayKeyDraft = null;
|
||||
if (gatewayKeyDraft === null && !$("#gatewayKeys .rename-in")) renderGatewayKeys();
|
||||
renderConnect();
|
||||
renderGatewayModels();
|
||||
renderArchive();
|
||||
@@ -2662,19 +2665,19 @@ const FLAVORS = {
|
||||
openai: {
|
||||
name: "OpenAI", base: (u) => u + "/v1", baseEnv: "OPENAI_BASE_URL", keyEnv: "OPENAI_API_KEY",
|
||||
note: "Chat Completions, the API most tools speak. Anything with an OpenAI base-URL setting works.",
|
||||
curl: (b, m) => ({ url: `${b}/chat/completions`, headers: ["Authorization: Bearer magpie"],
|
||||
curl: (b, m, k = "magpie") => ({ url: `${b}/chat/completions`, headers: [`Authorization: Bearer ${k}`],
|
||||
body: `{"model": "${m}",\n "messages": [{"role": "user", "content": "hi"}]}` }),
|
||||
python: (b, m) => `from openai import OpenAI
|
||||
python: (b, m, k = "magpie") => `from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="${b}", api_key="magpie")
|
||||
client = OpenAI(base_url="${b}", api_key="${k}")
|
||||
r = client.chat.completions.create(
|
||||
model="${m}",
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
)
|
||||
print(r.choices[0].message.content)`,
|
||||
node: (b, m) => `import OpenAI from "openai";
|
||||
node: (b, m, k = "magpie") => `import OpenAI from "openai";
|
||||
|
||||
const client = new OpenAI({ baseURL: "${b}", apiKey: "magpie" });
|
||||
const client = new OpenAI({ baseURL: "${b}", apiKey: "${k}" });
|
||||
const r = await client.chat.completions.create({
|
||||
model: "${m}",
|
||||
messages: [{ role: "user", content: "hi" }],
|
||||
@@ -2684,28 +2687,28 @@ console.log(r.choices[0].message.content);`,
|
||||
responses: {
|
||||
name: "Responses", base: (u) => u + "/v1", baseEnv: "OPENAI_BASE_URL", keyEnv: "OPENAI_API_KEY",
|
||||
note: "OpenAI's newer API: reasoning, built-in tool items, encrypted reasoning. Codex speaks this.",
|
||||
curl: (b, m) => ({ url: `${b}/responses`, headers: ["Authorization: Bearer magpie"],
|
||||
curl: (b, m, k = "magpie") => ({ url: `${b}/responses`, headers: [`Authorization: Bearer ${k}`],
|
||||
body: `{"model": "${m}", "input": "hi"}` }),
|
||||
python: (b, m) => `from openai import OpenAI
|
||||
python: (b, m, k = "magpie") => `from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="${b}", api_key="magpie")
|
||||
client = OpenAI(base_url="${b}", api_key="${k}")
|
||||
r = client.responses.create(model="${m}", input="hi")
|
||||
print(r.output_text)`,
|
||||
node: (b, m) => `import OpenAI from "openai";
|
||||
node: (b, m, k = "magpie") => `import OpenAI from "openai";
|
||||
|
||||
const client = new OpenAI({ baseURL: "${b}", apiKey: "magpie" });
|
||||
const client = new OpenAI({ baseURL: "${b}", apiKey: "${k}" });
|
||||
const r = await client.responses.create({ model: "${m}", input: "hi" });
|
||||
console.log(r.output_text);`,
|
||||
},
|
||||
anthropic: {
|
||||
name: "Anthropic", base: (u) => u, baseEnv: "ANTHROPIC_BASE_URL", keyEnv: "ANTHROPIC_API_KEY",
|
||||
note: "Messages API. Claude Code reads ANTHROPIC_AUTH_TOKEN instead of the key; the Agents tab sets that for you.",
|
||||
curl: (b, m) => ({ url: `${b}/v1/messages`, headers: ["x-api-key: magpie", "anthropic-version: 2023-06-01"],
|
||||
curl: (b, m, k = "magpie") => ({ url: `${b}/v1/messages`, headers: [`x-api-key: ${k}`, "anthropic-version: 2023-06-01"],
|
||||
body: `{"model": "${m}", "max_tokens": 1024,\n "messages": [{"role": "user", "content": "hi"}]}` }),
|
||||
python: (b, m) => `import anthropic
|
||||
python: (b, m, k = "magpie") => `import anthropic
|
||||
|
||||
client = anthropic.Anthropic(
|
||||
base_url="${b}", api_key="magpie",
|
||||
base_url="${b}", api_key="${k}",
|
||||
)
|
||||
m = client.messages.create(
|
||||
model="${m}",
|
||||
@@ -2713,9 +2716,9 @@ m = client.messages.create(
|
||||
messages=[{"role": "user", "content": "hi"}],
|
||||
)
|
||||
print(m.content[0].text)`,
|
||||
node: (b, m) => `import Anthropic from "@anthropic-ai/sdk";
|
||||
node: (b, m, k = "magpie") => `import Anthropic from "@anthropic-ai/sdk";
|
||||
|
||||
const client = new Anthropic({ baseURL: "${b}", apiKey: "magpie" });
|
||||
const client = new Anthropic({ baseURL: "${b}", apiKey: "${k}" });
|
||||
const m = await client.messages.create({
|
||||
model: "${m}",
|
||||
max_tokens: 1024,
|
||||
@@ -2726,17 +2729,17 @@ console.log(m.content[0].text);`,
|
||||
gemini: {
|
||||
name: "Gemini", base: (u) => u, baseEnv: "GOOGLE_GEMINI_BASE_URL", keyEnv: "GEMINI_API_KEY",
|
||||
note: "Google's generateContent API, v1beta. Gemini CLI and the google-genai SDKs speak this.",
|
||||
curl: (b, m) => ({ url: `${b}/v1beta/models/${m}:generateContent`, headers: ["x-goog-api-key: magpie"],
|
||||
curl: (b, m, k = "magpie") => ({ url: `${b}/v1beta/models/${m}:generateContent`, headers: [`x-goog-api-key: ${k}`],
|
||||
body: `{"contents": [{"parts": [{"text": "hi"}]}]}` }),
|
||||
python: (b, m) => `from google import genai
|
||||
python: (b, m, k = "magpie") => `from google import genai
|
||||
|
||||
client = genai.Client(api_key="magpie", http_options={"base_url": "${b}"})
|
||||
client = genai.Client(api_key="${k}", http_options={"base_url": "${b}"})
|
||||
r = client.models.generate_content(model="${m}", contents="hi")
|
||||
print(r.text)`,
|
||||
node: (b, m) => `import { GoogleGenAI } from "@google/genai";
|
||||
node: (b, m, k = "magpie") => `import { GoogleGenAI } from "@google/genai";
|
||||
|
||||
const ai = new GoogleGenAI({
|
||||
apiKey: "magpie",
|
||||
apiKey: "${k}",
|
||||
httpOptions: { baseUrl: "${b}" },
|
||||
});
|
||||
const r = await ai.models.generateContent({ model: "${m}", contents: "hi" });
|
||||
@@ -2811,7 +2814,20 @@ function renderConnect() {
|
||||
if (!models.some((m) => m.id === exampleModel)) exampleModel = models[0]?.id || "";
|
||||
const model = exampleModel || "provider/model";
|
||||
const f = FLAVORS[flavor] || FLAVORS.openai;
|
||||
const base = f.base(g.url);
|
||||
const urls = [g.url, ...(g.lanURLs || [])];
|
||||
if (!urls.includes(connectURL)) connectURL = g.url;
|
||||
const remote = connectURL !== g.url;
|
||||
const keys = g.lan ? (gatewayKeys || []).filter((k) => !k.off) : [];
|
||||
if (!keys.some((k) => k.id === connectKeyID)) {
|
||||
connectKeyID = "";
|
||||
connectSecret = null;
|
||||
}
|
||||
if (remote && !connectKeyID && keys.length) {
|
||||
queueMicrotask(() => selectConnectKey(keys[0].id));
|
||||
}
|
||||
const key = keys.find((k) => k.id === connectKeyID);
|
||||
const secret = key ? (connectSecret?.id === key.id ? connectSecret.secret : "") : remote ? "" : "magpie";
|
||||
const base = f.base(connectURL);
|
||||
const fold = $("#foldConnect");
|
||||
fold.setAttribute("aria-expanded", String(!connectFolded));
|
||||
fold.title = t(connectFolded ? "Show how to connect" : "Fold Connect away");
|
||||
@@ -2821,17 +2837,31 @@ function renderConnect() {
|
||||
note.replaceChildren();
|
||||
note.classList.toggle("brief", connectFolded);
|
||||
if (connectFolded) note.append(el("code", "", base), copyBtn(base, "Base URL"));
|
||||
else note.textContent = t("Loopback only · the key can be anything");
|
||||
else note.textContent = t(remote ? "Local network · an enabled gateway key is required" : g.lan && gatewayKeys?.length ? "Use a gateway key to track usage" : "Loopback only · the key can be anything");
|
||||
|
||||
box.append(...field("API", segs(Object.entries(FLAVORS).map(([k, v]) => [k, v.name]), flavor, (id) => { flavor = id; localStorage.setItem("magpie.flavor", id); renderConnect(); }), t(f.note)));
|
||||
|
||||
const b = el("div", "val");
|
||||
b.append(el("code", "", base), copyBtn(base, "Base URL"));
|
||||
if (urls.length > 1) b.append(connectPick("connectAddress", "Address", base,
|
||||
urls.map((u) => ({ v: u, name: f.base(u), note: t(u === g.url ? "This computer" : "Local network") })), connectURL,
|
||||
(u) => { connectURL = u; renderConnect(); }));
|
||||
else b.append(el("code", "", base));
|
||||
b.append(copyBtn(base, "Base URL"));
|
||||
box.append(...field("Base URL", b, t("What {env} takes.", { env: f.baseEnv })));
|
||||
|
||||
const k = el("div", "val");
|
||||
k.append(el("code", "", "magpie"), copyBtn("magpie", t("Key")));
|
||||
box.append(...field(t("API key"), k, t("{env}=magpie. The gateway trusts everything on loopback, so any value works.", { env: f.keyEnv })));
|
||||
const options = keys.map((k) => ({ v: k.id, name: k.name, literalName: true, note: k.masked }));
|
||||
if (!remote) options.unshift({ v: "", name: "This computer", note: t("Any key · no key attribution") });
|
||||
const keyLabel = remote || key ? "Gateway key" : "API key";
|
||||
if (!g.lan) k.append(el("code", "", "magpie"));
|
||||
else if (options.length) k.append(connectPick("connectKey", keyLabel, key ? key.name : remote ? t("Choose a gateway key") : t("This computer"),
|
||||
options, connectKeyID, selectConnectKey));
|
||||
else k.append(el("code", "", t("Create a gateway key above to connect")));
|
||||
if (key) k.append(copyCallerKeyBtn(key));
|
||||
else if (!remote) k.append(copyBtn("magpie", t("Key")));
|
||||
box.append(...field(t(keyLabel), k, t(g.lan && (remote || gatewayKeys?.length)
|
||||
? "Choose a gateway key to use as {env}; usage is tracked by key."
|
||||
: "{env}=magpie. The gateway trusts everything on loopback, so any value works.", { env: f.keyEnv })));
|
||||
|
||||
const m = el("div", "val");
|
||||
m.append(el("code", "", model), copyBtn(model, t("Model id")));
|
||||
@@ -2839,16 +2869,18 @@ function renderConnect() {
|
||||
|
||||
const ex = el("div", "stack");
|
||||
ex.append(segs(LANGS, lang, (id) => { lang = id; localStorage.setItem("magpie.lang", id); renderConnect(); }));
|
||||
const code = lang === "shell" ? envSnippet([[f.baseEnv, base], [f.keyEnv, "magpie"]])
|
||||
: lang === "curl" ? curlSnippet(f.curl(base, model))
|
||||
: f[lang](base, model);
|
||||
const code = !secret ? t(key ? "Loading gateway key…" : "Create a gateway key above to connect")
|
||||
: lang === "shell" ? envSnippet([[f.baseEnv, base], [f.keyEnv, secret]])
|
||||
: lang === "curl" ? curlSnippet(f.curl(base, model, secret))
|
||||
: f[lang](base, model, secret);
|
||||
const pre = el("pre", "snip");
|
||||
const c = el("code");
|
||||
c.append(highlight(code, lang));
|
||||
pre.append(c);
|
||||
// the button sits outside the scrolling box, so a long line doesn't carry it off
|
||||
const wrap = el("div", "snip-wrap");
|
||||
wrap.append(pre, copyBtn(code, t("Snippet")));
|
||||
wrap.append(pre);
|
||||
if (secret) wrap.append(copyBtn(code, t("Snippet")));
|
||||
ex.append(wrap);
|
||||
box.append(...field(t("Example"), ex, lang === "shell" ? t("Put these in the shell (or the tool's settings) and the tool talks to magpie instead of the vendor.") : ""));
|
||||
}
|
||||
@@ -6515,7 +6547,7 @@ function openProtoMenu(anchor, opts, value, choose, head = "Protocol this key sp
|
||||
b.type = "button";
|
||||
b.setAttribute("role", multi && o.v && o.v !== "\x00" ? "menuitemcheckbox" : "menuitemradio");
|
||||
const words = el("span", "pm-words");
|
||||
words.append(el("span", "pm-name", t(o.name)), el("span", "pm-note", t(o.note)));
|
||||
words.append(el("span", "pm-name", o.literalName ? o.name : t(o.name)), el("span", "pm-note", t(o.note)));
|
||||
b.append(el("span", "pm-tick"), words);
|
||||
tick(b, o);
|
||||
b.onclick = (e) => {
|
||||
@@ -6914,7 +6946,7 @@ function renderUsageLoading() {
|
||||
stats.append(tile);
|
||||
}
|
||||
$("#chart").hidden = true;
|
||||
for (const id of ["usageAgents", "usageModels"]) $("#" + id).hidden = true;
|
||||
for (const id of ["usageAgents", "usageModels", "usageKeys"]) $("#" + id).hidden = true;
|
||||
for (const h of $$("#view-usage .row-head")) h.hidden = true;
|
||||
$("#usageNote").textContent = "";
|
||||
}
|
||||
@@ -7509,7 +7541,7 @@ function autoResetButton(q, cls) {
|
||||
// askReset: spending a Codex account's reset can't be taken
|
||||
// back, so it asks first; then it says what came of it and reads the
|
||||
// usage again.
|
||||
let resetAsk = null;
|
||||
let confirmAsk = null;
|
||||
function askReset(q) {
|
||||
const ed = el("div", "editor reset-ask");
|
||||
const head = el("div", "ehead");
|
||||
@@ -7530,7 +7562,7 @@ function askReset(q) {
|
||||
go.classList.add("busy");
|
||||
try {
|
||||
const out = await api("usage/codex-reset", { user: q.user || "" });
|
||||
closeResetAsk();
|
||||
closeConfirmAsk();
|
||||
status(who + ": " + resetOutcome(out), out.code === "reset" ? "ok" : "err");
|
||||
loadQuotas();
|
||||
} catch (err) {
|
||||
@@ -7540,26 +7572,26 @@ function askReset(q) {
|
||||
}
|
||||
};
|
||||
const cancel = el("button", "text", t("Cancel"));
|
||||
cancel.onclick = (e) => { e.stopPropagation(); closeResetAsk(); };
|
||||
cancel.onclick = (e) => { e.stopPropagation(); closeConfirmAsk(); };
|
||||
bar.append(el("span", "grow"), cancel, go);
|
||||
ed.append(bar);
|
||||
resetAsk = ed;
|
||||
confirmAsk = ed;
|
||||
openModal(ed);
|
||||
$("#modal").classList.add("lib");
|
||||
go.focus();
|
||||
}
|
||||
function closeResetAsk() {
|
||||
if (!resetAsk) return;
|
||||
resetAsk = null;
|
||||
closeModal().then(() => { if (!resetAsk) $("#modal").classList.remove("lib"); });
|
||||
function closeConfirmAsk() {
|
||||
if (!confirmAsk) return;
|
||||
confirmAsk = null;
|
||||
closeModal().then(() => { if (!confirmAsk) $("#modal").classList.remove("lib"); });
|
||||
}
|
||||
// the dialog is the providers page's: while this asks, its backdrop and
|
||||
// Escape close only this (in the panel, Escape would hide the window)
|
||||
// Escape closes only the confirmation (in the panel it would hide the window)
|
||||
$("#modal").addEventListener("click", (e) => {
|
||||
if (resetAsk && e.target === e.currentTarget) { e.stopImmediatePropagation(); closeResetAsk(); }
|
||||
if (confirmAsk && e.target === e.currentTarget) { e.stopImmediatePropagation(); closeConfirmAsk(); }
|
||||
}, true);
|
||||
document.addEventListener("keydown", (e) => {
|
||||
if (resetAsk && e.key === "Escape") { e.preventDefault(); e.stopImmediatePropagation(); closeResetAsk(); }
|
||||
if (confirmAsk && e.key === "Escape") { e.preventDefault(); e.stopImmediatePropagation(); closeConfirmAsk(); }
|
||||
}, true);
|
||||
function resetOutcome(out) {
|
||||
switch (out.code) {
|
||||
@@ -7744,6 +7776,7 @@ function renderUsage() {
|
||||
$("#chart").hidden = empty;
|
||||
for (const id of ["usageAgents", "usageModels"]) $("#" + id).hidden = empty;
|
||||
for (const h of $$("#view-usage .row-head")) h.hidden = empty;
|
||||
$("#usageKeysHead").hidden = $("#usageKeys").hidden = empty || !u.callerKeys?.length;
|
||||
if (empty) {
|
||||
stats.classList.add("empty");
|
||||
const none = { today: "No calls today.", "7d": "No calls in the last 7 days.", "30d": "No calls in the last 30 days.", all: "No calls yet." }[period];
|
||||
@@ -7829,6 +7862,7 @@ function renderUsage() {
|
||||
};
|
||||
list("usageAgents", u.agents);
|
||||
list("usageModels", u.models);
|
||||
list("usageKeys", u.callerKeys || []);
|
||||
$("#usageNote").textContent = t("Counted from the providers' own usage reports on every call through the gateway · {path}", { path: u.path });
|
||||
}
|
||||
|
||||
@@ -7841,7 +7875,7 @@ function renderUsage() {
|
||||
// server pages it (/api/usage/requests) and saves it whole as CSV.
|
||||
|
||||
let ledger = null; // the page shown: { rows, offset, total, agents, …totals }
|
||||
let ledOffset = 0, ledAgent = "", ledProvider = "", ledFailed = false, ledQuery = "", ledModel = "", ledRoute = 0;
|
||||
let ledOffset = 0, ledAgent = "", ledProvider = "", ledCallerKey = "", ledFailed = false, ledQuery = "", ledModel = "", ledRoute = 0;
|
||||
const LED_PAGE = 100;
|
||||
// Remember the chart metric; start each app load split by model.
|
||||
let ledMetric = "tokens", ledSplit = "model";
|
||||
@@ -7852,10 +7886,10 @@ try {
|
||||
|
||||
let ledRouteInfo = null, ledBeforeRoute = null;
|
||||
window.openUsageRoute = (route) => {
|
||||
if (!ledBeforeRoute) ledBeforeRoute = { period, ledOffset, ledAgent, ledProvider, ledFailed, ledQuery, ledModel };
|
||||
if (!ledBeforeRoute) ledBeforeRoute = { period, ledOffset, ledAgent, ledProvider, ledCallerKey, ledFailed, ledQuery, ledModel };
|
||||
ledRoute = route.id;
|
||||
ledRouteInfo = route;
|
||||
ledOffset = 0; ledAgent = ""; ledProvider = ""; ledFailed = false; ledQuery = ""; ledModel = "";
|
||||
ledOffset = 0; ledAgent = ""; ledProvider = ""; ledCallerKey = ""; ledFailed = false; ledQuery = ""; ledModel = "";
|
||||
$("#ledQ").value = "";
|
||||
period = "all";
|
||||
usageTab = "requests";
|
||||
@@ -7867,6 +7901,7 @@ function ledParams(extra) {
|
||||
const q = new URLSearchParams({ period });
|
||||
if (ledAgent) q.set("agent", ledAgent);
|
||||
if (ledProvider) q.set("provider", ledProvider);
|
||||
if (ledCallerKey) q.set("callerKey", ledCallerKey);
|
||||
if (ledRoute) q.set("route", ledRoute);
|
||||
if (ledFailed) q.set("failed", "1");
|
||||
if (ledModel) q.set("model", ledModel);
|
||||
@@ -8393,6 +8428,14 @@ const LED_COLS = [
|
||||
|
||||
function renderLedger() {
|
||||
const l = ledger;
|
||||
const callers = l.callerKeys || [];
|
||||
if (ledCallerKey && !callers.some((k) => k.id === ledCallerKey)) {
|
||||
ledCallerKey = "";
|
||||
ledOffset = 0;
|
||||
// Reload the rows too: this response still belongs to the missing key.
|
||||
loadLedger().catch((e) => status(e.message, "err"));
|
||||
return;
|
||||
}
|
||||
const view = $("#view-usage");
|
||||
view.classList.remove("loading");
|
||||
view.removeAttribute("aria-busy");
|
||||
@@ -8413,6 +8456,9 @@ function renderLedger() {
|
||||
const providers = l.providers || [];
|
||||
if (ledProvider && !providers.some((p) => p.id === ledProvider)) ledProvider = "";
|
||||
sessPick($("#ledProvider"), "All providers", ledProvider, providers.map((p) => ({ v: p.id, name: t(p.name), note: "" })), "Provider", (v) => { ledProvider = v; ledOffset = 0; loadLedger().catch((e) => status(e.message, "err")); });
|
||||
sessPick($("#ledKey"), "All gateway keys", ledCallerKey, callers.map((k) => ({
|
||||
v: k.id, name: k.name, note: "",
|
||||
})), "Gateway keys", (v) => { ledCallerKey = v; ledOffset = 0; loadLedger().catch((e) => status(e.message, "err")); });
|
||||
const seg = $("#ledStatus");
|
||||
seg.replaceChildren();
|
||||
for (const [on, name] of [[false, "All"], [true, "Failed"]]) {
|
||||
@@ -8442,7 +8488,7 @@ function renderLedger() {
|
||||
$("#ledRouteClear").setAttribute("aria-label", t("Clear filter"));
|
||||
$("#ledRouteClear").onclick = () => {
|
||||
ledRoute = 0; ledRouteInfo = null;
|
||||
if (ledBeforeRoute) ({ period, ledOffset, ledAgent, ledProvider, ledFailed, ledQuery, ledModel } = ledBeforeRoute);
|
||||
if (ledBeforeRoute) ({ period, ledOffset, ledAgent, ledProvider, ledCallerKey, ledFailed, ledQuery, ledModel } = ledBeforeRoute);
|
||||
ledBeforeRoute = null;
|
||||
$("#ledQ").value = ledQuery;
|
||||
loadLedger().catch((e) => status(e.message, "err"));
|
||||
@@ -8453,7 +8499,7 @@ function renderLedger() {
|
||||
const pager = $("#ledPager");
|
||||
if (!l.total) {
|
||||
wrap.classList.add("none");
|
||||
const filtered = ledRoute || ledAgent || ledProvider || ledFailed || ledQuery.trim();
|
||||
const filtered = ledRoute || ledAgent || ledProvider || ledCallerKey || ledModel || ledFailed || ledQuery.trim();
|
||||
const none = { today: "No calls today.", "7d": "No calls in the last 7 days.", "30d": "No calls in the last 30 days.", all: "No calls yet." }[period];
|
||||
wrap.replaceChildren(el("div", "led-none", filtered ? t("No requests match these filters.") : t(none)));
|
||||
pager.hidden = true;
|
||||
@@ -8489,7 +8535,7 @@ function renderLedger() {
|
||||
const who = el("span", "who");
|
||||
// an agent on another computer, whose magpie passed the request on
|
||||
const name = r.agentName || r.agent;
|
||||
who.append(icon(r.icon || "generic"), el("span", "", r.via ? t("{agent} · via {host}", { agent: name, host: r.via }) : name));
|
||||
who.append(icon(r.icon || "generic"), el("span", "", [r.via ? t("{agent} · via {host}", { agent: name, host: r.via }) : name, r.callerKeyLabel || r.callerKeyName].filter(Boolean).join(" · ")));
|
||||
td(who, "", [r.kind, r.session && t("session {id}", { id: r.session })].filter(Boolean).join(" · "));
|
||||
td(r.req || "—", "model" + (r.req ? "" : " faint"), r.req || t("Not kept for requests before this version"));
|
||||
const local = r.source === "log";
|
||||
@@ -10914,7 +10960,7 @@ function renderRedactRules(s, row) {
|
||||
}
|
||||
|
||||
// renderLAN: the gateway shared on the local network, for agents on other
|
||||
// machines — the addresses they use it at and the key they must send.
|
||||
// machines. Gateway keys and connection examples live together in Gateway.
|
||||
let lanSelectedURL = "", lanProtocol = "openai";
|
||||
function renderLAN(s) {
|
||||
const box = $("#lanList");
|
||||
@@ -10933,7 +10979,7 @@ function renderLAN(s) {
|
||||
};
|
||||
const set = (body) => writingPrefs(api("settings/lan", body)).then((ns) => { prefs = ns; renderSettings(); })
|
||||
.catch((e) => { status(t(e.message), "err"); renderSettings(); });
|
||||
row(t("Share on local network"), t("Agents on other computers on this network can use magpie’s models, with the API key below"), "",
|
||||
row(t("Share on local network"), t("Agents on other computers on this network can use magpie’s models with a gateway key from Gateway"), "",
|
||||
segs([["off", t("Off")], ["on", t("On")]], s.lan ? "on" : "off", (v) => set({ on: v === "on" })));
|
||||
if (!s.lan) return;
|
||||
let urls = s.lanURLs || [], sub = "";
|
||||
@@ -10981,10 +11027,6 @@ function renderLAN(s) {
|
||||
update();
|
||||
}
|
||||
if (sub) row(t("In a container"), sub, "").classList.add("lan-container");
|
||||
const again = el("button", "text", t("New key"));
|
||||
again.onclick = () => set({ on: true, newKey: true });
|
||||
row(t("API key"), t("Other computers send it as their API key; a new one stops the old from working"),
|
||||
s.lanKey.slice(0, 14) + "…", copyBtn(s.lanKey, t("API key")), again);
|
||||
}
|
||||
|
||||
// renderUpdate fills in the version row: whether a newer magpie is out.
|
||||
@@ -11355,6 +11397,7 @@ function show(v) {
|
||||
closePicker();
|
||||
closeAgentModels();
|
||||
if (v !== "providers" && editing !== null) cancelEdit();
|
||||
if (v === "gateway") loadGatewayKeys();
|
||||
if (v === "providers" || v === "gateway" || v === "routing") loadProviders().then(back, (e) => status(e.message, "err"));
|
||||
if (v === "usage") loadUsage(true).then(back, (e) => status(e.message, "err"));
|
||||
if (v === "settings") loadSettings().then(back, (e) => status(e.message, "err"));
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
// Caller keys use the same account list and edit controls as provider keys.
|
||||
let gatewayKeys = null, gatewayKeyDraft = null, gatewayKeysBusy = false;
|
||||
let gatewayKeysRead = 0;
|
||||
let connectURL = "", connectKeyID = "", connectSecret = null;
|
||||
|
||||
function connectPick(id, label, text, options, value, choose) {
|
||||
const b = el("button", "val");
|
||||
b.id = id;
|
||||
b.type = "button";
|
||||
b.setAttribute("aria-label", t(label));
|
||||
b.append(el("code", "", text), svg(CHEV, 11, 1.6));
|
||||
b.onclick = () => {
|
||||
if (b.classList.contains("open")) return closeProtoMenu();
|
||||
openProtoMenu(b, options, value, choose, label);
|
||||
};
|
||||
return b;
|
||||
}
|
||||
|
||||
async function selectConnectKey(id) {
|
||||
connectKeyID = id;
|
||||
connectSecret = null;
|
||||
renderConnect();
|
||||
if (!id || !providers?.gateway.lan) return;
|
||||
const key = gatewayKeys?.find((k) => k.id === id && !k.off);
|
||||
if (!key) return;
|
||||
try {
|
||||
const out = await api("caller-keys/copy-key", { key: id });
|
||||
if (connectKeyID !== id || gatewayKeys?.find((k) => k.id === id) !== key) return;
|
||||
connectSecret = { id, secret: out.secret };
|
||||
if (view === "gateway") renderConnect();
|
||||
} catch (e) { status(t(e.message), "err"); }
|
||||
}
|
||||
async function loadGatewayKeys() {
|
||||
const read = ++gatewayKeysRead;
|
||||
try {
|
||||
const out = await api("caller-keys");
|
||||
if (read !== gatewayKeysRead) return;
|
||||
// A fresh list also refreshes a selected credential, which may have rotated.
|
||||
connectSecret = null;
|
||||
gatewayKeys = out.keys || [];
|
||||
renderGatewayKeys();
|
||||
if (view === "gateway" && providers) selectConnectKey(connectKeyID);
|
||||
} catch (e) {
|
||||
if (read !== gatewayKeysRead) return;
|
||||
$("#gatewayKeys").replaceChildren(el("div", "none", t(e.message)));
|
||||
}
|
||||
}
|
||||
|
||||
async function gatewayKeyAction(action, body) {
|
||||
if (gatewayKeysBusy) return null;
|
||||
gatewayKeysBusy = true;
|
||||
gatewayKeysRead++;
|
||||
try {
|
||||
const out = await api("caller-keys/" + action, body);
|
||||
gatewayKeysRead++;
|
||||
connectSecret = null;
|
||||
gatewayKeys = out.keys || [];
|
||||
gatewayKeyDraft = null;
|
||||
renderGatewayKeys();
|
||||
if (view === "gateway" && providers) selectConnectKey(connectKeyID);
|
||||
return out;
|
||||
} catch (e) {
|
||||
status(t(e.message), "err");
|
||||
return null;
|
||||
} finally {
|
||||
gatewayKeysBusy = false;
|
||||
}
|
||||
}
|
||||
|
||||
function gatewayRename(k) {
|
||||
const b = el("button", "n rename", k.name);
|
||||
b.title = t("Rename");
|
||||
b.onclick = () => {
|
||||
const i = input(k.name, t("Name"));
|
||||
i.className = "rename-in";
|
||||
i.setAttribute("aria-label", t("Name"));
|
||||
let finished = false;
|
||||
const done = async (save) => {
|
||||
if (finished) return;
|
||||
finished = true;
|
||||
// Keep the other controls in place when blur precedes their click.
|
||||
i.replaceWith(b);
|
||||
if (save && i.value.trim() !== k.name) await gatewayKeyAction("rename-key", { key: k.id, name: i.value });
|
||||
};
|
||||
i.onkeydown = (e) => { e.stopPropagation(); if (e.key === "Enter") done(true); else if (e.key === "Escape") done(false); };
|
||||
i.onblur = () => done(true);
|
||||
b.replaceWith(i);
|
||||
i.focus({ preventScroll: true });
|
||||
};
|
||||
return b;
|
||||
}
|
||||
|
||||
function gatewayKeyForm() {
|
||||
const box = el("div", "acc adding");
|
||||
const name = input(gatewayKeyDraft, t("Key name, e.g. Laptop"));
|
||||
name.setAttribute("aria-label", t("Gateway key name"));
|
||||
name.oninput = () => { gatewayKeyDraft = name.value; };
|
||||
const add = el("button", "text primary", t("Create"));
|
||||
const go = async () => {
|
||||
add.disabled = true;
|
||||
const out = await gatewayKeyAction("add-key", { name: name.value });
|
||||
if (!out) { add.disabled = false; return; }
|
||||
status(t("Gateway key created. Use Copy on its row to connect a client."), "ok");
|
||||
};
|
||||
add.onclick = go;
|
||||
name.onkeydown = (e) => {
|
||||
e.stopPropagation();
|
||||
if (e.key === "Enter" && !add.disabled) go();
|
||||
else if (e.key === "Escape") { gatewayKeyDraft = null; renderGatewayKeys(); }
|
||||
};
|
||||
const cancel = el("button", "text", t("Cancel"));
|
||||
cancel.onclick = () => { gatewayKeyDraft = null; renderGatewayKeys(); };
|
||||
const fields = el("div", "kf");
|
||||
fields.append(name);
|
||||
const bar = el("div", "kb");
|
||||
bar.append(el("span", "grow"), cancel, add);
|
||||
box.append(fields, bar);
|
||||
queueMicrotask(() => name.focus({ preventScroll: true }));
|
||||
return box;
|
||||
}
|
||||
|
||||
function copyCallerKeyBtn(k) {
|
||||
const cp = copyBtn("", t("Gateway key"));
|
||||
cp.setAttribute("aria-label", t("Copy gateway key"));
|
||||
cp.onclick = async () => {
|
||||
try {
|
||||
const out = await api("caller-keys/copy-key", { key: k.id });
|
||||
await copy(out.secret, t("Gateway key"), cp);
|
||||
} catch (e) { status(t(e.message), "err"); }
|
||||
};
|
||||
return cp;
|
||||
}
|
||||
|
||||
function renderGatewayKeys() {
|
||||
const box = $("#gatewayKeys");
|
||||
$("#gatewayKeysBlock").hidden = !providers?.gateway.lan;
|
||||
box.replaceChildren();
|
||||
$("#addGatewayKey").onclick = () => { gatewayKeyDraft = ""; renderGatewayKeys(); };
|
||||
if (gatewayKeys === null) return;
|
||||
const list = el("div", "accts");
|
||||
for (const k of gatewayKeys) {
|
||||
const row = el("div", "acc" + (k.off ? " off" : " in-use"));
|
||||
row.dataset.key = k.id;
|
||||
const tick = el("button", "dot tick");
|
||||
tick.title = t(k.off ? "Enable key" : "Disable key");
|
||||
tick.setAttribute("aria-label", tick.title);
|
||||
if (!k.off) tick.append(svg(CHECK, 10, 2.2));
|
||||
tick.onclick = () => gatewayKeyAction(k.off ? "on-key" : "off-key", { key: k.id });
|
||||
const rm = el("button", "text quiet", t("Remove"));
|
||||
rm.onclick = () => askGatewayKey(k, false);
|
||||
const rotate = el("button", "text quiet", t("Rotate key"));
|
||||
rotate.onclick = () => askGatewayKey(k, true);
|
||||
row.append(tick, gatewayRename(k), el("span", "plan mono", k.masked), el("span", "grow"), copyCallerKeyBtn(k), rotate, rm);
|
||||
list.append(row);
|
||||
}
|
||||
if (gatewayKeyDraft !== null) list.append(gatewayKeyForm());
|
||||
else if (!gatewayKeys.length) list.append(el("div", "none", t("No gateway keys yet")));
|
||||
box.append(list);
|
||||
}
|
||||
|
||||
function askGatewayKey(k, rotate) {
|
||||
const ed = el("div", "editor");
|
||||
const head = el("div", "ehead");
|
||||
head.append(el("b", "", t(rotate ? "Rotate gateway key?" : "Remove gateway key?")));
|
||||
ed.append(head, el("p", "lib-confirm", t(rotate
|
||||
? "Clients using {name} will need the new key. Its name and usage history stay the same."
|
||||
: "Clients using {name} will lose access from other computers. Its usage history is kept.", { name: k.name })));
|
||||
const bar = el("div", "bar");
|
||||
const cancel = el("button", "text", t("Cancel"));
|
||||
cancel.onclick = (e) => { e.stopPropagation(); closeConfirmAsk(); };
|
||||
const go = el("button", "text primary", t(rotate ? "Rotate key" : "Remove"));
|
||||
go.onclick = async (e) => {
|
||||
e.stopPropagation();
|
||||
if (go.disabled) return;
|
||||
go.disabled = true;
|
||||
const out = await gatewayKeyAction(rotate ? "rotate-key" : "remove-key", { key: k.id });
|
||||
if (!out) { go.disabled = false; return; }
|
||||
closeConfirmAsk();
|
||||
if (rotate) status(t("Gateway key rotated. Update clients on other computers."), "ok");
|
||||
};
|
||||
bar.append(el("span", "grow"), cancel, go);
|
||||
ed.append(bar);
|
||||
confirmAsk = ed;
|
||||
openModal(ed);
|
||||
$("#modal").classList.add("lib");
|
||||
cancel.focus({ preventScroll: true });
|
||||
}
|
||||
@@ -4,6 +4,35 @@
|
||||
// works. {name} slots are filled by t(str, vars).
|
||||
const I18N = {
|
||||
zh: {
|
||||
"Gateway keys": "网关密钥",
|
||||
"Gateway key": "网关密钥",
|
||||
"All gateway keys": "全部网关密钥",
|
||||
"Use a gateway key to track usage": "使用网关密钥统计用量",
|
||||
"Choose a gateway key to use as {env}; usage is tracked by key.": "选择填入 {env} 的网关密钥,用量按密钥统计。",
|
||||
"This computer": "本机",
|
||||
"Any key · no key attribution": "任意密钥 · 不归属到具体密钥",
|
||||
"Choose a gateway key": "选择网关密钥",
|
||||
"Create a gateway key above to connect": "请先在上方创建网关密钥",
|
||||
"Loading gateway key…": "正在读取网关密钥…",
|
||||
"Local network · an enabled gateway key is required": "局域网接入需要已启用的网关密钥",
|
||||
"Gateway key rotated. Update clients on other computers.": "网关密钥已轮换,请更新其他电脑上的客户端。",
|
||||
"Add gateway key": "添加网关密钥",
|
||||
"Create a gateway key for each client. Usage is counted separately for each key.": "为每个客户端创建一个网关密钥,分别统计用量。",
|
||||
"Gateway key name": "网关密钥名称",
|
||||
"Key name, e.g. Laptop": "密钥名称,例如笔记本",
|
||||
"Create": "创建",
|
||||
"Gateway key created. Use Copy on its row to connect a client.": "网关密钥已创建。点击该行的复制按钮即可配置客户端。",
|
||||
"Copy gateway key": "复制网关密钥",
|
||||
"Rotate key": "轮换密钥",
|
||||
"No gateway keys yet": "尚无网关密钥",
|
||||
"Rotate gateway key?": "轮换网关密钥?",
|
||||
"Remove gateway key?": "移除网关密钥?",
|
||||
"Clients using {name} will need the new key. Its name and usage history stay the same.": "使用 {name} 的客户端需要更新密钥,名称和用量历史保持不变。",
|
||||
"Clients using {name} will lose access from other computers. Its usage history is kept.": "使用 {name} 的客户端将无法从其他电脑接入,用量历史会保留。",
|
||||
"Enable key": "启用密钥",
|
||||
"Disable key": "停用密钥",
|
||||
"Use a name between 1 and 120 characters": "请输入 1 到 120 个字符的名称",
|
||||
"Key not found": "密钥不存在",
|
||||
// header, nav, footer
|
||||
"Agents": "Agent",
|
||||
"Providers": "供应商",
|
||||
@@ -138,13 +167,12 @@ const I18N = {
|
||||
"Share on local network": "局域网共享",
|
||||
"{label} address": "{label} 地址",
|
||||
"Copied {label} address": "已复制 {label} 地址",
|
||||
"Agents on other computers on this network can use magpie’s models, with the API key below": "同一网络里其他电脑上的 Agent 可以用下面的 API Key 使用 magpie 的模型",
|
||||
"Agents on other computers on this network can use magpie’s models with a gateway key from Gateway": "同一网络里其他电脑上的 Agent 可以使用 magpie 的模型,网关密钥在网关中管理",
|
||||
"This computer has no local network address right now": "这台电脑现在没有局域网地址",
|
||||
"In a container": "在容器中运行",
|
||||
"Where this page was opened, on the gateway’s port; MAGPIE_PUBLIC_URL sets another": "即打开本页所用的地址,端口为网关的端口;可用 MAGPIE_PUBLIC_URL 改成别的",
|
||||
"The container’s own addresses, which other devices can’t reach: set MAGPIE_PUBLIC_URL to the host’s": "这是容器自己的地址,其他设备连不上:请把 MAGPIE_PUBLIC_URL 设为宿主机的地址",
|
||||
"New key": "换新 Key",
|
||||
"Other computers send it as their API key; a new one stops the old from working": "其他电脑把它填作 API Key;换新后旧的立即失效",
|
||||
"API keys, private keys, tokens and passwords go to vendors as placeholders, and come back as they were": "API Key、私钥、Token 和密码以占位符发给模型厂商,回复里再还原",
|
||||
"Mask personal data": "脱敏个人信息",
|
||||
"Count me as a user": "计入用户数",
|
||||
|
||||
@@ -89,6 +89,11 @@
|
||||
<div class="list gwcard">
|
||||
<div class="row gw" id="gateway"></div>
|
||||
</div>
|
||||
<section id="gatewayKeysBlock" hidden>
|
||||
<div class="row-head"><span class="label" data-t>Gateway keys</span><span class="grow"></span><button class="text" id="addGatewayKey" data-t>Add gateway key</button></div>
|
||||
<p class="usage-note" data-t>Create a gateway key for each client. Usage is counted separately for each key.</p>
|
||||
<section id="gatewayKeys"></section>
|
||||
</section>
|
||||
<div class="row-head"><button class="fold" id="foldConnect" aria-expanded="true" aria-controls="connect"><span class="label" data-t>Connect</span></button><span class="grow"></span><span class="note" id="connectNote"></span></div>
|
||||
<section class="list connect" id="connect"></section>
|
||||
<div class="row-head"><button class="fold" id="foldModels" aria-expanded="true" aria-controls="gwModels"><span class="label" data-t>Models</span><span class="note" id="modelsCount"></span></button><span class="grow"></span><button class="text" id="copyModels" data-t>Copy all ids</button></div>
|
||||
@@ -133,12 +138,15 @@
|
||||
<div class="list" id="usageAgents"></div>
|
||||
<div class="row-head"><span class="label" data-t>Models</span></div>
|
||||
<div class="list" id="usageModels"></div>
|
||||
<div class="row-head" id="usageKeysHead" hidden><span class="label" data-t>Gateway keys</span></div>
|
||||
<div class="list" id="usageKeys" hidden></div>
|
||||
<p class="usage-note" id="usageNote"></p>
|
||||
</div>
|
||||
<div class="usage-pane" id="ledgerPane" hidden>
|
||||
<div class="sess-tools">
|
||||
<button class="sess-pick" id="ledAgent" type="button" hidden></button>
|
||||
<button class="sess-pick" id="ledProvider" type="button" hidden></button>
|
||||
<button class="sess-pick" id="ledKey" type="button" hidden></button>
|
||||
<div class="segs" id="ledStatus"></div>
|
||||
<span id="ledRoute" hidden><span id="ledRouteLabel"></span><button class="text" id="ledRouteClear" type="button" data-tt="Clear filter" aria-label="Clear filter">×</button></span>
|
||||
<span class="grow"></span>
|
||||
@@ -349,6 +357,7 @@
|
||||
|
||||
<script src="compat.js"></script>
|
||||
<script src="i18n.js"></script>
|
||||
<script src="caller-keys.js"></script>
|
||||
<script src="app.js"></script>
|
||||
<script src="routing.js"></script>
|
||||
<script src="library.js"></script>
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
package gui
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
)
|
||||
|
||||
func callerKeyRoutes(mux *http.ServeMux) {
|
||||
mux.HandleFunc("GET /api/caller-keys", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
keys, err := access.List()
|
||||
if err != nil {
|
||||
fail(w, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, map[string]any{"keys": keys})
|
||||
})
|
||||
mux.HandleFunc("POST /api/caller-keys/{action}", func(w http.ResponseWriter, r *http.Request) {
|
||||
var in access.Change
|
||||
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
|
||||
fail(w, err)
|
||||
return
|
||||
}
|
||||
access.MigrateLegacyLANKeyBestEffort()
|
||||
secret, err := access.Update(r.PathValue("action"), in)
|
||||
if err != nil {
|
||||
fail(w, err)
|
||||
return
|
||||
}
|
||||
keys, err := access.List()
|
||||
if err != nil {
|
||||
fail(w, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
writeJSON(w, map[string]any{"keys": keys, "secret": secret})
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
package gui
|
||||
|
||||
import (
|
||||
"encoding/csv"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/usage"
|
||||
)
|
||||
|
||||
func TestCallerKeyManagementAndUsageRoutes(t *testing.T) {
|
||||
sandboxHome(t)
|
||||
mux := http.NewServeMux()
|
||||
callerKeyRoutes(mux)
|
||||
usageRoutes(mux, folderOnly{})
|
||||
type state struct {
|
||||
Keys []access.Key `json:"keys"`
|
||||
Secret string `json:"secret"`
|
||||
}
|
||||
request := func(method, path, body string, out any) string {
|
||||
t.Helper()
|
||||
w := httptest.NewRecorder()
|
||||
mux.ServeHTTP(w, httptest.NewRequest(method, path, strings.NewReader(body)))
|
||||
if w.Code != 200 {
|
||||
t.Fatalf("%s: %d %s", path, w.Code, w.Body)
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(w.Body.Bytes(), out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return w.Body.String()
|
||||
}
|
||||
var s state
|
||||
request("POST", "/api/caller-keys/add-key", `{"name":"Laptop"}`, &s)
|
||||
secret, laptop := s.Secret, s.Keys[0].ID
|
||||
request("POST", "/api/caller-keys/add-key", `{"name":"Server"}`, &s)
|
||||
server := s.Keys[1].ID
|
||||
list := request("GET", "/api/caller-keys", "", &s)
|
||||
if strings.Contains(list, secret) || strings.Contains(list, `"secret"`) {
|
||||
t.Fatal("list leaked credentials")
|
||||
}
|
||||
for _, rec := range []usage.Record{
|
||||
{RouteID: 123, ProviderKeyID: "upstream", ProviderKeyName: "Upstream", CallerKeyID: laptop, CallerKeyName: "Laptop", Input: 100},
|
||||
{RouteID: 456, CallerKeyID: laptop, CallerKeyName: "Laptop", Input: 10},
|
||||
{RouteID: 123, CallerKeyID: server, CallerKeyName: "Server", Input: 200},
|
||||
{Input: 40},
|
||||
} {
|
||||
rec.Time, rec.Provider, rec.Model, rec.Status = time.Now(), "relay", "unknown", 200
|
||||
usage.Append(rec)
|
||||
}
|
||||
request("POST", "/api/caller-keys/rename-key", `{"key":"`+laptop+`","name":"Main"}`, &s)
|
||||
var summary usageJSON
|
||||
request("GET", "/api/usage?period=all", "", &summary)
|
||||
if len(summary.CallerKeys) != 2 || summary.Calls != 4 {
|
||||
t.Fatal(summary.CallerKeys)
|
||||
}
|
||||
for _, g := range summary.CallerKeys {
|
||||
if g.CallerKeyID == laptop && g.Name != "Main" {
|
||||
t.Fatal(g)
|
||||
}
|
||||
}
|
||||
var ledger ledgerJSON
|
||||
query := "?period=all&callerKey=" + laptop
|
||||
request("GET", "/api/usage/requests"+query, "", &ledger)
|
||||
if ledger.Total != 2 || ledger.Input != 110 || ledger.Rows[0].CallerKeyLabel != "Main" || len(ledger.CallerKeys) != 2 {
|
||||
t.Fatal(ledger)
|
||||
}
|
||||
raw := request("GET", "/api/usage/requests.csv"+query, "", nil)
|
||||
cells, err := csv.NewReader(strings.NewReader(raw)).ReadAll()
|
||||
if err != nil || len(cells) != 3 || cells[1][slices.Index(usage.CSVHeader, "caller_key_id")] != laptop || cells[2][slices.Index(usage.CSVHeader, "caller_key_id")] != laptop || strings.Contains(raw, secret) {
|
||||
t.Fatal("CSV", err, raw)
|
||||
}
|
||||
request("GET", "/api/usage/requests"+query+"&route=123", "", &ledger)
|
||||
if ledger.Total != 1 || ledger.Input != 100 || ledger.Rows[0].RouteID != 123 || ledger.Rows[0].ProviderKeyID != "upstream" || ledger.Rows[0].CallerKeyID != laptop {
|
||||
t.Fatal("combined route and caller filter", ledger)
|
||||
}
|
||||
raw = request("GET", "/api/usage/requests.csv"+query+"&route=123", "", nil)
|
||||
cells, err = csv.NewReader(strings.NewReader(raw)).ReadAll()
|
||||
if err != nil || len(cells) != 2 || cells[1][slices.Index(usage.CSVHeader, "route_id")] != "123" || cells[1][slices.Index(usage.CSVHeader, "provider_key_id")] != "upstream" || cells[1][slices.Index(usage.CSVHeader, "caller_key_id")] != laptop || strings.Contains(raw, secret) {
|
||||
t.Fatal("combined-filter CSV", err, raw)
|
||||
}
|
||||
request("POST", "/api/caller-keys/rotate-key", `{"key":"`+laptop+`"}`, &s)
|
||||
if s.Secret == "" || s.Secret == secret || s.Keys[0].ID != laptop || s.Keys[0].Name != "Main" {
|
||||
t.Fatal("rotation lost identity")
|
||||
}
|
||||
if _, ok := access.Authenticate(secret); ok {
|
||||
t.Fatal("rotated key still authenticates")
|
||||
}
|
||||
request("GET", "/api/usage/requests"+query, "", &ledger)
|
||||
if ledger.Total != 2 || ledger.Input != 110 || ledger.Rows[0].CallerKeyLabel != "Main" {
|
||||
t.Fatal("rotation lost usage history", ledger)
|
||||
}
|
||||
request("POST", "/api/caller-keys/remove-key", `{"key":"`+laptop+`"}`, &s)
|
||||
ledger = ledgerJSON{}
|
||||
request("GET", "/api/usage/requests"+query, "", &ledger)
|
||||
if ledger.Total != 2 || ledger.Rows[0].CallerKeyLabel != "Laptop" {
|
||||
t.Fatal("historical name", ledger)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
mux.ServeHTTP(w, httptest.NewRequest("POST", "/api/caller-keys/off-key", strings.NewReader(`{"key":"missing"}`)))
|
||||
if w.Code != 400 {
|
||||
t.Fatal("unknown key accepted", w.Code)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package gui
|
||||
|
||||
import (
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
func TestReadOnlyLANMigrationDoesNotBlockSettingsOrKeys(t *testing.T) {
|
||||
sandboxHome(t)
|
||||
s := settings.Settings{LAN: true, LANKey: "sk-magpie-fixture-readonly"}
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chmod(settings.Path(), 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { os.Chmod(settings.Path(), 0o600) })
|
||||
if err := settings.Save(s); err == nil {
|
||||
t.Skip("settings.json remains writable")
|
||||
}
|
||||
h := Handler(nil, nil)
|
||||
for _, path := range []string{"/api/settings", "/api/caller-keys", "/api/settings"} {
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, httptest.NewRequest("GET", path, nil))
|
||||
if w.Code != 200 {
|
||||
t.Fatal(path, w.Code, w.Body)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package gui
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/gateway"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
"github.com/yetone/magpie/internal/usage"
|
||||
)
|
||||
|
||||
func TestLANSettingSharesCallerKeyStore(t *testing.T) {
|
||||
sandboxHome(t)
|
||||
handler := Handler(nil, nil)
|
||||
post := func(path, body string) map[string]any {
|
||||
t.Helper()
|
||||
w := httptest.NewRecorder()
|
||||
handler.ServeHTTP(w, httptest.NewRequest(http.MethodPost, path, strings.NewReader(body)))
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("%s: %d %s", path, w.Code, w.Body)
|
||||
}
|
||||
var out map[string]any
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
s := post("/api/settings/lan", `{"on":true}`)
|
||||
if s["lan"] != true || s["lanKey"] != nil {
|
||||
t.Fatal("setting generated separate key", s)
|
||||
}
|
||||
keys, _ := access.List()
|
||||
key := keys[0]
|
||||
id := key.ID
|
||||
if keys, _ := access.List(); len(keys) != 1 || keys[0].ID != id || keys[0].Name != "Magpie" {
|
||||
t.Fatal("LAN shortcut is not a named key", keys)
|
||||
}
|
||||
old, err := access.Update("copy-key", access.Change{Key: id})
|
||||
if err != nil || strings.Contains(key.Masked, old) {
|
||||
t.Fatal("LAN settings leaked secret", key, err)
|
||||
}
|
||||
created := post("/api/caller-keys/add-key", `{"name":"Remote laptop"}`)
|
||||
secret := created["secret"].(string)
|
||||
if who, ok := access.Authenticate(secret); !ok || who.KeyName != "Remote laptop" {
|
||||
t.Fatal("named key is not shared", who, ok)
|
||||
}
|
||||
off := post("/api/settings/lan", `{"on":false,"newKey":true}`)
|
||||
if off["lan"] != nil || off["lanKey"] != nil {
|
||||
t.Fatal("legacy rotate request changed settings", off)
|
||||
}
|
||||
if _, ok := access.Authenticate(secret); !ok {
|
||||
t.Fatal("toggling sharing revoked a key")
|
||||
}
|
||||
post("/api/settings/lan", `{"on":true}`)
|
||||
if _, ok := access.Authenticate(old); !ok || settings.Load().LANKeyID != id {
|
||||
t.Fatal("toggling sharing replaced the LAN key")
|
||||
}
|
||||
post("/api/caller-keys/rename-key", `{"key":"`+id+`","name":"Desk"}`)
|
||||
usage.Append(usage.Record{Time: time.Now(), CallerKeyID: id, CallerKeyName: "Magpie", Input: 30, Status: 200})
|
||||
post("/api/settings/lan", `{"on":true,"newKey":true}`)
|
||||
keys, _ = access.List()
|
||||
key = keys[0]
|
||||
if key.ID != id || key.Name != "Desk" || key.Secret != "" {
|
||||
t.Fatal("rotation lost key identity or leaked secret", key)
|
||||
}
|
||||
if _, ok := access.Authenticate(old); ok {
|
||||
t.Fatal("rotated credential still works")
|
||||
}
|
||||
next, err := access.Update("copy-key", access.Change{Key: id})
|
||||
if err != nil || next == old {
|
||||
t.Fatal("key was not rotated", err)
|
||||
}
|
||||
if who, ok := access.Authenticate(next); !ok || who.KeyID != id {
|
||||
t.Fatal("rotated key is invalid", who, ok)
|
||||
}
|
||||
if _, ok := access.Authenticate(secret); !ok {
|
||||
t.Fatal("rotating LAN revoked another named key")
|
||||
}
|
||||
if recs := usage.Load(time.Time{}); len(recs) != 1 || recs[0].CallerKeyID != id || recs[0].Input != 30 {
|
||||
t.Fatal("rotation changed historical usage", recs)
|
||||
}
|
||||
post("/api/settings", `{"theme":"dark"}`)
|
||||
if settings.Load().LANKeyID != id {
|
||||
t.Fatal("saving unrelated preferences lost LAN key reference")
|
||||
}
|
||||
post("/api/caller-keys/off-key", `{"key":"`+id+`"}`)
|
||||
post("/api/settings/lan", `{"on":true,"newKey":true}`)
|
||||
keys, _ = access.List()
|
||||
if !keys[0].Off {
|
||||
t.Fatal("rotation re-enabled a disabled key")
|
||||
}
|
||||
post("/api/caller-keys/remove-key", `{"key":"`+id+`"}`)
|
||||
w := httptest.NewRecorder()
|
||||
handler.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/api/settings", nil))
|
||||
if keys, _ := access.List(); len(keys) != 1 {
|
||||
t.Fatal("settings resurrected a deleted key", w.Body)
|
||||
}
|
||||
post("/api/settings/lan", `{"on":true,"newKey":true}`)
|
||||
if settings.Load().LANKeyID == id {
|
||||
t.Fatal("replacement reused a deleted identity")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSettingsMigratesOldLANCredentialToManagedKey(t *testing.T) {
|
||||
sandboxHome(t)
|
||||
old := "sk-magpie-0123456789abcdef0123456789abcdef01234567"
|
||||
s := settings.Load()
|
||||
s.LAN, s.LANKey = true, old
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
handler := Handler(nil, nil)
|
||||
w := httptest.NewRecorder()
|
||||
handler.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/api/settings", nil))
|
||||
if w.Code != 200 || strings.Contains(w.Body.String(), old) || strings.Contains(w.Body.String(), `"lanKey"`) {
|
||||
t.Fatal("old credential leaked", w.Code, w.Body)
|
||||
}
|
||||
if settings.Load().LANKey != old {
|
||||
t.Fatal("migration did not retain the old credential for older Magpie")
|
||||
}
|
||||
keys, _ := access.List()
|
||||
if len(keys) != 1 || keys[0].Name != "Magpie" || settings.Load().LANKeyID != keys[0].ID {
|
||||
t.Fatal(keys)
|
||||
}
|
||||
who, ok := access.Authenticate(old)
|
||||
if !ok || who.KeyID != keys[0].ID {
|
||||
t.Fatal(who, ok)
|
||||
}
|
||||
w = httptest.NewRecorder()
|
||||
handler.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/api/caller-keys", nil))
|
||||
if w.Code != 200 || strings.Contains(w.Body.String(), old) {
|
||||
t.Fatal("list exposed credential", w.Code, w.Body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayConnectionAddressesFollowLANSharing(t *testing.T) {
|
||||
sandboxHome(t)
|
||||
s := settings.Load()
|
||||
for _, on := range []bool{false, true, false} {
|
||||
s.LAN = on
|
||||
if err := settings.Save(s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g := providersState().Gateway
|
||||
if g.LAN != on {
|
||||
t.Fatal("key block visibility does not follow sharing", g.LAN)
|
||||
}
|
||||
if !on && len(g.LANURLs) != 0 {
|
||||
t.Fatal("shared addresses remain after sharing is off", g.LANURLs)
|
||||
}
|
||||
if on && !slices.Equal(g.LANURLs, gateway.LANURLs()) {
|
||||
t.Fatal("Connect does not list the gateway's network addresses", g.LANURLs)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
"github.com/yetone/magpie/internal/catalog"
|
||||
"github.com/yetone/magpie/internal/gateway"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
"github.com/yetone/magpie/internal/settings"
|
||||
)
|
||||
|
||||
// The providers page: the vendors the user added, the presets they can add
|
||||
@@ -155,6 +156,8 @@ type presetJSON struct {
|
||||
|
||||
type gatewayJSON struct {
|
||||
URL string `json:"url"`
|
||||
LAN bool `json:"lan"`
|
||||
LANURLs []string `json:"lanURLs,omitempty"`
|
||||
Running bool `json:"running"`
|
||||
Mine bool `json:"mine"` // this process serves it
|
||||
Window bool `json:"window"` // the magpie serving it shows its routing
|
||||
@@ -429,6 +432,10 @@ func providersState() providersJSON {
|
||||
}
|
||||
cat := provider.Catalog()
|
||||
s.Gateway = gatewayJSON{URL: gateway.URL(), Models: len(cat), Calls: []gateway.Call{}, Groups: []gwGroupJSON{}}
|
||||
s.Gateway.LAN = settings.Load().LAN
|
||||
if s.Gateway.LAN {
|
||||
s.Gateway.LANURLs = gateway.LANURLs()
|
||||
}
|
||||
for _, e := range cat {
|
||||
if e.Group == "" {
|
||||
continue
|
||||
|
||||
@@ -78,6 +78,7 @@ func TestSettingsSaveKeepsWhatItDoesNotSend(t *testing.T) {
|
||||
RedactRules: []redact.Rule{{Kind: "prefix", Prefix: "oc_sk_"}},
|
||||
LAN: true,
|
||||
LANKey: "sk-lan",
|
||||
LANKeyID: "lan-key-id",
|
||||
RequestArchive: true,
|
||||
QuotaLeft: true,
|
||||
PlainNames: true,
|
||||
|
||||
@@ -1,5 +1,52 @@
|
||||
# Dropdown browser regression
|
||||
|
||||
## Gateway Caller Keys
|
||||
|
||||
`gateway-caller-keys.test.cjs` checks the named caller-key list on the
|
||||
Gateway page, including creation, copying, renaming, disabling, enabling,
|
||||
rotation and deletion. It verifies key-level usage overview, request filtering and
|
||||
CSV export in Chinese and English on Chromium and WebKit. The fixtures do
|
||||
not access local credentials. Gateway keys appear only while sharing is on, with one create entry point
|
||||
and confirmation before rotation/removal (Cancel and Escape send no mutation).
|
||||
Creation, renaming and confirmation focus their controls without scrolling.
|
||||
Gateway is the only key-management page;
|
||||
Settings controls sharing and shows addresses. The test checks that toggling
|
||||
sharing retains the key, and that a removed default key is recreated as
|
||||
Magpie and appears in Gateway without a reload.
|
||||
|
||||
`gateway-connect-keys.test.cjs` selects loopback and LAN addresses and enabled
|
||||
caller keys for Shell, curl, Python and Node examples across all four APIs.
|
||||
It checks rotation, disabling, removal, stale list responses, empty states,
|
||||
literal custom names and narrow layouts in Chinese and English on both engines.
|
||||
Sharing off keeps the original API-key field and hides the gateway-key picker;
|
||||
sharing on names the arbitrary local option separately from the Magpie key.
|
||||
|
||||
`api-key-usage.test.cjs` exercises the existing provider key list: adding,
|
||||
enabling and disabling, choosing the first key, renaming and removing.
|
||||
The Usage page's Gateway key rows, request filter and CSV exports identify
|
||||
client keys, not those provider credentials. A client using different
|
||||
upstream keys stays grouped together; renaming an upstream key does not
|
||||
rename the client. It also checks historical records, Chinese and English,
|
||||
and the narrow window in Chromium and WebKit.
|
||||
The API fixtures use test keys and never read local user configuration.
|
||||
The existing `usage-ledger.test.cjs` also checks request-route and caller-key
|
||||
filters together, CSV export, and restoring the previous caller filter when
|
||||
the request filter is cleared. A caller absent from the period's options clears
|
||||
the filter and reloads the unfiltered rows from the first page.
|
||||
|
||||
`api-key-theme.test.cjs` checks Gateway's enabled and disabled keys, creation
|
||||
and renaming inputs, rotation and key picker, plus Settings' LAN address
|
||||
controls against the global palette.
|
||||
Light and Dark override the OS; System follows live OS palette changes.
|
||||
The settings theme picker is also switched and reloaded in Chromium and
|
||||
WebKit. No separate colours are defined for gateway keys.
|
||||
|
||||
```sh
|
||||
node --test --test-concurrency=1 internal/gui/tests/gateway-caller-keys.test.cjs internal/gui/tests/gateway-connect-keys.test.cjs internal/gui/tests/api-key-usage.test.cjs internal/gui/tests/api-key-theme.test.cjs
|
||||
```
|
||||
|
||||
## Other Browser Regressions
|
||||
|
||||
`routing-sessions.test.cjs` checks the Routing request list in Chromium and
|
||||
WebKit, English and Chinese: the list defaults to By request and remembers
|
||||
the grouping choice across reloads; sessions are separated by agent and ID across
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const { test } = require("node:test");
|
||||
const { chromium, webkit } = require("playwright");
|
||||
const { fixture } = require("./fixtures/caller-keys.cjs");
|
||||
|
||||
async function palette(page, selectors) {
|
||||
return page.evaluate((selectors) => {
|
||||
const root = getComputedStyle(document.documentElement);
|
||||
const rgb = (name) => {
|
||||
const hex = root.getPropertyValue(name).trim();
|
||||
return `rgb(${[1, 3, 5].map((i) => parseInt(hex.slice(i, i + 2), 16)).join(", ")})`;
|
||||
};
|
||||
const colors = Object.fromEntries(["bg", "card", "card-2", "fg", "fg-2", "muted", "pill", "line", "accent", "accent-fg"]
|
||||
.map((name) => [name, rgb("--" + name)]));
|
||||
const elements = Object.fromEntries(Object.entries(selectors).map(([name, selector]) => {
|
||||
const s = getComputedStyle(document.querySelector(selector));
|
||||
return [name, { color: s.color, bg: s.backgroundColor, border: s.borderTopColor }];
|
||||
}));
|
||||
return { colors, elements };
|
||||
}, selectors);
|
||||
}
|
||||
|
||||
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
|
||||
for (const theme of (process.env.THEME ? [process.env.THEME] : ["light", "dark", "system"])) {
|
||||
test(`${engine} ${theme}: API key controls follow the global palette`, async (t) => {
|
||||
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
|
||||
t.after(() => browser.close());
|
||||
const context = await browser.newContext({ viewport: { width: 1000, height: 760 }, reducedMotion: "reduce", colorScheme: "light" });
|
||||
const page = await context.newPage();
|
||||
page.setDefaultTimeout(6000);
|
||||
const events = [], errors = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await page.route("**/*", fixture("en", theme, events, { lan: true }));
|
||||
await page.goto("http://magpie.test/?view=gateway");
|
||||
await page.locator('#gatewayKeys .acc[data-key="server"] .dot').click();
|
||||
await page.locator('#gatewayKeys .acc.off[data-key="server"]').waitFor();
|
||||
await page.locator("#addGatewayKey").click();
|
||||
await page.locator('#gatewayKeys .acc[data-key="laptop"] .rename').click();
|
||||
await page.locator("#gatewayKeys .rename-in").waitFor();
|
||||
await page.mouse.move(500, 20);
|
||||
const gatewaySelectors = { card: "#gatewayKeys .accts", enabled: '#gatewayKeys .acc[data-key="work"] .dot',
|
||||
disabled: "#gatewayKeys .acc.off .dot", masked: "#gatewayKeys .acc .plan", form: "#gatewayKeys .adding",
|
||||
input: "#gatewayKeys .adding input", rename: "#gatewayKeys .rename-in", create: "#gatewayKeys .primary", rotate: '#gatewayKeys .acc[data-key="work"] .text', keyPick: "#connectKey" };
|
||||
const gatewayPaints = [];
|
||||
for (const scheme of ["light", "dark"]) {
|
||||
await page.emulateMedia({ colorScheme: scheme });
|
||||
await page.waitForTimeout(200);
|
||||
const { colors: c, elements: e } = await palette(page, gatewaySelectors);
|
||||
assert.equal(e.card.bg, c.card);
|
||||
assert.equal(e.card.border, c.line);
|
||||
assert.equal(e.enabled.bg, c.fg);
|
||||
assert.equal(e.enabled.color, c.card);
|
||||
assert.equal(e.disabled.color, c.muted);
|
||||
assert.equal(e.masked.bg, c.pill);
|
||||
assert.equal(e.masked.color, c.muted);
|
||||
assert.equal(e.form.bg, c["card-2"]);
|
||||
for (const input of [e.input, e.rename]) {
|
||||
assert.equal(input.bg, c.card, "key inputs must not use the browser's default background");
|
||||
assert.equal(input.color, c.fg);
|
||||
}
|
||||
assert.equal(e.rotate.color, c.muted);
|
||||
assert.equal(e.keyPick.bg, "rgba(0, 0, 0, 0)");
|
||||
assert.equal(e.create.bg, c.accent);
|
||||
assert.equal(e.create.color, c["accent-fg"]);
|
||||
gatewayPaints.push(c.bg);
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${theme}-${scheme}-keys.png`) });
|
||||
}
|
||||
}
|
||||
assert.equal(gatewayPaints[0] === gatewayPaints[1], theme !== "system", "only System follows an OS palette change");
|
||||
|
||||
await page.locator('#gatewayKeys .acc[data-key="work"]').getByRole("button", { name: "Rotate key", exact: true }).click();
|
||||
await page.locator("#modal .lib-confirm").waitFor();
|
||||
for (const scheme of ["light", "dark"]) {
|
||||
await page.emulateMedia({ colorScheme: scheme });
|
||||
await page.waitForTimeout(200);
|
||||
const { colors: c, elements: e } = await palette(page, {
|
||||
dialog: "#modal .dialog", message: "#modal .lib-confirm", confirm: "#modal .primary",
|
||||
});
|
||||
assert.equal(e.dialog.bg, c.card);
|
||||
assert.equal(e.message.color, c["fg-2"]);
|
||||
assert.equal(e.confirm.bg, c.accent);
|
||||
assert.equal(e.confirm.color, c["accent-fg"]);
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${theme}-${scheme}-confirm.png`) });
|
||||
}
|
||||
}
|
||||
await page.locator("#modal").getByRole("button", { name: "Cancel", exact: true }).click();
|
||||
await page.locator("#modal").waitFor({ state: "hidden" });
|
||||
assert.equal(events.filter((e) => e.action === "rotate-key").length, 0);
|
||||
|
||||
await page.goto("http://magpie.test/?view=settings");
|
||||
const on = page.locator("#lanList").getByRole("button", { name: "On", exact: true });
|
||||
await on.waitFor();
|
||||
const bounds = await on.boundingBox();
|
||||
await page.mouse.move(500, 400);
|
||||
await page.mouse.wheel(0, bounds.y - 250);
|
||||
await on.click();
|
||||
await page.locator("#lanList .lan-url").waitFor();
|
||||
await page.mouse.move(500, 20);
|
||||
for (const scheme of ["light", "dark"]) {
|
||||
await page.emulateMedia({ colorScheme: scheme });
|
||||
await page.waitForTimeout(200);
|
||||
const { colors: c, elements: e } = await palette(page, { card: "#lanList", text: "#lanList .row.pref:last-child .name",
|
||||
key: "#lanList .lan-url", copy: "#lanList .row.pref:last-child .copy" });
|
||||
assert.equal(e.card.bg, c.card);
|
||||
assert.equal(e.text.color, c.fg);
|
||||
assert.equal(e.key.color, c["fg-2"]);
|
||||
assert.equal(e.copy.color, c.muted);
|
||||
}
|
||||
if (theme === "system") {
|
||||
await page.mouse.move(500, 400);
|
||||
await page.mouse.wheel(0, -5000);
|
||||
for (const choice of ["Dark", "Light", "System"]) {
|
||||
await page.locator("#themeSegs").getByRole("button", { name: choice, exact: true }).click();
|
||||
await page.waitForFunction((want) => (document.documentElement.dataset.theme || "system") === want, choice.toLowerCase());
|
||||
await page.waitForFunction(() => !document.documentElement.classList.contains("theming"));
|
||||
const { colors: c, elements: e } = await palette(page, { card: "#lanList", key: "#lanList .lan-url" });
|
||||
assert.equal(e.card.bg, c.card);
|
||||
assert.equal(e.key.color, c["fg-2"]);
|
||||
}
|
||||
await page.reload();
|
||||
await page.locator("#themeSegs button.on").waitFor();
|
||||
assert.equal(await page.locator("#themeSegs button.on").textContent(), "System");
|
||||
}
|
||||
assert.deepEqual(errors, []);
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
// The real provider editor and Usage page, with isolated API fixtures.
|
||||
// Provider keys stay manageable; Usage's gateway keys identify the calling client.
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const crypto = require("node:crypto");
|
||||
const { test } = require("node:test");
|
||||
const { chromium, webkit } = require("playwright");
|
||||
|
||||
const assets = path.resolve(__dirname, "../assets");
|
||||
const fingerprint = (key) => crypto.createHash("sha256").update(key).digest("hex").slice(0, 10);
|
||||
const PERSONAL = fingerprint("test-personal"), TEAM = fingerprint("test-team");
|
||||
const totals = (rows) => ({
|
||||
calls: rows.length, errors: rows.filter((r) => r.status >= 400).length,
|
||||
input: rows.reduce((n, r) => n + r.in, 0), output: rows.reduce((n, r) => n + r.out, 0),
|
||||
cache_read: rows.reduce((n, r) => n + (r.cache_read || 0), 0), cache_write: 0, reasoning: 0,
|
||||
cost: rows.reduce((n, r) => n + r.cost, 0), unpriced: 0,
|
||||
});
|
||||
function fixture(lang, theme, posts, requests) {
|
||||
const relay = {
|
||||
id: "relay", name: "Relay", icon: "generic", chat: "https://relay.example/v1", responses: "", anthropic: "",
|
||||
models: [{ id: "m", name: "Model", on: true }], agents: [], fallback: [], headers: {}, routing: "", catalog: "",
|
||||
key: { set: true, masked: "test-…onal" }, balanceToken: { takes: false, set: false },
|
||||
keyList: [
|
||||
{ id: PERSONAL, name: "Personal", masked: "test-…onal", active: true, on: true },
|
||||
{ id: TEAM, name: "Team", masked: "test-…team", active: false, on: true },
|
||||
],
|
||||
};
|
||||
const providers = { providers: [relay], presets: [], excluded: [], gateway: { running: true, window: true } };
|
||||
const rows = [
|
||||
{ callerKeyId: "server", callerKeyName: "Server", in: 2000, out: 200, cost: 0.4, cache_read: 1000, status: 200, agent: "claude", agentName: "Claude Code" },
|
||||
{ callerKeyId: "laptop", callerKeyName: "Laptop", in: 1000, out: 100, cost: 0.2, status: 200, agent: "codex", agentName: "Codex" },
|
||||
{ callerKeyId: "laptop", callerKeyName: "Laptop", in: 0, out: 0, cost: 0, status: 429, agent: "codex", agentName: "Codex" },
|
||||
{ in: 50, out: 5, cost: 0.01, status: 200, agent: "codex", agentName: "Codex" },
|
||||
].map((r, i) => ({ t: new Date(Date.now() - i * 60e3).toISOString(), provider: "relay", providerName: "Relay", host: "relay.example",
|
||||
model: "m", req: "relay/m", ms: 1000, priced: true, icon: "generic", ...r }));
|
||||
const ledger = (q) => {
|
||||
let filtered = rows;
|
||||
if (q.get("callerKey")) filtered = filtered.filter((r) => r.callerKeyId === q.get("callerKey"));
|
||||
if (q.get("agent")) filtered = filtered.filter((r) => r.agent === q.get("agent"));
|
||||
if (q.get("failed") === "1") filtered = filtered.filter((r) => r.status >= 400);
|
||||
if (q.get("q")) filtered = filtered.filter((r) => (r.callerKeyName || r.model).toLowerCase().includes(q.get("q").toLowerCase()));
|
||||
const offset = +q.get("offset") || 0;
|
||||
return { ...totals(filtered), total: filtered.length, offset, period: q.get("period"),
|
||||
rows: filtered.slice(offset, offset + (+q.get("limit") || 100)),
|
||||
callerKeys: callerKeys(), agents: [{ id: "codex", name: "Codex" }, { id: "claude", name: "Claude Code" }] };
|
||||
};
|
||||
const callerKeys = () => ["server", "laptop"].map((id) => {
|
||||
const rs = rows.filter((r) => r.callerKeyId === id);
|
||||
return { id, callerKeyId: id, name: rs[0].callerKeyName, icon: "generic", ...totals(rs) };
|
||||
});
|
||||
return async (route) => {
|
||||
const req = route.request(), url = new URL(req.url());
|
||||
const json = (data) => route.fulfill({ json: data });
|
||||
if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = ${JSON.stringify({ lang, theme, web: false })};` });
|
||||
if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
|
||||
if (url.pathname === "/api/state") return json({ agents: [], profiles: [], settings: { lang, theme } });
|
||||
if (url.pathname === "/api/providers") return json(providers);
|
||||
if (url.pathname === "/api/groups") return json({ groups: [], models: [] });
|
||||
if (url.pathname.startsWith("/api/keys/")) {
|
||||
const action = url.pathname.split("/").at(-1), body = req.postDataJSON();
|
||||
posts.push({ action, body });
|
||||
const key = relay.keyList.find((k) => k.id === body.ref);
|
||||
if (action === "add") relay.keyList.push({ id: fingerprint(body.key), name: body.name, masked: "test-…new", on: true, active: false });
|
||||
if (action === "rename") key.name = body.name;
|
||||
if (action === "off" || action === "on") key.on = action === "on";
|
||||
if (action === "use") { for (const k of relay.keyList) k.active = k === key; }
|
||||
if (action === "remove") relay.keyList = relay.keyList.filter((k) => k !== key);
|
||||
return json(providers);
|
||||
}
|
||||
if (url.pathname === "/api/usage/quotas") return json([]);
|
||||
if (url.pathname === "/api/usage") return json({ ...totals(rows), callerKeys: callerKeys(), agents: [], models: [], series: [], bucket: "day", path: "~/.config/magpie/usage.jsonl" });
|
||||
if (url.pathname === "/api/usage/requests") { requests.push(url.searchParams); return json(ledger(url.searchParams)); }
|
||||
if (url.pathname === "/api/usage/requests/export") {
|
||||
requests.push(Object.assign(url.searchParams, { method: req.method() }));
|
||||
return json({ path: "~/Downloads/keys.csv", rows: ledger(url.searchParams).total });
|
||||
}
|
||||
if (url.pathname.startsWith("/api/")) return json({});
|
||||
const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);
|
||||
await route.fulfill({ body: await fs.readFile(file), contentType: { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml" }[path.extname(file)] });
|
||||
};
|
||||
}
|
||||
|
||||
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
|
||||
for (const lang of ["en", "zh"]) {
|
||||
test(`${engine} ${lang}: manage keys and inspect their separate usage`, async (t) => {
|
||||
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
|
||||
t.after(() => browser.close());
|
||||
const page = await (await browser.newContext({ viewport: { width: 1000, height: 740 }, reducedMotion: "reduce" })).newPage();
|
||||
page.setDefaultTimeout(5000);
|
||||
const errors = [], posts = [], requests = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await page.route("**/*", fixture(lang, "light", posts, requests));
|
||||
const w = lang === "zh"
|
||||
? { addButton: "添加", first: "设为首选", remove: "移除", failed: "失败", all: "全部网关密钥" }
|
||||
: { addButton: "Add", first: "Make first", remove: "Remove", failed: "Failed", all: "All gateway keys" };
|
||||
await page.goto("http://magpie.test/?view=providers");
|
||||
await page.locator(".row.provider", { hasText: "Relay" }).click();
|
||||
assert.equal(await page.locator(".accts .acc:not(.add)").count(), 2);
|
||||
await page.locator(".accts .acc.add").click();
|
||||
await page.locator(".acc.adding .kf input").nth(0).fill("Backup");
|
||||
await page.locator(".acc.adding .kf input").nth(1).fill("test-new");
|
||||
await page.locator(".acc.adding").getByRole("button", { name: w.addButton, exact: true }).click();
|
||||
await page.locator(".accts .acc", { hasText: "Backup" }).waitFor();
|
||||
assert.equal(posts.at(-1).action, "add");
|
||||
assert.equal(posts.at(-1).body.key, "test-new");
|
||||
const backup = () => page.locator(".accts .acc", { hasText: "Backup" });
|
||||
await backup().locator(".dot.tick").click();
|
||||
await page.waitForFunction(() => [...document.querySelectorAll(".accts .acc.off")].some((r) => r.textContent.includes("Backup")));
|
||||
assert.equal(posts.at(-1).action, "off");
|
||||
await backup().locator(".dot.tick").click();
|
||||
await page.waitForFunction(() => [...document.querySelectorAll(".accts .acc.in-use")].some((r) => r.textContent.includes("Backup")));
|
||||
const team = () => page.locator(".accts .acc", { hasText: "Team" });
|
||||
await team().getByRole("button", { name: w.first, exact: true }).click();
|
||||
await team().locator(".using").waitFor();
|
||||
assert.equal(posts.at(-1).body.ref, TEAM);
|
||||
await team().locator(".rename").click();
|
||||
await page.locator(".rename-in").fill("Workspace");
|
||||
await page.locator(".rename-in").press("Enter");
|
||||
await page.locator(".accts .acc .rename", { hasText: "Workspace" }).waitFor();
|
||||
assert.equal(posts.at(-1).action, "rename");
|
||||
await backup().getByRole("button", { name: w.remove, exact: true }).click();
|
||||
await page.waitForFunction(() => ![...document.querySelectorAll(".accts .acc")].some((r) => r.textContent.includes("Backup")));
|
||||
|
||||
await page.locator('nav [data-view="usage"]').click();
|
||||
await page.locator("#usageKeys .row").first().waitFor();
|
||||
assert.deepEqual(await page.locator("#usageKeys .name").allTextContents(), ["Server", "Laptop"]);
|
||||
assert.deepEqual(await page.locator("#usageKeys .num b").allTextContents(), ["2.2K", "1.1K"]);
|
||||
assert.deepEqual(await page.locator("#usageKeys .cost").allTextContents(), ["≈$0.400", "≈$0.200"]);
|
||||
assert(!/Workspace|Personal/.test(await page.locator("#usageKeys").textContent()));
|
||||
await page.locator("#usageTab .opt").nth(1).click();
|
||||
await page.locator(".led tbody tr").first().waitFor();
|
||||
assert.equal(await page.locator(".led tbody tr").first().locator("td").nth(3).textContent(), "Relay · relay.example");
|
||||
assert((await page.locator(".led tbody tr").first().locator("td").nth(3).getAttribute("title")).includes("relay.example"));
|
||||
await page.locator("#ledKey").click();
|
||||
await page.locator(".sess-menu .pm-item", { hasText: "Laptop" }).click();
|
||||
await page.waitForFunction(() => document.querySelectorAll(".led tbody tr").length === 2);
|
||||
assert.equal(requests.at(-1).get("callerKey"), "laptop");
|
||||
assert(!requests.at(-1).has("key"));
|
||||
assert.equal(requests.at(-1).get("offset"), "0");
|
||||
await page.locator("#ledStatus .opt", { hasText: w.failed }).click();
|
||||
await page.waitForFunction(() => document.querySelectorAll(".led tbody tr").length === 1);
|
||||
assert.equal(requests.at(-1).get("callerKey"), "laptop");
|
||||
assert.equal(requests.at(-1).get("failed"), "1");
|
||||
await page.locator("#ledExport").click();
|
||||
await page.waitForFunction(() => document.querySelector("#status").textContent.includes("keys.csv"));
|
||||
const exported = requests.findLast((q) => q.method === "POST");
|
||||
assert.equal(exported.get("callerKey"), "laptop");
|
||||
assert(!exported.has("key"));
|
||||
assert.equal(exported.get("failed"), "1");
|
||||
assert(!exported.has("offset") && !exported.has("limit"));
|
||||
await page.locator("#ledStatus .opt").first().click();
|
||||
await page.locator("#ledKey").click();
|
||||
await page.locator(".sess-menu .pm-item", { hasText: w.all }).click();
|
||||
await page.waitForFunction(() => document.querySelectorAll(".led tbody tr").length === 4);
|
||||
assert(!requests.at(-1).has("key") && !requests.at(-1).has("callerKey"));
|
||||
assert.equal(await page.locator(".led tbody tr").last().locator("td").nth(3).textContent(), "Relay · relay.example");
|
||||
await page.setViewportSize({ width: 560, height: 700 });
|
||||
assert(await page.locator("#view-usage").evaluate((v) => v.scrollWidth <= v.clientWidth));
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-api-key-usage.png`) });
|
||||
}
|
||||
assert.deepEqual(errors, []);
|
||||
});
|
||||
}
|
||||
}
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const assets = path.resolve(__dirname, "../../assets");
|
||||
|
||||
const sum = (rows) => ({
|
||||
calls: rows.length, errors: rows.filter((r) => r.status >= 400).length,
|
||||
input: rows.reduce((n, r) => n + r.in, 0), output: rows.reduce((n, r) => n + r.out, 0),
|
||||
cache_read: 0, cache_write: 0, reasoning: 0, cost: rows.reduce((n, r) => n + r.cost, 0), unpriced: 0,
|
||||
});
|
||||
|
||||
function fixture(lang, theme, events, options = {}) {
|
||||
let keys = [
|
||||
{ id: "laptop", name: "Laptop", masked: "sk-magpie-key-…111111" },
|
||||
{ id: "server", name: "Server", masked: "sk-magpie-key-…222222" },
|
||||
{ id: "work", name: "Work", masked: "sk-magpie-key-…333333" },
|
||||
];
|
||||
let serial = 0;
|
||||
let lan = !!options.lan;
|
||||
let lanKeyID = "", rotations = 0, lanSecret = "";
|
||||
const secrets = new Map([["laptop", "fixture-laptop"], ["server", "fixture-server"], ["work", "fixture-work"]]);
|
||||
const lanURLs = ["http://192.168.1.10:3999", "http://10.0.0.10:3999"];
|
||||
const lanState = () => ({ lang, theme, lan,
|
||||
lanURLs: lan ? lanURLs : [], fx: { rate: 7.2, at: new Date().toISOString() } });
|
||||
const rows = [
|
||||
{ callerKeyId: "laptop", callerKeyName: "Laptop", in: 100, out: 10, cost: 0.1 },
|
||||
{ callerKeyId: "server", callerKeyName: "Server", in: 200, out: 20, cost: 0.2 },
|
||||
{ callerKeyId: "work", callerKeyName: "Work", in: 300, out: 30, cost: 0.3 },
|
||||
{ in: 40, out: 4, cost: 0.04 },
|
||||
].map((r, i) => ({
|
||||
t: new Date(Date.now() - i * 60e3).toISOString(), agent: "codex", agentName: "Codex", icon: "generic", provider: "relay", providerName: "Relay",
|
||||
model: "m", req: "relay/m", ms: 1000, status: 200, priced: true, ...r,
|
||||
}));
|
||||
const keyName = (r) => keys.find((k) => k.id === r.callerKeyId)?.name || r.callerKeyName;
|
||||
const groups = () => ({
|
||||
callerKeys: ["laptop", "server", "work"].map((id) => {
|
||||
const rs = rows.filter((r) => r.callerKeyId === id);
|
||||
return { id, callerKeyId: id, name: keyName(rs[0]), icon: "generic", ...sum(rs) };
|
||||
}),
|
||||
});
|
||||
const ledger = (q) => {
|
||||
let rs = rows;
|
||||
if (q.get("callerKey")) rs = rs.filter((r) => r.callerKeyId === q.get("callerKey"));
|
||||
if (q.get("failed") === "1") rs = rs.filter((r) => r.status >= 400);
|
||||
return { ...sum(rs), ...groups(), total: rs.length, offset: 0, agents: [{ id: "codex", name: "Codex" }],
|
||||
rows: rs.map((r) => ({ ...r, callerKeyLabel: keyName(r) })) };
|
||||
};
|
||||
return async (route) => {
|
||||
const req = route.request(), url = new URL(req.url());
|
||||
const json = (data) => route.fulfill({ json: data });
|
||||
if (url.pathname === "/boot.js") return route.fulfill({ contentType: "text/javascript", body: `window.bootPrefs = ${JSON.stringify({ lang, theme, web: false })};` });
|
||||
if (url.pathname === "/wails/runtime.js") return route.fulfill({ contentType: "text/javascript", body: "export const Window = {};" });
|
||||
if (url.pathname === "/favicon.ico") return route.fulfill({ status: 204 });
|
||||
if (url.pathname === "/api/state") return json({ agents: [], profiles: [], settings: { lang, theme } });
|
||||
if (url.pathname === "/api/providers") return json({
|
||||
providers: [{ id: "relay", name: "Relay", icon: "generic", models: [{ id: "m", name: "Model", on: true }], agents: [] }],
|
||||
gateway: { running: true, window: true, mine: true, url: "http://127.0.0.1:3999", lan, lanURLs: lan ? lanURLs : [], calls: [], groups: [] },
|
||||
});
|
||||
if (url.pathname === "/api/groups") return json({ groups: [], models: [] });
|
||||
if (url.pathname === "/api/settings/lan") {
|
||||
const body = req.postDataJSON();
|
||||
events.push({ action: "lan", body });
|
||||
lan = body.on;
|
||||
let key = keys.find((k) => k.id === lanKeyID);
|
||||
if (lan && (!key || body.newKey)) {
|
||||
if (!key) {
|
||||
key = { id: "lan-key-" + (++serial), name: "Magpie" };
|
||||
lanKeyID = key.id;
|
||||
keys.push(key);
|
||||
}
|
||||
lanSecret = "fixture-lan-" + (++rotations);
|
||||
key.masked = "sk-magpie-key-…" + lanSecret.slice(-6);
|
||||
}
|
||||
return json(lanState());
|
||||
}
|
||||
if (url.pathname === "/api/settings") {
|
||||
if (req.method() === "POST") {
|
||||
const body = req.postDataJSON();
|
||||
events.push({ action: "settings", body });
|
||||
theme = body.theme;
|
||||
}
|
||||
return json(lanState());
|
||||
}
|
||||
if (url.pathname === "/api/caller-keys") return json({ keys });
|
||||
if (url.pathname.startsWith("/api/caller-keys/")) {
|
||||
const action = url.pathname.split("/").at(-1), body = req.postDataJSON();
|
||||
events.push({ action, body });
|
||||
const k = keys.find((k) => k.id === body.key);
|
||||
let secret = "";
|
||||
if (action === "add-key") {
|
||||
if (!body.name.trim()) return route.fulfill({ status: 400, json: { error: "Use a name between 1 and 120 characters" } });
|
||||
const id = "new-key-" + (++serial);
|
||||
secret = "fixture-created-" + serial;
|
||||
secrets.set(id, secret);
|
||||
keys.push({ id, name: body.name, masked: "sk-magpie-key-…created" });
|
||||
}
|
||||
if (action === "rename-key") k.name = body.name;
|
||||
if (action === "rotate-key") {
|
||||
secret = "fixture-rotated-" + (++rotations);
|
||||
secrets.set(k.id, secret);
|
||||
if (k.id === lanKeyID) lanSecret = secret;
|
||||
k.masked = "sk-magpie-key-…" + secret.slice(-6);
|
||||
}
|
||||
if (action === "remove-key") keys = keys.filter((v) => v !== k);
|
||||
if (action === "on-key" || action === "off-key") k.off = action === "off-key";
|
||||
if (action === "copy-key") secret = k.id === lanKeyID ? lanSecret : secrets.get(k.id);
|
||||
return json({ keys, secret });
|
||||
}
|
||||
if (url.pathname === "/api/copy") { events.push({ action: "clipboard", body: req.postDataJSON() }); return json({}); }
|
||||
if (url.pathname === "/api/usage") return json({ ...sum(rows), ...groups(), agents: [], models: [], series: [], bucket: "day", path: "~/.config/magpie/usage.jsonl" });
|
||||
if (url.pathname === "/api/usage/requests") { events.push({ action: "ledger", query: url.search }); return json(ledger(url.searchParams)); }
|
||||
if (url.pathname === "/api/usage/requests/export") { events.push({ action: "export", query: url.search }); return json({ path: "~/Downloads/keys.csv", rows: ledger(url.searchParams).total }); }
|
||||
if (url.pathname === "/api/usage/quotas") return json([]);
|
||||
if (url.pathname.startsWith("/api/")) return json({});
|
||||
const file = path.join(assets, url.pathname === "/" ? "index.html" : url.pathname);
|
||||
await route.fulfill({ body: await fs.readFile(file), contentType: { ".html": "text/html", ".js": "text/javascript", ".css": "text/css", ".svg": "image/svg+xml", ".png": "image/png" }[path.extname(file)] });
|
||||
};
|
||||
}
|
||||
async function confirmKeyAction(page, row, label) {
|
||||
await row.getByRole("button", { name: label, exact: true }).click();
|
||||
await page.locator("#modal").getByRole("button", { name: label, exact: true }).click();
|
||||
await page.locator("#modal").waitFor({ state: "hidden" });
|
||||
}
|
||||
module.exports = { fixture, confirmKeyAction };
|
||||
@@ -0,0 +1,135 @@
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const { test } = require("node:test");
|
||||
const { chromium, webkit } = require("playwright");
|
||||
const { fixture, confirmKeyAction } = require("./fixtures/caller-keys.cjs");
|
||||
|
||||
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
|
||||
for (const lang of ["en", "zh"]) {
|
||||
test(`${engine} ${lang}: manage named caller keys and filter their usage`, async (t) => {
|
||||
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
|
||||
t.after(() => browser.close());
|
||||
const page = await (await browser.newContext({ viewport: { width: 1000, height: 760 }, reducedMotion: "reduce" })).newPage();
|
||||
page.setDefaultTimeout(6000);
|
||||
await page.addInitScript(() => {
|
||||
window.callerFocus = [];
|
||||
const focus = HTMLElement.prototype.focus;
|
||||
HTMLElement.prototype.focus = function (options) {
|
||||
if (this.closest("#gatewayKeys") || this.closest("#modal")) {
|
||||
callerFocus.push({ input: this.tagName === "INPUT", preventScroll: options?.preventScroll === true });
|
||||
}
|
||||
return focus.call(this, options);
|
||||
};
|
||||
});
|
||||
const events = [], errors = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await page.route("**/*", fixture(lang, "light", events, { lan: true }));
|
||||
const w = lang === "zh"
|
||||
? { create: "创建", name: "网关密钥名称", remove: "移除", copy: "复制网关密钥", disable: "停用密钥", enable: "启用密钥", requests: "请求", usage: "用量" }
|
||||
: { create: "Create", name: "Gateway key name", remove: "Remove", copy: "Copy gateway key", disable: "Disable key", enable: "Enable key", requests: "Requests", usage: "Usage" };
|
||||
await page.goto("http://magpie.test/?view=gateway");
|
||||
await page.locator("#gatewayKeys .acc[data-key]").last().waitFor();
|
||||
assert.equal(await page.locator("#gatewayKeys .accts").count(), 1);
|
||||
assert.equal(await page.locator("#gatewayKeysBlock").isVisible(), true);
|
||||
assert.equal(await page.locator("#gatewayKeys .acc.add").count(), 0, "creation is offered once, in the header");
|
||||
await page.locator("#addGatewayKey").click();
|
||||
assert.equal(await page.evaluate(() => callerFocus.at(-1)?.preventScroll), true, "creating a key focuses without scrolling");
|
||||
await page.getByRole("button", { name: w.create, exact: true }).click();
|
||||
await page.waitForFunction(() => document.querySelector("#gatewayKeys .adding .primary")?.disabled === false);
|
||||
await page.getByRole("textbox", { name: w.name, exact: true }).fill("Tablet");
|
||||
await page.getByRole("textbox", { name: w.name, exact: true }).press("Enter");
|
||||
let tablet = page.locator("#gatewayKeys .acc[data-key]", { hasText: "Tablet" });
|
||||
await tablet.waitFor();
|
||||
assert.equal(await page.locator("#gatewayKeys .acc[data-key]").count(), 4);
|
||||
await tablet.getByRole("button", { name: w.copy, exact: true }).click();
|
||||
await page.waitForFunction(() => document.querySelector("#gatewayKeys .copy.done"));
|
||||
assert(events.some((e) => e.action === "clipboard" && e.body.text === "fixture-created-1"));
|
||||
await tablet.getByRole("button", { name: w.disable, exact: true }).click();
|
||||
await tablet.getByRole("button", { name: w.enable, exact: true }).waitFor();
|
||||
await tablet.getByRole("button", { name: w.enable, exact: true }).click();
|
||||
await tablet.getByRole("button", { name: w.disable, exact: true }).waitFor();
|
||||
await tablet.getByRole("button", { name: "Tablet", exact: true }).click();
|
||||
assert.equal(await page.evaluate(() => callerFocus.at(-1)?.preventScroll), true, "renaming a key focuses without scrolling");
|
||||
await page.locator("#gatewayKeys .rename-in").fill("Travel");
|
||||
await page.locator("#gatewayKeys .rename-in").press("Enter");
|
||||
tablet = page.locator("#gatewayKeys .acc[data-key]", { hasText: "Travel" });
|
||||
await tablet.waitFor();
|
||||
await page.locator("#nav").getByRole("button", { name: w.usage, exact: true }).click();
|
||||
await page.locator("#usageKeys .row").last().waitFor();
|
||||
assert.equal(await page.locator("#usageKeys .row").count(), 3);
|
||||
assert.match(await page.locator("#usageKeys").textContent(), /Laptop/);
|
||||
assert.equal(await page.locator("#usageKeysHead").textContent(), lang === "zh" ? "网关密钥" : "Gateway keys");
|
||||
assert.equal(await page.locator("#usageCallerKeys").count(), 0, "there is only one API key section");
|
||||
await page.locator("#usageTab").getByRole("button", { name: w.requests, exact: true }).click();
|
||||
await page.locator("#ledKey").click();
|
||||
await page.locator(".sess-menu .pm-item", { hasText: "Laptop" }).click();
|
||||
await page.waitForFunction(() => document.querySelectorAll("#ledWrap tbody tr").length === 1);
|
||||
assert.match(await page.locator("#ledWrap").textContent(), /Laptop/);
|
||||
await page.locator("#ledExport").click();
|
||||
await page.waitForTimeout(150);
|
||||
assert(events.some((e) => e.action === "export" && e.query.includes("callerKey=laptop") && !e.query.includes("key=") && !e.query.includes("user=")));
|
||||
await page.reload();
|
||||
await page.locator("#prefs").click();
|
||||
const share = page.locator("#lanList").getByRole("button", { name: lang === "zh" ? "开启" : "On", exact: true });
|
||||
await share.waitFor();
|
||||
// Real wheel input lets the page remember the reader's scroll position.
|
||||
const bounds = await share.boundingBox();
|
||||
await page.mouse.move(500, 400);
|
||||
await page.mouse.wheel(0, bounds.y - 250);
|
||||
await share.click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor();
|
||||
assert.equal(await page.locator("#lanList").getByRole("button", { name: w.copy, exact: true }).count(), 0,
|
||||
"Settings does not duplicate key management");
|
||||
await page.locator("#nav").getByRole("button", { name: lang === "zh" ? "网关" : "Gateway", exact: true }).click();
|
||||
await page.waitForFunction(() => document.querySelectorAll("#gatewayKeys .acc[data-key]").length === 5);
|
||||
let lanRow = page.locator("#gatewayKeys .acc[data-key]", { has: page.locator(".rename", { hasText: /^Magpie$/ }) });
|
||||
const lanID = await lanRow.getAttribute("data-key");
|
||||
const before = await lanRow.locator(".plan").textContent();
|
||||
await lanRow.getByRole("button", { name: w.copy, exact: true }).click();
|
||||
await page.waitForFunction(() => document.querySelector("#gatewayKeys .copy.done"));
|
||||
assert(events.some((e) => e.action === "clipboard" && e.body.text === "fixture-lan-1"));
|
||||
const rotateLabel = lang === "zh" ? "轮换密钥" : "Rotate key";
|
||||
const cancelLabel = lang === "zh" ? "取消" : "Cancel";
|
||||
await lanRow.getByRole("button", { name: rotateLabel, exact: true }).click();
|
||||
assert.equal(await page.evaluate(() => callerFocus.at(-1)?.preventScroll), true, "confirmation focuses without scrolling");
|
||||
await page.locator("#modal").getByRole("button", { name: cancelLabel, exact: true }).click();
|
||||
await page.locator("#modal").waitFor({ state: "hidden" });
|
||||
assert.equal(events.filter((e) => e.action === "rotate-key").length, 0);
|
||||
await confirmKeyAction(page, lanRow, rotateLabel);
|
||||
await page.waitForFunction((masked) => document.querySelector('#gatewayKeys .acc[data-key^="lan-key"] .plan')?.textContent !== masked, before);
|
||||
assert.equal(await lanRow.getAttribute("data-key"), lanID, "rotation keeps the key's identity");
|
||||
await lanRow.getByRole("button", { name: w.copy, exact: true }).click();
|
||||
await page.waitForFunction(() => document.querySelector("#gatewayKeys .copy.done"));
|
||||
assert(events.some((e) => e.action === "clipboard" && e.body.text === "fixture-rotated-2"));
|
||||
await lanRow.getByRole("button", { name: w.remove, exact: true }).click();
|
||||
await page.keyboard.press("Escape");
|
||||
await page.locator("#modal").waitFor({ state: "hidden" });
|
||||
assert.equal(events.filter((e) => e.action === "remove-key").length, 0);
|
||||
await confirmKeyAction(page, lanRow, w.remove);
|
||||
await page.waitForFunction(() => document.querySelectorAll("#gatewayKeys .acc[data-key]").length === 4);
|
||||
await page.locator("#prefs").click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor();
|
||||
await page.locator("#lanList").getByRole("button", { name: lang === "zh" ? "关闭" : "Off", exact: true }).click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor({ state: "detached" });
|
||||
await share.click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor();
|
||||
await page.locator("#nav").getByRole("button", { name: lang === "zh" ? "网关" : "Gateway", exact: true }).click();
|
||||
await page.waitForFunction(() => document.querySelectorAll("#gatewayKeys .acc[data-key]").length === 5);
|
||||
lanRow = page.locator("#gatewayKeys .acc[data-key]", { has: page.locator(".rename", { hasText: /^Magpie$/ }) });
|
||||
assert.notEqual(await lanRow.getAttribute("data-key"), lanID, "recreated LAN key appears without a page reload");
|
||||
assert.equal(await page.locator(`#gatewayKeys .acc[data-key="${lanID}"]`).count(), 0);
|
||||
tablet = page.locator("#gatewayKeys .acc[data-key]", { hasText: "Travel" });
|
||||
await tablet.waitFor();
|
||||
await confirmKeyAction(page, tablet, w.remove);
|
||||
await page.waitForFunction(() => document.querySelectorAll("#gatewayKeys .acc[data-key]").length === 4);
|
||||
await page.setViewportSize({ width: 560, height: 740 });
|
||||
assert.equal(await page.evaluate(() => document.documentElement.scrollWidth > window.innerWidth), false);
|
||||
assert.deepEqual(errors, []);
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${lang}-gateway-keys.png`) });
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs/promises");
|
||||
const path = require("node:path");
|
||||
const { test } = require("node:test");
|
||||
const { chromium, webkit } = require("playwright");
|
||||
const { fixture, confirmKeyAction } = require("./fixtures/caller-keys.cjs");
|
||||
|
||||
for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium", "webkit"])) {
|
||||
for (const locale of ["en", "zh"]) {
|
||||
test(`${engine} ${locale}: Connect uses the selected network address and caller key`, async (t) => {
|
||||
const browser = await (engine === "webkit" ? webkit.launch() : chromium.launch({ channel: "chromium" }));
|
||||
t.after(() => browser.close());
|
||||
const page = await (await browser.newContext({ viewport: { width: 1000, height: 760 }, reducedMotion: "reduce" })).newPage();
|
||||
page.setDefaultTimeout(6000);
|
||||
const events = [], errors = [];
|
||||
page.on("pageerror", (e) => errors.push(e.message));
|
||||
await page.route("**/*", fixture(locale, "dark", events));
|
||||
const w = locale === "zh" ? { on: "开启", off: "关闭", gateway: "网关", disable: "停用密钥", enable: "启用密钥", remove: "移除", rotate: "轮换密钥", create: "创建", name: "网关密钥名称" }
|
||||
: { on: "On", off: "Off", gateway: "Gateway", disable: "Disable key", enable: "Enable key", remove: "Remove", rotate: "Rotate key", create: "Create", name: "Gateway key name" };
|
||||
const snippet = page.locator("#connect .snip");
|
||||
const pick = async (id, name) => {
|
||||
await page.locator(id).click();
|
||||
await page.locator(".proto-menu .pm-item", { hasText: name }).click();
|
||||
};
|
||||
const expectSecret = (secret) => page.waitForFunction((secret) => document.querySelector("#connect .snip")?.textContent.includes(secret), secret);
|
||||
const row = (id) => page.locator(`#gatewayKeys .acc[data-key="${id}"]`);
|
||||
await page.goto("http://magpie.test/?view=gateway");
|
||||
await snippet.waitFor();
|
||||
assert.equal(await page.locator("#connectAddress").count(), 0, "no LAN addresses before sharing is on");
|
||||
assert.equal(await page.locator("#gatewayKeysBlock").isVisible(), false, "most users never share");
|
||||
assert.equal(await page.locator("#connectKey").count(), 0, "local-only users see no gateway key picker");
|
||||
assert.equal(await page.locator("#connect").getByText(locale === "zh" ? "API 密钥" : "API key", { exact: true }).count(), 1);
|
||||
await expectSecret('OPENAI_API_KEY=magpie');
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${locale}-local-connect.png`) });
|
||||
}
|
||||
|
||||
await page.locator("#prefs").click();
|
||||
const on = page.locator("#lanList").getByRole("button", { name: w.on, exact: true });
|
||||
await on.waitFor();
|
||||
const bounds = await on.boundingBox();
|
||||
await page.mouse.move(500, 400);
|
||||
await page.mouse.wheel(0, bounds.y - 250);
|
||||
await on.click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor();
|
||||
await page.locator("#nav").getByRole("button", { name: w.gateway, exact: true }).click();
|
||||
await page.locator("#gatewayKeysBlock").waitFor({ state: "visible" });
|
||||
await page.locator("#connectKey").click();
|
||||
assert.equal(await page.locator(".proto-menu .pm-name", { hasText: /^Magpie$/ }).count(), 1);
|
||||
assert.equal(await page.locator(".proto-menu .pm-name", { hasText: /^magpie$/ }).count(), 0,
|
||||
"the arbitrary local token must not look like a second named Magpie key");
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${locale}-shared-key-picker.png`) });
|
||||
}
|
||||
await page.keyboard.press("Escape");
|
||||
const staleKeys = await page.locator("#gatewayKeys .acc[data-key]").evaluateAll((rows) => rows.map((r) => ({
|
||||
id: r.dataset.key, name: r.querySelector(".rename").textContent, masked: r.querySelector(".plan").textContent,
|
||||
})));
|
||||
let releaseRead, readStarted;
|
||||
const held = new Promise((resolve) => { releaseRead = resolve; });
|
||||
const started = new Promise((resolve) => { readStarted = resolve; });
|
||||
const lateList = async (route) => {
|
||||
readStarted();
|
||||
await held;
|
||||
await route.fulfill({ json: { keys: staleKeys } });
|
||||
};
|
||||
await page.route("**/api/caller-keys", lateList);
|
||||
await page.locator("#prefs").click();
|
||||
await page.locator("#lanList").waitFor();
|
||||
await page.locator("#nav").getByRole("button", { name: w.gateway, exact: true }).click();
|
||||
await started;
|
||||
await row("server").getByRole("button", { name: w.disable, exact: true }).click();
|
||||
await page.locator('#gatewayKeys .acc.off[data-key="server"]').waitFor();
|
||||
const reply = page.waitForResponse((r) => new URL(r.url()).pathname === "/api/caller-keys");
|
||||
releaseRead();
|
||||
await reply;
|
||||
await page.waitForTimeout(100);
|
||||
assert.equal(await page.locator('#gatewayKeys .acc.off[data-key="server"]').count(), 1,
|
||||
"a late list response cannot undo a key action");
|
||||
await page.unroute("**/api/caller-keys", lateList);
|
||||
await row("server").getByRole("button", { name: w.enable, exact: true }).click();
|
||||
await row("server").getByRole("button", { name: w.disable, exact: true }).waitFor();
|
||||
await pick("#connectKey", "Server");
|
||||
await expectSecret("fixture-server");
|
||||
|
||||
await page.locator("#connectAddress").waitFor();
|
||||
await pick("#connectAddress", "10.0.0.10");
|
||||
await expectSecret("fixture-server");
|
||||
await row("work").getByRole("button", { name: w.disable, exact: true }).click();
|
||||
await page.locator('#gatewayKeys .acc.off[data-key="work"]').waitFor();
|
||||
await page.locator("#connectKey").click();
|
||||
assert.equal(await page.locator(".proto-menu .pm-name", { hasText: /^Work$/ }).count(), 0);
|
||||
assert.equal(await page.locator(".proto-menu .pm-name", { hasText: /^Magpie$/ }).count(), 1,
|
||||
"the default key is named Magpie");
|
||||
assert.equal(await page.locator(".proto-menu .pm-name", { hasText: /^magpie$/ }).count(), 0, "arbitrary loopback tokens cannot be selected for LAN");
|
||||
await page.keyboard.press("Escape");
|
||||
await row("server").getByRole("button", { name: w.disable, exact: true }).click();
|
||||
await expectSecret("fixture-laptop");
|
||||
await row("server").getByRole("button", { name: w.enable, exact: true }).click();
|
||||
await row("server").getByRole("button", { name: w.disable, exact: true }).waitFor();
|
||||
await pick("#connectKey", "Server");
|
||||
await expectSecret("fixture-server");
|
||||
|
||||
for (const api of ["OpenAI", "Responses", "Anthropic", "Gemini"]) {
|
||||
await page.locator("#connect").getByRole("button", { name: api, exact: true }).click();
|
||||
const base = "http://10.0.0.10:3999" + (["OpenAI", "Responses"].includes(api) ? "/v1" : "");
|
||||
assert.equal(await page.locator("#connectAddress code").textContent(), base);
|
||||
for (const dialect of ["Shell", "curl", "Python", "Node"]) {
|
||||
await page.locator("#connect").getByRole("button", { name: dialect, exact: true }).click();
|
||||
const code = await snippet.textContent();
|
||||
assert(code.includes(base), `${api} ${dialect} must use the selected address`);
|
||||
assert(code.includes("fixture-server"), `${api} ${dialect} must use the selected key`);
|
||||
assert(!code.includes('"magpie"'), "named key examples never fall back to magpie");
|
||||
}
|
||||
}
|
||||
await row("server").locator(".rename").click();
|
||||
await page.locator("#gatewayKeys .rename-in").fill("Local network");
|
||||
await page.locator("#gatewayKeys .rename-in").press("Enter");
|
||||
await row("server").getByRole("button", { name: "Local network", exact: true }).waitFor();
|
||||
await page.locator("#connectKey").click();
|
||||
assert.equal(await page.locator(".proto-menu .pm-name", { hasText: /^Local network$/ }).count(), 1,
|
||||
"custom key names are not translated as interface labels");
|
||||
await page.keyboard.press("Escape");
|
||||
await confirmKeyAction(page, row("server"), w.rotate);
|
||||
await expectSecret("fixture-rotated-2");
|
||||
assert.equal(await row("server").locator(".rename").textContent(), "Local network");
|
||||
assert(!(await snippet.textContent()).includes("fixture-server"), "rotation refreshes the selected secret");
|
||||
const code = await snippet.textContent();
|
||||
await page.locator("#connect .snip-wrap .copy").click();
|
||||
await page.waitForFunction(() => document.querySelector("#connect .snip-wrap .copy.done"));
|
||||
assert(events.some((e) => e.action === "clipboard" && e.body.text === code));
|
||||
for (const width of [1000, 560, 390]) {
|
||||
await page.setViewportSize({ width, height: 760 });
|
||||
await page.waitForTimeout(200);
|
||||
if (width >= 560) {
|
||||
assert.equal(await page.evaluate(() => document.documentElement.scrollWidth > innerWidth), false);
|
||||
}
|
||||
// Below the native window's minimum, check the changed cards, not its header.
|
||||
assert.equal(await page.evaluate(() => ["#connect", "#gatewayKeys .accts"].every((s) => {
|
||||
const e = document.querySelector(s);
|
||||
return e.scrollWidth <= e.clientWidth;
|
||||
})), true);
|
||||
if (process.env.ARTIFACT_DIR) {
|
||||
await fs.mkdir(process.env.ARTIFACT_DIR, { recursive: true });
|
||||
await page.screenshot({ path: path.join(process.env.ARTIFACT_DIR, `${engine}-${locale}-connect-${width}.png`) });
|
||||
}
|
||||
}
|
||||
await page.setViewportSize({ width: 1000, height: 760 });
|
||||
await confirmKeyAction(page, row("server"), w.remove);
|
||||
await expectSecret("fixture-laptop");
|
||||
assert(!(await snippet.textContent()).includes("fixture-rotated-2"), "removal clears the selected credential");
|
||||
// A remote connection without enabled keys must not copy a broken command.
|
||||
for (const id of ["laptop", "work"]) {
|
||||
await confirmKeyAction(page, row(id), w.remove);
|
||||
await row(id).waitFor({ state: "detached" });
|
||||
}
|
||||
const lan = page.locator('#gatewayKeys .acc[data-key^="lan-key"]');
|
||||
await confirmKeyAction(page, lan, w.remove);
|
||||
await lan.waitFor({ state: "detached" });
|
||||
assert.equal(await page.locator("#connect .snip-wrap .copy").count(), 0);
|
||||
assert.equal(await page.locator("#connectKey").count(), 0);
|
||||
await page.locator("#addGatewayKey").click();
|
||||
await page.getByRole("textbox", { name: w.name, exact: true }).fill("Replacement");
|
||||
await page.getByRole("button", { name: w.create, exact: true }).click();
|
||||
await expectSecret("fixture-created-2");
|
||||
assert.equal(await page.locator("#connectKey code").textContent(), "Replacement");
|
||||
await page.locator("#prefs").click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor();
|
||||
await page.locator("#lanList").getByRole("button", { name: w.off, exact: true }).click();
|
||||
await page.locator("#lanList .lan-address-row").waitFor({ state: "detached" });
|
||||
await page.locator("#nav").getByRole("button", { name: w.gateway, exact: true }).click();
|
||||
await page.locator("#connectAddress").waitFor({ state: "detached" });
|
||||
assert.equal(await page.locator("#gatewayKeysBlock").isVisible(), false);
|
||||
assert.equal(await page.locator("#connectKey").count(), 0);
|
||||
assert.equal(await page.locator("#connect").getByText(locale === "zh" ? "API 密钥" : "API key", { exact: true }).count(), 1);
|
||||
assert((await snippet.textContent()).includes("http://127.0.0.1:3999"));
|
||||
assert(!(await snippet.textContent()).includes("fixture-created-2"), "sharing off clears the selected gateway credential");
|
||||
assert.deepEqual(errors, []);
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -80,6 +80,11 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
|
||||
// remembered across a reload
|
||||
await page.reload();
|
||||
await page.locator("#activity .call").first().waitFor();
|
||||
// Reduced-motion animations can still be on their first frame when
|
||||
// fast fixtures return. Compare the settled heading position.
|
||||
await page.locator(view).evaluate(async (v) => {
|
||||
await Promise.all(v.getAnimations().map((a) => a.finished.catch(() => {})));
|
||||
});
|
||||
assert.equal(await page.locator("#foldConnect").getAttribute("aria-expanded"), "false", "the fold is remembered");
|
||||
assert(await page.locator("#connect").evaluate((c) => c.hidden));
|
||||
|
||||
|
||||
@@ -132,6 +132,7 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
|
||||
for (let i = 0; i < 40 && !posts.some((p) => p.path === "/api/groups/save"); i++) await page.waitForTimeout(50);
|
||||
const saved = posts.find((p) => p.path === "/api/groups/save")?.body;
|
||||
assert.deepEqual(saved?.members, ["hunyuan/hy4"], "a group of hy4 saved");
|
||||
assert.deepEqual(saved.fast, [], "a model shortcut starts at standard speed");
|
||||
assert.equal(saved.id, "hy4");
|
||||
|
||||
// the provider's editor names the model in no group under the tick
|
||||
|
||||
@@ -30,10 +30,12 @@ const ROWS = [
|
||||
for (let i = 0; i < 126; i++) {
|
||||
ROWS.push({ t: new Date(now - (300 + i * 60) * 1e3).toISOString(), agent: i % 2 ? "claude" : "codex", agentName: i % 2 ? "Claude Code" : "Codex", icon: i % 2 ? "claudecode-color" : "codex-color", provider: "relay", providerName: "Relay", host: "team", req: "sol", model: "gpt-6-sol", served: "gpt-6-sol", in: 1000 + i, out: 100, ms: 900, status: 200, cost: 0.001, priced: true });
|
||||
}
|
||||
ROWS.forEach((r, i) => { r.callerKeyId = i % 2 ? "server" : "laptop"; });
|
||||
|
||||
function page(q) {
|
||||
let rows = ROWS;
|
||||
if (q.get("route")) rows = rows.filter((r) => r.route_id === Number(q.get("route")));
|
||||
if (q.get("callerKey")) rows = rows.filter((r) => r.callerKeyId === q.get("callerKey"));
|
||||
if (q.get("agent")) rows = rows.filter((r) => r.agent === q.get("agent"));
|
||||
if (q.get("failed") === "1") rows = rows.filter((r) => r.status >= 400);
|
||||
const s = (q.get("q") || "").toLowerCase();
|
||||
@@ -46,6 +48,7 @@ function page(q) {
|
||||
input: sum("in"), output: sum("out"), cache_read: sum("cache_read"), cache_write: sum("cache_write"), reasoning: sum("reasoning"),
|
||||
cost: sum("cost"), unpriced: rows.filter((r) => !r.priced && r.in).length,
|
||||
agents: [{ id: "claude", name: "Claude Code", icon: "claudecode-color" }, { id: "codex", name: "Codex", icon: "codex-color" }],
|
||||
callerKeys: [{ id: "laptop", name: "Laptop" }, { id: "server", name: "Server" }],
|
||||
};
|
||||
}
|
||||
|
||||
@@ -155,6 +158,28 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
|
||||
|
||||
for (const lang of ["en", "zh"]) {
|
||||
const w = L[lang];
|
||||
await t.test(lang + ": an unavailable caller filter is cleared", async () => {
|
||||
const asked = [];
|
||||
const { page: tab, errors } = await open(lang, "light", asked);
|
||||
await tab.locator("#ledKey").click();
|
||||
await tab.locator(".sess-menu .pm-item", { hasText: "Laptop" }).click();
|
||||
await lastAsked(tab, asked, q => q.get("callerKey") === "laptop");
|
||||
await tab.route("**/api/usage/requests?**", async (route) => {
|
||||
const q = new URL(route.request().url()).searchParams;
|
||||
asked.push(q);
|
||||
const data = page(q);
|
||||
data.callerKeys = [{ id: "server", name: "Server" }];
|
||||
data.rows = data.rows.filter(r => r.callerKeyId === "server");
|
||||
data.total = data.rows.length;
|
||||
await route.fulfill({ json: data });
|
||||
});
|
||||
await tab.locator("#usageReload").click();
|
||||
await lastAsked(tab, asked, q => !q.has("callerKey") && q.get("offset") === "0");
|
||||
assert.equal(await tab.locator("#ledKey").textContent(), lang === "zh" ? "全部网关密钥" : "All gateway keys");
|
||||
assert(await tab.locator(".led-row").count() > 0, "stale filtered empty data must be reloaded");
|
||||
assert.deepEqual(errors, []);
|
||||
await tab.close();
|
||||
});
|
||||
await t.test(lang, async () => {
|
||||
const asked = [];
|
||||
const { page, errors } = await open(lang, "light", asked);
|
||||
@@ -331,12 +356,18 @@ for (const engine of (process.env.BROWSER ? [process.env.BROWSER] : ["chromium",
|
||||
const asked = [];
|
||||
const { page, errors } = await open(lang, "light", asked);
|
||||
await page.evaluate(() => {
|
||||
period = "7d"; ledAgent = "codex"; ledFailed = true; ledQuery = "sol"; ledOffset = 100;
|
||||
period = "7d"; ledAgent = "codex"; ledCallerKey = "server"; ledFailed = true; ledQuery = "sol"; ledOffset = 100;
|
||||
window.openUsageRoute({ id: 123, time: new Date().toISOString(), model: "gpt-6-sol" });
|
||||
});
|
||||
await lastAsked(page, asked, q=>q.get("route") === "123");
|
||||
await lastAsked(page, asked, q=>q.get("route") === "123" && !q.has("callerKey"));
|
||||
assert.equal(await page.locator(".led tbody tr").count(), 2, "the previous caller filter must not hide this route");
|
||||
await page.locator("#ledKey").click();
|
||||
await page.locator(".sess-menu .pm-item", { hasText: "Laptop" }).click();
|
||||
await lastAsked(page, asked, q=>q.get("route") === "123" && q.get("callerKey") === "laptop");
|
||||
await page.locator("#ledExport").click();
|
||||
await lastAsked(page, asked, q=>q.method === "POST" && q.get("route") === "123" && q.get("callerKey") === "laptop" && !q.has("offset") && !q.has("limit"));
|
||||
await page.locator("#ledRouteClear").click();
|
||||
await lastAsked(page, asked, q=>!q.has("route") && q.get("period") === "7d" && q.get("agent") === "codex" && q.get("failed") === "1" && q.get("q") === "sol" && q.get("offset") === "100");
|
||||
await lastAsked(page, asked, q=>!q.has("route") && q.get("period") === "7d" && q.get("agent") === "codex" && q.get("callerKey") === "server" && q.get("failed") === "1" && q.get("q") === "sol" && q.get("offset") === "100");
|
||||
assert.equal(await page.locator("#ledQ").inputValue(), "sol");
|
||||
assert.deepEqual(errors, []);
|
||||
await page.close();
|
||||
|
||||
+44
-13
@@ -12,6 +12,7 @@ import (
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/agent"
|
||||
"github.com/yetone/magpie/internal/edit"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
@@ -29,9 +30,10 @@ type usageGroup struct {
|
||||
|
||||
type usageJSON struct {
|
||||
usage.Summary
|
||||
Agents []usageGroup `json:"agents"`
|
||||
Models []usageGroup `json:"models"`
|
||||
Path string `json:"path"`
|
||||
Agents []usageGroup `json:"agents"`
|
||||
Models []usageGroup `json:"models"`
|
||||
CallerKeys []usageGroup `json:"callerKeys"`
|
||||
Path string `json:"path"`
|
||||
}
|
||||
|
||||
func usageState(p usage.Period) usageJSON {
|
||||
@@ -65,9 +67,30 @@ func usageState(p usage.Period) usageJSON {
|
||||
}
|
||||
out.Models = append(out.Models, ug)
|
||||
}
|
||||
out.CallerKeys = callerUsageGroups(s)
|
||||
return out
|
||||
}
|
||||
|
||||
func callerUsageGroups(s usage.Summary) []usageGroup {
|
||||
keys := []usageGroup{}
|
||||
current, _ := access.List()
|
||||
names := map[string]string{}
|
||||
for _, k := range current {
|
||||
names[k.ID] = k.Name
|
||||
}
|
||||
for _, g := range s.CallerKeys {
|
||||
n := names[g.CallerKeyID]
|
||||
if n == "" {
|
||||
n = g.CallerKeyName
|
||||
}
|
||||
if n == "" {
|
||||
n = g.CallerKeyID
|
||||
}
|
||||
keys = append(keys, usageGroup{Group: g, Name: n, Icon: "generic"})
|
||||
}
|
||||
return keys
|
||||
}
|
||||
|
||||
func periodOf(s string) usage.Period {
|
||||
switch p := usage.Period(s); p {
|
||||
case usage.Today, usage.Week, usage.Month, usage.All:
|
||||
@@ -78,24 +101,26 @@ func periodOf(s string) usage.Period {
|
||||
|
||||
func ledgerFilter(q url.Values) usage.Filter {
|
||||
id, _ := strconv.ParseInt(q.Get("route"), 10, 64)
|
||||
return usage.Filter{RouteID: id, Model: q.Get("model"), Agent: q.Get("agent"), Provider: q.Get("provider"), Failed: q.Get("failed") == "1", Query: q.Get("q")}
|
||||
return usage.Filter{RouteID: id, Model: q.Get("model"), Agent: q.Get("agent"), Provider: q.Get("provider"), CallerKey: q.Get("callerKey"), Failed: q.Get("failed") == "1", Query: q.Get("q")}
|
||||
}
|
||||
|
||||
// ledgerRow is a usage.Row with the names the page shows it by.
|
||||
type ledgerRow struct {
|
||||
usage.Row
|
||||
AgentName string `json:"agentName"`
|
||||
Icon string `json:"icon"` // the agent's
|
||||
ProviderName string `json:"providerName"`
|
||||
Access string `json:"access,omitempty"` // known account/route type, independent of model maker
|
||||
PricingModel string `json:"pricing_model,omitempty"`
|
||||
CallerKeyLabel string `json:"callerKeyLabel,omitempty"`
|
||||
AgentName string `json:"agentName"`
|
||||
Icon string `json:"icon"` // the agent's
|
||||
ProviderName string `json:"providerName"`
|
||||
Access string `json:"access,omitempty"` // known account/route type, independent of model maker
|
||||
PricingModel string `json:"pricing_model,omitempty"`
|
||||
}
|
||||
|
||||
type ledgerJSON struct {
|
||||
Period usage.Period `json:"period"`
|
||||
Rows []ledgerRow `json:"rows"`
|
||||
Offset int `json:"offset"`
|
||||
Total int `json:"total"` // the rows the filter keeps, on every page
|
||||
CallerKeys []usageGroup `json:"callerKeys"`
|
||||
Period usage.Period `json:"period"`
|
||||
Rows []ledgerRow `json:"rows"`
|
||||
Offset int `json:"offset"`
|
||||
Total int `json:"total"` // the rows the filter keeps, on every page
|
||||
// Series: those rows by hour, day or week (Bucket), for the chart
|
||||
Bucket string `json:"bucket"`
|
||||
Series []usage.SeriesPoint `json:"series"`
|
||||
@@ -162,6 +187,11 @@ func ledgerPage(p usage.Period, f usage.Filter, offset, limit int) ledgerJSON {
|
||||
}
|
||||
out := ledgerJSON{Period: p, Rows: make([]ledgerRow, 0, len(page)), Offset: offset, Total: l.Total, Totals: l.Sum, Agents: []ledgerAgent{}, Providers: []ledgerAgent{}}
|
||||
out.Bucket, out.Series = l.Bucket, l.Series
|
||||
out.CallerKeys = callerUsageGroups(usage.Summary{CallerKeys: l.CallerKeys})
|
||||
callerLabels := map[string]string{}
|
||||
for _, g := range out.CallerKeys {
|
||||
callerLabels[g.CallerKeyID] = g.Name
|
||||
}
|
||||
for _, r := range page {
|
||||
a := who(r.Agent)
|
||||
lr := ledgerRow{Row: r, AgentName: a.Name, Icon: a.Icon, ProviderName: names[r.Provider]}
|
||||
@@ -176,6 +206,7 @@ func ledgerPage(p usage.Period, f usage.Filter, offset, limit int) ledgerJSON {
|
||||
if lr.ProviderName == "" {
|
||||
lr.ProviderName = r.Provider
|
||||
}
|
||||
lr.CallerKeyLabel = callerLabels[r.CallerKeyID]
|
||||
out.Rows = append(out.Rows, lr)
|
||||
}
|
||||
for _, id := range l.Agents {
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
@@ -27,7 +28,10 @@ func TestZhipuKeyTeamFields(t *testing.T) {
|
||||
}
|
||||
asked := map[string]http.Header{}
|
||||
hosts := map[string][]string{} // where each key's team quota was asked
|
||||
var mu sync.Mutex
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
key := r.Header.Get("Authorization")
|
||||
switch r.Header.Get("X-Host") + r.URL.Path {
|
||||
case "open.bigmodel.cn/api/monitor/usage/quota/limit", "bigmodel.cn/api/monitor/usage/quota/limit":
|
||||
@@ -89,6 +93,8 @@ func TestZhipuKeyTeamFields(t *testing.T) {
|
||||
for _, q := range PlanQuotas(context.Background()) {
|
||||
got[q.Provider] = q
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
q, ok := got["team"]
|
||||
if !ok || q.Error != "" || q.Plan != "GLM Coding Team" || len(q.Windows) != 2 ||
|
||||
q.Windows[0].Name != "5 hours" || q.Windows[0].Used != 42 || q.Windows[0].Span != 5*time.Hour ||
|
||||
|
||||
@@ -60,11 +60,12 @@ type Settings struct {
|
||||
RedactPersonal bool `json:"redactPersonal,omitempty"`
|
||||
RedactWords []string `json:"redactWords,omitempty"`
|
||||
RedactRules []redact.Rule `json:"redactRules,omitempty"`
|
||||
// LAN shares the gateway on the local network, for agents on other
|
||||
// machines; a request from one must carry LANKey as its API key, a
|
||||
// key magpie makes when LAN is first turned on.
|
||||
// LAN shares the gateway on the local network; remote callers must use
|
||||
// named caller keys. LANKey is retained for older Magpie versions.
|
||||
LAN bool `json:"lan,omitempty"`
|
||||
LANKey string `json:"lanKey,omitempty"`
|
||||
// LANKeyID remembers the default named key created when sharing is enabled.
|
||||
LANKeyID string `json:"lanKeyId,omitempty"`
|
||||
// RequestArchive keeps each call the gateway serves — its headers and
|
||||
// bodies both ways, secrets taken out — in the S3 bucket sync keeps
|
||||
// its backup in (gateway/archive.go), for looking into a request later.
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
package usage
|
||||
|
||||
import (
|
||||
"encoding/csv"
|
||||
"encoding/json"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/yetone/magpie/internal/catalog"
|
||||
"github.com/yetone/magpie/internal/sessions"
|
||||
)
|
||||
|
||||
func TestProviderAndCallerIdentitiesRemainIndependent(t *testing.T) {
|
||||
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
|
||||
t.Setenv("XDG_CACHE_HOME", t.TempDir())
|
||||
catalog.Reset()
|
||||
t.Cleanup(catalog.Reset)
|
||||
now := time.Now()
|
||||
recs := []Record{
|
||||
{Time: now.Add(-time.Minute), RouteID: 123, Provider: "relay", Model: "m", ProviderKeyID: "primary", ProviderKeyName: "Primary", CallerKeyID: "desk", CallerKeyName: "Desk", Input: 10, Status: 200},
|
||||
{Time: now, RouteID: 456, Provider: "relay", Model: "m", ProviderKeyID: "backup", ProviderKeyName: "Backup", CallerKeyID: "desk", CallerKeyName: "Desk", Input: 20, Status: 200},
|
||||
{Time: now, RouteID: 456, Provider: "relay", Model: "m", ProviderKeyID: "backup", ProviderKeyName: "Backup", CallerKeyID: "server", CallerKeyName: "Server", Input: 30, Status: 200},
|
||||
}
|
||||
s := summarize(All, now, recs)
|
||||
if len(s.ProviderKeys) != 2 || len(s.CallerKeys) != 2 || s.Input != 60 {
|
||||
t.Fatalf("independent summaries: %+v", s)
|
||||
}
|
||||
for _, g := range s.ProviderKeys {
|
||||
want := 10
|
||||
if g.ProviderKeyID == "backup" {
|
||||
want = 50
|
||||
}
|
||||
if g.Input != want || g.CallerKeyID != "" {
|
||||
t.Fatalf("provider grouping mixed with caller: %+v", g)
|
||||
}
|
||||
}
|
||||
for _, g := range s.CallerKeys {
|
||||
if g.Input != 30 || g.ProviderKeyID != "" {
|
||||
t.Fatalf("caller grouping mixed with provider: %+v", g)
|
||||
}
|
||||
}
|
||||
rows, total, _ := ledger(All.Since(now), Filter{CallerKey: "desk"}, recs)
|
||||
if len(rows) != 2 || total.Input != 30 {
|
||||
t.Fatalf("caller filter across upstream keys: %+v, %+v", rows, total)
|
||||
}
|
||||
if rows, total, _ := ledger(All.Since(now), Filter{RouteID: 456}, recs); len(rows) != 2 || total.Input != 50 {
|
||||
t.Fatalf("route filter across caller keys: %+v, %+v", rows, total)
|
||||
}
|
||||
if rows, total, _ := ledger(All.Since(now), Filter{RouteID: 456, CallerKey: "desk"}, recs); len(rows) != 1 || total.Input != 20 || rows[0].ProviderKeyID != "backup" || rows[0].CallerKeyID != "desk" {
|
||||
t.Fatalf("combined route and caller filter: %+v, %+v", rows, total)
|
||||
}
|
||||
if rows, total, _ := ledger(All.Since(now), Filter{RouteID: 123, CallerKey: "server"}, recs); len(rows) != 0 || total.Calls != 0 {
|
||||
t.Fatalf("combined filters must intersect: %+v, %+v", rows, total)
|
||||
}
|
||||
var out strings.Builder
|
||||
if err := WriteCSV(&out, rows); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cells, err := csv.NewReader(strings.NewReader(out.String())).ReadAll()
|
||||
if err != nil || len(cells) != 3 {
|
||||
t.Fatalf("CSV: %v, %s", err, out.String())
|
||||
}
|
||||
columns := []string{"provider_key_id", "provider_key_name", "route_id", "caller_key_id", "caller_key_name"}
|
||||
indexes := make([]int, len(columns))
|
||||
for i, name := range columns {
|
||||
indexes[i] = slices.Index(cells[0], name)
|
||||
if indexes[i] < 0 || (i > 0 && indexes[i] <= indexes[i-1]) {
|
||||
t.Fatal("missing or reordered identity columns", cells[0])
|
||||
}
|
||||
}
|
||||
if !slices.Equal(cells[0][len(cells[0])-2:], columns[3:]) {
|
||||
t.Fatal("provider columns must precede caller columns", cells[0])
|
||||
}
|
||||
for i, row := range rows {
|
||||
want := []string{row.ProviderKeyID, row.ProviderKeyName, strconv.FormatInt(row.RouteID, 10), row.CallerKeyID, row.CallerKeyName}
|
||||
for j, index := range indexes {
|
||||
if cells[i+1][index] != want[j] {
|
||||
t.Fatal("CSV lost an identity", cells[i+1], want)
|
||||
}
|
||||
}
|
||||
encoded, err := json.Marshal(row)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var fields map[string]any
|
||||
if err := json.Unmarshal(encoded, &fields); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fields["route_id"] != float64(row.RouteID) || fields["providerKeyId"] != row.ProviderKeyID || fields["providerKeyName"] != row.ProviderKeyName || fields["callerKeyId"] != "desk" || fields["callerKeyName"] != "Desk" || fields["keyId"] != nil || fields["keyName"] != nil {
|
||||
t.Fatal("JSON identities must be explicit", fields)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPackedRequestPageCallerAndRouteIdentity(t *testing.T) {
|
||||
pageHome(t)
|
||||
now := time.Now()
|
||||
for i := 0; i < 80; i++ {
|
||||
caller := "desk"
|
||||
if i%2 != 0 {
|
||||
caller = "server"
|
||||
}
|
||||
Append(Record{Time: now.Add(time.Duration(i) * time.Second), Agent: "codex", Provider: "relay", Model: "m", RouteID: 456, CallerKeyID: caller, CallerKeyName: strings.ToUpper(caller), ProviderKeyID: "primary", ProviderKeyName: "Primary", Input: 10, Output: 1, Status: 200})
|
||||
}
|
||||
// The first query creates the compressed snapshot; a different filter must
|
||||
// decode it instead of reusing the first materialized page.
|
||||
for _, filter := range []Filter{{}, {CallerKey: "desk", RouteID: 456, Provider: "relay", Model: "m"}, {CallerKey: "server"}} {
|
||||
for repeat := 0; repeat < 2; repeat++ {
|
||||
page := QueryPage(All, filter, 0, 100)
|
||||
want := 80
|
||||
if filter.CallerKey != "" {
|
||||
want = 40
|
||||
}
|
||||
if page.Total != want || len(page.Rows) != want || page.Sum.Input != want*10 {
|
||||
t.Fatalf("caller/route filter lost requests: %+v, rows=%d sum=%+v", filter, len(page.Rows), page.Sum)
|
||||
}
|
||||
if len(page.CallerKeys) != 2 {
|
||||
t.Fatalf("selected caller hid other keys: %+v", page.CallerKeys)
|
||||
}
|
||||
for _, group := range page.CallerKeys {
|
||||
if group.Calls != 40 || group.Input != 400 || group.CallerKeyName != strings.ToUpper(group.CallerKeyID) || group.ProviderKeyID != "" {
|
||||
t.Fatalf("caller grouping mixed with provider: %+v", group)
|
||||
}
|
||||
}
|
||||
equalPage(t, page, pageFromLedger(All, filter, 0, 100, LedgerOf(All, Filter{})))
|
||||
for _, row := range page.Rows {
|
||||
if row.RouteID != 456 || row.CallerKeyID == "" || row.CallerKeyName != strings.ToUpper(row.CallerKeyID) || row.ProviderKeyID != "primary" || row.ProviderKeyName != "Primary" || (filter.CallerKey != "" && row.CallerKeyID != filter.CallerKey) {
|
||||
t.Fatalf("compressed cache lost independent identities: %+v", row)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
requestCache.Lock()
|
||||
packed := len(requestCache.gateway.Archive) > 0
|
||||
requestCache.Unlock()
|
||||
if !packed {
|
||||
t.Fatal("test must exercise a compressed gateway snapshot")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCallerPageKeepsUnattributedSessionsSeparate(t *testing.T) {
|
||||
pageHome(t)
|
||||
now := time.Now()
|
||||
recs := []Record{
|
||||
{Time: now, Agent: "codex", Provider: "relay", Model: "m", CallerKeyID: "desk", CallerKeyName: "Desk", Input: 10, Status: 200},
|
||||
{Time: now, Agent: "codex", Provider: "relay", Model: "m", CallerKeyID: "server", CallerKeyName: "Server", Input: 20, Status: 200},
|
||||
{Time: now, Agent: "codex", Model: "unknown", CallerKeyID: "desk", CallerKeyName: "Desk", Status: 404, Rejected: true, Error: "unknown model"},
|
||||
{Time: now, Agent: "codex", Provider: "relay", Model: "m", Input: 30, Status: 200},
|
||||
}
|
||||
for i := range recs {
|
||||
recs[i].Time = now.Add(time.Duration(i) * time.Second)
|
||||
Append(recs[i])
|
||||
}
|
||||
logs := []sessions.Call{{Time: now.Add(-time.Second), Agent: "claude", Session: "local", Model: "m", Tokens: sessions.Tokens{Input: 40}}}
|
||||
rows, sum, agents, providers := ledgerWith(time.Time{}, Filter{}, recs, logs)
|
||||
all := Ledgered{Rows: rows, Sum: sum, Agents: agents, Providers: providers}
|
||||
gateway, local := &rowChunk{}, &rowChunk{Source: sessions.CallSource{Path: "/local"}}
|
||||
for i, row := range rows {
|
||||
if row.Source == "log" {
|
||||
local.add(row, "local-message", int64(i), false)
|
||||
} else {
|
||||
gateway.add(row, "", int64(i), false)
|
||||
}
|
||||
}
|
||||
for _, filter := range []Filter{{}, {CallerKey: "desk"}, {CallerKey: "server"}, {CallerKey: "desk", Failed: true}, {CallerKey: "missing"}} {
|
||||
page := buildRequestPage(All, filter, 0, 100, gateway, []*rowChunk{local})
|
||||
equalPage(t, page, pageFromLedger(All, filter, 0, 100, all))
|
||||
if len(page.CallerKeys) != 2 || page.CallerKeys[0].CallerKeyID != "server" || page.CallerKeys[0].Input != 20 || page.CallerKeys[1].CallerKeyID != "desk" || page.CallerKeys[1].Calls != 1 || page.CallerKeys[1].Input != 10 {
|
||||
t.Fatalf("unattributed sessions or rejections inflated caller groups: %+v", page.CallerKeys)
|
||||
}
|
||||
for _, row := range page.Rows {
|
||||
if filter.CallerKey != "" && row.Source == "log" {
|
||||
t.Fatal("local session wrongly attributed to a gateway key", row)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -44,15 +44,19 @@ func (r Record) Failed() bool { return r.Status >= 400 || r.Error != "" }
|
||||
// one provider, to the failed calls, and to the rows whose models, provider, host or
|
||||
// session hold Query (any case).
|
||||
type Filter struct {
|
||||
RouteID int64
|
||||
Model string // exact model selected in the ranking
|
||||
Agent string
|
||||
Provider string // a provider's id, as the ledger's rows have it
|
||||
Failed bool
|
||||
Query string
|
||||
CallerKey string
|
||||
RouteID int64
|
||||
Model string // exact model selected in the ranking
|
||||
Agent string
|
||||
Provider string // a provider's id, as the ledger's rows have it
|
||||
Failed bool
|
||||
Query string
|
||||
}
|
||||
|
||||
func (f Filter) keeps(r Record) bool {
|
||||
if f.CallerKey != "" && r.CallerKeyID != f.CallerKey {
|
||||
return false
|
||||
}
|
||||
if f.RouteID != 0 && r.RouteID != f.RouteID {
|
||||
return false
|
||||
}
|
||||
@@ -69,7 +73,7 @@ func (f Filter) keeps(r Record) bool {
|
||||
return false
|
||||
}
|
||||
if q := strings.ToLower(strings.TrimSpace(f.Query)); q != "" {
|
||||
return slices.ContainsFunc([]string{r.Requested, r.Model, r.Served, r.Provider, r.Host, r.Session, r.Effort, r.SessionProvider, r.SessionAccount, r.ProviderKeyID, r.ProviderKeyName}, func(s string) bool {
|
||||
return slices.ContainsFunc([]string{r.Requested, r.Model, r.Served, r.Provider, r.Host, r.Session, r.Effort, r.SessionProvider, r.SessionAccount, r.ProviderKeyID, r.ProviderKeyName, r.CallerKeyID, r.CallerKeyName}, func(s string) bool {
|
||||
return strings.Contains(strings.ToLower(s), q)
|
||||
})
|
||||
}
|
||||
@@ -383,7 +387,7 @@ func NewPricer() func([]Record) Totals {
|
||||
var CSVHeader = []string{"time", "agent", "requested_model", "provider", "host", "model", "served_model", "swapped",
|
||||
"effort", "input_tokens", "output_tokens", "cache_write_tokens", "cache_read_tokens", "reasoning_tokens",
|
||||
"cost_usd", "duration_ms", "ttft_ms", "status", "error", "session", "kind", "provider_key_id", "provider_key_name", "route_id",
|
||||
"request_id", "endpoint", "error_message", "error_type", "source", "rejected", "session_provider", "session_account", "session_official_login"}
|
||||
"request_id", "endpoint", "error_message", "error_type", "source", "rejected", "session_provider", "session_account", "session_official_login", "caller_key_id", "caller_key_name"}
|
||||
|
||||
// WriteCSV writes rows as CSV, a header first: times in RFC 3339 with
|
||||
// their offset, the cost in USD at the effective price (empty when unknown), error
|
||||
@@ -409,7 +413,7 @@ func WriteCSV(w io.Writer, rows []Row) error {
|
||||
cw.Write([]string{r.Time.Format(time.RFC3339), r.Agent, r.Requested, r.Provider, r.Host, r.Model, r.Served,
|
||||
strconv.FormatBool(r.Swapped), r.Effort, n(r.Input), n(r.Output), n(r.CacheWrite), n(r.CacheRead), n(r.Reasoning),
|
||||
cost, strconv.FormatInt(r.Millis, 10), ttft, n(r.Status), strconv.FormatBool(r.Failed()), r.Session, r.Kind, r.ProviderKeyID, r.ProviderKeyName, routeID,
|
||||
r.RequestID, r.Endpoint, r.Error, r.ErrType, r.Source, strconv.FormatBool(r.IsRejected()), r.SessionProvider, r.SessionAccount, strconv.FormatBool(r.SessionOfficialLogin)})
|
||||
r.RequestID, r.Endpoint, r.Error, r.ErrType, r.Source, strconv.FormatBool(r.IsRejected()), r.SessionProvider, r.SessionAccount, strconv.FormatBool(r.SessionOfficialLogin), r.CallerKeyID, r.CallerKeyName})
|
||||
}
|
||||
cw.Flush()
|
||||
return cw.Error()
|
||||
|
||||
@@ -89,8 +89,8 @@ func TestLedger(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := strings.Join(CSVHeader, ",") + "\n" +
|
||||
rows[1].Time.Format(time.RFC3339) + ",codex,relay/sol,relay,relay.example,sol,sol-2026-01-01,false,,10,1,0,0,0,0.000028,100,,200,false,,,,,,,,,,,false,,,false\n" +
|
||||
rows[2].Time.Format(time.RFC3339) + ",codex,fast,relay,relay.example,sol,luna,true,high,2000,500,1000,4000,0,0.012500,3200,400,200,false,s1,,,,123,,,,,,false,,,false\n"
|
||||
rows[1].Time.Format(time.RFC3339) + ",codex,relay/sol,relay,relay.example,sol,sol-2026-01-01,false,,10,1,0,0,0,0.000028,100,,200,false,,,,,,,,,,,false,,,false,,\n" +
|
||||
rows[2].Time.Format(time.RFC3339) + ",codex,fast,relay,relay.example,sol,luna,true,high,2000,500,1000,4000,0,0.012500,3200,400,200,false,s1,,,,123,,,,,,false,,,false,,\n"
|
||||
if b.String() != want {
|
||||
t.Fatalf("csv:\n%s\nwant:\n%s", b.String(), want)
|
||||
}
|
||||
@@ -257,7 +257,7 @@ func TestLedgerWithSessionLogCalls(t *testing.T) {
|
||||
if err := WriteCSV(&b, rows[2:4]); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{",req_log,,,,log,false,,,false\n", ",true,s2,,,,,req_lim,,You've hit your limit,rate_limit,log,false,,,false\n"} {
|
||||
for _, want := range []string{",req_log,,,,log,false,,,false,,\n", ",true,s2,,,,,req_lim,,You've hit your limit,rate_limit,log,false,,,false,,\n"} {
|
||||
if !strings.Contains(b.String(), want) {
|
||||
t.Fatalf("csv lacks %q:\n%s", want, b.String())
|
||||
}
|
||||
|
||||
@@ -33,6 +33,7 @@ type RequestPage struct {
|
||||
Sum Totals
|
||||
Total int
|
||||
Agents, Providers []string
|
||||
CallerKeys []Group
|
||||
Bucket string
|
||||
Series []SeriesPoint
|
||||
By map[string][]Share
|
||||
@@ -40,7 +41,7 @@ type RequestPage struct {
|
||||
|
||||
type packedRow struct {
|
||||
Time time.Time
|
||||
Text [21]uint32
|
||||
Text [23]uint32
|
||||
Tokens [5]int64
|
||||
Millis, TTFT, FirstText, Order int64
|
||||
RouteID int64
|
||||
@@ -59,8 +60,8 @@ type rowChunk struct {
|
||||
Used uint64
|
||||
}
|
||||
|
||||
func rowText(r *Row) [20]*string {
|
||||
return [20]*string{&r.Agent, &r.Provider, &r.Host, &r.SessionProvider, &r.SessionAccount, &r.Model, &r.Requested, &r.Served, &r.Effort, &r.Error, &r.ErrType, &r.RequestID, &r.Endpoint, &r.Session, &r.NativeSession, &r.Kind, &r.Source, &r.Via, &r.ProviderKeyID, &r.ProviderKeyName}
|
||||
func rowText(r *Row) [22]*string {
|
||||
return [22]*string{&r.Agent, &r.Provider, &r.Host, &r.SessionProvider, &r.SessionAccount, &r.Model, &r.Requested, &r.Served, &r.Effort, &r.Error, &r.ErrType, &r.RequestID, &r.Endpoint, &r.Session, &r.NativeSession, &r.Kind, &r.Source, &r.Via, &r.ProviderKeyID, &r.ProviderKeyName, &r.CallerKeyID, &r.CallerKeyName}
|
||||
}
|
||||
func (c *rowChunk) add(r Row, msg string, order int64, failed bool) {
|
||||
if c.dict == nil {
|
||||
@@ -81,7 +82,7 @@ func (c *rowChunk) add(r Row, msg string, order int64, failed bool) {
|
||||
for i, s := range rowText(&r) {
|
||||
p.Text[i] = intern(*s)
|
||||
}
|
||||
p.Text[20] = intern(msg)
|
||||
p.Text[22] = intern(msg)
|
||||
if r.Priced {
|
||||
p.Flags |= 1
|
||||
}
|
||||
@@ -436,7 +437,7 @@ func visibleLocal(chunks []*rowChunk) map[rowRef]bool {
|
||||
seen := map[string]bool{}
|
||||
for _, c := range chunks {
|
||||
for i, p := range c.Rows {
|
||||
msg := c.Strings[p.Text[20]]
|
||||
msg := c.Strings[p.Text[22]]
|
||||
if msg == "" {
|
||||
continue
|
||||
}
|
||||
@@ -625,6 +626,7 @@ func buildRequestPage(p Period, f Filter, offset, limit int, gateway *rowChunk,
|
||||
}
|
||||
out := RequestPage{Rows: []Row{}, Agents: []string{}, Providers: []string{}, By: map[string][]Share{}}
|
||||
agents, providers := map[string]bool{}, map[string]bool{}
|
||||
callers := map[string]*Group{}
|
||||
groups := map[string]map[string]*Share{}
|
||||
seriesGroups := map[string]map[string]*Share{}
|
||||
for _, d := range Dimensions {
|
||||
@@ -648,6 +650,7 @@ func buildRequestPage(p Period, f Filter, offset, limit int, gateway *rowChunk,
|
||||
if r.Provider != "" {
|
||||
providers[r.Provider] = true
|
||||
}
|
||||
addCallerRow(callers, r)
|
||||
keep := f.keeps(r.Record)
|
||||
if keep {
|
||||
out.Total++
|
||||
@@ -719,6 +722,7 @@ func buildRequestPage(p Period, f Filter, offset, limit int, gateway *rowChunk,
|
||||
out.Providers = append(out.Providers, a)
|
||||
}
|
||||
slices.Sort(out.Providers)
|
||||
out.CallerKeys = callerGroups(callers)
|
||||
for _, d := range Dimensions {
|
||||
out.By[d] = sharesOf(groups[d])
|
||||
}
|
||||
@@ -778,6 +782,11 @@ func buildRequestPage(p Period, f Filter, offset, limit int, gateway *rowChunk,
|
||||
func pageFromLedger(p Period, f Filter, offset, limit int, all Ledgered) RequestPage {
|
||||
l := all.Filtered(f)
|
||||
out := RequestPage{Sum: l.Sum, Total: len(l.Rows), Agents: l.Agents, Providers: l.Providers, By: map[string][]Share{}}
|
||||
callers := map[string]*Group{}
|
||||
for i := len(all.Rows) - 1; i >= 0; i-- {
|
||||
addCallerRow(callers, all.Rows[i])
|
||||
}
|
||||
out.CallerKeys = callerGroups(callers)
|
||||
offset = min(offset, len(l.Rows))
|
||||
out.Rows = l.Rows[offset:min(len(l.Rows), offset+limit)]
|
||||
out.Bucket, out.Series = LedgerSeries(p, l.Rows)
|
||||
@@ -796,3 +805,34 @@ func pageFromLedger(p Period, f Filter, offset, limit int, all Ledgered) Request
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Caller choices cover the period, not just the current page or selected key.
|
||||
func addCallerRow(groups map[string]*Group, r Row) {
|
||||
if r.CallerKeyID == "" || r.IsRejected() {
|
||||
return
|
||||
}
|
||||
g := groups[r.CallerKeyID]
|
||||
if g == nil {
|
||||
g = &Group{ID: r.CallerKeyID, CallerKeyID: r.CallerKeyID}
|
||||
groups[r.CallerKeyID] = g
|
||||
}
|
||||
g.CallerKeyName = r.CallerKeyName
|
||||
g.addRow(r)
|
||||
}
|
||||
|
||||
func callerGroups(groups map[string]*Group) []Group {
|
||||
out := make([]Group, 0, len(groups))
|
||||
for _, g := range groups {
|
||||
out = append(out, *g)
|
||||
}
|
||||
slices.SortFunc(out, func(a, b Group) int {
|
||||
if a.Tokens() != b.Tokens() {
|
||||
return b.Tokens() - a.Tokens()
|
||||
}
|
||||
if a.Calls != b.Calls {
|
||||
return b.Calls - a.Calls
|
||||
}
|
||||
return strings.Compare(a.ID, b.ID)
|
||||
})
|
||||
return out
|
||||
}
|
||||
@@ -50,7 +50,7 @@ func (c *rowChunk) pack() *rowChunk {
|
||||
for _, n := range p.Tokens {
|
||||
put(uint64(n))
|
||||
}
|
||||
for _, n := range []int64{p.Millis, p.TTFT, p.FirstText, p.Order} {
|
||||
for _, n := range []int64{p.Millis, p.TTFT, p.FirstText, p.Order, p.RouteID} {
|
||||
put(uint64(n))
|
||||
}
|
||||
put(math.Float64bits(p.Cost))
|
||||
@@ -130,6 +130,7 @@ func (c *rowChunk) unpack() *rowChunk {
|
||||
p.Tokens[j] = int64(take())
|
||||
}
|
||||
p.Millis, p.TTFT, p.FirstText, p.Order = int64(take()), int64(take()), int64(take()), int64(take())
|
||||
p.RouteID = int64(take())
|
||||
p.Cost, p.Status, p.Flags = math.Float64frombits(take()), int32(take()), uint8(take())
|
||||
}
|
||||
if !valid || len(data) != 0 {
|
||||
|
||||
@@ -123,7 +123,7 @@ func TestLedgerResolvesOAuthIdentityAndExportsIt(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := csv.NewReader(&out).ReadAll()
|
||||
if err != nil || len(data) != 2 || data[1][3] != UnknownProvider || data[1][4] != "" || data[1][len(data[1])-3] != "custom" || data[1][len(data[1])-2] != "matched@example.com" || data[1][len(data[1])-1] != "true" {
|
||||
if err != nil || len(data) != 2 || data[1][3] != UnknownProvider || data[1][4] != "" || data[1][slices.Index(CSVHeader, "session_provider")] != "custom" || data[1][slices.Index(CSVHeader, "session_account")] != "matched@example.com" || data[1][slices.Index(CSVHeader, "session_official_login")] != "true" {
|
||||
t.Fatal("CSV lost the resolved provider/account")
|
||||
}
|
||||
}
|
||||
@@ -164,7 +164,7 @@ func TestModelsNeverEstablishProviderOrAccount(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := csv.NewReader(&out).ReadAll()
|
||||
if err != nil || data[1][3] != UnknownProvider || data[1][len(CSVHeader)-3] != "relay" || data[1][len(CSVHeader)-2] != "" || data[1][len(CSVHeader)-1] != "false" {
|
||||
if err != nil || data[1][3] != UnknownProvider || data[1][slices.Index(CSVHeader, "session_provider")] != "relay" || data[1][slices.Index(CSVHeader, "session_account")] != "" || data[1][slices.Index(CSVHeader, "session_official_login")] != "false" {
|
||||
t.Fatalf("unconfirmed CSV route misattributed %+v: %v", data, err)
|
||||
}
|
||||
gateway := []Record{
|
||||
|
||||
+20
-1
@@ -29,6 +29,8 @@ type Record struct {
|
||||
Host string `json:"host,omitempty"` // where the call went: provider.Where then
|
||||
ProviderKeyID string `json:"providerKeyId,omitempty"` // fingerprint of the API key actually used
|
||||
ProviderKeyName string `json:"providerKeyName,omitempty"`
|
||||
CallerKeyID string `json:"callerKeyId,omitempty"`
|
||||
CallerKeyName string `json:"callerKeyName,omitempty"`
|
||||
// SessionProvider is the provider ID recorded by the agent. SessionAccount
|
||||
// identifies the session's creator, not the account used for an API request.
|
||||
// Neither field establishes an upstream route from today's configuration.
|
||||
@@ -291,6 +293,8 @@ type Group struct {
|
||||
Model string `json:"model,omitempty"`
|
||||
ProviderKeyID string `json:"providerKeyId,omitempty"`
|
||||
ProviderKeyName string `json:"providerKeyName,omitempty"`
|
||||
CallerKeyID string `json:"callerKeyId,omitempty"`
|
||||
CallerKeyName string `json:"callerKeyName,omitempty"`
|
||||
// Host is where the calls went, when the provider's id has gone to
|
||||
// more than one place, or elsewhere than the provider goes now: its
|
||||
// calls are then told apart by it, not summed under the id.
|
||||
@@ -329,6 +333,7 @@ type Summary struct {
|
||||
Agents []Group `json:"agents"`
|
||||
Models []Group `json:"models"`
|
||||
ProviderKeys []Group `json:"providerKeys"`
|
||||
CallerKeys []Group `json:"callerKeys"`
|
||||
// Sessions are the calls that named their session, by session.
|
||||
Sessions []Group `json:"sessions"`
|
||||
}
|
||||
@@ -348,7 +353,7 @@ func summarize(p Period, now time.Time, recs []Record) Summary {
|
||||
}
|
||||
}
|
||||
}
|
||||
s := Summary{Period: p, Bucket: "day", Agents: []Group{}, Models: []Group{}, ProviderKeys: []Group{}, Sessions: []Group{}, Series: []Point{}}
|
||||
s := Summary{Period: p, Bucket: "day", Agents: []Group{}, Models: []Group{}, ProviderKeys: []Group{}, CallerKeys: []Group{}, Sessions: []Group{}, Series: []Point{}}
|
||||
var first time.Time
|
||||
for _, r := range recs {
|
||||
if !r.IsRejected() && (first.IsZero() || r.Time.Before(first)) {
|
||||
@@ -388,6 +393,7 @@ func summarize(p Period, now time.Time, recs []Record) Summary {
|
||||
agents := map[string]*Group{}
|
||||
models := map[string]*Group{}
|
||||
keys := map[string]*Group{}
|
||||
callerKeys := map[string]*Group{}
|
||||
sessions := map[string]*Group{}
|
||||
for _, r := range recs {
|
||||
if r.IsRejected() {
|
||||
@@ -420,6 +426,15 @@ func summarize(p Period, now time.Time, recs []Record) Summary {
|
||||
models[k] = m
|
||||
}
|
||||
m.add(r, pr)
|
||||
if r.CallerKeyID != "" {
|
||||
g := callerKeys[r.CallerKeyID]
|
||||
if g == nil {
|
||||
g = &Group{ID: r.CallerKeyID, CallerKeyID: r.CallerKeyID}
|
||||
callerKeys[r.CallerKeyID] = g
|
||||
}
|
||||
g.CallerKeyName = r.CallerKeyName
|
||||
g.add(r, pr)
|
||||
}
|
||||
if keyProviders[r.Provider] {
|
||||
id := r.Provider + "#" + r.ProviderKeyID
|
||||
g := keys[id]
|
||||
@@ -462,6 +477,10 @@ func summarize(p Period, now time.Time, recs []Record) Summary {
|
||||
s.ProviderKeys = append(s.ProviderKeys, *g)
|
||||
}
|
||||
byTokens(s.ProviderKeys)
|
||||
for _, g := range callerKeys {
|
||||
s.CallerKeys = append(s.CallerKeys, *g)
|
||||
}
|
||||
byTokens(s.CallerKeys)
|
||||
for _, g := range sessions {
|
||||
s.Sessions = append(s.Sessions, *g)
|
||||
}
|
||||
|
||||
@@ -84,6 +84,7 @@ const usage = `magpie — one place to pick every agent's model
|
||||
|
||||
magpie serve run the gateway alone (the app runs it too)
|
||||
magpie healthcheck exit 0 when the gateway answers (a container's HEALTHCHECK)
|
||||
magpie gateway-key list|add <name>|rotate <id>|remove <id> manage the keys clients use to call a shared gateway
|
||||
magpie mcp image the image and video generation MCP server an agent is given from the library (stdio)
|
||||
magpie usage [today|7d|30d|all] tokens and cost per agent and model (30d)
|
||||
magpie usage --csv [today|7d|30d|all] every request as CSV: the model asked for, sent and served, tokens, cost, time, status
|
||||
@@ -228,6 +229,8 @@ func run(args []string) error {
|
||||
return groupCmd(args)
|
||||
case "serve":
|
||||
return serve()
|
||||
case "gateway-key":
|
||||
return gatewayKeys(args)
|
||||
case "accounts", "account":
|
||||
return accountsCmd(args)
|
||||
case "usage":
|
||||
|
||||
+2
-2
@@ -794,8 +794,8 @@ func refreshLive(ctx context.Context) {
|
||||
// unless MAGPIE_ADDR puts it on the network (a server, a Docker image)
|
||||
// without sharing it from Settings, when it is anyone who reaches it.
|
||||
func keyNote() string {
|
||||
if s := settings.Load(); s.LAN && s.LANKey != "" {
|
||||
return "(anything works from this machine; from others, the key under Settings → Share on local network)"
|
||||
if s := settings.Load(); s.LAN {
|
||||
return "(anything works from this machine; from others, an enabled gateway key — magpie gateway-key add <name>)"
|
||||
}
|
||||
if h, _, err := net.SplitHostPort(gateway.Addr()); err == nil && h != "localhost" && !net.ParseIP(h).IsLoopback() {
|
||||
return "(anything works, from anyone who reaches it — share it from Settings to require a key)"
|
||||
|
||||
+10
-6
@@ -27,12 +27,16 @@ func TestKeyNote(t *testing.T) {
|
||||
t.Errorf("MAGPIE_ADDR=%q: %q", c.addr, n)
|
||||
}
|
||||
}
|
||||
if err := settings.Save(settings.Settings{LAN: true, LANKey: "sk-magpie-k"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("MAGPIE_ADDR", "0.0.0.0:3425")
|
||||
if n := keyNote(); !strings.Contains(n, "Share on local network") {
|
||||
t.Error("shared:", n)
|
||||
for _, key := range []string{"", "sk-magpie-k"} {
|
||||
if err := settings.Save(settings.Settings{LAN: true, LANKey: key}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, addr := range []string{"", "0.0.0.0:3425"} {
|
||||
t.Setenv("MAGPIE_ADDR", addr)
|
||||
if n := keyNote(); !strings.Contains(n, "gateway key") || strings.Contains(n, "only listens on localhost") {
|
||||
t.Error("shared:", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -125,6 +125,14 @@ func usageTo(out io.Writer, args []string) error {
|
||||
return g.Provider + " / " + name
|
||||
})
|
||||
}
|
||||
if len(s.CallerKeys) > 0 {
|
||||
table("gateway keys", s.CallerKeys, func(g stats.Group) string {
|
||||
if g.CallerKeyName != "" {
|
||||
return g.CallerKeyName
|
||||
}
|
||||
return g.CallerKeyID
|
||||
})
|
||||
}
|
||||
if len(s.Sessions) > 0 {
|
||||
top := s.Sessions[:min(len(s.Sessions), 10)]
|
||||
head := "sessions"
|
||||
|
||||
Reference in new issue
Block a user