* usage: add opt-in OTLP HTTP metadata export
Export gateway traces and optional duration/token histograms asynchronously with a bounded queue, retries, proxy support and a bounded shutdown. Add bilingual Settings controls and MAGPIE_OTEL_* overrides. Keep prompts, replies, sessions and provider credentials local; use the same OTLP path for Langfuse.
Validated: full Go tests (nogui), usage race tests, focused go vet, and Chromium/WebKit settings tests in English and Chinese.
Refs #213
* otel: protect collector credentials in backups and settings
Remove all OTLP headers from keyless backups and restrict both new and existing settings files to 0600 before saving. Clarify SIGTERM export behavior. Addresses the two blocking review findings in #412.
* otel: preserve local credentials and read-only settings on restore
---------
Co-authored-by: raper <raper@raperdeMacBook-Air.local>
* feat(gui): show and filter usage for each API key
* feat(gateway): issue per-user caller keys and attribute usage
* feat(gui): manage gateway users and filter caller-key usage
* test: verify caller-key failover and document user management
* refactor(gateway): use named caller keys without user accounts
* feat(gui): manage named caller keys with existing account styles
* fix: remove user breakdown from CLI and leftover gateway styling
* fix(usage): drop stale user field from request response
* feat(gateway): authenticate LAN calls with named caller keys
* fix(gateway): preserve LAN key rotation in the named caller store
* fix(gui): retain native LAN key copy and rotation controls
* fix(gui): inherit the global theme for API key inputs
* fix(usage): show client API keys using the existing terminology
* fix(gateway): rotate named keys without changing usage identity
* fix(gui): unify gateway keys and selectable connection examples
Keep sharing controls in Settings and manage credentials only in Gateway. Refresh the named-key list on navigation, ignore stale reads after key actions, and generate all API examples from the selected address and enabled key.
* fix(gui): keep caller key names literal in connection menus
* fix(access): use Magpie as default key and drop user migration
* refactor(gui): name caller key source and tests consistently
* refactor(usage): split provider-key accounting out of caller-key PR
* fix(gateway): preserve LAN compatibility and add headless key management
* fix(gui): clarify gateway keys and confirm destructive actions
* test(gateway): guard caller attribution under 120 concurrent requests
* fix(gateway): keep open access, tolerate migration errors and revoke legacy keys
* fix(backup): preserve named gateway keys and honor no-keys restores
* fix(gui): keep local API-key semantics and disambiguate shared credentials
* docs(gateway): keep review scope focused and append browser test documentation
* fix(backup): retain existing default keys when old backups lack their store
* fix(gateway): attribute video creation usage to the caller key
* fix(access): explain default key revocation blocked by read-only settings
* test(usage): preserve provider and caller identities after rebase
* test(gui): reconcile caller key fixtures with merged main
* docs(usage): clarify caller coverage and independent provider accounting
* fix(gui): initialize fast members for model-shortcut groups
Latest main introduced fast-member arrays but omitted the model shortcut's draft. The existing unlisted-models browser regression fails unchanged on main. Initialize fast to an empty list and assert a standard-speed group is saved, without relaxing existing assertions.
* fix(access): retain legacy LAN access when the key store cannot be created
Allow only the configured legacy credential before either migration marker exists. Refuse revoked mirrors, disabled matches and unreadable stores; keep the fallback's caller identity stable when migration later succeeds. Cover read-only configuration directories, remote credential transports and revocation.
* fix(backup): save settings before replacing gateway credentials
Leave the key store untouched when settings validation or writes fail. Restore the previous settings association when the key store refuses its write, and report a rollback failure separately. Cover invalid settings and read-only settings/store failures.
* fix(gui): reset missing caller filters and focus key controls without scrolling
* test(provider): synchronize Zhipu quota fixture observations
A provider may serve a model under an id of its own: namespaced with a
prefix, dated, a `-latest` that is not what models.dev calls it, or simply
a different spelling. magpie sends the name it knows the model by, and the
provider answers that it has no such model.
magpie model wire relay-b/model-2 vendor-c/model-2-preview
magpie model wire 'relay-b/*' 'vendor-c/*'
magpie model wire relay-b/model-2 --reset
A `*` in the **value** stands for the model, so `relay-b/*` set to
`vendor-c/*` sends `vendor-c/model-3` for model-3. A value with no `*` sends
every model of that provider under one name, which is the case where a relay
really does serve them alike. The CLI shows the pattern the way it was
given, with a concrete line for one of the models, so what is in force is
not something you have to imagine. An exact key wins over the provider's.
The canonical id stays canonical. `Entry.Model` is not rewritten: it is the
key automatic routing groups are built from, what `Resolve` returns as the
model's identity, what `GET /models` lists, what an agent asks for, and what
a call is recorded and priced as. The alias goes into the request's model
field and nowhere else — which is the part that was wrong at first.
Handing it to the serializer gave the provider's name to the code that
decides what a request should look like: `buildChat` checks the model for
Gemini compatibility, `buildAnthropic` matches it for adaptive-only and
effort handling, `buildCodeAssist` maps by model. A provider serving a
model under a namespaced id got a request shaped as if it were that other
model. So the request is shaped from the model magpie knows and the field is
rewritten once at the end, with `rewriteModel` — which already existed to
put one provider's own model name in a body and keeps every other byte as
it was.
It has to reach every path. A request the provider understands as it came
is relayed by `passthrough` without being rebuilt, and the Anthropic token
count is asked of the provider directly; both rewrite the field too, so on
a provider speaking the client's own protocol — the common case for a relay
fronting one vendor — the alias still applies.
"Swapped" compared the canonical id with what the vendor's reply named, so
an alias that is not a prefix or a dated variant marked every call swapped.
It compares the name that was actually sent, in the gateway and in the
ledger alike; the ledger reads the map once for the lot, since its rows are
what a reader sets beside a bill. The Codex backend path is left alone: it
sends magpie's own pick and never an alias.
`provider.Test` asked with the canonical id, so a provider that only accepts
the alias reported a failure while its real requests worked. It asks with
the alias now and still reports the canonical id, since that is the one the
user picked.
A blank name was a hole: the setter trimmed what it was given, so a value of
only whitespace was removed on the way in, while the read path would have
used it had anything else written it — a hand-edited file, or a build from
before the trim. Both ends treat a blank as no name now. The key is
checked where it is written rather than in Save, so one hand-edited entry
cannot block every save, and #288's walk moves it on a provider rename
without a line here.
Co-authored-by: yetone <yetoneful@gmail.com>
* gateway: draw through a Grok subscription (Grok Imagine)
A Grok subscription (SuperGrok, X Premium+) could not draw: Drawers() returned
nothing for it and draw() refused with "Grok (SuperGrok) can't draw: magpie
draws only through an OpenAI-compatible API, and Grok (SuperGrok) has none".
Grok Build's own image_gen / image_edit tools call the backend's Imagine API
at <cli-chat-proxy.grok.com/v1>/images/generations and /images/edits with the
CLI's sign-in, which magpie already uses for chat.
- drawsGrok / grokDrawers: a Grok account draws with grok-imagine-image and
grok-imagine-image-quality, like a ChatGPT account does with GPT Image
- the Imagine API takes an aspect ratio, not a size or quality: size
WIDTHxHEIGHT is matched to the nearest ratio it accepts (aspectAmong,
aspectOf is unchanged); the image is asked for as base64
- an edit is JSON with the images as data URLs
- a plan's images cost nothing extra, so AutoDrawer ranks Grok with ChatGPT
(ChatGPT still wins a tie on release date)
- provider.grokBase is exported as GrokBase, like CodexBase, for the test
Tested against a real SuperGrok account through the gateway: generation,
16:9, the quality model and an edit. Video is not covered: the Grok CLI's
video tools need zero data retention off or a storage bucket.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* gateway: film through a Grok subscription, and a generate_video tool for agents
A Grok account that draws (c6578bc) can make videos too: Grok Imagine's
video endpoints sit behind the same sign-in.
The gateway speaks OpenAI's Videos API, so its SDK works as it is:
POST /v1/videos, GET /v1/videos/{id}, GET /v1/videos/{id}/content. A
request is taken on at once and polled; the id carries the provider and
the vendor's request id, so nothing is kept in the gateway and a restart
loses no video. Status is mapped (pending at 0% -> queued, pending ->
in_progress, done -> completed, failed/expired -> failed, with why).
seconds (1-15, the vendor's limit, which it says when exceeded), size
(-> aspect ratio and 480p/720p/1080p by the shorter side), input_reference
for image-to-video (URL, data URL, {image_url}, or a multipart file) and
reference_images are taken; a model that isn't a Grok video model is
refused by name. /content answers 409 until the video is ready.
`magpie mcp image` lists a second tool, generate_video: it starts the
video, waits (up to 10 minutes), saves the mp4 into the project
(generated-videos/ unless path says where, never over a file) and says
where, so an agent can animate an image it just drew by passing its path.
Checked against a real SuperGrok account: text-to-video, image-to-video
(input_reference), reference_images, both video models, the OpenAI
Python SDK (videos.create/retrieve/download_content), the MCP tool end to
end with two videos at once, and the error paths. Unit tests use a fake
Grok upstream.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* gateway: videos follow the Settings' off switch, and an id can't change the vendor's path
Review of the video change:
- Image generation "off" turns off the video a request that names no model
gets (videomaker, beside drawer), so generate_video says it is off too.
- A video id is put in the vendor's URL with the account's token: its parts
must be [A-Za-z0-9_-]+ (parseVideoID) and the vendor id is path-escaped, so
video_grok.x%3Fa=1.1 and %2F answer 404 and ask the vendor nothing.
- The id carries a short mark of the account that started it; after a switch
the video says it was started with another account instead of the vendor's
not-found.
- generate_video waits up to ten minutes, and Codex gives a tool a minute:
magpie-image's Codex entry gets tool_timeout_sec (Goose's timeout) of
eleven minutes, the user's own value kept.
- grok-imagine-video stops at 720p (the vendor turns 1080p away for it; 1.5
makes 1080p), so a 1080p size is asked at 720p there.
- A model may be named without its provider (grok-imagine-video).
- "seconds": 6.0 is 6; 6.5 is still refused.
- completed_at is left out: the vendor doesn't say when a video finished,
and a time made up at each poll changed each time.
Checked against a real SuperGrok account: bare model name, seconds 2.0,
1080p on the base model asked at 720p (the mp4 is 1280x720), 1080p accepted
on 1.5, both malformed ids, no completed_at.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* gateway: TestVideoModelByItsBareName asks its two models in a fixed order
It checked the body of the second request the upstream got, but asked the
two models by ranging over a map, so it failed about one run in seven
(Ubuntu CI). Ask them from a slice.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* library: an installed magpie-image gets the longer tool timeout on the next sync
magpie-image already ships, so many users have it in Codex and Goose as it was
written before: no tool_timeout_sec in Codex, and the timeout: 300 every Goose
server was given. syncMCP skipped an entry whose command, args and env were
unchanged, so an upgrade left Codex at its one-minute default, and Goose kept
300 as if the user had chosen it.
- mcpFile.behind says an entry lacks the timeout magpie gives the server now:
Codex's tool_timeout_sec is missing, or Goose's timeout is still exactly the
300 magpie wrote. syncMCP rewrites such an entry; merged() doesn't keep that
300 as the user's. A value the user chose (any other number) is left alone.
- Test: an entry as main wrote it is given 660 in both, one the user set stays,
and a second sync changes nothing. It fails without the change ("changed []").
Also, from the review:
- seconds is validated once in readFilming: a decimal number that is whole and
at least 1 (hex, inf, nan, underscores, negatives and 0 are refused), and the
object echoes 6 for 6.0.
- imagemcp: Run's doc comment is above Run again.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
A provider that serves a model the catalogue does not list, or lists at the
wrong size, has a window and a reply limit magpie cannot know: both fall
back to whatever the catalogue says about the model's maker, which for such
a provider is nothing at all.
magpie model context relay-a/gpt-5.5 262144
magpie model output relay-a/gpt-5.5 131072
magpie model context 'relay-a/*' 200000
magpie model output relay-a/gpt-5.5 --reset
Both are looked for in the same order as each other: this model, then
`<provider id>/*`, then the provider's own list, then the catalogue. A
model's own value wins over its provider's. The reply limit is new; the
window is not — `Provider.Contexts` has been there, with a `"*"` of its own,
settable through `magpie provider set <id> context.<model>=`, and this
reuses it rather than shadowing it.
That reuse is the point rather than a detail. A context is a routing input
and not only a number agents are shown: at 95% of a held member's window a
request moves to a member that takes more, so two sources of truth would
mean the gateway routing on one number while agents are told another.
`magpie model context` is a second way to set the same field, not a second
field.
Both reach the agents. Pi keeps a model's window and reply limit in its own
models.json, OpenCode keeps its limit, and Crush, Droid, Cline, Qoder and
Zcode keep their own; an agent reads those files when it starts, so a limit
set here that never reaches them is a limit that does not take effect.
`magpie provider set` has always told the agents, and this behaves the same
way rather than introducing a second, quieter path. The test asserts each
setter reaches them exactly once — the two paths differ (one saves the
provider file, the other settings and then says so) and that difference is
worth pinning. An end-to-end pass confirms it: with a Pi models.json in
place, `magpie model context relay-a/sol 999999` writes `contextWindow:
999999` into `providers.magpie.models`.
A `0` is refused rather than read as a reset, naming `--reset`; a mistyped
model id is refused the way `magpie model name` refuses the same id; a reset
says what is actually in force instead of claiming to be back to the
vendor's list while a provider-wide value still applies; and
`magpie model context <provider>/*` reports that value rather than saying
the provider serves no such model. Validation happens where the entry is
written, so one hand-edited bad key cannot stop a theme or a window size
from saving, and #288's walk moves these on a provider rename without a
line here.
Magpie reports models.dev's list price for every call. That is the right
number for a vendor's own key and the wrong one for a provider that
resells or exposes a model at a different tariff, and it leaves a model the
catalogue does not list with no cost at all — a provider's own /models
gives ids, not economics.
magpie model price relay-a/gpt-5.5 0.12,1.20,0.01,0.15
magpie model price relay-a/* 0.12,1.20,0.01,0.15
magpie model price relay-a/gpt-5.5 --reset
magpie model prices
Absolute USD per million tokens. All four parts are required: a price
missing one would understate the rest of every call, and a hand-edited file
that left a part out would bill it at zero, so each part is a pointer and
the setter refuses a price that is negative, NaN or infinite, naming the
part at fault. Zero stays distinct throughout — a model served at no cost
has a price, not the absence of one. A `<provider>/*` entry covers every
model of that provider and an exact entry wins over it; --reset removes
only the exact entry and says so when a provider-wide one is still in
force.
The order a price is looked for in is the price for this model, then
`<provider>/*`, then the provider's own catalogue, then models.dev for the
maker. A price is a provider's tariff, not the model's: the same model
through two providers has two prices, and each keeps its own. A provider is
resolved by its id, or by an id it was renamed from, and never by its
display name — Find answers those too, and a record naming one was counted
at another provider's price.
provider.EffectivePrice is the one lookup every cost caller shares, so the
usage ledger and a session's totals both go through it, and a session
naming a model bare is still priced at its maker's list price, which is a
different question from what one provider charges. catalog.PriceOf and
PricedBy keep returning the models.dev price for anything wanting an
unbiased number. The background-task pickers (draw, search, vision) still
rank by catalogue price: they choose which model answers, not what a call
is reported to cost.
Unlike every other per-model preference this does not call
catalog.Touched. What a call costs is not what an agent picks a model by,
and the model lists magpie keeps in the agents' own files are not its to
rewrite over a number in a cost report. A rename moves the prices with it
through the walk #288 adds, so nothing here has to say so, and an id
recorded before a rename still resolves.
Validation happens where the entry is written, not in Save: one hand-edited
bad price in the file must not make saving a theme or a window size fail. A
price for a model the provider does not serve is refused the way
`magpie model name` refuses the same id, since it is a price that never
applies and nothing reports it as dead; a removal is exempt, so an entry
left for a model that has since gone can still be taken away.
Three changes let a user say something about one model the catalogue does
not carry: what it costs, how large it is, and what the provider calls it
upstream. They are separate pull requests and none needs another's
feature, but all three need the two things here, so this lands first.
**CheckModelKey** — those keys are `"<provider id>/<model id>"`, and
`"<provider id>/*"` covers every model of a provider. A model id may contain
slashes of its own, so only the part before the first is the provider's. A
key that does not parse is one no lookup will ever match, which is worth
refusing where the entry is written rather than leaving as a silent no-op.
**One convention instead of two lists.** Settings saves by decoding the
whole struct out of a request body and copying back the fields it knows
about; renaming a provider moves the keys of the maps it knows about. Both
were hand-maintained lists, and a per-model map added outside either was
not saved by the first and not moved by the second.
`settings` now walks its own fields for the convention — a field named
`Model*` of type `map[string]X` — and does both jobs from it:
`RenamePerModel` for a provider that changes its id, `CarryPerModel` for the
settings page. A map added later needs no line in either file. A field whose
keys are not a model's is not one of these and is not named `Model*`:
`Visible`, which is by agent id, and `HiddenModels`, which is by agent and
model, among them.
`TestRenameMovesEveryPerModelMap` populates every per-model map at once and
renames once, so it fails both for a map left out and for a map that never
ran. `TestSettingsPageSaveKeepsWhatOtherPagesSet` drives the route that
saves rather than a copy of it: with three per-model maps a hand-written
list and this walk do the same thing, so only a helper that stops working
is distinguishable today — the test earns its keep the day a fourth map
appears and someone goes back to writing the list out.
* feat(gui): choose macOS session terminal
* session terminal: build the darwin CLI without cgo, drop the dead macOS 11 path, and open Terminal when the .command default is not a terminal
- terminal_apps_darwin.go imports "C" but had no cgo tag, and terminal_apps_other.go was !darwin only, so `CGO_ENABLED=0 GOOS=darwin go build -tags nogui .` (make release-cli's darwin targets) failed with undefined discoverTerminals. The cgo file and its test are now darwin && cgo; the stub is !darwin || !cgo.
- magpie needs macOS 12 (LSMinimumSystemVersion, -mmacosx-version-min=12.0), so the macOS 11 branch (temp .command file, deprecated LSCopyApplicationURLsForURL, CFRelease) never ran; it is gone along with the CoreServices link.
- With no saved choice, a default .command handler that is not a listed terminal, or a failed lookup, now opens Terminal as before the PR instead of failing; Settings labels the system option "System default (Terminal)" then, and the unused "System default (unavailable)" string is dropped.
- session-terminal.test.cjs covers an editor as the default in Chromium and WebKit, en and zh, and is on the README's node --test line.
---------
Co-authored-by: yetone <yetoneful@gmail.com>
* provider: a model can be told to accept images
Reasoning levels can be narrowed per model, but whether a model sees
images came only from the vendor's list. A relay or a model the list
hasn't caught yet could not be marked as taking images, or as text
only. The names editor now has Accepts images. The choice is kept
beside the name and the levels, and is what agents and the gateway's
image check are told. The same answer as the vendor's is not stored.
Restore default clears it.
Tested with TestModelImage. go build -tags nogui passes.
* settings: a save of the Settings page keeps the models' image choices, as it keeps their names and levels
POST /api/settings copied Visible, ModelNames and ModelEfforts over from the current settings but not the new ModelImages, and the Settings page never sends it, so switching the theme or language dropped every "Accepts images" override. Tested with TestSettingsSaveKeepsModelChoices (fails without the fix: images map[]).
---------
Co-authored-by: yetone <yetoneful@gmail.com>
The rate is internal/fx: a free, no-key endpoint (open.er-api.com/v6/latest/USD), cached in memory and in a small file beside magpie's other caches (XDG_CACHE_HOME/magpie/fxrate.json) for 12h, refetched only once that's stale, and falling back to the last rate it ever had — or a fixed 7.2 — while offline; the currency row's tooltip names the rate and when it was learned. usage.FormatCost carries the $/¥ formatting (2 decimals above $1, 3 below, whole numbers above 100) that usage_cli.go and internal/tui/pages.go used to each have their own copy of; both now read Settings' currency once per command/redraw rather than per row. internal/gui/api.go exposes the rate on /api/settings always and on /api/state only when cny is actually chosen, so usd users' hot path never touches the network for a rate nothing shows with.
Tested with new internal/fx/fx_test.go (a fake server: fetches and caches, answers from memory then from the cache file with no further calls, refetches once stale, falls back to a stale cache or Fallback when offline, ignores a reply with no CNY rate), TestFXInState in internal/gui/fx_test.go, TestFmtCostShowsChosenCurrency in internal/tui/usage_test.go, and a new internal/gui/tests/currency.test.cjs in Chromium and WebKit, English and Chinese (the Usage page's $ by default and ¥12.34×7.2 once cny is picked, the currency row's labels/state/tooltip, a real wheel scroll deep into Settings left alone by picking a currency, the choice surviving a reload). go build -tags nogui, GOOS=linux and GOOS=windows builds, and go test -tags nogui -count=1 ./... all pass clean.
A fetch that failed is not tried again for 10 minutes (the stale or fixed rate shown at once meanwhile), so offline each state() does not wait up to 4s on the network: TestGetDoesNotRetryAFailedFetchAtOnce.
The product, module path (github.com/yetone/magpie), binaries, env vars
(MAGPIE_ADDR, MAGPIE_DEV_UI, MAGPIE_THEME, MAGPIE_DEBUG), config and cache
folders (~/.config/magpie, ~/.cache/magpie), bundle id (com.yetone.magpie),
gateway provider id and token, and every UI string now say magpie.
On first run magpie copies ~/.config/dial and ~/.cache/dial to the new
folders when they don't exist yet (copy, not move, so a still-running dial
keeps working; the old folders can be deleted by hand later).
The logo is a small round magpie in profile: one path for the header and
the tray, and the app tile adds the white belly and the blue-green tail.
The header mark is drawn in ink rather than the accent colour, which
fought the text in light mode.
yetone