gateway: a custom provider's video models, marked "kind": "video" by its own model list, are started, polled and fetched at its OpenAI Videos API (#1446, Edward4226)
What changed
- Videomakers: a custom provider (no account, not a remote magpie, no
preset plan's videos, a Chat or Responses base) makes videos with the
models its fetched /v1/models marks "kind": "video"
(catalog.LiveVideomakers). Nothing is guessed from a name: sora-like-2
without the kind, a chat model and grok-imagine-video named on it all
still get the 400 and reach no upstream video route.
- /v1/videos, /v1/videos/{id} and /v1/videos/{id}/content forward to
{base}/videos the way Remote magpie's already do (filmsOpenAI), where
base is the Chat base, else the Responses base (videoBase). The vendor
gets its own User-Agent and no caller headers; passOnCaller stays
remote-magpie only.
- Custom video ids base64 the vendor's id (its own may hold dots) and end
with the key's fingerprint (customVideoID, markedKey), so a poll and its
download use the key that started the video, and a key turned off since
answers 404 "no longer on". Remote magpie's and Volcengine's id formats
are unchanged.
- Gateway key holds on all three routes: account/key holds before any
upstream call (videoAsker), model holds once the model is known
(videoModelHeld). Grok's and Volcengine's polls and downloads ignored a
key's model holds before; they now apply them too.
- videoObject's done/succeeded mapping is untouched: the custom path passes
the vendor's OpenAI object through (asOurs), so its "completed" reaches
the client as sent.
- docs/subsystems/gateway-routing.md says so under Remote magpie APIs.
Why
- Edward4226's provider serves OpenAI's Videos API, and magpie answered
400 "can't make videos" because only Grok, Volcengine and Remote magpie
were allowed.
What was run
- New tests in video_custom_test.go (httptest OpenAI Videos fake whose id
holds a dot; start, two polls, content bytes; exact upstream request log;
Responses-only base; two keys with holds; Grok poll/content held by key
models). On the old video.go 4 of 5 fail (videomakers [], 400 "can't
make videos", held Grok poll 200); the name-guard test passes on both,
as it should. Breaking markedKey by hand, or the content model check,
turns TestCustomProviderVideoKeys red.
- go test -tags nogui -count=1 ./internal/gateway/ ./internal/catalog/
./internal/provider/: ok. gofmt -l clean; go vet -tags nogui
./internal/gateway/ on darwin, GOOS=linux and GOOS=windows: clean.
- End to end: the built magpie (nogui) under a scratchpad HOME, `magpie
provider add Studio url=http://127.0.0.1:3561/v1 key=…`, `magpie serve`
on 127.0.0.1:3562, against a Go fake of the Videos API on 3561. curl
start -> queued, poll -> in_progress, poll -> completed, content ->
video/mp4 whose 200000 bytes cmp equal to what the fake served. The fake
saw POST /v1/videos with model vid-1, the polls and /content, each with
the provider's key and no X-Magpie-Agent.
- No real video vendor was used. Multipart uploads to a real OpenAI
upstream weren't tried. The GUI lists no video models per provider, so
it has nothing to check.
This commit is contained in:
1 parent
af1e477d83
commit
916c1cc7ea
3 files changed
+533
-21
No files matched your search
@@ -92,6 +92,17 @@ them. Decision, retrieval and media requests set magpie's User-Agent at
|
||||
their own call sites, so the remote accepts their attribution. Vendors
|
||||
receive no caller labels.
|
||||
|
||||
A custom provider (no account, no preset plan's videos) makes videos the
|
||||
same OpenAI way (#1446, `customVideomakers`), but only with the models its
|
||||
own fetched list marks `"kind": "video"` (`catalog.LiveVideomakers`); a
|
||||
name alone never makes a model a video maker. Start, poll and content go to
|
||||
`videoBase` (its Chat base, else its Responses base) `+ /videos`, with the
|
||||
vendor's own User-Agent and no caller headers. Its ids end with the key's
|
||||
fingerprint (`customVideoID`, `markedKey`), so a poll and its download go
|
||||
on the key that started the video. A gateway key's account and model holds
|
||||
apply to all three routes (`videoAsker`, `videoModelHeld`), and to Grok's
|
||||
and Volcengine's polls and downloads as well.
|
||||
|
||||
## Verification
|
||||
|
||||
```sh
|
||||
|
||||
+199
-21
@@ -43,6 +43,9 @@ import (
|
||||
// Another magpie's videos (a Remote magpie's, #545) are asked of it, at its
|
||||
// own videos API: its id for one goes, encoded, in the id given here, and
|
||||
// what it answers comes back with this magpie's id and name for the model.
|
||||
// A provider set up with a base URL whose own model list marks a model
|
||||
// "kind": "video" (#1446) is asked the same way, at the OpenAI-shaped
|
||||
// videos API under its base, on the key that started the video.
|
||||
|
||||
// grokVideoModels are the video models a Grok subscription makes videos with.
|
||||
var grokVideoModels = []catalog.Model{
|
||||
@@ -57,8 +60,9 @@ var grokVideoAspects = []string{"1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3
|
||||
const maxVideoBytes = 512 << 20
|
||||
|
||||
// Videomakers are the models a provider can make videos with: a Grok
|
||||
// subscription's, another magpie's, and a preset plan's models when magpie
|
||||
// implements that vendor's video API.
|
||||
// subscription's, another magpie's, a preset plan's models when magpie
|
||||
// implements that vendor's video API, and those a provider's own model list
|
||||
// marks as video models (customVideomakers).
|
||||
func Videomakers(p provider.Provider) []catalog.Model {
|
||||
if p.IsRemoteMagpie() {
|
||||
out := catalog.LiveVideomakers(p.ID)
|
||||
@@ -71,7 +75,7 @@ func Videomakers(p provider.Provider) []catalog.Model {
|
||||
return out
|
||||
}
|
||||
if !drawsGrok(p) {
|
||||
return nil
|
||||
return customVideomakers(p)
|
||||
}
|
||||
out := slices.Clone(grokVideoModels)
|
||||
for i := range out {
|
||||
@@ -80,6 +84,68 @@ func Videomakers(p provider.Provider) []catalog.Model {
|
||||
return out
|
||||
}
|
||||
|
||||
// customVideomakers are the video models of a provider set up with a base
|
||||
// URL — not a subscription's account, not another magpie, not a preset plan
|
||||
// whose own video API magpie speaks — that its fetched model list marks
|
||||
// "kind": "video" (catalog.LiveVideomakers, #1446). They are asked at the
|
||||
// OpenAI-shaped videos API under its base, as another magpie's are. A
|
||||
// model's name alone makes none of them one: a list that doesn't say so
|
||||
// leaves the provider making no videos.
|
||||
func customVideomakers(p provider.Provider) []catalog.Model {
|
||||
if p.Account != nil || p.IsRemoteMagpie() || len(p.PlanVideos()) > 0 || videoBase(p) == "" {
|
||||
return nil
|
||||
}
|
||||
out := catalog.LiveVideomakers(p.ID)
|
||||
for i := range out {
|
||||
out[i].Provider = p.ID
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// videoBase is the root of the OpenAI-shaped videos API of another magpie
|
||||
// or a provider set up with a base URL: its Chat base, or a Responses-only
|
||||
// relay's Responses base, under which it serves the images API too.
|
||||
func videoBase(p provider.Provider) string {
|
||||
return strings.TrimRight(cmp.Or(p.Base(provider.Chat), p.Base(provider.Responses)), "/")
|
||||
}
|
||||
|
||||
// filmsCustom is whether p makes model at its OpenAI-shaped videos API, as
|
||||
// its model list says (customVideomakers).
|
||||
func filmsCustom(p provider.Provider, model string) bool {
|
||||
return slices.ContainsFunc(customVideomakers(p), func(m catalog.Model) bool { return m.ID == model })
|
||||
}
|
||||
|
||||
// keyMark is the mark of the key p sends with, in the id of a video it
|
||||
// started at its OpenAI-shaped videos API: the vendor keeps a video for the
|
||||
// key that started it, so it is asked after on that key, whichever comes
|
||||
// first now. "0" for a provider with no key (a local server).
|
||||
func keyMark(p provider.Provider) string {
|
||||
if p.Key == "" {
|
||||
return "0"
|
||||
}
|
||||
return "k" + provider.KeyID(p.Key)
|
||||
}
|
||||
|
||||
// customVideoID is the id of a video p started at its OpenAI-shaped videos
|
||||
// API, by the vendor's id for it, which goes in base64 (it may hold dots).
|
||||
func customVideoID(p provider.Provider, theirs string, t time.Time) string {
|
||||
return "video_" + p.ID + "." + base64.RawURLEncoding.EncodeToString([]byte(theirs)) + "." + strconv.FormatInt(t.Unix(), 10) + "." + keyMark(p)
|
||||
}
|
||||
|
||||
// markedKey is p on its key whose mark is by (keyMark), while that key is
|
||||
// still on.
|
||||
func markedKey(p provider.Provider, by string) (provider.Provider, bool) {
|
||||
if p.Key == "" {
|
||||
return p, by == "0"
|
||||
}
|
||||
for _, k := range p.KeysOn() {
|
||||
if q := p.WithKey(k); keyMark(q) == by && videoBase(q) != "" {
|
||||
return q, true
|
||||
}
|
||||
}
|
||||
return p, false
|
||||
}
|
||||
|
||||
// videomaker is the model a request that names none makes its video with:
|
||||
// AutoVideomaker's, unless the Settings turned image generation off, which
|
||||
// turns video off with it. None when it is off or no provider makes videos.
|
||||
@@ -435,6 +501,14 @@ func videoMaker(id string) (p provider.Provider, vendorID string, started time.T
|
||||
return p, "", started, fmt.Errorf("%q isn't the id of a video magpie is making", id)
|
||||
}
|
||||
found, ferr := provider.Find(pid)
|
||||
if ferr == nil && !drawsGrok(*found) && !found.IsRemoteMagpie() && len(found.PlanVideos()) == 0 && len(customVideomakers(*found)) > 0 {
|
||||
// asked on the key that started it
|
||||
q, ok := markedKey(*found, by)
|
||||
if !ok {
|
||||
return p, "", started, fmt.Errorf("this video was started with a key of %s that is no longer on: turn it back on to get the video", found.Name)
|
||||
}
|
||||
return q, vendorID, started, nil
|
||||
}
|
||||
if ferr != nil || !drawsGrok(*found) && !found.IsRemoteMagpie() && len(found.PlanVideos()) == 0 {
|
||||
return p, "", started, fmt.Errorf("no provider %q makes videos here", pid)
|
||||
}
|
||||
@@ -494,10 +568,16 @@ func asOurs(b []byte, p provider.Provider, id string) (map[string]any, error) {
|
||||
return obj, nil
|
||||
}
|
||||
|
||||
// remoteVideoURL is where another magpie answers for a video, with suffix
|
||||
// ("" or "/content").
|
||||
// filmsOpenAI is whether p makes videos at an OpenAI-shaped videos API:
|
||||
// another magpie, or a provider whose model list marks video models.
|
||||
func filmsOpenAI(p provider.Provider) bool {
|
||||
return p.IsRemoteMagpie() || len(customVideomakers(p)) > 0
|
||||
}
|
||||
|
||||
// remoteVideoURL is where another magpie, or a provider's OpenAI-shaped
|
||||
// videos API, answers for a video, with suffix ("" or "/content").
|
||||
func remoteVideoURL(p provider.Provider, vendorID, suffix string) string {
|
||||
return strings.TrimRight(p.Base(provider.Chat), "/") + "/videos/" + url.PathEscape(theirVideoID(vendorID)) + suffix
|
||||
return videoBase(p) + "/videos/" + url.PathEscape(theirVideoID(vendorID)) + suffix
|
||||
}
|
||||
|
||||
// videoStatus asks the vendor how a video is going.
|
||||
@@ -547,7 +627,7 @@ func (s *Server) videosCreate(w http.ResponseWriter, r *http.Request) {
|
||||
if f.Model == "" {
|
||||
m, ok := videomaker()
|
||||
if !ok {
|
||||
fail(400, "no model to make videos with: sign in to a Grok subscription and leave Settings → Images → Image generation on, or name one")
|
||||
fail(400, "no model to make videos with: sign in to a Grok subscription or set up a provider that makes videos, and leave Settings → Images → Image generation on, or name one")
|
||||
return
|
||||
}
|
||||
f.Model, call.Model = m, m
|
||||
@@ -580,10 +660,11 @@ func (s *Server) videosCreate(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
call.Provider, call.To = p.ID, provider.Chat
|
||||
remote := p.IsRemoteMagpie()
|
||||
custom := !remote && filmsCustom(p, model)
|
||||
volcengine := p.Preset == "volcengine" && slices.ContainsFunc(p.PlanVideos(), func(m catalog.Model) bool { return m.ID == model })
|
||||
// any grok-imagine-video*, not only those listed: the vendor's newer ones work before magpie names them
|
||||
if !remote && !volcengine && (len(Videomakers(p)) == 0 || !strings.HasPrefix(model, "grok-imagine-video")) {
|
||||
fail(400, fmt.Sprintf("%s/%s can't make videos: magpie makes videos with a Grok subscription's grok-imagine-video or Volcengine Agent Plan's Seedance models", p.ID, model))
|
||||
if !remote && !custom && !volcengine && (!drawsGrok(p) || !strings.HasPrefix(model, "grok-imagine-video")) {
|
||||
fail(400, fmt.Sprintf(`%s/%s can't make videos: magpie makes videos with a Grok subscription's grok-imagine-video, Volcengine Agent Plan's Seedance models, another magpie's, and the models a provider's own model list marks "kind": "video"`, p.ID, model))
|
||||
return
|
||||
}
|
||||
var unmask func()
|
||||
@@ -594,8 +675,10 @@ func (s *Server) videosCreate(w http.ResponseWriter, r *http.Request) {
|
||||
at, field := strings.TrimRight(p.Base(provider.Responses), "/")+"/videos/generations", "request_id"
|
||||
var body []byte
|
||||
switch {
|
||||
case remote:
|
||||
at, field, body = strings.TrimRight(p.Base(provider.Chat), "/")+"/videos", "id", remoteVideoBody(model, f)
|
||||
case remote, custom:
|
||||
// a provider's OpenAI-shaped videos API is asked as another
|
||||
// magpie's is: OpenAI's body, its answer its own
|
||||
at, field, body = videoBase(p)+"/videos", "id", remoteVideoBody(model, f)
|
||||
case volcengine:
|
||||
at, field, body = strings.TrimRight(p.Base(provider.Responses), "/")+"/contents/generations/tasks", "id", volcengineVideoBody(model, f)
|
||||
default:
|
||||
@@ -632,9 +715,71 @@ func (s *Server) videosCreate(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, 200, obj)
|
||||
return
|
||||
}
|
||||
if custom {
|
||||
obj, _ := asOurs(b, p, customVideoID(p, theirs, start))
|
||||
if _, ok := obj["model"]; !ok {
|
||||
obj["model"] = p.ID + "/" + model
|
||||
}
|
||||
writeJSON(w, 200, obj)
|
||||
return
|
||||
}
|
||||
writeJSON(w, 200, videoObject(videoID(p, theirs, start), p.ID+"/"+model, start, videoState{}, f))
|
||||
}
|
||||
|
||||
// videoAsker is p on a key or account the calling key may use, to ask
|
||||
// after a video (#905), as starting one is held: a video at the
|
||||
// OpenAI-shaped videos API of a provider set up with a base URL on the key
|
||||
// that started it, which the calling key must be allowed; any other on the
|
||||
// first key or account it may use, as videosCreate picks. What the calling
|
||||
// key is told when it may use none, else "".
|
||||
func videoAsker(r *http.Request, p provider.Provider) (provider.Provider, string) {
|
||||
who, held := accountHolds(r)
|
||||
if !held {
|
||||
return p, ""
|
||||
}
|
||||
if !p.IsRemoteMagpie() && filmsOpenAI(p) {
|
||||
if p.Key == "" && p.Account == nil || accountAllowed(who, candidate{p: p}) {
|
||||
return p, ""
|
||||
}
|
||||
return p, keyAccountsError(who, p.ID)
|
||||
}
|
||||
q, ok := allowedKey(who, p, "")
|
||||
if !ok {
|
||||
return p, keyAccountsError(who, p.ID)
|
||||
}
|
||||
return q, ""
|
||||
}
|
||||
|
||||
// videoModelHeld is what the calling key is told when its models hold it
|
||||
// from a video of p's model (#882), as they hold it from starting one: ""
|
||||
// when they don't. A video whose model the vendor doesn't say is held.
|
||||
func videoModelHeld(r *http.Request, p provider.Provider, model string) string {
|
||||
who, held := keyHolds(r)
|
||||
if !held || model != "" && modelAllowed(who, p, model) {
|
||||
return ""
|
||||
}
|
||||
if model == "" {
|
||||
return fmt.Sprintf("The gateway key %q may not ask after this video: %s didn't say which model makes it.", who.KeyName, p.Name)
|
||||
}
|
||||
return keyModelError(who, p.ID+"/"+model)
|
||||
}
|
||||
|
||||
// openAIVideo is how a video at an OpenAI-shaped videos API (another
|
||||
// magpie's, a provider's) is going, with this magpie's id for it, and the
|
||||
// model making it as that API names it.
|
||||
func (s *Server) openAIVideo(ctx context.Context, p provider.Provider, vendorID, id string) (map[string]any, string, int, error) {
|
||||
b, code, err := s.sendAs(ctx, p, http.MethodGet, remoteVideoURL(p, vendorID, ""), "", nil, true)
|
||||
if err != nil {
|
||||
return nil, "", code, err
|
||||
}
|
||||
obj, err := asOurs(b, p, id)
|
||||
if err != nil {
|
||||
return nil, "", 502, err
|
||||
}
|
||||
model, _ := obj["model"].(string)
|
||||
return obj, strings.TrimPrefix(model, p.ID+"/"), 200, nil
|
||||
}
|
||||
|
||||
// videosGet answers how a video is going.
|
||||
func (s *Server) videosGet(w http.ResponseWriter, r *http.Request) {
|
||||
// A later poll can echo the prompt masked when the video was created.
|
||||
@@ -647,17 +792,21 @@ func (s *Server) videosGet(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, provider.Chat, 404, err.Error())
|
||||
return
|
||||
}
|
||||
p, held := videoAsker(r, p)
|
||||
if held != "" {
|
||||
writeError(w, provider.Chat, 403, held)
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), drawTimeout)
|
||||
defer cancel()
|
||||
if p.IsRemoteMagpie() {
|
||||
b, code, err := s.sendAs(ctx, p, http.MethodGet, remoteVideoURL(p, vendorID, ""), "", nil, true)
|
||||
if filmsOpenAI(p) {
|
||||
obj, model, code, err := s.openAIVideo(ctx, p, vendorID, id)
|
||||
if err != nil {
|
||||
writeError(w, provider.Chat, code, err.Error())
|
||||
return
|
||||
}
|
||||
obj, err := asOurs(b, p, id)
|
||||
if err != nil {
|
||||
writeError(w, provider.Chat, 502, err.Error())
|
||||
if held := videoModelHeld(r, p, model); held != "" {
|
||||
writeError(w, provider.Chat, 403, held)
|
||||
return
|
||||
}
|
||||
writeJSON(w, 200, obj)
|
||||
@@ -672,6 +821,10 @@ func (s *Server) videosGet(w http.ResponseWriter, r *http.Request) {
|
||||
if model == "" {
|
||||
model = "grok-imagine-video"
|
||||
}
|
||||
if held := videoModelHeld(r, p, model); held != "" {
|
||||
writeError(w, provider.Chat, 403, held)
|
||||
return
|
||||
}
|
||||
writeJSON(w, 200, videoObject(id, p.ID+"/"+model, started, st, filming{}))
|
||||
}
|
||||
|
||||
@@ -683,9 +836,26 @@ func (s *Server) videosContent(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, provider.Chat, 404, err.Error())
|
||||
return
|
||||
}
|
||||
p, held := videoAsker(r, p)
|
||||
if held != "" {
|
||||
writeError(w, provider.Chat, 403, held)
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), drawTimeout)
|
||||
defer cancel()
|
||||
if p.IsRemoteMagpie() {
|
||||
if filmsOpenAI(p) {
|
||||
if _, models := keyHolds(r); models {
|
||||
// the bytes say no model: the video's own answer does
|
||||
_, model, code, err := s.openAIVideo(ctx, p, vendorID, id)
|
||||
if err != nil {
|
||||
writeError(w, provider.Chat, code, err.Error())
|
||||
return
|
||||
}
|
||||
if held := videoModelHeld(r, p, model); held != "" {
|
||||
writeError(w, provider.Chat, 403, held)
|
||||
return
|
||||
}
|
||||
}
|
||||
s.remoteVideoContent(ctx, w, p, vendorID)
|
||||
return
|
||||
}
|
||||
@@ -694,6 +864,10 @@ func (s *Server) videosContent(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, provider.Chat, code, err.Error())
|
||||
return
|
||||
}
|
||||
if held := videoModelHeld(r, p, cmp.Or(st.Model, "grok-imagine-video")); held != "" {
|
||||
writeError(w, provider.Chat, 403, held)
|
||||
return
|
||||
}
|
||||
if st.Status != "done" && st.Status != "succeeded" {
|
||||
obj := videoObject(id, "", time.Time{}, st, filming{})
|
||||
msg := fmt.Sprintf("the video isn't ready: it is %v", obj["status"])
|
||||
@@ -733,8 +907,9 @@ func (s *Server) videosContent(w http.ResponseWriter, r *http.Request) {
|
||||
io.Copy(w, io.LimitReader(res.Body, maxVideoBytes))
|
||||
}
|
||||
|
||||
// remoteVideoContent passes on the bytes of a video another magpie made, or
|
||||
// what it says of one it hasn't.
|
||||
// remoteVideoContent passes on the bytes of a video another magpie, or a
|
||||
// provider's OpenAI-shaped videos API, made, or what it says of one it
|
||||
// hasn't.
|
||||
func (s *Server) remoteVideoContent(ctx context.Context, w http.ResponseWriter, p provider.Provider, vendorID string) {
|
||||
ctx = p.Via(ctx)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, remoteVideoURL(p, vendorID, "/content"), nil)
|
||||
@@ -742,8 +917,11 @@ func (s *Server) remoteVideoContent(ctx context.Context, w http.ResponseWriter,
|
||||
writeError(w, provider.Chat, 502, err.Error())
|
||||
return
|
||||
}
|
||||
req.Header.Set("User-Agent", "magpie/"+Version)
|
||||
passOnCaller(ctx, req)
|
||||
if p.IsRemoteMagpie() {
|
||||
// vendors are told no caller
|
||||
req.Header.Set("User-Agent", "magpie/"+Version)
|
||||
passOnCaller(ctx, req)
|
||||
}
|
||||
if err := p.Sign(ctx, req, provider.Chat, nil); err != nil {
|
||||
writeError(w, provider.Chat, 502, err.Error())
|
||||
return
|
||||
|
||||
@@ -0,0 +1,323 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/yetone/magpie/internal/access"
|
||||
"github.com/yetone/magpie/internal/provider"
|
||||
)
|
||||
|
||||
// openAIVideos is an upstream serving OpenAI's Videos API under /v1, as a
|
||||
// provider set up with a base URL does (#1446): its model list marks
|
||||
// vid-1 "kind": "video" and lists sora-like-2 with no kind, a video takes
|
||||
// two polls to be made, and what each request was sent and signed with is
|
||||
// kept.
|
||||
type openAIVideos struct {
|
||||
*httptest.Server
|
||||
mu sync.Mutex
|
||||
asked []string // "METHOD path key"
|
||||
bodies []string // bodies of POST /v1/videos
|
||||
heads []http.Header
|
||||
polls int
|
||||
}
|
||||
|
||||
func newOpenAIVideos(t *testing.T) *openAIVideos {
|
||||
t.Helper()
|
||||
u := &openAIVideos{}
|
||||
u.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
b, _ := io.ReadAll(r.Body)
|
||||
u.mu.Lock()
|
||||
defer u.mu.Unlock()
|
||||
key := strings.TrimPrefix(r.Header.Get("Authorization"), "Bearer ")
|
||||
u.asked = append(u.asked, r.Method+" "+r.URL.Path+" "+key)
|
||||
u.heads = append(u.heads, r.Header.Clone())
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
// the vendor's id has a dot, which magpie's own ids can't carry bare
|
||||
const vid = "video_68d7.abc"
|
||||
switch {
|
||||
case r.Method == "GET" && r.URL.Path == "/v1/models":
|
||||
io.WriteString(w, `{"object":"list","data":[{"id":"gpt-5.5","object":"model"},{"id":"vid-1","object":"model","kind":"video"},{"id":"sora-like-2","object":"model"}]}`)
|
||||
case r.Method == "POST" && r.URL.Path == "/v1/videos":
|
||||
u.bodies = append(u.bodies, string(b))
|
||||
io.WriteString(w, `{"id":"`+vid+`","object":"video","created_at":1790000000,"status":"queued","progress":0,"model":"vid-1","seconds":"4","size":"1280x720"}`)
|
||||
case r.Method == "GET" && r.URL.Path == "/v1/videos/"+vid:
|
||||
u.polls++
|
||||
switch u.polls {
|
||||
case 1:
|
||||
io.WriteString(w, `{"id":"`+vid+`","object":"video","status":"in_progress","progress":40,"model":"vid-1"}`)
|
||||
default:
|
||||
io.WriteString(w, `{"id":"`+vid+`","object":"video","status":"completed","progress":100,"model":"vid-1","seconds":"4"}`)
|
||||
}
|
||||
case r.Method == "GET" && r.URL.Path == "/v1/videos/"+vid+"/content":
|
||||
w.Header().Set("Content-Type", "video/mp4")
|
||||
io.WriteString(w, "\x00\x00\x00\x18ftypmp42fake-mp4")
|
||||
default:
|
||||
w.WriteHeader(404)
|
||||
io.WriteString(w, `{"error":{"message":"no route `+r.Method+" "+r.URL.Path+`","type":"invalid_request_error"}}`)
|
||||
}
|
||||
}))
|
||||
t.Cleanup(u.Close)
|
||||
return u
|
||||
}
|
||||
|
||||
func (u *openAIVideos) log() []string {
|
||||
u.mu.Lock()
|
||||
defer u.mu.Unlock()
|
||||
return slices.Clone(u.asked)
|
||||
}
|
||||
|
||||
// saveFetched saves p and fetches its model list, as adding it does.
|
||||
func saveFetched(t *testing.T, p provider.Provider) provider.Provider {
|
||||
t.Helper()
|
||||
if err := provider.Save(p); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found, err := provider.Find(p.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := found.Fetch(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return *found
|
||||
}
|
||||
|
||||
func asKey(t *testing.T, h http.Handler, secret, method, path, body string) (int, map[string]any, []byte) {
|
||||
t.Helper()
|
||||
r := httptest.NewRequest(method, path, strings.NewReader(body))
|
||||
if body != "" {
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
if secret != "" {
|
||||
r.Header.Set("Authorization", "Bearer "+secret)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
var obj map[string]any
|
||||
json.Unmarshal(w.Body.Bytes(), &obj)
|
||||
return w.Code, obj, w.Body.Bytes()
|
||||
}
|
||||
|
||||
// A provider set up with a base URL whose model list marks a model "kind":
|
||||
// "video" makes videos with it: started, polled and fetched at its
|
||||
// OpenAI-shaped videos API (#1446). The old gateway turned it away 400 as
|
||||
// not a Grok or Volcengine model.
|
||||
func TestCustomProviderMakesVideos(t *testing.T) {
|
||||
fresh(t)
|
||||
up := newOpenAIVideos(t)
|
||||
p := saveFetched(t, provider.Provider{ID: "studio", Name: "Studio", Chat: up.URL + "/v1", Key: "sk-studio"})
|
||||
ms := Videomakers(p)
|
||||
if len(ms) != 1 || ms[0].ID != "vid-1" || ms[0].Provider != "studio" {
|
||||
t.Fatalf("videomakers %v: the list marks vid-1 alone a video model", ms)
|
||||
}
|
||||
h := New().Handler()
|
||||
code, obj, raw := asKey(t, h, "", "POST", "/v1/videos", `{"model":"studio/vid-1","prompt":"a cat on a fence","seconds":4,"size":"1280x720"}`)
|
||||
id, _ := obj["id"].(string)
|
||||
if code != 200 || !strings.HasPrefix(id, "video_studio.") || obj["model"] != "studio/vid-1" || obj["status"] != "queued" {
|
||||
t.Fatalf("start: %d %s", code, raw)
|
||||
}
|
||||
var sent map[string]any
|
||||
json.Unmarshal([]byte(up.bodies[0]), &sent)
|
||||
if sent["model"] != "vid-1" || sent["prompt"] != "a cat on a fence" || sent["seconds"] != "4" || sent["size"] != "1280x720" {
|
||||
t.Fatalf("sent %v", sent)
|
||||
}
|
||||
for i, want := range []string{"in_progress", "completed"} {
|
||||
code, obj, raw = asKey(t, h, "", "GET", "/v1/videos/"+id, "")
|
||||
if code != 200 || obj["id"] != id || obj["status"] != want || obj["model"] != "studio/vid-1" {
|
||||
t.Fatalf("poll %d: %d %s", i, code, raw)
|
||||
}
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest("GET", "/v1/videos/"+id+"/content", nil))
|
||||
if rec.Code != 200 || rec.Body.String() != "\x00\x00\x00\x18ftypmp42fake-mp4" || rec.Header().Get("Content-Type") != "video/mp4" {
|
||||
t.Fatalf("content: %d %q %v", rec.Code, rec.Body.String(), rec.Header())
|
||||
}
|
||||
want := []string{"GET /v1/models sk-studio", "POST /v1/videos sk-studio", "GET /v1/videos/video_68d7.abc sk-studio", "GET /v1/videos/video_68d7.abc sk-studio", "GET /v1/videos/video_68d7.abc/content sk-studio"}
|
||||
if got := up.log(); !slices.Equal(got, want) {
|
||||
t.Fatalf("asked %v\nwant %v", got, want)
|
||||
}
|
||||
// a vendor is told nothing of who is calling
|
||||
for _, hd := range up.heads {
|
||||
if hd.Get(AgentHeader) != "" || hd.Get(SessionHeader) != "" {
|
||||
t.Fatalf("caller headers went to the vendor: %v", hd)
|
||||
}
|
||||
}
|
||||
// by its bare name too, and with none named when it is the only maker
|
||||
if code, obj, raw = asKey(t, h, "", "POST", "/v1/videos", `{"model":"vid-1","prompt":"a dog"}`); code != 200 || obj["model"] != "studio/vid-1" {
|
||||
t.Fatalf("bare name: %d %s", code, raw)
|
||||
}
|
||||
if code, obj, raw = asKey(t, h, "", "POST", "/v1/videos", `{"prompt":"a dog"}`); code != 200 || obj["model"] != "studio/vid-1" {
|
||||
t.Fatalf("no model named: %d %s", code, raw)
|
||||
}
|
||||
// another magpie asking this one for its video models is told of it
|
||||
if objs := videomakerObjects(); len(objs) != 1 || objs[0]["id"] != "studio/vid-1" {
|
||||
t.Fatalf("videomakerObjects %v", objs)
|
||||
}
|
||||
}
|
||||
|
||||
// A model's name alone makes no video model: one the provider's list
|
||||
// doesn't mark "kind": "video" is still turned away before any request,
|
||||
// and so is every model of a provider whose list marks none.
|
||||
func TestCustomProviderVideoNeedsTheListsWord(t *testing.T) {
|
||||
fresh(t)
|
||||
up := newOpenAIVideos(t)
|
||||
saveFetched(t, provider.Provider{ID: "studio", Name: "Studio", Chat: up.URL + "/v1", Key: "sk-studio"})
|
||||
h := New().Handler()
|
||||
for _, model := range []string{"studio/sora-like-2", "studio/gpt-5.5", "studio/grok-imagine-video"} {
|
||||
code, obj, raw := asKey(t, h, "", "POST", "/v1/videos", `{"model":"`+model+`","prompt":"a cat"}`)
|
||||
if code != 400 || !strings.Contains(errorOf(obj), "can't make videos") {
|
||||
t.Fatalf("%s: %d %s", model, code, raw)
|
||||
}
|
||||
}
|
||||
for _, a := range up.log() {
|
||||
if strings.Contains(a, "/videos") {
|
||||
t.Fatalf("a video was asked for: %v", up.log())
|
||||
}
|
||||
}
|
||||
if ms := Videomakers(provider.Provider{ID: "unlisted", Name: "Unlisted", Chat: up.URL + "/v1", Key: "k"}); len(ms) != 0 {
|
||||
t.Fatalf("a provider with no fetched list makes videos: %v", ms)
|
||||
}
|
||||
}
|
||||
|
||||
// A Responses-only relay has no Chat base: its video is started, polled
|
||||
// and fetched under its Responses base all the same, as its images are.
|
||||
func TestCustomProviderVideoOnAResponsesBase(t *testing.T) {
|
||||
fresh(t)
|
||||
up := newOpenAIVideos(t)
|
||||
saveFetched(t, provider.Provider{ID: "relay", Name: "Relay", Responses: up.URL + "/v1", Key: "sk-relay"})
|
||||
h := New().Handler()
|
||||
code, obj, raw := asKey(t, h, "", "POST", "/v1/videos", `{"model":"relay/vid-1","prompt":"a kite"}`)
|
||||
id, _ := obj["id"].(string)
|
||||
if code != 200 || id == "" {
|
||||
t.Fatalf("start: %d %s", code, raw)
|
||||
}
|
||||
asKey(t, h, "", "GET", "/v1/videos/"+id, "")
|
||||
if code, obj, raw = asKey(t, h, "", "GET", "/v1/videos/"+id, ""); code != 200 || obj["status"] != "completed" {
|
||||
t.Fatalf("poll: %d %s", code, raw)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest("GET", "/v1/videos/"+id+"/content", nil))
|
||||
if rec.Code != 200 || !strings.HasSuffix(rec.Body.String(), "fake-mp4") {
|
||||
t.Fatalf("content: %d %s", rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
// A video started on a provider's later key is asked after on that key,
|
||||
// whichever is first: the vendor keeps it for the key that started it.
|
||||
// A gateway key held to some accounts (#905) asks after it only on one it
|
||||
// may use, and one held to some models (#882) only after a model it may
|
||||
// use, on the poll and the content as on the start.
|
||||
func TestCustomProviderVideoKeys(t *testing.T) {
|
||||
fresh(t)
|
||||
up := newOpenAIVideos(t)
|
||||
saveFetched(t, provider.Provider{ID: "studio", Name: "Studio", Chat: up.URL + "/v1", Key: "kA", Keys: []provider.KeyAccount{{Key: "kB"}}})
|
||||
keys, secrets := newCaller(t, "OnB", "OnA", "OtherModels", "ThisModel")
|
||||
for i, c := range []access.Change{
|
||||
{Key: keys[0].ID, Accounts: []string{"studio/" + provider.KeyID("kB")}},
|
||||
{Key: keys[1].ID, Accounts: []string{"studio/" + provider.KeyID("kA")}},
|
||||
{Key: keys[2].ID, Models: []string{"elsewhere/*"}},
|
||||
{Key: keys[3].ID, Models: []string{"studio/vid-1"}},
|
||||
} {
|
||||
if _, err := access.Update([]string{"accounts-key", "accounts-key", "models-key", "models-key"}[i], c); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
h := New().Handler()
|
||||
code, obj, raw := asKey(t, h, secrets[0], "POST", "/v1/videos", `{"model":"studio/vid-1","prompt":"a cat"}`)
|
||||
id, _ := obj["id"].(string)
|
||||
if code != 200 || id == "" {
|
||||
t.Fatalf("start: %d %s", code, raw)
|
||||
}
|
||||
if code, _, raw = asKey(t, h, secrets[0], "GET", "/v1/videos/"+id, ""); code != 200 {
|
||||
t.Fatalf("poll on kB: %d %s", code, raw)
|
||||
}
|
||||
// a caller with no restriction asks on kB too, not on the first key
|
||||
if code, _, raw = asKey(t, h, "", "GET", "/v1/videos/"+id, ""); code != 200 {
|
||||
t.Fatalf("poll: %d %s", code, raw)
|
||||
}
|
||||
if code, _, raw = asKey(t, h, secrets[0], "GET", "/v1/videos/"+id+"/content", ""); code != 200 || !strings.HasSuffix(string(raw), "fake-mp4") {
|
||||
t.Fatalf("content on kB: %d %s", code, raw)
|
||||
}
|
||||
for _, a := range up.log()[1:] {
|
||||
if !strings.HasSuffix(a, " kB") {
|
||||
t.Fatalf("asked on another key than kB: %v", up.log())
|
||||
}
|
||||
}
|
||||
before := len(up.log())
|
||||
// held to kA: kB's video is not its to ask after
|
||||
for _, path := range []string{"/v1/videos/" + id, "/v1/videos/" + id + "/content"} {
|
||||
if code, obj, raw = asKey(t, h, secrets[1], "GET", path, ""); code != 403 || !strings.Contains(errorOf(obj), "not allowed to use the accounts") {
|
||||
t.Fatalf("held to kA, %s: %d %s", path, code, raw)
|
||||
}
|
||||
}
|
||||
if len(up.log()) != before {
|
||||
t.Fatalf("a key held from the account had the vendor asked: %v", up.log()[before:])
|
||||
}
|
||||
// held to other models: neither the poll nor the bytes
|
||||
for _, path := range []string{"/v1/videos/" + id, "/v1/videos/" + id + "/content"} {
|
||||
if code, obj, raw = asKey(t, h, secrets[2], "GET", path, ""); code != 403 || !strings.Contains(errorOf(obj), "may not use studio/vid-1") {
|
||||
t.Fatalf("held to other models, %s: %d %s", path, code, raw)
|
||||
}
|
||||
}
|
||||
for _, a := range up.log()[before:] {
|
||||
if strings.HasSuffix(strings.Fields(a)[1], "/content") {
|
||||
t.Fatalf("a key held from the model was sent the video: %v", up.log()[before:])
|
||||
}
|
||||
}
|
||||
// held to this model: both
|
||||
if code, _, raw = asKey(t, h, secrets[3], "GET", "/v1/videos/"+id, ""); code != 200 {
|
||||
t.Fatalf("held to vid-1, poll: %d %s", code, raw)
|
||||
}
|
||||
if code, _, raw = asKey(t, h, secrets[3], "GET", "/v1/videos/"+id+"/content", ""); code != 200 || !strings.HasSuffix(string(raw), "fake-mp4") {
|
||||
t.Fatalf("held to vid-1, content: %d %s", code, raw)
|
||||
}
|
||||
// and a key taken off, its videos say so rather than ask on another
|
||||
if err := provider.SetKeyOn("studio", provider.KeyID("kB"), false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code, obj, raw = asKey(t, h, "", "GET", "/v1/videos/"+id, ""); code != 404 || !strings.Contains(errorOf(obj), "no longer on") {
|
||||
t.Fatalf("kB off: %d %s", code, raw)
|
||||
}
|
||||
}
|
||||
|
||||
// A Grok subscription's video is held by a gateway key's models on its
|
||||
// poll and content as on its start; one the key may use is answered as
|
||||
// before.
|
||||
func TestGrokVideoPollHeldByKeyModels(t *testing.T) {
|
||||
grokSignedIn(t)
|
||||
newGrokMedia(t)
|
||||
keys, secrets := newCaller(t, "OtherModels", "Grok")
|
||||
if _, err := access.Update("models-key", access.Change{Key: keys[0].ID, Models: []string{"elsewhere/*"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := access.Update("models-key", access.Change{Key: keys[1].ID, Models: []string{"grok/*"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h := New().Handler()
|
||||
code, obj, raw := asKey(t, h, secrets[1], "POST", "/v1/videos", `{"model":"grok/grok-imagine-video","prompt":"a kite"}`)
|
||||
id, _ := obj["id"].(string)
|
||||
if code != 200 || id == "" {
|
||||
t.Fatalf("start: %d %s", code, raw)
|
||||
}
|
||||
for _, path := range []string{"/v1/videos/" + id, "/v1/videos/" + id + "/content"} {
|
||||
if code, obj, raw = asKey(t, h, secrets[0], "GET", path, ""); code != 403 || !strings.Contains(errorOf(obj), "may not use grok/") {
|
||||
t.Fatalf("held, %s: %d %s", path, code, raw)
|
||||
}
|
||||
}
|
||||
// the two held asks read the vendor's state, as a poll does: the
|
||||
// third ask finds it made
|
||||
if code, obj, raw = asKey(t, h, secrets[1], "GET", "/v1/videos/"+id, ""); code != 200 || obj["status"] != "completed" || obj["model"] != "grok/grok-imagine-video" {
|
||||
t.Fatalf("poll: %d %s", code, raw)
|
||||
}
|
||||
if code, _, raw = asKey(t, h, secrets[1], "GET", "/v1/videos/"+id+"/content", ""); code != 200 || string(raw) != "fake-mp4-bytes" {
|
||||
t.Fatalf("content: %d %s", code, raw)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user