diff --git a/docs/协议文档/阿里系列/视频-wan3.0.md b/docs/协议文档/阿里系列/视频-wan3.0.md new file mode 100644 index 0000000..f510c2b --- /dev/null +++ b/docs/协议文档/阿里系列/视频-wan3.0.md @@ -0,0 +1,1119 @@ +> ## Documentation Index +> Fetch the complete documentation index at: https://docs.bailian.console.aliyun.com/llms.txt +> Use this file to discover all available pages before exploring further. + +# 万相3.0-视频生成API参考 + +> 万相3.0是全能参考视频生成模型(All-in-One),统一支持 文生视频 、 图生视频 (首帧/首尾帧)和 参考生视频 等多种用法。最长可生成30秒视频,输出帧率为30fps。 + +使用指南请参见[万相3.0-视频生成](/zh/model-studio/wan3-video-generation-guide)。 + +## 适用范围 + +为确保调用成功,请务必保证模型、Endpoint URL 和 API Key 均属于同一地域。跨地域调用将会失败。 + +- [选择模型](https://bailian.console.aliyun.com/cn-beijing/model/market):前往模型广场选择模型,并确认其所属地域。 +- 选择 URL:选择对应的地域 Endpoint URL。 +- 配置 API Key:选择地域并[获取与配置 API Key](/zh/model-studio/get-api-key),再[配置API Key到环境变量](/zh/model-studio/configure-api-key-through-environment-variables)。 + + + 本文的示例代码适用于北京地域。 + + +## HTTP调用 + +由于视频生成任务耗时较长(通常为1-5分钟),API采用异步调用。整个流程包含 "创建任务 -> 轮询获取" 两个核心步骤,具体如下: + +### 步骤1:创建任务获取任务ID + + + + `POST https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis` + + + + `POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis` + + + + `POST https://{WorkspaceId}.ap-northeast-1.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis` + + + + `POST https://{WorkspaceId}.eu-central-1.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis` + + + + `POST https://{WorkspaceId}.us-east-1.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis` + + + + `POST https://{WorkspaceId}.cn-hongkong.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis` + + + +调用时请将`{WorkspaceId}`替换为真实的[业务空间ID](/zh/model-studio/regions#h2_migrate_domain)。 + + + - 创建成功后,使用接口返回的 `task_id` 查询结果,task\_id 有效期为 24 小时。请勿重复创建任务,轮询获取即可。 + - 新手指引请参见[Postman](/zh/model-studio/first-call-to-image-and-video-api)。 + + + + + + + + + + + + + + + + +
+ #### 请求参数 + + ##### 请求头(Headers) + + Content-Type`string`(必选) + + 请求内容类型。此参数必须设置为`application/json`。 + + Authorization`string`(必选) + + 请求身份认证。接口使用阿里云百炼API Key进行身份认证。示例值:Bearer sk-xxxx。 + + X-DashScope-Async`string`(必选) + + 异步处理配置参数。HTTP请求只支持异步,必须设置为`enable`。 + + + 缺少此请求头将报错:“current user api does not support synchronous calls”。 + + + ##### 请求体(Request Body) + + model `string` (必选) + + 模型名称。可选值: + + - `wan3.0-video-prime`:高速版,能力对齐标准版,端到端速度显著提升。 + - `wan3.0-video`:标准版。 + + input `object` (必选) + + 输入的基本信息。`prompt` 和 `media` 必填其一。 + + + prompt `string` (条件必选) + + 文本提示词,用来描述期望生成的视频内容。和 `media` 必填其一。 + + 支持中英文,每个汉字/字母占一个字符,不超过20000个字符,超过部分会自动截断。 + + 在全能参考模式下,prompt中可以用"图1""视频1""音频1"等指代 media 数组中对应顺序的媒体素材。 + + media `array` (条件必选) + + 媒体素材数组,支持图像、视频、音频、文件和网页作为输入。和 `prompt` 必填其一。 + + - 数组中每个元素为一个媒体对象,包含 `type` 与 `url` 字段。 + - 在参考生视频模式下,按照数组顺序定义 `prompt` 中素材引用的顺序。图和视频分别计数,即可同时存在图1、视频1。 + + - 数组中的第 1 个 `reference_video` 对应 视频1,第 2 个对应 视频2,以此类推。 + - 数组中的第 1 个 `reference_image` 对应 图1,第 2 个对应 图2,以此类推。 + - 数组中的第 1 个 `reference_audio` 对应 音频1,第 2 个对应 音频2,以此类推。 + + + type `string` (必选) + + 媒体素材类型。可选值为: + + - `first_frame`:首帧图像。最多1张,严格作为视频第一帧。 + - `last_frame`:尾帧图像。最多1张,严格作为视频最后一帧。 + - `reference_image`:参考图像。最多10张。 + - `reference_video`:参考视频。最多5段,总时长不大于15秒。 + - `reference_audio`:参考音频。最多5段,总时长不大于15秒。 + - `file`:文件。最多1个,不可与 link 同时输入。 + - `link`:网页链接。最多1个,不可与 file 同时输入。 + + + 不同 `type` 的组合规则和限制请参见[素材组合](#w30media_combo_t)。 + + + url `string` (必选) + + 媒体素材URL或Base64 编码数据。 + + + 图像URL或Base64 编码数据。 + + 图像限制: + + - 格式:JPEG、JPG、PNG(不支持透明通道)、BMP、WEBP。 + - 分辨率:单边\[240, 8000]像素。 + - 长宽比:不超过8:1。 + - 文件大小:不超过20MB。 + + 支持输入的格式: + + 1. 公网URL: + + - 支持HTTP或HTTPS协议。 + - 示例值:[https://xxx/xxx.png](https://xxx/xxx.png)。 + 2. 临时URL: + + - 支持OSS协议,必须通过[上传文件获取临时 URL](/zh/model-studio/get-temporary-file-url)。 + - 示例值:oss\://dashscope-instant/xxx/xxx.png。 + 3. Base64 编码图像后的字符串: + + - 数据格式:`data:{MIME_type};base64,{base64_data}`。 + - 示例值:data:image/png;base64,GDU7MtCZzEbTbmRZ......。(编码字符串过长,仅展示片段) + - 详情请参见[传入图像](/zh/model-studio/image-to-video-guide#32d9db99f1fk0)。 + + + + 参考视频URL。 + + 视频限制: + + - 格式:mp4、mov。 + - 时长:单个\[1, 15]秒,总时长不大于15秒。 + - 帧率:≥16 fps。 + - 分辨率:单边\[240, 4096]像素。 + - 长宽比:不超过8:1。 + - 单文件大小:不超过100MB。 + + 支持输入的格式: + + 1. 公网URL: + + - 支持HTTP或HTTPS协议。 + - 示例值:[https://xxx/xxx.mp4](https://xxx/xxx.mp4)。 + 2. 临时URL: + + - 支持OSS协议,必须通过[上传文件获取临时 URL](/zh/model-studio/get-temporary-file-url)。 + - 示例值:oss\://dashscope-instant/xxx/xxx.mp4。 + + + + 参考音频URL。 + + 音频限制: + + - 格式:wav、mp3。 + - 时长:单个\[1, 15]秒,总时长不大于15秒。 + - 文件大小:不超过15MB。 + + 支持输入的格式: + + 1. 公网URL: + + - 支持HTTP或HTTPS协议。 + - 示例值:[https://xxx/xxx.mp3](https://xxx/xxx.mp3)。 + 2. 临时URL: + + - 支持OSS协议,必须通过[上传文件获取临时 URL](/zh/model-studio/get-temporary-file-url)。 + - 示例值:oss\://dashscope-instant/xxx/xxx.mp3。 + + + + 文件URL。 + + 文件限制: + + - 格式:docx、doc、xlsx、xls、pptx、ppt、pdf、txt、key、pages、numbers、md。 + - 文件大小:不超过100MB。 + - 页数限制:不超过50页(对pdf、docx、doc、pptx、ppt、key、pages格式校验)。 + + 支持输入的格式: + + 1. 公网URL: + + - 支持HTTP或HTTPS协议。 + - 示例值:[https://xxx/xxx.pdf](https://xxx/xxx.pdf)。 + 2. 临时URL: + + - 支持OSS协议,必须通过[上传文件获取临时 URL](/zh/model-studio/get-temporary-file-url)。 + - 示例值:oss\://dashscope-instant/xxx/xxx.pdf。 + + + + 公开网页的URL地址。仅支持解析无需登录的公开网页(如新闻、博客、公众号等)。 + + 支持输入的格式: + + 1. 公网URL: + + - 支持HTTP或HTTPS协议。 + - 示例值:[https://xxx/article/xxx](https://xxx/article/xxx)。 + + + + + parameters `object` (可选) + + 视频处理参数。 + + + resolution `string` (可选) + + 生成视频的分辨率档位。默认值为 `1080P`。可选值: + + - `1080P` + - `720P` + - `480P` + + ratio `string` (可选) + + 生成视频的宽高比。可选值: + + - `adaptive`(默认值):自适应长宽比,根据输入媒体比例和意图自动推荐合适的长宽比。 + - `21:9` + - `16:9` + - `4:3` + - `1:1` + - `3:4` + - `9:16` + + duration `integer` (可选) + + 生成视频的时长,单位为秒。默认值为5。 + + - 无视频输入时:取值范围为\[2, 30]的整数。 + - 有视频输入时:输入视频总时长 + 输出视频时长不超过30秒。 + - 传 `-1` 时:智能时长模式,模型根据输入的 prompt、内容和富媒体自动推荐合适时长生成。 + + audio `boolean` (可选) + + 输出视频是否包含音频。 + + - `true`:默认值,输出视频包含声音。 + - `false`:输出视频不包含音轨。 + + 开关声音价格相同。 + + seed `integer` (可选) + + 随机种子,用于复现生成结果。取值范围:-1或\[0, 2147483647]。传入-1或未指定时,系统自动生成随机种子。即使使用相同seed,也不能保证每次生成结果完全一致。 + + prompt\_extend `boolean` (可选) + + 是否开启prompt智能改写。开启后使用大模型对输入prompt进行智能改写。对于较短的prompt生成效果提升明显,但会增加耗时。 + + - `true`:默认值,开启智能改写。 + - `false`:不开启智能改写。 + + + 当传入文档(`file`)或网页(`link`)素材时,`prompt_extend` 取值必须为 `true`。 + + + watermark `boolean` (可选) + + 是否添加水印标识。 + + - `false`:默认值,不添加水印。 + - `true`:添加水印。 + + + + + 通过 `file` 类型传入文件,模型自动理解文件内容生成视频。 + + ```bash + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "一支高端智能眼镜产品广告,整体风格极简、未来感、时尚高级,光影克制,画面以黑色、银灰色、冰蓝色为主色调,局部点缀柔和白光与参数UI图形。开场在纯黑背景中,一副智能眼镜从黑暗中缓缓浮现,镜腿边缘掠过精致高光,镜框轮廓在冷冽边缘光下被勾勒出来,镜头超近距离掠过镜片、鼻托、转轴、镜腿与材质细节,展现金属与高性能复合材料的细腻质感,表面处理高级克制,线条轻薄流畅。随后产品在空中缓慢旋转,画面以极简动态图形同步展示核心参数信息。随后镜头快速收拢,所有零件精准回归组装成完整产品,切换到年轻模特佩戴展示,模特五官立体、气质自信,穿着简洁高级的都市时尚服装,在极简空间和城市光影环境中自然转头、抬手、行走、微笑,镜头从正面、侧面、斜后方展示眼镜佩戴状态,突出轻薄贴合、时尚轮廓与日常百搭属性。结尾在纯色背景中,产品悬浮定格,镜头缓慢推进到品牌logo和核心slogan,整体音乐极简电子氛围配合精准鼓点,节奏干净有力,画面质感高级、克制、纯粹,具有强烈品牌记忆点和国际化科技审美。", + "media": [ + { + "type": "file", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260806/ebapmr/glass.pptx" + } + ] + }, + "parameters": { + "resolution": "480P", + "ratio": "adaptive", + "duration": 10, + "prompt_extend": true + } + }' + ``` + + + + 通过 `input.media` 传入参考图片、视频、音频、文件或网页链接,模型自动理解意图生成视频。 + + ```bash expandable + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "视频1抱着图3,在图4的椅子上弹奏一支舒缓的乡村民谣,并说道:"今天的阳光真好。"图1手中拿着图2,路过视频1,把手中的图2放到视频1旁边的桌子上,并说道:"真好听,能不能再唱一遍"。", + "media": [ + { + "type": "reference_image", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260408/sjuytr/wan-r2v-object-girl.jpg" + }, + { + "type": "reference_video", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260129/qigswt/wan-r2v-role2.mp4" + }, + { + "type": "reference_image", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260129/rtjeqf/wan-r2v-object3.png" + }, + { + "type": "reference_image", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260129/qpzxps/wan-r2v-object4.png" + }, + { + "type": "reference_image", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260129/wfjikw/wan-r2v-backgroud5.png" + } + ] + }, + "parameters": { + "resolution": "480P", + "ratio": "adaptive", + "duration": 5, + "prompt_extend": true + } + }' + ``` + + + + 仅通过 `prompt` 生成视频,不传入任何媒体文件。 + + ```bash + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "一只小猫在月光下的屋顶上奔跑,城市的霓虹灯在远处闪烁,电影级画质,流畅运镜。" + }, + "parameters": { + "resolution": "480P", + "ratio": "adaptive", + "duration": 5, + "prompt_extend": true + } + }' + ``` + + + + 仅传入 `first_frame`,严格指定视频首帧图像。 + + ```bash + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "一只猫在草地上奔跑", + "media": [ + { + "type": "first_frame", + "url": "https://cdn.translate.alibaba.com/r/wanx-demo-1.png" + } + ] + }, + "parameters": { + "resolution": "480P", + "ratio": "adaptive", + "duration": 5, + "prompt_extend": true + } + }' + ``` + + + + 仅支持传入 `first_frame` 和 `last_frame`,严格指定视频的首帧和尾帧图像。 + + ```bash expandable + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "清晨太阳刚刚升起,在南瓜地里面,有一颗小南瓜上面挂着露珠,突然小南瓜咔擦一声,出现了裂缝,从裂缝中透出金光,小南瓜伴随着金光裂开,出现一团白雾,一只小兔子在南瓜裂开的南瓜中央出现。", + "media": [ + { + "type": "first_frame", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260414/welyei/wan2.7-i2v-first-frame.webp" + }, + { + "type": "last_frame", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260414/zongha/wan2.7-i2v-last-frame.webp" + } + ] + }, + "parameters": { + "resolution": "480P", + "ratio": "adaptive", + "duration": 5, + "prompt_extend": true + } + }' + ``` + + + + 通过 `reference_video` 类型传入待编辑视频,结合 `prompt` 指令编辑视频内容。 + + ```bash + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "将整个画面转换为黏土风格", + "media": [ + { + "type": "reference_video", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260402/ldnfdf/wan2.7-videoedit-style-change.mp4" + } + ] + }, + "parameters": { + "resolution": "720P", + "prompt_extend": true + } + }' + ``` + + + + 通过 `reference_video` 类型传入原始视频,结合含延长意图关键词的 `prompt` 延长视频内容。需将 `ratio` 设为 `adaptive`。 + + ```bash + curl --location 'https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis' \ + -H 'X-DashScope-Async: enable' \ + -H "Authorization: Bearer $DASHSCOPE_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "wan3.0-video", + "input": { + "prompt": "将视频1向后延长,面包师端上刷好的面包,将刷子放到一旁,镜头跟随面包师,去斜后方的烤炉进行烤制", + "media": [ + { + "type": "reference_video", + "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260402/ldnfdf/wan2.7-videoedit-style-change.mp4" + } + ] + }, + "parameters": { + "resolution": "720P", + "ratio": "adaptive", + "prompt_extend": true + } + }' + ``` + + +
+ + + + + + + + + + + + + + + +
+ #### 响应参数 + + output `object` + + 任务输出信息。 + + + task\_id `string` + + 任务ID。查询有效期24小时。 + + task\_status `string` + + 任务状态。 + + + - PENDING:任务排队中 + - RUNNING:任务处理中 + - SUCCEEDED:任务执行成功 + - FAILED:任务执行失败 + - CANCELED:任务已取消 + - UNKNOWN:任务不存在或状态未知 + + + + request\_id`string` + + 请求唯一标识。可用于请求明细溯源和问题排查。 + + code`string` + + 请求失败的错误码。请求成功时不会返回此参数,详情请参见[错误码](/zh/model-studio/error-code)。 + + message`string` + + 请求失败的详细信息。请求成功时不会返回此参数,详情请参见[错误码](/zh/model-studio/error-code)。 + + + + 请保存 task\_id,用于查询任务状态与结果。 + + ```json + { + "output": { + "task_status": "PENDING", + "task_id": "0385dc79-5ff8-4d82-bcb6-xxxxxx" + }, + "request_id": "4909100c-7b5a-9f92-bfe5-xxxxxx" + } + ``` + + + + 创建任务失败,请参见[错误码](/zh/model-studio/error-code)进行解决。 + + ```json + { + "code": "InvalidApiKey", + "message": "No API-key provided.", + "request_id": "7438d53d-6eb8-4596-8835-xxxxxx" + } + ``` + + +
+ +### 步骤2:根据任务ID查询结果 + + + + `GET https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/tasks/{task_id}` + + + + `GET https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1/tasks/{task_id}` + + + + `GET https://{WorkspaceId}.ap-northeast-1.maas.aliyuncs.com/api/v1/tasks/{task_id}` + + + + `GET https://{WorkspaceId}.eu-central-1.maas.aliyuncs.com/api/v1/tasks/{task_id}` + + + + `GET https://{WorkspaceId}.us-east-1.maas.aliyuncs.com/api/v1/tasks/{task_id}` + + + + `GET https://{WorkspaceId}.cn-hongkong.maas.aliyuncs.com/api/v1/tasks/{task_id}` + + + + + - 轮询建议:视频生成过程约需数分钟,建议采用轮询机制,并设置合理的查询间隔(如 15 秒)来获取结果。 + - 任务状态流转:PENDING(排队中)→ RUNNING(处理中)→ SUCCEEDED(成功)/ FAILED(失败)。 + - 结果链接:任务成功后返回视频链接,有效期为 24 小时。建议在获取链接后立即下载并转存至永久存储(如[阿里云 OSS](https://help.aliyun.com/zh/oss/user-guide/what-is-oss))。 + - task\_id 有效期:24小时,超时后将无法查询结果,接口将返回任务状态为`UNKNOWN`。 + - RPS 限制:查询接口默认RPS为20。如需更高频查询或事件通知,建议[配置异步任务回调](/zh/model-studio/async-task-api)。 + - 更多操作:如需批量查询、取消任务等操作,请参见[管理异步任务](/zh/model-studio/manage-asynchronous-tasks)。 + + + + + + + + + + + + + + + + +
+ #### 请求参数 + + ##### 请求头(Headers) + + Authorization`string`(必选) + + 请求身份认证。接口使用阿里云百炼API Key进行身份认证。示例值:Bearer sk-xxxx。 + + ##### URL路径参数(Path parameters) + + task\_id `string`(必选) + + 任务ID。 + + + + 将`{task_id}`完整替换为上一步接口返回的`task_id`的值。`task_id`查询有效期为24小时,并请将`{WorkspaceId}`替换为真实的[业务空间ID](/zh/model-studio/regions#h2_migrate_domain)。 + + ```bash + curl -X GET https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1/tasks/{task_id} \ + --header "Authorization: Bearer $DASHSCOPE_API_KEY" + ``` + + +
+ + + + + + + + + + + + + + + +
+ #### 响应参数 + + output `object` + + 任务输出信息。 + + + task\_id `string`(必选) + + 任务ID。 + + task\_status `string` + + 任务状态。 + + + - PENDING:任务排队中 + - RUNNING:任务处理中 + - SUCCEEDED:任务执行成功 + - FAILED:任务执行失败 + - CANCELED:任务已取消 + - UNKNOWN:任务不存在或状态未知 + + + submit\_time `string` + + 任务提交时间。格式为 YYYY-MM-DD HH:mm:ss.SSS。 + + scheduled\_time `string` + + 任务执行时间。格式为 YYYY-MM-DD HH:mm:ss.SSS。 + + end\_time `string` + + 任务完成时间。格式为 YYYY-MM-DD HH:mm:ss.SSS。 + + orig\_prompt `string` + + 原始输入的提示词。 + + video\_url `string` + + 生成视频的URL地址。任务成功时返回。 + + code`string` + + 请求失败的错误码。请求成功时不会返回此参数,详情请参见[错误码](/zh/model-studio/error-code)。 + + message`string` + + 请求失败的详细信息。请求成功时不会返回此参数,详情请参见[错误码](/zh/model-studio/error-code)。 + + + usage `object` + + 输出信息统计。只对成功的结果计数。 + + + video\_count `integer` + + 生成视频的数量。固定为1。 + + duration `float` + + 生成视频的时长,单位为秒。 + + input\_video\_duration `float` + + 输入视频的时长,单位为秒。无视频输入时为0.0。 + + output\_video\_duration `float` + + 输出视频的时长,单位为秒。 + + fps `integer` + + 生成视频的帧率。默认值为30。 + + SR `integer` + + 生成视频的分辨率。示例值:720。 + + ratio `string` + + 生成视频的宽高比。示例值:16:9。 + + + request\_id`string` + + 请求唯一标识。可用于请求明细溯源和问题排查。 + + + + 视频URL仅保留24小时,超时后会被自动清除,请及时保存生成的视频。 + + ```json + { + "request_id": "78c9b768-0285-996c-b682-xxxxxx", + "output": { + "task_id": "17ed7e50-00cf-4509-aea1-xxxxxx", + "task_status": "SUCCEEDED", + "submit_time": "2026-08-06 10:01:35.452", + "scheduled_time": "2026-08-06 10:01:35.507", + "end_time": "2026-08-06 10:13:33.838", + "orig_prompt": "A golden retriever running on a sunny beach, waves crashing in the background, cinematic lighting", + "video_url": "https://dashscope-result-bj.oss-cn-beijing.aliyuncs.com/xxx/video.mp4" + }, + "usage": { + "video_count": 1, + "duration": 5.0, + "input_video_duration": 0.0, + "output_video_duration": 5.0, + "fps": 30, + "SR": 720, + "ratio": "16:9" + } + } + ``` + + + + 若任务执行失败,task\_status将置为 FAILED,并提供错误码和信息。请参见[错误码](/zh/model-studio/error-code)进行解决。 + + ```json + { + "request_id": "e5e57877-c0fc-47ed-8fad-xxxxxx", + "output": { + "task_id": "eff1443c-ccab-4676-aad3-xxxxxx", + "task_status": "FAILED", + "code": "InvalidParameter", + "message": "The two modes are mutually exclusive. Do not pass reference_xx and first_frame/last_frame at the same time." + } + } + ``` + + + + task\_id查询有效期为 24 小时,超时后将无法查询,返回以下报错信息。 + + ```json + { + "request_id": "a4de7c32-7057-9f82-8581-xxxxxx", + "output": { + "task_id": "502a00b1-19d9-4839-a82f-xxxxxx", + "task_status": "UNKNOWN" + } + } + ``` + + +
+ +## 素材组合 + + + 仅支持以下特定的素材组合,传入其他组合将报错。 + + +**文生视频**:仅传入 `prompt`,不支持传入 `media`。 + +```json +// 文生视频 +{ + "input": { + "prompt": "..." + } +} +``` + +**图生视频、参考生视频、视频编辑、视频延长等模式**需通过 `media` 数组传入素材,每个元素需指定 `type` 和 `url`。`type` 组合规则: + +- `first_frame`/`last_frame` 与 `reference_image`/`reference_video`/`reference_audio`/`file`/`link` 类型互斥,不能在同一请求中混用。 +- `file` 和 `link` 二选一,但均可与 `reference_image`/`reference_video`/`reference_audio` 组合使用。 +- `reference_image`/`reference_video`/`reference_audio` 之间可自由组合。 + + + ```json 首帧生视频 + // 仅首帧 + { + "media": [ + { "type": "first_frame", "url": "..." } + ] + } + ``` + + ```json 首尾帧生视频 + // 仅首帧+尾帧 + { + "media": [ + { "type": "first_frame", "url": "..." }, + { "type": "last_frame", "url": "..." } + ] + } + ``` + + ```json 参考生视频 + // reference_image、reference_video、reference_audio 可自由组合 + + // 仅参考图片 + { + "media": [ + { "type": "reference_image", "url": "..." } + ] + } + + // 仅参考视频 + { + "media": [ + { "type": "reference_video", "url": "..." } + ] + } + + // 仅参考音频 + { + "media": [ + { "type": "reference_audio", "url": "..." } + ] + } + + // 图片+视频 + { + "media": [ + { "type": "reference_image", "url": "..." }, + { "type": "reference_video", "url": "..." } + ] + } + + // 图片+音频 + { + "media": [ + { "type": "reference_image", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 视频+音频 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 图片+视频+音频 + { + "media": [ + { "type": "reference_image", "url": "..." }, + { "type": "reference_video", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + ``` + + ```json 文件/网页生视频 + // 必须包含 file 或 link(二选一),可与 reference_image/reference_video/reference_audio 自由组合 + + // 仅文件 + { + "media": [ + { "type": "file", "url": "..." } + ] + } + + // 文件+图片 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_image", "url": "..." } + ] + } + + // 文件+视频 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_video", "url": "..." } + ] + } + + // 文件+音频 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 文件+图片+视频 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_image", "url": "..." }, + { "type": "reference_video", "url": "..." } + ] + } + + // 文件+图片+音频 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_image", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 文件+视频+音频 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_video", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 文件+图片+视频+音频 + { + "media": [ + { "type": "file", "url": "..." }, + { "type": "reference_image", "url": "..." }, + { "type": "reference_video", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + ``` + + ```json 视频编辑 + // 必须包含 reference_video,可与 reference_image/reference_audio 组合 + // 提示词含编辑意图 + + // 仅参考视频 + { + "media": [ + { "type": "reference_video", "url": "..." } + ] + } + + // 视频+图片 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_image", "url": "..." } + ] + } + + // 视频+音频 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 视频+图片+音频 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_image", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + ``` + + ```json 视频延长 + // 必须包含 reference_video,可与 reference_image/reference_audio 组合 + // 提示词含延长意图 + + // 仅参考视频 + { + "media": [ + { "type": "reference_video", "url": "..." } + ] + } + + // 视频+图片 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_image", "url": "..." } + ] + } + + // 视频+音频 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + + // 视频+图片+音频 + { + "media": [ + { "type": "reference_video", "url": "..." }, + { "type": "reference_image", "url": "..." }, + { "type": "reference_audio", "url": "..." } + ] + } + ``` + + +## 常见问题 + +### 首尾帧模式下为什么不能同时传入音频等其他媒体素材? + +万相3.0 的首帧/首尾帧模式仅支持传入 `first_frame` 和 `last_frame`,不可同时传入音频等其他类型(详见[素材组合](#w30media_combo_t))。 + +与[万相2.7图生视频系列](/zh/model-studio/wan-image-to-video-guide)的区别:wan2.7 首帧/首尾帧模式支持同时传入 `driving_audio` 驱动音频;万相3.0 首帧/首尾帧模式不支持,如需音频驱动,请使用全模态参考模式(`reference_image` + `reference_audio` 组合)替代首尾帧模式。具体用法请参见[万相3.0-视频生成使用指南](/zh/model-studio/wan3-video-generation-guide)。 diff --git a/internal/cmd/cmd.go b/internal/cmd/cmd.go index ace1129..a4814f9 100644 --- a/internal/cmd/cmd.go +++ b/internal/cmd/cmd.go @@ -595,6 +595,14 @@ func registerRelayRoutes(server *ghttp.Server) { group.POST("/video_generation", relay.HandleMiniMaxVideoV1Submit) group.GET("/query/video_generation", relay.HandleMiniMaxVideoV1Retrieve) }) + + // 阿里 DashScope 官方视频协议端点(wan2.x / wan3.0 通用,阿里 SDK 换 base_url 直连; + // 查询回放官方响应形态,轮询计费由网关后台完成) + server.Group("/api/v1", func(group *ghttp.RouterGroup) { + group.Middleware(middleware.ApiMaintenance, middleware.MaintenanceMode, middleware.ApiKeyAuth, middleware.RelayBodyLimit, middleware.ContentFilter) + group.POST("/services/aigc/video-generation/video-synthesis", relay.HandleAliVideoSubmit) + group.GET("/tasks/{task_id}", relay.HandleAliVideoRetrieve) + }) } // registerLandingPage 在未内嵌前端(非 embedweb 构建)时,为后端根路径注册项目介绍页, diff --git a/internal/handler/relay/ali_video.go b/internal/handler/relay/ali_video.go new file mode 100644 index 0000000..553a34a --- /dev/null +++ b/internal/handler/relay/ali_video.go @@ -0,0 +1,112 @@ +package relay + +import ( + "github.com/gogf/gf/v2/net/ghttp" + + "github.com/qianfree/team-api/internal/logic/monitor" + "github.com/qianfree/team-api/internal/middleware" + relay_constant "github.com/qianfree/team-api/relay/constant" + relay_handler "github.com/qianfree/team-api/relay/handler" +) + +// 阿里 DashScope 官方视频协议入站胶水(/api/v1 端点): +// 创建请求在此完成官方请求体解析与通用任务体转换,复用任务提交管线 +// (selectTaskChannel 渠道调度 + relay_handler.HandleTaskSubmit 计费/提交/落库); +// 查询直连 relay_handler 的阿里视频处理器(回放 DashScope 官方响应形态)。 +// 协议文档:docs/协议文档/阿里系列/视频-wan3.0.md + +// HandleAliVideoSubmit 处理 POST /api/v1/services/aigc/video-generation/video-synthesis +// (创建视频生成任务,wan2.x / wan3.0 通用) +func HandleAliVideoSubmit(r *ghttp.Request) { + body := r.GetBody() + if len(body) == 0 { + writeVideosError(r, 400, "request body is empty", "") + return + } + + // 官方请求体 → 归一结构(input.media[] 原样保留) + req, taskErr := relay_handler.ParseAliVideoCreateRequest(body) + if taskErr != nil { + writeVideosError(r, taskErr.StatusCode, taskErr.Message, taskErr.ErrCode) + return + } + if taskErr := relay_handler.ValidateAliVideoCreateRequest(req); taskErr != nil { + writeVideosError(r, taskErr.StatusCode, taskErr.Message, taskErr.ErrCode) + return + } + + // 转成任务框架通用请求体(适配器消费 media/metadata + 双写通用词汇) + taskBody, err := relay_handler.BuildAliVideoTaskBody(req) + if err != nil { + writeVideosError(r, 500, "build task body failed", "") + return + } + + rc := &relay_handler.TaskRelayContext{ + TenantID: middleware.GetTenantID(r.Context()), + UserID: middleware.GetUserID(r.Context()), + ApiKeyID: middleware.GetApiKeyID(r.Context()), + ProjectID: middleware.GetProjectID(r.Context()), + RequestID: r.GetCtxVar("RequestId").String(), + Writer: r.Response.Writer, + Scope: r.GetCtxVar("ApiKeyScope").String(), + ClientIP: r.GetClientIp(), + KeyRateLimitQps: r.GetCtxVar(middleware.CtxKeyApiKeyRateLimitQps).Int(), + KeyConcurrency: r.GetCtxVar(middleware.CtxKeyApiKeyRateLimitConcurrency).Int(), + KeyIpWhitelist: r.GetCtxVar(middleware.CtxKeyApiKeyIpWhitelist).String(), + KeyTotalQuota: r.GetCtxVar(middleware.CtxKeyApiKeyTotalQuota).Float64(), + KeyUsedQuota: r.GetCtxVar(middleware.CtxKeyApiKeyUsedQuota).Float64(), + // 阿里 DashScope 官方协议:{"output":{"task_id":...}} 响应 + task_ 前缀 ID + // (协议标识与 relay/handler.aliVideoProtocol 对齐,对齐 videos.go 胶水层字面量惯例) + Protocol: "ali_native", + RelayMode: int(relay_constant.RelayModeAliVideo), + } + + // 选择渠道(模型权限校验 + 调度引擎,与 /v1/video/generations 同一链路) + channelMeta, err := selectTaskChannel(r, taskBody) + if err != nil { + writeTaskChannelSelectionError(r, err) + return + } + + // 响应捕获用于审计(对齐 HandleTaskSubmit 模式) + capture := NewResponseCaptureWriter(r.Response.Writer) + rc.Writer = capture + + registerAsyncTask(rc.RequestID, rc.TenantID, rc.UserID, rc.ProjectID, req.Model, channelMeta, r.URL.Path) + + relay_handler.HandleTaskSubmit( + r.Context(), taskBody, r.URL.Path, r.Header, + rc, taskDataProvider, taskBillingProvider, channelMeta, + ) + + // 提交成功后切换为任务 ID 跟踪(任务生命周期远超 HTTP 请求) + if rc.TaskID != "" { + monitor.SwitchToTaskID(rc.RequestID, rc.TaskID) + } else { + monitor.UnregisterRequest(rc.RequestID) + } + + rc.ForwardingTrace = buildTaskForwardingTrace(r.URL.Path, taskBody, channelMeta, capture.StatusCode()) + go recordTaskSubmitAudit(r, rc, capture, taskBody) +} + +// HandleAliVideoRetrieve 处理 GET /api/v1/tasks/{task_id}(查询视频任务,回放官方响应) +func HandleAliVideoRetrieve(r *ghttp.Request) { + taskID := r.Get("task_id").String() + if taskID == "" { + writeVideosError(r, 400, "task id is required", "") + return + } + + rc := &relay_handler.TaskRelayContext{ + TenantID: middleware.GetTenantID(r.Context()), + UserID: middleware.GetUserID(r.Context()), + ApiKeyID: middleware.GetApiKeyID(r.Context()), + ProjectID: middleware.GetProjectID(r.Context()), + RequestID: r.GetCtxVar("RequestId").String(), + Writer: r.Response.Writer, + } + + relay_handler.HandleAliVideoRetrieve(r.Context(), taskID, rc, taskDataProvider) +} diff --git a/internal/logic/billing/scheme.go b/internal/logic/billing/scheme.go index 5057bd2..acfd770 100644 --- a/internal/logic/billing/scheme.go +++ b/internal/logic/billing/scheme.go @@ -110,9 +110,33 @@ func (GenericScheme) ValidateSchemeConfig(cfg json.RawMessage) error { return nil } -// SettleTaskCost 通用引擎无素材计量语义:忽略 usage,回退预扣口径 -// (与 estimateTaskCost 的估算公式一致,结算保持「预扣即终价,token 重算另走 RecalculateByTokens」)。 -func (g GenericScheme) SettleTaskCost(pricing *PricingResult, ratios map[string]any, _ *rcommon.TaskMaterialUsage) decimal.Decimal { +// SettleTaskCost 通用引擎的结算口径: +// - per_second(按秒计费):上游返回实际输出秒数(usage.OutputSeconds > 0)时按 +// 「实际秒数 × 矩阵单价 × 租户乘数 × 时段乘数」结算,多退少补——查价键优先 +// usage.Resolution(实际输出分辨率,如阿里 wan3.0 usage.SR),回退 ratios 的 +// spec.resolution(提交时请求档位)。实际时长与请求时长不一致(智能时长模式、 +// 上游按内容微调)时以官方计量为准;usage 缺失回退预扣口径。 +// - 其余模式(token / per_request / tiered)无素材计量语义:忽略 usage,回退预扣口径 +// (与 estimateTaskCost 的估算公式一致,结算保持「预扣即终价,token 重算另走 RecalculateByTokens」)。 +func (g GenericScheme) SettleTaskCost(pricing *PricingResult, ratios map[string]any, usage *rcommon.TaskMaterialUsage) decimal.Decimal { + if pricing.BillingMode == "per_second" && usage != nil && usage.OutputSeconds > 0 { + duration := usage.OutputSeconds + // 结算秒数钳制上限与预扣一致,防御异常大值刷扣 + if duration > maxTaskDurationSeconds { + duration = maxTaskDurationSeconds + } + spec, _ := ratioString(ratios, "spec.resolution") + if usage.Resolution != "" { + spec = usage.Resolution + } + if price := LookupPerSecondPrice(pricing.PerSecondPrices, spec); price > 0 { + return NewFromFloat(price). + Mul(NewFromFloat(duration)). + Mul(NewFromFloat(pricing.TenantMultiplier)). + Mul(NewFromFloat(effectiveTimeMultiplier(pricing))) + } + // 矩阵为空/全零:落到下方回退(占位/预扣口径),与预扣分支的兜底行为一致 + } return g.EstimateTaskCost(pricing, ratios, nil) } diff --git a/internal/logic/billing/scheme_per_second_settle_test.go b/internal/logic/billing/scheme_per_second_settle_test.go new file mode 100644 index 0000000..0d65ee9 --- /dev/null +++ b/internal/logic/billing/scheme_per_second_settle_test.go @@ -0,0 +1,104 @@ +package billing + +import ( + "testing" + + rcommon "github.com/qianfree/team-api/relay/common" +) + +// generic per_second 按实际输出秒数结算(方案 A)的行为测试。 +// 矩阵:480P=0.02 / 720P=0.05 / 1080P=0.10,"*" 兜底 0.05(对齐 wan3.0 三档定价形态)。 + +// perSecondTestPricing 构造 generic per_second 测试定价(乘数恒 1,租户折扣单测另行覆盖) +func perSecondTestPricing(tenantMul float64) *PricingResult { + return &PricingResult{ + BillingMode: "per_second", + TenantMultiplier: tenantMul, + PerSecondPrices: map[string]float64{"480P": 0.02, "720P": 0.05, "1080P": 0.10, "*": 0.05}, + } +} + +// TestGenericPerSecond_SettleByActualSeconds 结算口径:上游返回实际输出秒数时按 +// 「实际秒数 × 档位单价 × 租户 × 时段」结算,多退少补;usage 缺失回退预扣口径。 +func TestGenericPerSecond_SettleByActualSeconds(t *testing.T) { + pricing := perSecondTestPricing(1.0) + scheme := GenericScheme{} + + // 实际 5.13s(上游浮点秒)× 720P 档:请求 5s 预扣与实际有零头差,按实际结算 + cost := scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "720P", + }, &rcommon.TaskMaterialUsage{OutputSeconds: 5.13, Resolution: "720P"}) + assertDecimal(t, cost, 5.13*0.05, "settle by actual seconds") + + // usage.Resolution 优先于 ratios 提交时档位(请求未指定档位,实际输出 480P) + cost = scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, + }, &rcommon.TaskMaterialUsage{OutputSeconds: 5, Resolution: "480P"}) + assertDecimal(t, cost, 5*0.02, "usage resolution wins over ratios") + + // usage.Resolution 缺失回退 ratios 提交时档位;实际秒数长于请求时补扣 + cost = scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "1080P", + }, &rcommon.TaskMaterialUsage{OutputSeconds: 12}) + assertDecimal(t, cost, 12*0.10, "fallback resolution, supplement longer actual") + + // 智能时长场景:预扣按上限 30s 冻结,实际生成 12s 结算退差 + cost = scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 30.0, "spec.resolution": "720P", + }, &rcommon.TaskMaterialUsage{OutputSeconds: 12, Resolution: "720P"}) + assertDecimal(t, cost, 12*0.05, "smart duration refunds difference") + + // 异常大秒数钳制 120s(与预扣钳制同源,防 usage 异常刷扣) + cost = scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "720P", + }, &rcommon.TaskMaterialUsage{OutputSeconds: 99999, Resolution: "720P"}) + assertDecimal(t, cost, 120*0.05, "clamp abnormal seconds") + + // 档位未命中矩阵 → "*" 兜底价 + cost = scheme.SettleTaskCost(pricing, map[string]any{}, + &rcommon.TaskMaterialUsage{OutputSeconds: 5, Resolution: "360P"}) + assertDecimal(t, cost, 5*0.05, "unknown resolution falls back to wildcard") + + // usage 为 nil:回退预扣口径(请求 5s × 720P) + cost = scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "720P", + }, nil) + assertDecimal(t, cost, 5*0.05, "nil usage falls back to estimate") + + // usage 存在但 OutputSeconds=0:同样回退预扣口径 + cost = scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "720P", + }, &rcommon.TaskMaterialUsage{Resolution: "720P"}) + assertDecimal(t, cost, 5*0.05, "zero output seconds falls back to estimate") + + // 矩阵为空/全零:落到预扣分支的占位兜底(与预扣行为一致) + empty := perSecondTestPricing(1.0) + empty.PerSecondPrices = nil + cost = scheme.SettleTaskCost(empty, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "720P", + }, &rcommon.TaskMaterialUsage{OutputSeconds: 5, Resolution: "720P"}) + assertDecimal(t, cost, imagePlaceholderPreDeduct, "empty matrix falls back to placeholder") +} + +// TestGenericPerSecond_SettleTenantMultiplier 租户折扣作用于实际秒数结算。 +func TestGenericPerSecond_SettleTenantMultiplier(t *testing.T) { + pricing := perSecondTestPricing(0.5) + scheme := GenericScheme{} + + cost := scheme.SettleTaskCost(pricing, map[string]any{ + "spec.duration": 5.0, "spec.resolution": "720P", + }, &rcommon.TaskMaterialUsage{OutputSeconds: 5.13, Resolution: "720P"}) + assertDecimal(t, cost, 5.13*0.05*0.5, "tenant multiplier applies to actual seconds") +} + +// TestGenericPerSecond_NonPerSecondModesIgnoreUsage 回归:非 per_second 模式(token 等) +// 的结算不受 usage 影响,保持「预扣即终价」。 +func TestGenericPerSecond_NonPerSecondModesIgnoreUsage(t *testing.T) { + scheme := GenericScheme{} + + // token 模式 + usage 带秒数:忽略 usage,走预扣口径(无时长信号 → 占位兜底) + tokenPricing := &PricingResult{BillingMode: "token", OutputPrice: 3.0, TenantMultiplier: 1.0} + cost := scheme.SettleTaskCost(tokenPricing, map[string]any{}, + &rcommon.TaskMaterialUsage{OutputSeconds: 100}) + assertDecimal(t, cost, imagePlaceholderPreDeduct, "token mode ignores usage") +} diff --git a/internal/logic/billing/task_billing.go b/internal/logic/billing/task_billing.go index 962c63c..edddb42 100644 --- a/internal/logic/billing/task_billing.go +++ b/internal/logic/billing/task_billing.go @@ -438,9 +438,11 @@ func (b *TaskBillingProviderImpl) RecalculateByMaterials(ctx context.Context, te // 取价失败不阻断结算(调用方回落预扣金额),仅返回 ok=false return Zero, false, err } - // 仅显式配置了计费方案的模型适用:generic 任务保持「预扣即终价」不被重估 - // (素材计费语义由方案声明,未声明素材的模型不应因 usage 附带字段改变结算来源) - if pricing.Scheme == "" { + // 仅显式配置了计费方案、或 generic per_second(消费上游实际输出秒数结算)的模型适用: + // 其余 generic 任务保持「预扣即终价」不被重估(素材计费语义由方案声明,未声明素材的 + // 模型不应因 usage 附带字段改变结算来源)。per_second 放行是安全的:存量按秒渠道 + // (kling/volcengine/sora)适配器均不产出 MaterialUsage,结算链路到不了这里,行为不变 + if pricing.Scheme == "" && pricing.BillingMode != "per_second" { return Zero, false, nil } costD := LookupScheme(pricing.Scheme).SettleTaskCost(pricing, ratios, usage) diff --git a/relay/common/task_adaptor.go b/relay/common/task_adaptor.go index 64c162d..80424bd 100644 --- a/relay/common/task_adaptor.go +++ b/relay/common/task_adaptor.go @@ -84,6 +84,7 @@ type TaskMaterialUsage struct { InputImageCount int // 输入图片张数 InputAudioSeconds float64 // 输入音频时长(秒) OutputSeconds float64 // 输出(生成)时长(秒) + Resolution string // 实际输出分辨率规格(如 "720P",对齐 per_second 矩阵键);空 = 上游未返回 } // TaskStatusEnum 任务状态枚举 diff --git a/relay/constant/relay_mode.go b/relay/constant/relay_mode.go index 3c75711..bf60f43 100644 --- a/relay/constant/relay_mode.go +++ b/relay/constant/relay_mode.go @@ -33,6 +33,7 @@ const ( RelayModeResponsesCompact // POST /v1/responses/compact RelayModeVideos // OpenAI Videos 协议 /v1/videos(创建/查询/删除/内容下载) RelayModeMiniMaxVideo // MiniMax 官方协议 /v2/video_generation(H3 v2 提交/查询/删除) + RelayModeAliVideo // 阿里 DashScope 官方协议 /api/v1/services/aigc/video-generation/video-synthesis(提交/查询) ) // Path2RelayMode 根据请求路径判断 RelayMode @@ -159,6 +160,8 @@ func (m RelayMode) String() string { return "videos" case RelayModeMiniMaxVideo: return "minimax_video" + case RelayModeAliVideo: + return "ali_video" case RelayModeMjSubmit: return "mj_submit" case RelayModeMjFetch: diff --git a/relay/handler/ali_video_convert.go b/relay/handler/ali_video_convert.go new file mode 100644 index 0000000..ae78a8d --- /dev/null +++ b/relay/handler/ali_video_convert.go @@ -0,0 +1,263 @@ +package handler + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "strconv" + "strings" + "time" + + "github.com/gogf/gf/v2/frame/g" + + "github.com/qianfree/team-api/relay/common" + "github.com/qianfree/team-api/relay/constant" +) + +// 阿里 DashScope 官方视频协议(POST /api/v1/services/aigc/video-generation/video-synthesis)入站转换层。 +// 原生请求体(model/input.prompt/input.media[]/parameters)解析校验后转成任务框架通用请求体, +// 复用任务提交管线(selectTaskChannel 渠道调度 + HandleTaskSubmit 计费/提交/落库); +// 查询端点 GET /api/v1/tasks/{task_id} 回放 DashScope 官方响应形态(协议文档 +// docs/协议文档/阿里系列/视频-wan3.0.md),阿里 dashscope SDK 换 base_url 即可直连。 +// +// 转换产物在保留原生字段(media[]/negative_prompt/audio_url)之外,双写通用词汇 +// (prompt/seconds/metadata)——参数倍率规则的匹配源是归一化任务体(与 MiniMax 官方 +// 协议入站同一约定),保证按通用路径配置的倍率规则对官方协议入站同样生效。 + +// aliVideoProtocol TaskRelayContext.Protocol 的阿里 DashScope 官方视频协议标识 +const aliVideoProtocol = "ali_native" + +// aliVideoCreateRequest DashScope 官方创建请求的归一形态(JSON 解析后的产物) +type aliVideoCreateRequest struct { + Model string + Prompt string + NegativePrompt string + AudioURL string + Media []aliVideoMedia + Parameters aliVideoParameters +} + +// aliVideoMedia wan3.0 input.media 数组元素 +type aliVideoMedia struct { + Type string `json:"type"` + URL string `json:"url"` +} + +// aliVideoParameters DashScope parameters(wan2.x 与 wan3.0 字段并集) +type aliVideoParameters struct { + Resolution string `json:"resolution,omitempty"` + Ratio string `json:"ratio,omitempty"` + Size string `json:"size,omitempty"` + Duration *int `json:"duration,omitempty"` // wan3.0:-1 = 智能时长 + Audio *bool `json:"audio,omitempty"` // wan3.0:输出音频开关 + Seed *int `json:"seed,omitempty"` + PromptExtend *bool `json:"prompt_extend,omitempty"` + Watermark *bool `json:"watermark,omitempty"` +} + +// jsonAliVideoInput 官方请求的 input 节(wan2.x 扁平字段 + wan3.0 media 数组并集) +type jsonAliVideoInput struct { + Prompt string `json:"prompt"` + NegativePrompt string `json:"negative_prompt"` + AudioURL string `json:"audio_url"` + Media []aliVideoMedia `json:"media"` +} + +// ParseAliVideoCreateRequest 解析官方创建请求。 +// 返回的 TaskError 已携带面向客户端的 HTTP 状态码与错误信息。 +func ParseAliVideoCreateRequest(body []byte) (*aliVideoCreateRequest, *common.TaskError) { + var raw struct { + Model string `json:"model"` + Input jsonAliVideoInput `json:"input"` + Parameters aliVideoParameters `json:"parameters"` + } + if err := json.Unmarshal(body, &raw); err != nil { + return nil, &common.TaskError{StatusCode: http.StatusBadRequest, Message: "invalid request body: " + err.Error()} + } + + return &aliVideoCreateRequest{ + Model: strings.TrimSpace(raw.Model), + Prompt: raw.Input.Prompt, + NegativePrompt: raw.Input.NegativePrompt, + AudioURL: raw.Input.AudioURL, + Media: raw.Input.Media, + Parameters: raw.Parameters, + }, nil +} + +// ValidateAliVideoCreateRequest 请求必填校验(对齐官方约束:model 必填,prompt 与 media 必填其一) +func ValidateAliVideoCreateRequest(req *aliVideoCreateRequest) *common.TaskError { + if req.Model == "" { + return &common.TaskError{StatusCode: http.StatusBadRequest, Message: "model is required"} + } + if strings.TrimSpace(req.Prompt) == "" && len(req.Media) == 0 { + return &common.TaskError{StatusCode: http.StatusBadRequest, Message: "one of input.prompt / input.media is required"} + } + return nil +} + +// BuildAliVideoTaskBody 把官方创建请求转成任务框架通用请求体(JSON 字节)。 +// media 数组放顶层原样透传(适配器 buildWan3Media 消费);通用词汇(seconds/metadata) +// 双写保证参数倍率规则与 per_second 预扣信号(spec.duration/spec.resolution)同源。 +func BuildAliVideoTaskBody(req *aliVideoCreateRequest) ([]byte, error) { + body := map[string]any{ + "model": req.Model, + "prompt": req.Prompt, + } + if len(req.Media) > 0 { + body["media"] = req.Media + } + + metadata := map[string]any{} + if req.NegativePrompt != "" { + body["negative_prompt"] = req.NegativePrompt + metadata["negative_prompt"] = req.NegativePrompt + } + if req.AudioURL != "" { + body["audio_url"] = req.AudioURL + metadata["audio_url"] = req.AudioURL + } + + p := req.Parameters + if p.Resolution != "" { + body["resolution"] = p.Resolution + metadata["resolution"] = p.Resolution + } + if p.Ratio != "" { + body["ratio"] = p.Ratio + metadata["ratio"] = p.Ratio + } + if p.Size != "" { + metadata["size"] = p.Size + } + // duration 保留原值(含 -1 智能时长信号,供适配器预扣按上限冻结);正数另写通用 seconds + if p.Duration != nil { + metadata["duration"] = *p.Duration + if *p.Duration > 0 { + body["seconds"] = strconv.Itoa(*p.Duration) + } + } + if p.Audio != nil { + metadata["sound"] = map[bool]string{true: "on", false: "off"}[*p.Audio] + metadata["generate_audio"] = *p.Audio + } + if p.Seed != nil { + metadata["seed"] = *p.Seed + } + if p.PromptExtend != nil { + metadata["prompt_extend"] = *p.PromptExtend + } + if p.Watermark != nil { + metadata["watermark"] = *p.Watermark + } + if len(metadata) > 0 { + body["metadata"] = metadata + } + + data, err := json.Marshal(body) + if err != nil { + return nil, fmt.Errorf("marshal ali video task body: %w", err) + } + return data, nil +} + +// ==================== 查询端点回放 ==================== + +// aliTaskResponse DashScope 任务查询响应形态(GET /api/v1/tasks/{task_id} 的回放产物)。 +// usage 用 RawMessage 原样保留上游计量(duration/output_video_duration/fps/SR/ratio) +type aliTaskResponse struct { + Output struct { + TaskID string `json:"task_id"` + TaskStatus string `json:"task_status"` + SubmitTime string `json:"submit_time,omitempty"` + ScheduledTime string `json:"scheduled_time,omitempty"` + EndTime string `json:"end_time,omitempty"` + OrigPrompt string `json:"orig_prompt,omitempty"` + VideoURL string `json:"video_url,omitempty"` + Code string `json:"code,omitempty"` + Message string `json:"message,omitempty"` + } `json:"output"` + Usage json.RawMessage `json:"usage,omitempty"` + RequestID string `json:"request_id"` +} + +// aliStatusFromTask 平台任务状态 → DashScope 官方状态机映射 +func aliStatusFromTask(status string) string { + switch status { + case "IN_PROGRESS": + return "RUNNING" + case "SUCCESS": + return "SUCCEEDED" + case "FAILURE": + return "FAILED" + default: + // NOT_START / SUBMITTED / QUEUED 及一切未知中间态都归入 PENDING + return "PENDING" + } +} + +// buildAliTaskResponse 平台任务 → DashScope 官方查询响应。 +// 优先回放 task.Data 缓存的上游查询响应(含 usage/orig_prompt/时间线,保真度最高), +// 仅覆写 output.task_id 为平台公开任务 ID、task_status 以平台状态为准(上游响应原文 +// 可能滞后一轮轮询);task.Data 不可解析(提交后首拍轮询前为提交响应形态或空)时 +// 构造最小响应。失败任务补 code/message(上游已带则保留) +func buildAliTaskResponse(task *common.AsyncTask) *aliTaskResponse { + resp := &aliTaskResponse{} + if len(task.Data) > 0 { + _ = json.Unmarshal(task.Data, resp) + } + status := aliStatusFromTask(task.Status) + + resp.Output.TaskID = task.PublicTaskID + resp.Output.TaskStatus = status + resp.RequestID = task.RequestID + + if status == "FAILED" { + if resp.Output.Code == "" { + resp.Output.Code = "TaskFailed" + } + if resp.Output.Message == "" { + resp.Output.Message = task.FailReason + if resp.Output.Message == "" { + resp.Output.Message = "video generation failed" + } + } + } + return resp +} + +// HandleAliVideoRetrieve GET /api/v1/tasks/{task_id} 查询管线(回放 DashScope 响应形态)。 +// 租户双键隔离(user_id + tenant_id)复用 GetTaskByPublicIDAndUser;平台门禁:该端点只服务 +// ali 平台任务,防止异构平台任务 ID 被塞进 DashScope 响应结构回放(对齐 MiniMax 官方端点做法) +func HandleAliVideoRetrieve( + ctx context.Context, + taskID string, + rc *TaskRelayContext, + dataProvider common.TaskDataProvider, +) { + task, err := dataProvider.GetTaskByPublicIDAndUser(ctx, taskID, rc.UserID, rc.TenantID) + if err != nil { + g.Log().Errorf(ctx, "AliVideoRetrieve: query task failed, taskID=%s, err=%v", taskID, err) + writeTaskError(rc.Writer, http.StatusInternalServerError, "query task failed", "") + return + } + if task == nil || task.Platform != string(constant.TaskPlatformAli) { + writeTaskError(rc.Writer, http.StatusNotFound, "task not found", "") + return + } + writeJSON(rc.Writer, http.StatusOK, buildAliTaskResponse(task)) +} + +// writeAliVideoSubmitResponse 提交成功响应:官方 DashScope 形态 +// ({"output":{"task_status":"PENDING","task_id":...},"request_id":...}) +func writeAliVideoSubmitResponse(w http.ResponseWriter, publicTaskID, requestID string, now time.Time) { + _ = now // 提交响应形态无时间字段,参数保留对齐其他协议 writer 的签名习惯 + writeJSON(w, http.StatusOK, map[string]any{ + "output": map[string]any{ + "task_status": "PENDING", + "task_id": publicTaskID, + }, + "request_id": requestID, + }) +} diff --git a/relay/handler/ali_video_convert_test.go b/relay/handler/ali_video_convert_test.go new file mode 100644 index 0000000..3355457 --- /dev/null +++ b/relay/handler/ali_video_convert_test.go @@ -0,0 +1,205 @@ +package handler + +import ( + "encoding/json" + "testing" + "time" + + "github.com/qianfree/team-api/relay/common" +) + +func TestParseAliVideoCreateRequest_Full(t *testing.T) { + body := `{ + "model": "wan3.0-video", + "input": { + "prompt": "一只小猫在月光下的屋顶上奔跑", + "media": [ + {"type": "first_frame", "url": "https://example.com/f.png"}, + {"type": "reference_image", "url": "https://example.com/r.png"} + ] + }, + "parameters": { + "resolution": "1080P", "ratio": "16:9", "duration": 5, + "audio": false, "seed": 42, "prompt_extend": true, "watermark": false + } + }` + req, taskErr := ParseAliVideoCreateRequest([]byte(body)) + if taskErr != nil { + t.Fatalf("unexpected error: %+v", taskErr) + } + if req.Model != "wan3.0-video" || req.Prompt != "一只小猫在月光下的屋顶上奔跑" { + t.Fatalf("unexpected request: %+v", req) + } + if len(req.Media) != 2 || req.Media[0].Type != "first_frame" || req.Media[1].URL != "https://example.com/r.png" { + t.Fatalf("unexpected media: %+v", req.Media) + } + if req.Parameters.Resolution != "1080P" || req.Parameters.Ratio != "16:9" { + t.Fatalf("unexpected parameters: %+v", req.Parameters) + } + if req.Parameters.Duration == nil || *req.Parameters.Duration != 5 { + t.Fatalf("unexpected duration: %+v", req.Parameters.Duration) + } + if req.Parameters.Audio == nil || *req.Parameters.Audio { + t.Fatalf("unexpected audio: %+v", req.Parameters.Audio) + } +} + +func TestParseAliVideoCreateRequest_LegacyWan2(t *testing.T) { + // wan2.x 旧格式(扁平 input,无 media)同样可解析 + body := `{"model":"wan2.2-t2v-plus","input":{"prompt":"p","negative_prompt":"np","audio_url":"https://a.mp3"},"parameters":{"size":"1024*576"}}` + req, taskErr := ParseAliVideoCreateRequest([]byte(body)) + if taskErr != nil { + t.Fatalf("unexpected error: %+v", taskErr) + } + if req.Model != "wan2.2-t2v-plus" || req.Prompt != "p" || req.NegativePrompt != "np" || req.AudioURL != "https://a.mp3" { + t.Fatalf("unexpected request: %+v", req) + } + if len(req.Media) != 0 { + t.Fatalf("wan2 request must not carry media: %+v", req.Media) + } +} + +func TestParseAliVideoCreateRequest_InvalidJSON(t *testing.T) { + if _, taskErr := ParseAliVideoCreateRequest([]byte(`{invalid`)); taskErr == nil || taskErr.StatusCode != 400 { + t.Fatalf("expected 400 task error, got %+v", taskErr) + } +} + +func TestValidateAliVideoCreateRequest(t *testing.T) { + // 缺 model + if taskErr := ValidateAliVideoCreateRequest(&aliVideoCreateRequest{Prompt: "p"}); taskErr == nil { + t.Error("missing model should be rejected") + } + // prompt 与 media 都缺(文档约束:必填其一) + if taskErr := ValidateAliVideoCreateRequest(&aliVideoCreateRequest{Model: "wan3.0-video"}); taskErr == nil { + t.Error("missing both prompt and media should be rejected") + } + // 纯 media(无 prompt)合法 + if taskErr := ValidateAliVideoCreateRequest(&aliVideoCreateRequest{ + Model: "wan3.0-video", + Media: []aliVideoMedia{{Type: "reference_video", URL: "https://v.mp4"}}, + }); taskErr != nil { + t.Errorf("media-only request should be accepted: %+v", taskErr) + } +} + +func TestBuildAliVideoTaskBody(t *testing.T) { + duration := 5 + audio := false + body, err := BuildAliVideoTaskBody(&aliVideoCreateRequest{ + Model: "wan3.0-video", + Prompt: "p", + Media: []aliVideoMedia{{Type: "first_frame", URL: "https://f.png"}}, + Parameters: aliVideoParameters{ + Resolution: "480P", Ratio: "16:9", Duration: &duration, Audio: &audio, + }, + }) + if err != nil { + t.Fatalf("build: %v", err) + } + var m map[string]any + if err := json.Unmarshal(body, &m); err != nil { + t.Fatalf("unmarshal: %v", err) + } + // 顶层保留:model/prompt/media/通用 seconds + if m["model"] != "wan3.0-video" || m["prompt"] != "p" || m["seconds"] != "5" { + t.Fatalf("unexpected top-level body: %s", body) + } + media, ok := m["media"].([]any) + if !ok || len(media) != 1 { + t.Fatalf("expected media passthrough: %s", body) + } + // metadata 双写:duration/resolution/ratio/sound/generate_audio + meta, _ := m["metadata"].(map[string]any) + if meta["duration"] != float64(5) || meta["resolution"] != "480P" || meta["ratio"] != "16:9" { + t.Fatalf("unexpected metadata: %v", meta) + } + if meta["sound"] != "off" || meta["generate_audio"] != false { + t.Fatalf("audio switch not normalized: %v", meta) + } +} + +func TestBuildAliVideoTaskBody_SmartDuration(t *testing.T) { + // duration=-1(智能时长):metadata 保留 -1 信号(适配器预扣按 30s 上限冻结),不写 seconds + minusOne := -1 + body, err := BuildAliVideoTaskBody(&aliVideoCreateRequest{ + Model: "wan3.0-video", + Prompt: "p", + Parameters: aliVideoParameters{Duration: &minusOne}, + }) + if err != nil { + t.Fatalf("build: %v", err) + } + var m map[string]any + _ = json.Unmarshal(body, &m) + if _, has := m["seconds"]; has { + t.Fatalf("smart duration must not emit seconds: %s", body) + } + meta, _ := m["metadata"].(map[string]any) + if meta["duration"] != float64(-1) { + t.Fatalf("smart duration signal lost: %v", meta) + } +} + +func TestBuildAliTaskResponse(t *testing.T) { + now := time.Now() + submitted := now.Add(-time.Minute) + + // 终态成功:回放上游查询响应(usage/orig_prompt 保真),覆写 task_id/status + task := &common.AsyncTask{ + PublicTaskID: "task_pub_1", + RequestID: "req_1", + Platform: "ali", + Status: "SUCCESS", + CreatedAt: submitted, + Data: []byte(`{"output":{"task_id":"up-stream-id","task_status":"RUNNING","video_url":"https://v.mp4","orig_prompt":"a cat"},"usage":{"video_count":1,"duration":5.0,"output_video_duration":5.13,"fps":30,"SR":720,"ratio":"16:9"},"request_id":"up-req"}`), + } + resp := buildAliTaskResponse(task) + if resp.Output.TaskID != "task_pub_1" || resp.Output.TaskStatus != "SUCCEEDED" { + t.Fatalf("unexpected replay: %+v", resp.Output) + } + if resp.Output.VideoURL != "https://v.mp4" || resp.Output.OrigPrompt != "a cat" { + t.Fatalf("upstream fields must be preserved: %+v", resp.Output) + } + var usage map[string]any + if err := json.Unmarshal(resp.Usage, &usage); err != nil { + t.Fatalf("usage not preserved: %v", err) + } + if usage["output_video_duration"] != 5.13 || usage["SR"] != float64(720) { + t.Fatalf("usage metering lost: %v", usage) + } + if resp.RequestID != "req_1" { + t.Fatalf("request id should be platform's: %s", resp.RequestID) + } + + // 非终态(提交后首拍,Data 为提交响应形态):构造 PENDING + pending := &common.AsyncTask{ + PublicTaskID: "task_pub_2", + RequestID: "req_2", + Platform: "ali", + Status: "SUBMITTED", + Data: []byte(`{"output":{"task_status":"PENDING","task_id":"up-2"},"request_id":"up-req-2"}`), + } + resp = buildAliTaskResponse(pending) + if resp.Output.TaskID != "task_pub_2" || resp.Output.TaskStatus != "PENDING" { + t.Fatalf("unexpected pending replay: %+v", resp.Output) + } + + // Data 为空:构造最小响应 + empty := &common.AsyncTask{PublicTaskID: "task_pub_3", RequestID: "req_3", Platform: "ali", Status: "IN_PROGRESS"} + resp = buildAliTaskResponse(empty) + if resp.Output.TaskID != "task_pub_3" || resp.Output.TaskStatus != "RUNNING" { + t.Fatalf("unexpected empty-data replay: %+v", resp.Output) + } + + // 失败任务:上游 code/message 保留;缺失时用平台 FailReason 兜底 + failed := &common.AsyncTask{ + PublicTaskID: "task_pub_4", RequestID: "req_4", Platform: "ali", Status: "FAILURE", + FailReason: "upstream quota exceeded", + Data: []byte(`{"output":{"task_id":"up-4","task_status":"FAILED","code":"Throttling"}}`), + } + resp = buildAliTaskResponse(failed) + if resp.Output.Code != "Throttling" || resp.Output.Message != "upstream quota exceeded" { + t.Fatalf("unexpected failure replay: %+v", resp.Output) + } +} diff --git a/relay/handler/ali_video_pipeline_test.go b/relay/handler/ali_video_pipeline_test.go new file mode 100644 index 0000000..34d7ef0 --- /dev/null +++ b/relay/handler/ali_video_pipeline_test.go @@ -0,0 +1,208 @@ +package handler + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/qianfree/team-api/relay/common" + "github.com/qianfree/team-api/relay/constant" +) + +// 端到端集成测试:阿里 DashScope 官方协议 / OpenAI Videos 协议 → 转换 → 任务提交管线 +// (ali 适配器 + mock 上游)→ task_ 前缀 ID + DashScope 官方响应形态 + 落库 platform=ali +// + 上游收到 media 数组与映射参数。 + +// newAliUpstream 构造 mock 阿里 DashScope 上游(校验端点/异步头,记录请求体) +func newAliUpstream(t *testing.T, received *map[string]any) *httptest.Server { + t.Helper() + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if !strings.HasSuffix(r.URL.Path, "/api/v1/services/aigc/video-generation/video-synthesis") { + t.Errorf("unexpected upstream path: %s", r.URL.Path) + } + if r.Header.Get("X-DashScope-Async") != "enable" { + t.Errorf("missing X-DashScope-Async header: %v", r.Header) + } + body, _ := io.ReadAll(r.Body) + if err := json.Unmarshal(body, received); err != nil { + t.Errorf("upstream body not json: %v", err) + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]any{ + "output": map[string]any{"task_status": "PENDING", "task_id": "ali-up-1"}, + "request_id": "up-req-1", + }) + })) +} + +func newAliChannelMeta(upstreamURL string) *common.ChannelMeta { + return &common.ChannelMeta{ + ChannelID: 21, + ChannelType: int(constant.ProviderAli), + ChannelName: "mock-ali", + BaseURL: upstreamURL, + ApiKey: "sk-test", + UpstreamModelName: "wan3.0-video", + } +} + +func TestHandleTaskSubmit_AliNativeProtocol_FullPipeline(t *testing.T) { + var upstreamReceived map[string]any + upstream := newAliUpstream(t, &upstreamReceived) + defer upstream.Close() + + // 官方请求体 → 归一 → 通用任务体(与胶水层 HandleAliVideoSubmit 相同链路) + raw := `{ + "model": "wan3.0-video", + "input": { + "prompt": "小猫奔跑", + "media": [{"type": "first_frame", "url": "https://example.com/f.png"}] + }, + "parameters": {"resolution": "480P", "ratio": "16:9", "duration": 5, "audio": false} + }` + req, taskErr := ParseAliVideoCreateRequest([]byte(raw)) + if taskErr != nil { + t.Fatalf("parse: %+v", taskErr) + } + taskBody, err := BuildAliVideoTaskBody(req) + if err != nil { + t.Fatalf("build task body: %v", err) + } + + rec := httptest.NewRecorder() + rc := &TaskRelayContext{ + TenantID: 1, + UserID: 2, + ApiKeyID: 3, + RequestID: "req_ali_test", + Writer: rec, + Protocol: aliVideoProtocol, + RelayMode: int(constant.RelayModeAliVideo), + } + data := &stubVideosSubmitProvider{} + + HandleTaskSubmit(context.Background(), taskBody, "/api/v1/services/aigc/video-generation/video-synthesis", + http.Header{}, rc, data, &stubVideosBillingProvider{}, newAliChannelMeta(upstream.URL)) + + // 客户端响应:DashScope 官方形态 + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d, body=%s", rec.Code, rec.Body.String()) + } + resp := decodeVideosJSON(t, rec) + output, _ := resp["output"].(map[string]any) + if output == nil || output["task_status"] != "PENDING" { + t.Fatalf("unexpected dashscope output: %s", rec.Body.String()) + } + taskID, _ := output["task_id"].(string) + if !strings.HasPrefix(taskID, "task_") { + t.Fatalf("expected task_ prefixed id, got %q", taskID) + } + if resp["request_id"] != "req_ali_test" { + t.Fatalf("expected platform request id: %s", rec.Body.String()) + } + if rc.TaskID != taskID { + t.Fatalf("rc.TaskID=%q should match response id %q", rc.TaskID, taskID) + } + + // 落库任务:ali 平台 + 上游任务 ID + if data.created == nil || data.created.Platform != "ali" { + t.Fatalf("expected ali task record, got %+v", data.created) + } + var pd struct { + UpstreamTaskID string `json:"upstream_task_id"` + } + if err := json.Unmarshal(data.created.PrivateData, &pd); err != nil { + t.Fatalf("private data unmarshal: %v", err) + } + if pd.UpstreamTaskID != "ali-up-1" { + t.Fatalf("expected upstream task id captured, got %q", pd.UpstreamTaskID) + } + + // 上游收到:media 数组透传 + parameters 映射(resolution/ratio/duration/audio) + input, _ := upstreamReceived["input"].(map[string]any) + if input == nil || input["prompt"] != "小猫奔跑" { + t.Fatalf("upstream input: %v", upstreamReceived["input"]) + } + media, _ := input["media"].([]any) + if len(media) != 1 { + t.Fatalf("expected media array forwarded, got %v", input["media"]) + } + first, _ := media[0].(map[string]any) + if first["type"] != "first_frame" || first["url"] != "https://example.com/f.png" { + t.Fatalf("unexpected media item: %v", media[0]) + } + params, _ := upstreamReceived["parameters"].(map[string]any) + if params == nil || params["resolution"] != "480P" || params["ratio"] != "16:9" || + params["duration"] != float64(5) || params["audio"] != false { + t.Fatalf("upstream parameters: %v", upstreamReceived["parameters"]) + } +} + +func TestHandleTaskSubmit_OpenAIVideosToWan3_Pipeline(t *testing.T) { + // OpenAI Videos 协议(multipart)→ BuildVideosTaskBody → ali 适配器: + // input_reference → media[{type:first_frame}]、seconds → duration、size 短边 → resolution、 + // sound → audio + var upstreamReceived map[string]any + upstream := newAliUpstream(t, &upstreamReceived) + defer upstream.Close() + + mBody := []byte(`{"model":"wan3.0-video","prompt":"小猫对着镜头打哈欠","seconds":"8","size":"1280x720","aspect_ratio":"9:16","sound":"on","input_reference":{"image_url":"https://example.com/ref.png"}}`) + req, taskErr := ParseVideosCreateRequest(mBody, "application/json") + if taskErr != nil { + t.Fatalf("parse: %+v", taskErr) + } + taskBody, err := BuildVideosTaskBody(req) + if err != nil { + t.Fatalf("build task body: %v", err) + } + + rec := httptest.NewRecorder() + rc := &TaskRelayContext{ + TenantID: 1, + UserID: 2, + ApiKeyID: 3, + RequestID: "req_ali_openai_test", + Writer: rec, + Protocol: aliVideoProtocol, + RelayMode: int(constant.RelayModeAliVideo), + } + data := &stubVideosSubmitProvider{} + + HandleTaskSubmit(context.Background(), taskBody, "/api/v1/services/aigc/video-generation/video-synthesis", + http.Header{}, rc, data, &stubVideosBillingProvider{}, newAliChannelMeta(upstream.URL)) + + if rec.Code != http.StatusOK { + t.Fatalf("expected 200, got %d, body=%s", rec.Code, rec.Body.String()) + } + if data.created == nil || data.created.Platform != "ali" { + t.Fatalf("expected ali task record, got %+v", data.created) + } + + // 上游收到映射后的 wan3.0 参数 + input, _ := upstreamReceived["input"].(map[string]any) + media, _ := input["media"].([]any) + if len(media) != 1 { + t.Fatalf("expected first_frame media from input_reference, got %v", input["media"]) + } + first, _ := media[0].(map[string]any) + if first["type"] != "first_frame" { + t.Fatalf("input_reference should map to first_frame: %v", media[0]) + } + params, _ := upstreamReceived["parameters"].(map[string]any) + if params == nil || params["duration"] != float64(8) { + t.Fatalf("seconds → duration mapping lost: %v", upstreamReceived["parameters"]) + } + if params["resolution"] != "720P" { + t.Fatalf("size short-edge → resolution mapping lost: %v", upstreamReceived["parameters"]) + } + if params["ratio"] != "9:16" { + t.Fatalf("aspect_ratio → ratio mapping lost: %v", upstreamReceived["parameters"]) + } + if params["audio"] != true { + t.Fatalf("sound → audio mapping lost: %v", upstreamReceived["parameters"]) + } +} diff --git a/relay/handler/task_handler.go b/relay/handler/task_handler.go index edb5401..450402f 100644 --- a/relay/handler/task_handler.go +++ b/relay/handler/task_handler.go @@ -369,7 +369,8 @@ func HandleTaskSubmit( dbgAttempt.MarkFinal(nil) // 11. 返回响应(OpenAI Videos 协议返回官方 Video 对象;MiniMax 官方协议返回 {"task_id"} - //(v1 含 base_resp 信封);legacy 保持原格式) + //(v1 含 base_resp 信封);阿里 DashScope 官方协议返回 {"output":{"task_id":...}}; + // legacy 保持原格式) if rc.Protocol == videosProtocolOpenAI { writeVideosSubmitResponse(rc.Writer, publicTaskID, modelName, now, rc.RequestEcho) return @@ -382,6 +383,10 @@ func HandleTaskSubmit( writeMiniMaxV1SubmitResponse(rc.Writer, publicTaskID) return } + if rc.Protocol == aliVideoProtocol { + writeAliVideoSubmitResponse(rc.Writer, publicTaskID, rc.RequestID, now) + return + } respBody := map[string]any{ "id": publicTaskID, "status": "SUBMITTED", diff --git a/relay/taskchannel/ali/adaptor.go b/relay/taskchannel/ali/adaptor.go index ed50fe8..414a746 100644 --- a/relay/taskchannel/ali/adaptor.go +++ b/relay/taskchannel/ali/adaptor.go @@ -6,6 +6,7 @@ import ( "fmt" "io" "net/http" + "strconv" "strings" "github.com/qianfree/team-api/relay/common" @@ -21,6 +22,10 @@ func init() { // ==================== 请求/响应结构体 ==================== +// wan3MaxDurationSeconds wan3.0 智能时长模式(duration=-1)的输出上限(秒)。 +// 智能时长下输出长度不可预知,预扣按该上限冻结,结算按上游 usage.output_video_duration 多退少补 +const wan3MaxDurationSeconds = 30.0 + // dashScopeVideoRequest DashScope 视频生成请求 type dashScopeVideoRequest struct { Model string `json:"model"` @@ -29,9 +34,18 @@ type dashScopeVideoRequest struct { } type dashScopeVideoInput struct { - Prompt string `json:"prompt"` + Prompt string `json:"prompt,omitempty"` NegativePrompt string `json:"negative_prompt,omitempty"` AudioURL string `json:"audio_url,omitempty"` + // Media wan3.0 全能参考模式的媒体素材数组(first_frame/last_frame/reference_image/ + // reference_video/reference_audio/file/link);wan2.x 旧格式不填 + Media []dashScopeMedia `json:"media,omitempty"` +} + +// dashScopeMedia wan3.0 input.media 数组元素 +type dashScopeMedia struct { + Type string `json:"type"` + URL string `json:"url"` } type dashScopeVideoParams struct { @@ -42,6 +56,8 @@ type dashScopeVideoParams struct { PromptExtend *bool `json:"prompt_extend,omitempty"` Watermark *bool `json:"watermark,omitempty"` Seed *int `json:"seed,omitempty"` + // Audio wan3.0 输出视频是否包含音频(开关价格相同) + Audio *bool `json:"audio,omitempty"` } // aliMetadata Ali DashScope metadata 参数结构体(用于 UnmarshalMetadata 映射) @@ -55,6 +71,7 @@ type aliMetadata struct { PromptExtend *bool `json:"prompt_extend,omitempty"` Watermark *bool `json:"watermark,omitempty"` Seed *int `json:"seed,omitempty"` + // Audio wan3.0 输出音频开关("on"/"off" 通用词汇由 buildVideoRequest 归一,此处不承载) } // dashScopeImageRequest DashScope 图片生成请求 @@ -95,7 +112,7 @@ type dashScopeTaskResponse struct { Usage struct { Duration float64 `json:"duration"` VideoCount int `json:"video_count"` - OutputVideoDuration int `json:"output_video_duration"` + OutputVideoDuration float64 `json:"output_video_duration"` SR int `json:"SR"` Ratio string `json:"ratio"` ImageCount int `json:"image_count"` @@ -119,14 +136,14 @@ func (a *AliAdaptor) Init(info *common.RelayInfo) { } // detectVideo 根据模型名判断是否为视频生成。 -// 注意:万相图片模型(-t2i 文生图、-i2i 图像编辑)即使以 wan2. 开头也不是视频,必须先排除, +// 注意:万相图片模型(-t2i 文生图、-i2i 图像编辑)即使以 wan2./wan3. 开头也不是视频,必须先排除, // 否则会被误判为视频而 POST 到 video-synthesis 端点,被上游以 "url error" 拒绝。 func (a *AliAdaptor) detectVideo(modelName string) bool { m := strings.ToLower(modelName) if strings.Contains(m, "-t2i") || strings.Contains(m, "-i2i") { return false } - return strings.HasPrefix(m, "wan2.") || strings.HasPrefix(m, "wanx2.1-t2v") + return strings.HasPrefix(m, "wan2.") || strings.HasPrefix(m, "wan3.") || strings.HasPrefix(m, "wanx2.1-t2v") } func (a *AliAdaptor) ValidateRequest(_ context.Context, _ *common.RelayInfo, body []byte) *common.TaskError { @@ -140,6 +157,10 @@ func (a *AliAdaptor) ValidateRequest(_ context.Context, _ *common.RelayInfo, bod return nil } +// EstimateBilling 估算任务费用(提交前)。 +// 注意:此处自行从 body 提取模型名判定视频/图片,不依赖 BuildRequestBody 的 a.isVideo 副作用—— +// 管线中 EstimateBilling(第 6 步)先于 BuildRequestBody(第 7 步)执行,旧实现读 a.isVideo +// 恒为 false,视频时长信号从未上报(预扣恒按引擎默认 5s 估),本次修复。 func (a *AliAdaptor) EstimateBilling(_ context.Context, _ *common.RelayInfo, body []byte) map[string]any { ratios := map[string]any{"base": 1.0} var req map[string]any @@ -147,16 +168,29 @@ func (a *AliAdaptor) EstimateBilling(_ context.Context, _ *common.RelayInfo, bod return ratios } - if a.isVideo { - // duration 影响计费:spec.duration 为真实秒数(per_second 用),duration 为存量信号 + modelName, _ := req["model"].(string) + if a.detectVideo(modelName) { metadata := taskchannel.ExtractMetadata(req) - var meta struct { - Duration *int `json:"duration"` - } - if taskchannel.UnmarshalMetadata(metadata, &meta) == nil && meta.Duration != nil && *meta.Duration > 0 { - ratios["duration"] = float64(*meta.Duration) - ratios["spec.duration"] = float64(*meta.Duration) + var meta aliMetadata + if taskchannel.UnmarshalMetadata(metadata, &meta) == nil { + // 分辨率事实值:per_second 矩阵查价键(480P/720P/1080P 原值) + if meta.Resolution != "" { + ratios["spec.resolution"] = meta.Resolution + } + // 时长事实值:duration>0 用请求值;duration=-1(wan3.0 智能时长)按输出上限冻结, + // 结算以 usage.output_video_duration 多退少补 + if meta.Duration != nil { + d := float64(*meta.Duration) + if d == -1 { + d = wan3MaxDurationSeconds + } + if d > 0 { + ratios["duration"] = d + ratios["spec.duration"] = d + } + } } + // 顶层 seconds(OpenAI Videos 通用词汇)优先于 metadata if seconds, ok := req["seconds"].(string); ok { if d, err := parseInt(seconds); err == nil && d > 0 { ratios["duration"] = float64(d) @@ -250,8 +284,28 @@ func (a *AliAdaptor) buildVideoRequest(info *common.RelayInfo, req map[string]an } } + // wan3.0 专属适配:媒体素材数组 + 音频开关 + size 档位映射。 + // wan2.x 保持旧格式不动(其 size 是原生尺寸语义,不可参与档位映射) + if strings.HasPrefix(strings.ToLower(modelName), "wan3.") { + dsReq.Input.Media = buildWan3Media(req, metadata) + if sound, ok := metadata["sound"].(string); ok && sound != "" { + audio := sound == "on" + params.Audio = &audio + } + if params.Resolution == "" && meta.Size != "" { + params.Resolution = resolutionFromSize(meta.Size) + } + // ratio 兜底:通用词汇 aspect_ratio(OpenAI Videos 双写键,部分链路只写该键) + if params.Ratio == "" { + if ar, ok := metadata["aspect_ratio"].(string); ok { + params.Ratio = ar + } + } + } + if params.Resolution != "" || params.Ratio != "" || params.Size != "" || - params.Duration != nil || params.PromptExtend != nil || params.Watermark != nil || params.Seed != nil { + params.Duration != nil || params.PromptExtend != nil || params.Watermark != nil || + params.Seed != nil || params.Audio != nil { dsReq.Parameters = params } @@ -262,6 +316,71 @@ func (a *AliAdaptor) buildVideoRequest(info *common.RelayInfo, req map[string]an return strings.NewReader(string(data)), nil } +// buildWan3Media 构建 wan3.0 input.media 数组(全能参考模式)。 +// 优先级:顶层 media(阿里原生入站原样透传)> metadata.image(OpenAI input_reference, +// 首帧语义对齐 Sora)> 顶层 images[](通用多图词汇 → 参考图)。均无则返回 nil(纯文生视频) +func buildWan3Media(req map[string]any, metadata map[string]any) []dashScopeMedia { + if raw, ok := req["media"].([]any); ok && len(raw) > 0 { + media := make([]dashScopeMedia, 0, len(raw)) + for _, item := range raw { + m, ok := item.(map[string]any) + if !ok { + continue + } + t, _ := m["type"].(string) + u, _ := m["url"].(string) + if t == "" || u == "" { + continue + } + media = append(media, dashScopeMedia{Type: t, URL: u}) + } + if len(media) > 0 { + return media + } + } + if img, ok := metadata["image"].(string); ok && img != "" { + return []dashScopeMedia{{Type: "first_frame", URL: img}} + } + if imgs, ok := req["images"].([]any); ok && len(imgs) > 0 { + media := make([]dashScopeMedia, 0, len(imgs)) + for _, item := range imgs { + if u, ok := item.(string); ok && u != "" { + media = append(media, dashScopeMedia{Type: "reference_image", URL: u}) + } + } + if len(media) > 0 { + return media + } + } + return nil +} + +// resolutionFromSize 从尺寸字符串(如 "1280x720")按短边映射 wan3.0 分辨率档位。 +// 竖版尺寸(720x1280)同样按短边归档;解析失败返回空串(上游取默认 1080P) +func resolutionFromSize(size string) string { + parts := strings.SplitN(strings.ToLower(strings.TrimSpace(size)), "x", 2) + if len(parts) != 2 { + return "" + } + w, errW := strconv.Atoi(strings.TrimSpace(parts[0])) + h, errH := strconv.Atoi(strings.TrimSpace(parts[1])) + if errW != nil || errH != nil || w <= 0 || h <= 0 { + return "" + } + short := w + if h < short { + short = h + } + switch { + case short <= 500: + return "480P" + case short <= 800: + return "720P" + default: + return "1080P" + } +} + func (a *AliAdaptor) buildImageRequest(info *common.RelayInfo, req map[string]any, modelName string) (io.Reader, error) { dsReq := dashScopeImageRequest{ Input: dashScopeImageInput{}, @@ -409,6 +528,14 @@ func (a *AliAdaptor) ParseTaskResult(body []byte) (*common.TaskInfo, error) { if resp.Output.VideoURL != "" { info.ResultURL = resp.Output.VideoURL } + // 素材计量(wan3.0 按秒计费):实际输出秒数 + 实际分辨率, + // 供 generic per_second 结算按「实际秒数 × 实际档位单价」多退少补(方案 A) + if resp.Usage.OutputVideoDuration > 0 { + info.MaterialUsage = &common.TaskMaterialUsage{ + OutputSeconds: resp.Usage.OutputVideoDuration, + Resolution: resolutionFromSR(resp.Usage.SR), + } + } // 图片结果 if len(resp.Output.Results) > 0 { info.ResultURL = resp.Output.Results[0].URL @@ -448,3 +575,12 @@ func parseInt(s string) (int, error) { _, err := fmt.Sscanf(s, "%d", &n) return n, err } + +// resolutionFromSR 上游 usage.SR(分辨率短边整数,如 720)→ per_second 矩阵键("720P")。 +// 0/异常值返回空串(结算查价回退 ratios 的 spec.resolution) +func resolutionFromSR(sr int) string { + if sr <= 0 { + return "" + } + return strconv.Itoa(sr) + "P" +} diff --git a/relay/taskchannel/ali/adaptor_test.go b/relay/taskchannel/ali/adaptor_test.go new file mode 100644 index 0000000..a392fbc --- /dev/null +++ b/relay/taskchannel/ali/adaptor_test.go @@ -0,0 +1,254 @@ +package ali + +import ( + "context" + "encoding/json" + "testing" + + "github.com/qianfree/team-api/relay/common" +) + +func newTestRelayInfo() *common.RelayInfo { + // ChannelMeta 必须非空(BuildRequestBody 读取 IsModelMapped/UpstreamModelName) + return &common.RelayInfo{ChannelMeta: &common.ChannelMeta{}} +} + +func TestDetectVideo(t *testing.T) { + a := &AliAdaptor{} + video := []string{ + "wan3.0-video", "wan3.0-video-prime", "WAN3.0-VIDEO", + "wan2.7-t2v-2026-04-25", "wan2.2-t2v-plus", "wanx2.1-t2v-turbo", + } + for _, m := range video { + if !a.detectVideo(m) { + t.Errorf("%s should be detected as video", m) + } + } + image := []string{ + "wanx2.1-t2i-ediff", "wanx2.5-image-edit", "wanx-v1", + "qwen-image-generation", "flux-dev", + } + for _, m := range image { + if a.detectVideo(m) { + t.Errorf("%s should be detected as image", m) + } + } +} + +// 回归:EstimateBilling 不再依赖 BuildRequestBody 的 isVideo 副作用(管线中前者先于后者执行, +// 旧实现视频时长信号从未上报),此处直接对原始 body 求值。 +func TestEstimateBilling_VideoSpecSignals(t *testing.T) { + a := &AliAdaptor{} + + // wan3.0 + metadata duration/resolution → spec.* 上报 + ratios := a.EstimateBilling(context.Background(), newTestRelayInfo(), []byte( + `{"model":"wan3.0-video","prompt":"p","metadata":{"duration":8,"resolution":"720P"}}`)) + if ratios["spec.duration"] != 8.0 || ratios["duration"] != 8.0 { + t.Fatalf("spec.duration signal lost: %v", ratios) + } + if ratios["spec.resolution"] != "720P" { + t.Fatalf("spec.resolution signal lost: %v", ratios) + } + + // 智能时长 -1:按 wan3.0 上限 30s 冻结(结算按实际秒数多退少补) + ratios = a.EstimateBilling(context.Background(), newTestRelayInfo(), []byte( + `{"model":"wan3.0-video","prompt":"p","metadata":{"duration":-1,"resolution":"1080P"}}`)) + if ratios["spec.duration"] != 30.0 { + t.Fatalf("smart duration should freeze at 30s: %v", ratios) + } + if ratios["spec.resolution"] != "1080P" { + t.Fatalf("spec.resolution signal lost: %v", ratios) + } + + // 顶层 seconds(OpenAI Videos 通用词汇)优先于 metadata + ratios = a.EstimateBilling(context.Background(), newTestRelayInfo(), []byte( + `{"model":"wan3.0-video","prompt":"p","seconds":"12","metadata":{"duration":5}}`)) + if ratios["spec.duration"] != 12.0 { + t.Fatalf("seconds should override metadata duration: %v", ratios) + } + + // 未传时长:不产出 spec.duration(引擎按默认 5s 估) + ratios = a.EstimateBilling(context.Background(), newTestRelayInfo(), []byte( + `{"model":"wan3.0-video","prompt":"p"}`)) + if _, has := ratios["spec.duration"]; has { + t.Fatalf("no duration signal expected: %v", ratios) + } + + // 图片模型:n>1 产出 count,不产出时长/分辨率信号 + ratios = a.EstimateBilling(context.Background(), newTestRelayInfo(), []byte( + `{"model":"wanx-v1","prompt":"p","n":4}`)) + if ratios["count"] != 4.0 { + t.Fatalf("image count signal lost: %v", ratios) + } + if _, has := ratios["spec.duration"]; has { + t.Fatalf("image model must not carry duration signal: %v", ratios) + } +} + +func TestBuildVideoRequest_Wan3NativeMedia(t *testing.T) { + a := &AliAdaptor{} + body := []byte(`{ + "model": "wan3.0-video", + "prompt": "p", + "media": [ + {"type": "reference_video", "url": "https://v.mp4"}, + {"type": "reference_audio", "url": "https://a.mp3"} + ], + "metadata": {"resolution": "480P", "ratio": "16:9", "duration": 5, "sound": "off"} + }`) + r, err := a.BuildRequestBody(context.Background(), newTestRelayInfo(), body) + if err != nil { + t.Fatalf("build: %v", err) + } + var req dashScopeVideoRequest + if err := json.NewDecoder(r).Decode(&req); err != nil { + t.Fatalf("decode: %v", err) + } + if len(req.Input.Media) != 2 || req.Input.Media[0].Type != "reference_video" || req.Input.Media[1].URL != "https://a.mp3" { + t.Fatalf("media passthrough lost: %+v", req.Input.Media) + } + if req.Parameters == nil || req.Parameters.Resolution != "480P" || req.Parameters.Ratio != "16:9" { + t.Fatalf("parameters lost: %+v", req.Parameters) + } + if req.Parameters.Audio == nil || *req.Parameters.Audio { + t.Fatalf("sound=off should map to audio=false: %+v", req.Parameters.Audio) + } +} + +func TestBuildVideoRequest_OpenAIVocabulary(t *testing.T) { + a := &AliAdaptor{} + // OpenAI Videos 通用词汇:metadata.image → first_frame;size 短边 → resolution;sound → audio + body := []byte(`{ + "model": "wan3.0-video", + "prompt": "p", + "seconds": "8", + "images": ["https://x/1.png"], + "metadata": {"image": "https://x/ref.png", "size": "1280x720", "sound": "on", "aspect_ratio": "9:16"} + }`) + r, err := a.BuildRequestBody(context.Background(), newTestRelayInfo(), body) + if err != nil { + t.Fatalf("build: %v", err) + } + var req dashScopeVideoRequest + if err := json.NewDecoder(r).Decode(&req); err != nil { + t.Fatalf("decode: %v", err) + } + if len(req.Input.Media) != 1 || req.Input.Media[0].Type != "first_frame" || req.Input.Media[0].URL != "https://x/ref.png" { + t.Fatalf("metadata.image should map to first_frame: %+v", req.Input.Media) + } + if req.Parameters == nil || req.Parameters.Duration == nil || *req.Parameters.Duration != 8 { + t.Fatalf("seconds → duration lost: %+v", req.Parameters) + } + if req.Parameters.Resolution != "720P" { + t.Fatalf("size short-edge → resolution lost: %+v", req.Parameters) + } + if req.Parameters.Ratio != "9:16" { + t.Fatalf("aspect_ratio → ratio lost: %+v", req.Parameters) + } + if req.Parameters.Audio == nil || !*req.Parameters.Audio { + t.Fatalf("sound=on should map to audio=true: %+v", req.Parameters.Audio) + } +} + +func TestBuildVideoRequest_MultiImagesFallback(t *testing.T) { + a := &AliAdaptor{} + // 无 metadata.image 时,顶层 images[] → reference_image 数组 + body := []byte(`{"model":"wan3.0-video","prompt":"p","images":["https://x/1.png","https://x/2.png"]}`) + r, err := a.BuildRequestBody(context.Background(), newTestRelayInfo(), body) + if err != nil { + t.Fatalf("build: %v", err) + } + var req dashScopeVideoRequest + if err := json.NewDecoder(r).Decode(&req); err != nil { + t.Fatalf("decode: %v", err) + } + if len(req.Input.Media) != 2 || req.Input.Media[0].Type != "reference_image" || req.Input.Media[1].URL != "https://x/2.png" { + t.Fatalf("images fallback lost: %+v", req.Input.Media) + } +} + +// 回归:wan2.x 旧格式不引入 wan3.0 专属行为(无 media、无 audio、size 不派生档位) +func TestBuildVideoRequest_Wan2Unchanged(t *testing.T) { + a := &AliAdaptor{} + body := []byte(`{"model":"wan2.2-t2v-plus","prompt":"p","metadata":{"size":"1024x576","duration":5}}`) + r, err := a.BuildRequestBody(context.Background(), newTestRelayInfo(), body) + if err != nil { + t.Fatalf("build: %v", err) + } + var req dashScopeVideoRequest + if err := json.NewDecoder(r).Decode(&req); err != nil { + t.Fatalf("decode: %v", err) + } + if len(req.Input.Media) != 0 { + t.Fatalf("wan2 request must not carry media: %+v", req.Input.Media) + } + if req.Parameters != nil && req.Parameters.Audio != nil { + t.Fatalf("wan2 request must not carry audio: %+v", req.Parameters) + } + if req.Parameters == nil || req.Parameters.Resolution != "" { + t.Fatalf("wan2 size must not derive resolution: %+v", req.Parameters) + } + if req.Parameters.Size != "1024*576" { + t.Fatalf("wan2 size passthrough broken: %+v", req.Parameters) + } +} + +func TestParseTaskResult_Wan3MaterialUsage(t *testing.T) { + a := &AliAdaptor{} + body := []byte(`{ + "output": {"task_id": "t1", "task_status": "SUCCEEDED", "video_url": "https://v.mp4"}, + "usage": {"video_count": 1, "duration": 5.0, "output_video_duration": 5.13, "fps": 30, "SR": 720, "ratio": "16:9"} + }`) + info, err := a.ParseTaskResult(body) + if err != nil { + t.Fatalf("parse: %v", err) + } + if info.Status != common.TaskStatusSuccess || info.ResultURL != "https://v.mp4" { + t.Fatalf("unexpected info: %+v", info) + } + if info.MaterialUsage == nil { + t.Fatal("expected material usage for per_second settlement") + } + if info.MaterialUsage.OutputSeconds != 5.13 { + t.Fatalf("output seconds lost: %+v", info.MaterialUsage) + } + if info.MaterialUsage.Resolution != "720P" { + t.Fatalf("SR → resolution normalization broken: %+v", info.MaterialUsage) + } + + // usage 缺失(wan2.x 响应):MaterialUsage 为 nil,结算回退预扣口径 + info, err = a.ParseTaskResult([]byte(`{"output":{"task_id":"t2","task_status":"SUCCEEDED","video_url":"https://v2.mp4"}}`)) + if err != nil { + t.Fatalf("parse: %v", err) + } + if info.MaterialUsage != nil { + t.Fatalf("no usage → no material usage: %+v", info.MaterialUsage) + } + + // SR=0:Resolution 留空(结算查价回退 ratios 档位) + info, err = a.ParseTaskResult([]byte(`{"output":{"task_id":"t3","task_status":"SUCCEEDED","video_url":"u"},"usage":{"output_video_duration":5.0,"SR":0}}`)) + if err != nil { + t.Fatalf("parse: %v", err) + } + if info.MaterialUsage == nil || info.MaterialUsage.Resolution != "" { + t.Fatalf("zero SR should yield empty resolution: %+v", info.MaterialUsage) + } +} + +func TestResolutionFromSize(t *testing.T) { + cases := map[string]string{ + "1280x720": "720P", + "720x1280": "720P", // 竖版按短边 + "854x480": "480P", + "1920x1080": "1080P", + "1080x1920": "1080P", + "1024x1024": "1080P", // 正方形短边 1024 → 1080P + "invalid": "", + "ax720": "", + } + for size, want := range cases { + if got := resolutionFromSize(size); got != want { + t.Errorf("resolutionFromSize(%q) = %q, want %q", size, got, want) + } + } +} diff --git a/relay/taskchannel/ali/constants.go b/relay/taskchannel/ali/constants.go index d00161d..b10f9e6 100644 --- a/relay/taskchannel/ali/constants.go +++ b/relay/taskchannel/ali/constants.go @@ -5,6 +5,8 @@ const channelName = "Ali" // ModelList 阿里云 DashScope 支持异步生成的模型列表(图片 + 视频) var ModelList = []string{ // 视频生成 + "wan3.0-video", + "wan3.0-video-prime", "wan2.7-t2v-2026-04-25", "wan2.6-t2v", "wan2.6-t2v-us",