feat(asr): 支持讯飞实时语音转写(RTASR)提供商(#842) (#894)
- 新增 asr/xfyun.rs RTASR 流式客户端:signa = Base64(HmacSHA1(MD5(appid+ts), apiKey))
鉴权,1280B/40ms 分块发送,{"end": true} 收尾,seg_id 去重拼接,断连 partial 兜底,
鉴权/限流/超时错误分类可读。
- 全链路接入:dictation 主流程、QA 面板、重新转录、取消、设置页凭据预检与
"验证"探活;凭据 xfyun.app_id / xfyun.api_key 存系统凭据库,不落日志。
- 设置页新增「讯飞实时语音转写」选项(AppID + API Key),5 种语言 i18n。
- 文档:docs/xfyun-asr.md + README/README.zh/USAGE 同步。
已知限制:标准版 RTASR 无请求级热词参数(在讯飞控制台配置),语种默认中文普通话。
This commit is contained in:
1 parent
a3d014b687
commit
5be28b1dbd
23 files changed
+1269
-16
No files matched your search
@@ -159,7 +159,7 @@ The closest alternatives are subscription SaaS products: a monthly fee, no optio
|
||||
OpenLess targets the same end-user experience, but:
|
||||
|
||||
- **Fully open source, local-first.** The code lives in this repository, and all of your data stays on your machine.
|
||||
- **Bring your own cloud credentials.** Volcengine streaming ASR with Ark / DeepSeek-compatible chat completions. No vendor lock-in.
|
||||
- **Bring your own cloud credentials.** A dozen-plus ASR and polish providers — Volcengine, iFlytek, Alibaba Cloud Bailian, StepFun, OpenAI-compatible endpoints, and more — or any endpoint you bring. No vendor lock-in.
|
||||
- **Tuned for AI prompts.** The structured mode reshapes loose speech into a prompt complete with context, constraints, and requests — ready to paste into ChatGPT, Claude, or Cursor.
|
||||
- **It will not answer for you.** The model only cleans up your text. If you say "what features does this app still need?", it returns that as a clean question — it does not hand you a feature list. For that, ask the AI itself.
|
||||
|
||||
@@ -198,7 +198,7 @@ OpenLess does one thing: it **turns speech into usable written text — AI promp
|
||||
| [Lazy](https://heylazy.com) | Closed-source notes / capture tool | Not a notes container — inserts straight into any input field |
|
||||
| [Superwhisper](https://superwhisper.com) | Closed-source macOS, subscription | Open source; cloud ASR today, local ASR on the roadmap |
|
||||
|
||||
## Status (v1.3.6)
|
||||
## Status
|
||||
|
||||
Every item below is one more layer sedimented into a default — a capability you authorize once and then never manage again. This is the infrastructure you stand on after launch:
|
||||
|
||||
@@ -206,9 +206,9 @@ Every item below is one more layer sedimented into a default — a capability yo
|
||||
- 🎨 **Style Pack Marketplace** — browse, install, and like community **style packs** from the in-app Marketplace, and publish your own (custom system prompt per pack, switchable by hotkey). Backed by a moderated marketplace backend; uploads are reviewed before they go public.
|
||||
- ⚡ **Streaming insertion** — polished text is written to the cursor character by character to reduce perceived latency, with an automatic one-shot-paste fallback. Toggle in Settings → Recording.
|
||||
- **Toggle and push-to-talk** recording modes, plus a **MediaPlayPause trigger** so wired-earbud inline controls can start and stop recording. `Esc` cancels at any phase, including polish and insertion.
|
||||
- **Cloud ASR**: Volcengine streaming ASR, OpenAI Whisper-compatible batch ASR, Apple Speech (macOS).
|
||||
- **Local ASR**: bundled Qwen3-ASR (0.6B / 1.7B) via vendored `Open-Less/qwen-asr`; Windows Foundry Local Whisper variants.
|
||||
- **Polish providers**: Ark / DeepSeek / OpenAI / Doubao / Anthropic-compatible chat completions, plus any OpenAI-compatible endpoint you bring.
|
||||
- **Cloud ASR**: Volcengine streaming ASR (bigasr), iFlytek realtime ASR (RTASR), Alibaba Cloud Bailian (classic realtime / Qwen3 realtime / Fun-ASR-Flash file transcription), StepFun StepAudio (batch + realtime), Zhipu GLM-ASR, Xiaomi MiMo ASR, ElevenLabs Scribe, OpenAI-compatible batch transcription (OpenAI Whisper / Groq / SiliconFlow SenseVoice / OpenRouter), and Apple Speech (macOS).
|
||||
- **Local ASR**: bundled Qwen3-ASR (0.6B / 1.7B) via vendored `Open-Less/qwen-asr` (macOS); Windows Foundry Local Whisper and sherpa-onnx (experimental) variants.
|
||||
- **Polish providers**: Ark (Volcengine), DeepSeek, OpenAI, Google Gemini, Codex OAuth, SiliconFlow, Atlas Cloud, Xiaomi MiMo, CometAPI, OpenRouter, Alibaba Cloud Coding Plan, CodingPlanX, MiniMax, and StepFun — plus any OpenAI-compatible endpoint you bring.
|
||||
- **Four output modes**: raw, light polish, structured (**AI-prompt mode**), and formal. Plus a **translation hotkey** that converts speech directly into the configured target language ([#43](../../issues/43)).
|
||||
- **Selection-ask QA panel** — a separate hotkey opens a floating panel that runs voice Q&A against the highlighted text in any app ([#118](../../issues/118)).
|
||||
- **Main window**: Overview / History / Vocab / Style / Marketplace / Settings. Persistent tray icon, plus a mini status capsule that floats on screen and follows the display you are typing on (multi-monitor).
|
||||
@@ -337,6 +337,7 @@ New credential writes do not persist plaintext secrets. The repository contains
|
||||
You will need:
|
||||
|
||||
- **Volcengine streaming ASR**: APP ID, Access Token, Resource ID.
|
||||
- **iFlytek realtime ASR (RTASR)**: AppID, API Key. See [docs/xfyun-asr.md](docs/xfyun-asr.md).
|
||||
- **Ark polish**: API Key, Model ID, Endpoint. The Ark default endpoint is `https://ark.cn-beijing.volces.com/api/v3/chat/completions`.
|
||||
|
||||
## Prompt-handling principles
|
||||
@@ -363,7 +364,7 @@ Long-term reference rewrites are stored as `raw → polished → rule` triples a
|
||||
The dictionary handles your proper nouns, product names, names of people, and new words. Today it supports:
|
||||
|
||||
- Manually adding the correct spelling, a category, and notes. You do not need to maintain misspellings or context hints.
|
||||
- Enabled entries are sent as Volcengine ASR `context.hotwords` so they are recognized correctly during transcription.
|
||||
- Enabled entries are sent to the ASR provider that supports hotwords (Volcengine `context.hotwords`, StepFun `hotwords`, Whisper-compatible `prompt`, Bailian vocabulary ID) so they are recognized correctly during transcription. iFlytek realtime ASR has no request-level hotword parameter — configure personalized hotwords in the iFlytek console instead.
|
||||
- Entries are also injected into the polish prompt: the model decides per sentence whether to substitute. If "Cloud" clearly refers to the AI product `Claude` in context, it is corrected; if it genuinely means cloud computing, it is left as is.
|
||||
- The app auto-learns candidate corrections such as `Claude`, `ChatGPT`, and `OpenLess` from your history and offers them later.
|
||||
|
||||
@@ -379,7 +380,7 @@ The active implementation is Tauri 2 (`openless-all/app/`). Releases are split i
|
||||
types.rs Pure value types: DictationSession, PolishMode, HotkeyBinding, errors
|
||||
hotkey.rs Global hotkey (CGEventTap on macOS, WH_KEYBOARD_LL on Windows, rdev on Linux)
|
||||
recorder.rs Mic → 16 kHz mono Int16 PCM, RMS callback
|
||||
asr/ Volcengine streaming ASR (WebSocket) + Whisper HTTP
|
||||
asr/ Streaming ASR clients (Volcengine / Bailian / Qwen3 / StepFun / iFlytek over WebSocket) + Whisper-compatible batch HTTP
|
||||
polish.rs OpenAI-compatible chat completions (Ark / DeepSeek / etc.)
|
||||
insertion.rs AX focused-element → clipboard + Cmd+V → copy-only fallback
|
||||
persistence.rs History / preferences / vocab JSON + platform credential vault
|
||||
@@ -398,7 +399,6 @@ See [CLAUDE.md](CLAUDE.md) for invariants and module-wiring rules.
|
||||
|
||||
Planned but not yet shipped:
|
||||
|
||||
- Dictation translation mode: hold a separate hotkey, speak in your language, insert in the target language ([#43](../../issues/43)).
|
||||
- Cross-session style memory: polish learns the user's tone over time ([#46](../../issues/46)).
|
||||
- Snippets (no UI or trigger logic yet).
|
||||
- History enhancements: copy button, search, re-polish, re-insert.
|
||||
|
||||
+7
-7
@@ -198,7 +198,7 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提
|
||||
| [Lazy](https://heylazy.com) | 闭源的笔记 / 速记工具 | 不是笔记容器——直接插入到任意输入框 |
|
||||
| [Superwhisper](https://superwhisper.com) | 闭源 macOS,订阅制 | 开源;目前云端 ASR,本地 ASR 在路线图中 |
|
||||
|
||||
## 当前状态(v1.3.6)
|
||||
## 当前状态
|
||||
|
||||
下面每一项,都是一层已经沉降为默认、你授权一次之后就不必再操心的能力——这就是开屏之后你所站立的基础设施:
|
||||
|
||||
@@ -206,9 +206,9 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提
|
||||
- 🎨 **风格包市场**——在应用内的 Marketplace 浏览、安装、点赞社区**风格包**,并发布自己的(每个包一套自定义系统提示词,可用快捷键切换)。由经过审核的市场后端支撑;上传内容公开前会经过审核。
|
||||
- ⚡ **流式插入**——润色后的文本逐字符写入光标以降低感知延迟,并带有自动的一次性粘贴回退。可在 设置 → 录音 中切换。
|
||||
- **切换式与按住说话(push-to-talk)** 两种录音模式,外加 **MediaPlayPause 触发**,让有线耳机的线控也能开始 / 停止录音。`Esc` 可在任意阶段取消,包括润色与插入。
|
||||
- **云端 ASR**:Volcengine 流式 ASR、OpenAI Whisper 兼容的批量 ASR、Apple Speech(macOS)。
|
||||
- **本地 ASR**:通过 vendored 的 `Open-Less/qwen-asr` 内置 Qwen3-ASR(0.6B / 1.7B);Windows 上的 Foundry Local Whisper 变体。
|
||||
- **润色提供方**:Ark / DeepSeek / OpenAI / Doubao / Anthropic 兼容的 chat completions,以及你自带的任意 OpenAI 兼容端点。
|
||||
- **云端 ASR**:Volcengine 流式 ASR(bigasr)、讯飞实时语音转写(RTASR)、阿里云百炼(经典实时 / Qwen3 实时 / Fun-ASR-Flash 录音文件)、阶跃星辰 StepAudio(批式 + 实时)、智谱 GLM-ASR、小米 MiMo ASR、ElevenLabs Scribe、OpenAI 兼容批量转写(OpenAI Whisper / Groq / 硅基流动 SenseVoice / OpenRouter),以及 Apple Speech(macOS)。
|
||||
- **本地 ASR**:通过 vendored 的 `Open-Less/qwen-asr` 内置 Qwen3-ASR(0.6B / 1.7B)(macOS);Windows 上的 Foundry Local Whisper 与 sherpa-onnx(实验性)变体。
|
||||
- **润色提供方**:Ark(火山方舟)、DeepSeek、OpenAI、Google Gemini、Codex OAuth、硅基流动、Atlas Cloud、小米 MiMo、CometAPI、OpenRouter、阿里云 Coding Plan、CodingPlanX、MiniMax、StepFun,以及你自带的任意 OpenAI 兼容端点。
|
||||
- **四种输出模式**:原文、轻度润色、结构化(**AI 提示词模式**)、正式。另有一个**翻译快捷键**,将语音直接转换为所配置的目标语言([#43](../../issues/43))。
|
||||
- **选区问答面板**——一个独立快捷键打开浮动面板,针对任意应用中被高亮选中的文本进行语音问答([#118](../../issues/118))。
|
||||
- **主窗口**:概览 / 历史 / 词典 / 风格 / 市场 / 设置。常驻托盘图标,以及一个浮于屏幕、并跟随你正在输入的显示器的迷你状态胶囊(多显示器)。
|
||||
@@ -218,7 +218,7 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提
|
||||
- **Beta 频道(可选加入)**——设置 → 关于 → 加入 Beta 频道,可下载最新预发布版本进行手动安装。Beta 版本绝不会自动推送给 Stable 用户(见[贡献流程](#贡献流程))。
|
||||
- **分发渠道**——从 [Releases](../../releases) 直接下载 DMG/EXE、Homebrew Cask(`brew install --cask openless`)、Windows 安装包。
|
||||
- **单实例锁**——防止两个 OpenLess 进程争抢同一个快捷键边沿。
|
||||
- 词典条目作为 Volcengine ASR 的 `context.hotwords` 注入,并在润色时作为语义提示;命中次数按会话累计。
|
||||
- 词典条目注入到支持热词的 ASR 提供方(Volcengine 的 `context.hotwords`、StepFun 的 `hotwords`、Whisper 兼容的 `prompt`、百炼的 vocabulary_id),并在润色时作为语义提示;命中次数按会话累计。讯飞实时语音转写标准版没有请求级热词参数,需在讯飞控制台配置个性化热词。
|
||||
- 平台原生全局快捷键:macOS 上为 CGEventTap,Windows 上为低级键盘钩子(`WH_KEYBOARD_LL`)。
|
||||
|
||||
## 下载与安装(终端用户)
|
||||
@@ -337,6 +337,7 @@ Windows: %APPDATA%\OpenLess\credentials.json
|
||||
你需要准备:
|
||||
|
||||
- **Volcengine 流式 ASR**:APP ID、Access Token、Resource ID。
|
||||
- **讯飞实时语音转写(RTASR)**:AppID、API Key。见 [docs/xfyun-asr.md](docs/xfyun-asr.md)。
|
||||
- **Ark 润色**:API Key、Model ID、Endpoint。Ark 默认端点为 `https://ark.cn-beijing.volces.com/api/v3/chat/completions`。
|
||||
|
||||
## 文本处理原则
|
||||
@@ -379,7 +380,7 @@ OpenLess 的润色模型只重塑文本。它不回答问题、不执行任务
|
||||
types.rs Pure value types: DictationSession, PolishMode, HotkeyBinding, errors
|
||||
hotkey.rs Global hotkey (CGEventTap on macOS, WH_KEYBOARD_LL on Windows, rdev on Linux)
|
||||
recorder.rs Mic → 16 kHz mono Int16 PCM, RMS callback
|
||||
asr/ Volcengine streaming ASR (WebSocket) + Whisper HTTP
|
||||
asr/ 流式 ASR 客户端(Volcengine / Bailian / Qwen3 / StepFun / iFlytek 走 WebSocket)+ Whisper 兼容批量 HTTP
|
||||
polish.rs OpenAI-compatible chat completions (Ark / DeepSeek / etc.)
|
||||
insertion.rs AX focused-element → clipboard + Cmd+V → copy-only fallback
|
||||
persistence.rs History / preferences / vocab JSON + platform credential vault
|
||||
@@ -398,7 +399,6 @@ commands.rs Tauri IPC surface
|
||||
|
||||
已规划但尚未发布:
|
||||
|
||||
- 听写翻译模式:按住一个独立快捷键,用你的语言说,插入为目标语言([#43](../../issues/43))。
|
||||
- 跨会话风格记忆:润色随时间学习用户的语气([#46](../../issues/46))。
|
||||
- 片段(Snippets,尚无 UI 或触发逻辑)。
|
||||
- 历史增强:复制按钮、搜索、重新润色、重新插入。
|
||||
|
||||
@@ -35,6 +35,8 @@
|
||||
不会配火山 ASR 的话,先看这篇图文引导:
|
||||
[OpenLess 火山 ASR 配置](docs/volcengine-setup.md)
|
||||
|
||||
用讯飞实时语音转写的话,看 [OpenLess 讯飞 ASR 配置](docs/xfyun-asr.md)(AppID + API Key,设置 → 语音提供商 → 讯飞实时语音转写)。
|
||||
|
||||
打开 OpenLess → **设置**,填入以下字段:
|
||||
|
||||
| 字段 | 说明 |
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
# OpenLess 讯飞 ASR 配置
|
||||
|
||||
OpenLess 支持讯飞开放平台的**实时语音转写(RTASR,标准版)**,用于把录音实时转写成文字。
|
||||
|
||||
## 所需服务
|
||||
|
||||
- 服务:讯飞开放平台 **实时语音转写(标准版)**
|
||||
- 接口文档:<https://www.xfyun.cn/doc/asr/rtasr/API.html>
|
||||
|
||||
> 说明:录音文件转写(IFASR)、实时语音转写大模型(202 方言 / 37 语种免切)目前
|
||||
> **未接入**。若你有这些需求,欢迎在 issue #842 留言。
|
||||
|
||||
## 凭证获取
|
||||
|
||||
1. 登录讯飞开放平台:<https://www.xfyun.cn/>
|
||||
2. 控制台 → 我的应用 → 创建应用,添加 **实时语音转写** 服务(需先在服务页领取免费额度或购买)。
|
||||
3. 在服务管理页查看并复制 **AppID** 与 **API Key**。
|
||||
4. 打开 OpenLess `Settings → Providers → ASR`,选择「讯飞实时语音转写」,填入 AppID 与 API Key。
|
||||
|
||||
## 音频约束
|
||||
|
||||
- 采样率 16 kHz、位深 16 bit、单声道 PCM(与 OpenLess 录音输出完全一致,无需转码)。
|
||||
- 支持中文普通话、中英混合与英文;方言/小语种需在讯飞控制台开通后由 `lang` 参数指定,
|
||||
首期 OpenLess 固定使用中文普通话(`lang=cn`)。
|
||||
- IP 白名单:若在讯飞控制台开启了 IP 白名单,请把运行 OpenLess 机器的公网 IP 加入,
|
||||
否则握手会返回 `10105 illegal access`。
|
||||
|
||||
## 已知限制
|
||||
|
||||
- 标准版 RTASR **没有请求参数级热词**:OpenLess 的词典热词不会传给讯飞,个性化热词
|
||||
请在讯飞控制台上传。
|
||||
- 实时出字由服务端驱动:OpenLess 的胶囊 UI 在收尾阶段一次性展示最终文本(与其它
|
||||
流式 provider 行为一致)。
|
||||
|
||||
## 排障
|
||||
|
||||
- `10105` / `10110`:AppID / API Key 错误、IP 白名单未放行,或账号未开通实时语音转写服务。
|
||||
- `10800`:超过授权连接数(并发路数已满)。
|
||||
- `37005`:超过 15 秒未发送音频(弱网时偶发,可重试)。
|
||||
Generated
+12
@@ -3144,6 +3144,16 @@ version = "0.7.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0e7465ac9959cc2b1404e8e2367b43684a6d13790fe23056cc8c6c5a6b7bcb94"
|
||||
|
||||
[[package]]
|
||||
name = "md-5"
|
||||
version = "0.10.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d89e7ee0cfbedfc4da3340218492196241d89eefb6dab27de5df917a6d2e78cf"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"digest",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "memchr"
|
||||
version = "2.8.2"
|
||||
@@ -3941,6 +3951,7 @@ dependencies = [
|
||||
"libc",
|
||||
"local-ip-address",
|
||||
"log",
|
||||
"md-5",
|
||||
"minisign-verify",
|
||||
"ndk-context",
|
||||
"objc2 0.5.2",
|
||||
@@ -3954,6 +3965,7 @@ dependencies = [
|
||||
"rustls",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha1",
|
||||
"sha2",
|
||||
"sherpa-onnx",
|
||||
"simplelog",
|
||||
|
||||
@@ -29,6 +29,10 @@ serde_json = "1"
|
||||
# OpenRouter ASR 把音频以标准 base64(带 padding)放进 JSON body(issue #582)。
|
||||
base64 = "0.22"
|
||||
sha2 = "0.10"
|
||||
# 讯飞 RTASR/IFASR 签名:signa = Base64(HmacSHA1(MD5(appid + ts), apiKey))。
|
||||
md-5 = "0.10"
|
||||
# HmacSHA1 的摘要实现(signa 签名用);已在依赖树(传递依赖),提升为直接依赖零成本。
|
||||
sha1 = "0.10"
|
||||
# issue #609 F-03:history.json 完整性校验(HMAC-SHA256),密钥存系统凭据库。
|
||||
hmac = "0.12"
|
||||
# issue #609 H-02:HMAC 比较走常量时间,避免计时侧信道。已在依赖树(sha2/hmac 传递),
|
||||
|
||||
@@ -17,6 +17,7 @@ pub mod stepfun_realtime;
|
||||
pub mod volcengine;
|
||||
pub mod wav;
|
||||
pub mod whisper;
|
||||
pub mod xfyun;
|
||||
|
||||
pub use bailian::{BailianCredentials, BailianRealtimeASR};
|
||||
pub use dashscope_multimodal::DashScopeMultimodalASR;
|
||||
@@ -26,6 +27,7 @@ pub use qwen_realtime::{Qwen3RealtimeASR, Qwen3RealtimeCredentials};
|
||||
pub use stepfun_realtime::{StepfunRealtimeASR, StepfunRealtimeCredentials};
|
||||
pub use volcengine::{VolcengineCredentials, VolcengineStreamingASR};
|
||||
pub use whisper::WhisperBatchASR;
|
||||
pub use xfyun::{XfyunCredentials, XfyunStreamingASR};
|
||||
|
||||
/// Sink for raw 16 kHz / 16-bit / mono PCM bytes coming off the recorder.
|
||||
///
|
||||
|
||||
@@ -0,0 +1,863 @@
|
||||
//! iFlytek(讯飞开放平台)实时语音转写(RTASR)流式客户端。
|
||||
//!
|
||||
//! 官方文档:https://www.xfyun.cn/doc/asr/rtasr/API.html
|
||||
//!
|
||||
//! 协议要点(标准版):
|
||||
//! - 端点:`wss://rtasr.xfyun.cn/v1/ws`,鉴权走查询参数
|
||||
//! `appid` + `ts` + `signa`,其中 `signa = Base64(HmacSHA1(MD5(appid + ts), apiKey))`;
|
||||
//! - 音频:16 kHz / 16-bit / 单声道 PCM,与 OpenLess recorder 输出完全一致;
|
||||
//! - 建议每 40ms 发送 1280 字节,发送过快可能触发引擎报错;
|
||||
//! - 上传结束:发送二进制消息 `{"end": true}`;
|
||||
//! - 结果:服务端以 text message 返回 `{"action":"result","data":"<json 字符串>"}`,
|
||||
//! `data.cn.st.type` 为 `0`(最终结果)/ `1`(中间结果),全部结果发完后服务端断开连接。
|
||||
//!
|
||||
//! 已知限制:标准版 RTASR 没有请求参数级热词(个性化热词只能在讯飞控制台上传);
|
||||
//! 方言/小语种需在控制台开通后通过 `lang` 参数指定,首期固定中文普通话(`cn`)。
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use base64::Engine;
|
||||
use futures_util::{SinkExt, StreamExt};
|
||||
use hmac::{Hmac, Mac};
|
||||
use md5::{Digest as Md5Digest, Md5};
|
||||
use parking_lot::Mutex as ParkingMutex;
|
||||
use serde_json::Value;
|
||||
use sha1::Sha1;
|
||||
use tokio::net::TcpStream;
|
||||
use tokio::runtime::Handle;
|
||||
use tokio::sync::{mpsc, oneshot, Mutex as AsyncMutex, Notify};
|
||||
use tokio_tungstenite::tungstenite::client::IntoClientRequest;
|
||||
use tokio_tungstenite::tungstenite::Message;
|
||||
use tokio_tungstenite::{connect_async, MaybeTlsStream, WebSocketStream};
|
||||
|
||||
use super::{AudioConsumer, RawTranscript};
|
||||
|
||||
pub const PROVIDER_ID: &str = "iflytek";
|
||||
pub const DEFAULT_ENDPOINT: &str = "wss://rtasr.xfyun.cn/v1/ws";
|
||||
/// RTASR 文档建议:每 40ms 发送 1280 字节(16k/16-bit/mono = 32000 B/s)。
|
||||
pub const TARGET_AUDIO_CHUNK_BYTES: usize = 1_280;
|
||||
/// 16 kHz · 16-bit · mono = 32 000 bytes/sec → 32 bytes/ms。
|
||||
const BYTES_PER_MS: u64 = 32;
|
||||
const FINAL_RESULT_TIMEOUT: Duration = Duration::from_secs(12);
|
||||
/// WebSocket 建连(TCP + TLS + HTTP upgrade)上限,避免弱网下握手挂死热键线程。
|
||||
const CONNECT_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
/// 握手阶段等待 `action=started` 的上限。鉴权失败(10105/10110)应在此窗口内快速失败,
|
||||
/// 而不是把错误拖到收尾阶段才暴露。
|
||||
const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
/// 默认语种:中文普通话。方言/小语种需在讯飞控制台开通后传对应 `lang` 参数。
|
||||
const DEFAULT_LANG: &str = "cn";
|
||||
|
||||
type WsStream = WebSocketStream<MaybeTlsStream<TcpStream>>;
|
||||
type WsSink = futures_util::stream::SplitSink<WsStream, Message>;
|
||||
type SharedWriter = Arc<AsyncMutex<Option<WsSink>>>;
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct XfyunCredentials {
|
||||
/// 讯飞开放平台应用 ID。
|
||||
pub app_id: String,
|
||||
/// 实时语音转写服务对应的 APIKey(接口密钥)。
|
||||
pub api_key: String,
|
||||
}
|
||||
|
||||
impl XfyunCredentials {
|
||||
pub fn auth_ok(&self) -> bool {
|
||||
!self.app_id.trim().is_empty() && !self.api_key.trim().is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, thiserror::Error)]
|
||||
pub enum XfyunASRError {
|
||||
#[error("credentials missing")]
|
||||
CredentialsMissing,
|
||||
#[error("连接失败: {0}")]
|
||||
ConnectionFailed(String),
|
||||
/// 握手阶段服务端返回 10105 / 10110:AppID / APIKey 错误、IP 白名单未配置、
|
||||
/// 或账号未开通实时语音转写服务。
|
||||
#[error("凭据被拒或未开通服务({0})")]
|
||||
AuthRejected(String),
|
||||
/// 10800:超过授权连接数 / 并发受限。
|
||||
#[error("并发受限({0})")]
|
||||
RateLimited(String),
|
||||
#[error("识别失败: {0}")]
|
||||
TaskFailed(String),
|
||||
#[error("no final result")]
|
||||
NoFinalResult,
|
||||
#[error("final result timed out")]
|
||||
FinalResultTimeout,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct SyncState {
|
||||
pending_audio: Vec<u8>,
|
||||
bytes_sent: u64,
|
||||
started: bool,
|
||||
finished: bool,
|
||||
runtime: Option<Handle>,
|
||||
start: Option<Instant>,
|
||||
final_tx: Option<oneshot::Sender<Result<RawTranscript, XfyunASRError>>>,
|
||||
/// seg_id → 最终(type=0)分段文本。同一 seg_id 的后到结果覆盖前一个。
|
||||
final_segments: BTreeMap<i64, String>,
|
||||
/// seg_id → 最近一次中间(type=1)分段文本,服务端在 final 前断连时兜底用。
|
||||
partial_segments: BTreeMap<i64, String>,
|
||||
last_result_text: String,
|
||||
}
|
||||
|
||||
pub struct XfyunStreamingASR {
|
||||
credentials: XfyunCredentials,
|
||||
state: ParkingMutex<SyncState>,
|
||||
writer: SharedWriter,
|
||||
final_rx: ParkingMutex<Option<oneshot::Receiver<Result<RawTranscript, XfyunASRError>>>>,
|
||||
/// 握手结果通道:receive loop 收到 `action=started` 后发 Ok,收到 error 发 Err。
|
||||
/// `open_session` 等待它以在鉴权失败时快速失败。
|
||||
handshake_tx: ParkingMutex<Option<oneshot::Sender<Result<(), XfyunASRError>>>>,
|
||||
/// 音频发送队列:consume_pcm_chunk 入队,唯一 worker 串行 send,保证时序。
|
||||
audio_tx: ParkingMutex<Option<mpsc::UnboundedSender<Vec<u8>>>>,
|
||||
/// 队列里 + worker 在飞的 audio 帧总数。send_last_frame 必须等它归零再发
|
||||
/// `{"end": true}`,否则末帧先到、尾部音频被服务端当「end 之后的数据」丢弃。
|
||||
pending_sends: Arc<AtomicUsize>,
|
||||
send_done: Arc<Notify>,
|
||||
}
|
||||
|
||||
impl XfyunStreamingASR {
|
||||
pub fn new(credentials: XfyunCredentials) -> Self {
|
||||
Self {
|
||||
credentials,
|
||||
state: ParkingMutex::new(SyncState::default()),
|
||||
writer: Arc::new(AsyncMutex::new(None)),
|
||||
final_rx: ParkingMutex::new(None),
|
||||
handshake_tx: ParkingMutex::new(None),
|
||||
audio_tx: ParkingMutex::new(None),
|
||||
pending_sends: Arc::new(AtomicUsize::new(0)),
|
||||
send_done: Arc::new(Notify::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// 构建带鉴权参数的 WebSocket 地址:
|
||||
/// `wss://rtasr.xfyun.cn/v1/ws?appid=..&ts=..&signa=..&lang=cn`。
|
||||
pub fn connect_url(&self) -> String {
|
||||
connect_url(&self.credentials)
|
||||
}
|
||||
|
||||
pub async fn open_session(self: &Arc<Self>) -> Result<(), XfyunASRError> {
|
||||
if !self.credentials.auth_ok() {
|
||||
return Err(XfyunASRError::CredentialsMissing);
|
||||
}
|
||||
|
||||
let request = self
|
||||
.connect_url()
|
||||
.into_client_request()
|
||||
.map_err(|e| XfyunASRError::ConnectionFailed(e.to_string()))?;
|
||||
let (ws, _resp) = tokio::time::timeout(CONNECT_TIMEOUT, connect_async(request))
|
||||
.await
|
||||
.map_err(|_| {
|
||||
XfyunASRError::ConnectionFailed(format!(
|
||||
"连接超时({} ms)",
|
||||
CONNECT_TIMEOUT.as_millis()
|
||||
))
|
||||
})?
|
||||
.map_err(|e| XfyunASRError::ConnectionFailed(e.to_string()))?;
|
||||
let (write, read) = ws.split();
|
||||
*self.writer.lock().await = Some(write);
|
||||
|
||||
let (final_tx, final_rx) = oneshot::channel();
|
||||
let (audio_tx, mut audio_rx) = mpsc::unbounded_channel::<Vec<u8>>();
|
||||
let (handshake_tx, handshake_rx) = oneshot::channel();
|
||||
{
|
||||
let mut st = self.state.lock();
|
||||
*st = SyncState::default();
|
||||
st.runtime = Some(Handle::current());
|
||||
st.start = Some(Instant::now());
|
||||
st.final_tx = Some(final_tx);
|
||||
}
|
||||
*self.final_rx.lock() = Some(final_rx);
|
||||
*self.audio_tx.lock() = Some(audio_tx);
|
||||
*self.handshake_tx.lock() = Some(handshake_tx);
|
||||
self.pending_sends.store(0, Ordering::SeqCst);
|
||||
|
||||
// 音频 worker:FIFO recv + 串行 send_binary,保证 chunk 顺序。
|
||||
let writer_for_worker = Arc::clone(&self.writer);
|
||||
let pending_for_worker = Arc::clone(&self.pending_sends);
|
||||
let notify_for_worker = Arc::clone(&self.send_done);
|
||||
tokio::spawn(async move {
|
||||
while let Some(chunk) = audio_rx.recv().await {
|
||||
if let Err(e) = send_binary(&writer_for_worker, chunk).await {
|
||||
log::error!("[xfyun-asr] audio frame send failed: {e}");
|
||||
}
|
||||
if pending_for_worker.fetch_sub(1, Ordering::SeqCst) == 1 {
|
||||
notify_for_worker.notify_waiters();
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// receive loop:处理 started / result / error,以及服务端断开。
|
||||
let weak_self = Arc::downgrade(self);
|
||||
tokio::spawn(async move {
|
||||
let mut read = read;
|
||||
while let Some(msg) = read.next().await {
|
||||
let Some(this) = weak_self.upgrade() else {
|
||||
break;
|
||||
};
|
||||
match msg {
|
||||
Ok(Message::Text(text)) => {
|
||||
if !this.handle_text_message(&text) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Ok(Message::Close(_)) => {
|
||||
// 服务端在全部结果发完后主动断开;也覆盖 37005(15s 无音频)
|
||||
// 等中断场景 —— finish_on_close 会按已有内容兜底。
|
||||
this.finish_on_close();
|
||||
break;
|
||||
}
|
||||
Ok(_) => { /* ignore binary/ping/pong */ }
|
||||
Err(e) => {
|
||||
log::error!("[xfyun-asr] receive loop error: {e}");
|
||||
this.finish_with_partial_or_error(XfyunASRError::ConnectionFailed(
|
||||
e.to_string(),
|
||||
));
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// 等待握手结果:鉴权错误 / 连接被拒在这里快速失败,不等用户说完话。
|
||||
match tokio::time::timeout(HANDSHAKE_TIMEOUT, handshake_rx).await {
|
||||
Ok(Ok(Ok(()))) => Ok(()),
|
||||
Ok(Ok(Err(e))) => {
|
||||
self.cancel();
|
||||
Err(e)
|
||||
}
|
||||
Ok(Err(_)) => {
|
||||
self.cancel();
|
||||
Err(XfyunASRError::ConnectionFailed(
|
||||
"握手通道提前关闭".to_string(),
|
||||
))
|
||||
}
|
||||
Err(_) => {
|
||||
self.cancel();
|
||||
Err(XfyunASRError::ConnectionFailed(format!(
|
||||
"握手超时({} ms)",
|
||||
HANDSHAKE_TIMEOUT.as_millis()
|
||||
)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn send_last_frame(&self) -> Result<(), XfyunASRError> {
|
||||
// 等所有在途音频帧发完,再发 `{"end": true}`(上限 800ms 防极端网络下永远等)。
|
||||
let drain_deadline = Instant::now() + Duration::from_millis(800);
|
||||
while self.pending_sends.load(Ordering::SeqCst) > 0 {
|
||||
let remaining = drain_deadline.saturating_duration_since(Instant::now());
|
||||
if remaining.is_zero() {
|
||||
log::warn!(
|
||||
"[xfyun-asr] send_last_frame: pending {} 帧未发送完,超时强制继续",
|
||||
self.pending_sends.load(Ordering::SeqCst)
|
||||
);
|
||||
break;
|
||||
}
|
||||
let _ = tokio::time::timeout(remaining, self.send_done.notified()).await;
|
||||
}
|
||||
|
||||
// 冲刷尾部不足一块的残余音频。
|
||||
let leftover = {
|
||||
let mut st = self.state.lock();
|
||||
if st.pending_audio.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(std::mem::take(&mut st.pending_audio))
|
||||
}
|
||||
};
|
||||
if let Some(buf) = leftover {
|
||||
let len = buf.len() as u64;
|
||||
self.state.lock().bytes_sent += len;
|
||||
let Some(tx) = self.audio_tx.lock().as_ref().cloned() else {
|
||||
return Err(XfyunASRError::ConnectionFailed(
|
||||
"websocket not open".into(),
|
||||
));
|
||||
};
|
||||
self.pending_sends.fetch_add(1, Ordering::SeqCst);
|
||||
if tx.send(buf).is_err() {
|
||||
self.pending_sends.fetch_sub(1, Ordering::SeqCst);
|
||||
}
|
||||
}
|
||||
|
||||
// 等尾部音频也发完,再发结束标识(内容与文档一致,必须走 binary message)。
|
||||
let drain_deadline = Instant::now() + Duration::from_millis(800);
|
||||
while self.pending_sends.load(Ordering::SeqCst) > 0 {
|
||||
let remaining = drain_deadline.saturating_duration_since(Instant::now());
|
||||
if remaining.is_zero() {
|
||||
log::warn!(
|
||||
"[xfyun-asr] send_last_frame: tail {} 帧未发送完,超时强制结束",
|
||||
self.pending_sends.load(Ordering::SeqCst)
|
||||
);
|
||||
break;
|
||||
}
|
||||
let _ = tokio::time::timeout(remaining, self.send_done.notified()).await;
|
||||
}
|
||||
|
||||
send_binary(&self.writer, br#"{"end": true}"#.to_vec()).await
|
||||
}
|
||||
|
||||
pub async fn await_final_result(&self) -> Result<RawTranscript, XfyunASRError> {
|
||||
self.await_final_result_with_timeout(FINAL_RESULT_TIMEOUT)
|
||||
.await
|
||||
}
|
||||
|
||||
pub async fn await_final_result_with_timeout(
|
||||
&self,
|
||||
timeout: Duration,
|
||||
) -> Result<RawTranscript, XfyunASRError> {
|
||||
let rx = self.final_rx.lock().take();
|
||||
let Some(rx) = rx else {
|
||||
return Err(XfyunASRError::NoFinalResult);
|
||||
};
|
||||
match tokio::time::timeout(timeout, rx).await {
|
||||
Ok(Ok(result)) => result,
|
||||
Ok(Err(_)) => Err(XfyunASRError::NoFinalResult),
|
||||
Err(_) => {
|
||||
log::error!(
|
||||
"[xfyun-asr] final result timed out after {} ms",
|
||||
timeout.as_millis()
|
||||
);
|
||||
self.cancel();
|
||||
Err(XfyunASRError::FinalResultTimeout)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn cancel(&self) {
|
||||
let runtime = {
|
||||
let mut st = self.state.lock();
|
||||
st.pending_audio.clear();
|
||||
st.runtime.clone()
|
||||
};
|
||||
// 释放握手通道:open_session 若仍在等 started,会立刻收到 Err 返回。
|
||||
*self.handshake_tx.lock() = None;
|
||||
// 关闭音频队列 → worker 的 recv() 返回 None → 退出,不再 hold writer。
|
||||
*self.audio_tx.lock() = None;
|
||||
if let Some(runtime) = runtime {
|
||||
let writer = Arc::clone(&self.writer);
|
||||
runtime.spawn(async move {
|
||||
let mut guard = writer.lock().await;
|
||||
if let Some(mut ws) = guard.take() {
|
||||
let _ = ws.close().await;
|
||||
}
|
||||
});
|
||||
}
|
||||
self.signal_error(XfyunASRError::NoFinalResult);
|
||||
}
|
||||
|
||||
// ---- internals ----
|
||||
|
||||
fn handle_text_message(&self, text: &str) -> bool {
|
||||
let value: Value = match serde_json::from_str(text) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
log::warn!("[xfyun-asr] invalid json event: {e}");
|
||||
return true;
|
||||
}
|
||||
};
|
||||
let action = value
|
||||
.get("action")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or_default();
|
||||
match action {
|
||||
"started" => {
|
||||
self.mark_started();
|
||||
true
|
||||
}
|
||||
"result" => {
|
||||
self.record_result(&value);
|
||||
true
|
||||
}
|
||||
"error" => {
|
||||
let code = value
|
||||
.get("code")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
let desc = value
|
||||
.get("desc")
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
log::error!("[xfyun-asr] server error code={code} desc={desc}");
|
||||
let error = classify_server_error(&code, &desc);
|
||||
// 握手阶段就报错:把错误直接交给 open_session 的 handshake 等待方。
|
||||
let handed = self.handshake_tx.lock().take();
|
||||
if let Some(tx) = handed {
|
||||
let _ = tx.send(Err(error.clone()));
|
||||
}
|
||||
self.finish_error(error);
|
||||
false
|
||||
}
|
||||
_ => true,
|
||||
}
|
||||
}
|
||||
|
||||
fn mark_started(&self) {
|
||||
self.state.lock().started = true;
|
||||
let tx = self.handshake_tx.lock().take();
|
||||
if let Some(tx) = tx {
|
||||
let _ = tx.send(Ok(()));
|
||||
}
|
||||
}
|
||||
|
||||
fn record_result(&self, value: &Value) {
|
||||
let Some(data_str) = value.get("data").and_then(Value::as_str) else {
|
||||
return;
|
||||
};
|
||||
let Ok(data) = serde_json::from_str::<Value>(data_str) else {
|
||||
log::warn!("[xfyun-asr] result data is not valid JSON");
|
||||
return;
|
||||
};
|
||||
let Some(seg_id) = data.get("seg_id").and_then(Value::as_i64) else {
|
||||
return;
|
||||
};
|
||||
let Some(st) = data.get("cn").and_then(|c| c.get("st")) else {
|
||||
return;
|
||||
};
|
||||
// `type` 字段文档为字符串("0" 最终 / "1" 中间)。防御性兼容数字形态
|
||||
// (0/1):若服务端以数字返回而只认字符串,多句最终结果会被整体降级成
|
||||
// 中间结果,收尾 fallback 只剩最后一句 —— 前句丢失。
|
||||
let is_final = st
|
||||
.get("type")
|
||||
.and_then(|v| {
|
||||
v.as_str()
|
||||
.map(|s| s == "0")
|
||||
.or_else(|| v.as_i64().map(|n| n == 0))
|
||||
})
|
||||
.unwrap_or(false);
|
||||
let text = extract_words(st);
|
||||
let trimmed = text.trim();
|
||||
if trimmed.is_empty() {
|
||||
return;
|
||||
}
|
||||
|
||||
let mut state = self.state.lock();
|
||||
state.last_result_text = trimmed.to_string();
|
||||
if is_final {
|
||||
// 最终结果:以 seg_id 去重覆盖,收尾按 seg_id 顺序拼接。
|
||||
state
|
||||
.final_segments
|
||||
.insert(seg_id, trimmed.to_string());
|
||||
state.partial_segments.remove(&seg_id);
|
||||
} else {
|
||||
state
|
||||
.partial_segments
|
||||
.insert(seg_id, trimmed.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
/// 服务端断开连接:正常路径(全部结果已发完)或异常中断。
|
||||
/// 握手尚未完成就被关闭时,优先把错误交给 open_session 的等待方。
|
||||
fn finish_on_close(&self) {
|
||||
let handshake = self.handshake_tx.lock().take();
|
||||
if let Some(tx) = handshake {
|
||||
let _ = tx.send(Err(XfyunASRError::ConnectionFailed(
|
||||
"连接在握手完成前被关闭".to_string(),
|
||||
)));
|
||||
return;
|
||||
}
|
||||
self.finish_with_partial_or_error(XfyunASRError::NoFinalResult);
|
||||
}
|
||||
|
||||
/// 有已识别内容(最终或中间结果)就兜底返回,否则报错 —— 与 Volcengine / Bailian
|
||||
/// 的「服务端在 final 前断连不丢已识别文字」策略保持一致。
|
||||
fn finish_with_partial_or_error(&self, error: XfyunASRError) {
|
||||
let has_partial = {
|
||||
let st = self.state.lock();
|
||||
!st.last_result_text.trim().is_empty() || !st.partial_segments.is_empty()
|
||||
};
|
||||
if has_partial {
|
||||
self.finish_success();
|
||||
} else {
|
||||
self.finish_error(error);
|
||||
}
|
||||
}
|
||||
|
||||
fn finish_success(&self) {
|
||||
let (tx, text, duration_ms) = {
|
||||
let mut st = self.state.lock();
|
||||
if st.finished {
|
||||
return;
|
||||
}
|
||||
st.finished = true;
|
||||
st.pending_audio.clear();
|
||||
let text = if st.final_segments.is_empty() {
|
||||
st.last_result_text.clone()
|
||||
} else {
|
||||
let segments: Vec<String> = st.final_segments.values().cloned().collect();
|
||||
super::mimo::join_transcript_chunks(&segments)
|
||||
};
|
||||
let duration_ms = if st.bytes_sent > 0 {
|
||||
st.bytes_sent / BYTES_PER_MS
|
||||
} else {
|
||||
st.start
|
||||
.map(|start| start.elapsed().as_millis() as u64)
|
||||
.unwrap_or_default()
|
||||
};
|
||||
(st.final_tx.take(), text, duration_ms)
|
||||
};
|
||||
*self.audio_tx.lock() = None;
|
||||
if let Some(tx) = tx {
|
||||
let _ = tx.send(Ok(RawTranscript { text, duration_ms }));
|
||||
}
|
||||
self.close_writer();
|
||||
}
|
||||
|
||||
fn signal_error(&self, error: XfyunASRError) {
|
||||
let tx = {
|
||||
let mut st = self.state.lock();
|
||||
if st.finished {
|
||||
return;
|
||||
}
|
||||
st.finished = true;
|
||||
st.final_tx.take()
|
||||
};
|
||||
*self.audio_tx.lock() = None;
|
||||
if let Some(tx) = tx {
|
||||
let _ = tx.send(Err(error));
|
||||
}
|
||||
}
|
||||
|
||||
fn finish_error(&self, error: XfyunASRError) {
|
||||
// 握手尚未完成就出错(如 receive loop 网络中断、服务端 error 消息)时,
|
||||
// 必须把错误交给 open_session 的 handshake 等待方 —— 否则它只能空等
|
||||
// HANDSHAKE_TIMEOUT(5s)才返回。已过握手的会话这里 take 到 None,幂等跳过。
|
||||
let handshake = self.handshake_tx.lock().take();
|
||||
if let Some(tx) = handshake {
|
||||
let _ = tx.send(Err(error.clone()));
|
||||
}
|
||||
self.signal_error(error);
|
||||
self.close_writer();
|
||||
}
|
||||
|
||||
fn close_writer(&self) {
|
||||
let writer = Arc::clone(&self.writer);
|
||||
if let Some(handle) = self.state.lock().runtime.clone() {
|
||||
handle.spawn(async move {
|
||||
let mut guard = writer.lock().await;
|
||||
if let Some(mut ws) = guard.take() {
|
||||
let _ = ws.close().await;
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AudioConsumer for XfyunStreamingASR {
|
||||
fn consume_pcm_chunk(&self, pcm: &[u8]) {
|
||||
let chunks: Vec<Vec<u8>> = {
|
||||
let mut st = self.state.lock();
|
||||
if !st.started || st.finished {
|
||||
return;
|
||||
}
|
||||
st.pending_audio.extend_from_slice(pcm);
|
||||
let mut out = Vec::new();
|
||||
while st.pending_audio.len() >= TARGET_AUDIO_CHUNK_BYTES {
|
||||
let chunk: Vec<u8> = st
|
||||
.pending_audio
|
||||
.drain(..TARGET_AUDIO_CHUNK_BYTES)
|
||||
.collect();
|
||||
st.bytes_sent += chunk.len() as u64;
|
||||
out.push(chunk);
|
||||
}
|
||||
out
|
||||
};
|
||||
if chunks.is_empty() {
|
||||
return;
|
||||
}
|
||||
let Some(tx) = self.audio_tx.lock().as_ref().cloned() else {
|
||||
return;
|
||||
};
|
||||
for chunk in chunks {
|
||||
// pending_sends 必须先 +1 再入队:否则 worker 可能先 recv + 发送 + 减 1,
|
||||
// 把 usize 计数器 underflow。
|
||||
self.pending_sends.fetch_add(1, Ordering::SeqCst);
|
||||
if tx.send(chunk).is_err() {
|
||||
if self.pending_sends.fetch_sub(1, Ordering::SeqCst) == 1 {
|
||||
self.send_done.notify_waiters();
|
||||
}
|
||||
log::warn!("[xfyun-asr] audio queue closed; dropping subsequent frames");
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn send_binary(writer: &SharedWriter, data: Vec<u8>) -> Result<(), XfyunASRError> {
|
||||
let mut guard = writer.lock().await;
|
||||
let Some(ws) = guard.as_mut() else {
|
||||
return Err(XfyunASRError::ConnectionFailed(
|
||||
"websocket not open".to_string(),
|
||||
));
|
||||
};
|
||||
ws.send(Message::Binary(data))
|
||||
.await
|
||||
.map_err(|e| XfyunASRError::ConnectionFailed(e.to_string()))
|
||||
}
|
||||
|
||||
fn connect_url(credentials: &XfyunCredentials) -> String {
|
||||
let ts = chrono::Utc::now().timestamp().to_string();
|
||||
let signa = compute_signa(&credentials.app_id, &credentials.api_key, &ts);
|
||||
let mut url = url::Url::parse(DEFAULT_ENDPOINT).expect("static endpoint parses");
|
||||
url.query_pairs_mut()
|
||||
.append_pair("appid", credentials.app_id.trim())
|
||||
.append_pair("ts", &ts)
|
||||
.append_pair("signa", &signa)
|
||||
.append_pair("lang", DEFAULT_LANG);
|
||||
url.to_string()
|
||||
}
|
||||
|
||||
/// `signa = Base64(HmacSHA1(MD5(appid + ts), apiKey))`,与讯飞开放平台文档公式一致。
|
||||
pub fn compute_signa(app_id: &str, api_key: &str, ts: &str) -> String {
|
||||
let base = format!("{app_id}{ts}");
|
||||
let md5_hex = md5_hex(base.as_bytes());
|
||||
let mut mac = Hmac::<Sha1>::new_from_slice(api_key.trim().as_bytes())
|
||||
.expect("HMAC accepts any key length");
|
||||
mac.update(md5_hex.as_bytes());
|
||||
base64::engine::general_purpose::STANDARD.encode(mac.finalize().into_bytes())
|
||||
}
|
||||
|
||||
fn md5_hex(input: &[u8]) -> String {
|
||||
let digest = Md5::digest(input);
|
||||
digest
|
||||
.iter()
|
||||
.map(|byte| format!("{byte:02x}"))
|
||||
.collect::<String>()
|
||||
}
|
||||
|
||||
/// 从 `cn.st` 节点提取全部词:`rt[].ws[].cw[]` 取第一个候选的 `w` 依次拼接。
|
||||
fn extract_words(st: &Value) -> String {
|
||||
let mut out = String::new();
|
||||
let Some(rt) = st.get("rt").and_then(Value::as_array) else {
|
||||
return out;
|
||||
};
|
||||
for sentence in rt {
|
||||
let Some(ws) = sentence.get("ws").and_then(Value::as_array) else {
|
||||
continue;
|
||||
};
|
||||
for word in ws {
|
||||
let Some(cw) = word.get("cw").and_then(Value::as_array) else {
|
||||
continue;
|
||||
};
|
||||
for candidate in cw {
|
||||
if let Some(w) = candidate.get("w").and_then(Value::as_str) {
|
||||
out.push_str(w);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// 把讯飞错误码归类为对用户可读的类别:鉴权/授权问题与并发限制单独分类,
|
||||
/// 其余归通用识别失败(避免 capsule 文案笼统指向「网络失败」)。
|
||||
fn classify_server_error(code: &str, desc: &str) -> XfyunASRError {
|
||||
match code {
|
||||
"10105" | "10110" => XfyunASRError::AuthRejected(format!("{code} {desc}")),
|
||||
"10800" => XfyunASRError::RateLimited(format!("{code} {desc}")),
|
||||
_ => XfyunASRError::TaskFailed(format!("{code} {desc}")),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn signa_matches_official_documentation_example() {
|
||||
// 官方文档示例:appid=595f23df,ts=1512041814,apiKey=d9f4aa7ea6d94faca62cd88a28fd5234
|
||||
// → signa = IrrzsJeOFk1NGfJHW6SkHUoN9CU=
|
||||
let signa = compute_signa(
|
||||
"595f23df",
|
||||
"d9f4aa7ea6d94faca62cd88a28fd5234",
|
||||
"1512041814",
|
||||
);
|
||||
assert_eq!(signa, "IrrzsJeOFk1NGfJHW6SkHUoN9CU=");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn md5_hex_matches_known_vector() {
|
||||
// MD5("595f23df1512041814") = 0829d4012497c14a30e7e72aeebe565e(文档示例)
|
||||
assert_eq!(
|
||||
md5_hex(b"595f23df1512041814"),
|
||||
"0829d4012497c14a30e7e72aeebe565e"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn connect_url_contains_encoded_auth_params() {
|
||||
let creds = XfyunCredentials {
|
||||
app_id: "595f23df".into(),
|
||||
api_key: "d9f4aa7ea6d94faca62cd88a28fd5234".into(),
|
||||
};
|
||||
let url = connect_url(&creds);
|
||||
assert!(url.starts_with("wss://rtasr.xfyun.cn/v1/ws?"));
|
||||
assert!(url.contains("appid=595f23df"));
|
||||
assert!(url.contains("ts="));
|
||||
assert!(url.contains("signa="));
|
||||
assert!(url.contains("lang=cn"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extract_words_joins_candidates_in_order() {
|
||||
let data = serde_json::json!({
|
||||
"cn": {
|
||||
"st": {
|
||||
"bg": "820",
|
||||
"ed": "0",
|
||||
"rt": [
|
||||
{"ws": [
|
||||
{"cw": [{"w": "啊", "wp": "n"}], "wb": 0, "we": 0},
|
||||
{"cw": [{"w": "喂", "wp": "n"}], "wb": 0, "we": 0},
|
||||
{"cw": [{"w": "!", "wp": "p"}], "wb": 0, "we": 0},
|
||||
{"cw": [{"w": "你好", "wp": "n"}], "wb": 0, "we": 0}
|
||||
]}
|
||||
],
|
||||
"type": "1"
|
||||
}
|
||||
},
|
||||
"seg_id": 5
|
||||
});
|
||||
let text = extract_words(&data["cn"]["st"]);
|
||||
assert_eq!(text, "啊喂!你好");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn record_result_stores_final_and_partial_separately() {
|
||||
let asr = XfyunStreamingASR::new(XfyunCredentials {
|
||||
app_id: "app".into(),
|
||||
api_key: "key".into(),
|
||||
});
|
||||
asr.record_result(&serde_json::json!({
|
||||
"action": "result",
|
||||
"data": "{\"cn\":{\"st\":{\"rt\":[{\"ws\":[{\"cw\":[{\"w\":\"中间\"}]}]}],\"type\":\"1\"}},\"seg_id\":1}"
|
||||
}));
|
||||
{
|
||||
let st = asr.state.lock();
|
||||
assert!(st.final_segments.is_empty());
|
||||
assert_eq!(st.partial_segments.get(&1).unwrap(), "中间");
|
||||
}
|
||||
asr.record_result(&serde_json::json!({
|
||||
"action": "result",
|
||||
"data": "{\"cn\":{\"st\":{\"rt\":[{\"ws\":[{\"cw\":[{\"w\":\"最终\"}]}]}],\"type\":\"0\"}},\"seg_id\":1}"
|
||||
}));
|
||||
{
|
||||
let st = asr.state.lock();
|
||||
assert_eq!(st.final_segments.get(&1).unwrap(), "最终");
|
||||
assert!(st.partial_segments.is_empty(), "final 应清除同 seg 的 partial");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn record_result_accepts_numeric_type_for_final() {
|
||||
// 防御性:服务端若以数字 0/1 返回 type(而非文档字符串 "0"/"1"),
|
||||
// 最终结果仍要落 final_segments,不能整体降级成中间结果丢句。
|
||||
let asr = XfyunStreamingASR::new(XfyunCredentials {
|
||||
app_id: "app".into(),
|
||||
api_key: "key".into(),
|
||||
});
|
||||
asr.record_result(&serde_json::json!({
|
||||
"action": "result",
|
||||
"data": "{\"cn\":{\"st\":{\"rt\":[{\"ws\":[{\"cw\":[{\"w\":\"第一句\"}]}]}],\"type\":0}},\"seg_id\":1}"
|
||||
}));
|
||||
asr.record_result(&serde_json::json!({
|
||||
"action": "result",
|
||||
"data": "{\"cn\":{\"st\":{\"rt\":[{\"ws\":[{\"cw\":[{\"w\":\"第二句\"}]}]}],\"type\":0}},\"seg_id\":2}"
|
||||
}));
|
||||
let st = asr.state.lock();
|
||||
assert_eq!(st.final_segments.len(), 2);
|
||||
assert_eq!(st.final_segments.get(&1).unwrap(), "第一句");
|
||||
assert_eq!(st.final_segments.get(&2).unwrap(), "第二句");
|
||||
assert!(st.partial_segments.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn duplicate_final_segment_overwrites_not_duplicates() {
|
||||
let asr = XfyunStreamingASR::new(XfyunCredentials {
|
||||
app_id: "app".into(),
|
||||
api_key: "key".into(),
|
||||
});
|
||||
let event = |text: &str| {
|
||||
serde_json::json!({
|
||||
"action": "result",
|
||||
"data": format!(
|
||||
"{{\"cn\":{{\"st\":{{\"rt\":[{{\"ws\":[{{\"cw\":[{{\"w\":\"{text}\"}}]}}]}}],\"type\":\"0\"}}}},\"seg_id\":2}}"
|
||||
)
|
||||
})
|
||||
};
|
||||
asr.record_result(&event("第一版"));
|
||||
asr.record_result(&event("第二版"));
|
||||
let st = asr.state.lock();
|
||||
assert_eq!(st.final_segments.len(), 1);
|
||||
assert_eq!(st.final_segments.get(&2).unwrap(), "第二版");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn auth_and_license_errors_classify_as_auth_rejected() {
|
||||
assert!(matches!(
|
||||
classify_server_error("10105", "illegal access|illegal client_ip"),
|
||||
XfyunASRError::AuthRejected(_)
|
||||
));
|
||||
assert!(matches!(
|
||||
classify_server_error("10110", "no license"),
|
||||
XfyunASRError::AuthRejected(_)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn concurrency_limit_classifies_as_rate_limited() {
|
||||
assert!(matches!(
|
||||
classify_server_error("10800", "over max connect limit"),
|
||||
XfyunASRError::RateLimited(_)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generic_errors_stay_task_failed() {
|
||||
assert!(matches!(
|
||||
classify_server_error("37005", "no audio data"),
|
||||
XfyunASRError::TaskFailed(_)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn credentials_require_both_fields_trimmed() {
|
||||
let ok = XfyunCredentials {
|
||||
app_id: "app".into(),
|
||||
api_key: "key".into(),
|
||||
};
|
||||
assert!(ok.auth_ok());
|
||||
assert!(!XfyunCredentials { app_id: "".into(), api_key: "key".into() }.auth_ok());
|
||||
assert!(!XfyunCredentials { app_id: "app".into(), api_key: " ".into() }.auth_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn finish_error_notifies_pending_handshake_waiter() {
|
||||
// 握手前出错(如网络中断)必须唤醒 open_session 的 handshake 等待方,
|
||||
// 否则其空等 HANDSHAKE_TIMEOUT。已过握手的会话 take 到 None,幂等。
|
||||
let asr = XfyunStreamingASR::new(XfyunCredentials {
|
||||
app_id: "app".into(),
|
||||
api_key: "key".into(),
|
||||
});
|
||||
let (tx, mut rx) = oneshot::channel();
|
||||
*asr.handshake_tx.lock() = Some(tx);
|
||||
|
||||
asr.finish_error(XfyunASRError::ConnectionFailed("boom".into()));
|
||||
|
||||
match rx.try_recv() {
|
||||
Ok(Err(XfyunASRError::ConnectionFailed(_))) => {}
|
||||
other => panic!("handshake 等待方应收到错误,实际: {other:?}"),
|
||||
}
|
||||
// 已 take:重复调用不 panic、不重复通知。
|
||||
asr.finish_error(XfyunASRError::ConnectionFailed("again".into()));
|
||||
assert!(asr.handshake_tx.lock().is_none());
|
||||
}
|
||||
}
|
||||
@@ -76,6 +76,9 @@ pub(crate) fn asr_configured_for_provider(provider: &str, snap: &CredentialsSnap
|
||||
configured(&snap.asr_endpoint) && configured(&snap.asr_model)
|
||||
}
|
||||
AsrConfiguredFields::VolcAppKey => volcengine_configured(snap),
|
||||
AsrConfiguredFields::XfyunAppKey => {
|
||||
configured(&snap.xfyun_app_id) && configured(&snap.xfyun_api_key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -213,6 +216,8 @@ pub async fn set_credential(
|
||||
| CredentialAccount::AsrEndpoint
|
||||
| CredentialAccount::AsrModel
|
||||
| CredentialAccount::AsrVocabularyId
|
||||
| CredentialAccount::XfyunAppId
|
||||
| CredentialAccount::XfyunApiKey
|
||||
) {
|
||||
return Err("provider-scoped credential must be an ASR account".to_string());
|
||||
}
|
||||
@@ -355,6 +360,8 @@ fn parse_account(s: &str) -> Result<CredentialAccount, String> {
|
||||
"asr.endpoint" => Ok(CredentialAccount::AsrEndpoint),
|
||||
"asr.model" => Ok(CredentialAccount::AsrModel),
|
||||
"asr.vocabulary_id" => Ok(CredentialAccount::AsrVocabularyId),
|
||||
"xfyun.app_id" => Ok(CredentialAccount::XfyunAppId),
|
||||
"xfyun.api_key" => Ok(CredentialAccount::XfyunApiKey),
|
||||
_ => Err(format!("unknown account: {s}")),
|
||||
}
|
||||
}
|
||||
@@ -280,6 +280,9 @@ async fn validate_asr_provider() -> Result<(), String> {
|
||||
if active_asr == crate::asr::elevenlabs::PROVIDER_ID {
|
||||
return validate_elevenlabs_asr_provider().await;
|
||||
}
|
||||
if active_asr == crate::asr::xfyun::PROVIDER_ID {
|
||||
return validate_xfyun_asr_provider().await;
|
||||
}
|
||||
// StepFun 一入口双协议:`*-stream` 模型走实时 WS 验证,其余走批式
|
||||
// /audio/transcriptions(与 build 侧 resolve_effective_asr_provider 同判据)。
|
||||
if active_asr == "stepfun" || active_asr == crate::asr::stepfun_realtime::PROVIDER_ID {
|
||||
@@ -301,6 +304,38 @@ async fn validate_asr_provider() -> Result<(), String> {
|
||||
validate_asr_transcription(&config, model.trim()).await
|
||||
}
|
||||
|
||||
/// 讯飞 RTASR 验证:真连 + 500ms 静音 + 收尾。鉴权错误(10105 / 10110)在握手阶段
|
||||
/// 即返回;纯静音会话服务端可能直接关闭且不返回任何 result(等价于「没说话」),
|
||||
/// 这类 `NoFinalResult` 不算验证失败 —— 握手成功已经证明 AppID/APIKey 有效。
|
||||
async fn validate_xfyun_asr_provider() -> Result<(), String> {
|
||||
let app_id = CredentialsVault::get(CredentialAccount::XfyunAppId)
|
||||
.map_err(|e| e.to_string())?
|
||||
.unwrap_or_default();
|
||||
if app_id.trim().is_empty() {
|
||||
return Err("讯飞 AppID 为空".to_string());
|
||||
}
|
||||
let api_key = CredentialsVault::get(CredentialAccount::XfyunApiKey)
|
||||
.map_err(|e| e.to_string())?
|
||||
.unwrap_or_default();
|
||||
if api_key.trim().is_empty() {
|
||||
return Err("讯飞 API Key 为空".to_string());
|
||||
}
|
||||
let asr = std::sync::Arc::new(crate::asr::XfyunStreamingASR::new(
|
||||
crate::asr::XfyunCredentials { app_id, api_key },
|
||||
));
|
||||
asr.open_session().await.map_err(|e| e.to_string())?;
|
||||
crate::asr::AudioConsumer::consume_pcm_chunk(
|
||||
&*asr,
|
||||
&vec![0u8; crate::asr::xfyun::TARGET_AUDIO_CHUNK_BYTES * 5],
|
||||
);
|
||||
asr.send_last_frame().await.map_err(|e| e.to_string())?;
|
||||
match asr.await_final_result().await {
|
||||
Ok(_) => Ok(()),
|
||||
Err(crate::asr::xfyun::XfyunASRError::NoFinalResult) => Ok(()),
|
||||
Err(e) => Err(e.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
/// StepFun 实时 WS 验证:真连 + session.update + 500ms 静音 + 收尾。
|
||||
/// 协议无 finish 事件,收尾走静音帧 + 宽限期(纯静音会话以空文本成功返回,
|
||||
/// 见 stepfun_realtime 模块注释),全程 ~2s。
|
||||
|
||||
@@ -194,6 +194,8 @@ enum ActiveAsr {
|
||||
Qwen3Realtime(Arc<Qwen3RealtimeASR>),
|
||||
/// 阶跃星辰 StepAudio 实时(OpenAI Realtime 风格 WS,收尾靠静音帧驱动 VAD)。
|
||||
StepfunRealtime(Arc<crate::asr::StepfunRealtimeASR>),
|
||||
/// 讯飞开放平台实时语音转写(RTASR)流式。
|
||||
Xfyun(Arc<crate::asr::XfyunStreamingASR>),
|
||||
#[cfg(target_os = "windows")]
|
||||
FoundryLocalWhisper(Arc<FoundryLocalWhisperAsr>),
|
||||
/// Windows sherpa-onnx 本地 ASR(offline batch + 实验 online streaming)。
|
||||
@@ -237,6 +239,7 @@ pub(crate) enum ActiveAsrProviderKind {
|
||||
ElevenLabs,
|
||||
WhisperCompatible,
|
||||
Volcengine,
|
||||
Xfyun,
|
||||
}
|
||||
|
||||
/// 「能否开始一次会话」所需的凭据形态(对应 `ensure_asr_credentials` 预检门)。
|
||||
@@ -246,6 +249,8 @@ pub(crate) enum AsrPreflightCredential {
|
||||
AsrApiKey,
|
||||
/// 需要火山引擎 App Key + Access Key。
|
||||
VolcAppKey,
|
||||
/// 需要讯飞 AppID + APIKey。
|
||||
XfyunAppKey,
|
||||
}
|
||||
|
||||
/// 概览页「已配置 / 未配置」状态所需的字段(对应 `asr_configured_for_provider`)。
|
||||
@@ -260,6 +265,8 @@ pub(crate) enum AsrConfiguredFields {
|
||||
EndpointModelOnly,
|
||||
/// 火山引擎三件套。
|
||||
VolcAppKey,
|
||||
/// 讯飞 AppID + APIKey。
|
||||
XfyunAppKey,
|
||||
}
|
||||
|
||||
impl ActiveAsrProviderKind {
|
||||
@@ -273,6 +280,7 @@ impl ActiveAsrProviderKind {
|
||||
| ActiveAsrProviderKind::ElevenLabs
|
||||
| ActiveAsrProviderKind::WhisperCompatible => AsrPreflightCredential::AsrApiKey,
|
||||
ActiveAsrProviderKind::Volcengine => AsrPreflightCredential::VolcAppKey,
|
||||
ActiveAsrProviderKind::Xfyun => AsrPreflightCredential::XfyunAppKey,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -292,6 +300,7 @@ impl ActiveAsrProviderKind {
|
||||
ActiveAsrProviderKind::WhisperCompatible
|
||||
| ActiveAsrProviderKind::StepfunRealtime => AsrConfiguredFields::EndpointModelOnly,
|
||||
ActiveAsrProviderKind::Volcengine => AsrConfiguredFields::VolcAppKey,
|
||||
ActiveAsrProviderKind::Xfyun => AsrConfiguredFields::XfyunAppKey,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -311,6 +320,8 @@ pub(crate) fn active_asr_provider_kind(id: &str) -> ActiveAsrProviderKind {
|
||||
ActiveAsrProviderKind::ElevenLabs
|
||||
} else if is_whisper_compatible_provider(id) {
|
||||
ActiveAsrProviderKind::WhisperCompatible
|
||||
} else if is_xfyun_provider(id) {
|
||||
ActiveAsrProviderKind::Xfyun
|
||||
} else {
|
||||
ActiveAsrProviderKind::Volcengine
|
||||
}
|
||||
@@ -2046,6 +2057,13 @@ impl Coordinator {
|
||||
.map_err(|_| "重新转录超时".to_string())?
|
||||
.map_err(|e| e.to_string())?
|
||||
}
|
||||
ActiveAsr::Xfyun(asr) => {
|
||||
asr.send_last_frame().await.map_err(|e| e.to_string())?;
|
||||
tokio::time::timeout(timeout, asr.await_final_result())
|
||||
.await
|
||||
.map_err(|_| "重新转录超时".to_string())?
|
||||
.map_err(|e| e.to_string())?
|
||||
}
|
||||
ActiveAsr::Whisper(w) => tokio::time::timeout(timeout, w.transcribe())
|
||||
.await
|
||||
.map_err(|_| "重新转录超时".to_string())?
|
||||
@@ -2662,6 +2680,18 @@ fn read_volc_credentials() -> VolcengineCredentials {
|
||||
}
|
||||
}
|
||||
|
||||
fn read_xfyun_credentials() -> crate::asr::XfyunCredentials {
|
||||
let app_id = CredentialsVault::get(CredentialAccount::XfyunAppId)
|
||||
.ok()
|
||||
.flatten()
|
||||
.unwrap_or_default();
|
||||
let api_key = CredentialsVault::get(CredentialAccount::XfyunApiKey)
|
||||
.ok()
|
||||
.flatten()
|
||||
.unwrap_or_default();
|
||||
crate::asr::XfyunCredentials { app_id, api_key }
|
||||
}
|
||||
|
||||
fn enabled_hotwords(inner: &Arc<Inner>) -> Vec<DictionaryHotword> {
|
||||
inner
|
||||
.vocab
|
||||
@@ -3185,6 +3215,7 @@ mod tests {
|
||||
assert_eq!(ElevenLabs.preflight_credential(), AsrApiKey);
|
||||
assert_eq!(WhisperCompatible.preflight_credential(), AsrApiKey);
|
||||
assert_eq!(Volcengine.preflight_credential(), VolcAppKey);
|
||||
assert_eq!(Xfyun.preflight_credential(), XfyunAppKey);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -3326,6 +3357,7 @@ mod tests {
|
||||
assert_eq!(ElevenLabs.configured_fields(), ApiKeyOnly);
|
||||
assert_eq!(WhisperCompatible.configured_fields(), EndpointModelOnly);
|
||||
assert_eq!(Volcengine.configured_fields(), VolcAppKey);
|
||||
assert_eq!(Xfyun.configured_fields(), XfyunAppKey);
|
||||
}
|
||||
|
||||
#[cfg(target_os = "windows")]
|
||||
|
||||
@@ -136,6 +136,14 @@ pub(super) fn ensure_asr_credentials() -> Result<(), String> {
|
||||
}
|
||||
}
|
||||
}
|
||||
AsrPreflightCredential::XfyunAppKey => {
|
||||
let creds = read_xfyun_credentials();
|
||||
if creds.auth_ok() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err("请先在设置中填写讯飞 AppID 和 API Key".to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -415,6 +423,10 @@ pub(super) fn is_elevenlabs_provider(id: &str) -> bool {
|
||||
id == crate::asr::elevenlabs::PROVIDER_ID
|
||||
}
|
||||
|
||||
pub(super) fn is_xfyun_provider(id: &str) -> bool {
|
||||
id == crate::asr::xfyun::PROVIDER_ID
|
||||
}
|
||||
|
||||
pub(super) fn apply_chinese_script_preference(text: &str, pref: ChineseScriptPreference) -> String {
|
||||
if text.is_empty() {
|
||||
return String::new();
|
||||
@@ -453,6 +465,10 @@ pub(super) enum QaAsrStart {
|
||||
asr: Arc<crate::asr::StepfunRealtimeASR>,
|
||||
bridge: Arc<DeferredAsrBridge>,
|
||||
},
|
||||
Xfyun {
|
||||
asr: Arc<crate::asr::XfyunStreamingASR>,
|
||||
bridge: Arc<DeferredAsrBridge>,
|
||||
},
|
||||
Ready {
|
||||
active: ActiveAsr,
|
||||
consumer: Arc<dyn crate::recorder::AudioConsumer>,
|
||||
@@ -466,6 +482,7 @@ impl QaAsrStart {
|
||||
QaAsrStart::Bailian { asr, .. } => ActiveAsr::Bailian(Arc::clone(asr)),
|
||||
QaAsrStart::Qwen3Realtime { asr, .. } => ActiveAsr::Qwen3Realtime(Arc::clone(asr)),
|
||||
QaAsrStart::StepfunRealtime { asr, .. } => ActiveAsr::StepfunRealtime(Arc::clone(asr)),
|
||||
QaAsrStart::Xfyun { asr, .. } => ActiveAsr::Xfyun(Arc::clone(asr)),
|
||||
QaAsrStart::Ready { active, .. } => active.clone(),
|
||||
}
|
||||
}
|
||||
@@ -476,6 +493,7 @@ impl QaAsrStart {
|
||||
QaAsrStart::Bailian { bridge, .. } => Arc::clone(bridge) as _,
|
||||
QaAsrStart::Qwen3Realtime { bridge, .. } => Arc::clone(bridge) as _,
|
||||
QaAsrStart::StepfunRealtime { bridge, .. } => Arc::clone(bridge) as _,
|
||||
QaAsrStart::Xfyun { bridge, .. } => Arc::clone(bridge) as _,
|
||||
QaAsrStart::Ready { consumer, .. } => Arc::clone(consumer),
|
||||
}
|
||||
}
|
||||
@@ -516,6 +534,15 @@ impl QaAsrStart {
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
QaAsrStart::Xfyun { asr, bridge } => {
|
||||
asr.open_session().await.map_err(|e| e.to_string())?;
|
||||
let target: Arc<dyn crate::asr::AudioConsumer> = Arc::clone(asr) as _;
|
||||
let flushed = bridge.attach(target);
|
||||
log::info!(
|
||||
"[coord] QA iFlytek ASR connected; flushed {flushed} deferred audio bytes"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
QaAsrStart::Ready { .. } => Ok(()),
|
||||
}
|
||||
}
|
||||
@@ -710,5 +737,16 @@ pub(super) async fn build_qa_asr_start(
|
||||
label,
|
||||
))
|
||||
}
|
||||
ActiveAsrProviderKind::Xfyun => {
|
||||
let creds = read_xfyun_credentials();
|
||||
let label = AsrCallLabel::new(effective_asr.clone(), None);
|
||||
Ok((
|
||||
QaAsrStart::Xfyun {
|
||||
asr: Arc::new(crate::asr::XfyunStreamingASR::new(creds)),
|
||||
bridge: Arc::new(DeferredAsrBridge::new()),
|
||||
},
|
||||
label,
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1861,7 +1861,8 @@ pub(super) async fn begin_session_as(
|
||||
| ActiveAsrProviderKind::DashScopeMultimodal
|
||||
| ActiveAsrProviderKind::ElevenLabs
|
||||
| ActiveAsrProviderKind::WhisperCompatible
|
||||
| ActiveAsrProviderKind::Volcengine => {}
|
||||
| ActiveAsrProviderKind::Volcengine
|
||||
| ActiveAsrProviderKind::Xfyun => {}
|
||||
}
|
||||
|
||||
if is_bailian_provider(&effective_asr) {
|
||||
@@ -2173,6 +2174,82 @@ pub(super) async fn begin_session_as(
|
||||
let consumer: Arc<dyn crate::recorder::AudioConsumer> = whisper;
|
||||
start_recorder_and_enter_listening(inner, current_session_id, &active_asr, consumer)
|
||||
.await?;
|
||||
} else if is_xfyun_provider(&effective_asr) {
|
||||
// 讯飞 RTASR 实时流式:与 Bailian / 火山同构(open_session → 录音 → end → final)。
|
||||
let creds = read_xfyun_credentials();
|
||||
let asr_call_label = AsrCallLabel::new(effective_asr.clone(), None);
|
||||
let asr = Arc::new(crate::asr::XfyunStreamingASR::new(creds));
|
||||
let bridge = Arc::new(DeferredAsrBridge::new());
|
||||
let consumer: Arc<dyn crate::recorder::AudioConsumer> = bridge.clone();
|
||||
store_asr_for_session(
|
||||
inner,
|
||||
current_session_id,
|
||||
ActiveAsr::Xfyun(Arc::clone(&asr)),
|
||||
asr_call_label,
|
||||
);
|
||||
start_recorder_for_starting(inner, current_session_id, &active_asr, consumer).await?;
|
||||
|
||||
if let Err(e) = asr.open_session().await {
|
||||
log::error!("[coord] open iFlytek ASR session failed: {e}");
|
||||
match startup_race_status_for_starting(inner, current_session_id) {
|
||||
StartupRaceStatus::StaleContinuation => {
|
||||
log::info!(
|
||||
"[coord] stale iFlytek ASR open_session error from session {current_session_id} — ignoring"
|
||||
);
|
||||
asr.cancel();
|
||||
discard_startup_resources_for_session(inner, current_session_id);
|
||||
restore_prepared_windows_ime_session(inner, current_session_id);
|
||||
return Ok(());
|
||||
}
|
||||
StartupRaceStatus::CancelRaced => {
|
||||
asr.cancel();
|
||||
discard_startup_resources_for_session(inner, current_session_id);
|
||||
restore_prepared_windows_ime_session(inner, current_session_id);
|
||||
set_phase_idle_if_session_matches(inner, current_session_id);
|
||||
return Ok(());
|
||||
}
|
||||
StartupRaceStatus::ActiveStarting => {
|
||||
asr.cancel();
|
||||
}
|
||||
}
|
||||
discard_startup_resources_for_session(inner, current_session_id);
|
||||
emit_capsule(
|
||||
inner,
|
||||
CapsuleState::Error,
|
||||
0.0,
|
||||
0,
|
||||
Some(format!("ASR 连接失败: {e}")),
|
||||
None,
|
||||
);
|
||||
restore_prepared_windows_ime_session(inner, current_session_id);
|
||||
set_phase_idle_if_session_matches(inner, current_session_id);
|
||||
schedule_capsule_idle(inner, CAPSULE_AUTO_HIDE_DELAY_MS);
|
||||
return Err(e.to_string());
|
||||
}
|
||||
match startup_race_status_for_starting(inner, current_session_id) {
|
||||
StartupRaceStatus::ActiveStarting => {}
|
||||
StartupRaceStatus::CancelRaced => {
|
||||
log::info!("[coord] cancel raced during iFlytek ASR open_session — aborting begin");
|
||||
asr.cancel();
|
||||
discard_startup_resources_for_session(inner, current_session_id);
|
||||
restore_prepared_windows_ime_session(inner, current_session_id);
|
||||
set_phase_idle_if_session_matches(inner, current_session_id);
|
||||
return Ok(());
|
||||
}
|
||||
StartupRaceStatus::StaleContinuation => {
|
||||
log::info!(
|
||||
"[coord] stale iFlytek ASR open_session continuation from session {current_session_id} — ignoring"
|
||||
);
|
||||
asr.cancel();
|
||||
discard_startup_resources_for_session(inner, current_session_id);
|
||||
restore_prepared_windows_ime_session(inner, current_session_id);
|
||||
return Ok(());
|
||||
}
|
||||
}
|
||||
let target: Arc<dyn crate::asr::AudioConsumer> = asr;
|
||||
let flushed_bytes = bridge.attach(target);
|
||||
log::info!("[coord] iFlytek ASR connected; flushed {flushed_bytes} deferred audio bytes");
|
||||
finish_starting_session(inner, current_session_id).await;
|
||||
} else {
|
||||
let hotwords = enabled_hotwords(inner);
|
||||
let creds = read_volc_credentials();
|
||||
@@ -3048,6 +3125,34 @@ pub(super) async fn end_session(inner: &Arc<Inner>) -> Result<(), String> {
|
||||
}
|
||||
}
|
||||
}
|
||||
ActiveAsr::Xfyun(asr) => {
|
||||
debug_assert!(uses_global_timeout);
|
||||
if let Err(e) = asr.send_last_frame().await {
|
||||
log::error!("[coord] iFlytek ASR send last frame failed: {e}");
|
||||
}
|
||||
let timeout_duration =
|
||||
std::time::Duration::from_secs(COORDINATOR_GLOBAL_TIMEOUT_SECS);
|
||||
match tokio::time::timeout(timeout_duration, asr.await_final_result()).await {
|
||||
Ok(Ok(r)) => Ok(r),
|
||||
Ok(Err(e)) => {
|
||||
log::error!("[coord] iFlytek ASR await final failed: {e}");
|
||||
// 关闭 WebSocket 连接,避免流式 ASR 资源泄漏
|
||||
asr.cancel();
|
||||
Err(TranscribeFail::new(format!("识别失败: {e}"), e.to_string()))
|
||||
}
|
||||
Err(_) => {
|
||||
log::error!(
|
||||
"[coord] iFlytek ASR 全局超时 {} 秒",
|
||||
COORDINATOR_GLOBAL_TIMEOUT_SECS
|
||||
);
|
||||
asr.cancel();
|
||||
Err(TranscribeFail::new(
|
||||
"识别超时".to_string(),
|
||||
"xfyun global timeout".to_string(),
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
ActiveAsr::FoundryLocalWhisper(local) => {
|
||||
debug_assert!(!uses_global_timeout);
|
||||
|
||||
@@ -459,6 +459,21 @@ pub(super) async fn transcribe_overlay_dictation_asr(
|
||||
}
|
||||
}
|
||||
}
|
||||
ActiveAsr::Xfyun(asr) => {
|
||||
debug_assert!(uses_global_timeout);
|
||||
if let Err(error) = asr.send_last_frame().await {
|
||||
log::error!("[coord] overlay QA: iFlytek ASR send last frame failed: {error}");
|
||||
}
|
||||
let timeout_duration = std::time::Duration::from_secs(COORDINATOR_GLOBAL_TIMEOUT_SECS);
|
||||
match tokio::time::timeout(timeout_duration, asr.await_final_result()).await {
|
||||
Ok(Ok(raw)) => Ok(raw),
|
||||
Ok(Err(error)) => Err(error.to_string()),
|
||||
Err(_) => {
|
||||
asr.cancel();
|
||||
Err("xfyun global timeout".to_string())
|
||||
}
|
||||
}
|
||||
}
|
||||
ActiveAsr::Whisper(whisper) => {
|
||||
debug_assert!(uses_global_timeout);
|
||||
let timeout_duration = std::time::Duration::from_secs(COORDINATOR_GLOBAL_TIMEOUT_SECS);
|
||||
@@ -1085,6 +1100,30 @@ pub(super) async fn end_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
|
||||
}
|
||||
}
|
||||
}
|
||||
ActiveAsr::Xfyun(asr) => {
|
||||
debug_assert!(uses_global_timeout);
|
||||
if let Err(e) = asr.send_last_frame().await {
|
||||
log::error!("[coord] QA: iFlytek ASR send last frame failed: {e}");
|
||||
}
|
||||
let timeout_duration = std::time::Duration::from_secs(COORDINATOR_GLOBAL_TIMEOUT_SECS);
|
||||
match tokio::time::timeout(timeout_duration, asr.await_final_result()).await {
|
||||
Ok(Ok(r)) => r,
|
||||
Ok(Err(e)) => {
|
||||
log::error!("[coord] QA: iFlytek ASR await final failed: {e}");
|
||||
finish_qa_with_error_if_current(inner, session_id, format!("识别失败: {e}"));
|
||||
return Err(e.to_string());
|
||||
}
|
||||
Err(_) => {
|
||||
log::error!(
|
||||
"[coord] QA: iFlytek ASR 全局超时 {} 秒",
|
||||
COORDINATOR_GLOBAL_TIMEOUT_SECS
|
||||
);
|
||||
asr.cancel();
|
||||
finish_qa_with_error_if_current(inner, session_id, "识别超时".to_string());
|
||||
return Err("xfyun global timeout".to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
ActiveAsr::Qwen3Realtime(asr) => {
|
||||
debug_assert!(uses_global_timeout);
|
||||
if let Err(e) = asr.send_last_frame().await {
|
||||
|
||||
@@ -91,6 +91,7 @@ pub(super) fn cancel_active_asr(asr: ActiveAsr) {
|
||||
ActiveAsr::Bailian(b) => b.cancel(),
|
||||
ActiveAsr::Qwen3Realtime(q) => q.cancel(),
|
||||
ActiveAsr::StepfunRealtime(s) => s.cancel(),
|
||||
ActiveAsr::Xfyun(x) => x.cancel(),
|
||||
#[cfg(target_os = "windows")]
|
||||
ActiveAsr::FoundryLocalWhisper(local) => local.cancel(),
|
||||
#[cfg(target_os = "windows")]
|
||||
|
||||
@@ -220,6 +220,12 @@ struct CredsAsrEntry {
|
||||
volcengineApiKey: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
vocabularyId: Option<String>,
|
||||
/// 讯飞开放平台应用 ID(RTASR/IFASR 鉴权用)。
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
xfyunAppId: Option<String>,
|
||||
/// 讯飞实时语音转写 APIKey(接口密钥)。
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
xfyunApiKey: Option<String>,
|
||||
}
|
||||
|
||||
impl CredsAsrEntry {
|
||||
@@ -233,6 +239,8 @@ impl CredsAsrEntry {
|
||||
&& self.authMode.as_deref().unwrap_or("").is_empty()
|
||||
&& self.volcengineApiKey.as_deref().unwrap_or("").is_empty()
|
||||
&& self.vocabularyId.as_deref().unwrap_or("").is_empty()
|
||||
&& self.xfyunAppId.as_deref().unwrap_or("").is_empty()
|
||||
&& self.xfyunApiKey.as_deref().unwrap_or("").is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1126,6 +1134,8 @@ fn lookup_account(root: &CredsRoot, account: CredentialAccount) -> Option<String
|
||||
CredentialAccount::AsrEndpoint => asr.and_then(|e| pick(&e.baseURL)),
|
||||
CredentialAccount::AsrModel => asr.and_then(|e| pick(&e.model)),
|
||||
CredentialAccount::AsrVocabularyId => asr.and_then(|e| pick(&e.vocabularyId)),
|
||||
CredentialAccount::XfyunAppId => asr.and_then(|e| pick(&e.xfyunAppId)),
|
||||
CredentialAccount::XfyunApiKey => asr.and_then(|e| pick(&e.xfyunApiKey)),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1182,6 +1192,14 @@ fn write_account(root: &mut CredsRoot, account: CredentialAccount, value: Option
|
||||
let entry = root.providers.asr.entry(asr_id).or_default();
|
||||
entry.vocabularyId = normalized;
|
||||
}
|
||||
CredentialAccount::XfyunAppId => {
|
||||
let entry = root.providers.asr.entry(asr_id).or_default();
|
||||
entry.xfyunAppId = normalized;
|
||||
}
|
||||
CredentialAccount::XfyunApiKey => {
|
||||
let entry = root.providers.asr.entry(asr_id).or_default();
|
||||
entry.xfyunApiKey = normalized;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1204,6 +1222,10 @@ pub enum CredentialAccount {
|
||||
AsrModel,
|
||||
/// Active ASR provider's optional hotword vocabulary ID.
|
||||
AsrVocabularyId,
|
||||
/// 讯飞开放平台应用 ID。
|
||||
XfyunAppId,
|
||||
/// 讯飞实时语音转写 APIKey。
|
||||
XfyunApiKey,
|
||||
}
|
||||
|
||||
impl CredentialAccount {
|
||||
@@ -1224,6 +1246,8 @@ impl CredentialAccount {
|
||||
CredentialAccount::AsrEndpoint => "asr.endpoint",
|
||||
CredentialAccount::AsrModel => "asr.model",
|
||||
CredentialAccount::AsrVocabularyId => "asr.vocabulary_id",
|
||||
CredentialAccount::XfyunAppId => "xfyun.app_id",
|
||||
CredentialAccount::XfyunApiKey => "xfyun.api_key",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1241,6 +1265,8 @@ impl CredentialAccount {
|
||||
CredentialAccount::AsrEndpoint,
|
||||
CredentialAccount::AsrModel,
|
||||
CredentialAccount::AsrVocabularyId,
|
||||
CredentialAccount::XfyunAppId,
|
||||
CredentialAccount::XfyunApiKey,
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -1256,6 +1282,8 @@ pub struct CredentialsSnapshot {
|
||||
pub asr_api_key: Option<String>,
|
||||
pub asr_endpoint: Option<String>,
|
||||
pub asr_model: Option<String>,
|
||||
pub xfyun_app_id: Option<String>,
|
||||
pub xfyun_api_key: Option<String>,
|
||||
pub ark_api_key: Option<String>,
|
||||
pub ark_model_id: Option<String>,
|
||||
pub ark_endpoint: Option<String>,
|
||||
@@ -1476,6 +1504,8 @@ impl CredentialsVault {
|
||||
asr_api_key: lookup_account(&root, CredentialAccount::AsrApiKey),
|
||||
asr_endpoint: lookup_account(&root, CredentialAccount::AsrEndpoint),
|
||||
asr_model: lookup_account(&root, CredentialAccount::AsrModel),
|
||||
xfyun_app_id: lookup_account(&root, CredentialAccount::XfyunAppId),
|
||||
xfyun_api_key: lookup_account(&root, CredentialAccount::XfyunApiKey),
|
||||
ark_api_key: lookup_account(&root, CredentialAccount::ArkApiKey),
|
||||
ark_model_id: lookup_account(&root, CredentialAccount::ArkModelId),
|
||||
ark_endpoint: lookup_account(&root, CredentialAccount::ArkEndpoint),
|
||||
|
||||
@@ -859,6 +859,7 @@ export const en: typeof zhCN = {
|
||||
asrWhisper: 'OpenAI Whisper (compatible)',
|
||||
asrOpenrouter: 'OpenRouter Whisper',
|
||||
asrXiaomiMimo: 'Xiaomi MiMo ASR',
|
||||
asrIflytek: 'iFlytek Realtime ASR',
|
||||
asrElevenLabs: 'ElevenLabs Scribe',
|
||||
asrSherpaOnnxLocal: 'Local sherpa-onnx (experimental)',
|
||||
asrFoundryLocalWhisper: 'Local Whisper (Foundry Local)',
|
||||
@@ -875,6 +876,9 @@ export const en: typeof zhCN = {
|
||||
volcengineAuthModeApiKey: 'Ark API Key',
|
||||
volcengineMappingNote: 'Secret Key is not required right now. Resource ID defaults to volc.seedasr.sauc.duration.',
|
||||
volcengineApiKeyNote: 'Authenticate with Volcengine Ark speech model API Key, no APP ID needed. Resource ID defaults to volc.seedasr.sauc.duration.',
|
||||
xfyunAppIdLabel: 'AppID',
|
||||
xfyunApiKeyLabel: 'API Key',
|
||||
xfyunNote: 'Get AppID and API Key from the iFlytek Open Platform "Realtime ASR" service page. Audio is 16 kHz / 16-bit / mono PCM; the standard API has no hotword parameter (configure personalized hotwords in the iFlytek console), and the language defaults to Mandarin Chinese.',
|
||||
localAsrActiveNotice: 'Local ASR ({{name}}) is currently active. Switch or disable it from the Advanced tab.',
|
||||
localAsrTakeoverHint: 'Once "{{name}}" is enabled, the ASR provider will be taken over.',
|
||||
asrProviderTakenOver: 'A local engine is active. Pick another provider in the dropdown above to switch (the local engine stops automatically); manage local models under Advanced → Local models.',
|
||||
|
||||
@@ -861,6 +861,7 @@ export const ja: typeof zhCN = {
|
||||
asrWhisper: 'OpenAI Whisper(互換)',
|
||||
asrOpenrouter: 'OpenRouter Whisper',
|
||||
asrXiaomiMimo: 'Xiaomi MiMo ASR',
|
||||
asrIflytek: 'iFlytek リアルタイム音声認識',
|
||||
asrElevenLabs: 'ElevenLabs Scribe',
|
||||
asrSherpaOnnxLocal: 'ローカル sherpa-onnx(実験的)',
|
||||
asrFoundryLocalWhisper: 'ローカル Whisper(Foundry Local)',
|
||||
@@ -877,6 +878,9 @@ export const ja: typeof zhCN = {
|
||||
volcengineAuthModeApiKey: 'Ark API Key',
|
||||
volcengineMappingNote: 'Secret Key は現在不要。Resource ID のデフォルトは volc.seedasr.sauc.duration。',
|
||||
volcengineApiKeyNote: 'Volcengine Ark 音声モデルの API Key で認証、APP ID は不要です。Resource ID のデフォルトは volc.seedasr.sauc.duration。',
|
||||
xfyunAppIdLabel: 'AppID',
|
||||
xfyunApiKeyLabel: 'API Key',
|
||||
xfyunNote: 'iFlytek オープンプラットフォームの「リアルタイム音声認識」サービスページで AppID と API Key を取得します。音声は 16kHz / 16bit / モノラル PCM。標準版 API にホットワード引数はありません(iFlytek コンソールで個別ホットワードを設定)。言語はデフォルトで中国語(普通話)です。',
|
||||
localAsrActiveNotice: '現在「{{name}}」を使用中。「詳細設定」タブから切り替えまたは無効化できます。',
|
||||
localAsrTakeoverHint: '「{{name}}」を有効化すると ASR プロバイダーが引き継がれます。',
|
||||
asrProviderTakenOver: 'ASR プロバイダーは引き継ぎ済み',
|
||||
|
||||
@@ -861,6 +861,7 @@ export const ko: typeof zhCN = {
|
||||
asrWhisper: 'OpenAI Whisper(호환)',
|
||||
asrOpenrouter: 'OpenRouter Whisper',
|
||||
asrXiaomiMimo: 'Xiaomi MiMo ASR',
|
||||
asrIflytek: 'iFlytek 실시간 음성 인식',
|
||||
asrElevenLabs: 'ElevenLabs Scribe',
|
||||
asrSherpaOnnxLocal: '로컬 sherpa-onnx(실험적)',
|
||||
asrFoundryLocalWhisper: '로컬 Whisper(Foundry Local)',
|
||||
@@ -877,6 +878,9 @@ export const ko: typeof zhCN = {
|
||||
volcengineAuthModeApiKey: 'Ark API Key',
|
||||
volcengineMappingNote: 'Secret Key 는 현재 입력 불필요. Resource ID 기본값은 volc.seedasr.sauc.duration.',
|
||||
volcengineApiKeyNote: 'Volcengine Ark 음성 모델 API 키로 인증하며 APP ID는 필요하지 않습니다. Resource ID 기본값은 volc.seedasr.sauc.duration입니다.',
|
||||
xfyunAppIdLabel: 'AppID',
|
||||
xfyunApiKeyLabel: 'API Key',
|
||||
xfyunNote: 'iFlytek 오픈 플랫폼 "실시간 음성 인식" 서비스 페이지에서 AppID와 API Key를 가져옵니다. 오디오는 16kHz/16bit/모노 PCM입니다. 표준 API에는 핫워드 매개변수가 없으며(iFlytek 콘솔에서 개별 핫워드 설정), 언어는 기본적으로 중국어(보통화)입니다.',
|
||||
localAsrActiveNotice: '현재 "{{name}}" 사용 중. "고급" 탭에서 전환 또는 비활성화할 수 있습니다.',
|
||||
localAsrTakeoverHint: '"{{name}}" 활성화 시 ASR 프로바이더가 인수됩니다.',
|
||||
asrProviderTakenOver: 'ASR 프로바이더 인수 완료',
|
||||
|
||||
@@ -857,6 +857,7 @@ export const zhCN = {
|
||||
asrWhisper: 'OpenAI Whisper(兼容)',
|
||||
asrOpenrouter: 'OpenRouter Whisper',
|
||||
asrXiaomiMimo: '小米 MiMo ASR',
|
||||
asrIflytek: '讯飞实时语音转写',
|
||||
asrElevenLabs: 'ElevenLabs Scribe',
|
||||
asrSherpaOnnxLocal: '本地 sherpa-onnx(实验性)',
|
||||
asrFoundryLocalWhisper: '本地 Whisper(Foundry Local)',
|
||||
@@ -873,6 +874,9 @@ export const zhCN = {
|
||||
volcengineAuthModeApiKey: '方舟 API Key',
|
||||
volcengineMappingNote: 'Secret Key 当前无需填写。Resource ID 默认使用 volc.seedasr.sauc.duration。',
|
||||
volcengineApiKeyNote: '使用火山方舟语音模型的 API Key 鉴权,无需 APP ID。Resource ID 默认使用 volc.seedasr.sauc.duration。',
|
||||
xfyunAppIdLabel: 'AppID',
|
||||
xfyunApiKeyLabel: 'API Key',
|
||||
xfyunNote: '在讯飞开放平台「实时语音转写」服务页获取 AppID 与 API Key。音频为 16kHz/16bit/单声道 PCM;标准版接口暂不支持热词参数(可在讯飞控制台配置个性化热词),语种默认中文普通话。',
|
||||
localAsrActiveNotice: '当前已启用「{{name}}」,可在「高级」中切换或禁用。',
|
||||
localAsrTakeoverHint: '启动「{{name}}」后,ASR 提供商将被接管。',
|
||||
asrProviderTakenOver: '当前用的是本地引擎,在上方下拉直接选其它供应商即可切换(本地引擎会自动停用);本地模型在「高级 → 本地模型」里管理。',
|
||||
|
||||
@@ -859,6 +859,7 @@ export const zhTW: typeof zhCN = {
|
||||
asrWhisper: 'OpenAI Whisper(兼容)',
|
||||
asrOpenrouter: 'OpenRouter Whisper',
|
||||
asrXiaomiMimo: '小米 MiMo ASR',
|
||||
asrIflytek: '訊飛即時語音轉寫',
|
||||
asrElevenLabs: 'ElevenLabs Scribe',
|
||||
asrSherpaOnnxLocal: '本地 sherpa-onnx(實驗性)',
|
||||
asrFoundryLocalWhisper: '本地 Whisper(Foundry Local)',
|
||||
@@ -875,6 +876,9 @@ export const zhTW: typeof zhCN = {
|
||||
volcengineAuthModeApiKey: '方舟 API Key',
|
||||
volcengineMappingNote: 'Secret Key 當前無需填寫。Resource ID 默認使用 volc.seedasr.sauc.duration。',
|
||||
volcengineApiKeyNote: '使用火山方舟語音模型的 API Key 鑑權,無需 APP ID。Resource ID 預設使用 volc.seedasr.sauc.duration。',
|
||||
xfyunAppIdLabel: 'AppID',
|
||||
xfyunApiKeyLabel: 'API Key',
|
||||
xfyunNote: '在訊飛開放平台「即時語音轉寫」服務頁取得 AppID 與 API Key。音訊為 16kHz/16bit/單聲道 PCM;標準版介面暫不支援熱詞參數(可在訊飛控制台設定個人化熱詞),語種預設中文普通話。',
|
||||
localAsrActiveNotice: '當前已啓用「{{name}}」,可在「高級」中切換或停用。',
|
||||
localAsrTakeoverHint: '啓動「{{name}}」後,ASR 提供商將被接管。',
|
||||
asrProviderTakenOver: 'ASR 提供商已被接管',
|
||||
|
||||
@@ -588,6 +588,27 @@ export function ProvidersSection({ kind = 'all' }: ProvidersSectionProps = {}) {
|
||||
: t('settings.providers.volcengineMappingNote')}
|
||||
</div>
|
||||
</>
|
||||
) : committedAsrProvider === 'iflytek' ? (
|
||||
<>
|
||||
<CredentialField
|
||||
key={`${committedAsrProvider}:app_id`}
|
||||
label={t('settings.providers.xfyunAppIdLabel')}
|
||||
account="xfyun.app_id"
|
||||
provider={committedAsrProvider}
|
||||
mono
|
||||
/>
|
||||
<CredentialField
|
||||
key={`${committedAsrProvider}:api_key`}
|
||||
label={t('settings.providers.xfyunApiKeyLabel')}
|
||||
account="xfyun.api_key"
|
||||
provider={committedAsrProvider}
|
||||
mono
|
||||
mask
|
||||
/>
|
||||
<div style={{ marginTop: 2, fontSize: 11.5, color: 'var(--ol-ink-4)', lineHeight: 1.6 }}>
|
||||
{t('settings.providers.xfyunNote')}
|
||||
</div>
|
||||
</>
|
||||
) : committedAsrProvider === 'local-qwen3' || committedAsrProvider === 'foundry-local-whisper' || committedAsrProvider === 'sherpa-onnx-local' || committedAsrProvider === 'apple-speech' ? (
|
||||
// 用户已经在用本地 ASR——dropdown 行的 asrProviderTakenOver 已经把
|
||||
// "在高级中切换或禁用"讲清楚了,body 不再重复。
|
||||
|
||||
@@ -238,6 +238,9 @@ export const ASR_PRESETS = [
|
||||
// 小米 MiMo ASR 按官方文档走 /chat/completions + input_audio,不是
|
||||
// Whisper /audio/transcriptions;后端由 asr/mimo.rs 专用 client 处理。
|
||||
{ id: 'xiaomi-mimo-asr', nameKey: 'asrXiaomiMimo', baseUrl: 'https://api.xiaomimimo.com/v1', model: 'mimo-v2.5-asr' },
|
||||
// 讯飞实时语音转写(RTASR)流式:无 baseUrl/model 配置,凭据是 AppID + APIKey
|
||||
// 两字段(asr/xfyun.rs);音频 16k/16bit/mono,与 recorder 输出一致。
|
||||
{ id: 'iflytek', nameKey: 'asrIflytek', baseUrl: '', model: '' },
|
||||
{ id: 'foundry-local-whisper', nameKey: 'asrFoundryLocalWhisper', baseUrl: '', model: '' },
|
||||
// 本地引擎(Foundry / sherpa-onnx / Qwen3):无 baseUrl/model 配置,
|
||||
// 模型在「高级 → 本地模型」里下载与切换。
|
||||
|
||||
Reference in new issue
Block a user