feat(asr): 接入 ZenMux 语音转写(JSON + base64)作为可选云端 ASR(#837) (#915)

* feat(asr): 接入 ZenMux 语音转写(JSON + base64)作为可选云端 ASR(#837)

- WhisperBatchASR 新增 ZenMuxJson 请求格式:application/json + base64
  input_audio,language 跟随工作语言(ISO 639-1 映射),enable_itn 可配置
- 新增 zenmux 命名预设:默认 https://zenmux.ai/api/v1 + qwen/qwen3-asr-flash,
  复用 Whisper 兼容路由与 30s 分片;模型 slug 自由输入
- 设置页新增 enable_itn 开关(per-provider 高级配置,默认开启)
- 补齐五语言 i18n、前端单测与 README

* fix(asr): ZenMux「测试连接」按 JSON 协议验证 + 词典不生效说明(#837)

- validate_asr_transcription 按 provider 真实请求格式发探针:OpenRouter /
  ZenMux 走 JSON+base64,其余 whisper 兼容厂商走 multipart,避免假阴性
- 设置页对 zenmux 预设增加说明:JSON 协议不发送词典热词(prompt/hotwords),
  词典仍进入润色链路但不参与识别偏置;README 同步标注例外
- coordinator 再导出 whisper_request_format 供验证路径复用
This commit is contained in:
Chris233 authored and GitHub committed 2026-08-04 23:37:09 +08:00
1 parent d31cafd2d8
commit 5f9586f739
17 files changed
+555 -89

No files matched your search

+2 -2
View File
@@ -180,7 +180,7 @@ Every item below is one more layer sedimented into a default — a capability yo
- 🎨 **Style Pack Marketplace** — browse, install, and like community **style packs** from the in-app Marketplace, and publish your own (custom system prompt per pack, switchable by hotkey). Backed by a moderated marketplace backend; uploads are reviewed before they go public. - 🎨 **Style Pack Marketplace** — browse, install, and like community **style packs** from the in-app Marketplace, and publish your own (custom system prompt per pack, switchable by hotkey). Backed by a moderated marketplace backend; uploads are reviewed before they go public.
- ⚡ **Streaming insertion** — polished text is written to the cursor character by character to reduce perceived latency, with an automatic one-shot-paste fallback. Toggle in Settings → Recording. - ⚡ **Streaming insertion** — polished text is written to the cursor character by character to reduce perceived latency, with an automatic one-shot-paste fallback. Toggle in Settings → Recording.
- **Toggle and push-to-talk** recording modes, plus a **MediaPlayPause trigger** so wired-earbud inline controls can start and stop recording. `Esc` cancels at any phase, including polish and insertion. - **Toggle and push-to-talk** recording modes, plus a **MediaPlayPause trigger** so wired-earbud inline controls can start and stop recording. `Esc` cancels at any phase, including polish and insertion.
- **Cloud ASR**: Volcengine streaming ASR (bigasr), iFlytek realtime ASR (RTASR), Alibaba Cloud Bailian (classic realtime / Qwen3 realtime / Fun-ASR-Flash file transcription), StepFun StepAudio (batch + realtime), Zhipu GLM-ASR, Xiaomi MiMo ASR, ElevenLabs Scribe, OpenAI-compatible batch transcription (OpenAI Whisper / Groq / SiliconFlow SenseVoice / OpenRouter), and Apple Speech (macOS). - **Cloud ASR**: Volcengine streaming ASR (bigasr), iFlytek realtime ASR (RTASR), Alibaba Cloud Bailian (classic realtime / Qwen3 realtime / Fun-ASR-Flash file transcription), StepFun StepAudio (batch + realtime), Zhipu GLM-ASR, Xiaomi MiMo ASR, ElevenLabs Scribe, OpenAI-compatible batch transcription (OpenAI Whisper / Groq / SiliconFlow SenseVoice / OpenRouter / ZenMux), and Apple Speech (macOS).
- **Local ASR**: bundled Qwen3-ASR (0.6B / 1.7B) via vendored `Open-Less/qwen-asr` (macOS); Windows Foundry Local Whisper and sherpa-onnx (experimental) variants. - **Local ASR**: bundled Qwen3-ASR (0.6B / 1.7B) via vendored `Open-Less/qwen-asr` (macOS); Windows Foundry Local Whisper and sherpa-onnx (experimental) variants.
- **Polish providers**: Ark (Volcengine), DeepSeek, OpenAI, Google Gemini, Codex OAuth, SiliconFlow, Atlas Cloud, Xiaomi MiMo, CometAPI, OpenRouter, Alibaba Cloud Coding Plan, CodingPlanX, MiniMax, and StepFun — plus any OpenAI-compatible endpoint you bring. - **Polish providers**: Ark (Volcengine), DeepSeek, OpenAI, Google Gemini, Codex OAuth, SiliconFlow, Atlas Cloud, Xiaomi MiMo, CometAPI, OpenRouter, Alibaba Cloud Coding Plan, CodingPlanX, MiniMax, and StepFun — plus any OpenAI-compatible endpoint you bring.
- **Four output modes**: raw, light polish, structured (**AI-prompt mode**), and formal. Plus a **translation hotkey** that converts speech directly into the configured target language ([#43](../../issues/43)). - **Four output modes**: raw, light polish, structured (**AI-prompt mode**), and formal. Plus a **translation hotkey** that converts speech directly into the configured target language ([#43](../../issues/43)).
@@ -338,7 +338,7 @@ Long-term reference rewrites are stored as `raw → polished → rule` triples a
The dictionary handles your proper nouns, product names, names of people, and new words. Today it supports: The dictionary handles your proper nouns, product names, names of people, and new words. Today it supports:
- Manually adding the correct spelling, a category, and notes. You do not need to maintain misspellings or context hints. - Manually adding the correct spelling, a category, and notes. You do not need to maintain misspellings or context hints.
- Enabled entries are sent to the ASR provider that supports hotwords (Volcengine `context.hotwords`, StepFun `hotwords`, Whisper-compatible `prompt`, Bailian vocabulary ID) so they are recognized correctly during transcription. iFlytek realtime ASR has no request-level hotword parameter — configure personalized hotwords in the iFlytek console instead. - Enabled entries are sent to the ASR provider that supports hotwords (Volcengine `context.hotwords`, StepFun `hotwords`, Whisper-compatible `prompt` — except ZenMux, whose JSON protocol does not carry `prompt`/`hotwords`, Bailian vocabulary ID) so they are recognized correctly during transcription. iFlytek realtime ASR has no request-level hotword parameter — configure personalized hotwords in the iFlytek console instead.
- Entries are also injected into the polish prompt: the model decides per sentence whether to substitute. If "Cloud" clearly refers to the AI product `Claude` in context, it is corrected; if it genuinely means cloud computing, it is left as is. - Entries are also injected into the polish prompt: the model decides per sentence whether to substitute. If "Cloud" clearly refers to the AI product `Claude` in context, it is corrected; if it genuinely means cloud computing, it is left as is.
- The app auto-learns candidate corrections such as `Claude`, `ChatGPT`, and `OpenLess` from your history and offers them later. - The app auto-learns candidate corrections such as `Claude`, `ChatGPT`, and `OpenLess` from your history and offers them later.
+2 -2
View File
@@ -180,7 +180,7 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提
- 🎨 **风格包市场**——在应用内的 Marketplace 浏览、安装、点赞社区**风格包**,并发布自己的(每个包一套自定义系统提示词,可用快捷键切换)。由经过审核的市场后端支撑;上传内容公开前会经过审核。 - 🎨 **风格包市场**——在应用内的 Marketplace 浏览、安装、点赞社区**风格包**,并发布自己的(每个包一套自定义系统提示词,可用快捷键切换)。由经过审核的市场后端支撑;上传内容公开前会经过审核。
- ⚡ **流式插入**——润色后的文本逐字符写入光标以降低感知延迟,并带有自动的一次性粘贴回退。可在 设置 → 录音 中切换。 - ⚡ **流式插入**——润色后的文本逐字符写入光标以降低感知延迟,并带有自动的一次性粘贴回退。可在 设置 → 录音 中切换。
- **切换式与按住说话(push-to-talk)** 两种录音模式,外加 **MediaPlayPause 触发**,让有线耳机的线控也能开始 / 停止录音。`Esc` 可在任意阶段取消,包括润色与插入。 - **切换式与按住说话(push-to-talk)** 两种录音模式,外加 **MediaPlayPause 触发**,让有线耳机的线控也能开始 / 停止录音。`Esc` 可在任意阶段取消,包括润色与插入。
- **云端 ASR**:Volcengine 流式 ASR(bigasr)、讯飞实时语音转写(RTASR)、阿里云百炼(经典实时 / Qwen3 实时 / Fun-ASR-Flash 录音文件)、阶跃星辰 StepAudio(批式 + 实时)、智谱 GLM-ASR、小米 MiMo ASR、ElevenLabs Scribe、OpenAI 兼容批量转写(OpenAI Whisper / Groq / 硅基流动 SenseVoice / OpenRouter),以及 Apple Speech(macOS)。 - **云端 ASR**:Volcengine 流式 ASR(bigasr)、讯飞实时语音转写(RTASR)、阿里云百炼(经典实时 / Qwen3 实时 / Fun-ASR-Flash 录音文件)、阶跃星辰 StepAudio(批式 + 实时)、智谱 GLM-ASR、小米 MiMo ASR、ElevenLabs Scribe、OpenAI 兼容批量转写(OpenAI Whisper / Groq / 硅基流动 SenseVoice / OpenRouter / ZenMux),以及 Apple Speech(macOS)。
- **本地 ASR**:通过 vendored 的 `Open-Less/qwen-asr` 内置 Qwen3-ASR(0.6B / 1.7B)(macOS);Windows 上的 Foundry Local Whisper 与 sherpa-onnx(实验性)变体。 - **本地 ASR**:通过 vendored 的 `Open-Less/qwen-asr` 内置 Qwen3-ASR(0.6B / 1.7B)(macOS);Windows 上的 Foundry Local Whisper 与 sherpa-onnx(实验性)变体。
- **润色提供方**:Ark(火山方舟)、DeepSeek、OpenAI、Google Gemini、Codex OAuth、硅基流动、Atlas Cloud、小米 MiMo、CometAPI、OpenRouter、阿里云 Coding Plan、CodingPlanX、MiniMax、StepFun,以及你自带的任意 OpenAI 兼容端点。 - **润色提供方**:Ark(火山方舟)、DeepSeek、OpenAI、Google Gemini、Codex OAuth、硅基流动、Atlas Cloud、小米 MiMo、CometAPI、OpenRouter、阿里云 Coding Plan、CodingPlanX、MiniMax、StepFun,以及你自带的任意 OpenAI 兼容端点。
- **四种输出模式**:原文、轻度润色、结构化(**AI 提示词模式**)、正式。另有一个**翻译快捷键**,将语音直接转换为所配置的目标语言([#43](../../issues/43))。 - **四种输出模式**:原文、轻度润色、结构化(**AI 提示词模式**)、正式。另有一个**翻译快捷键**,将语音直接转换为所配置的目标语言([#43](../../issues/43))。
@@ -192,7 +192,7 @@ OpenLess 只做一件事:**把语音变成可用的书面文字(尤其是 AI 提
- **Beta 频道(可选加入)**——设置 → 关于 → 加入 Beta 频道,可下载最新预发布版本进行手动安装。Beta 版本绝不会自动推送给 Stable 用户(见[贡献流程](#贡献流程))。 - **Beta 频道(可选加入)**——设置 → 关于 → 加入 Beta 频道,可下载最新预发布版本进行手动安装。Beta 版本绝不会自动推送给 Stable 用户(见[贡献流程](#贡献流程))。
- **分发渠道**——从 [Releases](../../releases) 直接下载 DMG/EXE、Homebrew Cask(`brew install --cask openless`)、Windows 安装包。 - **分发渠道**——从 [Releases](../../releases) 直接下载 DMG/EXE、Homebrew Cask(`brew install --cask openless`)、Windows 安装包。
- **单实例锁**——防止两个 OpenLess 进程争抢同一个快捷键边沿。 - **单实例锁**——防止两个 OpenLess 进程争抢同一个快捷键边沿。
- 词典条目注入到支持热词的 ASR 提供方(Volcengine 的 `context.hotwords`、StepFun 的 `hotwords`、Whisper 兼容的 `prompt`、百炼的 vocabulary_id),并在润色时作为语义提示;命中次数按会话累计。讯飞实时语音转写标准版没有请求级热词参数,需在讯飞控制台配置个性化热词。 - 词典条目注入到支持热词的 ASR 提供方(Volcengine 的 `context.hotwords`、StepFun 的 `hotwords`、Whisper 兼容的 `prompt`(ZenMux 除外——其 JSON 协议不携带 `prompt`/`hotwords`)、百炼的 vocabulary_id),并在润色时作为语义提示;命中次数按会话累计。讯飞实时语音转写标准版没有请求级热词参数,需在讯飞控制台配置个性化热词。
- 平台原生全局快捷键:macOS 上为 CGEventTap,Windows 上为低级键盘钩子(`WH_KEYBOARD_LL`)。 - 平台原生全局快捷键:macOS 上为 CGEventTap,Windows 上为低级键盘钩子(`WH_KEYBOARD_LL`)。
## 下载与安装(终端用户) ## 下载与安装(终端用户)
@@ -22,17 +22,28 @@ pub const PROMPT_CHAR_BUDGET: usize = 240;
/// 区切り文字(ASCII)。Whisper のトークナイザはどの言語でも安定して扱える。 /// 区切り文字(ASCII)。Whisper のトークナイザはどの言語でも安定して扱える。
const PROMPT_SEPARATOR: &str = ", "; const PROMPT_SEPARATOR: &str = ", ";
/// ZenMux 聚合平台的默认端点与模型(issue #837)。与前端 `ASR_PRESETS` 的
/// `zenmux` 条目保持一致;`read_whisper_credentials` 在 active 为 zenmux 且
/// 用户未填时回退到这里。
pub const ZENMUX_DEFAULT_ENDPOINT: &str = "https://zenmux.ai/api/v1";
pub const ZENMUX_DEFAULT_MODEL: &str = "qwen/qwen3-asr-flash";
/// `/audio/transcriptions` 请求体编码方式。 /// `/audio/transcriptions` 请求体编码方式。
/// ///
/// OpenAI 官方及多数兼容厂商用 `multipart/form-data`(file + model)。 /// OpenAI 官方及多数兼容厂商用 `multipart/form-data`(file + model)。
/// OpenRouter 虽路径相同、也走 Bearer,但请求体是 `application/json`: /// OpenRouter 虽路径相同、也走 Bearer,但请求体是 `application/json`:
/// `{model, input_audio:{data:<base64 wav>, format:"wav"}}`(issue #582)。 /// `{model, input_audio:{data:<base64 wav>, format:"wav"}}`(issue #582)。
/// ZenMux 与 OpenRouter 同形(issue #837),另支持可选的 `language` 与
/// `enable_itn`(数字归一化)字段,故单独一个变体,避免给 OpenRouter
/// 的请求体塞未知字段(部分实现会对未知字段 4xx)。
#[derive(Clone, Copy, PartialEq, Eq, Debug)] #[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum AsrRequestFormat { pub enum AsrRequestFormat {
/// `multipart/form-data`(既有行为,默认)。 /// `multipart/form-data`(既有行为,默认)。
Multipart, Multipart,
/// OpenRouter `application/json` + base64 音频。 /// OpenRouter `application/json` + base64 音频。
OpenRouterJson, OpenRouterJson,
/// ZenMux `application/json` + base64 音频(`language` 可选、`enable_itn` 恒发)。
ZenMuxJson,
} }
pub struct WhisperBatchASR { pub struct WhisperBatchASR {
@@ -52,6 +63,12 @@ pub struct WhisperBatchASR {
verbose_json: bool, verbose_json: bool,
/// 请求体编码方式。默认 `Multipart`,OpenRouter 走 `OpenRouterJson`。 /// 请求体编码方式。默认 `Multipart`,OpenRouter 走 `OpenRouterJson`。
request_format: AsrRequestFormat, request_format: AsrRequestFormat,
/// ZenMux 的 `language` 字段(ISO 639-1,如 `zh`)。None = 不发送,服务端
/// 自动检测。仅 `ZenMuxJson` 编码下生效。
language: Option<String>,
/// ZenMux 的 `enable_itn` 字段(数字/单位归一化)。默认 true,与 ZenMux
/// 文档示例及中文 ASR 预期一致;仅 `ZenMuxJson` 编码下生效。
enable_itn: bool,
/// 一等 `hotwords` 参数(JSON 数组字符串)。StepFun 等厂商不认 `prompt` /// 一等 `hotwords` 参数(JSON 数组字符串)。StepFun 等厂商不认 `prompt`
/// (静默忽略),但提供专门的热词字段——用它词典才真正生效。空 = 不发。 /// (静默忽略),但提供专门的热词字段——用它词典才真正生效。空 = 不发。
hotwords: Vec<String>, hotwords: Vec<String>,
@@ -75,6 +92,8 @@ impl WhisperBatchASR {
max_chunk_duration_ms, max_chunk_duration_ms,
verbose_json, verbose_json,
request_format: AsrRequestFormat::Multipart, request_format: AsrRequestFormat::Multipart,
language: None,
enable_itn: true,
hotwords: Vec::new(), hotwords: Vec::new(),
buffer: Mutex::new(Vec::new()), buffer: Mutex::new(Vec::new()),
} }
@@ -94,6 +113,20 @@ impl WhisperBatchASR {
self self
} }
/// 设置 ZenMux 的 `language` 字段(ISO 639-1 码)。None = 不发送(自动检测)。
/// 仅 `ZenMuxJson` 编码下生效。
pub fn with_language(mut self, language: Option<String>) -> Self {
self.language = language;
self
}
/// 设置 ZenMux 的 `enable_itn`(数字归一化)开关,默认 true。仅 `ZenMuxJson`
/// 编码下生效。
pub fn with_enable_itn(mut self, enable_itn: bool) -> Self {
self.enable_itn = enable_itn;
self
}
/// Stop collecting audio, encode the buffer as WAV, and POST to the /// Stop collecting audio, encode the buffer as WAV, and POST to the
/// Whisper transcriptions endpoint. /// Whisper transcriptions endpoint.
/// ///
@@ -218,6 +251,29 @@ impl WhisperBatchASR {
} }
request.json(&body) request.json(&body)
} }
AsrRequestFormat::ZenMuxJson => {
// ZenMux /audio/transcriptions(issue #837):application/json,
// 音频走标准 base64。语言跟随 OpenLess 工作语言映射;enable_itn
// 由设置页开关决定,恒显式发送。不带 multipart 专属字段。
let mut body = serde_json::json!({
"model": self.model,
"input_audio": {
"data": base64::engine::general_purpose::STANDARD.encode(&wav),
"format": "wav",
},
"enable_itn": self.enable_itn,
});
if let Some(language) = self.language.as_ref() {
if !language.trim().is_empty() {
body["language"] = serde_json::Value::String(language.trim().to_string());
}
}
let mut request = client.post(&url);
if !self.api_key.trim().is_empty() {
request = request.header("Authorization", format!("Bearer {}", self.api_key));
}
request.json(&body)
}
}; };
let resp = request let resp = request
@@ -858,6 +914,118 @@ mod tests {
server.join().unwrap(); server.join().unwrap();
} }
#[tokio::test]
async fn zenmux_format_posts_json_with_language_and_itn() {
// issue #837:ZenMuxJson 走 application/json + input_audio.data(base64),
// 语言有映射时发送、enable_itn 恒显式发送;响应仍按 {text} 解析。
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
listener.set_nonblocking(true).unwrap();
let addr = listener.local_addr().unwrap();
let server = thread::spawn(move || {
let deadline = Instant::now() + Duration::from_secs(5);
let mut stream = loop {
match listener.accept() {
Ok((stream, _)) => break stream,
Err(err) if err.kind() == std::io::ErrorKind::WouldBlock => {
assert!(
Instant::now() < deadline,
"timed out waiting for ASR test request"
);
thread::sleep(Duration::from_millis(10));
}
Err(err) => panic!("accept ASR test request failed: {err}"),
}
};
stream.set_nonblocking(false).unwrap();
stream
.set_read_timeout(Some(Duration::from_secs(5)))
.unwrap();
let request = read_http_request(&mut stream);
let request_text = String::from_utf8_lossy(&request);
let lower = request_text.to_ascii_lowercase();
assert!(request_text.starts_with("POST /audio/transcriptions HTTP/1.1"));
assert!(lower.contains("content-type: application/json"));
assert!(lower.contains("authorization: bearer key"));
assert!(request_text.contains("input_audio"));
assert!(request_text.contains(r#""format":"wav""#));
assert!(request_text.contains(r#""language":"zh""#));
assert!(request_text.contains(r#""enable_itn":true"#));
assert!(!lower.contains("multipart/form-data"));
write_json_response(&mut stream, r#"{"text":"zenmux ok"}"#);
});
let base_url = format!("http://{}", addr);
let asr = WhisperBatchASR::new(
"key".to_string(),
base_url,
"qwen/qwen3-asr-flash".to_string(),
None,
Some(30_000),
false,
)
.with_request_format(AsrRequestFormat::ZenMuxJson)
.with_language(Some("zh".to_string()))
.with_enable_itn(true);
let pcm = vec![0u8; 32_000 * 2];
asr.consume_pcm_chunk(&pcm);
let transcript = asr.transcribe().await.unwrap();
assert_eq!(transcript.text, "zenmux ok");
server.join().unwrap();
}
#[tokio::test]
async fn zenmux_format_omits_language_when_unset_and_sends_false_itn() {
// language 无映射(None)时不发送该字段;enable_itn=false 显式发送 false。
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
listener.set_nonblocking(true).unwrap();
let addr = listener.local_addr().unwrap();
let server = thread::spawn(move || {
let deadline = Instant::now() + Duration::from_secs(5);
let mut stream = loop {
match listener.accept() {
Ok((stream, _)) => break stream,
Err(err) if err.kind() == std::io::ErrorKind::WouldBlock => {
assert!(
Instant::now() < deadline,
"timed out waiting for ASR test request"
);
thread::sleep(Duration::from_millis(10));
}
Err(err) => panic!("accept ASR test request failed: {err}"),
}
};
stream.set_nonblocking(false).unwrap();
stream
.set_read_timeout(Some(Duration::from_secs(5)))
.unwrap();
let request = read_http_request(&mut stream);
let request_text = String::from_utf8_lossy(&request);
assert!(request_text.contains(r#""enable_itn":false"#));
assert!(!request_text.contains(r#""language""#));
write_json_response(&mut stream, r#"{"text":"zenmux no-lang ok"}"#);
});
let base_url = format!("http://{}", addr);
let asr = WhisperBatchASR::new(
"key".to_string(),
base_url,
"qwen/qwen3-asr-flash".to_string(),
None,
None,
false,
)
.with_request_format(AsrRequestFormat::ZenMuxJson)
.with_language(None)
.with_enable_itn(false);
let pcm = vec![0u8; 32_000 * 2];
asr.consume_pcm_chunk(&pcm);
let transcript = asr.transcribe().await.unwrap();
assert_eq!(transcript.text, "zenmux no-lang ok");
server.join().unwrap();
}
#[tokio::test] #[tokio::test]
async fn empty_api_key_omits_authorization_header() { async fn empty_api_key_omits_authorization_header() {
// openai-compatible 允许 API Key 留空(LAN 无鉴权端点):请求不得携带 // openai-compatible 允许 API Key 留空(LAN 无鉴权端点):请求不得携带
@@ -1,4 +1,5 @@
use super::*; use super::*;
use base64::Engine;
use std::collections::HashMap; use std::collections::HashMap;
#[derive(Serialize)] #[derive(Serialize)]
@@ -304,7 +305,10 @@ async fn validate_asr_provider() -> Result<(), String> {
.map_err(|e| e.to_string())? .map_err(|e| e.to_string())?
.filter(|s| !s.trim().is_empty()) .filter(|s| !s.trim().is_empty())
.ok_or_else(|| "asrModelMissing".to_string())?; .ok_or_else(|| "asrModelMissing".to_string())?;
validate_asr_transcription(&config, model.trim()).await // 验证请求体与真实转写保持一致:OpenRouter / ZenMux 走 JSON+base64,
// 其余 whisper 兼容厂商走 multipart——避免「测试连接」假阴性(issue #837)。
let request_format = crate::coordinator::whisper_request_format(&active_asr);
validate_asr_transcription(&config, model.trim(), request_format).await
} }
/// 讯飞 RTASR 验证:真连 + 500ms 静音 + 收尾。鉴权错误(10105 / 10110)在握手阶段 /// 讯飞 RTASR 验证:真连 + 500ms 静音 + 收尾。鉴权错误(10105 / 10110)在握手阶段
@@ -676,7 +680,11 @@ pub(crate) fn active_sherpa_asr_is_supported(provider: &str) -> bool {
} }
} }
async fn validate_asr_transcription(config: &ProviderConfig, model: &str) -> Result<(), String> { async fn validate_asr_transcription(
config: &ProviderConfig,
model: &str,
request_format: crate::asr::whisper::AsrRequestFormat,
) -> Result<(), String> {
const MAX_ASR_VALIDATE_BODY_BYTES: usize = 1024 * 1024; const MAX_ASR_VALIDATE_BODY_BYTES: usize = 1024 * 1024;
const MAX_ATTEMPTS: u32 = 6; const MAX_ATTEMPTS: u32 = 6;
let url = asr_transcriptions_url(&config.base_url)?; let url = asr_transcriptions_url(&config.base_url)?;
@@ -689,20 +697,57 @@ async fn validate_asr_transcription(config: &ProviderConfig, model: &str) -> Res
let mut attempt: u32 = 0; let mut attempt: u32 = 0;
let response = loop { let response = loop {
attempt += 1; attempt += 1;
let wav_part = reqwest::multipart::Part::bytes(wav.clone()) let request = match request_format {
.file_name("openless-asr-check.wav") crate::asr::whisper::AsrRequestFormat::Multipart => {
.mime_str("audio/wav") let wav_part = reqwest::multipart::Part::bytes(wav.clone())
.map_err(|e| format!("请求体构建失败: {e}"))?; .file_name("openless-asr-check.wav")
let form = reqwest::multipart::Form::new() .mime_str("audio/wav")
.part("file", wav_part) .map_err(|e| format!("请求体构建失败: {e}"))?;
.text("model", model.to_string()); let form = reqwest::multipart::Form::new()
// `openai-compatible` 允许 API Key 留空:此时不带 Authorization 头, .part("file", wav_part)
// 避免空 Bearer 被服务端 401 拒绝(与 fetch_provider_models 一致)。 .text("model", model.to_string());
let mut request = client.post(&url); let mut request = client.post(&url);
if !config.api_key.trim().is_empty() { if !config.api_key.trim().is_empty() {
request = request.header("Authorization", format!("Bearer {}", config.api_key)); request = request.header("Authorization", format!("Bearer {}", config.api_key));
} }
match request.multipart(form).send().await { request.multipart(form)
}
crate::asr::whisper::AsrRequestFormat::OpenRouterJson => {
// OpenRouter:application/json + base64(issue #582),与真实
// 转写请求同形;不带 multipart 专属字段。
let body = serde_json::json!({
"model": model,
"input_audio": {
"data": base64::engine::general_purpose::STANDARD.encode(&wav),
"format": "wav",
},
});
let mut request = client.post(&url);
if !config.api_key.trim().is_empty() {
request = request.header("Authorization", format!("Bearer {}", config.api_key));
}
request.json(&body)
}
crate::asr::whisper::AsrRequestFormat::ZenMuxJson => {
// ZenMux:application/json + base64,enable_itn 与真实请求默认
// 一致(true),language 留空走服务端自动检测(issue #837)。
let body = serde_json::json!({
"model": model,
"input_audio": {
"data": base64::engine::general_purpose::STANDARD.encode(&wav),
"format": "wav",
},
"enable_itn": true,
});
let mut request = client.post(&url);
if !config.api_key.trim().is_empty() {
request = request.header("Authorization", format!("Bearer {}", config.api_key));
}
request.json(&body)
}
};
match request.send().await
{
Ok(resp) => break resp, Ok(resp) => break resp,
Err(e) if e.is_timeout() => return Err("providerRequestTimeout".to_string()), Err(e) if e.is_timeout() => return Err("providerRequestTimeout".to_string()),
Err(e) if (e.is_connect() || e.is_request()) && attempt < MAX_ATTEMPTS => { Err(e) if (e.is_connect() || e.is_request()) && attempt < MAX_ATTEMPTS => {
+91 -13
View File
@@ -77,6 +77,9 @@ mod silence_auto_stop;
pub(crate) mod selection_polish; pub(crate) mod selection_polish;
use asr_wiring::*; use asr_wiring::*;
// providers.rs 的 ASR 验证路径按 provider 的真实请求格式发送探针(issue #837),
// 需要跨模块访问 whisper 兼容系的格式映射,显式再导出。
pub(crate) use asr_wiring::whisper_request_format;
use capsule_focus::*; use capsule_focus::*;
use hotkey_loops::*; use hotkey_loops::*;
use polish_flow::*; use polish_flow::*;
@@ -444,9 +447,9 @@ pub(crate) fn derive_bailian_endpoint(
fn batch_asr_chunk_limit_ms(provider_id: &str) -> Option<u64> { fn batch_asr_chunk_limit_ms(provider_id: &str) -> Option<u64> {
match provider_id { match provider_id {
// OpenRouter 把音频 base64 进 JSON body,体积比二进制大 ~33%,长录音易撞 // OpenRouter / ZenMux 把音频 base64 进 JSON body,体积比二进制大 ~33%,
// body/时长上限,保守按 30s 切分(与 zhipu 同)。 // 长录音易撞 body/时长上限,保守按 30s 切分(与 zhipu 同)。
"zhipu" | "openrouter" => Some(30_000), "zhipu" | "openrouter" | "zenmux" => Some(30_000),
// 其余预设默认不分片;openai-compatible 可由用户高级配置覆盖。 // 其余预设默认不分片;openai-compatible 可由用户高级配置覆盖。
_ => read_advanced_asr_config(provider_id).chunk_duration_ms, _ => read_advanced_asr_config(provider_id).chunk_duration_ms,
} }
@@ -457,13 +460,30 @@ fn batch_asr_chunk_limit_ms(provider_id: &str) -> Option<u64> {
/// 与分片时长由用户按 provider 配置(存凭据 vault)。 /// 与分片时长由用户按 provider 配置(存凭据 vault)。
pub(crate) const OPENAI_COMPATIBLE_ASR_PROVIDER_ID: &str = "openai-compatible"; pub(crate) const OPENAI_COMPATIBLE_ASR_PROVIDER_ID: &str = "openai-compatible";
/// `openai-compatible` 预设的高级配置(per-provider 存于凭据 vault)。 /// ZenMux ASR 预设 id(issue #837)。与前端 `ASR_PRESETS` 的 `zenmux` 条目一致,
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] /// 复用 Whisper 批式管线,但请求体走 JSON + base64(`ZenMuxJson`)。
pub(crate) const ZENMUX_ASR_PROVIDER_ID: &str = "zenmux";
/// `openai-compatible` 与 `zenmux` 预设的高级配置(per-provider 存于凭据 vault)。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) struct AdvancedAsrConfig { pub(crate) struct AdvancedAsrConfig {
/// 是否请求 `response_format=verbose_json`(配合幻听过滤;服务端不支持时保持 false)。 /// 是否请求 `response_format=verbose_json`(配合幻听过滤;服务端不支持时保持 false)。
pub(crate) verbose_json: bool, pub(crate) verbose_json: bool,
/// 单次请求的音频分片时长;None = 不分片整段发送。 /// 单次请求的音频分片时长;None = 不分片整段发送。
pub(crate) chunk_duration_ms: Option<u64>, pub(crate) chunk_duration_ms: Option<u64>,
/// ZenMux `enable_itn`(数字/单位归一化)。默认 true,与 ZenMux 文档示例及
/// 中文 ASR 预期一致;仅 zenmux 消费。
pub(crate) enable_itn: bool,
}
impl Default for AdvancedAsrConfig {
fn default() -> Self {
Self {
verbose_json: false,
chunk_duration_ms: None,
enable_itn: true,
}
}
} }
/// 解析 per-provider 高级配置 JSON;缺失/非法一律回落保守默认。 /// 解析 per-provider 高级配置 JSON;缺失/非法一律回落保守默认。
@@ -492,20 +512,25 @@ fn parse_advanced_asr_config(raw: Option<&str>) -> AdvancedAsrConfig {
.map(|ms| ms.floor() as u64) .map(|ms| ms.floor() as u64)
}) })
}), }),
// 缺失/非布尔回落默认 true(开启),与前端 advancedAsrConfig.ts 一致。
enable_itn: value
.get("enableItn")
.and_then(|v| v.as_bool())
.unwrap_or(true),
} }
} }
/// 按 provider id 决定高级配置:仅 `openai-compatible` 采用用户配置, /// 按 provider id 决定高级配置:`openai-compatible` 与 `zenmux` 采用用户配置,
/// 命名厂商一律返回默认(保持硬编码行为)。 /// 其余命名厂商一律返回默认(保持硬编码行为)。
fn advanced_asr_config_for(provider_id: &str, raw: Option<&str>) -> AdvancedAsrConfig { fn advanced_asr_config_for(provider_id: &str, raw: Option<&str>) -> AdvancedAsrConfig {
if provider_id != OPENAI_COMPATIBLE_ASR_PROVIDER_ID { if provider_id != OPENAI_COMPATIBLE_ASR_PROVIDER_ID && provider_id != ZENMUX_ASR_PROVIDER_ID {
return AdvancedAsrConfig::default(); return AdvancedAsrConfig::default();
} }
parse_advanced_asr_config(raw) parse_advanced_asr_config(raw)
} }
/// 读取某 ASR provider 的高级配置。仅 `openai-compatible` 读 vault;命名厂商 /// 读取某 ASR provider 的高级配置。仅 `openai-compatible` / `zenmux` 读 vault;
/// 走硬编码行为(这里返回默认值),避免破坏已测通的路径。 /// 其余命名厂商走硬编码行为(这里返回默认值),避免破坏已测通的路径。
fn read_advanced_asr_config(provider_id: &str) -> AdvancedAsrConfig { fn read_advanced_asr_config(provider_id: &str) -> AdvancedAsrConfig {
let raw = CredentialsVault::get_for_asr_provider( let raw = CredentialsVault::get_for_asr_provider(
provider_id, provider_id,
@@ -2547,18 +2572,35 @@ fn read_whisper_credentials() -> (String, String, String) {
.ok() .ok()
.flatten() .flatten()
.unwrap_or_default(); .unwrap_or_default();
let active_asr = CredentialsVault::get_active_asr();
let (default_endpoint, default_model) = whisper_credential_defaults(&active_asr);
let base_url = CredentialsVault::get(CredentialAccount::AsrEndpoint) let base_url = CredentialsVault::get(CredentialAccount::AsrEndpoint)
.ok() .ok()
.flatten() .flatten()
.unwrap_or_default(); .filter(|s| !s.trim().is_empty())
.unwrap_or(default_endpoint);
let model = CredentialsVault::get(CredentialAccount::AsrModel) let model = CredentialsVault::get(CredentialAccount::AsrModel)
.ok() .ok()
.flatten() .flatten()
.filter(|s| !s.is_empty()) .filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "whisper-1".to_string()); .unwrap_or(default_model);
(api_key, base_url, model) (api_key, base_url, model)
} }
/// whisper 兼容系 provider 的「空槽默认值」。zenmux 有厂商默认端点/模型
/// (与前端 preset 一致);其余预设沿用空 endpoint + `whisper-1`(默认值由
/// 前端切换 provider 时写入 vault)。纯函数,便于单测。
fn whisper_credential_defaults(provider_id: &str) -> (String, String) {
if provider_id == ZENMUX_ASR_PROVIDER_ID {
(
crate::asr::whisper::ZENMUX_DEFAULT_ENDPOINT.to_string(),
crate::asr::whisper::ZENMUX_DEFAULT_MODEL.to_string(),
)
} else {
(String::new(), "whisper-1".to_string())
}
}
fn read_mimo_credentials() -> (String, String, String) { fn read_mimo_credentials() -> (String, String, String) {
let api_key = CredentialsVault::get(CredentialAccount::AsrApiKey) let api_key = CredentialsVault::get(CredentialAccount::AsrApiKey)
.ok() .ok()
@@ -3232,6 +3274,7 @@ mod tests {
AdvancedAsrConfig { AdvancedAsrConfig {
verbose_json: true, verbose_json: true,
chunk_duration_ms: Some(30_000), chunk_duration_ms: Some(30_000),
enable_itn: true,
} }
); );
// 命名厂商忽略该配置,保持硬编码行为。 // 命名厂商忽略该配置,保持硬编码行为。
@@ -3266,6 +3309,7 @@ mod tests {
AdvancedAsrConfig { AdvancedAsrConfig {
verbose_json: true, verbose_json: true,
chunk_duration_ms: None, chunk_duration_ms: None,
enable_itn: true,
} }
); );
// 分片时长 0 或缺失 = 不分片。 // 分片时长 0 或缺失 = 不分片。
@@ -3280,6 +3324,7 @@ mod tests {
AdvancedAsrConfig { AdvancedAsrConfig {
verbose_json: false, verbose_json: false,
chunk_duration_ms: Some(30_000), chunk_duration_ms: Some(30_000),
enable_itn: true,
} }
); );
// 浮点分片时长与前端一致向下取整:30000.9 → 30000。 // 浮点分片时长与前端一致向下取整:30000.9 → 30000。
@@ -3288,6 +3333,7 @@ mod tests {
AdvancedAsrConfig { AdvancedAsrConfig {
verbose_json: false, verbose_json: false,
chunk_duration_ms: Some(30_000), chunk_duration_ms: Some(30_000),
enable_itn: true,
} }
); );
// 负数 / 字符串分片时长 → 不分片。 // 负数 / 字符串分片时长 → 不分片。
@@ -3322,6 +3368,38 @@ mod tests {
assert_eq!(batch_asr_chunk_limit_ms("openrouter"), Some(30_000)); assert_eq!(batch_asr_chunk_limit_ms("openrouter"), Some(30_000));
} }
#[test]
fn zenmux_credential_defaults_and_advanced_config() {
use crate::asr::whisper::{ZENMUX_DEFAULT_ENDPOINT, ZENMUX_DEFAULT_MODEL};
// base64 进 JSON body 体积膨胀,与 OpenRouter 同按 30s 切分。
assert_eq!(batch_asr_chunk_limit_ms("zenmux"), Some(30_000));
// 空槽默认值:zenmux 回落厂商默认端点/模型(与前端 preset 一致);
// 其余 whisper 兼容预设沿用空 endpoint + whisper-1。
assert_eq!(
whisper_credential_defaults("zenmux"),
(
ZENMUX_DEFAULT_ENDPOINT.to_string(),
ZENMUX_DEFAULT_MODEL.to_string()
)
);
assert_eq!(
whisper_credential_defaults("whisper"),
(String::new(), "whisper-1".to_string())
);
assert_eq!(
whisper_credential_defaults("openrouter"),
(String::new(), "whisper-1".to_string())
);
// enable_itn 默认 true;用户配置显式 false 可覆盖;仅 openai-compatible /
// zenmux 读用户配置,其余命名厂商忽略(保持硬编码行为)。
assert!(advanced_asr_config_for("zenmux", None).enable_itn);
assert!(advanced_asr_config_for("zenmux", Some(r#"{"verboseJson":true}"#)).enable_itn);
assert!(!advanced_asr_config_for("zenmux", Some(r#"{"enableItn":false}"#)).enable_itn);
assert!(advanced_asr_config_for("whisper", Some(r#"{"enableItn":false}"#)).enable_itn);
assert!(!advanced_asr_config_for("zenmux", Some(r#"{"enableItn":false}"#)).verbose_json);
}
#[test] #[test]
fn stepfun_is_whisper_compatible_with_hotwords_vocab() { fn stepfun_is_whisper_compatible_with_hotwords_vocab() {
use crate::asr::whisper::AsrRequestFormat; use crate::asr::whisper::AsrRequestFormat;
@@ -385,7 +385,7 @@ pub(super) fn build_apple_speech(
pub(super) fn is_whisper_compatible_provider(id: &str) -> bool { pub(super) fn is_whisper_compatible_provider(id: &str) -> bool {
matches!( matches!(
id, id,
"whisper" | "siliconflow" | "zhipu" | "groq" | "openrouter" | "stepfun" "whisper" | "siliconflow" | "zhipu" | "groq" | "openrouter" | "stepfun" | "zenmux"
) || id == OPENAI_COMPATIBLE_ASR_PROVIDER_ID ) || id == OPENAI_COMPATIBLE_ASR_PROVIDER_ID
} }
@@ -416,9 +416,11 @@ pub(super) fn whisper_vocab_for_provider(
/// 该 provider 的请求体编码方式。OpenRouter 的 `/audio/transcriptions` 是 /// 该 provider 的请求体编码方式。OpenRouter 的 `/audio/transcriptions` 是
/// `application/json` + base64 音频(issue #582),其余兼容厂商沿用 multipart。 /// `application/json` + base64 音频(issue #582),其余兼容厂商沿用 multipart。
pub(super) fn whisper_request_format(provider_id: &str) -> crate::asr::whisper::AsrRequestFormat { /// ZenMux 同形但带 `language` / `enable_itn`(issue #837),单独走 `ZenMuxJson`。
pub(crate) fn whisper_request_format(provider_id: &str) -> crate::asr::whisper::AsrRequestFormat {
match provider_id { match provider_id {
"openrouter" => crate::asr::whisper::AsrRequestFormat::OpenRouterJson, "openrouter" => crate::asr::whisper::AsrRequestFormat::OpenRouterJson,
"zenmux" => crate::asr::whisper::AsrRequestFormat::ZenMuxJson,
_ => crate::asr::whisper::AsrRequestFormat::Multipart, _ => crate::asr::whisper::AsrRequestFormat::Multipart,
} }
} }
@@ -437,11 +439,61 @@ pub(super) fn whisper_request_format(provider_id: &str) -> crate::asr::whisper::
pub(super) fn whisper_supports_verbose_json(provider_id: &str) -> bool { pub(super) fn whisper_supports_verbose_json(provider_id: &str) -> bool {
match provider_id { match provider_id {
"whisper" | "groq" => true, "whisper" | "groq" => true,
// ZenMux 的 JSON 请求体协议没有 response_format,恒关闭。
"zenmux" => false,
// openai-compatible 由用户高级配置决定;其余厂商保持关闭。 // openai-compatible 由用户高级配置决定;其余厂商保持关闭。
_ => read_advanced_asr_config(provider_id).verbose_json, _ => read_advanced_asr_config(provider_id).verbose_json,
} }
} }
/// OpenLess 工作语言(原生名,见前端 `SUPPORTED_LANGUAGES`)→ ZenMux `language`
/// 字段值(ISO 639-1 码)。取 `working_languages` 主语言映射;未收录的语言返回
/// None —— 请求体省略 `language`,由 ZenMux 服务端自动检测(issue #837)。
pub(super) fn zenmux_language_code(native_name: &str) -> Option<String> {
let code = match native_name.trim() {
"简体中文" | "繁体中文" => "zh",
"English" => "en",
"日本語" => "ja",
"한국어" => "ko",
"Français" => "fr",
"Deutsch" => "de",
"Español" => "es",
"Italiano" => "it",
"Português" => "pt",
"Русский" => "ru",
"العربية" => "ar",
"Tiếng Việt" => "vi",
"ไทย" => "th",
"हिन्दी" => "hi",
_ => return None,
};
Some(code.to_string())
}
/// 当前 prefs 的主工作语言 → ZenMux `language`(None = 不发送,自动检测)。
pub(super) fn zenmux_language_for_prefs(prefs: &crate::types::UserPreferences) -> Option<String> {
prefs
.working_languages
.first()
.and_then(|name| zenmux_language_code(name))
}
/// 构造完的 `WhisperBatchASR` 上注入 zenmux 专属选项(`language` 跟随工作语言、
/// `enable_itn` 读用户高级配置)。非 zenmux 原样返回,保持现有行为;QA 与听写
/// 两处构造点共用,避免重复逻辑。
pub(super) fn apply_zenmux_asr_options(
builder: crate::asr::whisper::WhisperBatchASR,
active_asr: &str,
inner: &Arc<Inner>,
) -> crate::asr::whisper::WhisperBatchASR {
if active_asr != ZENMUX_ASR_PROVIDER_ID {
return builder;
}
builder
.with_language(zenmux_language_for_prefs(&inner.prefs.get()))
.with_enable_itn(read_advanced_asr_config(ZENMUX_ASR_PROVIDER_ID).enable_itn)
}
pub(super) fn is_bailian_provider(id: &str) -> bool { pub(super) fn is_bailian_provider(id: &str) -> bool {
id == crate::asr::bailian::PROVIDER_ID id == crate::asr::bailian::PROVIDER_ID
} }
@@ -750,7 +802,7 @@ pub(super) async fn build_qa_asr_start(
let label = AsrCallLabel::new(effective_asr.clone(), Some(model.clone())); let label = AsrCallLabel::new(effective_asr.clone(), Some(model.clone()));
let (whisper_prompt, hotwords) = let (whisper_prompt, hotwords) =
whisper_vocab_for_provider(active_asr, enabled_phrases(inner)); whisper_vocab_for_provider(active_asr, enabled_phrases(inner));
let whisper = Arc::new( let whisper = Arc::new(apply_zenmux_asr_options(
WhisperBatchASR::new( WhisperBatchASR::new(
api_key, api_key,
base_url, base_url,
@@ -761,7 +813,9 @@ pub(super) async fn build_qa_asr_start(
) )
.with_request_format(whisper_request_format(active_asr)) .with_request_format(whisper_request_format(active_asr))
.with_hotwords(hotwords), .with_hotwords(hotwords),
); active_asr,
inner,
));
let active = ActiveAsr::Whisper(Arc::clone(&whisper)); let active = ActiveAsr::Whisper(Arc::clone(&whisper));
let consumer: Arc<dyn crate::recorder::AudioConsumer> = whisper; let consumer: Arc<dyn crate::recorder::AudioConsumer> = whisper;
Ok((QaAsrStart::Ready { active, consumer }, label)) Ok((QaAsrStart::Ready { active, consumer }, label))
@@ -798,6 +852,47 @@ pub(super) async fn build_qa_asr_start(
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn zenmux_is_whisper_compatible_json_provider() {
use crate::asr::whisper::AsrRequestFormat;
// issue #837:ZenMux 走 whisper 兼容路由,请求体 JSON+base64(ZenMuxJson),
// 与 OpenRouter 共用 30s 切分;JSON 协议不吃 response_format / hotwords。
assert!(is_whisper_compatible_provider("zenmux"));
assert_eq!(
active_asr_provider_kind("zenmux"),
ActiveAsrProviderKind::WhisperCompatible
);
assert_eq!(
whisper_request_format("zenmux"),
AsrRequestFormat::ZenMuxJson
);
assert!(!whisper_supports_verbose_json("zenmux"));
assert!(!whisper_uses_hotwords("zenmux"));
}
#[test]
fn zenmux_language_code_covers_supported_languages_and_omits_unknown() {
// 覆盖前端 SUPPORTED_LANGUAGES 的全部 15 种语言;未收录 → None(自动检测)。
assert_eq!(zenmux_language_code("简体中文").as_deref(), Some("zh"));
assert_eq!(zenmux_language_code("繁体中文").as_deref(), Some("zh"));
assert_eq!(zenmux_language_code("English").as_deref(), Some("en"));
assert_eq!(zenmux_language_code("日本語").as_deref(), Some("ja"));
assert_eq!(zenmux_language_code("한국어").as_deref(), Some("ko"));
assert_eq!(zenmux_language_code("Français").as_deref(), Some("fr"));
assert_eq!(zenmux_language_code("Deutsch").as_deref(), Some("de"));
assert_eq!(zenmux_language_code("Español").as_deref(), Some("es"));
assert_eq!(zenmux_language_code("Italiano").as_deref(), Some("it"));
assert_eq!(zenmux_language_code("Português").as_deref(), Some("pt"));
assert_eq!(zenmux_language_code("Русский").as_deref(), Some("ru"));
assert_eq!(zenmux_language_code("العربية").as_deref(), Some("ar"));
assert_eq!(zenmux_language_code("Tiếng Việt").as_deref(), Some("vi"));
assert_eq!(zenmux_language_code("ไทย").as_deref(), Some("th"));
assert_eq!(zenmux_language_code("हिन्दी").as_deref(), Some("hi"));
assert_eq!(zenmux_language_code(""), None);
assert_eq!(zenmux_language_code("Esperanto"), None);
assert_eq!(zenmux_language_code(" "), None);
}
#[test] #[test]
fn require_openai_compatible_fields_errors_on_missing_endpoint_or_model() { fn require_openai_compatible_fields_errors_on_missing_endpoint_or_model() {
// endpoint 缺失(含纯空白)→ 明确报错,绝不静默回落 whisper-1。 // endpoint 缺失(含纯空白)→ 明确报错,绝不静默回落 whisper-1。
@@ -2153,7 +2153,7 @@ pub(super) async fn begin_session_as(
let (whisper_prompt, hotwords) = let (whisper_prompt, hotwords) =
whisper_vocab_for_provider(&active_asr, enabled_phrases(inner)); whisper_vocab_for_provider(&active_asr, enabled_phrases(inner));
let asr_call_label = AsrCallLabel::new(effective_asr.clone(), Some(model.clone())); let asr_call_label = AsrCallLabel::new(effective_asr.clone(), Some(model.clone()));
let whisper = Arc::new( let whisper = Arc::new(apply_zenmux_asr_options(
WhisperBatchASR::new( WhisperBatchASR::new(
api_key, api_key,
base_url, base_url,
@@ -2164,7 +2164,9 @@ pub(super) async fn begin_session_as(
) )
.with_request_format(whisper_request_format(&active_asr)) .with_request_format(whisper_request_format(&active_asr))
.with_hotwords(hotwords), .with_hotwords(hotwords),
); &active_asr,
inner,
));
store_asr_for_session( store_asr_for_session(
inner, inner,
current_session_id, current_session_id,
+5 -1
View File
@@ -868,6 +868,7 @@ export const en: typeof zhCN = {
asrGroq: 'Groq Whisper-large-v3', asrGroq: 'Groq Whisper-large-v3',
asrWhisper: 'OpenAI Whisper (compatible)', asrWhisper: 'OpenAI Whisper (compatible)',
asrOpenrouter: 'OpenRouter Whisper', asrOpenrouter: 'OpenRouter Whisper',
asrZenmux: 'ZenMux',
asrOpenAiCompatible: 'Custom OpenAI-compatible', asrOpenAiCompatible: 'Custom OpenAI-compatible',
asrXiaomiMimo: 'Xiaomi MiMo ASR', asrXiaomiMimo: 'Xiaomi MiMo ASR',
asrIflytek: 'iFlytek Realtime ASR', asrIflytek: 'iFlytek Realtime ASR',
@@ -878,11 +879,14 @@ export const en: typeof zhCN = {
asrAppleSpeech: 'Apple Speech (macOS)', asrAppleSpeech: 'Apple Speech (macOS)',
}, },
elevenLabsUploadNotice: 'ElevenLabs uploads recorded audio to the configured endpoint for batch transcription.', elevenLabsUploadNotice: 'ElevenLabs uploads recorded audio to the configured endpoint for batch transcription.',
asrAdvancedNote: 'Advanced options below only affect the Custom OpenAI-compatible preset; named provider presets keep their built-in behavior.', zenmuxVocabularyNote: 'ZenMux uses a JSON transcription protocol and does not receive dictionary hotwords (prompt/hotwords); the dictionary still feeds the polish step but does not bias speech recognition.',
asrAdvancedNote: 'Advanced options below only affect the Custom OpenAI-compatible and ZenMux presets; other named provider presets keep their built-in behavior.',
asrAdvancedVerboseJsonLabel: 'Segment metrics (verbose_json)', asrAdvancedVerboseJsonLabel: 'Segment metrics (verbose_json)',
asrAdvancedVerboseJsonHint: 'Requests segment metrics for hallucination filtering when the server supports it; keep off for self-hosted servers that do not.', asrAdvancedVerboseJsonHint: 'Requests segment metrics for hallucination filtering when the server supports it; keep off for self-hosted servers that do not.',
asrAdvancedChunkLabel: 'Chunk duration (ms)', asrAdvancedChunkLabel: 'Chunk duration (ms)',
asrAdvancedChunkHint: '0 = no chunking, send the whole clip at once. Chunked requests suit long recordings or servers with per-request duration limits.', asrAdvancedChunkHint: '0 = no chunking, send the whole clip at once. Chunked requests suit long recordings or servers with per-request duration limits.',
asrAdvancedEnableItnLabel: 'Number normalization (enable_itn)',
asrAdvancedEnableItnHint: 'Normalizes spoken numbers/units into Arabic numerals (e.g. “twenty twenty-six” → “2026”). Turn off to keep the raw text.',
volcengineAppKeyLabel: 'APP ID', volcengineAppKeyLabel: 'APP ID',
volcengineAccessKeyLabel: 'Access Token', volcengineAccessKeyLabel: 'Access Token',
volcengineApiKeyLabel: 'API Key', volcengineApiKeyLabel: 'API Key',
+5 -1
View File
@@ -870,6 +870,7 @@ export const ja: typeof zhCN = {
asrGroq: 'Groq Whisper-large-v3', asrGroq: 'Groq Whisper-large-v3',
asrWhisper: 'OpenAI Whisper(互換)', asrWhisper: 'OpenAI Whisper(互換)',
asrOpenrouter: 'OpenRouter Whisper', asrOpenrouter: 'OpenRouter Whisper',
asrZenmux: 'ZenMux',
asrOpenAiCompatible: 'カスタム OpenAI 互換', asrOpenAiCompatible: 'カスタム OpenAI 互換',
asrXiaomiMimo: 'Xiaomi MiMo ASR', asrXiaomiMimo: 'Xiaomi MiMo ASR',
asrIflytek: 'iFlytek リアルタイム音声認識', asrIflytek: 'iFlytek リアルタイム音声認識',
@@ -880,11 +881,14 @@ export const ja: typeof zhCN = {
asrAppleSpeech: 'Apple 音声認識 (macOS)', asrAppleSpeech: 'Apple 音声認識 (macOS)',
}, },
elevenLabsUploadNotice: 'ElevenLabs は録音音声を設定済みのエンドポイントへアップロードしてバッチ文字起こしします。', elevenLabsUploadNotice: 'ElevenLabs は録音音声を設定済みのエンドポイントへアップロードしてバッチ文字起こしします。',
asrAdvancedNote: '以下の詳細オプションは「カスタム OpenAI 互換」プリセットのみに影響します。名前付きプロバイダーのプリセットは内蔵動作のままです。', zenmuxVocabularyNote: 'ZenMux は JSON 文字起こしプロトコルを使用し、辞書ホットワード(prompt/hotwords)は送信されません。辞書は依然として潤色段階には渡りますが、音声認識のバイアスには使用されません。',
asrAdvancedNote: '以下の詳細オプションは「カスタム OpenAI 互換」と「ZenMux」のプリセットのみに影響します。その他の名前付きプロバイダーのプリセットは内蔵動作のままです。',
asrAdvancedVerboseJsonLabel: 'セグメント指標 (verbose_json)', asrAdvancedVerboseJsonLabel: 'セグメント指標 (verbose_json)',
asrAdvancedVerboseJsonHint: 'サーバーが対応する場合に幻聴フィルタ用の segments 指標を要求します。非対応の自前サーバーではオフのままにしてください。', asrAdvancedVerboseJsonHint: 'サーバーが対応する場合に幻聴フィルタ用の segments 指標を要求します。非対応の自前サーバーではオフのままにしてください。',
asrAdvancedChunkLabel: '分割時間 (ms)', asrAdvancedChunkLabel: '分割時間 (ms)',
asrAdvancedChunkHint: '0 = 分割なし(全体を一括送信)。長い録音や1リクエストの時間制限があるサーバー向けに分割送信できます。', asrAdvancedChunkHint: '0 = 分割なし(全体を一括送信)。長い録音や1リクエストの時間制限があるサーバー向けに分割送信できます。',
asrAdvancedEnableItnLabel: '数字正規化 (enable_itn)',
asrAdvancedEnableItnHint: '口頭の数字・単位を算用数字に正規化します(例:「にせんにじゅうろく」→「2026」)。オフにすると元の表記を保持します。',
volcengineAppKeyLabel: 'APP ID', volcengineAppKeyLabel: 'APP ID',
volcengineAccessKeyLabel: 'Access Token', volcengineAccessKeyLabel: 'Access Token',
volcengineApiKeyLabel: 'API Key', volcengineApiKeyLabel: 'API Key',
+5 -1
View File
@@ -870,6 +870,7 @@ export const ko: typeof zhCN = {
asrGroq: 'Groq Whisper-large-v3', asrGroq: 'Groq Whisper-large-v3',
asrWhisper: 'OpenAI Whisper(호환)', asrWhisper: 'OpenAI Whisper(호환)',
asrOpenrouter: 'OpenRouter Whisper', asrOpenrouter: 'OpenRouter Whisper',
asrZenmux: 'ZenMux',
asrOpenAiCompatible: '커스텀 OpenAI 호환', asrOpenAiCompatible: '커스텀 OpenAI 호환',
asrXiaomiMimo: 'Xiaomi MiMo ASR', asrXiaomiMimo: 'Xiaomi MiMo ASR',
asrIflytek: 'iFlytek 실시간 음성 인식', asrIflytek: 'iFlytek 실시간 음성 인식',
@@ -880,11 +881,14 @@ export const ko: typeof zhCN = {
asrAppleSpeech: 'Apple 음성 (macOS)', asrAppleSpeech: 'Apple 음성 (macOS)',
}, },
elevenLabsUploadNotice: 'ElevenLabs는 녹음 오디오를 설정된 엔드포인트에 업로드해 일괄 전사합니다.', elevenLabsUploadNotice: 'ElevenLabs는 녹음 오디오를 설정된 엔드포인트에 업로드해 일괄 전사합니다.',
asrAdvancedNote: '아래 고급 옵션은「커스텀 OpenAI 호환」프리셋에만 적용됩니다. 명명된 공급자 프리셋은 내장 동작을 유지합니다.', zenmuxVocabularyNote: 'ZenMux는 JSON 전사 프로토콜을 사용하며 사전 핫워드(prompt/hotwords)를 보내지 않습니다. 사전은 여전히 다듬기 단계에 전달되지만 음성 인식 편향에는 사용되지 않습니다.',
asrAdvancedNote: '아래 고급 옵션은「커스텀 OpenAI 호환」및「ZenMux」프리셋에만 적용됩니다. 다른 명명된 공급자 프리셋은 내장 동작을 유지합니다.',
asrAdvancedVerboseJsonLabel: '세그먼트 지표 (verbose_json)', asrAdvancedVerboseJsonLabel: '세그먼트 지표 (verbose_json)',
asrAdvancedVerboseJsonHint: '서버가 지원할 때 환각 필터링용 segments 지표를 요청합니다. 지원하지 않는 자체 호스팅 서버에서는 꺼두세요.', asrAdvancedVerboseJsonHint: '서버가 지원할 때 환각 필터링용 segments 지표를 요청합니다. 지원하지 않는 자체 호스팅 서버에서는 꺼두세요.',
asrAdvancedChunkLabel: '분할 시간 (ms)', asrAdvancedChunkLabel: '분할 시간 (ms)',
asrAdvancedChunkHint: '0 = 분할 없음(전체를 한 번에 전송). 긴 녹음이나 요청당 시간 제한이 있는 서버에 적합합니다.', asrAdvancedChunkHint: '0 = 분할 없음(전체를 한 번에 전송). 긴 녹음이나 요청당 시간 제한이 있는 서버에 적합합니다.',
asrAdvancedEnableItnLabel: '숫자 정규화 (enable_itn)',
asrAdvancedEnableItnHint: '구어 숫자/단위를 아라비아 숫자로 정규화합니다(예: "이천이십육" → "2026"). 끄면 원문을 유지합니다.',
volcengineAppKeyLabel: 'APP ID', volcengineAppKeyLabel: 'APP ID',
volcengineAccessKeyLabel: 'Access Token', volcengineAccessKeyLabel: 'Access Token',
volcengineApiKeyLabel: 'API Key', volcengineApiKeyLabel: 'API Key',
+5 -1
View File
@@ -866,6 +866,7 @@ export const zhCN = {
asrGroq: 'Groq Whisper-large-v3', asrGroq: 'Groq Whisper-large-v3',
asrWhisper: 'OpenAI Whisper(兼容)', asrWhisper: 'OpenAI Whisper(兼容)',
asrOpenrouter: 'OpenRouter Whisper', asrOpenrouter: 'OpenRouter Whisper',
asrZenmux: 'ZenMux',
asrOpenAiCompatible: '自定义 OpenAI 兼容', asrOpenAiCompatible: '自定义 OpenAI 兼容',
asrXiaomiMimo: '小米 MiMo ASR', asrXiaomiMimo: '小米 MiMo ASR',
asrIflytek: '讯飞实时语音转写', asrIflytek: '讯飞实时语音转写',
@@ -876,11 +877,14 @@ export const zhCN = {
asrAppleSpeech: 'Apple 语音(macOS)', asrAppleSpeech: 'Apple 语音(macOS)',
}, },
elevenLabsUploadNotice: 'ElevenLabs 会将录音上传到所配置的端点进行批量转写。', elevenLabsUploadNotice: 'ElevenLabs 会将录音上传到所配置的端点进行批量转写。',
asrAdvancedNote: '以下高级选项仅影响「自定义 OpenAI 兼容」预设;命名厂商预设保持内置行为。', zenmuxVocabularyNote: 'ZenMux 走 JSON 转写协议,不发送词典热词(prompt/hotwords);词典仍会进入润色链路,但不会参与语音识别偏置。',
asrAdvancedNote: '以下高级选项仅影响「自定义 OpenAI 兼容」与「ZenMux」预设;其余命名厂商预设保持内置行为。',
asrAdvancedVerboseJsonLabel: '分段指标 (verbose_json)', asrAdvancedVerboseJsonLabel: '分段指标 (verbose_json)',
asrAdvancedVerboseJsonHint: '服务端支持时返回 segments 指标,用于幻听过滤;自建服务若不支持请保持关闭。', asrAdvancedVerboseJsonHint: '服务端支持时返回 segments 指标,用于幻听过滤;自建服务若不支持请保持关闭。',
asrAdvancedChunkLabel: '分片时长 (ms)', asrAdvancedChunkLabel: '分片时长 (ms)',
asrAdvancedChunkHint: '0 = 不分片,整段发送;按片段多次请求,适合长录音或服务端单次请求时长受限。', asrAdvancedChunkHint: '0 = 不分片,整段发送;按片段多次请求,适合长录音或服务端单次请求时长受限。',
asrAdvancedEnableItnLabel: '数字归一化 (enable_itn)',
asrAdvancedEnableItnHint: '把口语数字/单位归一化为阿拉伯数字(如“二零二六年”→“2026年”)。关闭后保留原始文字。',
volcengineAppKeyLabel: 'APP ID', volcengineAppKeyLabel: 'APP ID',
volcengineAccessKeyLabel: 'Access Token', volcengineAccessKeyLabel: 'Access Token',
volcengineApiKeyLabel: 'API Key', volcengineApiKeyLabel: 'API Key',
+5 -1
View File
@@ -868,6 +868,7 @@ export const zhTW: typeof zhCN = {
asrGroq: 'Groq Whisper-large-v3', asrGroq: 'Groq Whisper-large-v3',
asrWhisper: 'OpenAI Whisper(兼容)', asrWhisper: 'OpenAI Whisper(兼容)',
asrOpenrouter: 'OpenRouter Whisper', asrOpenrouter: 'OpenRouter Whisper',
asrZenmux: 'ZenMux',
asrOpenAiCompatible: '自訂 OpenAI 相容', asrOpenAiCompatible: '自訂 OpenAI 相容',
asrXiaomiMimo: '小米 MiMo ASR', asrXiaomiMimo: '小米 MiMo ASR',
asrIflytek: '訊飛即時語音轉寫', asrIflytek: '訊飛即時語音轉寫',
@@ -878,11 +879,14 @@ export const zhTW: typeof zhCN = {
asrAppleSpeech: 'Apple 語音(macOS)', asrAppleSpeech: 'Apple 語音(macOS)',
}, },
elevenLabsUploadNotice: 'ElevenLabs 會將錄音上傳至已設定的端點進行批次轉寫。', elevenLabsUploadNotice: 'ElevenLabs 會將錄音上傳至已設定的端點進行批次轉寫。',
asrAdvancedNote: '以下進階選項僅影響「自訂 OpenAI 相容」預設;具名廠商預設維持內建行為。', zenmuxVocabularyNote: 'ZenMux 走 JSON 轉寫協定,不傳送詞典熱詞(prompt/hotwords);詞典仍會進入潤色鏈路,但不會參與語音辨識偏置。',
asrAdvancedNote: '以下進階選項僅影響「自訂 OpenAI 相容」與「ZenMux」預設;其餘具名廠商預設維持內建行為。',
asrAdvancedVerboseJsonLabel: '分段指標 (verbose_json)', asrAdvancedVerboseJsonLabel: '分段指標 (verbose_json)',
asrAdvancedVerboseJsonHint: '服務端支援時回傳 segments 指標,用於幻聽過濾;自建服務若不支援請保持關閉。', asrAdvancedVerboseJsonHint: '服務端支援時回傳 segments 指標,用於幻聽過濾;自建服務若不支援請保持關閉。',
asrAdvancedChunkLabel: '分片時長 (ms)', asrAdvancedChunkLabel: '分片時長 (ms)',
asrAdvancedChunkHint: '0 = 不分片,整段傳送;按片段多次請求,適合長錄音或服務端單次請求時長受限。', asrAdvancedChunkHint: '0 = 不分片,整段傳送;按片段多次請求,適合長錄音或服務端單次請求時長受限。',
asrAdvancedEnableItnLabel: '數字正規化 (enable_itn)',
asrAdvancedEnableItnHint: '把口語數字/單位正規化為阿拉伯數字(如「二零二六年」→「2026年」)。關閉後保留原始文字。',
volcengineAppKeyLabel: 'APP ID', volcengineAppKeyLabel: 'APP ID',
volcengineAccessKeyLabel: 'Access Token', volcengineAccessKeyLabel: 'Access Token',
volcengineApiKeyLabel: 'API Key', volcengineApiKeyLabel: 'API Key',
@@ -11,54 +11,64 @@ function assertEqual(actual: unknown, expected: unknown, name: string) {
assertEqual( assertEqual(
parseAdvancedAsrConfig(null), parseAdvancedAsrConfig(null),
{ verboseJson: false, chunkDurationMs: null }, { verboseJson: false, chunkDurationMs: null, enableItn: true },
'null raw falls back to conservative defaults', 'null raw falls back to conservative defaults',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig(''), parseAdvancedAsrConfig(''),
{ verboseJson: false, chunkDurationMs: null }, { verboseJson: false, chunkDurationMs: null, enableItn: true },
'empty raw falls back to conservative defaults', 'empty raw falls back to conservative defaults',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig('not-json'), parseAdvancedAsrConfig('not-json'),
{ verboseJson: false, chunkDurationMs: null }, { verboseJson: false, chunkDurationMs: null, enableItn: true },
'invalid JSON falls back to conservative defaults', 'invalid JSON falls back to conservative defaults',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig('[1,2]'), parseAdvancedAsrConfig('[1,2]'),
{ verboseJson: false, chunkDurationMs: null }, { verboseJson: false, chunkDurationMs: null, enableItn: true },
'non-object JSON falls back to conservative defaults', 'non-object JSON falls back to conservative defaults',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig('{"verboseJson":true}'), parseAdvancedAsrConfig('{"verboseJson":true}'),
{ verboseJson: true, chunkDurationMs: null }, { verboseJson: true, chunkDurationMs: null, enableItn: true },
'missing chunkDurationMs stays null', 'missing chunkDurationMs stays null',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig('{"chunkDurationMs":0}'), parseAdvancedAsrConfig('{"chunkDurationMs":0}'),
{ verboseJson: false, chunkDurationMs: null }, { verboseJson: false, chunkDurationMs: null, enableItn: true },
'zero chunk duration means no chunking', 'zero chunk duration means no chunking',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig('{"chunkDurationMs":30000.9,"verboseJson":false}'), parseAdvancedAsrConfig('{"chunkDurationMs":30000.9,"verboseJson":false}'),
{ verboseJson: false, chunkDurationMs: 30000 }, { verboseJson: false, chunkDurationMs: 30000, enableItn: true },
'chunk duration is floored to integer', 'chunk duration is floored to integer',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig('{"verboseJson":"yes"}'), parseAdvancedAsrConfig('{"verboseJson":"yes"}'),
{ verboseJson: false, chunkDurationMs: null }, { verboseJson: false, chunkDurationMs: null, enableItn: true },
'non-boolean verboseJson falls back to false', 'non-boolean verboseJson falls back to false',
) )
assertEqual( assertEqual(
parseAdvancedAsrConfig(serializeAdvancedAsrConfig({ verboseJson: true, chunkDurationMs: 30000 })), parseAdvancedAsrConfig('{"enableItn":false}'),
{ verboseJson: true, chunkDurationMs: 30000 }, { verboseJson: false, chunkDurationMs: null, enableItn: false },
'explicit false enableItn is honored',
)
assertEqual(
parseAdvancedAsrConfig(serializeAdvancedAsrConfig({
verboseJson: true,
chunkDurationMs: 30000,
enableItn: false,
})),
{ verboseJson: true, chunkDurationMs: 30000, enableItn: false },
'serialize/parse round-trip preserves config', 'serialize/parse round-trip preserves config',
) )
+11 -6
View File
@@ -1,25 +1,27 @@
// 通用 OpenAI 兼容 ASR(openai-compatible)的高级配置。 // 通用 OpenAI 兼容 ASR(openai-compatible)与 ZenMux(zenmux)的高级配置。
// JSON 形状与后端 coordinator.rs::AdvancedAsrConfig 保持一致: // JSON 形状与后端 coordinator.rs::AdvancedAsrConfig 保持一致:
// {"verboseJson": bool, "chunkDurationMs": number|null}。 // {"verboseJson": bool, "chunkDurationMs": number|null, "enableItn": bool}。
// 解析策略与后端一致:缺失 / 非法一律回落保守默认(不发 response_format、不分片)。 // 解析策略与后端一致:缺失 / 非法一律回落保守默认(不发 response_format、不分片、
// enable_itn 默认开启)。
export interface AdvancedAsrConfig { export interface AdvancedAsrConfig {
verboseJson: boolean verboseJson: boolean
chunkDurationMs: number | null chunkDurationMs: number | null
enableItn: boolean
} }
export function parseAdvancedAsrConfig(raw: string | null): AdvancedAsrConfig { export function parseAdvancedAsrConfig(raw: string | null): AdvancedAsrConfig {
if (!raw) { if (!raw) {
return { verboseJson: false, chunkDurationMs: null } return { verboseJson: false, chunkDurationMs: null, enableItn: true }
} }
let value: unknown let value: unknown
try { try {
value = JSON.parse(raw) value = JSON.parse(raw)
} catch { } catch {
return { verboseJson: false, chunkDurationMs: null } return { verboseJson: false, chunkDurationMs: null, enableItn: true }
} }
if (typeof value !== 'object' || value === null) { if (typeof value !== 'object' || value === null) {
return { verboseJson: false, chunkDurationMs: null } return { verboseJson: false, chunkDurationMs: null, enableItn: true }
} }
const record = value as Record<string, unknown> const record = value as Record<string, unknown>
const rawMs = record.chunkDurationMs const rawMs = record.chunkDurationMs
@@ -30,6 +32,8 @@ export function parseAdvancedAsrConfig(raw: string | null): AdvancedAsrConfig {
return { return {
verboseJson: record.verboseJson === true, verboseJson: record.verboseJson === true,
chunkDurationMs, chunkDurationMs,
// 缺失 / 非布尔回落默认开启(与后端 parse_advanced_asr_config 一致)。
enableItn: record.enableItn !== false,
} }
} }
@@ -37,5 +41,6 @@ export function serializeAdvancedAsrConfig(config: AdvancedAsrConfig): string {
return JSON.stringify({ return JSON.stringify({
verboseJson: config.verboseJson, verboseJson: config.verboseJson,
chunkDurationMs: config.chunkDurationMs, chunkDurationMs: config.chunkDurationMs,
enableItn: config.enableItn,
}) })
} }
@@ -26,3 +26,17 @@ if (openAiCompatiblePreset.baseUrl !== '' || openAiCompatiblePreset.model !== ''
`Custom OpenAI-compatible ASR preset must have no defaults (got baseUrl=${openAiCompatiblePreset.baseUrl}, model=${openAiCompatiblePreset.model})`, `Custom OpenAI-compatible ASR preset must have no defaults (got baseUrl=${openAiCompatiblePreset.baseUrl}, model=${openAiCompatiblePreset.model})`,
); );
} }
const zenmuxPreset = ASR_PRESETS.find(p => p.id === 'zenmux');
if (!zenmuxPreset) {
throw new Error('ZenMux ASR preset is missing');
}
if (zenmuxPreset.baseUrl !== 'https://zenmux.ai/api/v1') {
throw new Error(`unexpected ZenMux base URL: ${zenmuxPreset.baseUrl}`);
}
if (zenmuxPreset.model !== 'qwen/qwen3-asr-flash') {
throw new Error(`unexpected ZenMux default model: ${zenmuxPreset.model}`);
}
@@ -659,9 +659,14 @@ export function ProvidersSection({ kind = 'all' }: ProvidersSectionProps = {}) {
{t('settings.providers.elevenLabsUploadNotice')} {t('settings.providers.elevenLabsUploadNotice')}
</div> </div>
)} )}
{committedAsrProvider === 'zenmux' && (
<div role="note" style={{ marginTop: 2, fontSize: 11.5, color: 'var(--ol-ink-4)', lineHeight: 1.6 }}>
{t('settings.providers.zenmuxVocabularyNote')}
</div>
)}
{/* 统一百炼「拉取模型」只写 model,不覆盖用户选择的区域或工作空间 endpoint。 */} {/* 统一百炼「拉取模型」只写 model,不覆盖用户选择的区域或工作空间 endpoint。 */}
<ProviderTools kind="asr" modelAccount="asr.model" provider={committedAsrProvider} onModelSelected={() => setAsrModelRevision(v => v + 1)} /> <ProviderTools kind="asr" modelAccount="asr.model" provider={committedAsrProvider} onModelSelected={() => setAsrModelRevision(v => v + 1)} />
{committedAsrProvider === 'openai-compatible' && ( {(committedAsrProvider === 'openai-compatible' || committedAsrProvider === 'zenmux') && (
<AsrAdvancedOptions provider={committedAsrProvider} /> <AsrAdvancedOptions provider={committedAsrProvider} />
)} )}
</> </>
@@ -672,12 +677,14 @@ export function ProvidersSection({ kind = 'all' }: ProvidersSectionProps = {}) {
); );
} }
// 通用 OpenAI 兼容 ASR 的高级选项:仅对 openai-compatible 预设显示。 // ASR 高级选项:openai-compatible 与 zenmux 两个预设显示。
// 命名厂商预设的怪癖开关(verbose_json / 分片等)是测过的硬编码行为,不开放。 // openai-compatible 暴露 verbose_json / 分片时长(其余命名厂商保持硬编码行为);
// zenmux 暴露 enable_itn(数字归一化)开关,verbose_json / 分片对其无意义。
function AsrAdvancedOptions({ provider }: { provider: string }) { function AsrAdvancedOptions({ provider }: { provider: string }) {
const { t } = useTranslation(); const { t } = useTranslation();
const [verboseJson, setVerboseJson] = useState(false); const [verboseJson, setVerboseJson] = useState(false);
const [chunkDraft, setChunkDraft] = useState(''); const [chunkDraft, setChunkDraft] = useState('');
const [enableItn, setEnableItn] = useState(true);
const [status, setStatus] = useState<'idle' | 'saving' | 'error'>('idle'); const [status, setStatus] = useState<'idle' | 'saving' | 'error'>('idle');
const [error, setError] = useState(''); const [error, setError] = useState('');
@@ -692,6 +699,7 @@ function AsrAdvancedOptions({ provider }: { provider: string }) {
const config = parseAdvancedAsrConfig(raw); const config = parseAdvancedAsrConfig(raw);
setVerboseJson(config.verboseJson); setVerboseJson(config.verboseJson);
setChunkDraft(config.chunkDurationMs ? String(config.chunkDurationMs) : ''); setChunkDraft(config.chunkDurationMs ? String(config.chunkDurationMs) : '');
setEnableItn(config.enableItn);
} catch (err) { } catch (err) {
if (!cancelled) { if (!cancelled) {
setStatus('error'); setStatus('error');
@@ -710,7 +718,11 @@ function AsrAdvancedOptions({ provider }: { provider: string }) {
return Math.floor(value); return Math.floor(value);
}; };
const save = async (partial: { verboseJson?: boolean; chunkDurationMs?: number | null }) => { const save = async (partial: {
verboseJson?: boolean
chunkDurationMs?: number | null
enableItn?: boolean
}) => {
setStatus('saving'); setStatus('saving');
setError(''); setError('');
const next: AdvancedAsrConfig = { const next: AdvancedAsrConfig = {
@@ -719,11 +731,13 @@ function AsrAdvancedOptions({ provider }: { provider: string }) {
partial.chunkDurationMs !== undefined partial.chunkDurationMs !== undefined
? partial.chunkDurationMs ? partial.chunkDurationMs
: parseChunkDraft(chunkDraft), : parseChunkDraft(chunkDraft),
enableItn: partial.enableItn ?? enableItn,
}; };
try { try {
await setCredential('asr.advanced_config', serializeAdvancedAsrConfig(next), provider); await setCredential('asr.advanced_config', serializeAdvancedAsrConfig(next), provider);
setVerboseJson(next.verboseJson); setVerboseJson(next.verboseJson);
setChunkDraft(next.chunkDurationMs ? String(next.chunkDurationMs) : ''); setChunkDraft(next.chunkDurationMs ? String(next.chunkDurationMs) : '');
setEnableItn(next.enableItn);
setStatus('idle'); setStatus('idle');
} catch (err) { } catch (err) {
setStatus('error'); setStatus('error');
@@ -744,31 +758,42 @@ function AsrAdvancedOptions({ provider }: { provider: string }) {
> >
{t('settings.providers.asrAdvancedNote')} {t('settings.providers.asrAdvancedNote')}
</div> </div>
<SettingRow {provider === 'zenmux' ? (
label={t('settings.providers.asrAdvancedVerboseJsonLabel')} <SettingRow
desc={t('settings.providers.asrAdvancedVerboseJsonHint')} label={t('settings.providers.asrAdvancedEnableItnLabel')}
> desc={t('settings.providers.asrAdvancedEnableItnHint')}
<Toggle on={verboseJson} onToggle={(next) => void save({ verboseJson: next })} /> >
</SettingRow> <Toggle on={enableItn} onToggle={(next) => void save({ enableItn: next })} />
<SettingRow </SettingRow>
label={t('settings.providers.asrAdvancedChunkLabel')} ) : (
desc={t('settings.providers.asrAdvancedChunkHint')} <>
> <SettingRow
<input label={t('settings.providers.asrAdvancedVerboseJsonLabel')}
type="number" desc={t('settings.providers.asrAdvancedVerboseJsonHint')}
min={0} >
step={1000} <Toggle on={verboseJson} onToggle={(next) => void save({ verboseJson: next })} />
value={chunkDraft} </SettingRow>
placeholder="0" <SettingRow
disabled={status === 'saving'} label={t('settings.providers.asrAdvancedChunkLabel')}
onChange={(e) => setChunkDraft(e.target.value)} desc={t('settings.providers.asrAdvancedChunkHint')}
onBlur={() => void save({ chunkDurationMs: parseChunkDraft(chunkDraft) })} >
onKeyDown={(e) => { <input
if (e.key === 'Enter') (e.target as HTMLInputElement).blur(); type="number"
}} min={0}
style={inputStyle} step={1000}
/> value={chunkDraft}
</SettingRow> placeholder="0"
disabled={status === 'saving'}
onChange={(e) => setChunkDraft(e.target.value)}
onBlur={() => void save({ chunkDurationMs: parseChunkDraft(chunkDraft) })}
onKeyDown={(e) => {
if (e.key === 'Enter') (e.target as HTMLInputElement).blur();
}}
style={inputStyle}
/>
</SettingRow>
</>
)}
{status === 'error' && ( {status === 'error' && (
<div style={{ fontSize: 11, color: 'var(--ol-warn)', lineHeight: 1.4 }}> <div style={{ fontSize: 11, color: 'var(--ol-warn)', lineHeight: 1.4 }}>
{t('common.operationFailed')}: {error} {t('common.operationFailed')}: {error}
@@ -235,6 +235,10 @@ export const ASR_PRESETS = [
// OpenRouter 的 /audio/transcriptions 走 application/json + base64(issue #582), // OpenRouter 的 /audio/transcriptions 走 application/json + base64(issue #582),
// 后端 coordinator.rs::whisper_request_format 对该 id 切换到 OpenRouterJson 编码。 // 后端 coordinator.rs::whisper_request_format 对该 id 切换到 OpenRouterJson 编码。
{ id: 'openrouter', nameKey: 'asrOpenrouter', baseUrl: 'https://openrouter.ai/api/v1', model: 'openai/whisper-large-v3-turbo' }, { id: 'openrouter', nameKey: 'asrOpenrouter', baseUrl: 'https://openrouter.ai/api/v1', model: 'openai/whisper-large-v3-turbo' },
// ZenMux 聚合平台的 /audio/transcriptions 同为 application/json + base64
// (issue #837),后端 whisper_request_format 对该 id 切 ZenMuxJson;
// 语言跟随工作语言,enable_itn 开关见 AsrAdvancedOptions(仅该预设显示)。
{ id: 'zenmux', nameKey: 'asrZenmux', baseUrl: 'https://zenmux.ai/api/v1', model: 'qwen/qwen3-asr-flash' },
// 通用 OpenAI 兼容端点(自建 / 局域网 llama.cpp 等):无默认 endpoint/model, // 通用 OpenAI 兼容端点(自建 / 局域网 llama.cpp 等):无默认 endpoint/model,
// 用户必填;verbose_json 与分片时长由高级选项按 provider 配置(见 // 用户必填;verbose_json 与分片时长由高级选项按 provider 配置(见
// ProvidersSection 的 AsrAdvancedOptions),默认行为最保守。 // ProvidersSection 的 AsrAdvancedOptions),默认行为最保守。