Files
openless/openless-all
17fe67c59b feat(asr): 接入阶跃星辰 StepAudio 2.5 批式转写,词典走一等 hotwords (#835)
* feat(asr): 接入阶跃星辰 StepAudio 2.5 批式转写,词典走一等 hotwords

StepFun 的 /v1/audio/transcriptions 是标准 OpenAI multipart(实测 2026-07),
归入 Whisper 兼容路由即可,无需新协议客户端:

- 前端新增 stepfun preset(https://api.stepfun.com/v1 + stepaudio-2.5-asr),
  i18n 五语言补齐;
- StepFun 静默忽略 prompt(实测返回 200 但不参与偏置),词汇偏置走专门的
  hotwords 参数(JSON 数组字符串)——WhisperBatchASR 新增 with_hotwords,
  wiring 按 whisper_uses_hotwords 在 prompt / hotwords 间二选一路由,
  用户词典在 StepFun 下才真正生效;
- response_format 只认 json/text → verbose_json 幻听过滤保持关闭;
- 修「验证」按钮对静音敏感厂商的假阴性:探针是 250ms 纯静音,StepFun 返回
  400 "no speech found",但走到内容拒收说明鉴权(错 key=401)与模型名
  (错模型=404)均已通过——判为验证成功而非凭据错误。

实测矩阵(真实 key 逐项打点):真实语音+json/缺省/带 prompt/带 hotwords 均
200 且转写准确(5.5s 音频约 0.95s 往返);verbose_json 400;静音 400
no speech;错 key 401;错模型 404。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(asr): StepFun 流式实时转写——同一入口按模型名自动切协议

照百炼「一个入口、按模型二次路由」的先例:stepfun preset 下模型填
`stepaudio-2.5-asr-stream`(或任意 `*-stream`)即走新的实时 WS 客户端
(asr/stepfun_realtime.rs),其余模型维持批式 Whisper 兼容路径。凭据与
「已配置」判定仍看真实 active `stepfun`,一把 key、同一个 https base
(wss URL 由 client 自动派生)。

协议与 qwen_realtime 同为 OpenAI Realtime 风格,但四处关键差异
(2026-07-16 真实接口逐项实测确认):

- 模型在 session.update 里传(session.audio.input.transcription.model),
  session 配置是 audio.input.{format,transcription,turn_detection} 嵌套形状;
- delta 的 text/stash 是「已确定前缀 + 未定尾巴」的拼接关系(Qwen 是互斥取一);
- 没有服务端结束事件:session.finish 回 transcript.response.error,server_vad
  下 commit 被静默忽略。收尾靠客户端状态机:松手时冲刷尾音频 + 补送 700ms
  静音帧逼 VAD 关段,finishing && open_segments==0 即完成;纯静音会话由
  1.2s 宽限期兜底空文本成功(验证按钮复用该路径,全程 ~2s);
- 实时协议词汇偏置走 transcription.prompt(批式恰相反只认 hotwords),
  词典按 effective 协议自动选通道。

真机端到端(真实 key、模拟 recorder 时序):session 建立 186ms,
松手→最终结果 251ms(批式 ~950ms),5.5s 中文语音转写一字不差,
多句段 VAD 断句由 join_segments 复用 qwen 拼接规则。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(asr): wait for StepFun tail frame before finishing

* fix(asr): guard StepFun completion before tail ack

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Chris233 <h-chris233@outlook.com>
2026-07-18 15:43:24 +08:00
..

OpenLess All-Platform

This is the current cross-platform OpenLess workspace.

App Directory

The runnable Tauri app lives in app/. The macOS build links a vendored C ASR engine (Open-Less/qwen-asr, forked from antirez/qwen-asr) tracked as a git submodule under app/src-tauri/vendor/qwen-asr/, so initialize submodules on first clone.

# First clone only — pull in vendored submodules
git submodule update --init --recursive

cd app
npm ci
npm run tauri dev

macOS Build

Use the project build script instead of calling tauri build directly:

cd app
INSTALL=0 ./scripts/build-mac.sh

Generated macOS artifacts:

  • app/src-tauri/target/release/bundle/macos/OpenLess.app
  • app/src-tauri/target/release/bundle/dmg/OpenLess_1.1.0_aarch64.dmg

For local install during development:

cd app
./scripts/build-mac.sh

Windows Build

The runnable Tauri app is still app/. Windows contributors should run a preflight before building so missing MSVC, Windows SDK, or MinGW tools fail with actionable messages.

cd app
powershell -ExecutionPolicy Bypass -File .\scripts\windows-preflight.ps1

MSVC Route

Use this route when Visual Studio Build Tools and the Windows SDK are installed. Open a Developer PowerShell, or call vcvars64.bat, then run:

cd app
npm ci
npm run tauri -- build

Required Visual Studio Installer components:

  • Microsoft.VisualStudio.Workload.VCTools
  • MSVC v143 x64/x86 build tools
  • Windows 10/11 SDK that provides kernel32.lib

If link.exe or kernel32.lib is missing, rerun:

powershell -ExecutionPolicy Bypass -File .\scripts\windows-preflight.ps1 -Toolchain msvc

GNU / MinGW Route

Use this route when MSVC/Windows SDK is unavailable. The app now lives under the no-space openless-all directory to avoid GNU/MinGW path quoting issues while generating import libraries. Use the helper script to keep the GNU build environment and target setup consistent.

cd app
scoop install rustup mingw
rustup toolchain install stable-x86_64-pc-windows-gnu
rustup target add x86_64-pc-windows-gnu
powershell -ExecutionPolicy Bypass -File .\scripts\windows-preflight.ps1 -Toolchain gnu
powershell -ExecutionPolicy Bypass -File .\scripts\windows-build-gnu.ps1

Generated GNU artifacts:

  • %TEMP%\openless-windows-gnu\src-tauri\target\x86_64-pc-windows-gnu\release\openless.exe
  • %TEMP%\openless-windows-gnu\src-tauri\target\x86_64-pc-windows-gnu\release\bundle\msi\OpenLess_*_x64_en-US.msi
  • %TEMP%\openless-windows-gnu\src-tauri\target\x86_64-pc-windows-gnu\release\bundle\nsis\OpenLess_*_x64-setup.exe

Hotkey Injection Gate

Use this gate before/after Windows hotkey changes when a physical keyboard regression is unavailable. It injects a dev/test-only hotkey click through the coordinator handle_pressed / handle_released path, asserts the log contains [coord] hotkey pressed, and cancels the dry-run session automatically.

cd app
npm run check:hotkey-injection

Windows Runtime Notes

  • Windows does not need the macOS Accessibility permission. Use Settings -> Permissions -> Global hotkey to inspect listener status.
  • Microphone permission is checked by opening a short-lived input stream, so a device-format query alone is not treated as permission granted.
  • Text insertion through Ctrl+V is treated as copy fallback unless the app can confirm insertion.

Release Signing

Tagged releases (v*-tauri) must be Developer ID signed and notarized so users can download and open the macOS app without manually removing quarantine attributes.

Required GitHub secrets:

  • APPLE_CERTIFICATE
  • APPLE_CERTIFICATE_PASSWORD
  • APPLE_ID
  • APPLE_PASSWORD
  • APPLE_TEAM_ID

Optional:

  • APPLE_PROVIDER_SHORT_NAME
  • KEYCHAIN_PASSWORD

Manual workflow runs can still produce ad-hoc signed test builds, but tagged macOS releases fail if signing/notarization secrets are missing.

Ignored Local Output

The following are intentionally local-only:

  • app/node_modules/
  • app/dist/
  • app/src-tauri/target/
  • app/src-tauri/gen/
  • .DS_Store