chore(repo): 移除内部 AI 协作 / 规划 / 审计文档，仅本地保留
公共仓库不再发布 CLAUDE.md / AGENTS.md / 内部 audit + roadmap + github-tracking + superpowers + windows-*-tracking 等共 45 个文件。 - 全部用 `git rm --cached`，本地仍保留以供日常协作； - `.gitignore` 加规则防止再次被 `git add` 拉回； - 用户 docs（volcengine-setup / tauri-csp / references / 图片素材） 以及 ISSUE_TEMPLATE / workflows / pull_request_template 仍保留发布。 密钥层面审计同步完成：未发现 API key / token / 私钥泄露， tauri pubkey 与 GitHub OAuth client_id 均为设计上公开值。
This commit is contained in:
baiqing committed
1 parent
e6749b1d71
commit
d6d1fee77e
46 files changed
+34
-11241
No files matched your search
@@ -1,334 +0,0 @@
|
||||
# Windows terminal clipboard restore investigation (2026-05-02)
|
||||
|
||||
Scope: `openless-all/app/src-tauri/src/insertion.rs`
|
||||
|
||||
## Problem statement
|
||||
|
||||
On Windows terminal-style text entry, OpenLess could:
|
||||
|
||||
1. put the new dictated text into the clipboard
|
||||
2. send `Ctrl+V`
|
||||
3. restore the old clipboard too early
|
||||
4. let the terminal paste the old clipboard instead of the dictated text
|
||||
|
||||
## Baseline code path
|
||||
|
||||
- `Coordinator::end_session()` treats Windows synthetic paste as `InsertStatus::PasteSent`, not `Inserted`.
|
||||
- `TextInserter::insert()` calls `insert_with_clipboard_restore()`.
|
||||
- Baseline Windows/Linux behavior restored the previous clipboard after a fixed `150ms`.
|
||||
- That fixed delay assumed the target app had already consumed the clipboard by then.
|
||||
|
||||
## Automated evidence
|
||||
|
||||
### 1. GUI automation boundary in this session
|
||||
|
||||
Commands used:
|
||||
|
||||
```powershell
|
||||
Start-Process notepad.exe -PassThru
|
||||
Start-Process cmd.exe -PassThru
|
||||
EnumWindows(...)
|
||||
```
|
||||
|
||||
Observed result:
|
||||
|
||||
- `explorer.exe` exists in `SessionId=1`
|
||||
- newly started `notepad.exe`, `cmd.exe`, and even a local WinForms probe form did not expose enumerable top-level windows in this thread
|
||||
|
||||
Conclusion:
|
||||
|
||||
- this Codex desktop thread can compile and manipulate the Windows clipboard
|
||||
- it cannot reliably drive newly created GUI windows in the current desktop context
|
||||
- therefore the strongest fully automated evidence in this session must come from clipboard-timing experiments, not end-to-end GUI paste readback
|
||||
|
||||
### 2. Clipboard timing matrix
|
||||
|
||||
Script:
|
||||
|
||||
- `openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1`
|
||||
|
||||
Command:
|
||||
|
||||
```powershell
|
||||
$cases = @(
|
||||
@{ consumer = 50; restore = 150 },
|
||||
@{ consumer = 250; restore = 150 },
|
||||
@{ consumer = 250; restore = 750 }
|
||||
)
|
||||
foreach ($case in $cases) {
|
||||
powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1 -ConsumerDelayMs $case.consumer -RestoreDelayMs $case.restore
|
||||
}
|
||||
```
|
||||
|
||||
Observed outputs:
|
||||
|
||||
```json
|
||||
{"consumerDelayMs":50,"restoreDelayMs":150,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_DICTATED_TEXT","matchedInserted":true}
|
||||
{"consumerDelayMs":250,"restoreDelayMs":150,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_OLDER_CLIPBOARD","matchedInserted":false}
|
||||
{"consumerDelayMs":250,"restoreDelayMs":750,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_DICTATED_TEXT","matchedInserted":true}
|
||||
```
|
||||
|
||||
Interpretation:
|
||||
|
||||
- a fast consumer (`50ms`) succeeds with the old `150ms` restore window
|
||||
- a slower consumer (`250ms`) fails with the old `150ms` restore window
|
||||
- the same slower consumer succeeds once restore is delayed to `750ms`
|
||||
|
||||
This isolates the bug to clipboard restore timing, independent of ASR, polish, QA hotkey, or selection logic.
|
||||
|
||||
### 3. Real app end-to-end regression in a stable desktop automation stack
|
||||
|
||||
Environment:
|
||||
|
||||
- Python `pywinauto` + `pywin32`
|
||||
- Real desktop windows, not mock controls
|
||||
- Targets:
|
||||
- Windows Terminal `cmd.exe` tab
|
||||
- Windows Terminal `PowerShell` tab
|
||||
- Notepad
|
||||
|
||||
Method:
|
||||
|
||||
- Put a command or text payload into the real Windows clipboard
|
||||
- Send synthetic `Ctrl+V`
|
||||
- Wait either `150ms` or `750ms`
|
||||
- Restore the previous clipboard
|
||||
- Verify the target app actually received the intended payload
|
||||
|
||||
Observed outputs:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"target": "Windows Terminal CMD",
|
||||
"restoreDelayMs": 150,
|
||||
"expected": "CMD_150_OK",
|
||||
"succeeded": true
|
||||
},
|
||||
{
|
||||
"target": "Windows Terminal CMD",
|
||||
"restoreDelayMs": 750,
|
||||
"expected": "CMD_750_OK",
|
||||
"succeeded": true
|
||||
},
|
||||
{
|
||||
"target": "Windows Terminal PowerShell",
|
||||
"restoreDelayMs": 150,
|
||||
"expected": "POWERSHELL_150_OK",
|
||||
"succeeded": true
|
||||
},
|
||||
{
|
||||
"target": "Windows Terminal PowerShell",
|
||||
"restoreDelayMs": 750,
|
||||
"expected": "POWERSHELL_750_OK",
|
||||
"succeeded": true
|
||||
},
|
||||
{
|
||||
"target": "Notepad",
|
||||
"restoreDelayMs": 150,
|
||||
"expected": "NOTEPAD_150_OK",
|
||||
"succeeded": true
|
||||
},
|
||||
{
|
||||
"target": "Notepad",
|
||||
"restoreDelayMs": 750,
|
||||
"expected": "NOTEPAD_750_OK",
|
||||
"succeeded": true
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Interpretation:
|
||||
|
||||
- the isolated clipboard/paste/restore harness does **not** reproduce the stale-paste bug on the current Windows Terminal `CMD` tab
|
||||
- it also does **not** reproduce it on the current Windows Terminal `PowerShell` tab
|
||||
- Notepad behaves as expected in both timing windows
|
||||
- therefore the user-reported failure is not a blanket “all terminal paste on Windows fails at 150ms” statement
|
||||
- the failure requires an additional condition beyond “target is a terminal”, such as a slower paste consumer, extra lifecycle delay, or OpenLess-specific sequencing around focus restoration and session completion
|
||||
|
||||
### 4. Full OpenLess lifecycle evidence on `wt-cmd`
|
||||
|
||||
To go beyond isolated paste harnesses, the automation was pushed through the real OpenLess lifecycle:
|
||||
|
||||
- synthetic hold-mode hotkey press on Windows (`VK_LCONTROL`, observed by the low-level hook)
|
||||
- real recorder startup
|
||||
- real Volcengine ASR session connection
|
||||
- real LLM polish
|
||||
- real insertion into a Windows Terminal `cmd.exe` tab
|
||||
|
||||
Because the desktop automation session could not reliably feed text into the real microphone path, a debug-only test hook was added for automation:
|
||||
|
||||
- if a debug transcript file is configured and ASR returns an empty transcript, OpenLess substitutes that transcript and continues through the normal post-ASR insertion path
|
||||
|
||||
One captured successful run produced the following evidence:
|
||||
|
||||
- OpenLess log:
|
||||
- `[hotkey] Windows trigger pressed vk=162`
|
||||
- `[coord] front_app captured: C:\WINDOWS\system32\cmd.exe`
|
||||
- `[coord] recorder started (asr=volcengine, phase=Starting)`
|
||||
- `[coord] ASR connected; flushed ... deferred audio bytes`
|
||||
- `[coord] session started`
|
||||
- `[hotkey] Windows trigger released vk=162`
|
||||
- `[llm] HTTP 200 ...`
|
||||
|
||||
- History record:
|
||||
|
||||
```json
|
||||
{
|
||||
"rawTranscript": "瀑布它的白沫其实非常喜欢。",
|
||||
"finalText": "瀑布的白沫其实非常喜欢。",
|
||||
"insertStatus": "pasteSent"
|
||||
}
|
||||
```
|
||||
|
||||
- Windows Terminal `cmd.exe` tab tail:
|
||||
|
||||
```text
|
||||
D:\Users\cooper\Practice-Project\202604\openless>瀑布的白沫其实非常喜欢。
|
||||
```
|
||||
|
||||
Interpretation:
|
||||
|
||||
- this is a true OpenLess session, not a bare clipboard harness
|
||||
- the target front app captured by OpenLess was the Windows Terminal `cmd.exe` tab
|
||||
- the final inserted text visible at the terminal prompt matched the polished `finalText`
|
||||
- in this captured run, the terminal did **not** paste the pre-dictation clipboard contents
|
||||
|
||||
Residual caveat:
|
||||
|
||||
- repeated re-runs in the same desktop session later hit intermittent startup/hook-install flakiness before the test reached insertion again
|
||||
- that flakiness affected test repeatability, but it does not invalidate the already captured successful full-lifecycle evidence above
|
||||
|
||||
## 5. Repeatable full-lifecycle regression after automation hardening
|
||||
|
||||
After hardening the automation path, the full OpenLess lifecycle was run through a stable route:
|
||||
|
||||
- launch OpenLess with WebView2 remote debugging enabled
|
||||
- drive lifecycle by invoking Tauri commands from the main webview (`start_dictation` / `stop_dictation`)
|
||||
- keep real focus-target capture and real insertion behavior
|
||||
- use a debug-only transcript override only when ASR would otherwise be empty in this desktop environment
|
||||
- read back target content directly from UIA controls instead of recycling clipboard-based readback
|
||||
|
||||
Targets exercised:
|
||||
|
||||
- `Windows Terminal` `cmd.exe` tab
|
||||
- `Windows Terminal` `PowerShell` tab
|
||||
- `Notepad`
|
||||
|
||||
Representative results:
|
||||
|
||||
```json
|
||||
{
|
||||
"target": "wt-cmd",
|
||||
"historyFinalText": "openless terminal regression success",
|
||||
"insertStatus": "pasteSent",
|
||||
"targetContainsFinalText": true,
|
||||
"targetContainsClipboardSentinel": false
|
||||
}
|
||||
{
|
||||
"target": "wt-powershell",
|
||||
"historyFinalText": "openless terminal regression success",
|
||||
"insertStatus": "pasteSent",
|
||||
"targetContainsFinalText": true,
|
||||
"targetContainsClipboardSentinel": false
|
||||
}
|
||||
{
|
||||
"target": "notepad",
|
||||
"historyFinalText": "openless terminal regression success",
|
||||
"insertStatus": "pasteSent",
|
||||
"targetContainsFinalText": true,
|
||||
"targetContainsClipboardSentinel": false
|
||||
}
|
||||
```
|
||||
|
||||
Repeatability observed in the current session:
|
||||
|
||||
- `wt-cmd`: multiple successful runs with final text visible at the terminal prompt
|
||||
- `wt-powershell`: successful run with final text visible at the terminal prompt
|
||||
- `notepad`: two consecutive successful runs after switching readback from clipboard-based copy to direct UIA text capture
|
||||
|
||||
Updated interpretation:
|
||||
|
||||
- the originally suspected “terminal paste always restores the old clipboard before paste lands” is **not** reproducible as a general rule in the current full-lifecycle automation
|
||||
- once the automation path is stabilized, all three tested targets receive the intended final text while `insertStatus` remains `pasteSent`
|
||||
- the clipboard timing race is still real in isolation for slow consumers, but the complete OpenLess lifecycle on this machine does not reproduce the stale-clipboard failure for:
|
||||
- `wt-cmd`
|
||||
- `wt-powershell`
|
||||
- `notepad`
|
||||
|
||||
Most likely current conclusion:
|
||||
|
||||
- the user-reported bug depends on an additional condition not captured in the hardened automation path
|
||||
- plausible candidates remain:
|
||||
- a different terminal host/session state
|
||||
- a different target application than the tested Windows Terminal tabs
|
||||
- another timing-sensitive environment factor outside the core insertion code
|
||||
|
||||
## Root cause
|
||||
|
||||
Root cause: Windows `PasteSent` semantics were treated as if they implied paste completion.
|
||||
|
||||
- `PasteSent` only means OpenLess sent synthetic `Ctrl+V`
|
||||
- it does not mean the target application has already read clipboard contents
|
||||
- terminal-style targets can consume the clipboard later than standard text inputs
|
||||
- restoring the old clipboard at a fixed `150ms` can therefore race ahead of actual paste consumption
|
||||
- current real-app regression suggests this is conditional, not universal: some terminal sessions consume quickly enough to beat `150ms`, while slower consumers still fail
|
||||
|
||||
Classification:
|
||||
|
||||
- primary layer: `clipboard lifecycle`
|
||||
- secondary layer: `insertion lifecycle`
|
||||
- not primary: `focus restore`
|
||||
- manifestation: terminal-specific and likely any slower Windows paste consumer
|
||||
- not evidence of a global Windows clipboard bug by itself
|
||||
|
||||
## Fix applied
|
||||
|
||||
File:
|
||||
|
||||
- `openless-all/app/src-tauri/src/insertion.rs`
|
||||
|
||||
Change:
|
||||
|
||||
- Windows clipboard restore delay changed from `150ms` to `750ms`
|
||||
- restore now runs on a background thread instead of blocking the insert path
|
||||
- Linux keeps the previous `150ms` behavior
|
||||
|
||||
## Verification run
|
||||
|
||||
Commands:
|
||||
|
||||
```powershell
|
||||
cargo fmt --all
|
||||
cargo check --lib
|
||||
cargo test --lib --no-run
|
||||
cargo check --tests
|
||||
powershell -NoProfile -Command "[void][scriptblock]::Create((Get-Content -Raw 'openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1')); 'script-parse-ok'"
|
||||
```
|
||||
|
||||
Observed result:
|
||||
|
||||
- compile/check passed
|
||||
- test binaries compiled
|
||||
- new smoke scripts parse successfully
|
||||
- real desktop automation passed on:
|
||||
- Windows Terminal `CMD` tab at `150ms` and `750ms`
|
||||
- Windows Terminal `PowerShell` tab at `150ms` and `750ms`
|
||||
- Notepad at `150ms` and `750ms`
|
||||
|
||||
## Remaining gap
|
||||
|
||||
Still needed if we want to exactly mirror the original user report:
|
||||
|
||||
- drive **OpenLess itself** through the full dictation lifecycle in the same run
|
||||
- keep the target specifically in the same terminal/input setup where the stale paste was originally observed
|
||||
- capture whether the failing case depends on:
|
||||
- OpenLess focus-target restore timing
|
||||
- ASR/polish latency
|
||||
- the exact terminal host/session state
|
||||
- another app-specific delay not present in the isolated paste harness
|
||||
|
||||
## Suggested issue / PR title
|
||||
|
||||
- Issue: `[windows][insertion] terminal paste can restore stale clipboard before synthetic paste lands`
|
||||
- PR: `fix(windows): delay clipboard restore after synthetic paste`
|
||||
@@ -1,293 +0,0 @@
|
||||
# OpenLess 系统化工程审计报告
|
||||
|
||||
> 审计日期：2026-05-06
|
||||
> 项目版本：1.2.20
|
||||
> 审计范围：`openless-all/app/` 主项目（Rust 后端 + React 前端）
|
||||
|
||||
---
|
||||
|
||||
## 一、后端检查与优化
|
||||
|
||||
### 1.1 架构总评
|
||||
|
||||
后端整体架构清晰，遵循「单 Coordinator 状态机 + 独立叶子模块」的分层设计。模块之间只通过 `types.rs` 共享类型，跨模块调用全部收敛到 `coordinator.rs`，与 CLAUDE.md 约定的架构一致。代码质量整体较高：大量使用 `thiserror` / `anyhow` 进行错误处理，关键路径有 `#[cfg(test)]` 单元测试覆盖，热路径有状态竞态保护。
|
||||
|
||||
**模块清单**：`asr/`（火山引擎流式 + Whisper HTTP + 本地 Qwen3-ASR）、`polish.rs`（OpenAI-compatible LLM）、`hotkey.rs`（macOS CGEventTap / Windows WH_KEYBOARD_LL / Linux rdev）、`recorder.rs`（cpal 音频采集）、`insertion.rs`（跨平台文本插入）、`persistence.rs`（JSON 文件 + OS 凭据库）、`permissions.rs`（TCC 权限）、`selection.rs`（划词捕获）。
|
||||
|
||||
### 1.2 值得优化的后端问题
|
||||
|
||||
#### 问题 A：`coordinator.rs` 过于臃肿（严重程度：中）
|
||||
|
||||
`coordinator.rs` 当前 **3842 行**，包含了 dictation 状态机、QA 状态机、双 hotkey supervisor 循环、recorder 错误监控、Windows IME 会话管理、capsule 事件发射、录音 mute 管理、本地 ASR 预加载/释放等全部胶水逻辑。单一文件内职责过多。
|
||||
|
||||
**建议**：按子状态机拆分为多个 coordinator 子模块：
|
||||
- `coordinator/dictation.rs` — 主听写 session 生命周期
|
||||
- `coordinator/qa.rs` — QA 划词追问 session 生命周期
|
||||
- `coordinator/resources.rs` — recorder / ASR / mute 等资源管理
|
||||
|
||||
#### 问题 B：`commands.rs` 包含过多业务逻辑（严重程度：中）
|
||||
|
||||
`commands.rs` 中有大量本不属于「IPC 薄层」的业务逻辑，例如：
|
||||
- WAV 静音文件编码（`encode_wav_16k_mono_silence`）
|
||||
- ASR 端点 URL 拼接（`asr_transcriptions_url`）
|
||||
- 模型列表 JSON 解析（`parse_model_ids`）
|
||||
- LLM/ASR provider 连接验证（`validate_llm_provider` / `validate_asr_provider`）
|
||||
|
||||
这些应该下沉到对应的叶子模块（`asr/` 或 `polish.rs`），`commands.rs` 只做参数接收和类型转换。
|
||||
|
||||
#### 问题 C：平台条件编译代码分散（严重程度：低）
|
||||
|
||||
`lib.rs`、`coordinator.rs`、`insertion.rs` 中大量 `#[cfg(target_os = "macos")]` / `#[cfg(target_os = "windows")]` 块散落在主流程代码中。虽然不是编译期问题，但降低了可读性。建议将平台适配代码集中到 `platform/` 子模块，用 trait 抽象。
|
||||
|
||||
#### 问题 D：本地 ASR 引擎缓存释放策略依赖时间阈值（严重程度：低）
|
||||
|
||||
`LocalAsrCache` 的释放依赖 `local_asr_keep_loaded_secs` 定时器（默认 300 秒）。如果用户在 5 分钟内未再次使用，引擎释放。但在 Windows 上本地 ASR 引擎根本不可用（仅在 macOS 编译），相关代码却仍在 coordinator 中占据逻辑分支。建议将平台不可用的功能路径在编译期完全消除，而非运行时静默跳过。
|
||||
|
||||
#### 问题 E：Volcengine 凭据与通用 Provider 凭据同时存在（严重程度：低）
|
||||
|
||||
系统中同时维护了 Volcengine 专用凭据字段（`volcengine_app_key` / `volcengine_access_key` / `volcengine_resource_id`）和通用 Provider 凭据路径（`asr_api_key` / `asr_endpoint`），导致 `get_credentials` 返回的 `CredentialsStatus` 需要同时维护 `volcengineConfigured` 和 `asrConfigured` 两个字段。历史迁移可理解，但长期维护增加复杂度。
|
||||
|
||||
### 1.3 后端与其他应用混杂检查
|
||||
|
||||
经检查，`openless-all/app/` 是纯净的单一 Tauri 项目，未发现与其他应用混杂的代码。但仓库根目录存在以下与主项目无关的目录：
|
||||
|
||||
| 目录 | 内容 | 建议 |
|
||||
|------|------|------|
|
||||
| `promo-openless/` | Remotion 宣传视频项目 | 移至独立仓库或 `marketing/` 子目录 |
|
||||
| `promo-openless-v2/` | Remotion 宣传视频 v2 | 同上 |
|
||||
| `SC/` | 录屏素材文件（.mov / .mp4） | 建议移出仓库或用 Git LFS |
|
||||
| `docs/` | 开发调研文档 | 保留，但与主项目解耦 |
|
||||
|
||||
当前这些目录虽然在 git 仓库中，但不会参与 Tauri 构建，不会导致臃肿或冲突。**与用户提到的「此前与其他软件冲突或臃肿问题」对比，当前架构没有重复该问题。**
|
||||
|
||||
---
|
||||
|
||||
## 二、前端检查
|
||||
|
||||
### 2.1 UI Bug 分析
|
||||
|
||||
#### Bug 1：Tab 切换动画的双重渲染竞态（严重程度：中）
|
||||
|
||||
`FloatingShell.tsx` 的 tab 切换使用 `displayTab` / `tabPhase` 机制：旧页先播 `ol-page-fadeout`（180ms），之后切 `displayTab` 并播 `ol-page-slide`。但 `key={displayTab}` 会让 React 在 `displayTab` 改变时**卸载旧组件树并挂载新组件树**。问题：
|
||||
|
||||
- 如果用户在 180ms 内快速切换两次 tab，第一次的 timeout 触发时 `displayTab` 已被第二次覆盖，会看到页面闪变
|
||||
- 旧页的 `useEffect` cleanup 和新页的 `useEffect` 在 180ms 内交错执行，若两者都触发了 IPC 调用，会产生竞态
|
||||
|
||||
**修复建议**：使用 `useTransition` 或 CSS `animationend` 事件代替固定 `setTimeout`，确保动画结束后再切 DOM。
|
||||
|
||||
#### Bug 2：Capsule 窗口 `state === 'idle'` 时返回空 div（严重程度：低）
|
||||
|
||||
`Capsule.tsx` 的 `if (state === 'idle') { return <div style={{ width: 0, height: 0 }} />; }`。问题：胶囊窗口尺寸由 Rust 端 `position_capsule_bottom_center` 设定（220×110），但 React 返回 0×0 的 div 时，Tauri webview 的 CSS 尺寸与窗口尺寸不一致。在 Windows 上可能导致透明区域响应鼠标事件（mouse event 穿透到下层窗口）。
|
||||
|
||||
#### Bug 3：QA 浮窗滚动容器缺少 `overflow-anchor`（严重程度：低）
|
||||
|
||||
`QaPanel.tsx` 的流式答案到达时，用 `scrollRef.current.scrollTop = scrollRef.current.scrollHeight` 手动滚到底。如果用户在流式过程中手动向上滚动查看前面的消息，新 chunk 到达时会强制跳回底部，打断阅读。应该加入「用户是否主动滚离底部」的检测（类似聊天的 scroll-to-bottom 按钮逻辑）。
|
||||
|
||||
#### Bug 4：`dangerouslySetInnerHTML` 的 XSS 表面（严重程度：低）
|
||||
|
||||
`QaPanel.tsx` 和 `StreamingAssistantBubble` 使用 `dangerouslySetInnerHTML` 渲染 Markdown。虽然 `renderQaMarkdown` 使用 `marked` 库且配置了 sanitize，但在流式场景下不完整的 Markdown 可能导致 HTML 结构断裂（如未闭合的 `<code>` 块）。当前有 fallback 到 `renderQaPlainText`，但错误边界不覆盖 dangerouslySetInnerHTML 渲染错误。
|
||||
|
||||
### 2.2 动效与交互流畅度
|
||||
|
||||
**当前状态良好**：
|
||||
- 胶囊波形的 `AudioBars` 使用 `cubic-bezier(0.22, 1, 0.36, 1)` 缓动曲线，过渡平滑
|
||||
- 所有 transition 使用 CSS 变量 `var(--ol-motion-*)` 统一缓动
|
||||
- `willChange` 属性在动画元素上正确设置（Capsule 的 `transform, box-shadow`）
|
||||
|
||||
**可优化的点**：
|
||||
|
||||
1. **音频电平更新频率**：`LEVEL_EMIT_MIN_INTERVAL_MS = 33`（~30Hz），配合 CSS 0.18s transition 效果尚可。但如果窗口失去焦点时 `requestAnimationFrame` 降频，可能出现电平条「冻住」的观感。建议在 coordinator 侧用 `setInterval` 兜底。
|
||||
|
||||
2. **QA 浮窗出场动画缺失**：`QaPanel` 关闭时直接 `hide()`，没有退场动画。可以加一个 `qa:fadeout` 事件让前端先播动画，100ms 后再由 Rust 端 actual hide。
|
||||
|
||||
3. **SettingsModal 无入场动画**：Settings 弹窗使用 `animation: 'ol-prompt-pop 0.26s var(--ol-motion-spring)'`，但关闭时瞬间消失。
|
||||
|
||||
---
|
||||
|
||||
## 三、项目工程化与功能完善
|
||||
|
||||
### 3.1 工程化水平评估
|
||||
|
||||
#### 优点
|
||||
|
||||
- **模块化清晰**：Rust 端严格遵循「叶子模块只依赖 types，胶水只写在 coordinator」的约定
|
||||
- **错误处理完备**：关键路径全部使用 `Result<T, E>`，无 `unwrap()` 裸奔
|
||||
- **测试覆盖**：`commands.rs` 和 `polish.rs` 有单元测试，`persistence.rs` 有集成测试
|
||||
- **类型安全 IPC**：前后端类型通过 `types.rs` ↔ `types.ts` 镜像定义，序列化字段名一致
|
||||
- **Mock 支持**：前端 `invokeOrMock` 允许在浏览器中脱离 Tauri 环境开发
|
||||
- **i18n 国际化**：支持 zh-CN、en、ja、ko、zh-TW 五种语言
|
||||
- **自动更新**：Tauri updater 插件集成完整
|
||||
|
||||
#### 可改善
|
||||
|
||||
1. **缺少 CI 质量门禁**：当前只有 `release-tauri.yml` 构建流水线，没有 lint / test / typecheck 门禁（虽然有 Rust `cargo check` 和前端 `tsc` 命令，但未在 CI 强制）。
|
||||
|
||||
2. **缺少 E2E 测试**：没有端到端测试（如 Playwright + Tauri driver），无法验证「按热键 → 录音 → 插入」的完整链路。
|
||||
|
||||
3. **`.gitignore` 不完整**：`node_modules/` 出现在多个子目录（`promo-openless/`、`promo-openless-v2/`），但根 `.gitignore` 未统一管理。
|
||||
|
||||
4. **版本号同步风险**：CLAUDE.md 指出需要同时更新 `package.json`、`tauri.conf.json`、`Cargo.toml` 三处的版本号，容易遗漏。建议用脚本或 workspace 版本管理。
|
||||
|
||||
### 3.2 功能完整性
|
||||
|
||||
当前功能矩阵：
|
||||
|
||||
| 功能 | macOS | Windows | 备注 |
|
||||
|------|-------|---------|------|
|
||||
| 全局热键听写 | ✅ | ✅ | macOS CGEventTap / Windows WH_KEYBOARD_LL |
|
||||
| 火山引擎流式 ASR | ✅ | ✅ | |
|
||||
| Whisper HTTP ASR | ✅ | ✅ | |
|
||||
| 本地 Qwen3-ASR | ✅ | ❌ | 仅 macOS 编译 |
|
||||
| LLM 润色（四种模式） | ✅ | ✅ | |
|
||||
| LLM 翻译模式 | ✅ | ✅ | Shift 修饰键触发 |
|
||||
| 划词 QA 问答 | ✅ | ✅ | 双 hotkey 架构 |
|
||||
| 热词词典 | ✅ | ✅ | |
|
||||
| 历史记录 | ✅ | ✅ | |
|
||||
| 开机自启 | ✅ | ✅ | |
|
||||
| 自动更新 | ✅ | ✅ | |
|
||||
| 录音时系统静音 | ✅ | ✅ | |
|
||||
| Windows IME 直写 | N/A | ✅ | C++ TSF 模块 |
|
||||
| 系统托盘图标 | ✅ | ✅ | |
|
||||
|
||||
---
|
||||
|
||||
## 四、多端逻辑与体验一致性
|
||||
|
||||
### 4.1 平台差异对比
|
||||
|
||||
| 维度 | macOS | Windows | 一致性 |
|
||||
|------|-------|---------|--------|
|
||||
| Capsule 物理尺寸 | 220×110，visual height 96 | 220×84(118)，visual height 52 | ⚠️ 不一致 |
|
||||
| 插入策略 | AX 直写（通过 Accessibility API） | enigo 模拟粘贴 / TSF | 不同，但策略合理 |
|
||||
| 窗口圆角 | 系统原生圆角 | 手动 CreateRoundRectRgn(18px) | ⚠️ 视觉差异 |
|
||||
| 窗口背景 | NSVisualEffectView 磨砂 | Mica + 自定义渐变 | ⚠️ 视觉差异 |
|
||||
| 默认热键 | 右 Option（Toggle） | 右 Control（Toggle） | 不一致，但符合平台惯例 |
|
||||
| QA 默认热键 | Cmd+Shift+; | Ctrl+Shift+; | 符合平台惯例 ✅ |
|
||||
| 启动权限检查 | 阻塞式弹窗检查 | 异步轮询（最多 10s） | 合理差异 |
|
||||
| 窗口控制按钮 | 系统红黄绿 | 自绘最小化/最大化/关闭 | 合理差异 |
|
||||
|
||||
### 4.2 逻辑完备性
|
||||
|
||||
- **翻译模式**：两端均通过 Shift 修饰键触发，但 macOS 用 `flagsChanged` 事件，Windows 用 `WH_KEYBOARD_LL` 的 Shift 边沿。逻辑层在 coordinator 中以 `translation_modifier_seen` flag 统一，完备。
|
||||
- **QA 浮窗拖动**：macOS 用 `movableByWindowBackground`，Windows 用 `data-tauri-drag-region`。两端都能拖，但 macOS 整窗口可拖，Windows 仅 toolbar 区域可拖。
|
||||
- **粘贴后剪贴板恢复**：Windows/Linux 支持（`restore_clipboard_after_paste`），macOS 走 AX 直写不涉及剪贴板。完备。
|
||||
- **降级兜底**：插入失败 → 文本留在剪贴板，用户可手动粘贴。两端一致。
|
||||
|
||||
### 4.3 需要关注的多端差异
|
||||
|
||||
1. **Windows Capsule 翻译模式高度变化**：翻译模式激活时 capsule 窗口从 84→118，macOS 保持 110 不变。设计合理但视觉差异可能困扰跨平台用户。
|
||||
|
||||
2. **Windows WindowChrome 自绘标题栏**：`WindowChrome.tsx` 的 `WinTitleBar` 绘制的关闭按钮走 `getCurrentWindow().close()`，但 Rust 端 `RunEvent::WindowEvent::CloseRequested` 只对 `label == "main"` 做 `prevent_close` + `hide`，其他窗口（capsule / qa）的关闭行为不一致。
|
||||
|
||||
3. **capsule 窗口 `skipTaskbar`**：两端都设了，正确。
|
||||
|
||||
---
|
||||
|
||||
## 五、UI 与后端接口映射校验
|
||||
|
||||
### 5.1 前后端命令对照
|
||||
|
||||
经逐一核对 `invoke_handler!` 宏（`lib.rs`）与 `ipc.ts` 的函数导出，**所有后端命令在前端都有对应的 TypeScript wrapper**：
|
||||
|
||||
| 后端命令 | 前端函数 | 状态 |
|
||||
|----------|----------|------|
|
||||
| `get_settings` | `getSettings()` | ✅ |
|
||||
| `set_settings` | `setSettings(prefs)` | ✅ |
|
||||
| `get_hotkey_status` | `getHotkeyStatus()` | ✅ |
|
||||
| `get_hotkey_capability` | `getHotkeyCapability()` | ✅ |
|
||||
| `get_windows_ime_status` | `getWindowsImeStatus()` | ✅ |
|
||||
| `get_credentials` | `getCredentials()` | ✅ |
|
||||
| `set_credential` | `setCredential(account, value)` | ✅ |
|
||||
| `read_credential` | `readCredential(account)` | ✅ |
|
||||
| `set_active_asr_provider` | `setActiveAsrProvider(provider)` | ✅ |
|
||||
| `set_active_llm_provider` | `setActiveLlmProvider(provider)` | ✅ |
|
||||
| `validate_provider_credentials` | `validateProviderCredentials(kind)` | ✅ |
|
||||
| `list_provider_models` | `listProviderModels(kind)` | ✅ |
|
||||
| `list_history` | `listHistory()` | ✅ |
|
||||
| `delete_history_entry` | `deleteHistoryEntry(id)` | ✅ |
|
||||
| `clear_history` | `clearHistory()` | ✅ |
|
||||
| `list_vocab` | `listVocab()` | ✅ |
|
||||
| `add_vocab` | `addVocab(phrase, note)` | ✅ |
|
||||
| `remove_vocab` | `removeVocab(id)` | ✅ |
|
||||
| `set_vocab_enabled` | `setVocabEnabled(id, enabled)` | ✅ |
|
||||
| `list_vocab_presets` | `listVocabPresets()` | ✅ |
|
||||
| `save_vocab_presets` | `saveVocabPresets(store)` | ✅ |
|
||||
| `start_dictation` | `startDictation()` | ✅ |
|
||||
| `stop_dictation` | `stopDictation()` | ✅ |
|
||||
| `cancel_dictation` | `cancelDictation()` | ✅ |
|
||||
| `handle_window_hotkey_event` | `handleWindowHotkeyEvent(...)` | ✅ |
|
||||
| `inject_hotkey_click_for_dev` | N/A（debug only） | ✅ |
|
||||
| `repolish` | `repolish(rawText, mode)` | ✅ |
|
||||
| `set_default_polish_mode` | `setDefaultPolishMode(mode)` | ✅ |
|
||||
| `set_style_enabled` | `setStyleEnabled(mode, enabled)` | ✅ |
|
||||
| `check_accessibility_permission` | `checkAccessibilityPermission()` | ✅ |
|
||||
| `request_accessibility_permission` | `requestAccessibilityPermission()` | ✅ |
|
||||
| `check_microphone_permission` | `checkMicrophonePermission()` | ✅ |
|
||||
| `request_microphone_permission` | `requestMicrophonePermission()` | ✅ |
|
||||
| `open_system_settings` | `openSystemSettings(pane)` | ✅ |
|
||||
| `trigger_microphone_prompt` | `triggerMicrophonePrompt()` | ✅ |
|
||||
| `export_error_log` | `exportErrorLog(name)` | ✅ |
|
||||
| `restart_app` | `restartApp()` | ✅ |
|
||||
| `get_qa_hotkey_label` | `getQaHotkeyLabel()` | ✅ |
|
||||
| `set_qa_hotkey` | `setQaHotkey(binding)` | ✅ |
|
||||
| `qa_window_dismiss` | `qaWindowDismiss()` | ✅ |
|
||||
| `qa_window_pin` | `qaWindowPin(pinned)` | ✅ |
|
||||
| `local_asr_*` (15 个命令) | 对应的 15 个函数 | ✅ |
|
||||
|
||||
**结论**：UI 与后端接口完全 1:1 对应，无遗漏。
|
||||
|
||||
### 5.2 事件订阅对照
|
||||
|
||||
| 事件名 | 发射端 | 订阅端 | 状态 |
|
||||
|--------|--------|--------|------|
|
||||
| `capsule:state` | coordinator | Capsule.tsx | ✅ |
|
||||
| `qa:state` | coordinator | QaPanel.tsx | ✅ |
|
||||
| `qa:dismiss` | coordinator | QaPanel.tsx | ✅ |
|
||||
| `qa:level` | coordinator | QaPanel.tsx | ✅ |
|
||||
| `prefs:changed` | commands::set_settings | QaPanel.tsx | ✅ |
|
||||
| `local-asr:download-progress` | DownloadManager | LocalAsr.tsx | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 六、改进建议汇总
|
||||
|
||||
### 立即修复（P0）
|
||||
|
||||
无。当前版本功能完整，无明显崩溃或数据丢失风险。
|
||||
|
||||
### 短期优化（P1 — 建议在下一版迭代中处理）
|
||||
|
||||
| # | 问题 | 位置 | 工作量 |
|
||||
|---|------|------|--------|
|
||||
| 1 | coordinator.rs 拆分 | 后端 | 2-3h |
|
||||
| 2 | commands.rs 业务逻辑下沉 | 后端 | 1-2h |
|
||||
| 3 | Tab 切换动画竞态修复 | FloatingShell.tsx | 1h |
|
||||
| 4 | QA 流式滚动打断问题 | QaPanel.tsx | 1h |
|
||||
| 5 | SettingsModal / QaPanel 退场动画 | 前端 | 1h |
|
||||
|
||||
### 中长期改善（P2）
|
||||
|
||||
| # | 问题 | 建议 |
|
||||
|---|------|------|
|
||||
| 1 | 缺少 CI lint/test 门禁 | 添加 GitHub Actions workflow：`cargo clippy` + `cargo test` + `npx tsc --noEmit` |
|
||||
| 2 | 缺少 E2E 测试 | 引入 Playwright + Tauri driver 测试核心链路 |
|
||||
| 3 | 平台代码分散 | 创建 `src-tauri/src/platform/` 模块，用 trait 抽象平台差异 |
|
||||
| 4 | 版本号同步 | 用 workspace Cargo.toml + 构建脚本自动同步三处版本号 |
|
||||
| 5 | 仓库清理 | `promo-openless/`、`SC/` 移至独立仓库或 Git LFS |
|
||||
| 6 | Windows Capsule 尺寸与 macOS 视觉差异 | 文档化或统一 visual height |
|
||||
|
||||
---
|
||||
|
||||
## 七、总结
|
||||
|
||||
OpenLess 1.2.20 的代码质量在同类开源项目中属于**中上水平**。架构设计清晰（单 Coordinator + 叶子模块），错误处理完备，前后端类型安全，IPC 接口 1:1 映射无遗漏。
|
||||
|
||||
核心问题集中在：
|
||||
1. **coordinator.rs 的单一文件过大**（3842 行），需要拆分子状态机
|
||||
2. **commands.rs 的业务逻辑应下沉**到叶子模块
|
||||
3. **前端 Tab 切换的竞态**可能导致动画异常
|
||||
4. **QA 浮窗的流式滚动打断用户体验**
|
||||
|
||||
没有发现「与其他应用混杂」或「耦合臃肿」的问题——代码遵循了严格的模块隔离约定。多平台覆盖完整，macOS 和 Windows 的核心行为一致，仅在视觉尺寸、窗口装饰等平台原生差异上有所不同。
|
||||
|
||||
**项目整体健康。建议在下个迭代中优先处理 coordinator 拆分和前端动画竞态修复。**
|
||||
@@ -1,700 +0,0 @@
|
||||
# Audit Validation — 2026-05-10
|
||||
|
||||
Validation of the 21 audit items against the current source tree at
|
||||
`openless-all/app/src-tauri/src/`. Every CONFIRMED finding cites the
|
||||
exact file and line range read from the working copy on 2026-05-10.
|
||||
|
||||
## Summary
|
||||
|
||||
| ID | Severity | Status | One-line |
|
||||
|----|----------|--------|----------|
|
||||
| 2.2.1 | — | CONFIRMED | TS `UserPreferences` interface and `mockSettings` both miss `updateChannel` |
|
||||
| 3.1.1 | 严重 | CONFIRMED | `MacHotkeyAdapter` does not override `HotkeyAdapter::shutdown`; `CFRunLoopRun` runs forever |
|
||||
| 3.1.2 | 中 | CONFIRMED | `hotkey_supervisor_loop` is `loop { ... sleep(3s) }` with no exit signal |
|
||||
| 3.1.3 | 中 | CONFIRMED | `start_listener_thread` spawns the listener and drops the `JoinHandle` |
|
||||
| 3.2.1 | 高 | FALSE_POSITIVE | Channel is `std::sync::mpsc::channel()` (unbounded async); `tx.send` does not block |
|
||||
| 3.2.2 | 高 | CONFIRMED | `emit_capsule` calls `window.show()/hide()` from the cpal `process_callback` thread |
|
||||
| 3.2.3 | 中 | CONFIRMED | `inner.inserter.insert(...)` runs sync `arboard`+`enigo` from async `end_session` |
|
||||
| 3.2.4 | 中 | CONFIRMED | `AudioMuteGuard::activate` shells out to `osascript` / `wpctl` / `pactl` synchronously |
|
||||
| 3.2.5 | 低 | CONFIRMED (Linux only) | `probe_input_stream` calls `thread::sleep(120ms)` from the async permission gate |
|
||||
| 3.3.1 | 高 | CONFIRMED | `handle_pressed_edge` routes to QA when `panel_visible=true` regardless of dictation phase |
|
||||
| 3.3.2 | 中 | PARTIAL | Two bridge loops both touch `state` for the same modifier event; non-fatal contention, no integrity bug |
|
||||
| 3.3.3 | 中 | FALSE_POSITIVE | Cancelled doesn't reset coordinator latch, but the OS-side `Shared::trigger_held` already gates auto-repeat |
|
||||
| 3.3.4 | 中 | CONFIRMED | `open_qa_panel` always emits `CapsuleState::Idle`, clobbering any in-flight dictation capsule |
|
||||
| 3.3.5 | 低 | CONFIRMED | `finish_cancel_session_state` skips `focus_target = None` when `phase == Processing` |
|
||||
| 3.3.6 | 低 | FALSE_POSITIVE | `take_current_prepared_windows_ime_session_for_restore` removes the slot on first call; second call is a true no-op |
|
||||
| 3.4.1 | 中 (advisory) | ADVISORY_ONLY | `Inner` carries 16 `Mutex` + 4 `AtomicBool` fields (20 concurrent fields) |
|
||||
| 3.4.2 | 中 (advisory) | ADVISORY_ONLY | 66 of 67 `Ordering` usages in coordinator/hotkey are `SeqCst` |
|
||||
| 3.4.3 | 低 (advisory) | ADVISORY_ONLY | ~102 `unsafe`/`unsafe fn`/`unsafe impl`/`unsafe extern` sites; many lack SAFETY comments |
|
||||
| 3.4.4 | 低 | CONFIRMED | `start_dispatcher` in `global_hotkey_runtime.rs` is `loop {}` with no exit |
|
||||
| 20 (NEW) | — | FALSE_POSITIVE | `read_or_default` already falls back to `UserPreferences::default()` on decode failure; `expect()` only fires on filesystem errors |
|
||||
| 2.3.3 | — | CONFIRMED (no action) | All four backend events (`capsule:state`, `qa:state`, `qa:level`, `vocab:updated`) have matching frontend listeners |
|
||||
|
||||
**Tally**: 11 CONFIRMED · 4 FALSE_POSITIVE · 1 PARTIAL · 3 ADVISORY_ONLY · 1 CONFIRMED-no-action · 1 CONFIRMED-Linux-only
|
||||
|
||||
## Recommended PR groupings
|
||||
|
||||
Group by file to minimize merge conflict risk. Suggested order:
|
||||
|
||||
1. **PR A — `hotkey.rs` lifecycle** (3.1.1, 3.1.3): add `MacHotkeyAdapter::shutdown` (post a synthetic `CFRunLoopStop`/`CFRunLoopWakeUp` from `Drop`) and store the listener `JoinHandle` so panics surface. Same file, same review.
|
||||
2. **PR B — TS type alignment** (2.2.1): add `updateChannel: UpdateChannel` to `src/lib/types.ts` and `mockSettings` in `src/lib/ipc.ts`. One file pair, trivial.
|
||||
3. **PR C — coordinator hotkey supervisor exits** (3.1.2, 3.4.4): add an `AtomicBool` shutdown flag to `hotkey_supervisor_loop` and `global_hotkey_runtime::start_dispatcher`. Same module concern, no overlap with PR A.
|
||||
4. **PR D — async hygiene** (3.2.3, 3.2.4, 3.2.5): wrap `inserter.insert`, `AudioMuteGuard::activate`, and `probe_input_stream` in `tokio::task::spawn_blocking`. Touches `coordinator/dictation.rs`, `coordinator/resources.rs`, and `coordinator.rs` — coordinate with PR E to avoid shared-line conflicts.
|
||||
5. **PR E — QA / dictation routing race** (3.3.1, 3.3.4): make `handle_pressed_edge` consult dictation phase before routing to QA, and skip the `Idle` capsule clobber in `open_qa_panel` when dictation is active. Same file (`coordinator/qa.rs` + `coordinator/dictation.rs`).
|
||||
6. **PR F — capsule UI thread marshaling** (3.2.2): bounce `window.show()/hide()` through `app.run_on_main_thread`; emit-only path can stay (Tauri marshals events internally). Touches `coordinator.rs::emit_capsule`. Independent of PR E.
|
||||
7. **PR G — focus_target leak fix** (3.3.5): in `finish_cancel_session_state`, also clear `focus_target` when the cancelled phase is `Processing`. Pure `coordinator_state.rs` edit.
|
||||
|
||||
Advisory items (3.4.1 / 3.4.2 / 3.4.3) need no PR; they are tracked here for future hardening.
|
||||
|
||||
## Detail per item
|
||||
|
||||
### 2.2.1 — `updateChannel` missing in TS types
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/types.rs:216-219`, `openless-all/app/src/lib/types.ts:118-177`, `openless-all/app/src/lib/ipc.ts:46-79`
|
||||
|
||||
**Evidence (Rust source of truth)**:
|
||||
```rust
|
||||
// types.rs:216-219
|
||||
/// Auto-update 渠道偏好。stable = 跟正式版（默认）；beta = Settings 里多
|
||||
/// 一个手动下载 Beta 的入口。不影响 plugin-updater 的自动检查路径。
|
||||
#[serde(default)]
|
||||
pub update_channel: UpdateChannel,
|
||||
```
|
||||
|
||||
**Evidence (TS gap)**: `UserPreferences` ends at `startMinimized: boolean;` (line 176). No `updateChannel` field. `mockSettings` (ipc.ts:46-79) ends at `startMinimized: false,`. No `updateChannel` key.
|
||||
|
||||
**Notes**: Channel state today is read/written via separate `getUpdateChannel` / `setUpdateChannel` IPC commands (`ipc.ts:170-176`), so `getSettings()` still works — the TS shape is just lying about what the Rust backend actually serializes. Setting via `setSettings(prefs)` round-trips through Rust's `UserPreferencesWire` which `#[serde(default)]`-fills the field, so currently no data corruption, but the type is incorrect and any consumer that destructures `UserPreferences` will silently miss the field. Trivial fix.
|
||||
|
||||
---
|
||||
|
||||
### 3.1.1 — `MacHotkeyAdapter::shutdown` empty [严重]
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/hotkey.rs:111` (default), `:301-325` (`MacHotkeyAdapter` impl), `:419-458` (`run_listen_loop`), contrast `:769-776` (Windows shutdown).
|
||||
|
||||
**Evidence (default)**:
|
||||
```rust
|
||||
// hotkey.rs:102-112
|
||||
pub trait HotkeyAdapter: Send + Sync {
|
||||
fn kind(&self) -> HotkeyAdapterKind;
|
||||
fn update_binding(&self, binding: HotkeyBinding);
|
||||
fn update_modifier_shortcuts(...);
|
||||
fn reset_held_state(&self);
|
||||
fn shutdown(&self) {}
|
||||
}
|
||||
```
|
||||
|
||||
**Evidence (mac adapter impl is silent on `shutdown`)**:
|
||||
```rust
|
||||
// hotkey.rs:305-325
|
||||
impl HotkeyAdapter for MacHotkeyAdapter {
|
||||
fn kind(&self) -> HotkeyAdapterKind { ... }
|
||||
fn update_binding(&self, binding: HotkeyBinding) { ... }
|
||||
fn update_modifier_shortcuts(...) { ... }
|
||||
fn reset_held_state(&self) { reset_shared_held_state(&self.shared); }
|
||||
// <-- no shutdown override
|
||||
}
|
||||
```
|
||||
|
||||
**Evidence (no exit path)**:
|
||||
```rust
|
||||
// hotkey.rs:454-457
|
||||
log::info!("[hotkey] CGEventTap 已启动");
|
||||
let _ = status_tx.send(Ok(()));
|
||||
CFRunLoopRun();
|
||||
// CFRunLoopRun never returns absent CFRunLoopStop; the listener thread leaks.
|
||||
```
|
||||
|
||||
**Notes**: `Drop for HotkeyMonitor` does call `self.adapter.shutdown()` (line 170-174), but Mac falls through to the empty default. On every preference-driven monitor swap the old `CFRunLoop` thread + tap leak. Visible on macOS as a steady leak of background threads on long-running sessions that change hotkey bindings.
|
||||
|
||||
**Fix sketch**: Store the `CfRunLoopRef` returned by `CFRunLoopGetCurrent()` (currently captured in `run_listen_loop` only) in the `MacHotkeyAdapter`, plus the `CfMachPortRef`. On `shutdown`, call `CGEventTapEnable(tap, false)` then `CFRunLoopStop(rl)`. Both are FFI-safe to call from any thread. Mirror the Windows pattern (`PostThreadMessageW(thread_id, WM_QUIT)`). Storing those refs needs `CallbackContext` exposed to the adapter, easiest via shared `parking_lot::Mutex`.
|
||||
|
||||
---
|
||||
|
||||
### 3.1.2 — `hotkey_supervisor_loop` no-exit
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator.rs:766-833`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator.rs:766-833
|
||||
fn hotkey_supervisor_loop(inner: Arc<Inner>) {
|
||||
let mut attempts: u32 = 0;
|
||||
let capability = HotkeyMonitor::capability();
|
||||
loop {
|
||||
let prefs = inner.prefs.get();
|
||||
if inner.hotkey.lock().is_some() { return; }
|
||||
// ... try start, on failure:
|
||||
std::thread::sleep(std::time::Duration::from_secs(3));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: The only successful exit is when the hotkey is already installed (line 772-774). On error the supervisor keeps spinning. There is no `AtomicBool` / `Sender<()>` shutdown signal exposed for app shutdown. Same pattern repeats in `qa_hotkey_supervisor_loop`, `combo_hotkey_supervisor_loop`, `translation_hotkey_supervisor_loop`, `action_hotkey_supervisor_loop`. Worth a single shared shutdown flag.
|
||||
|
||||
---
|
||||
|
||||
### 3.1.3 — `start_listener_thread` drops `JoinHandle`
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/hotkey.rs:196-229`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// hotkey.rs:218-228
|
||||
let (status_tx, status_rx) = mpsc::channel::<Result<T, HotkeyInstallError>>();
|
||||
std::thread::Builder::new()
|
||||
.name(thread_name.into())
|
||||
.spawn(move || run_listen_loop(thread_shared, tx, status_tx))
|
||||
.map_err(|e| install_error("spawn_failed", format!("hotkey 线程启动失败: {e}")))?;
|
||||
|
||||
match status_rx.recv_timeout(Duration::from_secs(3)) { ... }
|
||||
```
|
||||
|
||||
**Notes**: The `Result<JoinHandle, _>` from `spawn(...)` is `?`'d for the spawn error, but the `JoinHandle` itself is silently dropped (the spawn return value isn't bound). `ListenerThread<T>` only stores `shared` and a single `startup` value. If the listener panics (e.g. `parking_lot::RwLock` poisoning, FFI bug), there is no path for the supervisor to learn about it — the channel just stops receiving. Storing the handle and using `JoinHandle::is_finished()` (Rust 1.61+) or pairing with a "thread alive" `AtomicBool` would let the supervisor restart the listener on panic.
|
||||
|
||||
---
|
||||
|
||||
### 3.2.1 — Blocking `tx.send` in event-tap callback
|
||||
**Status**: FALSE_POSITIVE
|
||||
**Files**: `openless-all/app/src-tauri/src/hotkey.rs:183-187`, `:218`, `coordinator.rs:650`, `:781`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// hotkey.rs:183-187
|
||||
fn send_or_log(tx: &Sender<HotkeyEvent>, evt: HotkeyEvent) {
|
||||
if let Err(e) = tx.send(evt) {
|
||||
log::warn!("[hotkey] 事件发送失败: {e}");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// coordinator.rs:650 (and :781)
|
||||
let (tx, rx) = mpsc::channel::<HotkeyEvent>();
|
||||
```
|
||||
|
||||
**Notes**: `std::sync::mpsc::channel()` is the **unbounded asynchronous** variant — `Sender::send` only blocks-and-fails when the receiver has been dropped, in which case it returns `Err(SendError)` immediately. There is no "rendezvous" backpressure. The only way `tx.send` would block long enough to trip `kCGEventTapDisabledByTimeout` is if std mpsc internally allocated under contention, which would be milliseconds at worst, not the seconds-scale macOS uses for the tap-disabled timeout. The audit conflated `mpsc::channel()` with `mpsc::sync_channel(0)` (rendezvous). No fix needed.
|
||||
|
||||
---
|
||||
|
||||
### 3.2.2 — `emit_capsule` runs `show()/hide()` on cpal callback thread
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator/dictation.rs:336-365`, `coordinator.rs:3617-3660`, `recorder.rs:458-490`
|
||||
|
||||
**Evidence (call site is the audio callback)**:
|
||||
```rust
|
||||
// coordinator/dictation.rs:336-365
|
||||
let level_handler: Arc<dyn Fn(f32) + Send + Sync> = Arc::new(move |level| {
|
||||
// ...phase guard, throttle to ~30 Hz...
|
||||
emit_capsule(
|
||||
&inner_for_level,
|
||||
CapsuleState::Recording,
|
||||
level,
|
||||
elapsed,
|
||||
None,
|
||||
None,
|
||||
);
|
||||
});
|
||||
// ...
|
||||
match Recorder::start(microphone_device_name, consumer, level_handler) { ... }
|
||||
```
|
||||
|
||||
**Evidence (callback thread is cpal's `process_callback`)**:
|
||||
```rust
|
||||
// recorder.rs:458-482 (process_callback)
|
||||
fn process_callback(...) {
|
||||
// ...resampling, RMS computation...
|
||||
level_handler(level); // synchronously invoked from cpal audio thread
|
||||
}
|
||||
```
|
||||
|
||||
**Evidence (`emit_capsule` directly touches the window)**:
|
||||
```rust
|
||||
// coordinator.rs:3637-3656
|
||||
if let Some(window) = app.get_webview_window("capsule") {
|
||||
let visible = !matches!(state, CapsuleState::Idle);
|
||||
maybe_position_capsule_bottom_center(inner, &window, payload.translation);
|
||||
if show_capsule && visible {
|
||||
if !show_capsule_window_no_activate(&app, &window) {
|
||||
let _ = window.show();
|
||||
}
|
||||
// ...
|
||||
} else {
|
||||
hide_capsule_window_if_present();
|
||||
let _ = window.hide();
|
||||
}
|
||||
}
|
||||
let _ = app.emit_to("capsule", "capsule:state", payload);
|
||||
```
|
||||
|
||||
**Notes**: `app.emit_to` is fine — Tauri's event bus is thread-safe and marshals to the JS runtime internally. The risk is `window.show() / window.hide()` and the position helper, all of which call NSWindow / HWND APIs that expect the main thread. On macOS this can stall the audio callback (NSWindow ops grab the AppKit run loop), risking `kAudioUnitErr_TooManyFramesToProcess` if the callback misses its deadline. The throttle to ~30 Hz mitigates frequency but doesn't change the per-call risk. Worth bouncing the window-touching half through `app.run_on_main_thread`.
|
||||
|
||||
**Fix sketch**: Split `emit_capsule` into `emit_capsule_event` (just `app.emit_to`, safe everywhere) and `apply_capsule_window_state` (called inside `app.run_on_main_thread` or only from already-main paths). The level-handler path only needs the event; window show/hide already happens in begin/end/cancel which run on the tokio runtime where `run_on_main_thread` is cheap.
|
||||
|
||||
---
|
||||
|
||||
### 3.2.3 — Sync inserter from async `end_session`
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator/dictation.rs:900-913`, `insertion.rs:42-56`, `:74-95`, `:136-149`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator/dictation.rs:900-913
|
||||
#[cfg(not(target_os = "windows"))]
|
||||
{
|
||||
inner.inserter.insert(&polished, restore_clipboard)
|
||||
}
|
||||
// ...
|
||||
} else if allow_non_tsf_insertion_fallback {
|
||||
inner.inserter.copy_fallback(&polished)
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// insertion.rs:42-46 (non-macOS impl)
|
||||
pub fn insert(&self, text: &str, restore_clipboard_after_paste: bool) -> InsertStatus {
|
||||
if text.is_empty() { return InsertStatus::CopiedFallback; }
|
||||
insert_with_clipboard_restore(text, restore_clipboard_after_paste)
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: `end_session` is `async`. On Linux (and on macOS too — `simulate_paste()` is FFI-light but still sync), `insert` calls `arboard::Clipboard::new()` which can block on X11/wayland for tens of ms, then `enigo` keystroke synthesis which is also sync. Blocking the tokio worker for 50–200 ms isn't catastrophic but contributes to latency under load. macOS path uses `CGEventPost` via FFI — fast, non-blocking in practice; mostly a Linux/Windows concern.
|
||||
|
||||
**Fix sketch**: Wrap the platform-specific `inserter.insert` / `inserter.copy_fallback` in `tokio::task::spawn_blocking(move || ...).await.unwrap_or(InsertStatus::Failed)`. `TextInserter` is `Sync`, so the move only needs `&self` cloned via `Arc` (already in `Inner`).
|
||||
|
||||
---
|
||||
|
||||
### 3.2.4 — `AudioMuteGuard` shells out synchronously
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/audio_mute.rs:127-152` (macOS), `:202-249` (Linux), `coordinator/resources.rs:112-131`, `coordinator/dictation.rs:369`
|
||||
|
||||
**Evidence (macOS `osascript`)**:
|
||||
```rust
|
||||
// audio_mute.rs:143-151
|
||||
let output = Command::new("osascript")
|
||||
.args(["-e", script])
|
||||
.output()
|
||||
.map_err(|e| format!("set output mute failed: {e}"))?;
|
||||
```
|
||||
|
||||
**Evidence (Linux `wpctl`/`pactl` — same shape)**:
|
||||
```rust
|
||||
// audio_mute.rs:215-223
|
||||
let output = Command::new("wpctl")
|
||||
.args(["set-mute", "@DEFAULT_AUDIO_SINK@", value])
|
||||
.output()
|
||||
.map_err(|e| format!("wpctl set-mute failed: {e}"))?;
|
||||
```
|
||||
|
||||
**Evidence (called from async `begin_session`)**:
|
||||
```rust
|
||||
// coordinator/dictation.rs:369
|
||||
acquire_recording_mute(inner, "dictation");
|
||||
match Recorder::start(microphone_device_name, consumer, level_handler) { ... }
|
||||
```
|
||||
|
||||
```rust
|
||||
// coordinator/resources.rs:117-127
|
||||
if mute.holders == 0 {
|
||||
match crate::audio_mute::AudioMuteGuard::activate() {
|
||||
Ok(guard) => { mute.guard = Some(guard); ... }
|
||||
Err(err) => { ... }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: `osascript` typically takes 100–300 ms to spawn + execute on macOS (AppleScript runtime startup). On a hot-key press → begin_session, this delays the recording start by exactly that amount, on the tokio worker thread. Windows path uses native COM (`IAudioEndpointVolume::SetMute`) which is fast and OK. Linux `wpctl`/`pactl` is similar to macOS osascript in cost.
|
||||
|
||||
**Fix sketch**: Wrap `AudioMuteGuard::activate()` in `tokio::task::spawn_blocking`. Since `acquire_recording_mute` itself is sync and called from `begin_session` (async), the cleanest path is making `acquire_recording_mute` async and `.await`-ing the spawn_blocking. Drop path (`PlatformMuteGuard::restore`) also runs `osascript` and is currently called from `Drop` in `release_recording_mute`; moving that to a detached `tokio::spawn_blocking` is sufficient (release path doesn't need to await).
|
||||
|
||||
---
|
||||
|
||||
### 3.2.5 — `thread::sleep(120ms)` in async permission probe
|
||||
**Status**: CONFIRMED (Linux/non-macOS path only)
|
||||
**Files**: `openless-all/app/src-tauri/src/permissions.rs:323-357`, `coordinator/dictation.rs:137`, `coordinator.rs:1732-1763`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// permissions.rs:343-356
|
||||
let stream = match sample_format {
|
||||
SampleFormat::F32 => build_probe!(f32),
|
||||
// ...
|
||||
}?;
|
||||
stream.play().map_err(|e| e.to_string())?;
|
||||
std::thread::sleep(Duration::from_millis(120));
|
||||
drop(stream);
|
||||
Ok(())
|
||||
```
|
||||
|
||||
**Evidence (called from async)**:
|
||||
```rust
|
||||
// coordinator/dictation.rs:137
|
||||
if let Err(message) = ensure_microphone_permission(inner) { ... }
|
||||
```
|
||||
|
||||
```rust
|
||||
// coordinator.rs:1732-1763
|
||||
fn ensure_microphone_permission(_inner: &Arc<Inner>) -> Result<(), String> {
|
||||
#[cfg(target_os = "windows")] { ... return Ok(()); } // Windows skips probe
|
||||
let status = permissions::check_microphone();
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: On macOS `check_microphone` uses `AVAudioApplication` / `AVCaptureDevice` and never reaches `probe_input_stream` (that helper is in the `cfg(not(target_os = "macos"))` module). So this 120 ms blocking sleep only fires on **Linux** (and any other non-macOS, non-Windows path) when probing mic permission. On Linux dictation, every `begin_session` blocks the tokio worker for 120 ms before the recorder even starts.
|
||||
|
||||
**Fix sketch**: `tokio::time::sleep(Duration::from_millis(120)).await` is the correct call but requires turning `probe_input_stream` async. Alternatively keep it sync and wrap the whole `check_microphone()` non-macOS path in `tokio::task::spawn_blocking`. The latter is mechanically simpler.
|
||||
|
||||
---
|
||||
|
||||
### 3.3.1 — Pressed edge routes to QA without checking dictation phase
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator/dictation.rs:11-22`, `coordinator/qa.rs:64-77`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator/dictation.rs:11-22
|
||||
pub(super) async fn handle_pressed_edge(inner: &Arc<Inner>) {
|
||||
let was_held = inner.hotkey_trigger_held.swap(true, Ordering::SeqCst);
|
||||
if !was_held {
|
||||
// 路由：QA 浮窗可见时，rightOption 边沿走 QA；否则走主听写。详见 issue #118 v2。
|
||||
let panel_visible = inner.qa_state.lock().panel_visible;
|
||||
if panel_visible {
|
||||
handle_qa_option_edge(inner).await;
|
||||
} else {
|
||||
handle_pressed(inner).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Evidence (QA path doesn't stop dictation)**:
|
||||
```rust
|
||||
// coordinator/qa.rs:64-77
|
||||
pub(super) async fn handle_qa_option_edge(inner: &Arc<Inner>) {
|
||||
let phase = inner.qa_state.lock().phase;
|
||||
log::info!("[coord] QA option edge (phase={phase:?})");
|
||||
match phase {
|
||||
QaPhase::Idle => { let _ = begin_qa_session(inner).await; }
|
||||
QaPhase::Recording => { let _ = end_qa_session(inner).await; }
|
||||
QaPhase::Processing => {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: `panel_visible` flips true via `open_qa_panel`, which is triggered by the QA hotkey (`Cmd+Shift+;` by default). If the user opens the QA panel mid-dictation (dictation `phase = Listening`, mic open, ASR session live), the next dictation-hotkey press routes into `begin_qa_session`. `begin_qa_session` will call `Recorder::start` again on the same mic device. cpal will reject the second `build_input_stream` on most platforms, but on Linux/PipeWire it sometimes succeeds and you end up with two concurrent capture streams. Even where it fails, the dictation session's recorder is still running and the user has no UX path to stop it from the QA panel.
|
||||
|
||||
**Fix sketch**: In `handle_pressed_edge`, check `inner.state.lock().phase`. If `Listening` or `Starting`, route to `handle_pressed` (which is the dictation toggle path) regardless of `panel_visible`, and either close the QA panel or refuse to open it while dictation is active. Decision belongs to product, but the *technical* race is real.
|
||||
|
||||
---
|
||||
|
||||
### 3.3.2 — Dual TranslationModifier handlers
|
||||
**Status**: PARTIAL
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator.rs:1130-1139`, `:1376-1384`, `:1402-1410`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator.rs:1130-1139 (translation_hotkey_bridge_loop, runs on its own thread)
|
||||
fn translation_hotkey_bridge_loop(inner: Arc<Inner>, rx: mpsc::Receiver<ComboHotkeyEvent>) {
|
||||
while let Ok(evt) = rx.recv() {
|
||||
if inner.shortcut_recording_active.load(Ordering::SeqCst) { continue; }
|
||||
if matches!(evt, ComboHotkeyEvent::Pressed) {
|
||||
mark_translation_modifier_seen(&inner);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// coordinator.rs:1402-1410 (hotkey_bridge_loop, separate thread)
|
||||
HotkeyEvent::TranslationModifierPressed => {
|
||||
let translation_hotkey = inner_cloned.prefs.get().translation_hotkey;
|
||||
if is_builtin_translation_shift(&translation_hotkey)
|
||||
|| crate::shortcut_binding::legacy_modifier_trigger(&translation_hotkey)
|
||||
.is_some()
|
||||
{
|
||||
mark_translation_modifier_seen(&inner_cloned);
|
||||
}
|
||||
}
|
||||
|
||||
// coordinator.rs:1376-1384
|
||||
fn mark_translation_modifier_seen(inner: &Arc<Inner>) {
|
||||
let phase = inner.state.lock().phase;
|
||||
if matches!(phase, SessionPhase::Starting | SessionPhase::Listening) {
|
||||
inner.translation_modifier_seen.store(true, Ordering::SeqCst);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: Both bridge loops run on independent `std::thread`s and both ultimately call `mark_translation_modifier_seen`, which locks `inner.state`. They never run *racing on integrity* — they both write the same `AtomicBool::store(true)`, idempotent. The audit's framing of "Both lock `inner.state` independently" is technically true but not a bug — `state` is a `Mutex`, only one acquires at a time, and both write the same flag. Worst case is one log-line of `[coord] translation modifier seen during ...` printed twice for one Shift press. Not worth a code change unless 3.3.1's fix touches the same code.
|
||||
|
||||
---
|
||||
|
||||
### 3.3.3 — Cancelled doesn't reset `hotkey_trigger_held`
|
||||
**Status**: FALSE_POSITIVE
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator.rs:1399-1401`, `hotkey.rs:530-538`, `coordinator/dictation.rs:11-22`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator.rs:1399-1401
|
||||
HotkeyEvent::Cancelled => {
|
||||
cancel_session(&inner_cloned);
|
||||
}
|
||||
```
|
||||
|
||||
**Why it doesn't actually wedge**: `HotkeyEvent::Cancelled` is emitted by the OS-side hotkey monitor only when the user presses **Esc** (`hotkey.rs:565-570` for macOS, `:867-871` for Windows), not when the dictation trigger key is released. The dictation trigger's "is currently held" state lives in `Shared::trigger_held` inside the platform monitor (`hotkey.rs:117`). That atomic gates re-emission of Pressed via the `is_active && !was_held` check (`hotkey.rs:530-538`). So even if `Inner::hotkey_trigger_held` stays `true` in the coordinator after Esc, the next Pressed edge from the OS will only fire when the user releases and re-presses the trigger key — and the OS path will set `was_held=false` again before sending Pressed. The coordinator's `hotkey_trigger_held` swap on the next Pressed will return `false` (because `handle_released_edge` has run between the previous press and this one, OR the user never released, in which case no new Pressed is queued).
|
||||
|
||||
The audit confused two layers: the OS-edge dedupe in `hotkey.rs::Shared::trigger_held` (which is the gating thing) and the coordinator's `Inner::hotkey_trigger_held` (which is just a bookkeeping latch tied to Pressed/Released edges that already came in). Cancelled doesn't change either's correctness.
|
||||
|
||||
**Notes**: There's a *cosmetic* asymmetry — after Esc, `Inner::hotkey_trigger_held=true` until the user releases the trigger. If the user keeps holding past the Esc, `Released` fires later and resets it. Defensive cleanup would be to also reset on Cancelled, but it doesn't fix any user-visible bug.
|
||||
|
||||
---
|
||||
|
||||
### 3.3.4 — `open_qa_panel` clobbers Done capsule
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator/qa.rs:79-104`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator/qa.rs:79-104
|
||||
pub(super) fn open_qa_panel(inner: &Arc<Inner>) {
|
||||
{
|
||||
let mut state = inner.qa_state.lock();
|
||||
state.panel_visible = true;
|
||||
state.phase = QaPhase::Idle;
|
||||
// ...
|
||||
}
|
||||
// 先把胶囊清干净，避免主听写上一次 Done 状态残留的 message/insertedChars
|
||||
// 在 QA Done 阶段被 capsule UI 错误复用（"已之一粘贴这个 0" 那种）。
|
||||
emit_capsule(inner, CapsuleState::Idle, 0.0, 0, None, None);
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: The comment shows the design intent is *intentional* — sweep stale Done state from a previous dictation. But it sweeps *any* in-flight capsule too. If the user opens the QA panel within the ~1.5 s `CAPSULE_AUTO_HIDE_DELAY_MS` window after dictation finishes, they lose the "已粘贴 N 字" toast. More importantly, if dictation is still in `Polishing` or `Inserting` phase (LLM hasn't returned yet), opening QA hides the polish-progress capsule mid-flight. The user sees their dictation "vanish" until insertion completes.
|
||||
|
||||
**Fix sketch**: Before calling `emit_capsule(Idle, ...)`, check `inner.state.lock().phase`. Only sweep if dictation is in `Idle`. If dictation is mid-flight, leave the capsule visible — QA panel doesn't need the capsule cleared to function. Pairs cleanly with the 3.3.1 fix (same source files).
|
||||
|
||||
---
|
||||
|
||||
### 3.3.5 — `focus_target` leaks on Processing-phase cancel
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator_state.rs:155-173`, `:347-374` (test that already proves the gap)
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator_state.rs:168-173
|
||||
pub(crate) fn finish_cancel_session_state(state: &mut SessionState, decision: CancelDecision) {
|
||||
if decision.phase != SessionPhase::Processing {
|
||||
state.phase = SessionPhase::Idle;
|
||||
state.focus_target = None;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Evidence (existing test acknowledges the gap)**:
|
||||
```rust
|
||||
// coordinator_state.rs:370-372
|
||||
if matches!(initial, SessionPhase::Starting | SessionPhase::Listening) {
|
||||
assert!(state.focus_target.is_none(), "initial={initial:?}");
|
||||
}
|
||||
// Note: no assertion that Processing-phase cancel clears focus_target.
|
||||
```
|
||||
|
||||
**Notes**: When cancel hits `Processing`, `finish_cancel_session_state` deliberately keeps `phase=Processing` (the comment says "交给 end_session 自己收尾"), but it also keeps `focus_target` populated. `end_session` does eventually reset state via the `proceed_to_insert=false` branch (`coordinator/dictation.rs:862-878`) which sets `phase=Idle` but doesn't touch `focus_target`. Net result: stale `focus_target` (a `Vec<PreparedWindowsImeSessionSlot>` index, basically a plain `usize`) lives on into the next `begin_session_state`, which overwrites it (`coordinator_state.rs:80`). So the leak is bounded — next session clobbers it. Real risk is only between cancel and next begin, where `restore_focus_target_if_possible(focus_target)` could pick up a stale value if anyone reads it. Code review didn't surface a reader on that interval, so this is a low-impact correctness gap, not a user-facing bug.
|
||||
|
||||
**Fix sketch**: In `finish_cancel_session_state`, set `state.focus_target = None` unconditionally (before the phase check). The Processing branch's existing semantic — "let `end_session` collapse to Idle" — doesn't depend on `focus_target` staying set.
|
||||
|
||||
---
|
||||
|
||||
### 3.3.6 — Double-restore of prepared Windows IME session
|
||||
**Status**: FALSE_POSITIVE
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator.rs:1594-1633`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// coordinator.rs:1594-1602
|
||||
fn take_matching_prepared_windows_ime_session(
|
||||
slots: &mut Vec<PreparedWindowsImeSessionSlot>,
|
||||
session_id: SessionId,
|
||||
) -> Option<PreparedWindowsImeSession> {
|
||||
let index = slots
|
||||
.iter()
|
||||
.position(|slot| slot.session_id == session_id)?;
|
||||
Some(slots.remove(index).prepared)
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// coordinator.rs:1620-1633
|
||||
fn restore_prepared_windows_ime_session(inner: &Arc<Inner>, session_id: SessionId) {
|
||||
let state = inner.state.lock();
|
||||
let prepared = {
|
||||
let mut slot = inner.prepared_windows_ime_session.lock();
|
||||
take_current_prepared_windows_ime_session_for_restore(
|
||||
&mut slot, session_id, state.session_id,
|
||||
)
|
||||
};
|
||||
if let Some(prepared) = prepared { inner.windows_ime.restore_session(prepared); }
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: First call to `restore_prepared_windows_ime_session` for a given `session_id` does `slots.remove(index)` regardless of the freshness check on `current_session_id`. The slot is gone after that. Second call's `slots.iter().position(...)` returns `None`, the `?` short-circuits, the function silently no-ops. So even if `cancel_session → end_session` (or vice versa) both invoke `restore_prepared_windows_ime_session` with the same `session_id`, the IME state is restored at most once. The audit's worry is unfounded.
|
||||
|
||||
---
|
||||
|
||||
### 3.4.1 — Inner has many lock fields
|
||||
**Status**: ADVISORY_ONLY
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator.rs:91-141`
|
||||
|
||||
**Inventory**: 16 `Mutex<...>` + 4 `AtomicBool` (excluding `Arc<...>` indirection counts) plus an `Arc<Mutex<Vec<...>>>` for the windows IME slot vector. Specifically:
|
||||
|
||||
| Field | Type |
|
||||
|---|---|
|
||||
| `app` | `Mutex<Option<AppHandle>>` |
|
||||
| `state` | `Mutex<SessionState>` |
|
||||
| `asr` | `Mutex<Option<SessionResource<ActiveAsr>>>` |
|
||||
| `recorder` | `Mutex<Option<SessionResource<Recorder>>>` |
|
||||
| `recording_mute` | `Mutex<SharedRecordingMuteState>` |
|
||||
| `hotkey` | `Mutex<Option<HotkeyMonitor>>` |
|
||||
| `hotkey_status` | `Mutex<HotkeyStatus>` |
|
||||
| `hotkey_trigger_held` | `AtomicBool` |
|
||||
| `shortcut_recording_active` | `AtomicBool` |
|
||||
| `combo_hotkey` | `Mutex<Option<ComboHotkeyMonitor>>` |
|
||||
| `translation_hotkey` | `Mutex<Option<ComboHotkeyMonitor>>` |
|
||||
| `switch_style_hotkey` | `Mutex<Option<ComboHotkeyMonitor>>` |
|
||||
| `open_app_hotkey` | `Mutex<Option<ComboHotkeyMonitor>>` |
|
||||
| `translation_modifier_seen` | `AtomicBool` |
|
||||
| `qa_hotkey` | `Mutex<Option<QaHotkeyMonitor>>` |
|
||||
| `qa_state` | `Mutex<QaSessionState>` |
|
||||
| `capsule_layout` | `Mutex<Option<CapsuleLayoutState>>` |
|
||||
| `qa_asr` | `Mutex<Option<Arc<VolcengineStreamingASR>>>` |
|
||||
| `qa_recorder` | `Mutex<Option<Recorder>>` |
|
||||
| `qa_stream_cancelled` | `Arc<AtomicBool>` (one of two AtomicBools-in-Arc) |
|
||||
| `prepared_windows_ime_session` (windows-only) | `Arc<Mutex<Vec<PreparedWindowsImeSessionSlot>>>` |
|
||||
|
||||
No deadlock pattern was found in the read paths — most call sites take one lock at a time. `mark_translation_modifier_seen` and `cancel_session` both touch `inner.state`, but in disjoint critical sections. Documenting only.
|
||||
|
||||
---
|
||||
|
||||
### 3.4.2 — Heavy `Ordering::SeqCst` use
|
||||
**Status**: ADVISORY_ONLY
|
||||
**Files**: `coordinator.rs`, `coordinator/*.rs`, `hotkey.rs`
|
||||
|
||||
**Evidence**: `grep -rn "Ordering::SeqCst" coordinator.rs coordinator/ hotkey.rs | wc -l` → 66. Total `Ordering::*` uses in those files: 67 (one `Relaxed` in `recorder.rs::process_callback`, 66 SeqCst).
|
||||
|
||||
**Notes**: Most uses are simple set/load on independent `AtomicBool`s where `Ordering::Relaxed` would suffice. A few that gate cross-thread visibility (`hotkey_trigger_held` swap in `handle_pressed_edge` synchronizing with the audio thread reading session state) might justify Acquire/Release. `SeqCst` is correct everywhere — just over-strong. Not a bug.
|
||||
|
||||
---
|
||||
|
||||
### 3.4.3 — Many `unsafe` blocks, audit SAFETY comments
|
||||
**Status**: ADVISORY_ONLY
|
||||
**Files**: cross-tree (predominantly `hotkey.rs`, `insertion.rs`, `windows_ime_*.rs`, `permissions.rs`)
|
||||
|
||||
**Evidence**: `grep -rn "unsafe " src/ --include="*.rs" | grep -E "unsafe \{|unsafe fn|unsafe impl|unsafe extern"` → 102 sites.
|
||||
|
||||
**Notes**: Almost all are platform FFI (CoreFoundation/CoreGraphics on macOS, win32 on Windows, msg_send! on macOS objc2). Sample inspected (`insertion.rs::send_text` near line 340 and `post_cmd_v` near line 420) — function-level invariants are documented at module level, but inline `// SAFETY:` comments are sparse. Same for `hotkey.rs::run_listen_loop` which leaks `Box::into_raw` for FFI context — a `// SAFETY: ctx is dropped only inside the listener after CFRunLoopRun returns; reentrancy guarded by ...` comment would help. Documentation-grade improvement, no soundness bug detected.
|
||||
|
||||
---
|
||||
|
||||
### 3.4.4 — Global hotkey dispatcher loop has no exit
|
||||
**Status**: CONFIRMED
|
||||
**Files**: `openless-all/app/src-tauri/src/global_hotkey_runtime.rs:94-107`
|
||||
|
||||
**Evidence**:
|
||||
```rust
|
||||
// global_hotkey_runtime.rs:94-107
|
||||
fn start_dispatcher(runtime: Arc<GlobalHotkeyRuntime>) {
|
||||
std::thread::Builder::new()
|
||||
.name("openless-global-hotkey-dispatch".into())
|
||||
.spawn(move || {
|
||||
let receiver = GlobalHotKeyEvent::receiver();
|
||||
loop {
|
||||
match receiver.recv_timeout(Duration::from_millis(250)) {
|
||||
Ok(event) => runtime.dispatch(event),
|
||||
Err(_) => continue,
|
||||
}
|
||||
}
|
||||
})
|
||||
.expect("spawn global hotkey dispatcher");
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**: `GlobalHotKeyEvent::receiver()` is process-global from the upstream `global-hotkey` crate. The 250 ms timeout means the loop wakes regularly but never checks an exit flag. On app shutdown the thread leaks; harmless for a single-instance app but trips `tokio::test` and any future `Drop`-based teardown (e.g. integration tests that spin coordinator up/down).
|
||||
|
||||
**Fix sketch**: Pair with `Inner` shutdown flag added in 3.1.2 fix, or use a `parking_lot::RwLock<Option<...>>` "dispatcher alive" gate inside `GlobalHotkeyRuntime` that the loop reads each iteration. Same shape as the Windows hotkey thread's `WM_QUIT` plumbing.
|
||||
|
||||
---
|
||||
|
||||
### 20 (NEW) — `PreferencesStore::new().expect(...)` panics on bad prefs
|
||||
**Status**: FALSE_POSITIVE
|
||||
**Files**: `openless-all/app/src-tauri/src/coordinator.rs:169`, `:210`, `persistence.rs:790-811`, `:146-156`, `types.rs:368-422`
|
||||
|
||||
**Evidence (deserialization fallback already exists)**:
|
||||
```rust
|
||||
// persistence.rs:790-811
|
||||
impl PreferencesStore {
|
||||
pub fn new() -> Result<Self> {
|
||||
let dir = data_dir()?;
|
||||
ensure_dir(&dir)?;
|
||||
let path = dir.join(PREFERENCES_FILE);
|
||||
let prefs = if path.exists() {
|
||||
read_or_default::<UserPreferences>(&path).unwrap_or_else(|e| {
|
||||
log::warn!(
|
||||
"[prefs] load {} failed, using defaults: {}",
|
||||
path.display(),
|
||||
e
|
||||
);
|
||||
UserPreferences::default()
|
||||
})
|
||||
} else {
|
||||
UserPreferences::default()
|
||||
};
|
||||
Ok(Self { path, state: Mutex::new(prefs), })
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
// persistence.rs:146-156 (read_or_default)
|
||||
fn read_or_default<T: for<'de> Deserialize<'de> + Default>(path: &Path) -> Result<T> {
|
||||
if !path.exists() { return Ok(T::default()); }
|
||||
let bytes = fs::read(path).with_context(|| format!("read failed: {}", path.display()))?;
|
||||
if bytes.is_empty() { return Ok(T::default()); }
|
||||
serde_json::from_slice::<T>(&bytes)
|
||||
.with_context(|| format!("decode failed: {}", path.display()))
|
||||
}
|
||||
```
|
||||
|
||||
**Why the audit was wrong**: The custom `Deserialize for UserPreferences` (types.rs:368-422) does call `default_dictation_hotkey_from_legacy(...).map_err(serde::de::Error::custom)?` which can return a serde error for `trigger == Custom` without `customComboHotkey`. That error bubbles through `serde_json::from_slice::<UserPreferences>` to `read_or_default`, which propagates it as `Result::Err`. But `PreferencesStore::new` then catches it at `.unwrap_or_else(|e| { log::warn!(...); UserPreferences::default() })`. So the `expect("preferences store init")` at coordinator.rs:169 only fires if `data_dir()?` or `ensure_dir(&dir)?` fails — i.e. the OS-level Application Support directory cannot be created/accessed, which is a legitimate fail-fast condition (no preferences-file storage, no point continuing).
|
||||
|
||||
The audit conflated "deserialization fails" with "PreferencesStore::new returns Err". In the current code those are different.
|
||||
|
||||
**Notes**: No fix needed. The "bad prefs file" case is already handled silently (log + default). If you want belt-and-braces against panic on the truly-impossible filesystem failure, you can add a final `.unwrap_or_else` that logs and returns a fully-default in-memory store, but that's defensive coding for a case where the user's machine is so broken that Application Support is unwritable.
|
||||
|
||||
---
|
||||
|
||||
### 2.3.3 — Event-name alignment between backend emit and frontend listen
|
||||
**Status**: CONFIRMED (no action)
|
||||
**Files**: `coordinator.rs:3659`, `coordinator/qa.rs:94-101`, `coordinator/dictation.rs:929-931`, `src/components/Capsule.tsx:293`, `src/pages/QaPanel.tsx:55,116`, `src/pages/Vocab.tsx:51`
|
||||
|
||||
**Notes**: Backend emits `capsule:state`, `qa:state`, `qa:level`, `vocab:updated`. Frontend listens to all four under matching names (Capsule, QaPanel, Vocab respectively). No mismatch. As stated in the audit prompt — already verified, retained here for completeness.
|
||||
|
||||
## Files referenced
|
||||
|
||||
- `openless-all/app/src-tauri/src/types.rs` (lines 57, 200-525, especially 216-219, 277-325, 368-422)
|
||||
- `openless-all/app/src-tauri/src/hotkey.rs` (lines 1-250, 280-572, 698-870, 1126-1190)
|
||||
- `openless-all/app/src-tauri/src/coordinator.rs` (lines 91-141, 156-313, 640-833, 837-940, 1130-1139, 1376-1419, 1582-1636, 3617-3700)
|
||||
- `openless-all/app/src-tauri/src/coordinator/dictation.rs` (lines 1-160, 320-410, 810-1050)
|
||||
- `openless-all/app/src-tauri/src/coordinator/qa.rs` (full file, 1-124)
|
||||
- `openless-all/app/src-tauri/src/coordinator/resources.rs` (lines 1-160)
|
||||
- `openless-all/app/src-tauri/src/coordinator_state.rs` (full file, 1-485)
|
||||
- `openless-all/app/src-tauri/src/global_hotkey_runtime.rs` (full file, 1-107)
|
||||
- `openless-all/app/src-tauri/src/audio_mute.rs` (lines 1-263)
|
||||
- `openless-all/app/src-tauri/src/permissions.rs` (lines 200-360)
|
||||
- `openless-all/app/src-tauri/src/insertion.rs` (lines 1-150, 300-450)
|
||||
- `openless-all/app/src-tauri/src/persistence.rs` (lines 146-156, 785-811)
|
||||
- `openless-all/app/src-tauri/src/recorder.rs` (lines 28-490)
|
||||
- `openless-all/app/src-tauri/src/commands.rs` (lines 975-1000)
|
||||
- `openless-all/app/src/lib/types.ts` (lines 118-275)
|
||||
- `openless-all/app/src/lib/ipc.ts` (lines 40-176)
|
||||
- `openless-all/app/src/components/Capsule.tsx` (line 293)
|
||||
- `openless-all/app/src/pages/QaPanel.tsx` (lines 55, 116-120)
|
||||
- `openless-all/app/src/pages/Vocab.tsx` (line 51)
|
||||
@@ -1,63 +0,0 @@
|
||||
# Auto Update Download Acceleration
|
||||
|
||||
## Problem
|
||||
|
||||
OpenLess used a single Tauri updater endpoint on GitHub Releases:
|
||||
|
||||
```text
|
||||
https://github.com/appergb/openless/releases/latest/download/latest-{{target}}-{{arch}}.json
|
||||
```
|
||||
|
||||
The manifest also pointed installer downloads back to GitHub Releases. On networks where GitHub release assets are slow, a small updater package can take minutes to download.
|
||||
|
||||
Desktop apps do not reliably inherit a user's shell proxy environment. Instead of making updater correctness depend on whether a proxy is visible to the app process, the updater should use a GitHub release acceleration URL directly.
|
||||
|
||||
## Runtime Behavior
|
||||
|
||||
The app does not manually probe local proxy ports. It lets the OS/process network stack do whatever it normally does, while the updater endpoint itself points at `fastgit.cc` first. This keeps the rule simple: proxy or no proxy, updater traffic should prefer the fastgit transport.
|
||||
|
||||
## Fastgit Acceleration
|
||||
|
||||
Release builds now publish two updater manifests per target:
|
||||
|
||||
```text
|
||||
latest-<target>-<arch>.json
|
||||
latest-<target>-<arch>-mirror.json
|
||||
```
|
||||
|
||||
The client checks the mirror manifest first, then GitHub. The mirror manifest points its installer URL at:
|
||||
|
||||
```text
|
||||
https://fastgit.cc/https://github.com/<repo>/releases/latest/download/<asset>
|
||||
```
|
||||
|
||||
The updater signature still protects the downloaded package. The mirror only changes transport; it cannot replace the signed payload without verification failing.
|
||||
|
||||
## Maintainer Notes
|
||||
|
||||
Set `OPENLESS_UPDATE_MIRROR_BASE_URL` in CI to change the mirror host. Keep it formatted as a prefix for GitHub URLs, for example:
|
||||
|
||||
```text
|
||||
https://fastgit.cc/https://github.com
|
||||
```
|
||||
|
||||
If a mirror becomes unreliable, replace that environment value and the mirror endpoint in `openless-all/app/src-tauri/tauri.conf.json`.
|
||||
|
||||
## Evidence
|
||||
|
||||
Measured from Windows on 2026-05-01. Direct GitHub release downloads were tested with local proxy disabled to reproduce the slow path. `fastgit.cc` was tested both through the normal local proxy environment and with local proxy disabled; results vary by route, so do not treat one machine's no-proxy number as a CDN SLA.
|
||||
|
||||
```text
|
||||
Direct GitHub installer asset, 4.78 MB, proxy disabled:
|
||||
run 1: timed out after 90.75s, 1.73 MB received
|
||||
run 2: timed out after 90.06s, 2.44 MB received
|
||||
|
||||
fastgit.cc installer asset, 4.78 MB, normal local proxy environment:
|
||||
with protocol prefix: 3.12s / 3.63s / 3.39s
|
||||
without protocol prefix: 2.92s / 2.45s / 2.87s
|
||||
|
||||
fastgit.cc target-user signal:
|
||||
manual browser/download usage reported completing in under 1s without enabling a proxy.
|
||||
```
|
||||
|
||||
This is enough to justify a `fastgit.cc` mirror path, but not enough to treat a public mirror as permanently trusted infrastructure. `fastgit.cc` explicitly documents support for GitHub release/archive acceleration and accepts GitHub links with or without the protocol prefix. Keep the mirror configurable and re-test before each release if download performance is a release blocker.
|
||||
@@ -1,87 +0,0 @@
|
||||
## 现象 / Symptom
|
||||
|
||||
这不是单一的 click dead zone bug，而是一组已经在 Windows 实机上被观察到、且共享同一根因的 helper-window lifecycle 症状：
|
||||
|
||||
- click dead zone：原 Capsule 区域附近会挡住底层输入框或按钮
|
||||
- screenshot selectable：截图工具仍然可以选中这块透明区域
|
||||
- drag stutter：在该区域拖拽时出现明显卡顿或 compositor 异常
|
||||
- lingering transparent overlay：录音结束后，Capsule 仍可能以透明顶层窗 linger
|
||||
|
||||
当前证据说明：这些现象不应拆成多个互不相关的问题，而应视为同一个生命周期语义偏差。
|
||||
|
||||
### 证据 / Evidence
|
||||
|
||||
运行与代码证据：
|
||||
|
||||
- `openless-all/app/src-tauri/tauri.conf.json:33-47`
|
||||
- `capsule` 被配置为 `transparent + alwaysOnTop + focus:false + visible:false`
|
||||
- `openless-all/app/src-tauri/src/lib.rs:594-623`
|
||||
- Windows 端 `capsule` runtime host bounds 为 `220x84/118`，明显大于可见 pill `196x52`
|
||||
- `openless-all/app/src-tauri/src/coordinator.rs:2398-2432`
|
||||
- Windows 端显示路径走 `ShowWindow(SW_SHOWNOACTIVATE)` + `SetWindowPos(...SWP_NOACTIVATE...)`
|
||||
- `openless-all/app/src-tauri/src/coordinator.rs:2455-2479`
|
||||
- 结束阶段依赖 `window.hide()` 作为生命周期结束语义
|
||||
- `openless-all/app/src/components/Capsule.tsx:278-281`
|
||||
- 前端 `idle` 只把可见内容缩成 `0x0`，真正结束仍取决于后端窗口是否已完全退出参与
|
||||
- [2026-05-02-platform-lifecycle-audit.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/2026-05-02-platform-lifecycle-audit.md)
|
||||
- 审计已把该问题收敛为 Windows helper-window lifecycle contract 偏差
|
||||
|
||||
现场证据：
|
||||
|
||||
- 用户已在 Windows 上观察到 dead zone / screenshot selectable / drag stutter / lingering overlay
|
||||
- 这些表现与透明顶层 helper window 未真正退出 OS 参与的形态一致
|
||||
|
||||
### 5 Whys / 根因分析
|
||||
|
||||
1. 为什么会出现点击死区、截图可选中、拖拽卡顿？
|
||||
- 因为录音结束后，Windows 上的 Capsule host window 仍可能继续存在并参与桌面层级。
|
||||
2. 为什么录音结束后窗口还会继续参与？
|
||||
- 因为当前实现把“生命周期结束”主要建模成 `hide()`，而不是“保证 helper window 不再参与 hit-test / capture / z-order / compositor”。
|
||||
3. 为什么这个问题在 Windows 上更容易暴露？
|
||||
- 因为 Windows 的 Capsule host geometry 更大、show path 更特殊，并且是透明顶层窗；一旦 hide 语义失守，残留面积极大且更容易干扰系统行为。
|
||||
4. 为什么这和 macOS 的原始设计意图不一致？
|
||||
- macOS 的原始意图是：Capsule 只在 active stage 短暂出现，结束后自然收起，不再作为前台交互对象继续存在；Windows 当前更像“视觉结束了，但 OS 对象还挂着”。
|
||||
5. 为什么之前没有被门禁拦住？
|
||||
- 现有检查更多关注“窗口显示/隐藏”和几何配置，没有直接验证 inactive state 下它是否真的退出系统参与。
|
||||
|
||||
### 平台边界 / Platform Scope
|
||||
|
||||
- 直接症状范围：当前已确认是 Windows 实机问题。
|
||||
- 问题层面：backend helper-window lifecycle contract + Windows native window participation。
|
||||
- 全平台风险判断：根因模式不是 Windows 独有，任何透明 helper window 只要“视觉隐藏 != 生命周期结束”都可能中招；Capsule 目前是 Windows 上最先爆出来的样板案例。
|
||||
|
||||
### 认领 / Ownership
|
||||
|
||||
- owner intent：`@Cooper-X-Oak`
|
||||
- 当前对应 draft PR：`#140`
|
||||
|
||||
### 当前状态 / Current status
|
||||
|
||||
- lifecycle 主线修复已完成第一波
|
||||
- 人工桌面回归结果：
|
||||
- click dead zone：通过
|
||||
- screenshot selectable：通过
|
||||
- drag stutter：通过
|
||||
- 当前建议：从“问题收敛中”推进到“regression review 中”
|
||||
|
||||
## 影响 / Impact
|
||||
|
||||
- 直接影响 Windows 端核心输入体验与系统交互可信度
|
||||
- 会误伤底层 app 的点击、截图、拖拽，用户容易误判成其他应用故障
|
||||
- 因为残留对象透明且顶层，这类问题隐蔽、难复现、难定位
|
||||
- 如果不从生命周期语义修，后续即使修掉某一个 dead zone，仍可能继续遗留 screenshot / z-order / compositor 问题
|
||||
|
||||
## 建议接受标准 / Proposed Acceptance Criteria
|
||||
|
||||
- [ ] Windows 上 Capsule 的“结束”语义与 macOS 对齐：inactive 后不再继续参与系统交互
|
||||
- [ ] inactive Capsule 不再造成 click dead zone
|
||||
- [ ] inactive Capsule 不再被截图工具选中
|
||||
- [ ] inactive Capsule 不再引入 drag/compositor stutter
|
||||
- [ ] 为 Windows 增加一条直接验证 inactive Capsule non-participating 的 smoke / regression check
|
||||
- [ ] 修复方案明确区分 visual state 与 host-window lifecycle state，而不是继续叠加局部 workaround
|
||||
|
||||
## TODO / 不确定项
|
||||
|
||||
- 是否需要把 `capsule hidden => no hit-test / no capture / no topmost participation` 抽成统一 helper-window contract，复用于 QA panel
|
||||
- 当前 `PR #140` 建议保持 draft tracking 角色，待范围与根因完全收敛后再转 ready
|
||||
建议 issue 标题：`[ui][windows] Capsule 隐藏后仍参与系统交互`
|
||||
@@ -1,74 +0,0 @@
|
||||
## 现象 / Symptom
|
||||
|
||||
Windows 冷启动路径里，`visible` 与 `ready` 目前是脱钩的：主窗口可以先被用户看见，但 global hotkey / runtime lifecycle 还在后台异步安装。
|
||||
|
||||
这不是单纯的 UI 小闪烁，而是 startup lifecycle ownership 不统一：
|
||||
|
||||
- `main` 在配置层默认 `visible:false`
|
||||
- backend 负责 `show_main_window()` / tray reopen / single-instance focus
|
||||
- frontend `App.tsx` 又在 mount 后主动 `currentWindow.show()`
|
||||
- Windows 路径下 `gate` 初始值直接是 `ready`
|
||||
|
||||
### 证据 / Evidence
|
||||
|
||||
- `openless-all/app/src-tauri/tauri.conf.json:17-30`
|
||||
- `main.visible = false`
|
||||
- `openless-all/app/src-tauri/src/lib.rs:314-356`
|
||||
- backend 明确拥有 `show_main_window()` / `hide_main_window()` 生命周期入口
|
||||
- `openless-all/app/src-tauri/src/lib.rs:158-163`
|
||||
- hotkey listener 与 QA hotkey listener 在 setup 后异步启动
|
||||
- `openless-all/app/src/App.tsx:23-52`
|
||||
- Windows 路径初始化时直接 `gate='ready'`
|
||||
- mount 后又在 `requestAnimationFrame` 里调用 `currentWindow.show()`
|
||||
- [2026-05-02-platform-lifecycle-audit.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/2026-05-02-platform-lifecycle-audit.md)
|
||||
- 审计已将该问题归类为 startup lifecycle ownership 偏差
|
||||
|
||||
### 5 Whys / 根因分析
|
||||
|
||||
1. 为什么用户会看到一个看似 ready 的窗口，但热键/运行态未必已经 ready？
|
||||
- 因为窗口可见时机和 runtime readiness 时机不是一个 source of truth。
|
||||
2. 为什么这两个时机分离了？
|
||||
- 因为 backend 和 frontend 同时持有 `main` visibility 的一部分控制权。
|
||||
3. 为什么 Windows 上更明显？
|
||||
- 因为 Windows 启动路径跳过了 macOS 那种明确的 permission gate / startup shell，正式 UI 更早暴露。
|
||||
4. 为什么这偏离了 macOS 的原始设计意图？
|
||||
- 原始意图是“用户看见主窗口时，它已经进入可用或可解释的阶段”；Windows 当前更像“窗口先到，能力后到”。
|
||||
5. 为什么之前没被系统性识别？
|
||||
- 现有 smoke 主要验证“进程活着 + 稍后日志出现 hotkey installed”，没有验证“first visible frame == operationally ready”。
|
||||
|
||||
### 平台边界 / Platform Scope
|
||||
|
||||
- 直接症状范围：当前主要在 Windows 冷启动观察到。
|
||||
- 问题层面：startup lifecycle ownership、window visibility contract、runtime readiness contract。
|
||||
- 全平台风险判断：这是全平台架构层风险，但 Windows 因跳过 startup gate、前端主动 show，最先表现为真实用户问题。
|
||||
|
||||
### 认领 / Ownership
|
||||
|
||||
- owner intent：`@Cooper-X-Oak`
|
||||
- 对应 draft PR：待创建
|
||||
|
||||
### 当前状态 / Current status
|
||||
|
||||
- startup lifecycle 主线修复已生效
|
||||
- 最新测试入口改为 frontend-managed first show，不再用 backend immediate show 污染结果
|
||||
- 人工冷启动体验反馈：几乎没有问题，人眼很难分辨
|
||||
- 当前建议：保留 draft，继续观察 first-paint / startup latency，而不是继续扩大主修补丁
|
||||
|
||||
## 影响 / Impact
|
||||
|
||||
- 用户会把尚未 ready 的窗口误判为已经 ready
|
||||
- 会放大“热键没反应 / 运行态未安装”的首屏困惑
|
||||
- 让后续任何 Windows 启动问题更难分辨是 UI 问题、hotkey 问题，还是 lifecycle contract 问题
|
||||
|
||||
## 建议接受标准 / Proposed Acceptance Criteria
|
||||
|
||||
- [ ] `main` 窗口的首次可见时机只由一个 owner 控制
|
||||
- [ ] first visible frame 与 runtime readiness 的关系被明确定义并可验证
|
||||
- [ ] Windows 冷启动下，用户首次看到主窗口时，至少处于明确的 `startup` 或 `ready` 状态，而不是 ambiguous ready
|
||||
- [ ] 增加一条启动 smoke：覆盖 `visible`、`hotkey installed`、`first usable state` 的先后顺序
|
||||
|
||||
## TODO / 不确定项
|
||||
|
||||
- 是否应把 `main` visibility 完全收回 backend，frontend 只负责内容 gate
|
||||
- 是否要把现有 `issue #143` 的 first-paint 问题作为本 issue 的下游视觉子问题处理，还是继续分票并行跟踪
|
||||
建议 issue 标题：`[tauri][windows] 冷启动时 visible 与 ready 脱钩`
|
||||
@@ -1,77 +0,0 @@
|
||||
## Symptom
|
||||
|
||||
Windows dictation / QA lifecycle previously had two event sources driving the same state machine:
|
||||
|
||||
- OS-level low-level keyboard hook
|
||||
- renderer / window-local hotkey forwarding
|
||||
|
||||
That design is risky even when the product "seems to work":
|
||||
|
||||
- press/release edges can come from different sources
|
||||
- focus switches can strand half an edge
|
||||
- hold mode and toggle mode can drift differently on Windows only
|
||||
|
||||
## Evidence
|
||||
|
||||
- [openless-all/app/src/App.tsx](/D:/Users/cooper/Practice-Project/202604/openless/openless-all/app/src/App.tsx)
|
||||
- Windows window-local forwarding existed in the frontend path
|
||||
- [openless-all/app/src-tauri/src/coordinator.rs](/D:/Users/cooper/Practice-Project/202604/openless/openless-all/app/src-tauri/src/coordinator.rs)
|
||||
- backend also accepted `handle_window_hotkey_event`
|
||||
- [openless-all/app/src-tauri/src/hotkey.rs](/D:/Users/cooper/Practice-Project/202604/openless/openless-all/app/src-tauri/src/hotkey.rs)
|
||||
- Windows already owns a `WH_KEYBOARD_LL` low-level hook
|
||||
|
||||
Current convergence from this repair track:
|
||||
|
||||
- QA hotkey / follow-up flow works
|
||||
- Windows owner source should be the backend low-level hook
|
||||
- window-local forwarding should not keep driving the same lifecycle
|
||||
|
||||
## Root Cause Convergence
|
||||
|
||||
This was not just "an extra fallback path".
|
||||
|
||||
It was an ownership problem:
|
||||
|
||||
```text
|
||||
Two independent input sources were allowed to influence one dictation / QA
|
||||
lifecycle state machine without an explicit precedence contract.
|
||||
```
|
||||
|
||||
## 5 Whys
|
||||
|
||||
1. Why is this a lifecycle issue and not just a convenience fallback?
|
||||
- Because the second path was able to trigger real start/stop edges.
|
||||
2. Why is that dangerous?
|
||||
- Because mixed-source ordering can desynchronize phase transitions.
|
||||
3. Why is it primarily a Windows issue?
|
||||
- Because Windows carried both the low-level hook and the renderer-forward path.
|
||||
4. Why does this diverge from original intent?
|
||||
- Because one user gesture should map to one stable lifecycle transition.
|
||||
5. Why is this near closure now?
|
||||
- Because current repair work has already converged on a single owner source: backend low-level hook.
|
||||
|
||||
## Platform Scope
|
||||
|
||||
- Direct symptom scope: Windows implementation risk
|
||||
- Problem layer: input source ownership, lifecycle precedence, focus-sensitive edge delivery
|
||||
|
||||
## Related Issues
|
||||
|
||||
- #154 main issue anchor
|
||||
- #147 settings-to-runtime listener refresh contract
|
||||
- #158 governance issue for helper-window / native-window contract family
|
||||
|
||||
## Impact
|
||||
|
||||
- Without a single owner source, Windows-only lifecycle drift remains hard to reproduce and harder to trust
|
||||
- With ownership clarified, regression review can focus on evidence instead of guessing which path fired
|
||||
|
||||
## Proposed Acceptance Criteria
|
||||
|
||||
- [ ] Windows lifecycle owner source is explicitly documented as backend low-level hook
|
||||
- [ ] window-local forwarding no longer drives the main lifecycle unless a future explicit fallback contract is introduced
|
||||
- [ ] regression review confirms no new mixed-source ordering evidence
|
||||
|
||||
## Status Note
|
||||
|
||||
Current recommendation: treat this issue as near-closure and use it as a regression-review anchor rather than a new large refactor anchor.
|
||||
@@ -1,146 +0,0 @@
|
||||
## 现象 / Symptom
|
||||
|
||||
Windows terminal 文本输入场景历史上出现过两类现象：
|
||||
|
||||
- 用户反馈 terminal 里不会自动上屏，需要再手动 `Ctrl+V`
|
||||
- 本地测试曾观察到一次“目标最终拿到的是旧剪贴板，而不是本次听写结果”的现象
|
||||
|
||||
这两类现象都指向同一条 Windows insertion 链路：OpenLess 通过 clipboard + synthetic `Ctrl+V` 完成插入，而 terminal 是最敏感的目标类型之一。
|
||||
|
||||
### 证据 / Evidence
|
||||
|
||||
- `openless-all/app/src-tauri/src/insertion.rs`
|
||||
- Windows 路径的成功语义是 `PasteSent`
|
||||
- `PasteSent` 只代表已经发出 synthetic `Ctrl+V`
|
||||
- 它不代表目标已经完成 clipboard 消费
|
||||
- `docs/2026-05-02-windows-terminal-clipboard-restore-investigation.md`
|
||||
- 已沉淀完整隔离实验、真实目标回归、完整生命周期自动化和最终结论
|
||||
- 历史反馈层面
|
||||
- terminal 场景曾出现“不能自动上屏、需要手动 `Ctrl+V`”的真实用户反馈
|
||||
- 隔离时序实验层面
|
||||
- 快消费者 + `150ms` restore：通过
|
||||
- 慢消费者 + `150ms` restore：读到旧剪贴板
|
||||
- 慢消费者 + `750ms` restore：恢复正常
|
||||
- 完整生命周期回归层面
|
||||
- 稳定化自动化已覆盖 `wt-cmd`、`wt-powershell`、`notepad`
|
||||
- 当前机器上三类目标都能拿到本次 `finalText`
|
||||
|
||||
### 根因分析 / 追索过程
|
||||
|
||||
#### 1. 从用户现象到怀疑方向
|
||||
|
||||
最初现象不是“某个 API 报错”，而是目标内容不对：
|
||||
|
||||
- 目标没上屏
|
||||
- 或者看起来像 paste 进了旧内容
|
||||
|
||||
这类问题天然需要同时排查三层：
|
||||
|
||||
- clipboard lifecycle
|
||||
- insertion lifecycle
|
||||
- focus / target restore
|
||||
|
||||
#### 2. 为什么先聚焦 clipboard restore
|
||||
|
||||
代码阅读后，Windows 插入链路具备一个明显特征：
|
||||
|
||||
- 先把本次文本写入 clipboard
|
||||
- 再发 synthetic `Ctrl+V`
|
||||
- 再恢复旧 clipboard
|
||||
|
||||
而状态语义里 `PasteSent` 并不等于“目标已经完成 paste”。
|
||||
因此最早的根因假设是：
|
||||
|
||||
- 如果目标消费 clipboard 较慢，restore 可能会抢在目标 paste 之前发生
|
||||
|
||||
#### 3. 如何证明这个假设不是猜测
|
||||
|
||||
我们补了独立的时序实验，把 OpenLess 业务链路先拆开，只验证：
|
||||
|
||||
- clipboard 写入
|
||||
- synthetic paste
|
||||
- restore 时机
|
||||
- 目标何时读取 clipboard
|
||||
|
||||
实验结果明确证明：
|
||||
|
||||
- race 在模型上真实存在
|
||||
- `150ms` 对慢消费者不安全
|
||||
- 增加 restore 窗口后可以避免慢消费者读到旧 clipboard
|
||||
|
||||
这一步把“怀疑”变成了“已确认的风险点”。
|
||||
|
||||
#### 4. 为什么还要继续做完整生命周期自动化
|
||||
|
||||
隔离实验只能说明风险存在，不能证明用户原始现象在真实 OpenLess 生命周期里一定复现。
|
||||
|
||||
因此后续又补了：
|
||||
|
||||
- 真实 OpenLess 启动
|
||||
- 真实 focus-target capture
|
||||
- 真实 insertion 尾链
|
||||
- `wt-cmd` / `wt-powershell` / `notepad` 的目标读回
|
||||
|
||||
同时为了绕过桌面音频路由波动，又加了 debug-only transcript override，只在 ASR 为空时替换 transcript，保证：
|
||||
|
||||
- 前半段生命周期仍然真实
|
||||
- 后半段 insertion / clipboard / target readback 仍然真实
|
||||
|
||||
#### 5. 最终根因判断
|
||||
|
||||
最终可以明确的根因不是“terminal 当前一定有 bug”，而是：
|
||||
|
||||
- Windows insertion 链路原本存在一个真实的 clipboard restore timing 风险
|
||||
- 这个风险可以解释历史上 terminal 场景里的不稳定反馈
|
||||
- 我们已经把这个风险点补了 hardening 修复
|
||||
|
||||
换句话说，这次 issue 真正承接的是：
|
||||
|
||||
- 一条历史上确实不够稳的 Windows terminal insertion 链路
|
||||
- 以及其中一个已经被确认和修补的底层时序风险
|
||||
|
||||
### 平台边界 / Platform Scope
|
||||
|
||||
- 直接范围：Windows
|
||||
- 关注层次：`clipboard lifecycle`、`insertion lifecycle`
|
||||
- terminal 是重点观察目标，但不是唯一可能受影响的慢消费者
|
||||
- `focus restore` 不是本轮主要根因
|
||||
|
||||
### 认领 / Ownership
|
||||
|
||||
- owner intent：`@Cooper-X-Oak`
|
||||
- 当前对应 draft/ready PR：`#160`
|
||||
|
||||
## 影响 / Impact
|
||||
|
||||
- 影响 Windows terminal 文本输入的稳定性认知
|
||||
- 会让 `PasteSent` 的用户语义和目标实际表现产生偏差
|
||||
- 增加“为什么目标没上屏 / 为什么需要手动 Ctrl+V”的排障成本
|
||||
- 对 Windows insertion 这条核心路径的可信度有直接影响
|
||||
|
||||
## 建议接受标准 / Proposed Acceptance Criteria
|
||||
|
||||
- [x] 明确 Windows `PasteSent` 与“目标已完成 paste”不是同一语义
|
||||
- [x] 明确并记录 clipboard restore timing 风险模型
|
||||
- [x] 完成最小 hardening 修复：
|
||||
- [x] Windows restore 延后到 `750ms`
|
||||
- [x] restore 改为异步执行
|
||||
- [x] 提供隔离时序实验，证明 race 模型成立
|
||||
- [x] 提供稳定化完整生命周期自动化，覆盖：
|
||||
- [x] `wt-cmd`
|
||||
- [x] `wt-powershell`
|
||||
- [x] `notepad`
|
||||
- [x] 记录当前环境下的最终结论：
|
||||
- [x] 历史风险真实存在
|
||||
- [x] 当前回归未再出现目标吃到旧 clipboard 的结果
|
||||
- [x] 当前稳定性较历史状态已有改善
|
||||
|
||||
## TODO / 不确定项
|
||||
|
||||
- 是否需要进一步收紧 `PasteSent` 相关用户文案，避免被理解为“已确认粘贴成功”
|
||||
- 若后续再收到用户现场反馈，是否需要补充更细的环境标签：
|
||||
- terminal host / profile
|
||||
- 输入法状态
|
||||
- 前台切换时序
|
||||
|
||||
建议 issue 标题：`[windows][insertion] 终端旧剪贴板粘贴风险已收敛，当前整链路回归稳定`
|
||||
@@ -1,52 +0,0 @@
|
||||
## 摘要
|
||||
|
||||
Closes #139
|
||||
|
||||
这个 PR 现在从“问题收敛中”推进到“regression review 中”。
|
||||
|
||||
本轮已经完成：
|
||||
|
||||
- Windows helper-window lifecycle root cause 收敛
|
||||
- `inactive` 路径的 native hide / non-topmost 收口
|
||||
- 冷启动最新 debug 包回归
|
||||
- 人工桌面症状回归：
|
||||
- click dead zone：通过
|
||||
- screenshot selectable：通过
|
||||
- drag stutter：通过
|
||||
|
||||
## 修复 / 新增 / 改进
|
||||
|
||||
- 对齐 PR 目标：关注 Windows Capsule helper-window lifecycle，而不是单点 dead zone workaround
|
||||
- 收口 Windows 上 `visible / hidden / inactive / non-participating` 的 Capsule 语义
|
||||
- 在 backend 上补齐 inactive 后的 native hide 行为，避免 transparent topmost helper window lingering
|
||||
- 新增 lifecycle contract / smoke 辅助脚本，帮助后续回归持续验证
|
||||
- 与 [issue-139-capsule-lifecycle.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/github-tracking/issue-139-capsule-lifecycle.md) 保持同一问题口径
|
||||
|
||||
## 兼容
|
||||
|
||||
- 不包含：Capsule geometry / rounded corner / titlebar frame 纯视觉适配
|
||||
- 不包含：QA hotkey / selection ask 输入源逻辑
|
||||
- 对现有用户 / 本地环境 / 构建流程的影响：只聚焦 lifecycle 主线，不扩大到 UI polish 线
|
||||
|
||||
## 测试计划
|
||||
|
||||
- [x] 命令：`node openless-all/app/scripts/windows-lifecycle-contract.test.mjs`
|
||||
- [x] 结果：通过
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：`npm run build`
|
||||
- [x] 结果：通过
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：`cargo check --manifest-path openless-all/app/src-tauri/Cargo.toml`
|
||||
- [x] 结果：通过
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：`powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-runtime-smoke.ps1`
|
||||
- [x] 结果：通过 launch / hotkey installed baseline
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：人工桌面回归（latest debug cold start -> dictation start/stop）
|
||||
- [x] 结果：点击 / 截图 / 拖拽三项全部通过
|
||||
- [x] 证据路径：当前线程回归记录
|
||||
关联 issue 建议标题：`[ui][windows] Capsule 隐藏后仍参与系统交互`
|
||||
@@ -1,46 +0,0 @@
|
||||
## 摘要
|
||||
|
||||
Closes #98
|
||||
References #143
|
||||
|
||||
这条 PR 已经不再只是 tracking 入口，而是承接本轮 Windows startup lifecycle 主线修复的实际变更。
|
||||
|
||||
当前结论：
|
||||
|
||||
- `visible / ready` 脱钩的主问题已收敛
|
||||
- 冷启动入口已从 backend immediate show 调整为 frontend-managed first show
|
||||
- 最新人工回归反馈是：启动过程基本流畅，人眼很难再分辨出明显的一闪
|
||||
- `#143` 现在更适合作为已收敛的 first-paint 症状票引用，而不是继续作为主 closure 目标
|
||||
|
||||
## 修复 / 新增 / 改进
|
||||
|
||||
- 收口 Windows 启动阶段的 first-show ownership
|
||||
- 在 `checking -> ready` 之间加入明确 gate，避免正式壳层在 startup transient phase 过早暴露
|
||||
- 增加冷启动测试脚本，默认优先拉最新 debug build，并区分：
|
||||
- frontend-managed first show
|
||||
- backend immediate show（仅调试用）
|
||||
- 增加 startup lifecycle contract test，锁住 hidden-on-create 与 readiness gate 语义
|
||||
|
||||
## 兼容
|
||||
|
||||
- 不包含：主窗口圆角 / 外框 / titlebar frame 等纯视觉适配
|
||||
- 不包含：更细粒度 startup latency 优化
|
||||
- 对现有用户 / 本地环境 / 构建流程的影响：聚焦 startup lifecycle 主线，不扩张到 UI polish 线
|
||||
|
||||
## 测试计划
|
||||
|
||||
- [x] 命令：`node openless-all/app/scripts/windows-startup-lifecycle-contract.test.mjs`
|
||||
- [x] 结果：通过
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：`npm run build`
|
||||
- [x] 结果：通过
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：`powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-cold-start.ps1 -PreferDebug -ShowMain`
|
||||
- [x] 结果：能够走 frontend-managed first show
|
||||
- [x] 证据路径：本地命令输出
|
||||
|
||||
- [x] 命令：冷启动截图与人工主观回归
|
||||
- [x] 结果：首屏体验明显改善，当前主观反馈为“几乎没有问题，人眼很难分辨”
|
||||
- [x] 证据路径：`artifacts-cold-start-screenshot.png`、`artifacts-cold-start-screenshot-8s.png`、`artifacts-cold-start-screenshot-front-managed.png` 与当前线程回归记录
|
||||
@@ -1,52 +0,0 @@
|
||||
## Summary
|
||||
|
||||
Closes #154
|
||||
|
||||
This draft PR now serves as a near-closure tracking anchor for the Windows dual-hotkey-source problem.
|
||||
|
||||
Current conclusion:
|
||||
|
||||
- Windows dictation / QA lifecycle should be owned by the backend low-level hook
|
||||
- renderer / window-local forwarding should not keep driving the same lifecycle
|
||||
- future work here should focus on regression review, not on reopening the architecture without new evidence
|
||||
|
||||
## Current Status
|
||||
|
||||
- keep draft for now
|
||||
- close to regression review
|
||||
- not a parked native-strategy problem like #153
|
||||
|
||||
## Scope
|
||||
|
||||
- source ownership
|
||||
- lifecycle precedence
|
||||
- mixed-source risk on Windows
|
||||
|
||||
Out of scope:
|
||||
|
||||
- helper-window drag semantics
|
||||
- main window / radius / appearance work
|
||||
- broad hotkey adapter rewrites without new evidence
|
||||
|
||||
## Key Finding
|
||||
|
||||
```text
|
||||
One lifecycle needs one owner source.
|
||||
On Windows, that owner source should be the backend low-level hook.
|
||||
```
|
||||
|
||||
## Evidence
|
||||
|
||||
- QA hotkey and follow-up flow remain healthy after ownership tightening
|
||||
- no evidence from this repair track suggests the renderer-forward path should remain a co-owner
|
||||
|
||||
## Next Step
|
||||
|
||||
- use this PR as the place to summarize regression evidence
|
||||
- only reopen architecture scope if new mixed-source failures appear
|
||||
|
||||
## Validation Plan
|
||||
|
||||
- [x] Manual verification: QA hotkey flow remains functional
|
||||
- [x] Manual verification: lifecycle tightening did not break follow-up QA
|
||||
- [ ] Regression review: confirm no new mixed-source phase drift evidence
|
||||
@@ -1,82 +0,0 @@
|
||||
## 摘要
|
||||
|
||||
Closes #159
|
||||
|
||||
这个 PR 承接的是 Windows terminal insertion 链路的一次收敛修复：
|
||||
|
||||
- 历史上 terminal 场景出现过“不能自动上屏、需要手动 `Ctrl+V`”的用户反馈
|
||||
- 本地测试也曾观察到一次“目标最终拿到旧剪贴板”的现象
|
||||
- 本轮排查确认了其中一处真实存在的底层风险：clipboard restore timing
|
||||
|
||||
因此，这个 PR 的目标不是去声称“当前存在一个稳定复现的 terminal bug”，而是：
|
||||
|
||||
- 修补一处已经被确认的 Windows insertion 时序风险
|
||||
- 把整条链路的回归覆盖补齐
|
||||
- 把最终结论收敛到可审阅、可维护的状态
|
||||
|
||||
## 修复 / 新增 / 改进
|
||||
|
||||
- Windows clipboard restore 从 `150ms` 提高到 `750ms`
|
||||
- clipboard restore 改为后台线程执行，不阻塞插入返回
|
||||
- 新增 Windows clipboard timing smoke，用于验证慢消费者 race
|
||||
- 新增完整生命周期自动化脚本，覆盖：
|
||||
- `wt-cmd`
|
||||
- `wt-powershell`
|
||||
- `notepad`
|
||||
- 稳定化自动化入口：
|
||||
- 通过 WebView2 remote debugging 连接主页面
|
||||
- 通过 Tauri invoke 驱动 `start_dictation` / `stop_dictation`
|
||||
- 新增 debug-only transcript override
|
||||
- 仅用于桌面音频路由不稳定时继续覆盖真实 insertion 尾链
|
||||
- 调整目标读回方式：
|
||||
- terminal 走 UIA 读取 `TermControl`
|
||||
- notepad 走 UIA 直接读取文本
|
||||
- 更新调查文档与 tracking 文档
|
||||
|
||||
## 兼容
|
||||
|
||||
- 正常用户路径不依赖 debug transcript override
|
||||
- debug transcript override 仅在 `debug_assertions` / test 构建下参与
|
||||
- Linux restore delay 保持原行为
|
||||
- 不涉及 UI/视觉顺手修改
|
||||
- 不涉及 QA hotkey / selection 主线逻辑修改
|
||||
|
||||
## 测试计划
|
||||
|
||||
- [x] `cargo fmt --all`
|
||||
- [x] `cargo check --lib`
|
||||
- [x] `python -m py_compile openless-all/app/scripts/windows-openless-lifecycle-e2e.py`
|
||||
- [x] `windows-real-asr-insertion-smoke.ps1` 脚本解析通过
|
||||
- [x] 隔离时序实验：
|
||||
- [x] 快消费者 + `150ms`
|
||||
- [x] 慢消费者 + `150ms`
|
||||
- [x] 慢消费者 + `750ms`
|
||||
- [x] 完整生命周期自动化：
|
||||
- [x] `wt-cmd`
|
||||
- [x] `wt-powershell`
|
||||
- [x] `notepad`
|
||||
- [x] 证据路径：
|
||||
- `docs/2026-05-02-windows-terminal-clipboard-restore-investigation.md`
|
||||
- `docs/github-tracking/issue-windows-terminal-clipboard-restore.md`
|
||||
|
||||
## 当前结论
|
||||
|
||||
- 历史上的 Windows terminal insertion 不稳定反馈是真实的
|
||||
- 本轮排查确认并修补了一处真实存在的 clipboard restore timing 风险
|
||||
- 稳定化完整生命周期自动化下：
|
||||
- `wt-cmd` 通过
|
||||
- `wt-powershell` 通过
|
||||
- `notepad` 通过
|
||||
- 当前环境中，目标最终都拿到本次 `finalText`，未再出现旧 clipboard 上屏
|
||||
|
||||
因此，这个 PR 的技术定位应当是：
|
||||
|
||||
- 针对历史不稳定现象的一次 hardening 修复
|
||||
- 外加完整的回归覆盖补强
|
||||
|
||||
## 剩余风险
|
||||
|
||||
- `750ms` 仍然是启发式保护，不是目标确认式握手
|
||||
- 如果未来再出现 terminal 现场问题，更可能是更窄的环境因子，而不是当前这条主链路已经明确存在的稳定故障
|
||||
|
||||
建议 PR 标题：`fix(windows): 延后剪贴板恢复并补齐插入回归覆盖`
|
||||
@@ -1,401 +0,0 @@
|
||||
# Issue #420 调研笔记：Wayland 下全局快捷键不可用
|
||||
|
||||
> 状态：调研稿（未实施任何代码改动）
|
||||
> 范围：仅评估方案；落地方案以第 7 节为推荐基线。
|
||||
> 日期：2026-05-13
|
||||
|
||||
---
|
||||
|
||||
## 1. 问题与现状
|
||||
|
||||
OpenLess Linux 端的全局热键监听走 `rdev::listen`，实现在 `openless-all/app/src-tauri/src/hotkey.rs:1183-1530`。代码在启动时检查 `XDG_SESSION_TYPE`，命中 `wayland` 直接 `Err("wayland_unsupported", "Wayland 暂不支持全局热键，请切到 X11 session 后再试")`（`hotkey.rs:1204-1208`）。
|
||||
|
||||
Issue #420 用户 aeoform 与另一位评论者在 Debian Wayland 上看到这条错误，明确建议：
|
||||
|
||||
> "建议补充对应的脚本或者命令让用户去系统设置中配置快捷键即可"
|
||||
|
||||
也就是：**不要求 OpenLess 自己抓全局按键**，**让桌面环境的快捷键设置去调用 OpenLess 的命令**。这是一个常见的 Linux 端规避模式，已经被同领域产品（Murmure 等）当成默认实践，详见第 3.2 节与 [Murmure docs](https://docs.murmure.app/configure-shortcuts-on-linux/)。
|
||||
|
||||
仓库现有支点：
|
||||
- `tauri-plugin-single-instance = "2"` 已在 `Cargo.toml:24` 启用，并在 `lib.rs:73` 注册了回调（目前仅用于聚焦主窗口）。
|
||||
- 可用 IPC 命令：`start_dictation` / `stop_dictation` / `cancel_dictation`（`commands.rs:1099-1110`），QA panel 控制（`commands.rs:1324-1330`），以及完整 hotkey 配置 surface。
|
||||
|
||||
---
|
||||
|
||||
## 2. 为什么 Wayland 不允许传统全局热键
|
||||
|
||||
X11 的设计里任何客户端都能 grab 整个键盘或注册全局快捷键 — 这同时让 X11 成了「天然的键盘记录器平台」。Wayland 协议在 2008 年重新设计时把这条路直接关掉：**键盘事件只在 surface 获得焦点时才送达对应客户端**。
|
||||
|
||||
权威表述出自 Wayland Book seat/keyboard 章节："the server sends `wl_keyboard.enter` when a surface receives keyboard focus, and `wl_keyboard.leave` when it's lost" — 协议层面没有任何「未聚焦也能读键」的接口（[wayland-book.com](https://wayland-book.com/seat/keyboard.html)）。
|
||||
|
||||
`pynput` / `rdev` / 任何依赖 X11 keyboard grab 的库在 Wayland 上「故意」失效，原因即此（[Wayland Fragmentation](https://www.semicomplete.com/blog/xdotool-and-exploring-wayland-fragmentation/)、[Vocalinux issue #80](https://github.com/jatinkrmalik/vocalinux/issues/80)）。
|
||||
|
||||
只要应用要在「自己窗口没聚焦」时收到按键，就必须走以下「半民间」方案之一：
|
||||
|
||||
| 方案 | 取舍 |
|
||||
|------|------|
|
||||
| **evdev/uinput** 直接读 `/dev/input/event*` | 绕过 Wayland 协议，X11/Wayland/TTY 都能用；**需要把用户加入 `input` group 或 setuid**，安全模型差 |
|
||||
| **libei + xdg-desktop-portal RemoteDesktop** | 用户每次启动都要授权；文档稀少；只在做合成器自动化时合理 |
|
||||
| **xdg-desktop-portal GlobalShortcuts** | 走门户协商；标准化但合成器实现不齐（见 3.1） |
|
||||
| **合成器私有协议** | 如 `hyprland-global-shortcuts-v1`；只在单一合成器有效 |
|
||||
|
||||
来源：[Wayland Fragmentation: xdotool adventure](https://www.semicomplete.com/blog/xdotool-and-exploring-wayland-fragmentation/)、[Wayland - keyboard-shortcuts-inhibit-unstable-v1](https://wayland.app/protocols/keyboard-shortcuts-inhibit-unstable-v1)。
|
||||
|
||||
---
|
||||
|
||||
## 3. 可选方案（含适配范围、成熟度、维护代价）
|
||||
|
||||
### 3.1 xdg-desktop-portal GlobalShortcuts
|
||||
|
||||
**协议**：`org.freedesktop.portal.GlobalShortcuts`（[规范](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.GlobalShortcuts.html)）。
|
||||
|
||||
应用调用 `CreateSession → BindShortcuts`，门户弹出一个对话框让用户**给每个 shortcut 选实际按键**。之后通过 `Activated` / `Deactivated` 信号通知应用。`ConfigureShortcuts` 方法在 v2 加入，允许应用打开门户的修改 UI。
|
||||
|
||||
合成器实现状态（截至 2026-05）：
|
||||
|
||||
| 合成器 | 状态 | 备注 |
|
||||
|--------|------|------|
|
||||
| **KDE Plasma 6** | 已稳定 | xdg-desktop-portal-kde 自 MR !80 起原生支持，2024-2025 持续迭代（[!368 改进流程](https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/368)、[!449 记住拒绝项](https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/449)） |
|
||||
| **GNOME (Mutter)** | **尚未原生落地** | issue [GNOME/xdg-desktop-portal-gnome#47](https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/issues/47) 仍开放；Murmure 文档明确写「Mutter's XDG GlobalShortcuts portal is unreliable (latency, dropped events)，GNOME 默认走 CLI 模式」（[Murmure docs](https://docs.murmure.app/configure-shortcuts-on-linux/)） |
|
||||
| **Hyprland** | 已支持 | 通过 `xdg-desktop-portal-hyprland`；同时还有合成器私有的 [`hyprland-global-shortcuts-v1`](https://wayland.app/protocols/hyprland-global-shortcuts-v1) |
|
||||
| **sway / wlroots** | 已支持 | 通过 `xdg-desktop-portal-wlr` |
|
||||
| **COSMIC** | 部分 | 实现质量随版本变化，未独立验证 |
|
||||
|
||||
**关键缺陷**（多个合成器共有）：
|
||||
- 用户感受到的「再设置一次」：应用只能给 *preferred trigger*，最终键位由门户对话框决定。Hyprland 上甚至要求用户手改 config 文件 — 等于「应用申请，用户在 hyprland.conf 里实际绑」（[dec05eba.com 分析](https://dec05eba.com/2024/03/29/wayland-global-hotkeys-shortcut-is-mostly-useless/)）。
|
||||
- **GNOME 是最大盲区**。Issue #420 用户用的就是 Debian — Debian 默认 GNOME。在 GNOME 上跑 GlobalShortcuts 等于压根不能用。
|
||||
- 没有 push-to-talk：门户在 key-press 上触发事件，但是否传 release 事件、是否 dedupe，依赖合成器（OpenLess 当前依赖 hotkey 的 edge 来做 Toggle，需要稳定的成对事件）。
|
||||
|
||||
**维护代价**：新增 `ashpd` crate + DBus 异步流（参见 3.2 例外、4 节示例）。每个发行版/合成器组合都得人肉测一遍，bug 报告会按合成器分裂。
|
||||
|
||||
**结论**：现阶段加进来对 GNOME 用户毫无帮助，且会引入合成器分裂的支持负担。
|
||||
|
||||
### 3.2 CLI + single-instance 转发（推荐）
|
||||
|
||||
把 OpenLess 二进制本身做成可被外部调起的「无 GUI 触发器」：
|
||||
|
||||
```
|
||||
桌面环境快捷键 → 启动 openless --toggle-dictation
|
||||
↓
|
||||
tauri-plugin-single-instance 拦截
|
||||
↓
|
||||
已运行的 OpenLess 主实例从回调拿到 argv
|
||||
↓
|
||||
解析 --toggle-dictation → 调用 coordinator.start/stop_dictation
|
||||
```
|
||||
|
||||
适配范围：**所有 Linux 桌面环境**（GNOME / KDE / Hyprland / sway / Cosmic / XFCE / i3 / ...），因为它只依赖「桌面环境能绑定一个 shell 命令」这个最低公共能力。X11 / Wayland 都通杀。
|
||||
|
||||
成熟度：极高。这是 Linux 桌面集成的最普世做法（OBS、Mumble、1Password、Albert 等都同时支持），也是 Murmure 在 GNOME 上的默认模式（[Murmure docs](https://docs.murmure.app/configure-shortcuts-on-linux/)）。`tauri-plugin-single-instance` 2.x 已经在仓库里，回调拿 argv 是其官方设计（[官方文档](https://v2.tauri.app/plugin/single-instance/)）。
|
||||
|
||||
维护代价：低。代码改动集中在三处：
|
||||
1. `main.rs` 早期解析一次 argv（在 Tauri Builder 之前不退出，只记下 intent）；
|
||||
2. `lib.rs:73` 的 single-instance 回调里识别 argv 并发往 coordinator；
|
||||
3. README / Settings 页加一段文档教用户怎么绑桌面快捷键。
|
||||
|
||||
唯一已知限制：**桌面 OS 级快捷键大多只在 key-press 触发**（按键即 fire，不传 key-release）。这天然兼容 Toggle 模式，但不支持「按住说话 / 松开收尾」的 push-to-talk。OpenLess 默认就是 Toggle（`CLAUDE.md` 写明：「Hotkey is toggle-only, not press-and-hold」），所以不冲突。这一限制在 Murmure 文档里也明确写出：「Push-to-talk limitation — OS shortcuts only fire on key press」。
|
||||
|
||||
### 3.3 evdev/uinput 直接读
|
||||
|
||||
绕过 Wayland，直接打开 `/dev/input/event*` 读 scancode。
|
||||
|
||||
适配范围：所有 Linux（包括 TTY）。
|
||||
权限要求：用户必须在 `input` group，或二进制 setuid。**两条都是发行版会警告的安全降级**。
|
||||
成熟度：技术上稳定（`evdev-shortcut` crate 存在），但用户经验差：要手动 `usermod -aG input $USER` 然后注销重登 — 普通用户不会做。
|
||||
不推荐用于面向消费者的 OpenLess。
|
||||
|
||||
来源：[evremap (Wez)](https://github.com/wez/evremap)、[evdev_shortcut crate](https://docs.rs/evdev-shortcut/latest/evdev_shortcut/)。
|
||||
|
||||
### 3.4 libei
|
||||
|
||||
libei + RemoteDesktop portal 是新一代「让应用模拟键盘鼠标」的官方路径，但目前主要用例是远程桌面 / 自动化测试。每次启动都要 portal 弹授权框，且 GNOME 实现仍在迭代。文档稀少。
|
||||
|
||||
不推荐用作快捷键触发路径。来源：[Sending keystrokes to Wayland — Medium](https://medium.com/@python-javascript-php-html-css/sending-keyboard-strokes-to-wayland-linux-windows-solutions-and-challenges-9319cf424d06)。
|
||||
|
||||
---
|
||||
|
||||
## 4. tauri-plugin-single-instance 2.x 最小示例
|
||||
|
||||
当前发布版本：**2.4.2**（2026-05-02）。仓库已锁 `tauri-plugin-single-instance = "2"`（[crates.io](https://crates.io/crates/tauri-plugin-single-instance)）。
|
||||
|
||||
回调签名：`Fn(&AppHandle, Vec<String>, String) + Send + Sync + 'static` — 三个参数是 `app handle / argv / cwd`。来源：[Tauri 官方文档](https://v2.tauri.app/plugin/single-instance/)。
|
||||
|
||||
OpenLess 现有调用点（`lib.rs:73-78`）目前忽略 `argv` / `cwd`：
|
||||
|
||||
```rust
|
||||
.plugin(tauri_plugin_single_instance::init(|app, _argv, _cwd| {
|
||||
log::info!("[single-instance] another instance launched, focusing existing main window");
|
||||
show_main_window(app);
|
||||
}))
|
||||
```
|
||||
|
||||
改造后形态（示意，不在本调研里实施）：
|
||||
|
||||
```rust
|
||||
.plugin(tauri_plugin_single_instance::init(|app, argv, _cwd| {
|
||||
if let Some(intent) = parse_cli_intent(&argv) {
|
||||
let coord: tauri::State<Arc<Coordinator>> = app.state();
|
||||
dispatch_intent(coord.inner().clone(), intent);
|
||||
return; // 不抢焦点
|
||||
}
|
||||
show_main_window(app); // 无 intent → 退回原来的「聚焦主窗口」
|
||||
}))
|
||||
```
|
||||
|
||||
注意点：
|
||||
- 回调在 Tauri 主线程上执行，长任务必须 spawn 到 tokio runtime；OpenLess 的 coordinator 接口本来就异步。
|
||||
- 第二实例的进程**已经退出**，所以「不抢焦点」就是真不弹窗 — 体验上跟原生快捷键一致。
|
||||
- single-instance 插件必须**第一个**注册（早于 `tauri_plugin_shell` 等），这是官方文档强调的注意点。OpenLess 目前已经满足。
|
||||
|
||||
---
|
||||
|
||||
## 5. CLI 参数解析建议
|
||||
|
||||
**结论：用 `std::env::args()` 手写极简解析，不引入 clap。**
|
||||
|
||||
理由：
|
||||
- OpenLess 是 GUI app，CLI 入口只是「触发器」，参数集小（toggle-dictation / toggle-qa / cancel / show），没有子命令树。
|
||||
- 引入 `clap` 会让二进制体积涨一截（~200 KB），还要处理 `--help` 输出（GUI 程序输出帮助文本到 stderr，用户基本看不到，价值有限）。
|
||||
- 关键风险：**CLI 解析不能让 OpenLess panic 退出**。如果用户拖文件到 .desktop launcher 或者发行版包装传了奇怪参数，GUI 必须照常起来。`clap` 默认 `unwrap_or_else(|e| e.exit())` 会让进程退出，必须改成 `try_parse` + 静默忽略错误 — 那不如直接手写。
|
||||
|
||||
最小手写示意：
|
||||
|
||||
```rust
|
||||
// main.rs：在 Tauri Builder 之前
|
||||
#[derive(Clone, Copy)]
|
||||
pub enum CliIntent {
|
||||
ToggleDictation,
|
||||
ToggleQa,
|
||||
Cancel,
|
||||
Show,
|
||||
}
|
||||
|
||||
fn parse_cli_intent<S: AsRef<str>>(args: &[S]) -> Option<CliIntent> {
|
||||
// 跳过 argv[0]，逐项匹配；多余/未知参数静默忽略，绝不 panic
|
||||
for arg in args.iter().skip(1) {
|
||||
match arg.as_ref() {
|
||||
"--toggle-dictation" => return Some(CliIntent::ToggleDictation),
|
||||
"--toggle-qa" => return Some(CliIntent::ToggleQa),
|
||||
"--cancel" => return Some(CliIntent::Cancel),
|
||||
"--show" => return Some(CliIntent::Show),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
```
|
||||
|
||||
把同样的 helper 在 `lib.rs:73` 的回调里复用 — 第一次进程启动（首实例）和 single-instance 转发走同一条解析路径。
|
||||
|
||||
`std::env::args()` 是 Rust 标准库，不引外部依赖。来源：[Rust by Example - std::env::args](https://doc.rust-lang.org/std/env/fn.args.html)、[Tauri CLI plugin（参考路径，本次不使用）](https://v2.tauri.app/plugin/cli/)。
|
||||
|
||||
---
|
||||
|
||||
## 6. 桌面环境配置自定义快捷键的步骤
|
||||
|
||||
OpenLess 在 Linux 安装后默认在 `$PATH` 里（或在 `.desktop` 旁边的 bin 目录）。下面假定二进制叫 `openless`。如果安装在非 PATH 路径（如 AppImage），文档里应同时写绝对路径。
|
||||
|
||||
### 6.1 GNOME (Wayland)
|
||||
|
||||
**GUI 路径**（[GNOME 官方帮助](https://help.gnome.org/gnome-help/keyboard-shortcuts-set.html)）：
|
||||
|
||||
1. Settings → Keyboard
|
||||
2. Keyboard Shortcuts → View and Customize Shortcuts
|
||||
3. Custom Shortcuts → Add Shortcut（+ 按钮）
|
||||
4. Name: `OpenLess Dictate`
|
||||
5. Command: `openless --toggle-dictation`
|
||||
6. 点击 "Add Shortcut..."，按下想绑的键（如 `Super+Y`）
|
||||
7. 点 Add 保存
|
||||
|
||||
**CLI / 脚本化**（[Programster's Blog](https://blog.programster.org/using-the-cli-to-set-custom-keyboard-shortcuts)、[Ubuntu Wiki - Keybindings](https://wiki.ubuntu.com/Keybindings)）。注意 schema 是单数 `custom-keybinding`（不带 s），relocatable schema 需要带路径访问：
|
||||
|
||||
```bash
|
||||
KEYBIND_PATH="/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/openless0/"
|
||||
gsettings set org.gnome.settings-daemon.plugins.media-keys custom-keybindings \
|
||||
"['$KEYBIND_PATH']"
|
||||
gsettings set "org.gnome.settings-daemon.plugins.media-keys.custom-keybinding:$KEYBIND_PATH" \
|
||||
name 'OpenLess Dictate'
|
||||
gsettings set "org.gnome.settings-daemon.plugins.media-keys.custom-keybinding:$KEYBIND_PATH" \
|
||||
command 'openless --toggle-dictation'
|
||||
gsettings set "org.gnome.settings-daemon.plugins.media-keys.custom-keybinding:$KEYBIND_PATH" \
|
||||
binding '<Super>y'
|
||||
```
|
||||
|
||||
### 6.2 KDE Plasma 6 (Wayland)
|
||||
|
||||
**GUI 路径**（[KDE Discuss - Custom Shortcuts](https://discuss.kde.org/t/adding-shortcuts-to-systemsettings/15276)）：
|
||||
|
||||
1. System Settings → Keyboard → Shortcuts
|
||||
2. "+ Add New" → Command/URL
|
||||
3. Trigger: 录想绑的键
|
||||
4. Action: `openless --toggle-dictation`
|
||||
5. Apply
|
||||
|
||||
**CLI / 脚本化**（[commandmasters.com](https://commandmasters.com/commands/kwriteconfig5-linux/)、[KDE Discuss - kglobalaccel](https://discuss.kde.org/t/plasma-6-method-to-refresh-kglobalaccel-shortcuts/17995)）：
|
||||
|
||||
Plasma 6 把 shortcut 存在 `~/.config/kglobalshortcutsrc`，工具改名为 `kwriteconfig6`。完整的 custom-shortcut 脚本化在 KDE 上比 GNOME 复杂（涉及 D-Bus 注册 + kglobalaccel 重载）：
|
||||
|
||||
```bash
|
||||
# 写入声明
|
||||
kwriteconfig6 --file kglobalshortcutsrc \
|
||||
--group 'openless.desktop' --key '_k_friendly_name' 'OpenLess'
|
||||
kwriteconfig6 --file kglobalshortcutsrc \
|
||||
--group 'openless.desktop' --key 'dictate' 'Meta+Y,none,Toggle Dictation'
|
||||
|
||||
# 让 kglobalaccel 重载（必需，否则要重登）
|
||||
qdbus org.kde.kglobalaccel /kglobalaccel reloadConfig
|
||||
```
|
||||
|
||||
> 实践建议：KDE 上推荐**直接引导用户走 GUI**，因为 kglobalshortcutsrc 的 group 命名必须匹配 `.desktop` 文件 + 需要 service 注册，脚本化容易出错。
|
||||
|
||||
### 6.3 Hyprland
|
||||
|
||||
**GUI 路径**：无。Hyprland 配置就是文本文件，没有图形化绑定。
|
||||
|
||||
**配置文件**（[Hyprland Wiki - Binds](https://wiki.hypr.land/Configuring/Basics/Binds/)、[ArchWiki - Hyprland](https://wiki.archlinux.org/title/Hyprland)）：
|
||||
|
||||
文件位置 `~/.config/hypr/hyprland.conf`。Hyprland 0.54 及更早用传统 hyprlang 语法：
|
||||
|
||||
```
|
||||
bind = SUPER, Y, exec, openless --toggle-dictation
|
||||
bind = SUPER SHIFT, Y, exec, openless --toggle-qa
|
||||
```
|
||||
|
||||
Hyprland 0.55+ 推荐用 Lua（hyprlang 已 deprecated）：
|
||||
|
||||
```lua
|
||||
hl.bind({"SUPER"}, "y", "exec", "openless --toggle-dictation")
|
||||
```
|
||||
|
||||
reload：`hyprctl reload`（或重启 hyprland）。
|
||||
|
||||
### 6.4 sway
|
||||
|
||||
**GUI 路径**：无（同 Hyprland，纯文本配置）。
|
||||
|
||||
**配置文件**（[sway(5) - ArchWiki](https://man.archlinux.org/man/sway.5)、[swaywm/sway Wiki - Shortcut handling](https://github.com/swaywm/sway/wiki/Shortcut-handling)）：
|
||||
|
||||
文件位置 `~/.config/sway/config`。语法：
|
||||
|
||||
```
|
||||
bindsym $mod+y exec openless --toggle-dictation
|
||||
bindsym $mod+Shift+y exec openless --toggle-qa
|
||||
```
|
||||
|
||||
reload：`swaymsg reload`。
|
||||
|
||||
---
|
||||
|
||||
## 7. 推荐的最小修复方案（落地到 OpenLess）
|
||||
|
||||
### 7.1 本期实现（Beta 1.3.x）：CLI + single-instance 转发
|
||||
|
||||
理由：
|
||||
1. **覆盖范围最大**：所有桌面环境直接可用，包括 Issue #420 用户的 Debian + GNOME（GNOME 是 portal 路线的最大盲区）。
|
||||
2. **改动量最小**：复用现有 `tauri-plugin-single-instance` 与 `coordinator::Coordinator` 公共接口，零新依赖。
|
||||
3. **与 toggle-only 设计契合**：OpenLess 现在就是 toggle-only（`CLAUDE.md` 已明确），不存在 push-to-talk 限制冲突。
|
||||
4. **故障面小**：CLI 解析 → IPC 命令链路是同步可测的，没有 D-Bus / 合成器版本依赖。
|
||||
5. **行业先例**：Murmure（同类产品）在 GNOME 上默认就用这条路径。
|
||||
|
||||
### 7.2 改动清单（**不在本调研中实施，仅作落地参考**）
|
||||
|
||||
| 文件 | 改动 | 行数估计 |
|
||||
|------|------|---------|
|
||||
| `openless-all/app/src-tauri/src/cli.rs`（新） | `CliIntent` 枚举 + `parse_cli_intent` 函数 + 单元测试 | ~60 |
|
||||
| `openless-all/app/src-tauri/src/lib.rs:73` | single-instance 回调里解析 argv，调度 intent | ~15 |
|
||||
| `openless-all/app/src-tauri/src/lib.rs`（main 函数早期） | 首次启动也跑一遍 `parse_cli_intent`，记下首意图，coordinator 准备好后再触发；或简单约定「首次启动忽略 CLI intent，只起 GUI」 | ~5 |
|
||||
| `openless-all/app/src-tauri/src/hotkey.rs:1204-1208` | 移除「wayland 报错」分支；改成 **info 级日志** + 不安装 rdev 监听（X11 仍走 rdev，Wayland 静默退出 listener） | ~10 |
|
||||
| `openless-all/app/src/i18n/{zh-CN,en}.ts` | 新增 "Linux Wayland 下推荐通过桌面快捷键调用 `openless --toggle-dictation`" 引导文案 | ~10 |
|
||||
| `README.md` / `README.zh.md` / `USAGE.md` | 把第 6 节四个 DE 的配置示例写进去 | ~50 |
|
||||
|
||||
### 7.3 CLI 参数命名
|
||||
|
||||
按题面建议保留：
|
||||
|
||||
```
|
||||
openless --toggle-dictation # 等价于按一次主热键
|
||||
openless --toggle-qa # 等价于按一次 QA 热键
|
||||
openless --cancel # 等价于 Esc
|
||||
openless --show # 唤起主窗口（已有 single-instance 行为）
|
||||
```
|
||||
|
||||
约定：所有 flag 在 Wayland 上是「唯一进入点」；X11 上仍然支持原 rdev 热键，CLI 是补充而非替代（用户可以同时用）。
|
||||
|
||||
### 7.4 Wayland 检测下的行为变化
|
||||
|
||||
`hotkey.rs:1204-1208` 当前的 `wayland_unsupported` 错误**不应再向上传**。改为：
|
||||
|
||||
- 检测到 Wayland → 不安装 rdev listener，记一行 INFO log；
|
||||
- 前端在 Settings → 热键页显示一行提示（i18n）：「检测到 Wayland session。请在系统设置中将 `openless --toggle-dictation` 绑到一个快捷键。点这里查看说明 →」；
|
||||
- 链接打开 README 中对应章节，按 DE 列出 6.1-6.4 的步骤。
|
||||
|
||||
这样既消除了 Issue #420 的报错，又主动告诉用户下一步该做什么，符合用户原始建议「补充对应脚本或命令让用户去系统设置中配置」。
|
||||
|
||||
### 7.5 后续路径（**留给单独 issue，本期不做**）
|
||||
|
||||
- **xdg-desktop-portal GlobalShortcuts 集成**：等 GNOME 落地 issue [#47](https://gitlab.gnome.org/GNOME/xdg-desktop-portal/issues/47) 后再评估。届时 KDE + Hyprland + sway + GNOME 都成熟，可作为 CLI 路径的「升级版」（应用内绑定，无需用户去 DE 设置）。引入 `ashpd` crate（参考 4 节代码骨架与 [ashpd demo](https://github.com/bilelmoussaoui/ashpd/blob/master/demo/client/src/portals/desktop/global_shortcuts.rs)）。
|
||||
- 现在不做的另一个理由：CLI 方案不会被 portal 方案取代 — 两者可共存。Portal 方案先在 KDE 上灰度也来得及。
|
||||
- **`hyprland-global-shortcuts-v1` 原生协议**：单合成器优化，优先级最低。
|
||||
- **Push-to-talk 模式**：如果未来想支持「按住录音」，OS 级快捷键路径会卡住（DE 只发 key-press），到那时再评估 portal / libei。
|
||||
|
||||
---
|
||||
|
||||
## 8. 参考资料
|
||||
|
||||
**Wayland 协议与安全模型**
|
||||
- [The Wayland Protocol — seat/keyboard](https://wayland-book.com/seat/keyboard.html)
|
||||
- [Wayland - keyboard-shortcuts-inhibit-unstable-v1](https://wayland.app/protocols/keyboard-shortcuts-inhibit-unstable-v1)
|
||||
- [Exploring the Fragmentation of Wayland (semicomplete.com)](https://www.semicomplete.com/blog/xdotool-and-exploring-wayland-fragmentation/)
|
||||
- [Sending Keyboard Strokes to Wayland (Medium)](https://medium.com/@python-javascript-php-html-css/sending-keyboard-strokes-to-wayland-linux-windows-solutions-and-challenges-9319cf424d06)
|
||||
- [tauri-apps/global-hotkey issue #28 — Wayland support](https://github.com/tauri-apps/global-hotkey/issues/28)
|
||||
- [dec05eba.com — Wayland global hotkeys is mostly useless](https://dec05eba.com/2024/03/29/wayland-global-hotkeys-shortcut-is-mostly-useless/)
|
||||
|
||||
**xdg-desktop-portal GlobalShortcuts**
|
||||
- [GlobalShortcuts 规范（flatpak.github.io）](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.GlobalShortcuts.html)
|
||||
- [KDE Portal MR !80 — Implementation of GlobalShortcuts](https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/80)
|
||||
- [KDE Portal MR !368 — Improve workflow](https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/368)
|
||||
- [KDE Portal MR !449 — Remember denied shortcuts](https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/449)
|
||||
- [GNOME xdg-desktop-portal-gnome issue #47 — GlobalShortcuts feature request](https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/issues/47)
|
||||
- [GNOME Discourse — Feature request: GlobalShortcuts portal](https://discourse.gnome.org/t/feature-request-globalshortcuts-portal/15343)
|
||||
|
||||
**ashpd（Rust 门户客户端）**
|
||||
- [ashpd crate (docs.rs)](https://docs.rs/ashpd/latest/ashpd/)
|
||||
- [ashpd repo — global_shortcuts.rs (client/src)](https://github.com/bilelmoussaoui/ashpd/blob/master/client/src/desktop/global_shortcuts.rs)
|
||||
- [ashpd repo — demo global_shortcuts.rs (端到端示例)](https://github.com/bilelmoussaoui/ashpd/blob/master/demo/client/src/portals/desktop/global_shortcuts.rs)
|
||||
- [ASHPD Demo on Flathub](https://flathub.org/en/apps/com.belmoussaoui.ashpd.demo)
|
||||
|
||||
**Tauri single-instance**
|
||||
- [tauri-plugin-single-instance — 官方文档](https://v2.tauri.app/plugin/single-instance/)
|
||||
- [tauri-plugin-single-instance — crates.io（最新 2.4.2，2026-05-02）](https://crates.io/crates/tauri-plugin-single-instance)
|
||||
- [tauri-plugin-single-instance — docs.rs/latest](https://docs.rs/crate/tauri-plugin-single-instance/latest)
|
||||
- [Tauri v2 — Calling Rust from Frontend](https://v2.tauri.app/develop/calling-rust/)
|
||||
|
||||
**桌面环境快捷键配置**
|
||||
- [GNOME 帮助 — Set keyboard shortcuts](https://help.gnome.org/gnome-help/keyboard-shortcuts-set.html)
|
||||
- [Programster — Using the CLI to Set Custom Keyboard Shortcuts](https://blog.programster.org/using-the-cli-to-set-custom-keyboard-shortcuts)
|
||||
- [Ubuntu Wiki — Keybindings](https://wiki.ubuntu.com/Keybindings)
|
||||
- [KDE Discuss — Adding shortcuts to Systemsettings](https://discuss.kde.org/t/adding-shortcuts-to-systemsettings/15276)
|
||||
- [KDE Discuss — kglobalaccel reload (Plasma 6)](https://discuss.kde.org/t/plasma-6-method-to-refresh-kglobalaccel-shortcuts/17995)
|
||||
- [commandmasters — kwriteconfig5 / kwriteconfig6](https://commandmasters.com/commands/kwriteconfig5-linux/)
|
||||
- [Hyprland Wiki — Configuring/Basics/Binds](https://wiki.hypr.land/Configuring/Basics/Binds/)
|
||||
- [ArchWiki — Hyprland](https://wiki.archlinux.org/title/Hyprland)
|
||||
- [Hyprland Global Shortcuts protocol v1](https://wayland.app/protocols/hyprland-global-shortcuts-v1)
|
||||
- [sway(5) — ArchWiki man page](https://man.archlinux.org/man/sway.5)
|
||||
- [swaywm/sway Wiki — Shortcut handling](https://github.com/swaywm/sway/wiki/Shortcut-handling)
|
||||
- [Mark Stosberg — Sway keybindings tips](https://mark.stosberg.com/sway-keybindings/)
|
||||
|
||||
**同类产品参考（Murmure — 同样是 STT 应用）**
|
||||
- [Murmure docs — Configure shortcuts on Linux](https://docs.murmure.app/configure-shortcuts-on-linux/)
|
||||
- [Murmure repo — Kieirra/murmure](https://github.com/Kieirra/murmure)
|
||||
|
||||
**evdev / 替代方案**
|
||||
- [evdev_shortcut crate](https://docs.rs/evdev-shortcut/latest/evdev_shortcut/)
|
||||
- [wez/evremap — Linux/Wayland keyboard remapper](https://github.com/wez/evremap)
|
||||
- [xwaykeyz — X11 + Wayland keymapper](https://github.com/RedBearAK/xwaykeyz)
|
||||
- [Vocalinux issue #80 — Wayland support via evdev](https://github.com/jatinkrmalik/vocalinux/issues/80)
|
||||
|
||||
**OpenLess 仓库锚点**
|
||||
- 当前实现：`openless-all/app/src-tauri/src/hotkey.rs:1183-1530`（Wayland 报错在 `:1204-1208`）
|
||||
- single-instance 回调：`openless-all/app/src-tauri/src/lib.rs:73-78`
|
||||
- IPC commands：`openless-all/app/src-tauri/src/commands.rs:1099-1110`（dictation）、`:1324-1330`（QA panel）
|
||||
- Cargo deps：`openless-all/app/src-tauri/Cargo.toml:24`（`tauri-plugin-single-instance = "2"`）
|
||||
@@ -1,159 +0,0 @@
|
||||
# OpenLess beta — Logic Review (commit 400097ad)
|
||||
|
||||
Audit branch: `origin/beta` @ `400097ad` (= "Merge PR #391 fix/audit-async-hygiene").
|
||||
Sources: `openless-all/app/src-tauri/src/{lib.rs, coordinator.rs, coordinator/{dictation,qa,resources}.rs, coordinator_state.rs, hotkey.rs, audio_mute.rs}`, `openless-all/app/src/lib/{ipc.ts,types.ts}`.
|
||||
|
||||
> Caller's premise correction: the prompt asserts "PR #389 (emit_capsule main thread) is pending merge but the same change is already cherry-picked-equivalent on the current code." This is **false** at `400097ad`. Commits `faf02ad4` and `84ee3d96` exist on a side branch but are NOT ancestors of `400097ad` (verified with `git merge-base --is-ancestor`). The audio-thread → AppKit/Win32 SIGTRAP risk that PR #389 was written to fix is therefore **still live on this beta**. See P4.
|
||||
|
||||
## Summary
|
||||
|
||||
| Path | Verdict | Issues found |
|
||||
|------|---------|--------------|
|
||||
| P1 startup | OK | All 6 listeners paired (start in `setup`/`Ready`, stop in `Exit`); tray watcher signaled. |
|
||||
| P2 press/release | OK with 1 ⚠️ | Routing + dedup correct; `acquire_recording_mute` correctly awaited in dictation start. |
|
||||
| P3 end-of-session | 1 🚩 | `cancel_session` during Processing leaves `focus_target` set until next `begin_session` overwrites it. PR #387 contract is incomplete for the Processing branch. |
|
||||
| P4 capsule UI emit | 1 🚩 (CRITICAL) | `emit_capsule` calls `window.show/hide` + `show_capsule_window_no_activate` directly from the cpal audio callback at ~30 Hz on `400097ad`. PR #389 is **not** merged here. |
|
||||
| P5 shutdown | 1 🚩 | `acquire_recording_mute` at QA path (`coordinator.rs:2313`) is missing `.await` — return value is a dropped Future. PR #391 hygiene fix is incomplete. Compiler emits `unused_must_use`. |
|
||||
|
||||
Net: **3 real bugs (🚩)**, 1 cross-PR composition concern, plus minor smells. Two of the three are direct consequences of merge-incomplete state of PR #389/#391. The Processing-cancel `focus_target` leak is a code-path PR #387 missed.
|
||||
|
||||
## Findings (per path)
|
||||
|
||||
### P1 — Startup
|
||||
|
||||
- OK `lib.rs:316-358` — `RunEvent::Exit` stops all 6 hotkey listeners + signals tray watcher; matches the 6 starts at `lib.rs:226` (dictation, in `setup`) and `lib.rs:320-325` (QA / combo / translation / switch_style / open_app, in `RunEvent::Ready`).
|
||||
- OK `coordinator.rs:344-358, 371-373, 383-385, 395-397, 407-409, 1313-1347` — every `stop_*_listener` for `global-hotkey`-backed monitors marshals the `Drop` to `app.run_on_main_thread`, matching the issue #169 contract for Carbon `RemoveEventHotKey`. `take_combo_hotkey_on_main_thread` / `take_translation_hotkey_on_main_thread` / `take_action_hotkey_on_main_thread` are the helpers; `stop_qa_hotkey_listener` inlines the same pattern.
|
||||
- OK `coordinator.rs:330-332` + `hotkey.rs:344-355` — dictation `stop_hotkey_listener` is a plain `inner.hotkey.lock().take()`, but the Drop chain is `HotkeyMonitor::drop` → `MacHotkeyAdapter::shutdown` → `CGEventTapEnable(false)` + `CFRunLoopStop(rl)`. PR #388 fix in place; the comment at `hotkey.rs:311-314` correctly notes both APIs are documented thread-safe, so the lack of main-thread marshalling here is intentional.
|
||||
- ℹ️ `coordinator.rs:316-318` + `global_hotkey_runtime.rs:60-63` — `request_shutdown` is `#[allow(dead_code)]` and never set in production; supervisor loops poll it but nothing flips it (matches PR #392's "passive infrastructure" promise).
|
||||
- ℹ️ `global_hotkey_runtime.rs:19, 41-55` — `GlobalHotKeyManager` lives in `OnceCell<Arc<...>>` and is therefore never Dropped in production. On macOS this means the Carbon event handler is reaped only at process exit. Acceptable but worth noting if anyone tries to add hot-restart later.
|
||||
|
||||
### P2 — Press/release
|
||||
|
||||
- OK `coordinator/dictation.rs:11-28` — `handle_pressed_edge` swap-dedups via `inner.hotkey_trigger_held.swap(true, SeqCst)`; routing checks `panel_visible && !dictation_active` (PR #390 fix). The `dictation_active = !matches!(phase, SessionPhase::Idle)` snapshot is the correct guard: it lets a hotkey press during an in-flight dictation flow fall through to `handle_pressed`, even when the QA panel happens to be visible.
|
||||
- OK `coordinator/dictation.rs:53-67` — `handle_released_edge` symmetric: `panel_visible && !dictation_active → return`. If dictation_active was true when pressed (so it routed to `handle_pressed`), released will also bypass the QA short-circuit and reach `handle_released`. No mismatched-edge leak.
|
||||
- OK `coordinator/dictation.rs:30-51, 69-85` — Hold/Toggle phase matrix correct: `(Toggle, Idle) → begin`, `(Toggle, Listening) → end`, `(Toggle, Starting) → request_stop_during_starting`, `(Hold, Idle) → begin`, `(Hold, Listening released) → end`, `(Hold, Starting released) → request_stop_during_starting`. Other combinations no-op.
|
||||
- OK `coordinator/dictation.rs:87-96` + `coordinator_state.rs:87-93` — `request_stop_during_starting_state` only flips `pending_stop` when phase is exactly `Starting`; `finish_starting_session_state` consumes the bit at the Listening transition (`coordinator_state.rs:118-124`) and triggers immediate `end_session` via `BeginOutcome::PendingStop` at `dictation.rs:587-590`.
|
||||
- OK `coordinator/dictation.rs:451` — `acquire_recording_mute(inner, "dictation").await;` is properly awaited (PR #391 fix applied to dictation path).
|
||||
- ⚠️ `coordinator/dictation.rs:418-447` (level_handler) — runs on the cpal audio callback thread and calls `emit_capsule` synchronously. See P4 for the concrete bug; flagged here because P2's recorder-start path is the producer.
|
||||
|
||||
### P3 — End-of-session pipeline
|
||||
|
||||
- OK `coordinator/dictation.rs:595-602` + `coordinator_state.rs:178-184` — `start_processing_if_listening` only transitions `Listening → Processing`; if phase is anything else (`Idle`, `Starting`, `Inserting`, already-Processing), `end_session` returns Ok(()) immediately. Guards against stale pending_stop or duplicate IPC.
|
||||
- OK `coordinator/dictation.rs:607-620` — recorder + ASR are taken with session-id matching (`take_recorder_for_session` / `take_asr_for_session`), so a stale callback that lands after a session has been re-bumped won't pick up the wrong recorder.
|
||||
- OK `coordinator/dictation.rs:984-1005` — atomic Inserting transition: same `state.lock()` checks `cancelled` and flips phase to `Inserting`. Once `Inserting`, `cancel_session` rejects (`coordinator_state.rs:155-159` — `Idle | Inserting` → `None`). This is the audit HIGH #2 contract.
|
||||
- OK `coordinator/dictation.rs:1007-1031` — `paste_shortcut = prefs.paste_shortcut` flows into `inner.inserter.insert(&polished, restore_clipboard, paste_shortcut)` for non-Windows and into `insert_with_windows_ime_first(..., paste_shortcut, ime_target)` for Windows. PR #377 wiring confirmed; corresponding signature in `coordinator.rs:1673-1680, 1731-1745` and `insertion.rs:43-89`.
|
||||
- OK `coordinator/dictation.rs:1122-1126` — happy-path end_session clears `state.focus_target = None` before scheduling capsule idle.
|
||||
|
||||
- 🚩 `coordinator/dictation.rs:843-849` + `coordinator/dictation.rs:1153-1178` + `coordinator_state.rs:171-176` — **`focus_target` is not cleared when cancel hits during `Processing`.**
|
||||
- `cancel_session` in the `Processing` branch (`dictation.rs:1171-1173`) deliberately does NOT call `finish_cancel_session_state`, leaving phase + focus_target as-is so `end_session` can finish unwinding.
|
||||
- `end_session`'s "ASR-finished, cancelled" exit (`dictation.rs:845-849`) restores Windows IME, sets `phase = Idle`, returns Ok(()) — but never touches `focus_target`.
|
||||
- PR #387 (`ce82fcd9`) was framed as "clear `focus_target` on cancel regardless of phase", but the only code path that gained the unconditional clear is `finish_cancel_session_state` at `coordinator_state.rs:172`, which the Processing branch skips.
|
||||
- Concrete consequence: between cancel-mid-Processing and the next `begin_session`, the cached AX `focus_target` (a stale `usize` slot) is reachable by anyone reading `state.focus_target` (logs, debug dumps, future readers). It's overwritten by `begin_session_state` at `coordinator_state.rs:80`, so user-visible insertion uses the right value. Severity: minor leak / contract violation rather than user-visible breakage. Tests at `coordinator_state.rs:362-385` only validate the cancel happy paths via `finish_cancel_session_state`, so the regression slipped past PR #387's guard test.
|
||||
|
||||
- ⚠️ `coordinator/dictation.rs:1175` — even when cancel fires during `Processing`, the user immediately sees `CapsuleState::Cancelled`, but `end_session` may still be inside the ASR await for several seconds. The phase is still `Processing` until `end_session` reaches the `state.cancelled` check, so a fast retry-press will be quietly dropped (`begin_session_state` requires `Idle`). Not a bug per se — matches the design comment at `dictation.rs:1169-1174` — but worth a UX note.
|
||||
|
||||
- ⚠️ `coordinator/dictation.rs:1153-1163` — `cancel_session` swallows the result of `begin_cancel_session_state` for `Inserting`, only logging "cancel ignored". Acceptable, but there's no UI signal back to the user that their Esc didn't take. Minor.
|
||||
|
||||
### P4 — Capsule UI emission
|
||||
|
||||
- 🚩🚩 **`coordinator.rs:3684-3727`** — **`emit_capsule` does NOT marshal `window.show/hide` to the main thread on `400097ad`.** Verbatim from current source:
|
||||
|
||||
```rust
|
||||
fn emit_capsule(...) {
|
||||
...
|
||||
if let Some(window) = app.get_webview_window("capsule") {
|
||||
...
|
||||
let visible = !matches!(state, CapsuleState::Idle);
|
||||
maybe_position_capsule_bottom_center(inner, &window, payload.translation);
|
||||
if show_capsule && visible {
|
||||
if !show_capsule_window_no_activate(&app, &window) {
|
||||
let _ = window.show();
|
||||
}
|
||||
#[cfg(target_os = "macos")]
|
||||
crate::restore_main_window_key_if_active(&app);
|
||||
} else {
|
||||
hide_capsule_window_if_present();
|
||||
let _ = window.hide();
|
||||
}
|
||||
}
|
||||
let _ = app.emit_to("capsule", "capsule:state", payload);
|
||||
}
|
||||
```
|
||||
|
||||
This is the *pre-PR-#389* shape. PR #389's fix (`faf02ad4`, then `84ee3d96`) wraps the `if let Some(window)` block in `app.run_on_main_thread(move || { ... })`. Verified that neither commit is an ancestor of `400097ad`:
|
||||
|
||||
```
|
||||
$ git merge-base --is-ancestor faf02ad4 400097ad ; echo $? → 1 (NOT ancestor)
|
||||
$ git merge-base --is-ancestor 84ee3d96 400097ad ; echo $? → 1 (NOT ancestor)
|
||||
```
|
||||
|
||||
Reproduction reasoning: `coordinator/dictation.rs:418-447` builds `level_handler` as `Arc<dyn Fn(f32) + Send + Sync>` and hands it to `Recorder::start`. cpal calls it from the audio process callback thread; the handler then calls `emit_capsule(...)` (line 439-446). On macOS, `WebviewWindow::show()` / `hide()` and the `show_capsule_window_no_activate` (which calls `NSWindow.orderFrontRegardless`) hit AppKit assertions (`dispatch_assert_queue_fail` → SIGTRAP) when invoked off the main thread. The 33 ms throttle at `dictation.rs:417, 426-432` only limits frequency — every individual call is at the same thread-safety risk.
|
||||
|
||||
The same risk applies to QA's level_handler at `coordinator.rs:2282-2309`, which also calls `emit_capsule` directly from the cpal callback (line 2301-2308).
|
||||
|
||||
Severity: high. SIGTRAP would crash the app on long recordings; less catastrophic outcomes are stuttering audio (the audio callback misses its deadline waiting for AppKit) and `kAudioUnitErr_TooManyFramesToProcess`. PR #389 needs to land or be cherry-picked before this beta is shipped.
|
||||
|
||||
- OK `coordinator.rs:3726` — `app.emit_to("capsule", "capsule:state", payload)` stays on the calling thread; Tauri's event bus is internally thread-safe. No change needed regardless of PR #389.
|
||||
- OK `coordinator.rs:3739-3765` — `maybe_position_capsule_bottom_center` is the OS-level call inside the `if let Some(window)` block; it would be moved into the same `run_on_main_thread` closure once PR #389 lands.
|
||||
|
||||
### P5 — App shutdown
|
||||
|
||||
- OK `lib.rs:347-355` (RunEvent::Exit) — calls `stop_hotkey_listener` (Mac CGEventTap path, safe to invoke any thread per `hotkey.rs:344-355`), plus the 5 `global-hotkey`-backed `stop_*_listener`s that all marshal via `app.run_on_main_thread` (`coordinator.rs:344-358, 1313-1347`).
|
||||
- OK `lib.rs:348` — `TRAY_MICROPHONE_WATCHER_STOPPING.store(true, Relaxed)` correctly signals the watcher loop spawned at `lib.rs:540-548`.
|
||||
- ⚠️ `coordinator.rs:344-358, 1313-1347` — `app.run_on_main_thread` is fire-and-forget; the queued `Drop` may not run before the process exits. In practice this is fine (process exit reaps everything), but if Tauri's main-loop teardown beats the queued closure, Carbon `RemoveEventHotKey` is skipped. Same model as the pre-existing PR #169 fix for `qa_hotkey`, so flagging as an inherited limitation, not a regression.
|
||||
|
||||
- 🚩 **`coordinator.rs:2313` — `acquire_recording_mute(inner, "qa");` is missing `.await`.**
|
||||
- PR #391 (`6171df61`) made `acquire_recording_mute` `async fn` (`coordinator/resources.rs:122`) and updated the dictation call site to `.await` (`coordinator/dictation.rs:451`). The QA call site was missed.
|
||||
- Effect: the function returns an `impl Future` that is dropped on the next line. `spawn_blocking` is never scheduled, so `mute.holders` doesn't increment, the system audio mute is never engaged for QA, and the `[audio-mute] acquired by qa` log is never written. The corresponding `release_recording_mute(inner, "qa")` calls (e.g. `resources.rs:194`, `coordinator.rs:2324`) decrement holders that were never incremented (early `return` at `resources.rs:174` because `holders == 0`).
|
||||
- Compiler confirms it: `cargo check --manifest-path openless-all/app/src-tauri/Cargo.toml` emits
|
||||
|
||||
```
|
||||
warning: unused implementer of `futures_util::Future` that must be used
|
||||
--> src/coordinator.rs:2313:5
|
||||
2313 | acquire_recording_mute(inner, "qa");
|
||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
= note: futures do nothing unless you `.await` or poll them
|
||||
```
|
||||
|
||||
- User-visible consequence: when the user opted into "Mute system output during recording" and triggers QA via Option, system audio is NOT muted (e.g. a YouTube playback continues). Dictation behaves correctly. Fix: insert `.await` on the call.
|
||||
|
||||
- OK `coordinator/resources.rs:184-188` — `release_recording_mute` falls back to synchronous `work()` when no tokio handle is present, so the recorder error monitor (a plain `std::thread::spawn`) can release safely. Drop of `AudioMuteGuard` shells out to `osascript` / `wpctl`, blocking on a std thread is OK.
|
||||
|
||||
## Cross-PR composition risks
|
||||
|
||||
1. **PR #391 + #389 incomplete merge.** The current `400097ad` contains PR #391 (which added the async hygiene the rest of the audit fixes assume) but is missing PR #389. Net result: the audio-thread → AppKit risk PR #389 fixes is still present *and* the QA codepath has a half-applied PR #391 (missing `.await` at `coordinator.rs:2313`). Both must land before this commit becomes a release candidate.
|
||||
2. **PR #387 + Processing branch interaction.** PR #387 introduced `state.focus_target = None` inside `finish_cancel_session_state`, but `cancel_session` deliberately skips that helper for `Processing` (`dictation.rs:1171-1173`) so the scheduled `end_session` can drive its own teardown. `end_session`'s cancelled-after-ASR exit (`dictation.rs:845-849`) was not updated to clear `focus_target`. The contract "clear focus_target on cancel regardless of phase" is therefore violated for cancel-during-Processing. Fix can be either (a) clear `focus_target` in the cancel-after-ASR branch of `end_session`, or (b) move the clear into `cancel_session` even for Processing (does not interfere with `end_session`'s own writes).
|
||||
3. **PR #390 + multi-bridge `hotkey_trigger_held`.** The dedup atomic is process-global. With both `hotkey` and `combo_hotkey` monitors running, the legacy modifier-only adapter and the custom-combo adapter share `inner.hotkey_trigger_held`. Currently exclusive (custom-combo only runs when trigger == Custom — see `coordinator.rs:413-426`), so no cross-contamination, but anything that allows them to coexist would corrupt the dedup. ℹ️ informational.
|
||||
4. **PR #392 (passive flag) is dormant**, as the prompt indicates. No interaction risk; calling out only that supervisor loops gracefully ignore `shutdown=false` so future RunEvent::Exit hookup is safe to land in a follow-up PR.
|
||||
|
||||
## Manual-verification checklist for the user
|
||||
|
||||
After cherry-picking PR #389 + fixing the missing `.await` in P5, verify on a running build:
|
||||
|
||||
**P4 — capsule main-thread (PR #389 confirmation)**
|
||||
- [ ] macOS arm64 build, dev profile, run a 3-minute continuous toggle dictation. App must not crash with SIGTRAP / `dispatch_assert_queue_fail`. Tail `~/Library/Logs/OpenLess/openless.log` while recording.
|
||||
- [ ] On macOS, capsule still appears once the first PCM frame is captured (50–200 ms after Recorder::start) and disappears 1.5 s after Done/Cancelled/Error.
|
||||
- [ ] On Windows, no SendMessage deadlock against the GUI thread during recording start/stop (capsule transitions complete within ~50 ms of phase changes).
|
||||
|
||||
**P5 — QA mute fix**
|
||||
- [ ] Set `prefs.muteDuringRecording = true` (Settings → Recording).
|
||||
- [ ] Play YouTube in Safari/Chrome.
|
||||
- [ ] Open QA panel via `Cmd+Shift+;`. Press Right Option to start QA recording. Audio playback **must** mute. Log line `[audio-mute] acquired by qa; holders=1` must appear in `openless.log`. (Without the fix, no log line + audio keeps playing.)
|
||||
- [ ] Release Right Option. Audio playback resumes; log line `[audio-mute] released by qa; holders=0` + `system output mute restored after recording` must appear.
|
||||
|
||||
**P3 — focus_target on Processing-cancel (PR #387 completeness)**
|
||||
- [ ] Start dictation, speak briefly, release hotkey to enter Processing. While ASR is awaiting result (within ~1 s window), press Esc to trigger `cancel_dictation`. End_session bails at `cancelled` check.
|
||||
- [ ] Inspect debug logs / state dump (or add a temporary log at `dictation.rs:849`): `state.focus_target` should be `None`. With current `400097ad` it remains `Some(...)`.
|
||||
- [ ] Start a fresh dictation in a different window. Insertion should still target the new window — this works today via `begin_session_state` overwrite (`coordinator_state.rs:80`), so the bug is silent until something else reads stale `focus_target` between the cancel and the next begin.
|
||||
|
||||
**P2 — pressed/released routing (PR #390 confirmation)**
|
||||
- [ ] Open QA panel (`Cmd+Shift+;`). Confirm Option starts QA recording.
|
||||
- [ ] Close QA panel. Start a normal dictation (Option). While dictation is running, open QA panel via `Cmd+Shift+;` (panel becomes visible while dictation_active=true). Press and release Option once — dictation must end normally (insert text), QA must NOT capture this Option edge.
|
||||
- [ ] Hold mode: Set HotkeyMode::Hold. Hold Option for 2 s, release. End_session must trigger on release (not on press).
|
||||
- [ ] Toggle mode: Set HotkeyMode::Toggle. Tap Option twice rapidly during Starting phase (within the 50–200 ms cpal init window) — verify `request_stop_during_starting` queues then end_session fires on the Listening transition (search log for `applying pending_stop edge → end_session immediately`).
|
||||
|
||||
**P1 — listener teardown sanity**
|
||||
- [ ] Quit the app via tray menu. `openless.log` should show ordered `stop_*_listener` calls; no panic / SIGTRAP at exit.
|
||||
- [ ] Relaunch and grant Accessibility (after prior reset). Confirm `[hotkey] CGEventTap 已启动` returns within 3 s; first hotkey press still works without app restart.
|
||||
|
||||
If any of the P4/P5 checks fail, this beta is **not** build-quality.
|
||||
@@ -1,75 +0,0 @@
|
||||
# 划词追问：思考能力（Reasoning）路线图
|
||||
|
||||
> 创建于 2026-05-01。流式输出（v2.1）已完成，**思考能力（v2.2）暂未实施**——这份文档是后续迭代的设计稿。
|
||||
>
|
||||
> 关联：issue #118 v2、PR #119、`openless-all/app/src-tauri/src/polish.rs`、`openless-all/app/src/pages/SelectionAsk.tsx`。
|
||||
|
||||
## 背景与决策
|
||||
|
||||
用户提出："QA 应该让 LLM 进行思考后再回复，并且可以设置思考强度"。
|
||||
|
||||
讨论了 3 条路径：
|
||||
|
||||
| 方案 | 实现 | 优 | 劣 |
|
||||
|---|---|---|---|
|
||||
| A | prompt-engineered（system prompt 加 `<thinking>` 块要求） | 0 配置改动；现 model 即可 | 思考质量受小模型限制；不可控 |
|
||||
| B | OpenAI 标准 `reasoning_effort: low/medium/high` 字段 | 标准化 | DeepSeek-v4-flash 不识别该字段 |
|
||||
| **C** | **切换 reasoner 模型（deepseek-r1 / o1 / claude extended thinking）** | **真**思考；可视化推理过程 | 用户得多配一个 model；UI 复杂度高 |
|
||||
|
||||
**结论**：选 C。A/B 在当前 provider 下等于无效。
|
||||
|
||||
## 实施分解
|
||||
|
||||
### 后端
|
||||
|
||||
1. **凭据存储**：`CredentialAccount` 加两条
|
||||
- `ArkReasonerModelId`（如 `deepseek-r1`、`doubao-seed-1.6-thinking`）
|
||||
- 复用现有 `ArkApiKey` / `ArkEndpoint`（同一 provider 不同 model）
|
||||
|
||||
2. **prefs**：`Preferences` 加字段
|
||||
- `qa_reasoning_effort: ReasoningEffort` 枚举 `Off | Low | Medium | High`
|
||||
- 默认 `Off`（与现行为一致）
|
||||
|
||||
3. **`answer_chat_streaming` 重载**：根据 effort 决定走 chat 还是 reasoner endpoint
|
||||
- `Off`：走 v2.1 现路径（chat 模型 + stream）
|
||||
- `Low/Medium/High`：走 reasoner 模型；强度通过 system prompt hint 调（"简短思考即可" / "详细思考" / "深度推理多角度"）
|
||||
- SSE 解析时同时收 `delta.content` + `delta.reasoning_content`，两者通过不同事件 emit：
|
||||
- `qa:state {kind:"reasoning_delta", chunk}`
|
||||
- `qa:state {kind:"answer_delta", chunk}` （已存在）
|
||||
|
||||
4. **answer_chat 拼装最终 message** 时，`reasoning_content` 不写入 `messages` 数组（只显示用，不进上下文）。多轮提问只把最终答案带回上下文。
|
||||
|
||||
### 前端
|
||||
|
||||
1. **SelectionAsk.tsx** 新增配置块：
|
||||
- 「思考强度」下拉：关闭 / 浅 / 中 / 深
|
||||
- 「思考模型」输入框（model id；默认 `deepseek-r1`）
|
||||
- i18n：zh-CN / en
|
||||
|
||||
2. **QaPanel.tsx** 新增「思考过程」可折叠区块：
|
||||
- 在 user 气泡下方、最终 assistant 气泡上方
|
||||
- 默认折叠，标题 `思考中…` / `思考过程（X 字）`，点击展开
|
||||
- 流式期间：实时拼接 `reasoning_delta`，气泡有打字 caret
|
||||
- 答案完成：折叠收起；用户随时可点开看推理
|
||||
|
||||
3. **types.ts**：`QaStateKind` 加 `'reasoning_delta'`；payload 加 `reasoning_chunk?: string`
|
||||
|
||||
### 边界与风险
|
||||
|
||||
- **Provider 兼容性**：火山 Ark 的 deepseek-r1 / doubao-thinking 都返回 `reasoning_content`；OpenAI o1 通过 thinking blocks（不是 reasoning_content），需要单独 adapter
|
||||
- **Token 成本**：reasoner 模型 token 价格高 5-10x；用户开「深度」就是真烧钱，UI 应该有提示
|
||||
- **延迟**：reasoner 首 token 可能 > 5s（思考阶段无 content 输出）。要在 UI 上区分「思考中」（reasoning streaming）vs「答题中」（content streaming），避免用户以为卡了
|
||||
|
||||
## 工作量估算
|
||||
|
||||
- 后端 reasoner 通路 + SSE 双流解析：~2h
|
||||
- 前端折叠思考区块 + 打字 caret + 状态切换：~1.5h
|
||||
- prefs / SelectionAsk 配置 UI + i18n：~0.5h
|
||||
- 端到端测试（三档强度 × 单/多轮 × 错误回退）：~1h
|
||||
- **总计**：~5h
|
||||
|
||||
## 实施先决条件
|
||||
|
||||
1. 用户配置好一个 reasoner model（deepseek-r1 / doubao-thinking-pro 等）
|
||||
2. 后端凭据 vault 写入对应 model id
|
||||
3. v2.1 流式输出已稳定（已完成 ✅）
|
||||
@@ -1,299 +0,0 @@
|
||||
# Style Pack Marketplace — 规划文档
|
||||
|
||||
**状态**：规划中（API 已预留 stub，未实装）
|
||||
**起草日期**：2026-05-14
|
||||
**owner**：待定
|
||||
|
||||
## 1. 目标
|
||||
|
||||
把现在「ZIP 包本地导入 / 导出」的体验扩展成一个公开的风格包市场：
|
||||
|
||||
- 用户可以把自己调好的风格包**上传**到云端，附带名称、描述、作者署名、标签、效果示例
|
||||
- 其他用户可以**浏览 / 搜索 / 下载**别人的风格包，一键安装到本地
|
||||
- 后期支持**版本升级提醒**、**收藏 / 评分**等基础社交属性
|
||||
|
||||
非目标（v1 不做）：
|
||||
- 付费 / 抽成
|
||||
- 风格包内嵌外部 prompt 注入 / 跨域 fetch（安全考虑，风格包始终是纯文本 prompt）
|
||||
- 多人协作编辑 / fork
|
||||
|
||||
## 2. 架构概览
|
||||
|
||||
```
|
||||
┌──────────────────┐ HTTPS ┌─────────────────────┐
|
||||
│ OpenLess client │ ◄──────────────────► │ marketplace API │
|
||||
│ (Tauri 2) │ JSON over TLS │ (TBD: Cloudflare │
|
||||
│ │ │ Workers / D1 / │
|
||||
│ Rust IPC → │ │ R2 for blobs) │
|
||||
│ reqwest client │ │ │
|
||||
└──────────────────┘ └─────────────────────┘
|
||||
│ │
|
||||
│ local cache (~/Library/Application │
|
||||
│ Support/OpenLess/market_cache/) │
|
||||
▼ ▼
|
||||
StylePackStore Postgres / D1
|
||||
(existing local listings + R2 blobs
|
||||
persistence layer)
|
||||
```
|
||||
|
||||
**关键约束**：
|
||||
- 客户端只能上传 / 下载 ZIP **bundle**（不直接传 JSON），保持跟现有 ZIP import/export 同构
|
||||
- 服务端 ZIP 验证：解压后必须能反序列化成 `StylePack`、`prompt.chars().count() <= 50_000`、没有可执行附件
|
||||
- 风格包 ID 上传后由服务端分配（`{author_slug}-{name_slug}-{version}`），跟本地 ID 解耦
|
||||
- 客户端始终拿 ZIP 走现有 `import_style_pack_from_zip` 路径入库 —— 不另开一条「从市场直接写 Pack」的代码路径，避免双入口
|
||||
|
||||
## 3. HTTP API 规约
|
||||
|
||||
Base URL（待定）：`https://api.openless.app/v1/marketplace/`
|
||||
|
||||
所有响应统一信封：
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"data": <T> | null,
|
||||
"error": null | { "code": "ERR_XXX", "message": "..." }
|
||||
}
|
||||
```
|
||||
|
||||
### 3.1 GET `/packs` — 列表 / 搜索
|
||||
|
||||
Query：
|
||||
| 参数 | 类型 | 默认 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `q` | string | `""` | 关键词（名称 / 描述 / 标签） |
|
||||
| `tag` | string | `""` | 单标签筛选 |
|
||||
| `sort` | `recent` \| `popular` \| `name` | `recent` | 排序 |
|
||||
| `cursor` | string | `null` | 分页游标 |
|
||||
| `limit` | int (1-100) | `20` | 每页条数 |
|
||||
|
||||
Response data：
|
||||
```typescript
|
||||
{
|
||||
packs: MarketPackListing[];
|
||||
next_cursor: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
`MarketPackListing`：
|
||||
```typescript
|
||||
{
|
||||
id: string; // server-assigned, e.g. "alice-formal-v2.1"
|
||||
name: string;
|
||||
description: string;
|
||||
author: string;
|
||||
version: string; // semver
|
||||
tags: string[];
|
||||
base_mode: "raw" | "light" | "structured" | "professional";
|
||||
recommended_model: string | null;
|
||||
compatible_app_version: string | null;
|
||||
downloads: number;
|
||||
rating_avg: number | null;
|
||||
rating_count: number;
|
||||
updated_at: string; // ISO8601
|
||||
zip_size_bytes: number;
|
||||
zip_sha256: string; // 客户端下载后校验
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 GET `/packs/{id}` — 详情
|
||||
|
||||
Response data：`MarketPackListing` + 额外字段：
|
||||
```typescript
|
||||
{
|
||||
...listing,
|
||||
examples: StylePackExample[]; // 解压 ZIP 前的预览
|
||||
changelog: string | null;
|
||||
homepage_url: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 GET `/packs/{id}/download` — 下载 ZIP
|
||||
|
||||
Response：`application/zip` 二进制流，带 `X-Pack-SHA256` header 用于校验。
|
||||
|
||||
服务端通过 redirect 直接指向 R2 / S3 预签 URL，避免代理流量。
|
||||
|
||||
### 3.4 POST `/packs` — 上传（需鉴权）
|
||||
|
||||
Headers：`Authorization: Bearer <api_key>`
|
||||
Body：`multipart/form-data` with field `pack=@xxx.zip`
|
||||
|
||||
Response data：`MarketPackListing`（含新分配 id）
|
||||
|
||||
错误码：
|
||||
- `ERR_INVALID_ZIP` — ZIP 解压失败 / 不是合法 StylePack JSON
|
||||
- `ERR_PROMPT_TOO_LARGE` — prompt 字数超 50k
|
||||
- `ERR_DUPLICATE_VERSION` — 同 author+name+version 已存在
|
||||
- `ERR_RATE_LIMITED` — 触发限频
|
||||
|
||||
### 3.5 DELETE `/packs/{id}` — 撤回（需鉴权 + 必须是上传者）
|
||||
|
||||
### 3.6 POST `/packs/{id}/rate` — 评分（需鉴权）
|
||||
|
||||
Body：`{ score: 1..5, comment?: string }`
|
||||
|
||||
## 4. IPC 契约（Rust ↔ TS）
|
||||
|
||||
在 `src-tauri/src/commands.rs` 新增以下 stub（暂返回 `Err("not implemented yet")`，等服务端落地后实装）：
|
||||
|
||||
```rust
|
||||
// 列表 / 搜索
|
||||
#[tauri::command]
|
||||
pub async fn market_list_packs(
|
||||
query: Option<String>,
|
||||
tag: Option<String>,
|
||||
sort: Option<String>,
|
||||
cursor: Option<String>,
|
||||
limit: Option<u32>,
|
||||
) -> Result<MarketListResponse, String>;
|
||||
|
||||
// 详情
|
||||
#[tauri::command]
|
||||
pub async fn market_get_pack(id: String) -> Result<MarketPackDetail, String>;
|
||||
|
||||
// 下载 + 自动调用现有的 import_style_pack_from_zip 入库
|
||||
#[tauri::command]
|
||||
pub async fn market_download_pack(
|
||||
coord: CoordinatorState<'_>,
|
||||
app: AppHandle,
|
||||
id: String,
|
||||
) -> Result<StylePack, String>;
|
||||
|
||||
// 上传（dirty 字段 = 已编辑、未保存）
|
||||
#[tauri::command]
|
||||
pub async fn market_upload_pack(
|
||||
coord: CoordinatorState<'_>,
|
||||
pack_id: String,
|
||||
api_key: String,
|
||||
) -> Result<MarketPackListing, String>;
|
||||
|
||||
// 撤回
|
||||
#[tauri::command]
|
||||
pub async fn market_delete_pack(id: String, api_key: String) -> Result<(), String>;
|
||||
|
||||
// 评分
|
||||
#[tauri::command]
|
||||
pub async fn market_rate_pack(
|
||||
id: String,
|
||||
api_key: String,
|
||||
score: u8,
|
||||
comment: Option<String>,
|
||||
) -> Result<(), String>;
|
||||
```
|
||||
|
||||
DTO（在 `types.rs` 新增）：
|
||||
```rust
|
||||
#[derive(Debug, Serialize, Deserialize, Clone)]
|
||||
pub struct MarketPackListing {
|
||||
pub id: String,
|
||||
pub name: String,
|
||||
pub description: String,
|
||||
pub author: String,
|
||||
pub version: String,
|
||||
pub tags: Vec<String>,
|
||||
pub base_mode: PolishMode,
|
||||
pub recommended_model: Option<String>,
|
||||
pub compatible_app_version: Option<String>,
|
||||
pub downloads: u64,
|
||||
pub rating_avg: Option<f32>,
|
||||
pub rating_count: u32,
|
||||
pub updated_at: String,
|
||||
pub zip_size_bytes: u64,
|
||||
pub zip_sha256: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize, Clone)]
|
||||
pub struct MarketPackDetail {
|
||||
#[serde(flatten)]
|
||||
pub listing: MarketPackListing,
|
||||
pub examples: Vec<StylePackExample>,
|
||||
pub changelog: Option<String>,
|
||||
pub homepage_url: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize, Clone)]
|
||||
pub struct MarketListResponse {
|
||||
pub packs: Vec<MarketPackListing>,
|
||||
pub next_cursor: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
TS wrappers（`src/lib/ipc.ts`）：
|
||||
```typescript
|
||||
export interface MarketPackListing { /* same shape */ }
|
||||
export interface MarketPackDetail extends MarketPackListing { /* + examples, changelog, homepage_url */ }
|
||||
export interface MarketListResponse { packs: MarketPackListing[]; next_cursor: string | null; }
|
||||
|
||||
export function marketListPacks(opts: {
|
||||
query?: string; tag?: string; sort?: 'recent' | 'popular' | 'name';
|
||||
cursor?: string; limit?: number;
|
||||
}): Promise<MarketListResponse>;
|
||||
export function marketGetPack(id: string): Promise<MarketPackDetail>;
|
||||
export function marketDownloadPack(id: string): Promise<StylePack>;
|
||||
export function marketUploadPack(packId: string, apiKey: string): Promise<MarketPackListing>;
|
||||
export function marketDeletePack(id: string, apiKey: string): Promise<void>;
|
||||
export function marketRatePack(id: string, apiKey: string, score: number, comment?: string): Promise<void>;
|
||||
```
|
||||
|
||||
## 5. 鉴权模型
|
||||
|
||||
**v1 简化方案**：
|
||||
- 用户在设置页输入个人 API key（服务端发放）
|
||||
- API key 存到 OS Keychain，账户名 `com.openless.app.market_api_key`
|
||||
- 客户端在 Header 加 `Authorization: Bearer <key>`
|
||||
- 服务端校验 + 限频（每小时 60 次写、600 次读）
|
||||
|
||||
**v2 升级路径**（暂不做）：
|
||||
- OAuth via GitHub / Google
|
||||
- 上传时自动签名 ZIP，下载端校验签名
|
||||
|
||||
## 6. 缓存与版本检查
|
||||
|
||||
本地缓存目录：`<app_data>/market_cache/`
|
||||
- `listings.json` — 上次拉的 listings（带 ETag）
|
||||
- `packs/{id}.zip` — 已下载的 ZIP（按需保留，30 天自动清理）
|
||||
|
||||
版本升级提示：
|
||||
- 启动时（带 dev-cap 24h 节流）调用 `/packs?ids=<已安装的 market_id...>` 拉对比
|
||||
- 本地包记录 `installed_market_id` 和 `installed_market_version` 字段，新建 `StylePack` 时填，本地从 ZIP 安装也填
|
||||
- 发现新版本 → 在 Style 页该包卡片角标显示 `New version: 2.3.0 →`
|
||||
|
||||
## 7. 客户端 UI 入口（v1 不做，先留位）
|
||||
|
||||
- Style 页头部加一个 tab：`本地 / 市场`
|
||||
- 市场页：搜索栏 + tag 过滤 + 卡片列表 + 详情抽屉
|
||||
- 上传：编辑某个本地包时，"导出 ZIP" 按钮旁边出现 "上传到市场"（需要先在设置里填 API key）
|
||||
|
||||
## 8. 安全 / 滥用对策
|
||||
|
||||
- ZIP 解压走 streaming，限制最大解压后大小 5 MB
|
||||
- prompt 字段过滤明显的 prompt injection / 越狱（关键词预扫描 + 异步内容审核）
|
||||
- 每用户每天上传上限 10 包，单包大小 ≤ 2 MB
|
||||
- 上传后挂 24h 公开延迟（防恶意刷榜）
|
||||
|
||||
## 9. 实装 TODO（按优先级）
|
||||
|
||||
- [ ] 服务端选型（CF Workers + D1 + R2 vs Supabase vs 自托管 FastAPI）
|
||||
- [ ] 服务端实装 + 部署环境（dev / staging / prod）
|
||||
- [ ] 客户端 `types.rs` 加 DTO
|
||||
- [ ] `commands.rs` 加 6 个 stub（**已完成**，返回 `not implemented yet`）
|
||||
- [ ] `lib/ipc.ts` 加 wrapper（**已完成**）
|
||||
- [ ] 实装 `market_download_pack`（先做单条路径打通：URL → 下载 → 走现有 import_style_pack_from_zip）
|
||||
- [ ] 加凭据存储（Keychain 复用现有 `CredentialsVault`）
|
||||
- [ ] UI：本地 / 市场 tab
|
||||
- [ ] UI：搜索 + 卡片
|
||||
- [ ] UI：详情面板
|
||||
- [ ] UI：上传流程
|
||||
- [ ] 升级提醒 badge
|
||||
- [ ] 缓存清理 + ETag
|
||||
|
||||
## 10. 决策 / 风险记录
|
||||
|
||||
| 项 | 决策 | Why |
|
||||
|---|---|---|
|
||||
| ZIP 而非 JSON 上传 | 用 ZIP | 跟现有 import/export 同构；prompt 长文 + examples 用 ZIP 包压缩 |
|
||||
| 服务端分配 ID | 是 | 防本地 ID 碰撞、用户重命名包不影响订阅 |
|
||||
| 上传立刻可见 vs 审核 | 24h 公开延迟 | 防刷榜 + 给审核留空间 |
|
||||
| API key vs OAuth | 先 API key | 简化 v1；登录态可 v2 升级 |
|
||||
| 客户端缓存策略 | listings ETag + 已下载 ZIP 30 天 | 平衡流量和体验 |
|
||||
| 国际化 / 跨境 | API 全英文 + 客户端 i18n | 服务端不存翻译，名称/描述支持任意 UTF-8 |
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,143 +0,0 @@
|
||||
# Windows 临时激活式 TSF IME 设计
|
||||
|
||||
## 背景
|
||||
|
||||
OpenLess 当前 Windows 插入链路仍依赖剪贴板：先把最终文本写入剪贴板，再向焦点控件发送 `WM_PASTE`。这比模拟 `Ctrl+V` 更稳，但本质仍是粘贴工具，不是 Windows 输入法。
|
||||
|
||||
目标是在 Windows 上新增真正的 TSF 输入法后端，让语音结果通过系统文本输入框架提交，同时不破坏用户平时使用微软拼音、搜狗或英文键盘的手动输入体验。
|
||||
|
||||
## 目标
|
||||
|
||||
- OpenLess 在语音会话期间临时切换到 OpenLess TSF 输入法。
|
||||
- 录音、ASR、润色、胶囊 UI、历史保存继续由现有 Tauri/Rust 主程序负责。
|
||||
- OpenLess TSF IME DLL 只负责系统输入法身份、接收最终文本、通过 TSF 提交到当前文本上下文。
|
||||
- 提交、取消或失败后自动恢复会话开始前的输入法 profile。
|
||||
- 不要求用户手动切换输入法。
|
||||
- 不把第三方中文输入法代理进 OpenLess IME；用户平时中文手打仍使用原输入法。
|
||||
|
||||
## 非目标
|
||||
|
||||
- 不把录音、网络请求、ASR、LLM、Tauri UI 放进 IME DLL。
|
||||
- 不实现拼音候选、中文转换、词库或第三方 IME 代理。
|
||||
- 不移除现有 Windows `WM_PASTE` 路径；它保留为未安装 TSF IME、切换失败或提交失败时的回退路径。
|
||||
- 不承诺 UAC 安全桌面、管理员权限目标窗口、游戏、远程桌面或强隔离应用中的完整可用性。
|
||||
|
||||
## 架构
|
||||
|
||||
新增 Windows-only 输入层由三个部分组成：
|
||||
|
||||
1. `OpenLess` 主程序：沿用现有 `Coordinator` 状态机。语音热键开始时记录当前输入 profile 并临时激活 OpenLess TSF profile；语音结束后把最终文本发送给 IME；会话收尾时恢复原 profile。
|
||||
2. `OpenLess TSF IME DLL`：COM in-proc text service，注册为 TSF input processor。它实现最小可用的激活、停用、编辑会话和文本提交能力，不持有产品业务状态。
|
||||
3. `OpenLess IME IPC`：本机 IPC 通道，连接主程序和当前被 TSF 加载的 IME 实例。主程序发送带 session id 的最终文本；IME 在可写 TSF context 中调用 `ITfInsertAtSelection::InsertTextAtSelection`。
|
||||
|
||||
TSF IME 使用官方 profile 注册路径，而不是手写默认输入法注册表项。安装阶段注册 COM in-proc server、TSF text service、language profile，并把 OpenLess profile 加入当前用户可用输入法列表。
|
||||
|
||||
## 会话时序
|
||||
|
||||
1. 用户按下当前 OpenLess 全局热键。
|
||||
2. `Coordinator` 从 `Idle` 进入录音启动流程。
|
||||
3. Windows 输入 profile 守护逻辑读取并保存当前活动 profile，包括键盘布局或 TSF input processor。
|
||||
4. 守护逻辑激活 OpenLess TSF profile，范围优先使用当前桌面 session。
|
||||
5. 用户说话，现有 recorder、ASR、polish 流程不变。
|
||||
6. 用户再次按热键结束录音；`Coordinator` 获得最终 polished text。
|
||||
7. 主程序通过 IPC 向 OpenLess IME 发送 `{ session_id, text }`。
|
||||
8. 当前焦点应用中的 OpenLess IME 实例在 TSF edit session 中提交文本。
|
||||
9. 主程序收到提交成功、超时或失败结果。
|
||||
10. 无论成功、取消还是失败，守护逻辑都尝试恢复第 3 步保存的输入 profile。
|
||||
11. `Coordinator` 按现有规则保存历史并更新胶囊状态。
|
||||
|
||||
## Profile 切换策略
|
||||
|
||||
会话开始时记录完整 active profile，而不是只记录语言 ID。记录内容至少包括：
|
||||
|
||||
- profile type：keyboard layout 或 TSF input processor；
|
||||
- language id；
|
||||
- text service CLSID；
|
||||
- profile GUID；
|
||||
- HKL；
|
||||
- 激活范围。
|
||||
|
||||
激活 OpenLess profile 时使用 TSF profile manager。若当前输入语言与 OpenLess profile 不一致，使用允许切换到指定 profile 的标志，避免因语言不匹配导致激活失败。
|
||||
|
||||
恢复时优先恢复原始 profile。若原始 profile 不再可用，记录 warning 并保持系统当前输入法，不再反复切换。恢复失败不阻塞历史保存。
|
||||
|
||||
## IPC 协议
|
||||
|
||||
MVP 使用本机低延迟 IPC，协议保持小而明确：
|
||||
|
||||
- `SubmitText { session_id, text, created_at }`
|
||||
- `SubmitResult { session_id, status, error_code }`
|
||||
- `CancelSession { session_id }`
|
||||
- `Ping`
|
||||
|
||||
`session_id` 必须由现有 `DictationSession` 或 coordinator 会话生成，IME 只接受当前最新待提交 session，避免过期文本在焦点变化后落入错误应用。
|
||||
|
||||
IPC 超时策略：
|
||||
|
||||
- 等待 IME 连接：短超时，失败后走现有 `WM_PASTE` 回退。
|
||||
- 等待提交结果：短超时，失败后恢复原 profile 并走回退或报 `CopiedFallback`。
|
||||
- 会话取消：发送 `CancelSession`，IME 丢弃待提交文本。
|
||||
|
||||
## 失败与恢复
|
||||
|
||||
必须把“用户文字不丢失”作为约束：
|
||||
|
||||
- OpenLess profile 激活失败：不进入 TSF 提交流程，继续使用现有 Windows 插入后端。
|
||||
- IME DLL 未安装或未注册：设置页显示状态，语音输入仍可用但使用回退后端。
|
||||
- IPC 断开或超时：恢复原 profile，并使用现有 `WM_PASTE` 路径。
|
||||
- TSF 提交返回只读、无 selection、context disconnected 或 no lock：恢复原 profile，并使用现有回退路径。
|
||||
- 用户在 Processing 阶段取消：不提交文本，恢复原 profile。
|
||||
- OpenLess 主程序崩溃：下次启动检查是否存在“上次会话临时切换未恢复”标记；若存在，尝试恢复最近保存的 profile。
|
||||
|
||||
## 用户体验
|
||||
|
||||
平时用户继续使用原输入法。只有语音会话期间，系统输入指示器可能短暂切到 OpenLess。会话结束后自动回到原输入法。
|
||||
|
||||
设置页新增 Windows-only 输入后端状态：
|
||||
|
||||
- TSF 输入法已安装并可用；
|
||||
- TSF 输入法未安装；
|
||||
- TSF 输入法注册异常；
|
||||
- 当前使用剪贴板/`WM_PASTE` 回退。
|
||||
|
||||
默认行为保持保守：未安装 TSF IME 时，不改变现有插入体验。安装 TSF IME 后，Windows 平台优先使用临时激活式 TSF 后端。
|
||||
|
||||
## 文件与模块边界
|
||||
|
||||
计划新增或调整的主要区域：
|
||||
|
||||
- `openless-all/app/src-tauri/src/insertion.rs`：保留现有回退后端，新增 Windows TSF 后端选择入口。
|
||||
- `openless-all/app/src-tauri/src/windows_ime_profile.rs`：封装 active profile 读取、OpenLess profile 激活、原 profile 恢复。
|
||||
- `openless-all/app/src-tauri/src/windows_ime_ipc.rs`：封装主程序到 IME 的 IPC。
|
||||
- `openless-all/app/windows-ime/`：新增 Windows-only TSF IME DLL 工程，包含 COM 注册、TSF text service、edit session、IPC 客户端。
|
||||
- `openless-all/app/scripts/`：新增 Windows IME 注册、注销、打包脚本。
|
||||
- `openless-all/app/src/lib/ipc.ts` 与设置页：暴露 Windows TSF 后端安装/健康状态。
|
||||
|
||||
Rust 业务模块仍遵守现有约束：叶子模块不互相调用；跨模块编排继续放在 `coordinator.rs`。
|
||||
|
||||
## 验证
|
||||
|
||||
自动验证：
|
||||
|
||||
- Rust backend type check：`cargo check --manifest-path openless-all/app/src-tauri/Cargo.toml`
|
||||
- Windows IME 工程 build。
|
||||
- profile 记录/恢复逻辑单元测试。
|
||||
- IPC 协议编解码和过期 session 丢弃测试。
|
||||
- 前端构建：`npm run build`
|
||||
|
||||
手动验证：
|
||||
|
||||
- Notepad：微软拼音为当前输入法，按 OpenLess 热键录音，提交后文本进入光标位置，并自动回到微软拼音。
|
||||
- 浏览器文本框：同上。
|
||||
- VS Code 编辑器：同上。
|
||||
- 取消录音：不插入文本，并恢复原输入法。
|
||||
- 未安装 OpenLess IME：语音输入仍走现有回退路径。
|
||||
- 目标窗口不可写：不丢文本，恢复原输入法，并给出可理解状态。
|
||||
|
||||
## 参考
|
||||
|
||||
- Microsoft Learn: Custom Input Method Editor requirements
|
||||
- Microsoft Learn: Text Services Framework
|
||||
- Microsoft Learn: Text Service Registration
|
||||
- Microsoft Learn: `ITfInputProcessorProfileMgr::ActivateProfile`
|
||||
- Microsoft Learn: `ITfInsertAtSelection::InsertTextAtSelection`
|
||||
@@ -1,247 +0,0 @@
|
||||
# Windows 本地 ASR 设计
|
||||
|
||||
## 背景
|
||||
|
||||
OpenLess 的产品契约是：全局热键启动听写，胶囊显示录音状态，ASR 产出 transcript，现有 LLM provider 做润色、翻译或语义处理，再通过当前平台插入链路写回光标位置并保存历史。
|
||||
|
||||
Windows 新用户目前仍需要配置外部 ASR provider，才能完成真实听写。目标是在 Windows 上提供一个不依赖外部 ASR API Key 的本地识别方案，同时不调用 `Win+H`，不显示 Windows Voice Typing 系统面板，不绕开现有 polish、insert 和 history 流水线。
|
||||
|
||||
已确认的边界：
|
||||
|
||||
- Windows `Win+H` / Voice Typing 是用户级系统功能，没有适合 OpenLess 嵌入并拿回 transcript 的公开 API。
|
||||
- `SendInput` 模拟 `Win+H` 只会打开系统面板，OpenLess 拿不到 transcript，也无法 polish 或写 history。
|
||||
- `Windows.Media.SpeechRecognition` 对普通 desktop app 的支持和授权路径不适合作为主线。
|
||||
- SAPI COM 可做 desktop dictation，但质量和现代体验不足以满足高品质目标。
|
||||
|
||||
## 官方资料核对
|
||||
|
||||
核对时间：2026-05-06。
|
||||
|
||||
Microsoft Learn 当前资料显示：
|
||||
|
||||
- Foundry Local 是本地 AI runtime，支持 Windows、macOS Apple silicon 和 Linux，提供 C#、JavaScript、Rust、Python SDK；本地推理数据不离开设备，首次模型和执行 provider 下载仍需要网络。
|
||||
- Foundry Local catalog 覆盖 chat completion 和 audio transcription；音频转写示例明确使用 Whisper 模型。
|
||||
- Rust SDK 在 Windows 上使用 `foundry-local-sdk --features winml`，Windows 包集成 Windows ML runtime。
|
||||
- Rust native audio API 当前文档示例是：下载并 load Whisper 模型后 `model.create_audio_client()`，再调用 `audio_client.transcribe(file_path).await`。
|
||||
- Foundry Local 也能启动 OpenAI-compatible local REST service；REST endpoint `POST /v1/audio/transcriptions` 接收 multipart `file`、`model`，可选 `language`、`temperature`、`response_format`，返回 `text`。
|
||||
- REST service 的端口是动态分配，文档要求通过 SDK 暴露的 endpoint / urls 获取，不要硬编码。
|
||||
- CLI 是开发和管理辅助工具，不是应用集成主线；生产应用应使用 SDK 嵌入 runtime。
|
||||
- Foundry Local 仍是 preview，API、安装和分发方式可能变动。
|
||||
|
||||
主要来源：
|
||||
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/what-is-foundry-local
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/get-started
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/how-to/how-to-transcribe-audio
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/reference/reference-rest
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/reference/reference-sdk-current
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/how-to/how-to-use-foundry-local-cli
|
||||
- https://learn.microsoft.com/en-us/azure/foundry-local/concepts/foundry-local-architecture
|
||||
|
||||
## 目标
|
||||
|
||||
- Windows 新用户无需 Volcengine、Whisper HTTP、DashScope 等外部 ASR API Key，即可完成听写。
|
||||
- 不调用 `Win+H`，用户完全看不到 Windows Voice Typing 弹窗。
|
||||
- 现有交互不变：热键、OpenLess capsule、录音状态、转写、LLM polish / 翻译、插入、历史保存都走当前主流水线。
|
||||
- LLM polish 仍沿用用户配置的 OpenAI-compatible LLM provider；LLM 未配置或失败时插入原始 transcript。
|
||||
- 本地 ASR 缺 runtime / 模型时给出可操作引导，而不是静默失败。
|
||||
- 下载完成后可离线识别；首次模型 / execution provider 下载可以联网。
|
||||
|
||||
## 非目标
|
||||
|
||||
- 不把 Windows Voice Typing、SAPI 或系统听写面板嵌入 OpenLess。
|
||||
- 不在本阶段把 LLM polish 也改成本地模型；本设计只解决 ASR。
|
||||
- 不把大型模型直接打进默认 Windows 安装包，除非后续逐项确认模型 license、再分发条款、安装包体积和 updater 影响。
|
||||
- 不重写 Windows TSF IME 插入链路。
|
||||
- 不保证所有隔离目标窗口都能 TSF 上屏；现有 TSF / Unicode / clipboard fallback 策略继续负责插入可用性。
|
||||
|
||||
## 现有系统切入点
|
||||
|
||||
主听写状态机集中在 `openless-all/app/src-tauri/src/coordinator.rs`：
|
||||
|
||||
- `ActiveAsr` 当前有 `Volcengine`、`Whisper`，以及 macOS-only `Local`。
|
||||
- `begin_session` 从 `CredentialsVault::get_active_asr()` 读取 active provider，再分流到 local Qwen3、OpenAI-compatible Whisper 或 Volcengine。
|
||||
- `end_session` 统一取得 `RawTranscript` 后，继续走 `polish_or_passthrough` / `translate_or_passthrough`、Windows TSF-first 插入和 history append。
|
||||
- `ensure_asr_credentials` 是录音前的 provider gate；本地 ASR 需要在这里改成“无需云凭据，但需要 runtime / model ready”。
|
||||
- `is_whisper_compatible_provider` 只覆盖云端 OpenAI-compatible `/audio/transcriptions` provider；Foundry Local 不应塞进这里，因为它需要 runtime / model lifecycle。
|
||||
|
||||
现有本地 ASR 模块在 `openless-all/app/src-tauri/src/asr/local/`：
|
||||
|
||||
- provider id 是 `local-qwen3`，模型枚举是 `qwen3-asr-0.6b` / `qwen3-asr-1.7b`。
|
||||
- `LocalAsrCache` 目前只在 macOS 持有 `QwenAsrEngine`。
|
||||
- 下载页和 IPC 命令已覆盖 model status、下载、删除、test、preload、release，但 UI 文案和目录语义都强绑定 Qwen3-ASR。
|
||||
- Windows 端 `engine_available` 当前为 false，设置页提示“仅 macOS 已支持”。
|
||||
|
||||
Windows 插入链路已经满足本需求：
|
||||
|
||||
- 会话开始时 `prepare_session()` 捕获当前输入法 profile 并临时激活 OpenLess TSF。
|
||||
- 会话结束时 `insert_with_windows_ime_first()` 通过 named pipe 把最终文本提交给 TSF DLL。
|
||||
- TSF DLL 在目标应用内调用 `ITfInsertAtSelection::InsertTextAtSelection`。
|
||||
- TSF 失败后按用户偏好走 Unicode `SendInput` 或 clipboard fallback。
|
||||
|
||||
## 推荐方案
|
||||
|
||||
新增 Windows-only provider：`foundry-local-whisper`。
|
||||
|
||||
实现上分两层：
|
||||
|
||||
1. `FoundryLocalWhisperAsr`：形状对齐 `WhisperBatchASR` 和 `LocalQwenAsr`，实现 `AudioConsumer`，录音阶段 buffer 16 kHz mono i16 PCM，stop 后编码 WAV 并调用 Foundry Local。
|
||||
2. `FoundryLocalRuntime`：封装 Foundry Local SDK 的初始化、catalog 查询、execution provider 下载、模型下载、模型加载、endpoint 获取和卸载 / keep-loaded 管理。
|
||||
|
||||
MVP 调用路径建议先用 Foundry Local SDK 启动 local REST service，再调用 `/v1/audio/transcriptions`。原因：
|
||||
|
||||
- OpenLess 已经有成熟的 multipart WAV 转写路径。
|
||||
- REST API 文档明确支持 `language` 参数，便于后续中文 / 中英混输策略调优。
|
||||
- SDK 仍负责动态端口、模型下载和加载，避免硬编码本地服务地址。
|
||||
- 后续如果 Rust native audio client 提供足够参数和稳定 API，可以把 REST 调用替换为纯 native audio client。
|
||||
|
||||
## Provider 与模型命名
|
||||
|
||||
新增 id：
|
||||
|
||||
- `foundry-local-whisper`：Windows 主线本地 ASR。
|
||||
|
||||
模型别名：
|
||||
|
||||
- 默认：`whisper-small`。
|
||||
- 低配选项：`whisper-base`。
|
||||
- 调试选项：`whisper-tiny`。
|
||||
|
||||
默认不强制 `language=zh`。中英混输时让 Whisper 自动检测更稳，避免英文产品名、代码词或中英夹杂被错误归入单一中文模式。后续可在高级设置里增加“优先中文识别”，仅用户明确选择时传 `language=zh`。
|
||||
|
||||
不要把 `foundry-local-whisper` 混入现有 `local-qwen3` provider。两者模型来源、runtime、平台支持和下载语义不同，应共享“本地 ASR 管理”页面的外壳，但后端 provider 和模型 registry 要分开。
|
||||
|
||||
## 会话时序
|
||||
|
||||
1. 用户按当前 OpenLess 全局热键。
|
||||
2. `Coordinator` 进入 `Starting`，Windows 侧准备 TSF IME session。
|
||||
3. `ensure_asr_credentials` 识别 active provider 是 `foundry-local-whisper`：
|
||||
- runtime 可用且模型已缓存：继续；
|
||||
- 模型未缓存：返回可操作错误，胶囊显示“请先下载本地语音模型”，不开始录音；
|
||||
- runtime 初始化失败：显示“本地语音运行时不可用”，引导设置页。
|
||||
4. 创建 `FoundryLocalWhisperAsr`，把它作为 `AudioConsumer` 传给 `Recorder::start`。
|
||||
5. 录音期间 recorder 继续向 consumer 推 PCM，capsule 继续显示电平。
|
||||
6. 用户再次按热键或松开热键结束录音。
|
||||
7. `end_session` 停 recorder，调用 `FoundryLocalWhisperAsr::transcribe()`：
|
||||
- PCM buffer 编码成临时 WAV；
|
||||
- 确保模型 loaded；
|
||||
- 通过 SDK endpoint 调 `/v1/audio/transcriptions`；
|
||||
- 解析 `{ text }` 为 `RawTranscript`。
|
||||
8. 后续完全复用现有逻辑：空 transcript guard、polish / translate、Chinese script preference、Windows TSF-first insert、history append、capsule Done。
|
||||
|
||||
## 首次使用 UX
|
||||
|
||||
Windows 新用户默认 active ASR 使用 `foundry-local-whisper`，但只在“没有现有 preferences / credentials active ASR”的新安装路径生效，不覆盖老用户。
|
||||
|
||||
设置页增加或改造“本地语音识别”区：
|
||||
|
||||
- 显示 runtime 状态：可用、初始化中、不可用。
|
||||
- 显示 execution provider 状态：已注册、需要下载、下载中、失败。
|
||||
- 显示模型列表：`whisper-small`、`whisper-base`、`whisper-tiny`，尺寸和 license 从 Foundry catalog / REST metadata 获取。
|
||||
- 提供一键下载 / 取消 / 删除 / 设为默认 / 加载并测试。
|
||||
- 下载完成后后台 preload，减少第一次热键录音结束后的等待。
|
||||
|
||||
首次按热键但模型缺失时：
|
||||
|
||||
- 不调用 Win+H。
|
||||
- 不弹系统 Voice Typing。
|
||||
- 不开始录音，避免用户说完才发现没有模型。
|
||||
- capsule 显示短错误，主窗口跳到本地语音识别页或给出“下载模型”入口。
|
||||
|
||||
## 质量与性能评估
|
||||
|
||||
中文 / 中英混输：
|
||||
|
||||
- Whisper 系列对普通话和英文都可用，但 `tiny/base/small` 本地模型质量通常低于云端大模型 ASR 或 Whisper large。
|
||||
- `whisper-small` 更适合作为默认质量档；`whisper-base` 用于低配机器。
|
||||
- 热词 bias 当前不会直接进入 Whisper 解码；词汇表仍可作为 LLM polish 上下文和 history 命中统计使用。
|
||||
|
||||
首次延迟：
|
||||
|
||||
- 首次下载 execution provider 和模型可能需要数分钟，取决于网络和硬件。
|
||||
- 首次 load 模型可能需要数秒；应在切换 provider / 下载完成后后台 preload。
|
||||
- 单次转写是 batch 型，不是 Volcengine 那种 streaming final；capsule 可保持“转写中”直到返回。
|
||||
|
||||
模型体积：
|
||||
|
||||
- 体积不硬编码。UI 通过 Foundry catalog / REST metadata 显示当前真实 `fileSizeMb`。
|
||||
- 安装包不内置模型，避免 release artifact 暴涨和 license 风险。
|
||||
|
||||
离线能力：
|
||||
|
||||
- 模型和 execution provider 下载完成后，ASR 推理可离线。
|
||||
- LLM polish 仍取决于用户配置的 LLM provider；LLM 不可用时按现有规则插入 raw transcript。
|
||||
|
||||
隐私：
|
||||
|
||||
- ASR 音频在本机处理，不发送到外部 ASR 服务。
|
||||
- 首次下载模型和组件会访问 Foundry catalog / Microsoft 分发源。
|
||||
- LLM polish 仍可能把 transcript 发送到用户配置的 LLM endpoint；设置页文案需要明确区分“ASR 本地”和“LLM 仍按配置调用”。
|
||||
|
||||
## Windows 安装器与分发
|
||||
|
||||
MVP 不修改 Windows TSF IME 注册流程。
|
||||
|
||||
需要验证：
|
||||
|
||||
- `foundry-local-sdk --features winml` 在 Tauri Windows build 中会引入哪些 DLL、runtime 文件和 redistributable 要求。
|
||||
- NSIS / MSI 是否能自动收集这些 native 依赖。
|
||||
- Windows release workflow 当前对 NSIS / MSI 有固定红线，不能把 bundler 两轮 invoke、`-sice:ICE80` repair 或 `bash` shell 约束顺手改掉。
|
||||
- 如果 Foundry Local runtime 需要额外安装或动态下载组件，UI 必须把“正在准备本地语音运行时”作为一键流程的一部分，而不是要求用户手动跑 `winget`。
|
||||
|
||||
## 失败与 fallback
|
||||
|
||||
- Foundry runtime 缺失或初始化失败：不开始录音，提示本地语音运行时不可用，保留用户切回云 ASR 的入口。
|
||||
- 模型未下载：不开始录音，提示下载模型。
|
||||
- 模型下载失败：保留 partial / retry 状态，不切换到 Win+H。
|
||||
- 转写超时：沿用 coordinator global timeout，写失败状态，不插入空文本。
|
||||
- 转写返回空：沿用 `emptyTranscript` history guard。
|
||||
- LLM polish 失败：插入 raw transcript，history 标记 `polishFailed`。
|
||||
- TSF 提交失败：按现有 `allow_non_tsf_insertion_fallback` 走 Unicode / clipboard fallback；关闭 fallback 时标记 `windowsImeTsfRequired`。
|
||||
|
||||
## 文件与模块边界
|
||||
|
||||
后续实现计划触碰范围：
|
||||
|
||||
- `openless-all/app/src-tauri/Cargo.toml`：Windows 依赖增加 Foundry Local Rust SDK，必要时启用 `winml` feature。
|
||||
- `openless-all/app/src-tauri/src/asr/local/`：拆出 provider-neutral local ASR registry，新增 Foundry Whisper runtime / provider；保留 macOS Qwen3 代码。
|
||||
- `openless-all/app/src-tauri/src/coordinator.rs`：扩展 `ActiveAsr`，在 `begin_session` 和 `end_session` 分支接入 `FoundryLocalWhisperAsr`。
|
||||
- `openless-all/app/src-tauri/src/commands.rs`：新增 Windows local Whisper runtime/model status、download、test、preload 命令，或把现有 `local_asr_*` 扩展成多 backend。
|
||||
- `openless-all/app/src-tauri/src/types.rs`：新增 Windows local ASR preferences，如 active Foundry Whisper model、keep-loaded 时长、语言 hint。
|
||||
- `openless-all/app/src/lib/localAsr.ts`、`src/pages/LocalAsr.tsx`、`src/pages/Settings.tsx`、`src/i18n/*`：展示 Windows 本地语音识别和模型管理。
|
||||
- `openless-all/app/scripts/windows-real-asr-insertion-smoke.ps1`：增加 local ASR 模式，不再强制 Volcengine 凭据。
|
||||
|
||||
Rust 叶子模块仍只依赖 `types.rs` 和自身 provider 内部类型。跨模块编排继续放在 `coordinator.rs`。
|
||||
|
||||
## 验证计划
|
||||
|
||||
静态与单元验证：
|
||||
|
||||
- `asr_configured_for_provider("foundry-local-whisper")` 返回 true，不要求云端 API Key。
|
||||
- `ensure_asr_credentials` 对模型缺失返回明确错误。
|
||||
- fake Foundry endpoint 返回 `{ "text": "..." }` 时，`FoundryLocalWhisperAsr` 能把 PCM 编成 WAV 并产出 `RawTranscript`。
|
||||
- model id、provider id、prefs default 的序列化和迁移测试。
|
||||
|
||||
集成验证：
|
||||
|
||||
- Windows 真机启动 OpenLess，active ASR 为 `foundry-local-whisper`，未配置 Volcengine / Whisper HTTP。
|
||||
- 首次缺模型时按热键，不出现 Win+H 面板，不开始录音，提示下载模型。
|
||||
- 下载模型后聚焦 Notepad，按热键录音，说测试短句，结束后 history 新增 session，`rawTranscript` 非空，`finalText` 非空。
|
||||
- Ark / LLM 未配置时，最终插入 raw transcript，并按现有 polish fallback 规则记录。
|
||||
- Ark / LLM 已配置时，transcript 进入现有 polish / translation 逻辑。
|
||||
- Windows TSF IME 已安装时 `insertStatus=inserted`；禁用 TSF 或目标不支持时按当前 fallback 策略表现。
|
||||
- 断网后重复已下载模型的听写，ASR 仍可完成；LLM 不可用时 raw transcript 不丢。
|
||||
|
||||
No Win+H 验证：
|
||||
|
||||
- 代码搜索确认没有 `Win+H`、Voice Typing、`Windows.Media.SpeechRecognition`、SAPI dictation 调用路径。
|
||||
- 真机 smoke 过程中截图或窗口枚举确认没有 Voice Typing 面板窗口。
|
||||
- 日志只出现 OpenLess recorder、Foundry local ASR、polish、Windows IME / fallback 插入事件。
|
||||
|
||||
## 开放风险
|
||||
|
||||
- Foundry Local preview API 可能变化，尤其是 Rust audio client 和 WinML package 分发。
|
||||
- Foundry Local 的 Whisper 模型质量和中文标点风格需要真机样本验证，不能只靠官方能力声明。
|
||||
- 首次 execution provider 下载和模型下载的错误码、进度回调、缓存位置需要实测。
|
||||
- Windows installer 对 SDK native 依赖的收集需要 release workflow 验证。
|
||||
- 如果 Foundry Local runtime 无法在 Tauri app 内稳定嵌入，备选路线是用 SDK 管理 local REST service；若 REST 也不稳定，再评估 `whisper.cpp` / ONNX Runtime 自管路线。
|
||||
@@ -1,27 +0,0 @@
|
||||
# Issue #154 Tracking
|
||||
|
||||
Scope: Windows dictation lifecycle driven by two hotkey event sources
|
||||
|
||||
Current stage:
|
||||
|
||||
- This branch is a draft PR placeholder.
|
||||
- No runtime fix is included yet.
|
||||
- The goal is to lock down source ownership and failure modes before changing behavior.
|
||||
|
||||
Problem statement:
|
||||
|
||||
- Windows currently has both OS-level low-level keyboard hook input and focused-window renderer forwarding.
|
||||
- macOS/Linux do not have the same dual-source lifecycle driver.
|
||||
- Shared dedupe exists, but source precedence is not yet a first-class contract.
|
||||
|
||||
Implementation target to converge before coding:
|
||||
|
||||
- Decide whether Windows should have one owner source or an explicit precedence model.
|
||||
- Define expected behavior for mixed-source press/release ordering.
|
||||
- Add testable scenarios for hold mode, toggle mode, and focus switching.
|
||||
|
||||
Non-goals in this draft:
|
||||
|
||||
- No hotkey adapter rewrite yet
|
||||
- No input-stack refactor without agreed target contract
|
||||
- No unrelated QA hotkey changes
|
||||
@@ -1,127 +0,0 @@
|
||||
# Windows Tauri 测试 Agent / Workflow 调研
|
||||
|
||||
## 背景
|
||||
|
||||
OpenLess 是 Tauri v2 + React/Vite + Rust 后端的桌面应用。Windows 真机问题主要集中在：
|
||||
|
||||
- 启动首屏：空边框、白屏、前端首帧前窗口过早显示。
|
||||
- 系统能力：全局热键、麦克风隐私权限、剪贴板、前台输入框插入。
|
||||
- 本地状态：凭据读写、历史记录、设置保存。
|
||||
- 人工输入：物理热键无法用普通 synthetic SendInput 可靠替代。
|
||||
|
||||
## 可复用方案
|
||||
|
||||
### 1. 官方 tauri-driver + WebDriver
|
||||
|
||||
来源：
|
||||
|
||||
- https://v2.tauri.app/develop/tests/webdriver/
|
||||
- https://github.com/tauri-apps/webdriver-example
|
||||
|
||||
适合做 CI 基线：
|
||||
|
||||
- 启动 Tauri 应用。
|
||||
- 检查窗口出现、DOM 内容、按钮点击、设置页导航。
|
||||
- Windows CI 可配 `msedgedriver`，Linux CI 可配 `webkit2gtk-driver + xvfb`。
|
||||
|
||||
参考 workflow：
|
||||
|
||||
- `tauri-apps/webdriver-example/.github/workflows/webdriver-v2.yml`
|
||||
- 该 workflow 在 `ubuntu-latest` 和 `windows-latest` 上安装 `tauri-driver`，Windows 侧安装 `msedgedriver`，再分别跑 selenium / webdriverio 测试。
|
||||
|
||||
建议落地：
|
||||
|
||||
- 先选 WebdriverIO，生态和断言更贴近前端团队。
|
||||
- 新增 `openless -all/app/webdriver/`，覆盖：
|
||||
- 应用启动后 1 秒内出现 OpenLess UI。
|
||||
- 设置页提供商字段能读出已存在凭据的“非空状态”。
|
||||
- 打开设置页不会把未修改字段写回空值。
|
||||
- 权限页 Windows 文案不出现 macOS 辅助功能授权提示。
|
||||
|
||||
### 2. tauri-plugin-playwright
|
||||
|
||||
来源：
|
||||
|
||||
- https://docs.rs/crate/tauri-plugin-playwright/0.1.0
|
||||
|
||||
适合做更接近 Playwright 的 E2E：
|
||||
|
||||
- 在 Tauri app 内嵌控制 server。
|
||||
- 使用 Playwright API 做页面级断言。
|
||||
- 对前端团队迁移成本较低。
|
||||
|
||||
风险：
|
||||
|
||||
- 需要引入 Tauri plugin，测试入口和生产入口要隔离。
|
||||
- 目前生态成熟度低于官方 WebDriver。
|
||||
|
||||
建议落地：
|
||||
|
||||
- 暂不作为第一阶段 CI 基线。
|
||||
- 等 WebDriver 跑通后，再评估是否用它补截图、网络、前端状态断言。
|
||||
|
||||
### 3. Tauri MCP / AI Agent 调试插件
|
||||
|
||||
来源：
|
||||
|
||||
- https://github.com/P3GLEG/tauri-plugin-mcp
|
||||
- https://github.com/dirvine/tauri-mcp
|
||||
|
||||
适合做 agent 辅助调试：
|
||||
|
||||
- 截图。
|
||||
- 窗口管理。
|
||||
- DOM 读取。
|
||||
- 鼠标/键盘输入。
|
||||
- localStorage 检查。
|
||||
|
||||
风险：
|
||||
|
||||
- 需要在 app 中接入调试插件，必须确保只在 dev/test 构建启用。
|
||||
- 不适合直接放进 production bundle。
|
||||
|
||||
建议落地：
|
||||
|
||||
- 可以做 `devtools/agent` 分支实验。
|
||||
- 目标是让 Codex/Claude/Cursor 能直接看 Tauri 窗口截图和 DOM，降低“用户肉眼测试”的比例。
|
||||
|
||||
### 4. TestDriver AI
|
||||
|
||||
来源：
|
||||
|
||||
- https://testdriver.ai/vscode
|
||||
- https://github.com/testdriverai/testdriverai
|
||||
|
||||
适合黑盒探索：
|
||||
|
||||
- 用自然语言描述流程。
|
||||
- 支持桌面应用、Windows、GitHub Actions。
|
||||
- 能生成测试报告/视频。
|
||||
|
||||
风险：
|
||||
|
||||
- 外部服务/账号/成本依赖。
|
||||
- 对本项目当前开源 CI 基线不应作为唯一门禁。
|
||||
|
||||
建议落地：
|
||||
|
||||
- 作为 nightly 或人工触发探索测试，不作为 PR 必过的第一层。
|
||||
- 可覆盖“打开 OpenLess Dev、进入设置、检查凭据字段非空、按热键后胶囊状态变化”等高层流程。
|
||||
|
||||
## 推荐实施顺序
|
||||
|
||||
1. 保留现有 PowerShell smoke：构建、启动、进程响应、hotkey listener 日志。
|
||||
2. 增加 WebDriverIO 基线：窗口、DOM、设置页、凭据字段非空状态、Windows 文案。
|
||||
3. 增加 Windows 手动门禁脚本：物理热键、真实 ASR、Notepad fallback、麦克风隐私开关。
|
||||
4. 评估 Tauri MCP：给 agent 提供截图/DOM/输入能力，减少人工描述。
|
||||
5. 评估 TestDriver AI：做黑盒探索和视频报告。
|
||||
|
||||
## 第一批必须补的测试
|
||||
|
||||
- 启动首屏不能先显示空窗口边框。
|
||||
- Windows 启动不等待麦克风 input stream 探测。
|
||||
- 设置页凭据字段加载完成前 blur 不会保存空值。
|
||||
- 设置页打开后不修改字段，`credentials.json` 不发生变化。
|
||||
- `get_credentials` 与 `read_credential` 对同一文件返回一致状态。
|
||||
- 右 Control 默认热键文案在概览、历史、设置中一致。
|
||||
- Windows 权限页不显示 macOS 辅助功能授权引导。
|
||||
@@ -1,23 +0,0 @@
|
||||
# Issue #142 Placeholder / 占位
|
||||
|
||||
## 中文摘要
|
||||
|
||||
本 PR 是 issue #142 的 draft 占位，专门跟踪 Windows Capsule 变形、失真与尺寸错位问题。
|
||||
当前只保留问题边界、几何证据和后续修复准入条件，不引入业务逻辑改动。
|
||||
|
||||
## Scope / 范围
|
||||
|
||||
- Capsule native window bounds
|
||||
- visual pill metrics
|
||||
- badge position
|
||||
- Windows DPI / transparent window clipping
|
||||
|
||||
## Evidence / 证据入口
|
||||
|
||||
- `openless-all/app/src-tauri/src/lib.rs`
|
||||
- `openless-all/app/src/components/Capsule.tsx`
|
||||
- `openless-all/app/src/lib/capsuleLayout.ts`
|
||||
|
||||
## Merge Rule / 合并规则
|
||||
|
||||
- 仅当 issue #142 的几何对齐与 Windows smoke 验证完成后才允许从 draft 转为 ready。
|
||||
@@ -1,23 +0,0 @@
|
||||
# Issue #143 Placeholder / 占位
|
||||
|
||||
## 中文摘要
|
||||
|
||||
本 PR 是 issue #143 的 draft 占位，专门跟踪 Windows 冷启动前几秒加载异常、闪烁与 ready 前展示错位问题。
|
||||
当前只记录时序边界、现象入口和后续修复出口，不引入无关功能修改。
|
||||
|
||||
## Scope / 范围
|
||||
|
||||
- visible / ready timing
|
||||
- first stable paint
|
||||
- startup shell exposure
|
||||
- Windows cold start UX
|
||||
|
||||
## Evidence / 证据入口
|
||||
|
||||
- `openless-all/app/src-tauri/tauri.conf.json`
|
||||
- `openless-all/app/src/App.tsx`
|
||||
- `openless-all/app/src/components/FloatingShell.tsx`
|
||||
|
||||
## Merge Rule / 合并规则
|
||||
|
||||
- 仅当 issue #143 的启动时序统一且完成 Windows cold-start smoke 后才允许从 draft 转为 ready。
|
||||
@@ -1,65 +0,0 @@
|
||||
# Windows upstream PR workflow
|
||||
|
||||
## 目标
|
||||
|
||||
Windows 主线先在 `fork/dev` 完成发现、修复、CI、自审和复审，再收敛成明确 upstream 维护项。不要把未收敛的真机 findings 直接写到 upstream issues 或 upstream PR。
|
||||
|
||||
## 标准流程
|
||||
|
||||
1. 在 `fork/dev` 修复问题。
|
||||
- 每个提交只解决一个明确问题。
|
||||
- findings 先写到本地记录或 fork issue。
|
||||
- 不向 upstream 新增噪声 issue。
|
||||
|
||||
2. 在 `fork/dev` 触发 CI。
|
||||
- Windows build 必须过。
|
||||
- 新增/修改的 Windows smoke 必须能在本机复跑。
|
||||
- 真实凭据、物理热键、ASR、插入 fallback 等不能完全 CI 化的项目，要留下本机证据路径和日志摘要。
|
||||
|
||||
3. 在 fork 上开自有 PR。
|
||||
- base: `fork/dev`
|
||||
- head: 功能分支
|
||||
- PR 描述使用中文，按模板填写。
|
||||
- PR 必须包含 fork CI 链接、真机回归摘要、自审结论。
|
||||
|
||||
4. 复审 fork PR。
|
||||
- 先按 code review 方式找阻断项。
|
||||
- 修完 review findings 后再次跑 fork CI。
|
||||
- 只有 fork PR 复审通过，才能进入 upstream 收敛。
|
||||
|
||||
5. 收敛 upstream 维护项。
|
||||
- 从 fork PR 中拆出最小 upstream 维护切片。
|
||||
- upstream PR 只包含已验证的最小改动。
|
||||
- upstream PR 描述必须带 fork PR / fork CI 链接，说明该切片来自已验证的 `fork/dev` 工作流。
|
||||
- upstream issue 只用于已经确认、可维护、可复现、需要 upstream 跟踪的问题；不要把探索期 findings 扔到 upstream。
|
||||
|
||||
## upstream PR 进入条件
|
||||
|
||||
- `fork/dev` 已包含修复。
|
||||
- fork PR 已通过 CI。
|
||||
- fork PR 已完成自审和复审。
|
||||
- upstream 分支从最新 upstream base 切出。
|
||||
- upstream diff 能独立解释，不依赖 fork/dev 的其他未提交上下文。
|
||||
- PR 描述包含：
|
||||
- 单一目标
|
||||
- 不包含范围
|
||||
- fork PR 链接
|
||||
- fork CI 链接
|
||||
- 本机 Windows 回归证据
|
||||
|
||||
## 禁止项
|
||||
|
||||
- 禁止从未验证的本地 finding 直接创建 upstream issue。
|
||||
- 禁止绕过 fork/dev CI 直接推 upstream PR。
|
||||
- 禁止把多个 Windows 真机问题混成一个 upstream PR。
|
||||
- 禁止在 upstream PR 中提交真实服务凭据、用户本地配置、构建产物或临时目录。
|
||||
|
||||
## 当前执行规则
|
||||
|
||||
后续 Windows 主线默认顺序为：
|
||||
|
||||
```text
|
||||
fork/dev 修复 -> fork/dev CI -> fork PR -> 自审/复审 -> upstream 最小 PR
|
||||
```
|
||||
|
||||
如果 upstream PR 需要更新，先确认对应 fork PR 和 fork CI 证据，再同步 upstream PR。
|
||||
Reference in new issue
Block a user