Merge branch 'appergb:main' into main

This commit is contained in:
Chris233 authored and GitHub committed 2026-05-02 16:11:42 +08:00
commit a259b57d5e
51 files changed
+3470 -213

No files matched your search

+4
View File
@@ -222,6 +222,7 @@ jobs:
OPENLESS_UPDATE_TARGET: ${{ matrix.updater-target }}
OPENLESS_UPDATE_ARCH: ${{ matrix.updater-arch }}
OPENLESS_UPDATE_REPO: appergb/openless
OPENLESS_UPDATE_MIRROR_BASE_URL: https://fastgit.cc/https://github.com
run: node scripts/write-updater-manifest.mjs
# ── 收集产物 ──
@@ -266,6 +267,7 @@ jobs:
openless-all/app/src-tauri/target/release/bundle/macos/*.app.tar.gz
openless-all/app/src-tauri/target/release/bundle/macos/*.app.tar.gz.sig
openless-all/app/src-tauri/target/release/bundle/latest-darwin-aarch64.json
openless-all/app/src-tauri/target/release/bundle/latest-darwin-aarch64-mirror.json
if-no-files-found: error
- name: Upload Windows artifacts
@@ -287,6 +289,7 @@ jobs:
openless-all/app/src-tauri/target/release/bundle/nsis/*.exe.sig
openless-all/app/src-tauri/target/release/bundle/msi/*.msi.sig
openless-all/app/src-tauri/target/release/bundle/latest-windows-x86_64.json
openless-all/app/src-tauri/target/release/bundle/latest-windows-x86_64-mirror.json
if-no-files-found: error
- name: Upload Linux artifacts
@@ -308,6 +311,7 @@ jobs:
path: |
openless-all/app/src-tauri/target/release/bundle/appimage/*.AppImage.sig
openless-all/app/src-tauri/target/release/bundle/latest-linux-x86_64.json
openless-all/app/src-tauri/target/release/bundle/latest-linux-x86_64-mirror.json
if-no-files-found: error
# ── tag 推送时,同步上传到 GitHub Release ──
@@ -0,0 +1,334 @@
# Windows terminal clipboard restore investigation (2026-05-02)
Scope: `openless-all/app/src-tauri/src/insertion.rs`
## Problem statement
On Windows terminal-style text entry, OpenLess could:
1. put the new dictated text into the clipboard
2. send `Ctrl+V`
3. restore the old clipboard too early
4. let the terminal paste the old clipboard instead of the dictated text
## Baseline code path
- `Coordinator::end_session()` treats Windows synthetic paste as `InsertStatus::PasteSent`, not `Inserted`.
- `TextInserter::insert()` calls `insert_with_clipboard_restore()`.
- Baseline Windows/Linux behavior restored the previous clipboard after a fixed `150ms`.
- That fixed delay assumed the target app had already consumed the clipboard by then.
## Automated evidence
### 1. GUI automation boundary in this session
Commands used:
```powershell
Start-Process notepad.exe -PassThru
Start-Process cmd.exe -PassThru
EnumWindows(...)
```
Observed result:
- `explorer.exe` exists in `SessionId=1`
- newly started `notepad.exe`, `cmd.exe`, and even a local WinForms probe form did not expose enumerable top-level windows in this thread
Conclusion:
- this Codex desktop thread can compile and manipulate the Windows clipboard
- it cannot reliably drive newly created GUI windows in the current desktop context
- therefore the strongest fully automated evidence in this session must come from clipboard-timing experiments, not end-to-end GUI paste readback
### 2. Clipboard timing matrix
Script:
- `openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1`
Command:
```powershell
$cases = @(
@{ consumer = 50; restore = 150 },
@{ consumer = 250; restore = 150 },
@{ consumer = 250; restore = 750 }
)
foreach ($case in $cases) {
powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1 -ConsumerDelayMs $case.consumer -RestoreDelayMs $case.restore
}
```
Observed outputs:
```json
{"consumerDelayMs":50,"restoreDelayMs":150,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_DICTATED_TEXT","matchedInserted":true}
{"consumerDelayMs":250,"restoreDelayMs":150,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_OLDER_CLIPBOARD","matchedInserted":false}
{"consumerDelayMs":250,"restoreDelayMs":750,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_DICTATED_TEXT","matchedInserted":true}
```
Interpretation:
- a fast consumer (`50ms`) succeeds with the old `150ms` restore window
- a slower consumer (`250ms`) fails with the old `150ms` restore window
- the same slower consumer succeeds once restore is delayed to `750ms`
This isolates the bug to clipboard restore timing, independent of ASR, polish, QA hotkey, or selection logic.
### 3. Real app end-to-end regression in a stable desktop automation stack
Environment:
- Python `pywinauto` + `pywin32`
- Real desktop windows, not mock controls
- Targets:
- Windows Terminal `cmd.exe` tab
- Windows Terminal `PowerShell` tab
- Notepad
Method:
- Put a command or text payload into the real Windows clipboard
- Send synthetic `Ctrl+V`
- Wait either `150ms` or `750ms`
- Restore the previous clipboard
- Verify the target app actually received the intended payload
Observed outputs:
```json
[
{
"target": "Windows Terminal CMD",
"restoreDelayMs": 150,
"expected": "CMD_150_OK",
"succeeded": true
},
{
"target": "Windows Terminal CMD",
"restoreDelayMs": 750,
"expected": "CMD_750_OK",
"succeeded": true
},
{
"target": "Windows Terminal PowerShell",
"restoreDelayMs": 150,
"expected": "POWERSHELL_150_OK",
"succeeded": true
},
{
"target": "Windows Terminal PowerShell",
"restoreDelayMs": 750,
"expected": "POWERSHELL_750_OK",
"succeeded": true
},
{
"target": "Notepad",
"restoreDelayMs": 150,
"expected": "NOTEPAD_150_OK",
"succeeded": true
},
{
"target": "Notepad",
"restoreDelayMs": 750,
"expected": "NOTEPAD_750_OK",
"succeeded": true
}
]
```
Interpretation:
- the isolated clipboard/paste/restore harness does **not** reproduce the stale-paste bug on the current Windows Terminal `CMD` tab
- it also does **not** reproduce it on the current Windows Terminal `PowerShell` tab
- Notepad behaves as expected in both timing windows
- therefore the user-reported failure is not a blanket “all terminal paste on Windows fails at 150ms” statement
- the failure requires an additional condition beyond “target is a terminal”, such as a slower paste consumer, extra lifecycle delay, or OpenLess-specific sequencing around focus restoration and session completion
### 4. Full OpenLess lifecycle evidence on `wt-cmd`
To go beyond isolated paste harnesses, the automation was pushed through the real OpenLess lifecycle:
- synthetic hold-mode hotkey press on Windows (`VK_LCONTROL`, observed by the low-level hook)
- real recorder startup
- real Volcengine ASR session connection
- real LLM polish
- real insertion into a Windows Terminal `cmd.exe` tab
Because the desktop automation session could not reliably feed text into the real microphone path, a debug-only test hook was added for automation:
- if a debug transcript file is configured and ASR returns an empty transcript, OpenLess substitutes that transcript and continues through the normal post-ASR insertion path
One captured successful run produced the following evidence:
- OpenLess log:
- `[hotkey] Windows trigger pressed vk=162`
- `[coord] front_app captured: C:\WINDOWS\system32\cmd.exe`
- `[coord] recorder started (asr=volcengine, phase=Starting)`
- `[coord] ASR connected; flushed ... deferred audio bytes`
- `[coord] session started`
- `[hotkey] Windows trigger released vk=162`
- `[llm] HTTP 200 ...`
- History record:
```json
{
"rawTranscript": "瀑布它的白沫其实非常喜欢。",
"finalText": "瀑布的白沫其实非常喜欢。",
"insertStatus": "pasteSent"
}
```
- Windows Terminal `cmd.exe` tab tail:
```text
D:\Users\cooper\Practice-Project\202604\openless>瀑布的白沫其实非常喜欢。
```
Interpretation:
- this is a true OpenLess session, not a bare clipboard harness
- the target front app captured by OpenLess was the Windows Terminal `cmd.exe` tab
- the final inserted text visible at the terminal prompt matched the polished `finalText`
- in this captured run, the terminal did **not** paste the pre-dictation clipboard contents
Residual caveat:
- repeated re-runs in the same desktop session later hit intermittent startup/hook-install flakiness before the test reached insertion again
- that flakiness affected test repeatability, but it does not invalidate the already captured successful full-lifecycle evidence above
## 5. Repeatable full-lifecycle regression after automation hardening
After hardening the automation path, the full OpenLess lifecycle was run through a stable route:
- launch OpenLess with WebView2 remote debugging enabled
- drive lifecycle by invoking Tauri commands from the main webview (`start_dictation` / `stop_dictation`)
- keep real focus-target capture and real insertion behavior
- use a debug-only transcript override only when ASR would otherwise be empty in this desktop environment
- read back target content directly from UIA controls instead of recycling clipboard-based readback
Targets exercised:
- `Windows Terminal` `cmd.exe` tab
- `Windows Terminal` `PowerShell` tab
- `Notepad`
Representative results:
```json
{
"target": "wt-cmd",
"historyFinalText": "openless terminal regression success",
"insertStatus": "pasteSent",
"targetContainsFinalText": true,
"targetContainsClipboardSentinel": false
}
{
"target": "wt-powershell",
"historyFinalText": "openless terminal regression success",
"insertStatus": "pasteSent",
"targetContainsFinalText": true,
"targetContainsClipboardSentinel": false
}
{
"target": "notepad",
"historyFinalText": "openless terminal regression success",
"insertStatus": "pasteSent",
"targetContainsFinalText": true,
"targetContainsClipboardSentinel": false
}
```
Repeatability observed in the current session:
- `wt-cmd`: multiple successful runs with final text visible at the terminal prompt
- `wt-powershell`: successful run with final text visible at the terminal prompt
- `notepad`: two consecutive successful runs after switching readback from clipboard-based copy to direct UIA text capture
Updated interpretation:
- the originally suspected “terminal paste always restores the old clipboard before paste lands” is **not** reproducible as a general rule in the current full-lifecycle automation
- once the automation path is stabilized, all three tested targets receive the intended final text while `insertStatus` remains `pasteSent`
- the clipboard timing race is still real in isolation for slow consumers, but the complete OpenLess lifecycle on this machine does not reproduce the stale-clipboard failure for:
- `wt-cmd`
- `wt-powershell`
- `notepad`
Most likely current conclusion:
- the user-reported bug depends on an additional condition not captured in the hardened automation path
- plausible candidates remain:
- a different terminal host/session state
- a different target application than the tested Windows Terminal tabs
- another timing-sensitive environment factor outside the core insertion code
## Root cause
Root cause: Windows `PasteSent` semantics were treated as if they implied paste completion.
- `PasteSent` only means OpenLess sent synthetic `Ctrl+V`
- it does not mean the target application has already read clipboard contents
- terminal-style targets can consume the clipboard later than standard text inputs
- restoring the old clipboard at a fixed `150ms` can therefore race ahead of actual paste consumption
- current real-app regression suggests this is conditional, not universal: some terminal sessions consume quickly enough to beat `150ms`, while slower consumers still fail
Classification:
- primary layer: `clipboard lifecycle`
- secondary layer: `insertion lifecycle`
- not primary: `focus restore`
- manifestation: terminal-specific and likely any slower Windows paste consumer
- not evidence of a global Windows clipboard bug by itself
## Fix applied
File:
- `openless-all/app/src-tauri/src/insertion.rs`
Change:
- Windows clipboard restore delay changed from `150ms` to `750ms`
- restore now runs on a background thread instead of blocking the insert path
- Linux keeps the previous `150ms` behavior
## Verification run
Commands:
```powershell
cargo fmt --all
cargo check --lib
cargo test --lib --no-run
cargo check --tests
powershell -NoProfile -Command "[void][scriptblock]::Create((Get-Content -Raw 'openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1')); 'script-parse-ok'"
```
Observed result:
- compile/check passed
- test binaries compiled
- new smoke scripts parse successfully
- real desktop automation passed on:
- Windows Terminal `CMD` tab at `150ms` and `750ms`
- Windows Terminal `PowerShell` tab at `150ms` and `750ms`
- Notepad at `150ms` and `750ms`
## Remaining gap
Still needed if we want to exactly mirror the original user report:
- drive **OpenLess itself** through the full dictation lifecycle in the same run
- keep the target specifically in the same terminal/input setup where the stale paste was originally observed
- capture whether the failing case depends on:
- OpenLess focus-target restore timing
- ASR/polish latency
- the exact terminal host/session state
- another app-specific delay not present in the isolated paste harness
## Suggested issue / PR title
- Issue: `[windows][insertion] terminal paste can restore stale clipboard before synthetic paste lands`
- PR: `fix(windows): delay clipboard restore after synthetic paste`
+63
View File
@@ -0,0 +1,63 @@
# Auto Update Download Acceleration
## Problem
OpenLess used a single Tauri updater endpoint on GitHub Releases:
```text
https://github.com/appergb/openless/releases/latest/download/latest-{{target}}-{{arch}}.json
```
The manifest also pointed installer downloads back to GitHub Releases. On networks where GitHub release assets are slow, a small updater package can take minutes to download.
Desktop apps do not reliably inherit a user's shell proxy environment. Instead of making updater correctness depend on whether a proxy is visible to the app process, the updater should use a GitHub release acceleration URL directly.
## Runtime Behavior
The app does not manually probe local proxy ports. It lets the OS/process network stack do whatever it normally does, while the updater endpoint itself points at `fastgit.cc` first. This keeps the rule simple: proxy or no proxy, updater traffic should prefer the fastgit transport.
## Fastgit Acceleration
Release builds now publish two updater manifests per target:
```text
latest-<target>-<arch>.json
latest-<target>-<arch>-mirror.json
```
The client checks the mirror manifest first, then GitHub. The mirror manifest points its installer URL at:
```text
https://fastgit.cc/https://github.com/<repo>/releases/latest/download/<asset>
```
The updater signature still protects the downloaded package. The mirror only changes transport; it cannot replace the signed payload without verification failing.
## Maintainer Notes
Set `OPENLESS_UPDATE_MIRROR_BASE_URL` in CI to change the mirror host. Keep it formatted as a prefix for GitHub URLs, for example:
```text
https://fastgit.cc/https://github.com
```
If a mirror becomes unreliable, replace that environment value and the mirror endpoint in `openless-all/app/src-tauri/tauri.conf.json`.
## Evidence
Measured from Windows on 2026-05-01. Direct GitHub release downloads were tested with local proxy disabled to reproduce the slow path. `fastgit.cc` was tested both through the normal local proxy environment and with local proxy disabled; results vary by route, so do not treat one machine's no-proxy number as a CDN SLA.
```text
Direct GitHub installer asset, 4.78 MB, proxy disabled:
run 1: timed out after 90.75s, 1.73 MB received
run 2: timed out after 90.06s, 2.44 MB received
fastgit.cc installer asset, 4.78 MB, normal local proxy environment:
with protocol prefix: 3.12s / 3.63s / 3.39s
without protocol prefix: 2.92s / 2.45s / 2.87s
fastgit.cc target-user signal:
manual browser/download usage reported completing in under 1s without enabling a proxy.
```
This is enough to justify a `fastgit.cc` mirror path, but not enough to treat a public mirror as permanently trusted infrastructure. `fastgit.cc` explicitly documents support for GitHub release/archive acceleration and accepts GitHub links with or without the protocol prefix. Keep the mirror configurable and re-test before each release if download performance is a release blocker.
@@ -0,0 +1,87 @@
## 现象 / Symptom
这不是单一的 click dead zone bug,而是一组已经在 Windows 实机上被观察到、且共享同一根因的 helper-window lifecycle 症状:
- click dead zone:原 Capsule 区域附近会挡住底层输入框或按钮
- screenshot selectable:截图工具仍然可以选中这块透明区域
- drag stutter:在该区域拖拽时出现明显卡顿或 compositor 异常
- lingering transparent overlay:录音结束后,Capsule 仍可能以透明顶层窗 linger
当前证据说明:这些现象不应拆成多个互不相关的问题,而应视为同一个生命周期语义偏差。
### 证据 / Evidence
运行与代码证据:
- `openless-all/app/src-tauri/tauri.conf.json:33-47`
- `capsule` 被配置为 `transparent + alwaysOnTop + focus:false + visible:false`
- `openless-all/app/src-tauri/src/lib.rs:594-623`
- Windows 端 `capsule` runtime host bounds 为 `220x84/118`,明显大于可见 pill `196x52`
- `openless-all/app/src-tauri/src/coordinator.rs:2398-2432`
- Windows 端显示路径走 `ShowWindow(SW_SHOWNOACTIVATE)` + `SetWindowPos(...SWP_NOACTIVATE...)`
- `openless-all/app/src-tauri/src/coordinator.rs:2455-2479`
- 结束阶段依赖 `window.hide()` 作为生命周期结束语义
- `openless-all/app/src/components/Capsule.tsx:278-281`
- 前端 `idle` 只把可见内容缩成 `0x0`,真正结束仍取决于后端窗口是否已完全退出参与
- [2026-05-02-platform-lifecycle-audit.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/2026-05-02-platform-lifecycle-audit.md)
- 审计已把该问题收敛为 Windows helper-window lifecycle contract 偏差
现场证据:
- 用户已在 Windows 上观察到 dead zone / screenshot selectable / drag stutter / lingering overlay
- 这些表现与透明顶层 helper window 未真正退出 OS 参与的形态一致
### 5 Whys / 根因分析
1. 为什么会出现点击死区、截图可选中、拖拽卡顿?
- 因为录音结束后,Windows 上的 Capsule host window 仍可能继续存在并参与桌面层级。
2. 为什么录音结束后窗口还会继续参与?
- 因为当前实现把“生命周期结束”主要建模成 `hide()`,而不是“保证 helper window 不再参与 hit-test / capture / z-order / compositor”。
3. 为什么这个问题在 Windows 上更容易暴露?
- 因为 Windows 的 Capsule host geometry 更大、show path 更特殊,并且是透明顶层窗;一旦 hide 语义失守,残留面积极大且更容易干扰系统行为。
4. 为什么这和 macOS 的原始设计意图不一致?
- macOS 的原始意图是:Capsule 只在 active stage 短暂出现,结束后自然收起,不再作为前台交互对象继续存在;Windows 当前更像“视觉结束了,但 OS 对象还挂着”。
5. 为什么之前没有被门禁拦住?
- 现有检查更多关注“窗口显示/隐藏”和几何配置,没有直接验证 inactive state 下它是否真的退出系统参与。
### 平台边界 / Platform Scope
- 直接症状范围:当前已确认是 Windows 实机问题。
- 问题层面:backend helper-window lifecycle contract + Windows native window participation。
- 全平台风险判断:根因模式不是 Windows 独有,任何透明 helper window 只要“视觉隐藏 != 生命周期结束”都可能中招;Capsule 目前是 Windows 上最先爆出来的样板案例。
### 认领 / Ownership
- owner intent:`@Cooper-X-Oak`
- 当前对应 draft PR:`#140`
### 当前状态 / Current status
- lifecycle 主线修复已完成第一波
- 人工桌面回归结果:
- click dead zone:通过
- screenshot selectable:通过
- drag stutter:通过
- 当前建议:从“问题收敛中”推进到“regression review 中”
## 影响 / Impact
- 直接影响 Windows 端核心输入体验与系统交互可信度
- 会误伤底层 app 的点击、截图、拖拽,用户容易误判成其他应用故障
- 因为残留对象透明且顶层,这类问题隐蔽、难复现、难定位
- 如果不从生命周期语义修,后续即使修掉某一个 dead zone,仍可能继续遗留 screenshot / z-order / compositor 问题
## 建议接受标准 / Proposed Acceptance Criteria
- [ ] Windows 上 Capsule 的“结束”语义与 macOS 对齐:inactive 后不再继续参与系统交互
- [ ] inactive Capsule 不再造成 click dead zone
- [ ] inactive Capsule 不再被截图工具选中
- [ ] inactive Capsule 不再引入 drag/compositor stutter
- [ ] 为 Windows 增加一条直接验证 inactive Capsule non-participating 的 smoke / regression check
- [ ] 修复方案明确区分 visual state 与 host-window lifecycle state,而不是继续叠加局部 workaround
## TODO / 不确定项
- 是否需要把 `capsule hidden => no hit-test / no capture / no topmost participation` 抽成统一 helper-window contract,复用于 QA panel
- 当前 `PR #140` 建议保持 draft tracking 角色,待范围与根因完全收敛后再转 ready
建议 issue 标题:`[ui][windows] Capsule 隐藏后仍参与系统交互`
@@ -0,0 +1,74 @@
## 现象 / Symptom
Windows 冷启动路径里,`visible` 与 `ready` 目前是脱钩的:主窗口可以先被用户看见,但 global hotkey / runtime lifecycle 还在后台异步安装。
这不是单纯的 UI 小闪烁,而是 startup lifecycle ownership 不统一:
- `main` 在配置层默认 `visible:false`
- backend 负责 `show_main_window()` / tray reopen / single-instance focus
- frontend `App.tsx` 又在 mount 后主动 `currentWindow.show()`
- Windows 路径下 `gate` 初始值直接是 `ready`
### 证据 / Evidence
- `openless-all/app/src-tauri/tauri.conf.json:17-30`
- `main.visible = false`
- `openless-all/app/src-tauri/src/lib.rs:314-356`
- backend 明确拥有 `show_main_window()` / `hide_main_window()` 生命周期入口
- `openless-all/app/src-tauri/src/lib.rs:158-163`
- hotkey listener 与 QA hotkey listener 在 setup 后异步启动
- `openless-all/app/src/App.tsx:23-52`
- Windows 路径初始化时直接 `gate='ready'`
- mount 后又在 `requestAnimationFrame` 里调用 `currentWindow.show()`
- [2026-05-02-platform-lifecycle-audit.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/2026-05-02-platform-lifecycle-audit.md)
- 审计已将该问题归类为 startup lifecycle ownership 偏差
### 5 Whys / 根因分析
1. 为什么用户会看到一个看似 ready 的窗口,但热键/运行态未必已经 ready?
- 因为窗口可见时机和 runtime readiness 时机不是一个 source of truth。
2. 为什么这两个时机分离了?
- 因为 backend 和 frontend 同时持有 `main` visibility 的一部分控制权。
3. 为什么 Windows 上更明显?
- 因为 Windows 启动路径跳过了 macOS 那种明确的 permission gate / startup shell,正式 UI 更早暴露。
4. 为什么这偏离了 macOS 的原始设计意图?
- 原始意图是“用户看见主窗口时,它已经进入可用或可解释的阶段”;Windows 当前更像“窗口先到,能力后到”。
5. 为什么之前没被系统性识别?
- 现有 smoke 主要验证“进程活着 + 稍后日志出现 hotkey installed”,没有验证“first visible frame == operationally ready”。
### 平台边界 / Platform Scope
- 直接症状范围:当前主要在 Windows 冷启动观察到。
- 问题层面:startup lifecycle ownership、window visibility contract、runtime readiness contract。
- 全平台风险判断:这是全平台架构层风险,但 Windows 因跳过 startup gate、前端主动 show,最先表现为真实用户问题。
### 认领 / Ownership
- owner intent:`@Cooper-X-Oak`
- 对应 draft PR:待创建
### 当前状态 / Current status
- startup lifecycle 主线修复已生效
- 最新测试入口改为 frontend-managed first show,不再用 backend immediate show 污染结果
- 人工冷启动体验反馈:几乎没有问题,人眼很难分辨
- 当前建议:保留 draft,继续观察 first-paint / startup latency,而不是继续扩大主修补丁
## 影响 / Impact
- 用户会把尚未 ready 的窗口误判为已经 ready
- 会放大“热键没反应 / 运行态未安装”的首屏困惑
- 让后续任何 Windows 启动问题更难分辨是 UI 问题、hotkey 问题,还是 lifecycle contract 问题
## 建议接受标准 / Proposed Acceptance Criteria
- [ ] `main` 窗口的首次可见时机只由一个 owner 控制
- [ ] first visible frame 与 runtime readiness 的关系被明确定义并可验证
- [ ] Windows 冷启动下,用户首次看到主窗口时,至少处于明确的 `startup` 或 `ready` 状态,而不是 ambiguous ready
- [ ] 增加一条启动 smoke:覆盖 `visible`、`hotkey installed`、`first usable state` 的先后顺序
## TODO / 不确定项
- 是否应把 `main` visibility 完全收回 backend,frontend 只负责内容 gate
- 是否要把现有 `issue #143` 的 first-paint 问题作为本 issue 的下游视觉子问题处理,还是继续分票并行跟踪
建议 issue 标题:`[tauri][windows] 冷启动时 visible 与 ready 脱钩`
@@ -0,0 +1,77 @@
## Symptom
Windows dictation / QA lifecycle previously had two event sources driving the same state machine:
- OS-level low-level keyboard hook
- renderer / window-local hotkey forwarding
That design is risky even when the product "seems to work":
- press/release edges can come from different sources
- focus switches can strand half an edge
- hold mode and toggle mode can drift differently on Windows only
## Evidence
- [openless-all/app/src/App.tsx](/D:/Users/cooper/Practice-Project/202604/openless/openless-all/app/src/App.tsx)
- Windows window-local forwarding existed in the frontend path
- [openless-all/app/src-tauri/src/coordinator.rs](/D:/Users/cooper/Practice-Project/202604/openless/openless-all/app/src-tauri/src/coordinator.rs)
- backend also accepted `handle_window_hotkey_event`
- [openless-all/app/src-tauri/src/hotkey.rs](/D:/Users/cooper/Practice-Project/202604/openless/openless-all/app/src-tauri/src/hotkey.rs)
- Windows already owns a `WH_KEYBOARD_LL` low-level hook
Current convergence from this repair track:
- QA hotkey / follow-up flow works
- Windows owner source should be the backend low-level hook
- window-local forwarding should not keep driving the same lifecycle
## Root Cause Convergence
This was not just "an extra fallback path".
It was an ownership problem:
```text
Two independent input sources were allowed to influence one dictation / QA
lifecycle state machine without an explicit precedence contract.
```
## 5 Whys
1. Why is this a lifecycle issue and not just a convenience fallback?
- Because the second path was able to trigger real start/stop edges.
2. Why is that dangerous?
- Because mixed-source ordering can desynchronize phase transitions.
3. Why is it primarily a Windows issue?
- Because Windows carried both the low-level hook and the renderer-forward path.
4. Why does this diverge from original intent?
- Because one user gesture should map to one stable lifecycle transition.
5. Why is this near closure now?
- Because current repair work has already converged on a single owner source: backend low-level hook.
## Platform Scope
- Direct symptom scope: Windows implementation risk
- Problem layer: input source ownership, lifecycle precedence, focus-sensitive edge delivery
## Related Issues
- #154 main issue anchor
- #147 settings-to-runtime listener refresh contract
- #158 governance issue for helper-window / native-window contract family
## Impact
- Without a single owner source, Windows-only lifecycle drift remains hard to reproduce and harder to trust
- With ownership clarified, regression review can focus on evidence instead of guessing which path fired
## Proposed Acceptance Criteria
- [ ] Windows lifecycle owner source is explicitly documented as backend low-level hook
- [ ] window-local forwarding no longer drives the main lifecycle unless a future explicit fallback contract is introduced
- [ ] regression review confirms no new mixed-source ordering evidence
## Status Note
Current recommendation: treat this issue as near-closure and use it as a regression-review anchor rather than a new large refactor anchor.
@@ -0,0 +1,146 @@
## 现象 / Symptom
Windows terminal 文本输入场景历史上出现过两类现象:
- 用户反馈 terminal 里不会自动上屏,需要再手动 `Ctrl+V`
- 本地测试曾观察到一次“目标最终拿到的是旧剪贴板,而不是本次听写结果”的现象
这两类现象都指向同一条 Windows insertion 链路:OpenLess 通过 clipboard + synthetic `Ctrl+V` 完成插入,而 terminal 是最敏感的目标类型之一。
### 证据 / Evidence
- `openless-all/app/src-tauri/src/insertion.rs`
- Windows 路径的成功语义是 `PasteSent`
- `PasteSent` 只代表已经发出 synthetic `Ctrl+V`
- 它不代表目标已经完成 clipboard 消费
- `docs/2026-05-02-windows-terminal-clipboard-restore-investigation.md`
- 已沉淀完整隔离实验、真实目标回归、完整生命周期自动化和最终结论
- 历史反馈层面
- terminal 场景曾出现“不能自动上屏、需要手动 `Ctrl+V`”的真实用户反馈
- 隔离时序实验层面
- 快消费者 + `150ms` restore:通过
- 慢消费者 + `150ms` restore:读到旧剪贴板
- 慢消费者 + `750ms` restore:恢复正常
- 完整生命周期回归层面
- 稳定化自动化已覆盖 `wt-cmd`、`wt-powershell`、`notepad`
- 当前机器上三类目标都能拿到本次 `finalText`
### 根因分析 / 追索过程
#### 1. 从用户现象到怀疑方向
最初现象不是“某个 API 报错”,而是目标内容不对:
- 目标没上屏
- 或者看起来像 paste 进了旧内容
这类问题天然需要同时排查三层:
- clipboard lifecycle
- insertion lifecycle
- focus / target restore
#### 2. 为什么先聚焦 clipboard restore
代码阅读后,Windows 插入链路具备一个明显特征:
- 先把本次文本写入 clipboard
- 再发 synthetic `Ctrl+V`
- 再恢复旧 clipboard
而状态语义里 `PasteSent` 并不等于“目标已经完成 paste”。
因此最早的根因假设是:
- 如果目标消费 clipboard 较慢,restore 可能会抢在目标 paste 之前发生
#### 3. 如何证明这个假设不是猜测
我们补了独立的时序实验,把 OpenLess 业务链路先拆开,只验证:
- clipboard 写入
- synthetic paste
- restore 时机
- 目标何时读取 clipboard
实验结果明确证明:
- race 在模型上真实存在
- `150ms` 对慢消费者不安全
- 增加 restore 窗口后可以避免慢消费者读到旧 clipboard
这一步把“怀疑”变成了“已确认的风险点”。
#### 4. 为什么还要继续做完整生命周期自动化
隔离实验只能说明风险存在,不能证明用户原始现象在真实 OpenLess 生命周期里一定复现。
因此后续又补了:
- 真实 OpenLess 启动
- 真实 focus-target capture
- 真实 insertion 尾链
- `wt-cmd` / `wt-powershell` / `notepad` 的目标读回
同时为了绕过桌面音频路由波动,又加了 debug-only transcript override,只在 ASR 为空时替换 transcript,保证:
- 前半段生命周期仍然真实
- 后半段 insertion / clipboard / target readback 仍然真实
#### 5. 最终根因判断
最终可以明确的根因不是“terminal 当前一定有 bug”,而是:
- Windows insertion 链路原本存在一个真实的 clipboard restore timing 风险
- 这个风险可以解释历史上 terminal 场景里的不稳定反馈
- 我们已经把这个风险点补了 hardening 修复
换句话说,这次 issue 真正承接的是:
- 一条历史上确实不够稳的 Windows terminal insertion 链路
- 以及其中一个已经被确认和修补的底层时序风险
### 平台边界 / Platform Scope
- 直接范围:Windows
- 关注层次:`clipboard lifecycle`、`insertion lifecycle`
- terminal 是重点观察目标,但不是唯一可能受影响的慢消费者
- `focus restore` 不是本轮主要根因
### 认领 / Ownership
- owner intent:`@Cooper-X-Oak`
- 当前对应 draft/ready PR:`#160`
## 影响 / Impact
- 影响 Windows terminal 文本输入的稳定性认知
- 会让 `PasteSent` 的用户语义和目标实际表现产生偏差
- 增加“为什么目标没上屏 / 为什么需要手动 Ctrl+V”的排障成本
- 对 Windows insertion 这条核心路径的可信度有直接影响
## 建议接受标准 / Proposed Acceptance Criteria
- [x] 明确 Windows `PasteSent` 与“目标已完成 paste”不是同一语义
- [x] 明确并记录 clipboard restore timing 风险模型
- [x] 完成最小 hardening 修复:
- [x] Windows restore 延后到 `750ms`
- [x] restore 改为异步执行
- [x] 提供隔离时序实验,证明 race 模型成立
- [x] 提供稳定化完整生命周期自动化,覆盖:
- [x] `wt-cmd`
- [x] `wt-powershell`
- [x] `notepad`
- [x] 记录当前环境下的最终结论:
- [x] 历史风险真实存在
- [x] 当前回归未再出现目标吃到旧 clipboard 的结果
- [x] 当前稳定性较历史状态已有改善
## TODO / 不确定项
- 是否需要进一步收紧 `PasteSent` 相关用户文案,避免被理解为“已确认粘贴成功”
- 若后续再收到用户现场反馈,是否需要补充更细的环境标签:
- terminal host / profile
- 输入法状态
- 前台切换时序
建议 issue 标题:`[windows][insertion] 终端旧剪贴板粘贴风险已收敛,当前整链路回归稳定`
@@ -0,0 +1,52 @@
## 摘要
Closes #139
这个 PR 现在从“问题收敛中”推进到“regression review 中”。
本轮已经完成:
- Windows helper-window lifecycle root cause 收敛
- `inactive` 路径的 native hide / non-topmost 收口
- 冷启动最新 debug 包回归
- 人工桌面症状回归:
- click dead zone:通过
- screenshot selectable:通过
- drag stutter:通过
## 修复 / 新增 / 改进
- 对齐 PR 目标:关注 Windows Capsule helper-window lifecycle,而不是单点 dead zone workaround
- 收口 Windows 上 `visible / hidden / inactive / non-participating` 的 Capsule 语义
- 在 backend 上补齐 inactive 后的 native hide 行为,避免 transparent topmost helper window lingering
- 新增 lifecycle contract / smoke 辅助脚本,帮助后续回归持续验证
- 与 [issue-139-capsule-lifecycle.md](/D:/Users/cooper/Practice-Project/202604/openless/docs/github-tracking/issue-139-capsule-lifecycle.md) 保持同一问题口径
## 兼容
- 不包含:Capsule geometry / rounded corner / titlebar frame 纯视觉适配
- 不包含:QA hotkey / selection ask 输入源逻辑
- 对现有用户 / 本地环境 / 构建流程的影响:只聚焦 lifecycle 主线,不扩大到 UI polish 线
## 测试计划
- [x] 命令:`node openless-all/app/scripts/windows-lifecycle-contract.test.mjs`
- [x] 结果:通过
- [x] 证据路径:本地命令输出
- [x] 命令:`npm run build`
- [x] 结果:通过
- [x] 证据路径:本地命令输出
- [x] 命令:`cargo check --manifest-path openless-all/app/src-tauri/Cargo.toml`
- [x] 结果:通过
- [x] 证据路径:本地命令输出
- [x] 命令:`powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-runtime-smoke.ps1`
- [x] 结果:通过 launch / hotkey installed baseline
- [x] 证据路径:本地命令输出
- [x] 命令:人工桌面回归(latest debug cold start -> dictation start/stop)
- [x] 结果:点击 / 截图 / 拖拽三项全部通过
- [x] 证据路径:当前线程回归记录
关联 issue 建议标题:`[ui][windows] Capsule 隐藏后仍参与系统交互`
@@ -0,0 +1,46 @@
## 摘要
Closes #98
References #143
这条 PR 已经不再只是 tracking 入口,而是承接本轮 Windows startup lifecycle 主线修复的实际变更。
当前结论:
- `visible / ready` 脱钩的主问题已收敛
- 冷启动入口已从 backend immediate show 调整为 frontend-managed first show
- 最新人工回归反馈是:启动过程基本流畅,人眼很难再分辨出明显的一闪
- `#143` 现在更适合作为已收敛的 first-paint 症状票引用,而不是继续作为主 closure 目标
## 修复 / 新增 / 改进
- 收口 Windows 启动阶段的 first-show ownership
- 在 `checking -> ready` 之间加入明确 gate,避免正式壳层在 startup transient phase 过早暴露
- 增加冷启动测试脚本,默认优先拉最新 debug build,并区分:
- frontend-managed first show
- backend immediate show(仅调试用)
- 增加 startup lifecycle contract test,锁住 hidden-on-create 与 readiness gate 语义
## 兼容
- 不包含:主窗口圆角 / 外框 / titlebar frame 等纯视觉适配
- 不包含:更细粒度 startup latency 优化
- 对现有用户 / 本地环境 / 构建流程的影响:聚焦 startup lifecycle 主线,不扩张到 UI polish 线
## 测试计划
- [x] 命令:`node openless-all/app/scripts/windows-startup-lifecycle-contract.test.mjs`
- [x] 结果:通过
- [x] 证据路径:本地命令输出
- [x] 命令:`npm run build`
- [x] 结果:通过
- [x] 证据路径:本地命令输出
- [x] 命令:`powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-cold-start.ps1 -PreferDebug -ShowMain`
- [x] 结果:能够走 frontend-managed first show
- [x] 证据路径:本地命令输出
- [x] 命令:冷启动截图与人工主观回归
- [x] 结果:首屏体验明显改善,当前主观反馈为“几乎没有问题,人眼很难分辨”
- [x] 证据路径:`artifacts-cold-start-screenshot.png`、`artifacts-cold-start-screenshot-8s.png`、`artifacts-cold-start-screenshot-front-managed.png` 与当前线程回归记录
@@ -0,0 +1,52 @@
## Summary
Closes #154
This draft PR now serves as a near-closure tracking anchor for the Windows dual-hotkey-source problem.
Current conclusion:
- Windows dictation / QA lifecycle should be owned by the backend low-level hook
- renderer / window-local forwarding should not keep driving the same lifecycle
- future work here should focus on regression review, not on reopening the architecture without new evidence
## Current Status
- keep draft for now
- close to regression review
- not a parked native-strategy problem like #153
## Scope
- source ownership
- lifecycle precedence
- mixed-source risk on Windows
Out of scope:
- helper-window drag semantics
- main window / radius / appearance work
- broad hotkey adapter rewrites without new evidence
## Key Finding
```text
One lifecycle needs one owner source.
On Windows, that owner source should be the backend low-level hook.
```
## Evidence
- QA hotkey and follow-up flow remain healthy after ownership tightening
- no evidence from this repair track suggests the renderer-forward path should remain a co-owner
## Next Step
- use this PR as the place to summarize regression evidence
- only reopen architecture scope if new mixed-source failures appear
## Validation Plan
- [x] Manual verification: QA hotkey flow remains functional
- [x] Manual verification: lifecycle tightening did not break follow-up QA
- [ ] Regression review: confirm no new mixed-source phase drift evidence
@@ -0,0 +1,82 @@
## 摘要
Closes #159
这个 PR 承接的是 Windows terminal insertion 链路的一次收敛修复:
- 历史上 terminal 场景出现过“不能自动上屏、需要手动 `Ctrl+V`”的用户反馈
- 本地测试也曾观察到一次“目标最终拿到旧剪贴板”的现象
- 本轮排查确认了其中一处真实存在的底层风险:clipboard restore timing
因此,这个 PR 的目标不是去声称“当前存在一个稳定复现的 terminal bug”,而是:
- 修补一处已经被确认的 Windows insertion 时序风险
- 把整条链路的回归覆盖补齐
- 把最终结论收敛到可审阅、可维护的状态
## 修复 / 新增 / 改进
- Windows clipboard restore 从 `150ms` 提高到 `750ms`
- clipboard restore 改为后台线程执行,不阻塞插入返回
- 新增 Windows clipboard timing smoke,用于验证慢消费者 race
- 新增完整生命周期自动化脚本,覆盖:
- `wt-cmd`
- `wt-powershell`
- `notepad`
- 稳定化自动化入口:
- 通过 WebView2 remote debugging 连接主页面
- 通过 Tauri invoke 驱动 `start_dictation` / `stop_dictation`
- 新增 debug-only transcript override
- 仅用于桌面音频路由不稳定时继续覆盖真实 insertion 尾链
- 调整目标读回方式:
- terminal 走 UIA 读取 `TermControl`
- notepad 走 UIA 直接读取文本
- 更新调查文档与 tracking 文档
## 兼容
- 正常用户路径不依赖 debug transcript override
- debug transcript override 仅在 `debug_assertions` / test 构建下参与
- Linux restore delay 保持原行为
- 不涉及 UI/视觉顺手修改
- 不涉及 QA hotkey / selection 主线逻辑修改
## 测试计划
- [x] `cargo fmt --all`
- [x] `cargo check --lib`
- [x] `python -m py_compile openless-all/app/scripts/windows-openless-lifecycle-e2e.py`
- [x] `windows-real-asr-insertion-smoke.ps1` 脚本解析通过
- [x] 隔离时序实验:
- [x] 快消费者 + `150ms`
- [x] 慢消费者 + `150ms`
- [x] 慢消费者 + `750ms`
- [x] 完整生命周期自动化:
- [x] `wt-cmd`
- [x] `wt-powershell`
- [x] `notepad`
- [x] 证据路径:
- `docs/2026-05-02-windows-terminal-clipboard-restore-investigation.md`
- `docs/github-tracking/issue-windows-terminal-clipboard-restore.md`
## 当前结论
- 历史上的 Windows terminal insertion 不稳定反馈是真实的
- 本轮排查确认并修补了一处真实存在的 clipboard restore timing 风险
- 稳定化完整生命周期自动化下:
- `wt-cmd` 通过
- `wt-powershell` 通过
- `notepad` 通过
- 当前环境中,目标最终都拿到本次 `finalText`,未再出现旧 clipboard 上屏
因此,这个 PR 的技术定位应当是:
- 针对历史不稳定现象的一次 hardening 修复
- 外加完整的回归覆盖补强
## 剩余风险
- `750ms` 仍然是启发式保护,不是目标确认式握手
- 如果未来再出现 terminal 现场问题,更可能是更窄的环境因子,而不是当前这条主链路已经明确存在的稳定故障
建议 PR 标题:`fix(windows): 延后剪贴板恢复并补齐插入回归覆盖`
@@ -0,0 +1,27 @@
# Issue #154 Tracking
Scope: Windows dictation lifecycle driven by two hotkey event sources
Current stage:
- This branch is a draft PR placeholder.
- No runtime fix is included yet.
- The goal is to lock down source ownership and failure modes before changing behavior.
Problem statement:
- Windows currently has both OS-level low-level keyboard hook input and focused-window renderer forwarding.
- macOS/Linux do not have the same dual-source lifecycle driver.
- Shared dedupe exists, but source precedence is not yet a first-class contract.
Implementation target to converge before coding:
- Decide whether Windows should have one owner source or an explicit precedence model.
- Define expected behavior for mixed-source press/release ordering.
- Add testable scenarios for hold mode, toggle mode, and focus switching.
Non-goals in this draft:
- No hotkey adapter rewrite yet
- No input-stack refactor without agreed target contract
- No unrelated QA hotkey changes
@@ -0,0 +1,23 @@
# Issue #142 Placeholder / 占位
## 中文摘要
本 PR 是 issue #142 的 draft 占位,专门跟踪 Windows Capsule 变形、失真与尺寸错位问题。
当前只保留问题边界、几何证据和后续修复准入条件,不引入业务逻辑改动。
## Scope / 范围
- Capsule native window bounds
- visual pill metrics
- badge position
- Windows DPI / transparent window clipping
## Evidence / 证据入口
- `openless-all/app/src-tauri/src/lib.rs`
- `openless-all/app/src/components/Capsule.tsx`
- `openless-all/app/src/lib/capsuleLayout.ts`
## Merge Rule / 合并规则
- 仅当 issue #142 的几何对齐与 Windows smoke 验证完成后才允许从 draft 转为 ready。
@@ -0,0 +1,23 @@
# Issue #143 Placeholder / 占位
## 中文摘要
本 PR 是 issue #143 的 draft 占位,专门跟踪 Windows 冷启动前几秒加载异常、闪烁与 ready 前展示错位问题。
当前只记录时序边界、现象入口和后续修复出口,不引入无关功能修改。
## Scope / 范围
- visible / ready timing
- first stable paint
- startup shell exposure
- Windows cold start UX
## Evidence / 证据入口
- `openless-all/app/src-tauri/tauri.conf.json`
- `openless-all/app/src/App.tsx`
- `openless-all/app/src/components/FloatingShell.tsx`
## Merge Rule / 合并规则
- 仅当 issue #143 的启动时序统一且完成 Windows cold-start smoke 后才允许从 draft 转为 ready。
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "openless-app",
"private": true,
"version": "1.2.11",
"version": "1.2.12",
"type": "module",
"scripts": {
"dev": "vite",
@@ -0,0 +1,128 @@
param(
[string]$ExePath = "",
[int]$TimeoutSeconds = 15
)
$ErrorActionPreference = "Stop"
if ([string]::IsNullOrWhiteSpace($ExePath)) {
$appRoot = (Resolve-Path (Join-Path $PSScriptRoot "..")).Path
$ExePath = Join-Path $appRoot "src-tauri\target\debug\openless.exe"
}
if (-not (Test-Path $ExePath)) {
throw "OpenLess executable not found: $ExePath"
}
$logPath = Join-Path $env:LOCALAPPDATA "OpenLess\Logs\openless.log"
Remove-Item -LiteralPath $logPath -Force -ErrorAction SilentlyContinue
Get-Process openless -ErrorAction SilentlyContinue | Stop-Process -Force
Add-Type @"
using System;
using System.Runtime.InteropServices;
public static class OpenLessCapsuleProbe {
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
public static extern IntPtr FindWindowW(string lpClassName, string lpWindowName);
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool IsWindowVisible(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern void keybd_event(byte bVk, byte bScan, int dwFlags, UIntPtr dwExtraInfo);
public const int KEYEVENTF_EXTENDEDKEY = 0x0001;
public const int KEYEVENTF_KEYUP = 0x0002;
}
"@
function Wait-LogPattern($Pattern, $TimeoutSeconds) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
if ((Test-Path $logPath) -and ((Get-Content -Raw $logPath) -match $Pattern)) {
return $true
}
Start-Sleep -Milliseconds 200
}
return $false
}
function Send-KeyEdge([byte]$Vk, [bool]$KeyUp) {
$flags = [OpenLessCapsuleProbe]::KEYEVENTF_EXTENDEDKEY
if ($KeyUp) {
$flags = $flags -bor [OpenLessCapsuleProbe]::KEYEVENTF_KEYUP
}
[OpenLessCapsuleProbe]::keybd_event($Vk, 0x1D, $flags, [UIntPtr]::Zero)
}
function Get-CapsuleWindowState() {
$hwnd = [OpenLessCapsuleProbe]::FindWindowW($null, "OpenLess Capsule")
if ($hwnd -eq [IntPtr]::Zero) {
return [pscustomobject]@{
Exists = $false
Visible = $false
Handle = "0x0"
}
}
return [pscustomobject]@{
Exists = $true
Visible = [OpenLessCapsuleProbe]::IsWindowVisible($hwnd)
Handle = ('0x{0:X}' -f $hwnd.ToInt64())
}
}
Write-Host "== Windows capsule lifecycle smoke =="
$env:OPENLESS_ACCEPT_SYNTHETIC_HOTKEY_EVENTS = "1"
$env:OPENLESS_HOTKEY_INJECTION_DRY_RUN = "1"
$process = Start-Process -FilePath $ExePath -WorkingDirectory (Split-Path $ExePath -Parent) -PassThru
try {
if (-not (Wait-LogPattern "hotkey listener installed" $TimeoutSeconds)) {
throw "Hotkey listener did not install within $TimeoutSeconds seconds."
}
Start-Sleep -Milliseconds 500
$before = Get-CapsuleWindowState
Send-KeyEdge 0xA3 $false
Start-Sleep -Milliseconds 120
Send-KeyEdge 0xA3 $true
$startedDryRun = Wait-LogPattern "session started \(hotkey-injection dry-run\)" 5
Start-Sleep -Milliseconds 400
$afterStart = Get-CapsuleWindowState
Send-KeyEdge 0xA3 $false
Start-Sleep -Milliseconds 120
Send-KeyEdge 0xA3 $true
Start-Sleep -Seconds 3
$afterStop = Get-CapsuleWindowState
[pscustomobject]@{
StartedDryRun = $startedDryRun
Before = "$($before.Handle) visible=$($before.Visible)"
AfterStart = "$($afterStart.Handle) visible=$($afterStart.Visible)"
AfterStop = "$($afterStop.Handle) visible=$($afterStop.Visible)"
} | Format-List
if (-not $startedDryRun) {
throw "Dry-run session did not start; cannot verify capsule lifecycle."
}
if (-not $afterStart.Visible) {
throw "Capsule did not become visible during synthetic recording start."
}
if ($afterStop.Visible) {
throw "Capsule is still visible after synthetic stop."
}
Write-Host "[ok] Capsule window is not visible after synthetic stop."
}
finally {
Remove-Item Env:OPENLESS_ACCEPT_SYNTHETIC_HOTKEY_EVENTS -ErrorAction SilentlyContinue
Remove-Item Env:OPENLESS_HOTKEY_INJECTION_DRY_RUN -ErrorAction SilentlyContinue
Get-Process openless -ErrorAction SilentlyContinue | Stop-Process -Force
}
@@ -0,0 +1,47 @@
param(
[int]$DurationSeconds = 20
)
$ErrorActionPreference = "Stop"
Add-Type @"
using System;
using System.Runtime.InteropServices;
public static class OpenLessCapsuleWatch {
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
public static extern IntPtr FindWindowW(string lpClassName, string lpWindowName);
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool IsWindowVisible(IntPtr hWnd);
}
"@
function Get-CapsuleState {
$hwnd = [OpenLessCapsuleWatch]::FindWindowW($null, "OpenLess Capsule")
if ($hwnd -eq [IntPtr]::Zero) {
return "missing"
}
if ([OpenLessCapsuleWatch]::IsWindowVisible($hwnd)) {
return "visible"
}
return "hidden"
}
Write-Host "== Windows capsule watch =="
Write-Host "Watch duration: $DurationSeconds seconds"
Write-Host "Please trigger dictation start/stop now."
$deadline = (Get-Date).AddSeconds($DurationSeconds)
$last = ""
while ((Get-Date) -lt $deadline) {
$state = Get-CapsuleState
if ($state -ne $last) {
Write-Host ("[{0:HH:mm:ss.fff}] capsule={1}" -f (Get-Date), $state)
$last = $state
}
Start-Sleep -Milliseconds 100
}
Write-Host "== Final capsule state: $last =="
@@ -0,0 +1,76 @@
param(
[int]$ConsumerDelayMs = 250,
[int]$RestoreDelayMs = 150,
[string]$InsertedText = "OPENLESS_DICTATED_TEXT",
[string]$PreviousText = "OPENLESS_OLDER_CLIPBOARD"
)
$ErrorActionPreference = "Stop"
Add-Type -AssemblyName System.Windows.Forms
Add-Type -ReferencedAssemblies @("System.Windows.Forms") @"
using System;
using System.Threading;
using System.Windows.Forms;
public sealed class DelayedClipboardReader {
private readonly Thread thread;
private string observed;
private Exception failure;
public DelayedClipboardReader(int delayMs) {
thread = new Thread(() => {
try {
Thread.Sleep(delayMs);
if (Clipboard.ContainsText()) {
observed = Clipboard.GetText();
}
} catch (Exception ex) {
failure = ex;
}
});
thread.SetApartmentState(ApartmentState.STA);
}
public void Start() {
thread.Start();
}
public string JoinAndGetResult() {
thread.Join();
if (failure != null) {
throw failure;
}
return observed;
}
}
"@
function Restore-ClipboardValue($Value) {
if ($null -eq $Value) {
cmd /c "echo off | clip" | Out-Null
return
}
Set-Clipboard -Value $Value
}
$originalClipboard = Get-Clipboard -Raw -ErrorAction SilentlyContinue
try {
Set-Clipboard -Value $InsertedText
$reader = [DelayedClipboardReader]::new($ConsumerDelayMs)
$reader.Start()
Start-Sleep -Milliseconds $RestoreDelayMs
Restore-ClipboardValue $PreviousText
$observedText = $reader.JoinAndGetResult()
$result = [pscustomobject]@{
consumerDelayMs = $ConsumerDelayMs
restoreDelayMs = $RestoreDelayMs
insertedText = $InsertedText
previousText = $PreviousText
observedText = $observedText
matchedInserted = ($observedText -eq $InsertedText)
}
$result | ConvertTo-Json -Compress
} finally {
Restore-ClipboardValue $originalClipboard
}
@@ -0,0 +1,142 @@
param(
[string]$ExePath = "",
[switch]$FreshBuild,
[switch]$PreferDebug,
[switch]$ShowMain,
[switch]$KeepLogs,
[switch]$ForceImmediateShow
)
$ErrorActionPreference = "Stop"
$appRoot = (Resolve-Path (Join-Path $PSScriptRoot "..")).Path
$artifactExe = Join-Path $appRoot ".artifacts\windows-gnu\dev\openless.exe"
$debugExe = Join-Path $appRoot "src-tauri\target\debug\openless.exe"
function Resolve-DefaultExePath {
param(
[string]$ArtifactExe,
[string]$DebugExe,
[switch]$PreferDebug
)
$artifactItem = if (Test-Path $ArtifactExe) { Get-Item $ArtifactExe } else { $null }
$debugItem = if (Test-Path $DebugExe) { Get-Item $DebugExe } else { $null }
if ($PreferDebug -and $debugItem) {
return $debugItem.FullName
}
if ($debugItem -and (-not $artifactItem -or $debugItem.LastWriteTime -gt $artifactItem.LastWriteTime)) {
return $debugItem.FullName
}
if ($artifactItem) {
return $artifactItem.FullName
}
if ($debugItem) {
return $debugItem.FullName
}
return $ArtifactExe
}
if ($FreshBuild) {
Push-Location $appRoot
try {
Write-Host "Building frontend dist..."
npm run build
Write-Host "Building backend debug exe..."
cargo build --manifest-path src-tauri/Cargo.toml
} finally {
Pop-Location
}
}
if ([string]::IsNullOrWhiteSpace($ExePath)) {
$ExePath = Resolve-DefaultExePath -ArtifactExe $artifactExe -DebugExe $debugExe -PreferDebug:$PreferDebug
}
if (-not (Test-Path $ExePath)) {
throw "OpenLess executable not found: $ExePath"
}
if (-not $env:SystemDrive) {
$env:SystemDrive = "C:"
}
if (-not $env:ProgramData) {
$env:ProgramData = Join-Path $env:SystemDrive "ProgramData"
}
$logPath = Join-Path $env:LOCALAPPDATA "OpenLess\Logs\openless.log"
$workingDirectory = Split-Path $ExePath -Parent
Add-Type @"
using System;
using System.Runtime.InteropServices;
public static class OpenLessWindow {
[DllImport("user32.dll")]
public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
}
"@
function Show-OpenLessWindow($Process) {
if ($null -eq $Process -or $Process.MainWindowHandle -eq 0) {
return $false
}
[OpenLessWindow]::ShowWindow($Process.MainWindowHandle, 9) | Out-Null
[OpenLessWindow]::SetForegroundWindow($Process.MainWindowHandle) | Out-Null
return $true
}
Write-Host "== Windows cold start =="
Write-Host "ExePath: $ExePath"
$running = Get-Process openless -ErrorAction SilentlyContinue
if ($running) {
Write-Host "Stopping existing OpenLess processes..."
$running | Stop-Process -Force
Start-Sleep -Milliseconds 600
}
if (-not $KeepLogs -and (Test-Path $logPath)) {
Remove-Item -LiteralPath $logPath -Force -ErrorAction SilentlyContinue
Write-Host "Cleared log: $logPath"
}
$env:PATH = "$env:USERPROFILE\.cargo\bin;$env:USERPROFILE\scoop\persist\rustup\.cargo\bin;$env:USERPROFILE\scoop\apps\rustup\current\.cargo\bin;$env:USERPROFILE\scoop\apps\mingw\current\bin;$env:PATH"
$useImmediateShow = $ShowMain -and $ForceImmediateShow
if ($useImmediateShow) {
$env:OPENLESS_SHOW_MAIN_ON_START = "1"
}
try {
$process = Start-Process -FilePath $ExePath -WorkingDirectory $workingDirectory -PassThru
} finally {
if ($useImmediateShow) {
Remove-Item Env:OPENLESS_SHOW_MAIN_ON_START -ErrorAction SilentlyContinue
}
}
Write-Host "Started OpenLess cold. pid=$($process.Id)"
Write-Host "Log path: $logPath"
if ($ShowMain) {
if ($ForceImmediateShow) {
Write-Host "Mode: backend immediate show (debug-only, may expose startup shell)"
} else {
Write-Host "Mode: frontend-managed first show (recommended for startup contract testing)"
$deadline = (Get-Date).AddSeconds(15)
while ((Get-Date) -lt $deadline) {
Start-Sleep -Milliseconds 250
$current = Get-Process -Id $process.Id -ErrorAction SilentlyContinue
if (Show-OpenLessWindow $current) {
Write-Host "OpenLess main window became visible and was brought to foreground. pid=$($current.Id)"
break
}
}
}
} else {
Write-Host "Mode: startup-default visibility"
}
@@ -0,0 +1,309 @@
import argparse
import json
import os
import subprocess
import sys
import tempfile
import time
import uuid
from pathlib import Path
import win32clipboard
import win32con
from pywinauto import Application, Desktop, keyboard
from websockets.sync.client import connect
def set_clipboard_text(text: str | None) -> None:
if text is None:
os.system("echo off | clip")
return
win32clipboard.OpenClipboard()
try:
win32clipboard.EmptyClipboard()
win32clipboard.SetClipboardText(text)
finally:
win32clipboard.CloseClipboard()
def get_clipboard_text() -> str | None:
try:
win32clipboard.OpenClipboard()
return win32clipboard.GetClipboardData(win32con.CF_UNICODETEXT)
except Exception:
return None
finally:
try:
win32clipboard.CloseClipboard()
except Exception:
pass
class CdpClient:
def __init__(self, websocket_url: str):
self.ws = connect(websocket_url)
self.next_id = 1
self._send("Runtime.enable")
def _send(self, method: str, params: dict | None = None) -> dict:
msg_id = self.next_id
self.next_id += 1
payload = {"id": msg_id, "method": method}
if params is not None:
payload["params"] = params
self.ws.send(json.dumps(payload))
while True:
message = json.loads(self.ws.recv())
if message.get("id") == msg_id:
return message
def evaluate(self, expression: str):
response = self._send(
"Runtime.evaluate",
{
"expression": expression,
"returnByValue": True,
"awaitPromise": True,
},
)
if "exceptionDetails" in response.get("result", {}):
raise RuntimeError(json.dumps(response["result"]["exceptionDetails"], ensure_ascii=False))
return response["result"]["result"].get("value")
def invoke(self, command: str, args: dict | None = None):
args_json = json.dumps(args or {}, ensure_ascii=False)
expression = f"""
(async () => {{
const value = await window.__TAURI__.core.invoke({json.dumps(command)}, {args_json});
return JSON.stringify(value ?? null);
}})()
"""
raw = self.evaluate(expression)
return json.loads(raw) if raw else None
def close(self):
self.ws.close()
def cdp_page_ws(port: int) -> str:
deadline = time.time() + 20
last_targets = []
while time.time() < deadline:
try:
response = subprocess.run(
[
"powershell",
"-NoProfile",
"-Command",
f"(Invoke-WebRequest -UseBasicParsing http://127.0.0.1:{port}/json/list).Content",
],
capture_output=True,
text=True,
check=True,
)
targets = json.loads(response.stdout)
last_targets = [target.get("url", "") for target in targets]
for target in targets:
url = target.get("url", "")
if url.startswith("http://tauri.localhost") and "?window=" not in url:
return target["webSocketDebuggerUrl"]
except Exception:
pass
time.sleep(0.5)
raise RuntimeError(f"Main Tauri page target was not found. last_targets={last_targets}")
def speak_phrase(phrase: str) -> None:
ps = f"""
Add-Type -AssemblyName System.Speech
$speaker = New-Object System.Speech.Synthesis.SpeechSynthesizer
$speaker.Rate = -1
$speaker.Volume = 100
$speaker.Speak(@'
{phrase}
'@)
"""
subprocess.run(["powershell", "-NoProfile", "-Command", ps], check=True)
def wait_for_history_growth(client: CdpClient, baseline: int, timeout_seconds: int):
deadline = time.time() + timeout_seconds
while time.time() < deadline:
history = client.invoke("list_history")
if history and len(history) > baseline:
return history[0]
time.sleep(0.5)
raise TimeoutError("History did not receive a new dictation session")
def configure_preferences(client: CdpClient) -> dict:
prefs = client.invoke("get_settings")
previous = json.loads(json.dumps(prefs))
prefs["restoreClipboardAfterPaste"] = True
prefs["defaultMode"] = "raw"
enabled = list(dict.fromkeys((prefs.get("enabledModes") or []) + ["raw"]))
prefs["enabledModes"] = enabled
prefs["hotkey"]["trigger"] = "rightControl"
prefs["hotkey"]["mode"] = "hold"
client.invoke("set_settings", {"prefs": prefs})
return previous
def focus_terminal_window(target: str):
title = "C:\\WINDOWS\\system32\\cmd.exe" if target == "wt-cmd" else "Windows PowerShell"
win = None
for candidate in Desktop(backend="uia").windows():
try:
if candidate.class_name() == "CASCADIA_HOSTING_WINDOW_CLASS" and candidate.window_text() == title:
win = candidate
break
except Exception:
continue
if win is None:
raise RuntimeError(f"terminal window not found for title={title}")
win.set_focus()
time.sleep(0.5)
keyboard.send_keys("{ESC}")
time.sleep(0.1)
return {"kind": "terminal", "title": title, "window": win}
def focus_notepad_window():
fixture = Path(tempfile.gettempdir()) / f"openless-lifecycle-{uuid.uuid4().hex}.txt"
fixture.write_text("", encoding="utf-8")
app = Application(backend="uia").start(f"notepad.exe {fixture}")
time.sleep(2.5)
title = f"{fixture.name} - Notepad"
win = Desktop(backend="uia").window(title=title)
doc = next(d for d in win.descendants() if d.class_name() == "RichEditD2DPT")
doc.set_focus()
time.sleep(0.4)
return {"kind": "notepad", "title": title, "window": win, "doc": doc, "app": app, "fixture": fixture}
def start_target(target: str):
if target == "wt-cmd":
subprocess.run(["wt.exe", "new-tab", "cmd.exe"], check=True)
time.sleep(2.5)
return focus_terminal_window(target)
if target == "wt-powershell":
subprocess.run(["wt.exe", "new-tab", "powershell.exe"], check=True)
time.sleep(2.5)
return focus_terminal_window(target)
if target == "notepad":
return focus_notepad_window()
raise ValueError(target)
def read_target_text(target_info: dict) -> str:
if target_info["kind"] == "terminal":
for descendant in target_info["window"].descendants():
if descendant.class_name() == "TermControl":
return descendant.window_text()
return ""
return target_info["doc"].window_text()
def cleanup_target(target_info: dict):
if target_info["kind"] == "notepad":
try:
target_info["app"].kill()
except Exception:
pass
try:
target_info["fixture"].unlink(missing_ok=True)
except Exception:
pass
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--exe-path", required=True)
parser.add_argument("--target", choices=["notepad", "wt-cmd", "wt-powershell"], required=True)
parser.add_argument("--phrase", default="openless terminal regression success")
parser.add_argument("--injected-transcript-text", default="")
parser.add_argument("--remote-debugging-port", type=int, default=9223)
parser.add_argument("--timeout-seconds", type=int, default=120)
args = parser.parse_args()
debug_transcript_path = ""
if args.injected_transcript_text.strip():
debug_transcript_path = str(Path(tempfile.gettempdir()) / "openless-debug-transcript-e2e.txt")
Path(debug_transcript_path).write_text(args.injected_transcript_text, encoding="utf-8")
launch_ps = f"""
$env:OPENLESS_SHOW_MAIN_ON_START='1'
$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS='--remote-debugging-port={args.remote_debugging_port}'
$env:OPENLESS_DEBUG_TRANSCRIPT_FILE='{debug_transcript_path}'
$proc = Start-Process -FilePath '{args.exe_path}' -PassThru
$proc.Id
"""
app_process = subprocess.run(
["powershell", "-NoProfile", "-Command", launch_ps],
check=True,
capture_output=True,
text=True,
)
app_pid = int(app_process.stdout.strip().splitlines()[-1])
client = None
target_info = None
previous_settings = None
previous_clipboard = get_clipboard_text()
clipboard_sentinel = f"OPENLESS_OLD_CLIPBOARD_SENTINEL_{uuid.uuid4().hex}"
try:
time.sleep(5)
client = CdpClient(cdp_page_ws(args.remote_debugging_port))
previous_settings = configure_preferences(client)
history = client.invoke("list_history") or []
baseline_count = len(history)
target_info = start_target(args.target)
set_clipboard_text(clipboard_sentinel)
client.invoke("start_dictation")
time.sleep(1.0)
if args.injected_transcript_text.strip():
time.sleep(1.0)
else:
speak_phrase(args.phrase)
time.sleep(0.8)
client.invoke("stop_dictation")
latest = wait_for_history_growth(client, baseline_count, args.timeout_seconds)
target_text = read_target_text(target_info)
result = {
"target": args.target,
"phrase": args.phrase,
"historyFinalText": latest.get("finalText"),
"historyRawTranscript": latest.get("rawTranscript"),
"insertStatus": latest.get("insertStatus"),
"targetContainsFinalText": bool(latest.get("finalText") and latest["finalText"] in target_text),
"targetContainsClipboardSentinel": clipboard_sentinel in target_text,
"targetTextTail": target_text[-400:],
}
print(json.dumps(result, ensure_ascii=False, indent=2))
finally:
if client and previous_settings is not None:
try:
client.invoke("set_settings", {"prefs": previous_settings})
except Exception:
pass
if client:
client.close()
cleanup_target(target_info) if target_info else None
set_clipboard_text(previous_clipboard)
subprocess.run(
["powershell", "-NoProfile", "-Command", f"Stop-Process -Id {app_pid} -Force -ErrorAction SilentlyContinue"],
check=False,
)
if debug_transcript_path:
try:
Path(debug_transcript_path).unlink(missing_ok=True)
except Exception:
pass
if __name__ == "__main__":
main()
@@ -1,10 +1,11 @@
param(
[string]$ExePath = "",
[ValidateSet("notepad", "browser")]
[ValidateSet("notepad", "browser", "wt-cmd", "wt-powershell")]
[string]$Target = "notepad",
[string]$Phrase = "OpenLess Windows real regression",
[int]$TimeoutSeconds = 120,
[int]$VirtualKey = 0xA3,
[int]$VirtualKey = 0xA2,
[string]$InjectedTranscriptText = "",
[int]$ManualSpeechSeconds = 8,
[switch]$ManualSpeech,
[switch]$DebugHotkeyEvents
@@ -83,6 +84,14 @@ function Write-TextUtf8($Path, $Text) {
[System.IO.File]::WriteAllText($Path, $Text, [System.Text.UTF8Encoding]::new($false))
}
function Restore-ClipboardValue($Value) {
if ($null -eq $Value) {
cmd /c "echo off | clip" | Out-Null
return
}
Set-Clipboard -Value $Value
}
function Set-HoldHotkeyPreference($Path) {
$previous = Read-TextUtf8 $Path
if ([string]::IsNullOrWhiteSpace($previous)) {
@@ -94,9 +103,9 @@ function Set-HoldHotkeyPreference($Path) {
$prefs | Add-Member -NotePropertyName hotkey -NotePropertyValue ([pscustomobject]@{})
}
if ($null -eq $prefs.hotkey.PSObject.Properties["trigger"]) {
$prefs.hotkey | Add-Member -NotePropertyName trigger -NotePropertyValue "rightControl"
$prefs.hotkey | Add-Member -NotePropertyName trigger -NotePropertyValue "leftControl"
} else {
$prefs.hotkey.trigger = "rightControl"
$prefs.hotkey.trigger = "leftControl"
}
if ($null -eq $prefs.hotkey.PSObject.Properties["mode"]) {
$prefs.hotkey | Add-Member -NotePropertyName mode -NotePropertyValue "hold"
@@ -109,6 +118,11 @@ function Set-HoldHotkeyPreference($Path) {
if ($null -eq $prefs.showCapsule) { $prefs | Add-Member -NotePropertyName showCapsule -NotePropertyValue $true }
if ($null -eq $prefs.activeAsrProvider) { $prefs | Add-Member -NotePropertyName activeAsrProvider -NotePropertyValue "volcengine" }
if ($null -eq $prefs.activeLlmProvider) { $prefs | Add-Member -NotePropertyName activeLlmProvider -NotePropertyValue "ark" }
if ($null -eq $prefs.restoreClipboardAfterPaste) {
$prefs | Add-Member -NotePropertyName restoreClipboardAfterPaste -NotePropertyValue $true
} else {
$prefs.restoreClipboardAfterPaste = $true
}
Write-TextUtf8 $Path ($prefs | ConvertTo-Json -Depth 8)
return $previous
}
@@ -184,6 +198,23 @@ function Release-Hotkey {
Send-KeyEdge $VirtualKey $true $true
}
function Ensure-TargetFocused($TargetInfo) {
if ($null -eq $TargetInfo) {
return $false
}
if ($TargetInfo.TargetTitle) {
$wshell = New-Object -ComObject WScript.Shell
if ($wshell.AppActivate($TargetInfo.TargetTitle)) {
Start-Sleep -Milliseconds 500
return $true
}
}
if ($null -ne $TargetInfo.Process) {
return (Focus-Window $TargetInfo.Process)
}
return $false
}
function Focus-Window($Process) {
if ($null -eq $Process -or $Process.MainWindowHandle -eq 0) {
return $false
@@ -280,12 +311,96 @@ function Stop-BrowserProfileProcesses($ProfilePath) {
function Start-InputTarget($TargetName) {
$startedAt = Get-Date
if ($TargetName -eq "notepad") {
Start-Process notepad.exe | Out-Null
$process = Wait-ProcessWindow "notepad" $startedAt 15
if (-not (Focus-Window $process)) {
throw "Notepad window could not be focused."
$fixture = Join-Path $env:TEMP "openless-notepad-input-fixture.txt"
Write-TextUtf8 $fixture ""
$process = Start-Process notepad.exe -ArgumentList $fixture -PassThru
Start-Sleep -Seconds 2
$title = "openless-notepad-input-fixture.txt - Notepad"
$activateScript = @"
import sys, time, win32com.client
title = sys.argv[1]
shell = win32com.client.Dispatch('WScript.Shell')
deadline = time.time() + 10
while time.time() < deadline:
if shell.AppActivate(title):
print('activated')
raise SystemExit(0)
time.sleep(0.2)
raise SystemExit(1)
"@
$activatePath = Join-Path $env:TEMP "openless-activate-notepad.py"
Write-TextUtf8 $activatePath $activateScript
try {
python $activatePath $title | Out-Null
} finally {
Remove-Item -LiteralPath $activatePath -Force -ErrorAction SilentlyContinue
}
Start-Sleep -Milliseconds 800
return [pscustomobject]@{
Process = $process
FixturePath = $fixture
ProfilePath = $null
TargetTitle = $title
TargetPid = $process.Id
TargetKind = "notepad"
}
}
if ($TargetName -in @("wt-cmd", "wt-powershell")) {
$wt = Get-Command wt.exe -ErrorAction SilentlyContinue
if ($null -eq $wt) {
throw "wt.exe was not found."
}
$profile = if ($TargetName -eq "wt-cmd") { "cmd.exe" } else { "powershell.exe" }
Start-Process -FilePath $wt.Source -ArgumentList @("new-tab", $profile) | Out-Null
Start-Sleep -Seconds 2
$title = if ($TargetName -eq "wt-cmd") { "C:\WINDOWS\system32\cmd.exe" } else { "Windows PowerShell" }
$activateScript = @"
import sys, time, win32com.client
title = sys.argv[1]
shell = win32com.client.Dispatch('WScript.Shell')
deadline = time.time() + 10
while time.time() < deadline:
if shell.AppActivate(title):
print('activated')
raise SystemExit(0)
time.sleep(0.2)
raise SystemExit(1)
"@
$activatePath = Join-Path $env:TEMP "openless-activate-target.py"
Write-TextUtf8 $activatePath $activateScript
try {
python $activatePath $title | Out-Null
} finally {
Remove-Item -LiteralPath $activatePath -Force -ErrorAction SilentlyContinue
}
$handleLookup = @"
import sys
from pywinauto import Desktop
title = sys.argv[1]
for window in Desktop(backend='uia').windows():
if window.class_name() == 'CASCADIA_HOSTING_WINDOW_CLASS' and window.window_text() == title:
print(window.handle)
raise SystemExit(0)
raise SystemExit(1)
"@
$handlePath = Join-Path $env:TEMP "openless-terminal-handle.py"
Write-TextUtf8 $handlePath $handleLookup
try {
$targetHandle = [int](python -X utf8 $handlePath $title)
} finally {
Remove-Item -LiteralPath $handlePath -Force -ErrorAction SilentlyContinue
}
Start-Sleep -Milliseconds 800
return [pscustomobject]@{
Process = $null
FixturePath = $null
ProfilePath = $null
TargetTitle = $title
TargetHandle = $targetHandle
TargetKind = "terminal"
}
return [pscustomobject]@{ Process = $process; FixturePath = $null; ProfilePath = $null }
}
$browserPath = Resolve-BrowserPath
@@ -307,7 +422,93 @@ function Start-InputTarget($TargetName) {
throw "Browser window could not be focused."
}
Start-Sleep -Seconds 1
return [pscustomobject]@{ Process = $process; FixturePath = $fixture; ProfilePath = $profilePath }
return [pscustomobject]@{ Process = $process; FixturePath = $fixture; ProfilePath = $profilePath; TargetKind = "browser" }
}
function Read-TargetContent($TargetInfo, $TargetName) {
if ($TargetName -eq "notepad") {
$readbackScript = @"
import sys
from pywinauto import Desktop
pid = int(sys.argv[1])
title = sys.argv[2]
out = sys.argv[3]
windows = [w for w in Desktop(backend='uia').windows() if getattr(w, 'process_id', lambda: None)() == pid]
win = None
for candidate in windows:
if candidate.window_text() == title:
win = candidate
break
if win is None and windows:
win = windows[0]
if win is None:
raise SystemExit(2)
for descendant in win.descendants():
if descendant.class_name() == 'RichEditD2DPT':
value = descendant.window_text()
open(out, 'w', encoding='utf-8').write(value)
raise SystemExit(0)
raise SystemExit(1)
"@
$readbackPath = Join-Path $env:TEMP "openless-notepad-readback.py"
$outputPath = Join-Path $env:TEMP "openless-notepad-readback.txt"
Write-TextUtf8 $readbackPath $readbackScript
try {
Remove-Item -LiteralPath $outputPath -Force -ErrorAction SilentlyContinue
python -X utf8 $readbackPath $TargetInfo.TargetPid $TargetInfo.TargetTitle $outputPath | Out-Null
Start-Sleep -Milliseconds 400
if (Test-Path $outputPath) {
return Get-Content -Raw -Encoding UTF8 $outputPath
}
return $null
} finally {
Remove-Item -LiteralPath $readbackPath -Force -ErrorAction SilentlyContinue
Remove-Item -LiteralPath $outputPath -Force -ErrorAction SilentlyContinue
}
}
if ($TargetName -eq "browser") {
Focus-Window $TargetInfo.Process | Out-Null
Start-Sleep -Milliseconds 400
Send-CtrlChord 0x41
Start-Sleep -Milliseconds 200
Send-CtrlChord 0x43
Start-Sleep -Milliseconds 400
return Get-Clipboard -Raw -ErrorAction SilentlyContinue
}
if ($TargetName -in @("wt-cmd", "wt-powershell")) {
$readbackScript = @"
import sys
from pywinauto import Desktop
handle = int(sys.argv[1])
out = sys.argv[2]
win = Desktop(backend='uia').window(handle=handle)
for descendant in win.descendants():
if descendant.class_name() == 'TermControl':
open(out, 'w', encoding='utf-8').write(descendant.window_text())
raise SystemExit(0)
raise SystemExit(1)
"@
$readbackPath = Join-Path $env:TEMP "openless-terminal-readback.py"
$outputPath = Join-Path $env:TEMP "openless-terminal-readback.txt"
Write-TextUtf8 $readbackPath $readbackScript
try {
Remove-Item -LiteralPath $outputPath -Force -ErrorAction SilentlyContinue
python -X utf8 $readbackPath $TargetInfo.TargetHandle $outputPath | Out-Null
if (Test-Path $outputPath) {
return Get-Content -Raw -Encoding UTF8 $outputPath
}
return $null
} finally {
Remove-Item -LiteralPath $readbackPath -Force -ErrorAction SilentlyContinue
Remove-Item -LiteralPath $outputPath -Force -ErrorAction SilentlyContinue
}
}
return $null
}
function Send-CtrlChord($Vk) {
@@ -338,6 +539,14 @@ $historyPath = Join-Path $env:APPDATA "OpenLess\history.json"
$preferencesPath = Join-Path $env:APPDATA "OpenLess\preferences.json"
$baselineCount = Get-HistoryCount $historyPath
$previousPreferences = Set-HoldHotkeyPreference $preferencesPath
$previousClipboard = Get-Clipboard -Raw -ErrorAction SilentlyContinue
$clipboardSentinel = "OPENLESS_OLD_CLIPBOARD_SENTINEL_$(Get-Date -Format 'yyyyMMddHHmmssfff')"
Restore-ClipboardValue $clipboardSentinel
$debugTranscriptPath = $null
if (-not [string]::IsNullOrWhiteSpace($InjectedTranscriptText)) {
$debugTranscriptPath = Join-Path $env:TEMP "openless-debug-transcript.txt"
Write-TextUtf8 $debugTranscriptPath $InjectedTranscriptText
}
Get-Process openless -ErrorAction SilentlyContinue | Stop-Process -Force
Remove-Item -LiteralPath $logPath -Force -ErrorAction SilentlyContinue
@@ -348,12 +557,16 @@ $env:OPENLESS_ACCEPT_SYNTHETIC_HOTKEY_EVENTS = "1"
if ($DebugHotkeyEvents) {
$env:OPENLESS_DEBUG_HOTKEY_EVENTS = "1"
}
if ($debugTranscriptPath) {
$env:OPENLESS_DEBUG_TRANSCRIPT_FILE = $debugTranscriptPath
}
try {
$openless = Start-Process -FilePath $ExePath -WorkingDirectory (Split-Path $ExePath -Parent) -PassThru
} finally {
Remove-Item Env:OPENLESS_SHOW_MAIN_ON_START -ErrorAction SilentlyContinue
Remove-Item Env:OPENLESS_ACCEPT_SYNTHETIC_HOTKEY_EVENTS -ErrorAction SilentlyContinue
Remove-Item Env:OPENLESS_DEBUG_HOTKEY_EVENTS -ErrorAction SilentlyContinue
Remove-Item Env:OPENLESS_DEBUG_TRANSCRIPT_FILE -ErrorAction SilentlyContinue
}
$inputTarget = $null
@@ -364,9 +577,18 @@ try {
$inputTarget = Start-InputTarget $Target
Press-Hotkey
if (-not (Wait-LogPattern $logPath "\[hotkey\] Windows trigger pressed" 10)) {
throw "Windows low-level hook did not observe the right Control press."
$observedPress = $false
for ($attempt = 1; $attempt -le 3 -and -not $observedPress; $attempt++) {
Ensure-TargetFocused $inputTarget | Out-Null
Press-Hotkey
$observedPress = Wait-LogPattern $logPath "\[hotkey\] Windows trigger pressed" 4
if (-not $observedPress) {
Release-Hotkey
Start-Sleep -Milliseconds 500
}
}
if (-not $observedPress) {
throw "Windows low-level hook did not observe the synthetic Control press."
}
if (-not (Wait-LogPattern $logPath "\[coord\] session started" 30)) {
throw "OpenLess recording session did not start."
@@ -399,18 +621,15 @@ try {
throw "Expected Windows insertStatus pasteSent after guarded foreground restore, got '$($latest.insertStatus)'."
}
Focus-Window $inputTarget.Process | Out-Null
Start-Sleep -Milliseconds 400
Send-CtrlChord 0x41
Start-Sleep -Milliseconds 200
Send-CtrlChord 0x43
Start-Sleep -Milliseconds 400
$targetText = Get-Clipboard -Raw -ErrorAction SilentlyContinue
$targetText = Read-TargetContent $inputTarget $Target
if ([string]::IsNullOrWhiteSpace($targetText)) {
throw "$Target clipboard readback is empty after Ctrl+A/C."
throw "$Target readback is empty."
}
if (-not $targetText.Contains($latest.finalText)) {
if ($targetText.Contains($clipboardSentinel)) {
throw "$Target readback contains the pre-dictation clipboard sentinel instead of latest finalText."
}
throw "$Target readback does not contain latest finalText; insertion was not proven at the target caret."
}
@@ -422,7 +641,7 @@ try {
if ($null -ne $inputTarget) {
if ($inputTarget.ProfilePath) {
Stop-BrowserProfileProcesses $inputTarget.ProfilePath
} else {
} elseif ($null -ne $inputTarget.Process) {
Stop-Process -Id $inputTarget.Process.Id -Force -ErrorAction SilentlyContinue
}
if ($inputTarget.FixturePath) {
@@ -438,6 +657,10 @@ try {
} else {
Write-TextUtf8 $preferencesPath $previousPreferences
}
Restore-ClipboardValue $previousClipboard
if ($debugTranscriptPath) {
Remove-Item -LiteralPath $debugTranscriptPath -Force -ErrorAction SilentlyContinue
}
}
Write-Host "Real ASR + insertion fallback smoke ($Target) passed."
@@ -0,0 +1,39 @@
import { readFile } from 'node:fs/promises';
function assertEqual(actual, expected, name) {
if (actual !== expected) {
throw new Error(`${name}: expected ${expected}, got ${actual}`);
}
}
function assertMatch(source, pattern, name) {
if (!pattern.test(source)) {
throw new Error(`${name}: pattern ${pattern} not found`);
}
}
const raw = await readFile(new URL('../src-tauri/tauri.conf.json', import.meta.url), 'utf-8');
const config = JSON.parse(raw);
const mainWindow = config.app.windows.find(window => window.label === 'main');
const appTsx = await readFile(new URL('../src/App.tsx', import.meta.url), 'utf-8');
if (!mainWindow) {
throw new Error('main window config missing');
}
assertEqual(mainWindow.visible, false, 'main window should stay hidden until startup contract allows first show');
assertMatch(
appTsx,
/const \[gate, setGate\] = useState<Gate>\(isTauri \? 'checking' : 'ready'\);/,
'desktop app should start in checking gate before claiming ready',
);
assertMatch(
appTsx,
/if \(os === 'win' && gate === 'checking'\) return;/,
'windows should not show the main shell while startup gate is still checking',
);
assertMatch(
appTsx,
/const pollHotkeyStatus = async \(\) => \{[\s\S]*?if \(status\.state !== 'starting'\) \{[\s\S]*?setGate\('ready'\);/m,
'windows startup should wait for hotkey status to leave the starting phase before entering ready',
);
@@ -0,0 +1,184 @@
param(
[ValidateSet("notepad", "powershell", "cmd", "wt")]
[string]$Target = "powershell",
[int]$RestoreDelayMs = 150,
[int]$PasteSettleDelayMs = 900
)
$ErrorActionPreference = "Stop"
Add-Type @"
using System;
using System.Runtime.InteropServices;
public static class ClipboardRestoreSmokeWin32 {
[DllImport("user32.dll")]
public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern void keybd_event(byte bVk, byte bScan, int dwFlags, UIntPtr dwExtraInfo);
public const int KEYEVENTF_EXTENDEDKEY = 0x0001;
public const int KEYEVENTF_KEYUP = 0x0002;
}
"@
function Send-KeyEdge($Vk, $KeyUp, $Extended = $false) {
$flags = 0
if ($Extended) {
$flags = $flags -bor [ClipboardRestoreSmokeWin32]::KEYEVENTF_EXTENDEDKEY
}
if ($KeyUp) {
$flags = $flags -bor [ClipboardRestoreSmokeWin32]::KEYEVENTF_KEYUP
}
$scanCode = if ($Vk -eq 0xA2 -or $Vk -eq 0xA3) { 0x1D } else { 0 }
[ClipboardRestoreSmokeWin32]::keybd_event([byte]$Vk, [byte]$scanCode, $flags, [UIntPtr]::Zero)
}
function Send-CtrlChord($Vk) {
Send-KeyEdge 0xA2 $false $false
Start-Sleep -Milliseconds 70
Send-KeyEdge $Vk $false $false
Start-Sleep -Milliseconds 70
Send-KeyEdge $Vk $true $false
Start-Sleep -Milliseconds 70
Send-KeyEdge 0xA2 $true $false
}
function Send-EnterKey {
Send-KeyEdge 0x0D $false $false
Start-Sleep -Milliseconds 70
Send-KeyEdge 0x0D $true $false
}
function Wait-ProcessWindow($ProcessName, $After, $TimeoutSeconds) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
$candidate = Get-Process $ProcessName -ErrorAction SilentlyContinue |
Where-Object { $_.StartTime -ge $After -and $_.MainWindowHandle -ne 0 } |
Sort-Object StartTime -Descending |
Select-Object -First 1
if ($null -ne $candidate) {
return $candidate
}
Start-Sleep -Milliseconds 300
}
return $null
}
function Focus-Window($Process) {
if ($null -eq $Process -or $Process.MainWindowHandle -eq 0) {
throw "Target window is unavailable."
}
[ClipboardRestoreSmokeWin32]::ShowWindow($Process.MainWindowHandle, 9) | Out-Null
[ClipboardRestoreSmokeWin32]::SetForegroundWindow($Process.MainWindowHandle) | Out-Null
Start-Sleep -Milliseconds 500
}
function Start-TargetWindow($TargetName) {
$startedAt = Get-Date
switch ($TargetName) {
"notepad" {
Start-Process notepad.exe | Out-Null
return Wait-ProcessWindow "notepad" $startedAt 15
}
"powershell" {
Start-Process powershell.exe -ArgumentList "-NoLogo" | Out-Null
return Wait-ProcessWindow "powershell" $startedAt 15
}
"cmd" {
Start-Process cmd.exe | Out-Null
return Wait-ProcessWindow "cmd" $startedAt 15
}
"wt" {
$wt = Get-Command wt.exe -ErrorAction SilentlyContinue
if ($null -eq $wt) {
throw "wt.exe was not found."
}
Start-Process $wt.Source -ArgumentList "new-tab", "powershell.exe", "-NoLogo" | Out-Null
return Wait-ProcessWindow "WindowsTerminal" $startedAt 20
}
}
}
function Wait-FileText($Path, $Expected, $TimeoutSeconds) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
if (Test-Path $Path) {
$content = Get-Content -Raw -Encoding UTF8 $Path
if ($content.Contains($Expected)) {
return $true
}
}
Start-Sleep -Milliseconds 200
}
return $false
}
function Restore-ClipboardValue($Value) {
if ($null -eq $Value) {
cmd /c "echo off | clip" | Out-Null
return
}
Set-Clipboard -Value $Value
}
$marker = "OPENLESS_CLIPBOARD_RESTORE_OK"
$outputPath = Join-Path $env:TEMP "openless-clipboard-restore-$Target.txt"
Remove-Item -LiteralPath $outputPath -Force -ErrorAction SilentlyContinue
$previousClipboard = Get-Clipboard -Raw -ErrorAction SilentlyContinue
switch ($Target) {
"notepad" {
$payload = $marker
}
"cmd" {
$payload = "echo $marker > `"$outputPath`""
}
default {
$payload = "Set-Content -Path `"$outputPath`" -Value `"$marker`""
}
}
$targetProcess = $null
try {
$targetProcess = Start-TargetWindow $Target
if ($null -eq $targetProcess) {
throw "Failed to start target window: $Target"
}
Focus-Window $targetProcess
Set-Clipboard -Value $payload
Start-Sleep -Milliseconds 150
Send-CtrlChord 0x56
Start-Sleep -Milliseconds $RestoreDelayMs
Restore-ClipboardValue $previousClipboard
if ($Target -eq "notepad") {
Start-Sleep -Milliseconds $PasteSettleDelayMs
Send-CtrlChord 0x41
Start-Sleep -Milliseconds 120
Send-CtrlChord 0x43
Start-Sleep -Milliseconds 200
$result = Get-Clipboard -Raw -ErrorAction SilentlyContinue
if (-not $result.Contains($marker)) {
throw "Notepad readback did not contain the pasted marker."
}
} else {
Start-Sleep -Milliseconds 120
Send-EnterKey
if (-not (Wait-FileText $outputPath $marker 6)) {
throw "Terminal target did not execute the pasted command before clipboard restore."
}
}
Write-Host "[ok] target=$Target restoreDelayMs=$RestoreDelayMs"
} finally {
Restore-ClipboardValue $previousClipboard
if ($null -ne $targetProcess) {
Stop-Process -Id $targetProcess.Id -Force -ErrorAction SilentlyContinue
}
}
@@ -6,12 +6,105 @@ function assertEqual(actual, expected, name) {
}
}
function assertMatch(source, pattern, name) {
if (!pattern.test(source)) {
throw new Error(`${name}: pattern ${pattern} not found`);
}
}
const raw = await readFile(new URL('../src-tauri/tauri.conf.json', import.meta.url), 'utf-8');
const config = JSON.parse(raw);
const capsuleWindow = config.app.windows.find((window) => window.label === 'capsule');
const mainWindow = config.app.windows.find((window) => window.label === 'main');
const libRs = await readFile(new URL('../src-tauri/src/lib.rs', import.meta.url), 'utf-8');
const coordinatorRs = await readFile(new URL('../src-tauri/src/coordinator.rs', import.meta.url), 'utf-8');
const capsuleTsx = await readFile(new URL('../src/components/Capsule.tsx', import.meta.url), 'utf-8');
const capsuleLayoutTs = await readFile(new URL('../src/lib/capsuleLayout.ts', import.meta.url), 'utf-8');
const windowChromeTsx = await readFile(new URL('../src/components/WindowChrome.tsx', import.meta.url), 'utf-8');
const floatingShellTsx = await readFile(new URL('../src/components/FloatingShell.tsx', import.meta.url), 'utf-8');
if (!capsuleWindow) {
throw new Error('capsule window config missing');
}
if (!mainWindow) {
throw new Error('main window config missing');
}
assertEqual(capsuleWindow.width, 220, 'windows capsule config keeps translation-capable width baseline');
assertEqual(capsuleWindow.height, 110, 'windows capsule config keeps translation-capable height baseline');
assertEqual(capsuleWindow.transparent, true, 'capsule window should keep transparent visuals');
assertEqual(capsuleWindow.alwaysOnTop, true, 'capsule window should stay above the focused app while recording');
assertEqual(mainWindow.decorations, false, 'windows main window should use only custom titlebar');
assertEqual(mainWindow.visible, false, 'windows main window should stay hidden until the intended first show point');
if (!/function WindowsResizeHandles\(\)/.test(windowChromeTsx)) {
throw new Error('windows frameless shell should expose explicit resize handles');
}
if (!/startResizeDragging\(direction\)/.test(windowChromeTsx)) {
throw new Error('windows resize handles should delegate edge dragging to Tauri');
}
if (!/borderRadius:\s*'var\(--ol-window-console-radius\)'/.test(floatingShellTsx)) {
throw new Error('floating shell should consume the shared window-console radius');
}
assertMatch(
coordinatorRs,
/let visible = !matches!\(state,\s*CapsuleState::Idle\);/,
'capsule should stay visible until the unified idle hide path runs',
);
assertMatch(
coordinatorRs,
/fn hide_capsule_window_if_present\(\)/,
'windows capsule lifecycle should include an explicit native hide helper',
);
assertMatch(
coordinatorRs,
/ShowWindow\(hwnd, SW_HIDE\)/,
'windows capsule hide helper should force the native window hidden',
);
assertMatch(
coordinatorRs,
/SetWindowPos\([\s\S]*?HWND_NOTOPMOST[\s\S]*?SWP_HIDEWINDOW/m,
'windows capsule hide helper should drop topmost participation when inactive',
);
if (!/export function getCapsuleHostMetrics\(\s*os: OS,\s*translationActive: boolean,\s*\): CapsuleHostMetrics/.test(capsuleLayoutTs)) {
throw new Error('capsule layout should define explicit host metrics separate from the visible pill metrics');
}
if (!/if \(os === 'win'\)\s*\{[\s\S]*?width: 220,[\s\S]*?height: translationActive \? 118 : 84,[\s\S]*?bottomInset: 12,[\s\S]*?badgeGap: 8[\s\S]*?\}/.test(capsuleLayoutTs)) {
throw new Error('windows capsule host metrics should leave room for shadow and badge geometry');
}
if (!/const hostMetrics = getCapsuleHostMetrics\(os,\s*translation\);/.test(capsuleTsx)) {
throw new Error('capsule should derive host metrics from the shared layout contract');
}
if (!/justifyContent:\s*os === 'win' \? 'flex-end' : 'center'/.test(capsuleTsx)) {
throw new Error('windows capsule host should anchor the pill to the bottom instead of centering it inside the larger native host window');
}
if (!/paddingBottom:\s*os === 'win' \? hostMetrics\.bottomInset : 0/.test(capsuleTsx)) {
throw new Error('windows capsule host should respect the shared bottom inset');
}
if (!/bottom:\s*`\$\{hostMetrics\.bottomInset \+ metrics\.height \+ hostMetrics\.badgeGap\}px`/.test(capsuleTsx)) {
throw new Error('windows translation badge should anchor from the shared host inset instead of a fixed center-based offset');
}
if (!/#\[cfg\(target_os = "windows"\)\][\s\S]*?width: 220\.0[\s\S]*?height: if translation_active \{ 118\.0 \} else \{ 84\.0 \}[\s\S]*?bottom_inset: 12\.0,/.test(libRs)) {
throw new Error('windows runtime capsule bounds should leave room for the native shadow while keeping a fixed visual pill');
}
if (!/#\[cfg\(target_os = "windows"\)\]\s*\{\s*52\.0\s*\}/.test(libRs)) {
throw new Error('windows capsule visual pill height should stay at 52px');
}
if (!/window\.set_size\(LogicalSize::new\(bounds\.width, bounds\.height\)\)\?/.test(libRs)) {
throw new Error('capsule positioning should resync runtime size with the computed layout');
}
if (!/let _ = window\.hide\(\);/.test(coordinatorRs)) {
throw new Error('capsule should be hidden once it leaves active states');
}
@@ -7,6 +7,7 @@ import { fileURLToPath } from 'node:url';
const target = process.env.OPENLESS_UPDATE_TARGET;
const arch = process.env.OPENLESS_UPDATE_ARCH;
const repo = process.env.OPENLESS_UPDATE_REPO || 'appergb/openless';
const mirrorBaseUrl = process.env.OPENLESS_UPDATE_MIRROR_BASE_URL || 'https://fastgit.cc/https://github.com';
if (!target || !arch) {
throw new Error('OPENLESS_UPDATE_TARGET and OPENLESS_UPDATE_ARCH are required');
@@ -52,12 +53,20 @@ if (!existsSync(signaturePath)) {
const assetName = basename(artifact);
const manifestName = `latest-${target}-${arch}.json`;
const mirrorManifestName = `latest-${target}-${arch}-mirror.json`;
const githubAssetUrl = `https://github.com/${repo}/releases/latest/download/${assetName}`;
const mirrorAssetUrl = `${mirrorBaseUrl.replace(/\/$/, '')}/${repo}/releases/latest/download/${assetName}`;
const manifest = {
version: packageJson.version,
pub_date: new Date().toISOString(),
url: `https://github.com/${repo}/releases/latest/download/${assetName}`,
url: githubAssetUrl,
signature: readFileSync(signaturePath, 'utf8').trim(),
};
const mirrorManifest = {
...manifest,
url: mirrorAssetUrl,
};
writeFileSync(join(bundleDir, manifestName), `${JSON.stringify(manifest, null, 2)}\n`);
console.log(`Wrote ${manifestName} for ${assetName}`);
writeFileSync(join(bundleDir, mirrorManifestName), `${JSON.stringify(mirrorManifest, null, 2)}\n`);
console.log(`Wrote ${manifestName} and ${mirrorManifestName} for ${assetName}`);
+2 -1
View File
@@ -3250,7 +3250,7 @@ dependencies = [
[[package]]
name = "openless"
version = "1.2.11"
version = "1.2.12"
dependencies = [
"anyhow",
"arboard",
@@ -3270,6 +3270,7 @@ dependencies = [
"objc2-foundation 0.2.2",
"once_cell",
"parking_lot",
"raw-window-handle",
"rdev",
"reqwest 0.12.28",
"serde",
+5 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "openless"
version = "1.2.11"
version = "1.2.12"
description = "OpenLess — local voice input that types where your cursor is"
authors = ["OpenLess"]
edition = "2021"
@@ -35,6 +35,7 @@ uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
bytes = "1"
url = "2"
raw-window-handle = "0.6"
# Hotkey + audio + insertion
global-hotkey = "0.6"
@@ -52,8 +53,11 @@ objc2-foundation = "0.2"
objc2-app-kit = "0.2"
[target.'cfg(target_os = "windows")'.dependencies]
raw-window-handle = "0.6"
windows = { version = "0.58", features = [
"Win32_Foundation",
"Win32_Graphics_Dwm",
"Win32_Graphics_Gdi",
"Win32_UI_Shell",
"Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging",
+1 -1
View File
@@ -9,7 +9,7 @@ mod frame;
pub mod volcengine;
pub mod whisper;
pub use volcengine::{VolcengineASRError, VolcengineCredentials, VolcengineStreamingASR};
pub use volcengine::{VolcengineCredentials, VolcengineStreamingASR};
pub use whisper::WhisperBatchASR;
/// Sink for raw 16 kHz / 16-bit / mono PCM bytes coming off the recorder.
+102 -4
View File
@@ -25,11 +25,50 @@ pub fn get_settings(coord: CoordinatorState<'_>) -> UserPreferences {
coord.prefs().get()
}
trait SettingsWriter {
fn write_settings(&self, prefs: UserPreferences) -> Result<(), String>;
fn refresh_dictation_hotkey(&self);
fn refresh_qa_hotkey(&self);
}
impl SettingsWriter for Coordinator {
fn write_settings(&self, prefs: UserPreferences) -> Result<(), String> {
self.prefs().set(prefs).map_err(|e| e.to_string())
}
fn refresh_dictation_hotkey(&self) {
self.update_hotkey_binding();
}
fn refresh_qa_hotkey(&self) {
self.update_qa_hotkey_binding();
}
}
impl SettingsWriter for Arc<Coordinator> {
fn write_settings(&self, prefs: UserPreferences) -> Result<(), String> {
self.prefs().set(prefs).map_err(|e| e.to_string())
}
fn refresh_dictation_hotkey(&self) {
self.update_hotkey_binding();
}
fn refresh_qa_hotkey(&self) {
self.update_qa_hotkey_binding();
}
}
fn persist_settings<T: SettingsWriter>(coord: &T, prefs: UserPreferences) -> Result<(), String> {
coord.write_settings(prefs)?;
coord.refresh_dictation_hotkey();
coord.refresh_qa_hotkey();
Ok(())
}
#[tauri::command]
pub fn set_settings(coord: CoordinatorState<'_>, prefs: UserPreferences) -> Result<(), String> {
coord.prefs().set(prefs).map_err(|e| e.to_string())?;
coord.update_hotkey_binding();
Ok(())
persist_settings(&*coord, prefs)
}
#[tauri::command]
@@ -506,7 +545,33 @@ fn _ensure_snapshot_used(_: CredentialsSnapshot) {}
#[cfg(test)]
mod tests {
use super::{models_url, parse_model_ids};
use super::{models_url, parse_model_ids, persist_settings, SettingsWriter};
use crate::types::{
HotkeyBinding, HotkeyMode, HotkeyTrigger, QaHotkeyBinding, UserPreferences,
};
use std::sync::Mutex;
#[derive(Default)]
struct FakeSettingsWriter {
saved: Mutex<Option<UserPreferences>>,
dictation_refreshes: Mutex<u32>,
qa_refreshes: Mutex<u32>,
}
impl SettingsWriter for FakeSettingsWriter {
fn write_settings(&self, prefs: UserPreferences) -> Result<(), String> {
*self.saved.lock().unwrap() = Some(prefs);
Ok(())
}
fn refresh_dictation_hotkey(&self) {
*self.dictation_refreshes.lock().unwrap() += 1;
}
fn refresh_qa_hotkey(&self) {
*self.qa_refreshes.lock().unwrap() += 1;
}
}
#[test]
fn models_url_accepts_base_or_chat_endpoint() {
@@ -527,4 +592,37 @@ mod tests {
.unwrap();
assert_eq!(models, vec!["a".to_string(), "b".to_string()]);
}
#[test]
fn persist_settings_refreshes_both_hotkey_pipelines() {
let writer = FakeSettingsWriter::default();
let prefs = UserPreferences {
hotkey: HotkeyBinding {
trigger: HotkeyTrigger::RightControl,
mode: HotkeyMode::Toggle,
},
qa_hotkey: Some(QaHotkeyBinding {
primary: ";".to_string(),
modifiers: vec!["ctrl".to_string(), "shift".to_string()],
}),
..Default::default()
};
persist_settings(&writer, prefs.clone()).unwrap();
let saved = writer
.saved
.lock()
.unwrap()
.clone()
.expect("settings saved");
assert_eq!(saved.hotkey.trigger, prefs.hotkey.trigger);
assert_eq!(saved.hotkey.mode, prefs.hotkey.mode);
assert_eq!(
saved.qa_hotkey.unwrap().primary,
prefs.qa_hotkey.unwrap().primary
);
assert_eq!(*writer.dictation_refreshes.lock().unwrap(), 1);
assert_eq!(*writer.qa_refreshes.lock().unwrap(), 1);
}
}
+205 -38
View File
@@ -118,6 +118,10 @@ struct Inner {
qa_asr: Mutex<Option<Arc<VolcengineStreamingASR>>>,
/// QA 用的 Recorder 句柄。
qa_recorder: Mutex<Option<Recorder>>,
/// QA SSE 流取消标志。begin_qa_session 重置为 false;cancel_qa_session 设 true;
/// polish::chat_completion_history_streaming 的 loop 每帧检查,true 时 break loop
/// 避免取消后 LLM 仍 drain HTTP body 烧 token。详见 issue #161。
qa_stream_cancelled: Arc<AtomicBool>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -187,6 +191,7 @@ impl Coordinator {
capsule_layout: Mutex::new(None),
qa_asr: Mutex::new(None),
qa_recorder: Mutex::new(None),
qa_stream_cancelled: Arc::new(AtomicBool::new(false)),
}),
}
}
@@ -220,7 +225,20 @@ impl Coordinator {
}
pub fn stop_qa_hotkey_listener(&self) {
self.inner.qa_hotkey.lock().take();
// QaHotkeyMonitor::drop 在 macOS 底层是 Carbon RemoveEventHotKey,要求主线程。
// RunEvent::Exit 回调不保证在 AppKit 主线程跑,drop 漏到 tokio worker 上会
// 触发 macOS dispatch_assert_queue_fail SIGTRAP。包到 run_on_main_thread 让
// drop 在主线程发生;AppHandle 已 None 时直接 drop(最坏 crash 也是退出时刻)。
// 详见 issue #169。
let app = self.inner.app.lock().clone();
if let Some(app) = app {
let inner = Arc::clone(&self.inner);
let _ = app.run_on_main_thread(move || {
inner.qa_hotkey.lock().take();
});
} else {
self.inner.qa_hotkey.lock().take();
}
}
/// 用户在设置里改了 QA 组合键时调用。先持久化(由 prefs.set 完成),
@@ -516,9 +534,7 @@ fn qa_hotkey_supervisor_loop(inner: Arc<Inner>) {
Err(e) => {
attempts += 1;
if attempts <= 3 || attempts % 10 == 0 {
log::warn!(
"[coord] QA hotkey 第 {attempts} 次注册失败: {e}; 3s 后重试"
);
log::warn!("[coord] QA hotkey 第 {attempts} 次注册失败: {e}; 3s 后重试");
}
std::thread::sleep(std::time::Duration::from_secs(3));
}
@@ -744,18 +760,38 @@ async fn handle_window_hotkey_event(
repeat: bool,
) -> Result<(), String> {
if event_type == "keydown" && key == "Escape" {
cancel_session(inner);
// Esc 路由(issue #161):QA 浮窗可见时优先取消 QA(不动 dictation);
// 否则走 dictation 取消通路。之前无条件 cancel_session 导致 QA 浮窗
// 按 Esc 杀的是 dictation 而 QA 流还在烧 token。
let qa_active = {
let st = inner.qa_state.lock();
st.panel_visible || st.phase != QaPhase::Idle
};
if qa_active {
close_qa_panel(inner);
} else {
cancel_session(inner);
}
return Ok(());
}
#[cfg(not(target_os = "windows"))]
{
let _ = (inner, event_type, key, code, repeat);
return Ok(());
Ok(())
}
#[cfg(target_os = "windows")]
{
if !window_hotkey_fallback_enabled() {
if event_type == "keydown" && !repeat {
log::info!(
"[window-hotkey] ignored because Windows lifecycle owner is the low-level hook"
);
}
return Ok(());
}
let trigger = inner.prefs.get().hotkey.trigger;
if !window_key_matches_trigger(trigger, &key, &code) {
return Ok(());
@@ -781,6 +817,10 @@ async fn handle_window_hotkey_event(
}
}
fn window_hotkey_fallback_enabled() -> bool {
crate::types::HotkeyCapability::current().explicit_fallback_available
}
#[cfg(any(target_os = "windows", test))]
fn window_key_matches_trigger(trigger: crate::types::HotkeyTrigger, key: &str, code: &str) -> bool {
use crate::types::HotkeyTrigger;
@@ -820,7 +860,9 @@ async fn begin_session(inner: &Arc<Inner>) -> Result<(), String> {
}
}
// 翻译模式标志重置;hotkey 监听器在 Shift down 时再 set true。
inner.translation_modifier_seen.store(false, Ordering::SeqCst);
inner
.translation_modifier_seen
.store(false, Ordering::SeqCst);
#[cfg(any(debug_assertions, test))]
if hotkey_injection_dry_run_enabled() {
@@ -1026,6 +1068,33 @@ fn spawn_recorder_error_monitor(inner: &Arc<Inner>, rx: mpsc::Receiver<RecorderE
.ok();
}
/// QA 录音 runtime error 监听器。镜像 `spawn_recorder_error_monitor` 的语义但走 QA
/// 收尾路径(`finish_qa_with_error` 替代 `abort_recording_with_error`)。
/// 用 qa_state.session_id 守卫 stale 事件。详见 issue #168。
fn spawn_qa_recorder_error_monitor(inner: &Arc<Inner>, rx: mpsc::Receiver<RecorderError>) {
let captured_session_id = inner.qa_state.lock().session_id;
let inner = Arc::clone(inner);
std::thread::Builder::new()
.name("openless-qa-recorder-error-monitor".into())
.spawn(move || {
if let Ok(err) = rx.recv() {
let current_session_id = inner.qa_state.lock().session_id;
if captured_session_id != current_session_id {
log::warn!(
"[coord] QA recorder error from stale session {} dropped (current={}, err={})",
captured_session_id,
current_session_id,
err
);
return;
}
log::error!("[coord] QA recorder runtime error: {err}");
finish_qa_with_error(&inner, format!("录音设备异常: {err}"));
}
})
.ok();
}
fn abort_recording_with_error(inner: &Arc<Inner>, message: String) {
let elapsed = {
let mut state = inner.state.lock();
@@ -1192,6 +1261,19 @@ async fn end_session(inner: &Arc<Inner>) -> Result<(), String> {
// ASR 返回空转写护栏(来自 PR #66):写一条 emptyTranscript 失败历史 + 错误胶囊,
// 与 main 上其它 error 路径保持一致(带 schedule_capsule_idle 让胶囊自动消失)。
let mut raw = raw;
#[cfg(any(debug_assertions, test))]
if raw.text.trim().is_empty() {
if let Some(debug_text) = debug_transcript_override_text() {
log::info!(
"[coord] using debug transcript override (chars={})",
debug_text.chars().count()
);
raw.text = debug_text;
}
}
if raw.text.trim().is_empty() {
let session = DictationSession {
id: Uuid::new_v4().to_string(),
@@ -1230,8 +1312,8 @@ async fn end_session(inner: &Arc<Inner>) -> Result<(), String> {
let working_languages = prefs.working_languages.clone();
let front_app = inner.state.lock().front_app.clone();
let translation_target = prefs.translation_target_language.trim().to_string();
let translation_active = inner.translation_modifier_seen.load(Ordering::SeqCst)
&& !translation_target.is_empty();
let translation_active =
inner.translation_modifier_seen.load(Ordering::SeqCst) && !translation_target.is_empty();
let (polished, polish_error) = if translation_active {
log::info!(
"[coord] translation mode → target=\u{300C}{}\u{300D} working={:?} front_app={:?}",
@@ -1411,12 +1493,23 @@ fn hotkey_injection_dry_run_enabled() -> bool {
std::env::var_os("OPENLESS_HOTKEY_INJECTION_DRY_RUN").is_some()
}
fn ensure_microphone_permission(inner: &Arc<Inner>) -> Result<(), String> {
#[cfg(any(debug_assertions, test))]
fn debug_transcript_override_text() -> Option<String> {
let path = std::env::var_os("OPENLESS_DEBUG_TRANSCRIPT_FILE")?;
let text = std::fs::read_to_string(path).ok()?;
let trimmed = text.trim().to_string();
if trimmed.is_empty() {
None
} else {
Some(trimmed)
}
}
fn ensure_microphone_permission(_inner: &Arc<Inner>) -> Result<(), String> {
use crate::permissions::{self, PermissionStatus};
#[cfg(target_os = "windows")]
{
let _ = inner;
if permissions::windows_microphone_access_explicitly_denied() {
return Err("需要麦克风权限,当前状态: Denied".to_string());
}
@@ -1431,11 +1524,10 @@ fn ensure_microphone_permission(inner: &Arc<Inner>) -> Result<(), String> {
return Ok(());
}
let requested = if let Some(app) = inner.app.lock().clone() {
crate::request_microphone_from_foreground(&app)
} else {
permissions::request_microphone()
};
// 听写路径不抢前台焦点:缺 mic 权限时直接请求系统授权,不再先 show_main_window。
// 用户在设置页手动点“请求权限”仍走 request_microphone_from_foreground,那是显式操作。
// 这里若系统不弹框,后续会通过 capsule error 引导用户主动去权限页处理。详见 #166。
let requested = permissions::request_microphone();
if matches!(
requested,
PermissionStatus::Granted | PermissionStatus::NotApplicable
@@ -1638,6 +1730,8 @@ async fn begin_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
state.front_app = capture_frontmost_app();
state.selection = None;
}
// 重置 SSE 取消标志:上一轮可能 set 过的 true 留着会让本轮流式立即 break。
inner.qa_stream_cancelled.store(false, Ordering::SeqCst);
// 抓选区。每轮按 Option 都重新抓一次:用户多轮提问中可以重新选别处文字。
// 浮窗 focus:false,原 app 仍是 frontmost,AX/Cmd+C fallback 都能拿到。
@@ -1702,11 +1796,7 @@ async fn begin_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
*last = Some(now);
}
if let Some(app) = inner_for_level.app.lock().clone() {
let _ = app.emit_to(
"qa",
"qa:level",
serde_json::json!({ "level": level }),
);
let _ = app.emit_to("qa", "qa:level", serde_json::json!({ "level": level }));
}
// 同步把电平推给底部胶囊,让 QA 录音也有跟主听写一致的可视反馈。
emit_capsule(
@@ -1720,8 +1810,11 @@ async fn begin_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
});
match Recorder::start(consumer, level_handler) {
Ok((rec, _runtime_errors)) => {
Ok((rec, runtime_errors)) => {
*inner.qa_recorder.lock() = Some(rec);
// QA 也跟主听写一样监听 cpal runtime error。设备中途消失 / panic 时
// 不能让 QA 永远卡在 Recording 没反馈。详见 issue #168。
spawn_qa_recorder_error_monitor(inner, runtime_errors);
}
Err(e) => {
log::error!("[coord] QA recorder start failed: {e}");
@@ -1778,11 +1871,7 @@ async fn end_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
emit_capsule(inner, CapsuleState::Transcribing, 0.0, 0, None, None);
if let Some(app) = inner.app.lock().clone() {
let _ = app.emit_to(
"qa",
"qa:state",
serde_json::json!({ "kind": "loading" }),
);
let _ = app.emit_to("qa", "qa:state", serde_json::json!({ "kind": "loading" }));
}
if let Some(rec) = inner.qa_recorder.lock().take() {
@@ -1845,10 +1934,14 @@ async fn end_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
}
};
inner.qa_state.lock().messages.push(crate::types::QaChatMessage {
role: "user".to_string(),
content: user_content,
});
inner
.qa_state
.lock()
.messages
.push(crate::types::QaChatMessage {
role: "user".to_string(),
content: user_content,
});
if let Some(app) = inner.app.lock().clone() {
let messages = inner.qa_state.lock().messages.clone();
@@ -1874,8 +1967,17 @@ async fn end_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
// 流式回调:每个 SSE delta 立刻推一帧 qa:state{kind:"answer_delta"} 给前端,
// 浮窗里气泡边收边长。最终的 messages 由 answer 事件统一下发(保证一致性)。
//
// session_id 守卫(issue #161):闭包捕获本会话 id;用户取消 → 关浮窗 → 开新浮窗
// 开新一轮时,旧的 in-flight LLM 流仍可能 emit chunk,必须在 emit 前比对当前
// qa_state.session_id == 捕获 id,否则跳过——避免旧会话的字漏进新气泡。
let captured_session_id = inner.qa_state.lock().session_id;
let inner_for_delta = Arc::clone(inner);
let on_delta = move |chunk: &str| {
let cur_id = inner_for_delta.qa_state.lock().session_id;
if cur_id != captured_session_id {
return; // 旧 session 漏来的 chunk,丢弃
}
if let Some(app) = inner_for_delta.app.lock().clone() {
let _ = app.emit_to(
"qa",
@@ -1888,11 +1990,17 @@ async fn end_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
}
};
// SSE 流取消旗标:cancel_qa_session / close_qa_panel 会 set true,
// polish 的 SSE loop 每帧检查 → break,释放 HTTP body。详见 issue #161。
let cancel_flag = Arc::clone(&inner.qa_stream_cancelled);
let should_cancel = move || cancel_flag.load(Ordering::Relaxed);
let answer = match answer_chat_dispatch(
&messages_for_llm,
&working_languages,
front_app.as_deref(),
on_delta,
should_cancel,
)
.await
{
@@ -1914,10 +2022,14 @@ async fn end_qa_session(inner: &Arc<Inner>) -> Result<(), String> {
return Ok(());
}
inner.qa_state.lock().messages.push(crate::types::QaChatMessage {
role: "assistant".to_string(),
content: answer.clone(),
});
inner
.qa_state
.lock()
.messages
.push(crate::types::QaChatMessage {
role: "assistant".to_string(),
content: answer.clone(),
});
if let Some(app) = inner.app.lock().clone() {
let messages = inner.qa_state.lock().messages.clone();
@@ -2012,6 +2124,10 @@ fn cancel_qa_session(inner: &Arc<Inner>) {
return;
}
inner.qa_state.lock().cancelled = true;
// SSE 流取消旗标——polish::chat_completion_history_streaming 的 loop 每帧检查
// 这个 flag,true 时立即 break 不再 drain HTTP body,避免取消后 LLM 仍烧 token。
// 详见 issue #161。
inner.qa_stream_cancelled.store(true, Ordering::SeqCst);
if let Some(rec) = inner.qa_recorder.lock().take() {
rec.stop();
}
@@ -2026,14 +2142,16 @@ fn cancel_qa_session(inner: &Arc<Inner>) {
log::info!("[coord] QA session cancelled (was {phase:?})");
}
async fn answer_chat_dispatch<F>(
async fn answer_chat_dispatch<F, C>(
messages: &[crate::types::QaChatMessage],
working_languages: &[String],
front_app: Option<&str>,
on_delta: F,
should_cancel: C,
) -> anyhow::Result<String>
where
F: Fn(&str) + Send + Sync,
C: Fn() -> bool + Send + Sync,
{
let api_key = CredentialsVault::get(CredentialAccount::ArkApiKey)?.unwrap_or_default();
if api_key.is_empty() {
@@ -2052,7 +2170,7 @@ where
let config = OpenAICompatibleConfig::new("ark", "Doubao Ark", base_url, api_key, model);
let provider = OpenAICompatibleLLMProvider::new(config);
Ok(provider
.answer_chat_streaming(messages, working_languages, front_app, on_delta)
.answer_chat_streaming(messages, working_languages, front_app, on_delta, should_cancel)
.await?)
}
@@ -2195,6 +2313,14 @@ mod tests {
);
assert!(coordinator.inner.hotkey_trigger_held.load(Ordering::SeqCst));
}
#[test]
fn window_hotkey_fallback_is_disabled_when_no_explicit_fallback_is_advertised() {
assert_eq!(
window_hotkey_fallback_enabled(),
crate::types::HotkeyCapability::current().explicit_fallback_available
);
}
}
fn enabled_phrases(inner: &Arc<Inner>) -> Vec<String> {
@@ -2433,6 +2559,42 @@ fn show_capsule_window_no_activate() -> bool {
false
}
#[cfg(target_os = "windows")]
fn hide_capsule_window_if_present() {
use std::iter::once;
use windows::core::PCWSTR;
use windows::Win32::Foundation::HWND;
use windows::Win32::UI::WindowsAndMessaging::{
FindWindowW, SetWindowPos, ShowWindow, HWND_NOTOPMOST, SWP_HIDEWINDOW, SWP_NOACTIVATE,
SWP_NOMOVE, SWP_NOSIZE, SW_HIDE,
};
let title: Vec<u16> = "OpenLess Capsule".encode_utf16().chain(once(0)).collect();
let hwnd = match unsafe { FindWindowW(PCWSTR::null(), PCWSTR(title.as_ptr())) } {
Ok(hwnd) => hwnd,
Err(_) => return,
};
if hwnd == HWND::default() || hwnd.0.is_null() {
return;
}
let _ = unsafe { ShowWindow(hwnd, SW_HIDE) };
let _ = unsafe {
SetWindowPos(
hwnd,
HWND_NOTOPMOST,
0,
0,
0,
0,
SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE | SWP_HIDEWINDOW,
)
};
}
#[cfg(not(target_os = "windows"))]
fn hide_capsule_window_if_present() {}
fn emit_capsule(
inner: &Arc<Inner>,
state: CapsuleState,
@@ -2454,6 +2616,11 @@ fn emit_capsule(
let show_capsule = inner.prefs.get().show_capsule;
if let Some(window) = app.get_webview_window("capsule") {
// 三平台统一:Done / Cancelled / Error 状态保留 ~1.5s toast
// (schedule_capsule_idle 之后会回 Idle 隐藏)。
// Windows 上 linger 的真实问题(截图选中 / 死区 / 拖拽卡顿)由 #140 加的
// `hide_capsule_window_if_present()` Win32 hard-hide 在 visible=false 分支
// 处理,不依赖把 Done/Cancelled/Error 打成 invisible。详见 PR #140 评论。
let visible = !matches!(state, CapsuleState::Idle);
maybe_position_capsule_bottom_center(inner, &window, payload.translation);
if show_capsule && visible {
@@ -2469,6 +2636,7 @@ fn emit_capsule(
#[cfg(target_os = "macos")]
crate::restore_main_window_key_if_active(&app);
} else {
hide_capsule_window_if_present();
let _ = window.hide();
}
}
@@ -2511,7 +2679,6 @@ fn maybe_position_capsule_bottom_center<R: tauri::Runtime>(
if crate::position_capsule_bottom_center(window, translation_active).is_ok() {
let mut last = inner.capsule_layout.lock();
*last = Some(next);
return;
}
}
+11 -1
View File
@@ -164,7 +164,7 @@ mod platform {
"hotkey hook 启动超时",
run_listen_loop,
)?;
let _ = listener.startup;
listener.startup;
Ok(Box::new(MacHotkeyAdapter {
shared: listener.shared,
}))
@@ -662,6 +662,16 @@ mod platform {
binding: HotkeyBinding,
tx: Sender<HotkeyEvent>,
) -> Result<Box<dyn HotkeyAdapter>, HotkeyInstallError> {
if std::env::var("XDG_SESSION_TYPE")
.ok()
.as_deref()
== Some("wayland")
{
return Err(install_error(
"wayland_unsupported",
"Wayland 暂不支持全局热键,请切到 X11 session 后再试",
));
}
let listener = start_listener_thread(
binding,
tx,
+122 -13
View File
@@ -10,12 +10,23 @@
//! → CGEventPost,跟我们这里完全同源。
//! 3. 其他平台 (Windows/Linux) 仍用 enigo。
#[cfg(not(target_os = "macos"))]
use std::sync::atomic::{AtomicU64, Ordering};
#[cfg(not(target_os = "macos"))]
use std::time::Duration;
#[cfg(not(target_os = "macos"))]
use once_cell::sync::Lazy;
#[cfg(not(target_os = "macos"))]
use parking_lot::Mutex;
use crate::types::InsertStatus;
#[cfg(not(target_os = "macos"))]
const CLIPBOARD_RESTORE_DELAY: Duration = Duration::from_millis(150);
#[cfg(target_os = "windows")]
const CLIPBOARD_RESTORE_DELAY: Duration = Duration::from_millis(750);
#[cfg(all(not(target_os = "macos"), not(target_os = "windows")))]
const CLIPBOARD_RESTORE_DELAY: Duration = Duration::from_millis(750);
pub struct TextInserter;
@@ -78,6 +89,20 @@ struct ClipboardRestorePlan {
previous_text: Option<String>,
}
#[cfg(not(target_os = "macos"))]
#[derive(Debug, Clone)]
struct PendingClipboardRestore {
latest_restore_id: u64,
original_text: Option<String>,
}
#[cfg(not(target_os = "macos"))]
static NEXT_CLIPBOARD_RESTORE_ID: AtomicU64 = AtomicU64::new(1);
#[cfg(not(target_os = "macos"))]
static PENDING_CLIPBOARD_RESTORE: Lazy<Mutex<Option<PendingClipboardRestore>>> =
Lazy::new(|| Mutex::new(None));
fn copy_to_clipboard(text: &str) -> bool {
let mut clipboard = match arboard::Clipboard::new() {
Ok(c) => c,
@@ -131,20 +156,50 @@ fn insert_with_clipboard_restore(text: &str, restore_clipboard_after_paste: bool
}
if restore_clipboard_after_paste {
maybe_restore_clipboard(restore_plan);
schedule_clipboard_restore(restore_plan);
}
// 关掉 → 听写文本留在剪贴板里,simulate_paste 没真正落地时用户能手动 Ctrl+V 找回。
insertion_success_status()
}
#[cfg(not(target_os = "macos"))]
fn maybe_restore_clipboard(plan: ClipboardRestorePlan) {
if plan.previous_text.is_none() {
fn schedule_clipboard_restore(plan: ClipboardRestorePlan) {
let restore_id = NEXT_CLIPBOARD_RESTORE_ID.fetch_add(1, Ordering::SeqCst);
let original_text = {
let mut pending = PENDING_CLIPBOARD_RESTORE.lock();
let original = pending
.as_ref()
.map(|batch| batch.original_text.clone())
.unwrap_or_else(|| plan.previous_text.clone());
*pending = Some(PendingClipboardRestore {
latest_restore_id: restore_id,
original_text: original.clone(),
});
original
};
std::thread::spawn(move || {
restore_clipboard_after_delay(
plan,
original_text,
restore_id,
CLIPBOARD_RESTORE_DELAY,
)
});
}
#[cfg(not(target_os = "macos"))]
fn restore_clipboard_after_delay(
plan: ClipboardRestorePlan,
original_text: Option<String>,
restore_id: u64,
delay: Duration,
) {
std::thread::sleep(delay);
if !is_latest_clipboard_restore(restore_id) {
return;
}
std::thread::sleep(CLIPBOARD_RESTORE_DELAY);
let mut clipboard = match arboard::Clipboard::new() {
Ok(clipboard) => clipboard,
Err(err) => {
@@ -152,6 +207,7 @@ fn maybe_restore_clipboard(plan: ClipboardRestorePlan) {
"[insertion] clipboard re-open failed during restore: {}",
err
);
clear_pending_clipboard_restore(restore_id);
return;
}
};
@@ -167,14 +223,34 @@ fn maybe_restore_clipboard(plan: ClipboardRestorePlan) {
}
};
if !should_restore_clipboard(current_text.as_deref(), &plan.inserted_text) {
return;
if should_restore_clipboard(current_text.as_deref(), &plan.inserted_text) {
if let Some(previous_text) = original_text {
if let Err(err) = clipboard.set_text(previous_text) {
log::warn!("[insertion] clipboard restore failed: {}", err);
}
}
} else {
log::info!(
"[insertion] skip clipboard restore: latest clipboard no longer matches inserted text"
);
}
if let Some(previous_text) = plan.previous_text {
if let Err(err) = clipboard.set_text(previous_text) {
log::warn!("[insertion] clipboard restore failed: {}", err);
}
clear_pending_clipboard_restore(restore_id);
}
#[cfg(not(target_os = "macos"))]
fn is_latest_clipboard_restore(restore_id: u64) -> bool {
matches!(
PENDING_CLIPBOARD_RESTORE.lock().as_ref(),
Some(batch) if batch.latest_restore_id == restore_id
)
}
#[cfg(not(target_os = "macos"))]
fn clear_pending_clipboard_restore(restore_id: u64) {
let mut pending = PENDING_CLIPBOARD_RESTORE.lock();
if matches!(pending.as_ref(), Some(batch) if batch.latest_restore_id == restore_id) {
pending.take();
}
}
@@ -303,6 +379,9 @@ mod macos {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
#[test]
#[cfg(not(target_os = "macos"))]
@@ -317,4 +396,34 @@ mod tests {
));
assert!(!should_restore_clipboard(None, "dictated text"));
}
#[test]
#[cfg(target_os = "windows")]
fn delayed_terminal_paste_must_see_dictated_text_before_clipboard_restore() {
let inserted_text = "dictated text".to_string();
let previous_text = "older clipboard".to_string();
let clipboard = Arc::new(Mutex::new(inserted_text.clone()));
let pasted = Arc::new(Mutex::new(None::<String>));
let clipboard_for_paste = Arc::clone(&clipboard);
let pasted_for_paste = Arc::clone(&pasted);
let reader = thread::spawn(move || {
thread::sleep(Duration::from_millis(250));
let seen = clipboard_for_paste.lock().unwrap().clone();
*pasted_for_paste.lock().unwrap() = Some(seen);
});
thread::sleep(CLIPBOARD_RESTORE_DELAY);
let current_text = Some(clipboard.lock().unwrap().clone());
if should_restore_clipboard(current_text.as_deref(), &inserted_text) {
*clipboard.lock().unwrap() = previous_text;
}
reader.join().unwrap();
assert_eq!(
pasted.lock().unwrap().as_deref(),
Some(inserted_text.as_str())
);
}
}
+207 -25
View File
@@ -105,12 +105,18 @@ pub fn run() {
#[cfg(target_os = "windows")]
{
use window_vibrancy::apply_mica;
if let Err(e) = apply_mica(&main, None) {
log::warn!("[main] mica failed: {e}");
}
// The window starts hidden so Windows native chrome can be disabled before
// the first show; doing this after the native frame is visible is unreliable.
if let Err(e) = main.set_decorations(false) {
log::warn!("[main] disable native decorations failed: {e}");
}
if let Err(e) = apply_mica(&main, None) {
log::warn!("[main] mica failed: {e}");
}
apply_windows_rounded_frame(&main);
}
if let Err(e) = main.show() {
log::warn!("[main] initial show failed: {e}");
}
}
@@ -214,10 +220,16 @@ pub fn run() {
RunEvent::Reopen { .. } => show_main_window(app),
RunEvent::WindowEvent { label, event, .. } => {
if label == "main" {
if let tauri::WindowEvent::CloseRequested { api, .. } = event {
if let tauri::WindowEvent::CloseRequested { ref api, .. } = event {
api.prevent_close();
hide_main_window(app);
}
#[cfg(target_os = "windows")]
if matches!(event, tauri::WindowEvent::Resized(_) | tauri::WindowEvent::ScaleFactorChanged { .. }) {
if let Some(main) = app.get_webview_window("main") {
apply_windows_rounded_frame(&main);
}
}
}
}
RunEvent::Exit => {
@@ -229,6 +241,97 @@ pub fn run() {
});
}
#[cfg(target_os = "windows")]
fn apply_windows_rounded_frame<R: Runtime>(window: &tauri::WebviewWindow<R>) {
use raw_window_handle::{HasWindowHandle, RawWindowHandle};
use windows::Win32::Foundation::{BOOL, HWND, RECT};
use windows::Win32::Graphics::Dwm::{
DwmSetWindowAttribute, DWMWA_BORDER_COLOR, DWMWA_WINDOW_CORNER_PREFERENCE, DWMWCP_ROUND,
};
use windows::Win32::Graphics::Gdi::{CreateRoundRectRgn, SetWindowRgn, HRGN};
use windows::Win32::UI::WindowsAndMessaging::{
GetWindowLongW, GetWindowRect, SetWindowLongW, SetWindowPos, GWL_STYLE, SWP_FRAMECHANGED,
SWP_NOMOVE, SWP_NOSIZE, SWP_NOZORDER, WS_CAPTION, WS_THICKFRAME,
};
let handle = match window.window_handle().map(|h| h.as_raw()) {
Ok(RawWindowHandle::Win32(handle)) => handle,
Ok(other) => {
log::warn!("[main] unexpected raw window handle for DWM frame: {other:?}");
return;
}
Err(e) => {
log::warn!("[main] read raw window handle failed: {e}");
return;
}
};
let hwnd = HWND(handle.hwnd.get() as *mut core::ffi::c_void);
unsafe {
let style = GetWindowLongW(hwnd, GWL_STYLE);
let desired_style = (style | WS_THICKFRAME.0 as i32) & !(WS_CAPTION.0 as i32);
if style != desired_style {
SetWindowLongW(hwnd, GWL_STYLE, desired_style);
if let Err(e) = SetWindowPos(
hwnd,
HWND::default(),
0,
0,
0,
0,
SWP_NOMOVE | SWP_NOSIZE | SWP_NOZORDER | SWP_FRAMECHANGED,
) {
log::warn!("[main] refresh native frame after style update failed: {e}");
}
}
if window.is_maximized().unwrap_or(false) {
let _ = SetWindowRgn(hwnd, HRGN::default(), BOOL(1));
return;
}
let corner_preference = DWMWCP_ROUND;
if let Err(e) = DwmSetWindowAttribute(
hwnd,
DWMWA_WINDOW_CORNER_PREFERENCE,
&corner_preference as *const _ as *const core::ffi::c_void,
std::mem::size_of_val(&corner_preference) as u32,
) {
log::warn!("[main] set DWM rounded corners failed: {e}");
}
// Remove DWM's fallback 1px light border; the React shell draws the visual stroke.
let border_color_none: u32 = 0xFFFFFFFE;
if let Err(e) = DwmSetWindowAttribute(
hwnd,
DWMWA_BORDER_COLOR,
&border_color_none as *const _ as *const core::ffi::c_void,
std::mem::size_of_val(&border_color_none) as u32,
) {
log::warn!("[main] remove DWM border color failed: {e}");
}
let mut rect = RECT::default();
if let Err(e) = GetWindowRect(hwnd, &mut rect) {
log::warn!("[main] read window rect for rounded region failed: {e}");
return;
}
let width = rect.right - rect.left;
let height = rect.bottom - rect.top;
if width <= 0 || height <= 0 {
return;
}
let region = CreateRoundRectRgn(0, 0, width + 1, height + 1, 18, 18);
if region.is_invalid() {
log::warn!("[main] create rounded window region failed");
return;
}
if SetWindowRgn(hwnd, region, BOOL(1)) == 0 {
log::warn!("[main] apply rounded window region failed");
}
}
}
#[tauri::command]
fn restart_app(app: AppHandle) {
// macOS:自动更新会让新装的 .app 带 com.apple.quarantine(无论 Tauri updater
@@ -450,7 +553,7 @@ fn position_qa_window<R: tauri::Runtime>(window: &tauri::WebviewWindow<R>) -> ta
let size = monitor.size();
let logical_w = size.width as f64 / scale;
let logical_h = size.height as f64 / scale;
let capsule_height = capsule_window_size(false).1;
let capsule_height = capsule_height_for_qa();
let x = ((logical_w - QA_WINDOW_WIDTH) / 2.0).max(0.0);
let y = (logical_h
- DOCK_BOTTOM_PADDING_FOR_QA
@@ -514,7 +617,16 @@ pub(crate) fn show_qa_window<R: tauri::Runtime>(app: &AppHandle<R>, content_kind
}
});
}
#[cfg(not(target_os = "macos"))]
#[cfg(target_os = "windows")]
{
if !show_qa_window_no_activate(&window) {
log::warn!("[qa] show_no_activate failed; falling back to window.show()");
if let Err(e) = window.show() {
log::warn!("[qa] show fallback failed: {e}");
}
}
}
#[cfg(all(not(target_os = "macos"), not(target_os = "windows")))]
if let Err(e) = window.show() {
log::warn!("[qa] show failed: {e}");
}
@@ -560,6 +672,41 @@ pub(crate) fn hide_qa_window<R: tauri::Runtime>(app: &AppHandle<R>) {
}
}
#[cfg(target_os = "windows")]
fn show_qa_window_no_activate<R: tauri::Runtime>(window: &tauri::WebviewWindow<R>) -> bool {
use raw_window_handle::{HasWindowHandle, RawWindowHandle};
use windows::Win32::Foundation::HWND;
use windows::Win32::UI::WindowsAndMessaging::{
SetWindowPos, ShowWindow, HWND_TOPMOST, SWP_NOACTIVATE, SWP_NOMOVE, SWP_NOSIZE,
SWP_SHOWWINDOW, SW_SHOWNOACTIVATE,
};
let Ok(handle) = window.window_handle() else {
return false;
};
let RawWindowHandle::Win32(raw) = handle.as_raw() else {
return false;
};
let hwnd = HWND(raw.hwnd.get() as *mut _);
if hwnd.0.is_null() {
return false;
}
let _ = unsafe { ShowWindow(hwnd, SW_SHOWNOACTIVATE) };
let _ = unsafe {
SetWindowPos(
hwnd,
HWND_TOPMOST,
0,
0,
0,
0,
SWP_NOMOVE | SWP_NOSIZE | SWP_SHOWWINDOW | SWP_NOACTIVATE,
)
};
true
}
/// 把 capsule 窗口移到屏幕底部居中,与 Swift `CapsuleWindowController.repositionToBottomCenter` 同效。
/// 留 80pt 给 macOS Dock;Windows 任务栏一般在底部 48pt 以内,整体也合适。
pub(crate) fn position_capsule_bottom_center<R: tauri::Runtime>(
@@ -570,59 +717,94 @@ pub(crate) fn position_capsule_bottom_center<R: tauri::Runtime>(
Some(m) => m,
None => return Ok(()),
};
let (cap_w, cap_h) = capsule_window_size(translation_active);
window.set_size(LogicalSize::new(cap_w, cap_h))?;
let bounds = capsule_window_bounds(translation_active);
window.set_size(LogicalSize::new(bounds.width, bounds.height))?;
let scale = monitor.scale_factor();
let size = monitor.size();
let logical_w = size.width as f64 / scale;
let logical_h = size.height as f64 / scale;
let x = ((logical_w - cap_w) / 2.0).max(0.0);
let y = (logical_h - cap_h - 80.0).max(0.0);
let x = ((logical_w - bounds.width) / 2.0).max(0.0);
let y = (logical_h - capsule_visual_height(translation_active) - 80.0 - bounds.bottom_inset)
.max(0.0);
window.set_position(LogicalPosition::new(x, y))?;
Ok(())
}
fn capsule_window_size(translation_active: bool) -> (f64, f64) {
#[derive(Clone, Copy, Debug, PartialEq)]
struct CapsuleWindowBounds {
width: f64,
height: f64,
bottom_inset: f64,
}
fn capsule_window_bounds(translation_active: bool) -> CapsuleWindowBounds {
#[cfg(target_os = "windows")]
{
let height = if translation_active { 110.0 } else { 52.0 };
(196.0, height)
CapsuleWindowBounds {
width: 220.0,
height: if translation_active { 118.0 } else { 84.0 },
bottom_inset: 12.0,
}
}
#[cfg(not(target_os = "windows"))]
{
let height = if translation_active { 110.0 } else { 42.0 };
(176.0, height)
CapsuleWindowBounds {
width: 176.0,
height: if translation_active { 110.0 } else { 42.0 },
bottom_inset: 0.0,
}
}
}
fn capsule_visual_height(_translation_active: bool) -> f64 {
#[cfg(target_os = "windows")]
{
52.0
}
#[cfg(not(target_os = "windows"))]
{
42.0
}
}
fn capsule_height_for_qa() -> f64 {
capsule_window_size(false).1
capsule_visual_height(false)
}
#[cfg(test)]
mod tests {
use super::{capsule_height_for_qa, capsule_window_size};
use super::{capsule_height_for_qa, capsule_visual_height, capsule_window_bounds};
#[test]
fn capsule_window_size_matches_visible_pill_when_not_translating() {
let (width, height) = capsule_window_size(false);
fn capsule_window_bounds_leave_room_for_windows_shadow() {
let bounds = capsule_window_bounds(false);
#[cfg(target_os = "windows")]
assert_eq!((width, height), (196.0, 52.0));
assert_eq!((bounds.width, bounds.height, bounds.bottom_inset), (220.0, 84.0, 12.0));
#[cfg(not(target_os = "windows"))]
assert_eq!((width, height), (176.0, 42.0));
assert_eq!((bounds.width, bounds.height, bounds.bottom_inset), (176.0, 42.0, 0.0));
}
#[test]
fn capsule_window_size_expands_for_translation_badge() {
let (width, height) = capsule_window_size(true);
fn capsule_window_bounds_expand_for_translation_badge() {
let bounds = capsule_window_bounds(true);
#[cfg(target_os = "windows")]
assert_eq!((width, height), (196.0, 110.0));
assert_eq!((bounds.width, bounds.height, bounds.bottom_inset), (220.0, 118.0, 12.0));
#[cfg(not(target_os = "windows"))]
assert_eq!((width, height), (176.0, 110.0));
assert_eq!((bounds.width, bounds.height, bounds.bottom_inset), (176.0, 110.0, 0.0));
}
#[test]
fn capsule_visual_height_matches_frontend_pill() {
#[cfg(target_os = "windows")]
assert_eq!(capsule_visual_height(true), 52.0);
#[cfg(not(target_os = "windows"))]
assert_eq!(capsule_visual_height(true), 42.0);
}
#[test]
+14 -5
View File
@@ -104,21 +104,23 @@ impl OpenAICompatibleLLMProvider {
/// 最后一条必须是新一轮的 user 提问。第一条 user 消息里如果有选区,调用方应在
/// content 里就把选区原文注入。`on_delta` 在每个 SSE chunk 到达时被调;最终返回
/// 拼好的完整字符串(用于写入 messages 历史)。详见 issue #118 v2。
pub async fn answer_chat_streaming<F>(
pub async fn answer_chat_streaming<F, C>(
&self,
messages: &[QaChatMessage],
working_languages: &[String],
front_app: Option<&str>,
on_delta: F,
should_cancel: C,
) -> Result<String, LLMError>
where
F: Fn(&str) + Send + Sync,
C: Fn() -> bool + Send + Sync,
{
let mut system_prompt = prompts::qa_system_prompt();
if let Some(premise) = context_premise(working_languages, front_app) {
system_prompt = format!("{}\n\n{}", premise, system_prompt);
}
self.chat_completion_history_streaming(&system_prompt, messages, on_delta)
self.chat_completion_history_streaming(&system_prompt, messages, on_delta, should_cancel)
.await
}
@@ -209,14 +211,16 @@ impl OpenAICompatibleLLMProvider {
/// 与 `chat_completion` 同条 HTTP 通路,但开 `stream: true` 并把 SSE chunk 一边
/// 解析、一边通过 `on_delta` 推给调用方(用于实时把答案塞进浮窗气泡)。
/// 最终返回拼好的完整字符串供调用方写入对话历史。
async fn chat_completion_history_streaming<F>(
async fn chat_completion_history_streaming<F, C>(
&self,
system_prompt: &str,
history: &[QaChatMessage],
on_delta: F,
should_cancel: C,
) -> Result<String, LLMError>
where
F: Fn(&str) + Send + Sync,
C: Fn() -> bool + Send + Sync,
{
if self.config.api_key.trim().is_empty() {
return Err(LLMError::MissingCredentials);
@@ -287,6 +291,12 @@ impl OpenAICompatibleLLMProvider {
let mut buffer = String::new();
let mut full_text = String::new();
loop {
// 取消旗标:用户取消 / 关浮窗时立即 break,不再 drain HTTP body。
// 否则 reqwest 会读完整个流(包括 LLM 后续 token)烧 quota。详见 issue #161。
if should_cancel() {
log::info!("[llm] stream cancelled by caller; breaking SSE loop");
break;
}
let chunk_opt = response
.chunk()
.await
@@ -606,10 +616,9 @@ const BOILERPLATE_END_CHARS: &[char] = &['。', ':', ':', ',', ',', '\n'];
fn strip_leading_boilerplate(text: &str) -> &str {
for prefix in LEADING_BOILERPLATE_PREFIXES {
if text.starts_with(prefix) {
if let Some(after_prefix) = text.strip_prefix(prefix) {
// Trim characters after the prefix up to (and including) the first
// sentence-ending punctuation or newline.
let after_prefix = &text[prefix.len()..];
for (idx, c) in after_prefix.char_indices() {
if BOILERPLATE_END_CHARS.contains(&c) {
let cut = prefix.len() + idx + c.len_utf8();
+32 -1
View File
@@ -173,7 +173,7 @@ fn forward_loop(
fn parse_binding(binding: &QaHotkeyBinding) -> Result<HotKey, QaHotkeyError> {
let mut mods = Modifiers::empty();
for raw in &binding.modifiers {
let tag = raw.trim().to_ascii_lowercase();
let tag = normalize_modifier_tag(raw);
let bit = match tag.as_str() {
"cmd" | "command" | "super" | "meta" | "win" => Modifiers::SUPER,
"ctrl" | "control" => Modifiers::CONTROL,
@@ -187,6 +187,17 @@ fn parse_binding(binding: &QaHotkeyBinding) -> Result<HotKey, QaHotkeyError> {
Ok(HotKey::new(Some(mods), code))
}
fn normalize_modifier_tag(raw: &str) -> String {
let tag = raw.trim().to_ascii_lowercase();
#[cfg(target_os = "windows")]
{
if matches!(tag.as_str(), "cmd" | "command") {
return "ctrl".to_string();
}
}
tag
}
/// 把用户配置的主键字符串解析成 keyboard_types::Code。
/// 支持单字符(字母 / 数字 / 符号)+ 常见命名键(F1..F12 / Enter / Tab / Escape / Space)。
fn parse_primary(raw: &str) -> Result<Code, QaHotkeyError> {
@@ -339,4 +350,24 @@ mod tests {
Err(QaHotkeyError::UnsupportedKey(_))
));
}
#[test]
fn cmd_modifier_normalizes_per_platform() {
let binding = QaHotkeyBinding {
primary: ";".into(),
modifiers: vec!["cmd".into(), "shift".into()],
};
let parsed = parse_binding(&binding).expect("binding parses");
#[cfg(target_os = "windows")]
{
assert!(parsed.mods.contains(Modifiers::CONTROL));
assert!(!parsed.mods.contains(Modifiers::SUPER));
}
#[cfg(not(target_os = "windows"))]
{
assert!(parsed.mods.contains(Modifiers::SUPER));
}
}
}
+53 -4
View File
@@ -75,11 +75,24 @@ pub fn capture_selection() -> Option<SelectionContext> {
}
}
// 3. Linux:暂不支持
// 3. Linux:best-effort 读 PRIMARY selection(wl-paste / xclip / xsel)。
#[cfg(all(not(target_os = "macos"), not(target_os = "windows")))]
{
let _ = source_app;
log::info!("[selection] platform unsupported, returning None");
if let Some(text) = linux_selection::read_selected_text() {
let trimmed = text.trim();
if !trimmed.is_empty() {
log::info!(
"[selection] linux primary selection OK ({} chars){}",
trimmed.chars().count(),
source_app
.as_deref()
.map(|a| format!(" front_app={a}"))
.unwrap_or_default()
);
return Some(SelectionContext {
text: truncate_selection(trimmed),
source_app,
});
}
}
None
@@ -177,6 +190,42 @@ fn post_copy_shortcut() -> bool {
windows_paste::send_ctrl_c().is_ok()
}
#[cfg(all(not(target_os = "macos"), not(target_os = "windows")))]
mod linux_selection {
use std::process::Command;
const PRIMARY_SELECTION_COMMANDS: &[(&str, &[&str])] = &[
("wl-paste", &["--primary", "--no-newline"]),
("xclip", &["-o", "-selection", "primary"]),
("xsel", &["--primary", "--output"]),
];
pub fn read_selected_text() -> Option<String> {
for (bin, args) in PRIMARY_SELECTION_COMMANDS {
if let Some(text) = run_capture(bin, args) {
return Some(text);
}
}
log::info!(
"[selection] linux primary selection unavailable (wl-paste/xclip/xsel all failed)"
);
None
}
fn run_capture(bin: &str, args: &[&str]) -> Option<String> {
let output = Command::new(bin).args(args).output().ok()?;
if !output.status.success() {
return None;
}
let text = String::from_utf8(output.stdout).ok()?;
let trimmed = text.trim();
if trimmed.is_empty() {
return None;
}
Some(trimmed.to_string())
}
}
// ─────────────────────────── macOS AX read ───────────────────────────
#[cfg(target_os = "macos")]
+5 -8
View File
@@ -4,18 +4,15 @@ use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
#[derive(Default)]
pub enum PolishMode {
Raw,
#[default]
Light,
Structured,
Formal,
}
impl Default for PolishMode {
fn default() -> Self {
PolishMode::Light
}
}
impl PolishMode {
pub fn display_name(&self) -> &'static str {
@@ -300,7 +297,7 @@ impl HotkeyCapability {
pub fn current() -> Self {
#[cfg(target_os = "macos")]
{
return Self {
Self {
adapter: HotkeyAdapterKind::MacEventTap,
available_triggers: vec![
HotkeyTrigger::RightOption,
@@ -315,7 +312,7 @@ impl HotkeyCapability {
supports_side_specific_modifiers: true,
explicit_fallback_available: false,
status_hint: Some("授权辅助功能后,通常需要完全退出并重新打开 OpenLess。".into()),
};
}
}
#[cfg(target_os = "windows")]
@@ -353,7 +350,7 @@ impl HotkeyCapability {
supports_side_specific_modifiers: true,
explicit_fallback_available: false,
status_hint: Some(
"Linux 仅 best-effort:不同桌面环境 / Wayland 组合可能限制全局热键。".into(),
"Linux 仅 best-effort:X11 可尝试 rdev 监听;Wayland 会明确提示暂不支持全局热键。".into(),
),
}
}
+4 -3
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "OpenLess",
"version": "1.2.11",
"version": "1.2.12",
"identifier": "com.openless.app",
"build": {
"beforeDevCommand": "npm run dev",
@@ -21,12 +21,12 @@
"minWidth": 980,
"minHeight": 640,
"resizable": true,
"decorations": true,
"decorations": false,
"transparent": true,
"shadow": true,
"hiddenTitle": true,
"titleBarStyle": "Overlay",
"visible": true,
"visible": false,
"acceptFirstMouse": true
},
{
@@ -88,6 +88,7 @@
"updater": {
"pubkey": "dW50cnVzdGVkIGNvbW1lbnQ6IG1pbmlzaWduIHB1YmxpYyBrZXk6IDFERUFBODAzNTY0QzMyM0YKUldRL01reFdBNmpxSGE1K0JadlpONXNWTzhJcGZCRGxjUVdIWExNNFJpeUNsSGZwazdlQThhemkK",
"endpoints": [
"https://fastgit.cc/https://github.com/appergb/openless/releases/latest/download/latest-{{target}}-{{arch}}-mirror.json",
"https://github.com/appergb/openless/releases/latest/download/latest-{{target}}-{{arch}}.json"
]
}
+48 -4
View File
@@ -6,6 +6,7 @@ import { detectOS } from './components/WindowChrome';
import {
checkAccessibilityPermission,
checkMicrophonePermission,
getHotkeyStatus,
handleWindowHotkeyEvent,
isTauri,
} from './lib/ipc';
@@ -29,25 +30,68 @@ export function App({ isCapsule, isQa }: AppProps) {
const os = detectOS();
// Windows 启动不应被权限探测阻塞首屏。
const [gate, setGate] = useState<Gate>(isTauri && os !== 'win' ? 'checking' : 'ready');
const [gate, setGate] = useState<Gate>(isTauri ? 'checking' : 'ready');
useEffect(() => {
if (!isTauri) return;
if (os === 'win' && gate === 'checking') return;
let cancelled = false;
requestAnimationFrame(() => {
if (cancelled) return;
import('@tauri-apps/api/window')
.then(({ getCurrentWindow }) => getCurrentWindow().show())
.then(async ({ getCurrentWindow }) => {
const currentWindow = getCurrentWindow();
if (!(await currentWindow.isVisible())) {
await currentWindow.show();
}
})
.catch(error => console.warn('[startup] show main window failed', error));
});
return () => {
cancelled = true;
};
}, []);
}, [gate, os]);
useEffect(() => {
if (!isTauri || os === 'win') return;
if (!isTauri) return;
let cancelled = false;
if (os === 'win') {
// 超时保护:50 次 × 200ms = 10s。hotkey hook 永远 starting(被反作弊 / EDR
// / UAC 拦)时不让 UI 死锁灰屏,过 10s 强 setGate('ready') 让用户进
// Permissions 页看 hotkey_status.lastError 处理。详见 issue #163。
const POLL_INTERVAL_MS = 200;
const POLL_MAX_ATTEMPTS = 50;
const pollHotkeyStatus = async () => {
let attempts = 0;
while (!cancelled && attempts < POLL_MAX_ATTEMPTS) {
attempts += 1;
const status = await getHotkeyStatus();
if (cancelled) return;
if (status.state !== 'starting') {
setGate('ready');
return;
}
await new Promise(resolve => window.setTimeout(resolve, POLL_INTERVAL_MS));
}
if (!cancelled) {
console.warn(
`[startup] hotkey gate timed out after ${POLL_MAX_ATTEMPTS * POLL_INTERVAL_MS}ms; forcing ready so user can reach Permissions page`
);
setGate('ready');
}
};
void pollHotkeyStatus().catch(error => {
console.warn('[startup] hotkey status polling failed', error);
if (!cancelled) {
setGate('ready');
}
});
return () => {
cancelled = true;
};
}
(async () => {
const [a, m] = await Promise.all([
checkAccessibilityPermission(),
@@ -7,6 +7,8 @@ import { useTranslation } from 'react-i18next';
import { isTauri, restartApp } from '../lib/ipc';
import { Btn } from '../pages/_atoms';
const UPDATE_CHECK_TIMEOUT_MS = 15_000;
export type UpdateStatus =
| 'idle'
| 'checking'
@@ -78,7 +80,7 @@ export function useAutoUpdate(): UseAutoUpdate {
return;
}
const { check } = await import('@tauri-apps/plugin-updater');
const next = await check();
const next = await check({ timeout: UPDATE_CHECK_TIMEOUT_MS });
updateRef.current = next;
setVersion(next?.version ?? '');
setStatus(next ? 'available' : 'none');
+10 -3
View File
@@ -2,6 +2,7 @@ import { useEffect, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { detectOS, type OS } from './WindowChrome';
import {
getCapsuleHostMetrics,
getCapsuleMessageLayout,
getCapsulePillMetrics,
} from '../lib/capsuleLayout';
@@ -259,11 +260,13 @@ function Pill({ os, state, level, insertedChars, message, onCancel, onConfirm }:
export function Capsule() {
const { t } = useTranslation();
const os = detectOS();
const metrics = getCapsulePillMetrics(os);
const [translation, setTranslation] = useState<boolean>(false);
const hostMetrics = getCapsuleHostMetrics(os, translation);
const [state, setState] = useState<CapsuleState>(isTauri ? 'idle' : 'recording');
const [level, setLevel] = useState<number>(isTauri ? 0 : 0.6);
const [insertedChars, setInsertedChars] = useState<number>(0);
const [message, setMessage] = useState<string | undefined>();
const [translation, setTranslation] = useState<boolean>(false);
useEffect(() => {
if (!isTauri) return;
@@ -309,7 +312,11 @@ export function Capsule() {
position: 'relative',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
justifyContent: os === 'win' ? 'flex-end' : 'center',
paddingTop: os === 'win'
? Math.max(0, hostMetrics.height - metrics.height - hostMetrics.bottomInset)
: 0,
paddingBottom: os === 'win' ? hostMetrics.bottomInset : 0,
background: 'transparent',
animation: os === 'win' ? 'none' : 'capsule-in .22s cubic-bezier(.2,.9,.3,1.1)',
}}
@@ -324,7 +331,7 @@ export function Capsule() {
left: '50%',
// bottom = 50%(pill 中线)+ pill 半高 21px(capsuleLayout mac=42)+ 8px 间隔。
// 只有翻译徽章可见时才需要额外高度;普通录音/转写状态由后端缩到 pill 本体,避免透明死区。
bottom: 'calc(50% + 21px + 8px)',
bottom: `${hostMetrics.bottomInset + metrics.height + hostMetrics.badgeGap}px`,
transform: 'translateX(-50%)',
pointerEvents: 'none',
}}
@@ -255,7 +255,7 @@ function FloatingShellBody({ os, initialTab, initialSettings }: { os: OS; initia
flex: 1, minWidth: 0,
overflow: 'hidden',
background: 'var(--ol-surface)',
borderRadius: os === 'mac' ? 20 : 14,
borderRadius: 'var(--ol-window-console-radius)',
border: '0.5px solid rgba(0,0,0,0.06)',
boxShadow: '0 1px 0 rgba(255,255,255,0.8) inset, 0 8px 24px -12px rgba(15,17,22,0.10), 0 2px 6px -2px rgba(15,17,22,0.06)',
display: 'flex',
+137 -40
View File
@@ -1,31 +1,38 @@
// WindowChrome.tsx — frosted outer frame + raised inner console pattern.
// The OUTER frame is a translucent shell with a tinted backdrop showing through.
// The INNER content lives in a single raised card that floats above it.
//
// Layout per window:
// ┌─ frosted outer ───────────────────────────────┐
// │ [titlebar] │
// │ ┌─ raised console (white, shadow) ─┐ │
// │ │ sidebar │ main │ │
// │ └──────────────────────────────────┘ │
// │ [icon footer] │
// └───────────────────────────────────────────────┘
import { type CSSProperties, type ReactNode } from 'react';
import { useState, type CSSProperties, type ReactNode } from 'react';
import { useTranslation } from 'react-i18next';
import { getCurrentWindow } from '@tauri-apps/api/window';
export type OS = 'mac' | 'win';
export type OS = 'mac' | 'win' | 'linux';
export function detectOS(): OS {
if (typeof navigator === 'undefined') return 'mac';
const ua = navigator.userAgent || '';
if (/Mac|iPhone|iPad|iPod/.test(ua)) return 'mac';
if (/Windows/.test(ua)) return 'win';
const uaDataPlatform = (
navigator as Navigator & { userAgentData?: { platform?: string } }
).userAgentData?.platform ?? '';
const hints = `${navigator.userAgent || ''} ${navigator.platform || ''} ${uaDataPlatform}`;
if (/Mac|iPhone|iPad|iPod/.test(hints)) return 'mac';
if (/Windows|Win32|Win64/.test(hints)) return 'win';
if (/Linux|X11|Wayland/.test(hints)) return 'linux';
return 'mac';
}
const MAC_TITLEBAR_HEIGHT = 36;
const MAC_SYSTEM_CONTROLS_RESERVED_WIDTH = 80;
export const WIN_TITLEBAR_HEIGHT = 36;
export const WIN_WINDOW_RADIUS = 10;
export const WIN_CONSOLE_RADIUS = 10;
const WIN_RESIZE_EDGE = 6;
const WIN_RESIZE_CORNER = 14;
type ResizeDirection =
| 'East'
| 'North'
| 'NorthEast'
| 'NorthWest'
| 'South'
| 'SouthEast'
| 'SouthWest'
| 'West';
interface WindowChromeProps {
os?: OS;
@@ -34,19 +41,32 @@ interface WindowChromeProps {
height?: number | string;
}
export function WindowChrome({ os = 'mac', title = 'OpenLess', children, height = 800 }: WindowChromeProps) {
export function WindowChrome({
os = 'mac',
title = 'OpenLess',
children,
height = 800,
}: WindowChromeProps) {
const shellRadius = os === 'mac' ? 20 : os === 'win' ? WIN_WINDOW_RADIUS : 14;
const consoleRadius = os === 'mac' ? 20 : os === 'win' ? WIN_CONSOLE_RADIUS : 14;
return (
<div
style={{
'--ol-window-shell-radius': `${shellRadius}px`,
'--ol-window-console-radius': `${consoleRadius}px`,
'--ol-window-titlebar-height': `${os === 'mac' ? MAC_TITLEBAR_HEIGHT : WIN_TITLEBAR_HEIGHT}px`,
width: '100%',
height,
position: 'relative',
borderRadius: os === 'mac' ? 20 : 14,
boxShadow: 'var(--ol-shadow-xl)',
borderRadius: 'var(--ol-window-shell-radius)',
boxShadow: os === 'win'
? '0 22px 54px -28px rgba(15, 17, 22, 0.38), 0 10px 30px -20px rgba(15, 17, 22, 0.22), 0 0 0 0.5px rgba(0, 0, 0, 0.10)'
: 'var(--ol-shadow-xl)',
overflow: 'hidden',
display: 'flex',
flexDirection: 'column',
border: '0.5px solid rgba(0,0,0,.10)',
border: os === 'win' ? '1px solid rgba(255,255,255,0.18)' : '0.5px solid rgba(0,0,0,.10)',
background: `
radial-gradient(120% 80% at 0% 0%, rgba(255,255,255,0.7) 0%, rgba(255,255,255,0) 60%),
radial-gradient(100% 70% at 100% 100%, rgba(37,99,235,0.07) 0%, rgba(37,99,235,0) 55%),
@@ -54,11 +74,10 @@ export function WindowChrome({ os = 'mac', title = 'OpenLess', children, height
`,
backdropFilter: 'blur(40px) saturate(180%)',
WebkitBackdropFilter: 'blur(40px) saturate(180%)',
}}
} as CSSProperties}
>
{os === 'win' && <WinTitleBar title={title} />}
{/* macOS:三色窗口按钮交给系统绘制和定位。这里只保留顶部拖动区,
并避开系统按钮热区,防止拖动层吞掉 close/minimize/zoom 点击。 */}
{os === 'win' && <WindowsResizeHandles />}
{os === 'mac' && (
<div
data-tauri-drag-region
@@ -88,12 +107,14 @@ function WinTitleBar({ title }: WinTitleBarProps) {
return (
<div
style={{
height: 36,
height: WIN_TITLEBAR_HEIGHT,
flexShrink: 0,
display: 'flex',
alignItems: 'stretch',
position: 'relative',
zIndex: 5,
zIndex: 70,
borderBottom: '0.5px solid rgba(0, 0, 0, 0.08)',
background: 'linear-gradient(180deg, rgba(255,255,255,0.20) 0%, rgba(255,255,255,0.08) 100%)',
}}
>
<div
@@ -103,24 +124,105 @@ function WinTitleBar({ title }: WinTitleBarProps) {
<img src="AppIcon.png" alt="" style={{ width: 14, height: 14, borderRadius: 3 }} />
<span style={{ fontSize: 12, color: 'var(--ol-ink-3)', fontWeight: 500 }}>{title}</span>
</div>
<div style={{ display: 'flex' }}>
<button style={winBtnStyle} title={t('windowChrome.minimize')} onClick={() => runWindowsWindowAction('minimize')}>
<div style={{ display: 'flex', pointerEvents: 'auto' }}>
<WinTitleButton title={t('windowChrome.minimize')} action="minimize">
<svg width="10" height="10" viewBox="0 0 10 10"><path d="M0 5h10" stroke="currentColor" strokeWidth="1" /></svg>
</button>
<button style={winBtnStyle} title={t('windowChrome.maximize')} onClick={() => runWindowsWindowAction('toggleMaximize')}>
</WinTitleButton>
<WinTitleButton title={t('windowChrome.maximize')} action="toggleMaximize">
<svg width="10" height="10" viewBox="0 0 10 10"><rect x="0.5" y="0.5" width="9" height="9" stroke="currentColor" strokeWidth="1" fill="none" /></svg>
</button>
<button style={winCloseBtnStyle} title={t('windowChrome.close')} onClick={() => runWindowsWindowAction('close')}>
</WinTitleButton>
<WinTitleButton title={t('windowChrome.close')} action="close" tone="danger">
<svg width="10" height="10" viewBox="0 0 10 10"><path d="M0 0L10 10M10 0L0 10" stroke="currentColor" strokeWidth="1" /></svg>
</button>
</WinTitleButton>
</div>
</div>
);
}
function WindowsResizeHandles() {
const handles: Array<{
direction: ResizeDirection;
cursor: CSSProperties['cursor'];
style: CSSProperties;
}> = [
{ direction: 'North', cursor: 'ns-resize', style: { top: 0, left: WIN_RESIZE_CORNER, right: WIN_RESIZE_CORNER, height: WIN_RESIZE_EDGE } },
{ direction: 'South', cursor: 'ns-resize', style: { bottom: 0, left: WIN_RESIZE_CORNER, right: WIN_RESIZE_CORNER, height: WIN_RESIZE_EDGE } },
{ direction: 'West', cursor: 'ew-resize', style: { top: WIN_RESIZE_CORNER, bottom: WIN_RESIZE_CORNER, left: 0, width: WIN_RESIZE_EDGE } },
{ direction: 'East', cursor: 'ew-resize', style: { top: WIN_RESIZE_CORNER, bottom: WIN_RESIZE_CORNER, right: 0, width: WIN_RESIZE_EDGE } },
{ direction: 'NorthWest', cursor: 'nwse-resize', style: { top: 0, left: 0, width: WIN_RESIZE_CORNER, height: WIN_RESIZE_CORNER } },
{ direction: 'NorthEast', cursor: 'nesw-resize', style: { top: 0, right: 0, width: WIN_RESIZE_CORNER, height: WIN_RESIZE_CORNER } },
{ direction: 'SouthWest', cursor: 'nesw-resize', style: { bottom: 0, left: 0, width: WIN_RESIZE_CORNER, height: WIN_RESIZE_CORNER } },
{ direction: 'SouthEast', cursor: 'nwse-resize', style: { bottom: 0, right: 0, width: WIN_RESIZE_CORNER, height: WIN_RESIZE_CORNER } },
];
return (
<div aria-hidden style={{ position: 'absolute', inset: 0, pointerEvents: 'none', zIndex: 60 }}>
{handles.map(handle => (
<div
key={handle.direction}
onMouseDown={event => {
if (event.button !== 0) return;
event.preventDefault();
event.stopPropagation();
void startResizeDragging(handle.direction);
}}
style={{
position: 'absolute',
pointerEvents: 'auto',
cursor: handle.cursor,
...handle.style,
}}
/>
))}
</div>
);
}
interface WinTitleButtonProps {
title: string;
action: 'minimize' | 'toggleMaximize' | 'close';
tone?: 'default' | 'danger';
children: ReactNode;
}
function WinTitleButton({ title, action, tone = 'default', children }: WinTitleButtonProps) {
const [hovered, setHovered] = useState(false);
const [pressed, setPressed] = useState(false);
const danger = tone === 'danger';
const background = pressed
? danger ? '#c42b1c' : 'rgba(0, 0, 0, 0.12)'
: hovered ? danger ? '#e81123' : 'rgba(0, 0, 0, 0.08)'
: 'transparent';
const color = danger && (hovered || pressed) ? '#fff' : 'var(--ol-ink-3)';
return (
<button
style={{ ...winBtnStyle, background, color }}
title={title}
onClick={() => runWindowsWindowAction(action)}
onMouseEnter={() => setHovered(true)}
onMouseLeave={() => {
setHovered(false);
setPressed(false);
}}
onMouseDown={() => setPressed(true)}
onMouseUp={() => setPressed(false)}
>
{children}
</button>
);
}
async function startResizeDragging(direction: ResizeDirection) {
try {
await getCurrentWindow().startResizeDragging(direction);
} catch (error) {
console.warn(`[window] Windows resize ${direction} failed`, error);
}
}
async function runWindowsWindowAction(action: 'minimize' | 'toggleMaximize' | 'close') {
try {
const { getCurrentWindow } = await import('@tauri-apps/api/window');
const currentWindow = getCurrentWindow();
if (action === 'minimize') {
await currentWindow.minimize();
@@ -146,8 +248,3 @@ const winBtnStyle: CSSProperties = {
cursor: 'default',
transition: 'background 0.12s ease-out, color 0.12s ease-out',
};
const winCloseBtnStyle: CSSProperties = {
...winBtnStyle,
color: 'var(--ol-ink-3)',
};
+4 -4
View File
@@ -212,12 +212,12 @@ export const en: typeof zhCN = {
selectionAsk: {
kicker: 'SELECTION ASK',
title: 'Selection Ask',
desc: 'Select text in any app, press Cmd+Shift+; to open the panel, then press Option to record a question. Multi-turn follow-ups stay in the same panel until you close it.',
desc: 'Select text in any app, press {{hotkey}} to open the panel, then press {{recordHotkey}} to record a question. Multi-turn follow-ups stay in the same panel until you close it.',
statusEnabled: 'Enabled',
statusDisabled: 'Disabled',
hotkey: {
title: 'Hotkey to open the panel',
desc: 'This only opens / closes the panel. Recording inside the panel reuses your main Option dictation key. Pick "Disabled" to turn the feature off.',
desc: 'This only opens / closes the panel. Recording inside the panel reuses your main {{recordHotkey}} dictation key. Pick "Disabled" to turn the feature off.',
optionDisabled: 'Disabled',
chordWarning: '',
},
@@ -229,8 +229,8 @@ export const en: typeof zhCN = {
title: 'How to use',
step1: 'Press「{{hotkey}}」any time to open the panel (no need to select first).',
step2: 'Select text in any app (browser, Mail, IDE, PDF reader…).',
step3: 'Press **Option** (rightOption — same key you use for dictation) to start recording. Press Option again to stop and submit; the answer shows in the panel.',
step4: 'Keep asking follow-ups in the same panel: press Option again to record, press again to submit. You can re-select different text for the next turn or skip selection entirely.',
step3: 'Press **{{recordHotkey}}** to start recording. Press {{recordHotkey}} again to stop and submit; the answer shows in the panel.',
step4: 'Keep asking follow-ups in the same panel: press {{recordHotkey}} again to record, press {{recordHotkey}} again to submit. You can re-select different text for the next turn or skip selection entirely.',
step5: 'Press Esc or the ✕ in the top-right to close — closing wipes the multi-turn history. Pressing「{{hotkey}}」again starts a fresh conversation.',
windowTitle: 'Position, drag, and pin',
windowDesc: 'The panel first opens above the recording capsule. The toolbar is draggable; once moved, it stays at the dragged position for the rest of the app session. The 📌 pin keeps the window open across follow-up turns.',
+4 -4
View File
@@ -210,12 +210,12 @@ export const zhCN = {
selectionAsk: {
kicker: 'SELECTION ASK',
title: '划词追问',
desc: '选中任意 app 里的一段文字,按 Cmd+Shift+; 弹出浮窗,再按 Option 录音提问。支持多轮追问,浮窗一直保留直到你手动关。',
desc: '选中任意 app 里的一段文字,按 {{hotkey}} 弹出浮窗,再按 {{recordHotkey}} 录音提问。支持多轮追问,浮窗一直保留直到你手动关。',
statusEnabled: '已启用',
statusDisabled: '未启用',
hotkey: {
title: '弹出浮窗的快捷键',
desc: '只决定「打开 / 关闭」浮窗。浮窗里录音 / 提问统一用 Option(与你的主听写键复用)。选「不启用」则关闭整个功能。',
desc: '只决定「打开 / 关闭」浮窗。浮窗里录音 / 提问统一用 {{recordHotkey}}(与你的主听写键复用)。选「不启用」则关闭整个功能。',
optionDisabled: '不启用',
chordWarning: '',
},
@@ -227,8 +227,8 @@ export const zhCN = {
title: '使用方法',
step1: '按「{{hotkey}}」在任意时刻打开浮窗(不需要先选文字)。',
step2: '在任意 app(浏览器、Mail、IDE、PDF reader…)里选中一段文字。',
step3: '按一下 **Option**(rightOption,跟你录音用的同一个键)——开始录音;再按一下 Option,停止并提交,AI 答案显示在浮窗里。',
step4: '同一个浮窗里可继续多轮追问:再按 Option 录音 → 再按 Option 提交。可以重新选文字让下一轮带新选区,也可以不选直接对话。',
step3: '按一下 **{{recordHotkey}}**——开始录音;再按一下 {{recordHotkey}},停止并提交,AI 答案显示在浮窗里。',
step4: '同一个浮窗里可继续多轮追问:再按 {{recordHotkey}} 录音 → 再按 {{recordHotkey}} 提交。可以重新选文字让下一轮带新选区,也可以不选直接对话。',
step5: '按 Esc 或浮窗右上角 ✕ 关闭,关闭即清空所有多轮历史。再按「{{hotkey}}」就是一段新的对话。',
windowTitle: '浮窗位置 + 拖动 + 钉住',
windowDesc: '浮窗第一次打开在屏幕底部录音胶囊正上方;标题栏可拖动,移到任意位置后下一次打开会保留位置(同一次启动期间)。右上角 📌 钉住时即使重新提问也保留窗口;不钉住按 Esc 即关。',
+18
View File
@@ -8,6 +8,13 @@ export interface CapsulePillMetrics {
textWidth: number;
}
export interface CapsuleHostMetrics {
width: number;
height: number;
bottomInset: number;
badgeGap: number;
}
export interface CapsuleMessageLayout {
allowWrap: boolean;
lineClamp: number;
@@ -21,6 +28,17 @@ export function getCapsulePillMetrics(os: OS): CapsulePillMetrics {
return { width: 176, height: 42, textWidth: 84 };
}
export function getCapsuleHostMetrics(
os: OS,
translationActive: boolean,
): CapsuleHostMetrics {
if (os === 'win') {
return { width: 220, height: translationActive ? 118 : 84, bottomInset: 12, badgeGap: 8 };
}
return { width: 176, height: translationActive ? 110 : 42, bottomInset: 0, badgeGap: 8 };
}
export function getCapsuleMessageLayout(
os: OS,
kind: CapsuleMessageKind,
+1
View File
@@ -105,6 +105,7 @@ export interface UserPreferences {
export type QaStateKind =
| 'idle'
| 'recording'
| 'loading'
| 'thinking'
| 'answer_delta'
| 'answer'
+2 -1
View File
@@ -34,6 +34,7 @@ function useModeLabel(): Record<PolishMode, string> {
export function History() {
const { t } = useTranslation();
const os = detectOS();
const FILTERS = useFilters();
const MODE_LABEL = useModeLabel();
const [filter, setFilter] = useState<'all' | PolishMode>('all');
@@ -203,7 +204,7 @@ export function History() {
: item.insertStatus === 'pasteSent'
? t('history.pasteSent')
: item.insertStatus === 'copiedFallback'
? t('history.copiedFallback', { shortcut: detectOS() === 'win' ? 'Ctrl+V' : '⌘V' })
? t('history.copiedFallback', { shortcut: os === 'mac' ? '⌘V' : 'Ctrl+V' })
: t('history.insertFailed')
}</span>
</div>
+56 -2
View File
@@ -31,6 +31,8 @@ export function QaPanel() {
const [pinned, setPinned] = useState(false);
/** 流式 LLM 答案:answer_delta 累积、answer 事件来时清空(最终内容已落到 messages)。 */
const [streamingAnswer, setStreamingAnswer] = useState<string>('');
/** 录音电平:0..1。后端每帧 33ms 通过 qa:level emit。详见 issue #162。 */
const [level, setLevel] = useState<number>(0);
const tRef = useRef(t);
tRef.current = t;
@@ -39,6 +41,7 @@ export function QaPanel() {
if (!isTauri) return;
let unlistenState: (() => void) | undefined;
let unlistenDismiss: (() => void) | undefined;
let unlistenLevel: (() => void) | undefined;
let cancelled = false;
(async () => {
try {
@@ -54,6 +57,7 @@ export function QaPanel() {
setSelectionPreview('');
setErrorMsg('');
setStreamingAnswer('');
setLevel(0);
break;
case 'recording':
setStatus('recording');
@@ -61,11 +65,21 @@ export function QaPanel() {
setErrorMsg('');
setStreamingAnswer('');
break;
case 'loading':
// ASR 在 finalize、user message 还没 push 的过渡帧。提前切到 thinking
// 视图避免 UI 卡 recording 几百 ms 反馈缺失。详见 issue #161。
setStatus('thinking');
setSelectionPreview('');
setErrorMsg('');
setStreamingAnswer('');
setLevel(0);
break;
case 'thinking':
setStatus('thinking');
setSelectionPreview('');
setErrorMsg('');
setStreamingAnswer('');
setLevel(0);
break;
case 'answer_delta':
// 流式增量。仍保持 thinking 状态——直到 answer 事件落定后才回 idle。
@@ -79,11 +93,13 @@ export function QaPanel() {
setErrorMsg('');
// messages 已被上面的 setMessages 落定,清掉流式 buffer 避免和最终气泡重影。
setStreamingAnswer('');
setLevel(0);
break;
case 'error':
setStatus('error');
setErrorMsg(payload.error ?? tRef.current('qa.error'));
setStreamingAnswer('');
setLevel(0);
break;
}
});
@@ -91,12 +107,18 @@ export function QaPanel() {
setPinned(false);
void qaWindowDismiss();
});
// qa:level — 录音电平,节流 ~33ms/帧。详见 issue #162。
const levelHandle = await listen<{ level: number }>('qa:level', event => {
setLevel(event.payload.level ?? 0);
});
if (cancelled) {
stateHandle();
dismissHandle();
levelHandle();
} else {
unlistenState = stateHandle;
unlistenDismiss = dismissHandle;
unlistenLevel = levelHandle;
}
} catch (error) {
console.error('[QaPanel] listener setup failed', error);
@@ -106,6 +128,7 @@ export function QaPanel() {
cancelled = true;
unlistenState?.();
unlistenDismiss?.();
unlistenLevel?.();
};
}, []);
@@ -145,11 +168,11 @@ export function QaPanel() {
<div ref={scrollRef} style={contentStyle}>
{messages.length === 0 && status === 'idle' && <EmptyHint t={t} />}
{messages.length === 0 && status === 'recording' && (
<RecordingHeader preview={selectionPreview} t={t} />
<RecordingHeader preview={selectionPreview} t={t} level={level} />
)}
<MessageList messages={messages} />
{status === 'recording' && messages.length > 0 && (
<TurnIndicator kind="recording" t={t} preview={selectionPreview} />
<TurnIndicator kind="recording" t={t} preview={selectionPreview} level={level} />
)}
{streamingAnswer && (
<StreamingAssistantBubble markdown={streamingAnswer} />
@@ -248,9 +271,11 @@ function EmptyHint({ t }: { t: ReturnType<typeof useTranslation>['t'] }) {
function RecordingHeader({
preview,
t,
level,
}: {
preview: string;
t: ReturnType<typeof useTranslation>['t'];
level: number;
}) {
const truncated = useMemo(() => truncate(preview, SELECTION_PREVIEW_MAX), [preview]);
return (
@@ -267,6 +292,32 @@ function RecordingHeader({
<span style={recordingDotStyle} />
{t('qa.recordingHint')}
</div>
<LevelBar level={level} />
</div>
);
}
/** QA 录音电平条。后端 qa:level 每帧 ~33ms 推一次 0..1。详见 issue #162。 */
function LevelBar({ level }: { level: number }) {
const pct = Math.min(100, Math.max(0, level * 100));
return (
<div
style={{
height: 4,
width: '100%',
background: 'rgba(0,0,0,0.06)',
borderRadius: 2,
overflow: 'hidden',
}}
>
<div
style={{
height: '100%',
width: `${pct}%`,
background: 'var(--ol-blue)',
transition: 'width 0.08s ease-out',
}}
/>
</div>
);
}
@@ -366,10 +417,12 @@ function TurnIndicator({
kind,
preview,
t,
level,
}: {
kind: 'recording' | 'thinking';
preview?: string;
t: ReturnType<typeof useTranslation>['t'];
level?: number;
}) {
if (kind === 'recording') {
const truncated = preview ? truncate(preview, SELECTION_PREVIEW_MAX) : '';
@@ -387,6 +440,7 @@ function TurnIndicator({
<span style={recordingDotStyle} />
{t('qa.recordingHint')}
</div>
<LevelBar level={level ?? 0} />
</div>
);
}
+54 -21
View File
@@ -10,7 +10,8 @@ import { Card, PageHeader } from './_atoms';
import { useHotkeySettings } from '../state/HotkeySettingsContext';
import { setQaHotkey } from '../lib/ipc';
import type { QaHotkeyBinding } from '../lib/types';
import { detectOS } from '../components/WindowChrome';
import { detectOS, type OS } from '../components/WindowChrome';
import { getHotkeyTriggerLabel } from '../lib/hotkey';
const QA_HOTKEY_DISABLED_ID = 'disabled' as const;
@@ -39,24 +40,50 @@ const QA_HOTKEY_PRESETS_WIN: readonly QaHotkeyPreset[] = [
{ id: 'ctrl+shift+,', label: 'Ctrl+Shift+,', binding: { primary: ',', modifiers: ['ctrl', 'shift'] } },
] as const;
const QA_HOTKEY_PRESETS: readonly QaHotkeyPreset[] =
detectOS() === 'mac' ? QA_HOTKEY_PRESETS_MAC : QA_HOTKEY_PRESETS_WIN;
// Linux:UI 展示用 Super,后端 binding 仍用 SUPER 同义词 `cmd` 透传到 global-hotkey。
const QA_HOTKEY_PRESETS_LINUX: readonly QaHotkeyPreset[] = [
{ id: 'super+shift+;', label: 'Super+Shift+;', binding: { primary: ';', modifiers: ['cmd', 'shift'] } },
{ id: 'super+shift+/', label: 'Super+Shift+/', binding: { primary: '/', modifiers: ['cmd', 'shift'] } },
{ id: 'super+shift+.', label: 'Super+Shift+.', binding: { primary: '.', modifiers: ['cmd', 'shift'] } },
{ id: 'super+shift+,', label: 'Super+Shift+,', binding: { primary: ',', modifiers: ['cmd', 'shift'] } },
] as const;
function bindingToPresetId(binding: QaHotkeyBinding | null): string {
function getQaHotkeyPresets(os: OS): readonly QaHotkeyPreset[] {
if (os === 'mac') return QA_HOTKEY_PRESETS_MAC;
if (os === 'linux') return QA_HOTKEY_PRESETS_LINUX;
return QA_HOTKEY_PRESETS_WIN;
}
function normalizeQaModifier(modifier: string): string {
const tag = modifier.toLowerCase();
if (tag === 'command' || tag === 'super' || tag === 'meta' || tag === 'win') {
return 'cmd';
}
return tag;
}
function bindingToPresetId(
binding: QaHotkeyBinding | null,
presets: readonly QaHotkeyPreset[],
): string {
if (!binding) return QA_HOTKEY_DISABLED_ID;
const sortedMods = [...binding.modifiers].map(m => m.toLowerCase()).sort();
const match = QA_HOTKEY_PRESETS.find(p => {
const pMods = [...p.binding.modifiers].sort();
const sortedMods = [...binding.modifiers].map(normalizeQaModifier).sort();
const match = presets.find(p => {
const pMods = [...p.binding.modifiers].map(normalizeQaModifier).sort();
return p.binding.primary === binding.primary
&& pMods.length === sortedMods.length
&& pMods.every((m, i) => m === sortedMods[i]);
});
return match ? match.id : QA_HOTKEY_PRESETS[0].id;
return match ? match.id : presets[0].id;
}
export function SelectionAsk() {
const { t } = useTranslation();
const { prefs, updatePrefs: savePrefs } = useHotkeySettings();
const { prefs, hotkey, updatePrefs: savePrefs } = useHotkeySettings();
const os = detectOS();
const qaHotkeyPresets = getQaHotkeyPresets(os);
const defaultHotkeyLabel = qaHotkeyPresets[0]?.label ?? '快捷键';
const recordHotkeyLabel = getHotkeyTriggerLabel(hotkey?.trigger);
if (!prefs) {
return (
@@ -64,7 +91,10 @@ export function SelectionAsk() {
<PageHeader
kicker={t('selectionAsk.kicker')}
title={t('selectionAsk.title')}
desc={t('selectionAsk.desc')}
desc={t('selectionAsk.desc', {
hotkey: defaultHotkeyLabel,
recordHotkey: recordHotkeyLabel,
})}
/>
<Card>
<div style={{ fontSize: 12, color: 'var(--ol-ink-4)' }}>{t('common.loading')}</div>
@@ -78,7 +108,7 @@ export function SelectionAsk() {
await savePrefs({ ...prefs, qaHotkey: null });
return;
}
const preset = QA_HOTKEY_PRESETS.find(p => p.id === id);
const preset = qaHotkeyPresets.find(p => p.id === id);
if (!preset) return;
// 先让后端真注册成功 → 再写盘 prefs。否则 prefs 跟实际生效的快捷键脱节,
// 会让用户陷入"UI 改了但按了没反应"的迷雾(issue #118 v1 实测过)。
@@ -96,15 +126,18 @@ export function SelectionAsk() {
savePrefs({ ...prefs, qaSaveHistory });
const enabled = prefs.qaHotkey !== null;
const currentId = bindingToPresetId(prefs.qaHotkey);
const currentLabel = QA_HOTKEY_PRESETS.find(p => p.id === currentId)?.label ?? '';
const currentId = bindingToPresetId(prefs.qaHotkey, qaHotkeyPresets);
const currentLabel = qaHotkeyPresets.find(p => p.id === currentId)?.label ?? defaultHotkeyLabel;
return (
<>
<PageHeader
kicker={t('selectionAsk.kicker')}
title={t('selectionAsk.title')}
desc={t('selectionAsk.desc')}
desc={t('selectionAsk.desc', {
hotkey: enabled ? currentLabel : defaultHotkeyLabel,
recordHotkey: recordHotkeyLabel,
})}
/>
<div style={{ display: 'flex', flexDirection: 'column', gap: 12 }}>
@@ -129,7 +162,7 @@ export function SelectionAsk() {
</span>
</div>
<div style={{ fontSize: 11.5, color: 'var(--ol-ink-4)', marginBottom: 12, lineHeight: 1.55 }}>
{t('selectionAsk.hotkey.desc')}
{t('selectionAsk.hotkey.desc', { recordHotkey: recordHotkeyLabel })}
</div>
<select
value={currentId}
@@ -149,7 +182,7 @@ export function SelectionAsk() {
}}
>
<option value={QA_HOTKEY_DISABLED_ID}>{t('selectionAsk.hotkey.optionDisabled')}</option>
{QA_HOTKEY_PRESETS.map(p => (
{qaHotkeyPresets.map(p => (
<option key={p.id} value={p.id}>{p.label}</option>
))}
</select>
@@ -195,11 +228,11 @@ export function SelectionAsk() {
<Card>
<div style={{ fontSize: 13, fontWeight: 600, marginBottom: 10 }}>{t('selectionAsk.howto.title')}</div>
<ol style={{ margin: 0, paddingLeft: 18, fontSize: 12.5, color: 'var(--ol-ink-2)', lineHeight: 1.7 }}>
<li>{t('selectionAsk.howto.step1')}</li>
<li>{t('selectionAsk.howto.step2', { hotkey: enabled ? currentLabel : '快捷键' })}</li>
<li>{t('selectionAsk.howto.step3')}</li>
<li>{t('selectionAsk.howto.step4', { hotkey: enabled ? currentLabel : '快捷键' })}</li>
<li>{t('selectionAsk.howto.step5')}</li>
<li>{t('selectionAsk.howto.step1', { hotkey: enabled ? currentLabel : defaultHotkeyLabel })}</li>
<li>{t('selectionAsk.howto.step2')}</li>
<li>{t('selectionAsk.howto.step3', { recordHotkey: recordHotkeyLabel })}</li>
<li>{t('selectionAsk.howto.step4', { recordHotkey: recordHotkeyLabel })}</li>
<li>{t('selectionAsk.howto.step5', { hotkey: enabled ? currentLabel : defaultHotkeyLabel })}</li>
</ol>
<div
+3 -1
View File
@@ -3,17 +3,19 @@ html, body, #root {
width: 100%;
margin: 0;
padding: 0;
background: transparent;
overflow: hidden;
}
body {
background: transparent;
user-select: none;
-webkit-user-select: none;
overflow: hidden;
}
#root {
display: flex;
isolation: isolate;
}
button {