fix(windows): delay clipboard restore after synthetic paste

This commit is contained in:
Cooper-X-Oak committed 2026-05-02 03:59:01 +08:00
1 parent 27dbeadcfd
commit 02a3e0a225
4 files changed
+448 -4

No files matched your search

@@ -0,0 +1,143 @@
# Windows terminal clipboard restore investigation (2026-05-02)
Scope: `openless-all/app/src-tauri/src/insertion.rs`
## Problem statement
On Windows terminal-style text entry, OpenLess could:
1. put the new dictated text into the clipboard
2. send `Ctrl+V`
3. restore the old clipboard too early
4. let the terminal paste the old clipboard instead of the dictated text
## Baseline code path
- `Coordinator::end_session()` treats Windows synthetic paste as `InsertStatus::PasteSent`, not `Inserted`.
- `TextInserter::insert()` calls `insert_with_clipboard_restore()`.
- Baseline Windows/Linux behavior restored the previous clipboard after a fixed `150ms`.
- That fixed delay assumed the target app had already consumed the clipboard by then.
## Automated evidence
### 1. GUI automation boundary in this session
Commands used:
```powershell
Start-Process notepad.exe -PassThru
Start-Process cmd.exe -PassThru
EnumWindows(...)
```
Observed result:
- `explorer.exe` exists in `SessionId=1`
- newly started `notepad.exe`, `cmd.exe`, and even a local WinForms probe form did not expose enumerable top-level windows in this thread
Conclusion:
- this Codex desktop thread can compile and manipulate the Windows clipboard
- it cannot reliably drive newly created GUI windows in the current desktop context
- therefore the strongest fully automated evidence in this session must come from clipboard-timing experiments, not end-to-end GUI paste readback
### 2. Clipboard timing matrix
Script:
- `openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1`
Command:
```powershell
$cases = @(
@{ consumer = 50; restore = 150 },
@{ consumer = 250; restore = 150 },
@{ consumer = 250; restore = 750 }
)
foreach ($case in $cases) {
powershell -ExecutionPolicy Bypass -File openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1 -ConsumerDelayMs $case.consumer -RestoreDelayMs $case.restore
}
```
Observed outputs:
```json
{"consumerDelayMs":50,"restoreDelayMs":150,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_DICTATED_TEXT","matchedInserted":true}
{"consumerDelayMs":250,"restoreDelayMs":150,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_OLDER_CLIPBOARD","matchedInserted":false}
{"consumerDelayMs":250,"restoreDelayMs":750,"insertedText":"OPENLESS_DICTATED_TEXT","previousText":"OPENLESS_OLDER_CLIPBOARD","observedText":"OPENLESS_DICTATED_TEXT","matchedInserted":true}
```
Interpretation:
- a fast consumer (`50ms`) succeeds with the old `150ms` restore window
- a slower consumer (`250ms`) fails with the old `150ms` restore window
- the same slower consumer succeeds once restore is delayed to `750ms`
This isolates the bug to clipboard restore timing, independent of ASR, polish, QA hotkey, or selection logic.
## Root cause
Root cause: Windows `PasteSent` semantics were treated as if they implied paste completion.
- `PasteSent` only means OpenLess sent synthetic `Ctrl+V`
- it does not mean the target application has already read clipboard contents
- terminal-style targets can consume the clipboard later than standard text inputs
- restoring the old clipboard at a fixed `150ms` can therefore race ahead of actual paste consumption
Classification:
- primary layer: `clipboard lifecycle`
- secondary layer: `insertion lifecycle`
- not primary: `focus restore`
- manifestation: terminal-specific and likely any slower Windows paste consumer
- not evidence of a global Windows clipboard bug by itself
## Fix applied
File:
- `openless-all/app/src-tauri/src/insertion.rs`
Change:
- Windows clipboard restore delay changed from `150ms` to `750ms`
- restore now runs on a background thread instead of blocking the insert path
- Linux keeps the previous `150ms` behavior
## Verification run
Commands:
```powershell
cargo fmt --all
cargo check --lib
cargo test --lib --no-run
cargo check --tests
powershell -NoProfile -Command "[void][scriptblock]::Create((Get-Content -Raw 'openless-all/app/scripts/windows-clipboard-consumer-timing-smoke.ps1')); 'script-parse-ok'"
```
Observed result:
- compile/check passed
- test binaries compiled
- new smoke scripts parse successfully
## Remaining gap
Still needed on a fully interactive Windows desktop:
- `Windows Terminal`
- `PowerShell`
- `CMD`
- `Notepad`
Goal:
- confirm that the real terminal input line reproduces under the old restore timing
- confirm the patched build no longer pastes stale clipboard content
## Suggested issue / PR title
- Issue: `[windows][insertion] terminal paste can restore stale clipboard before synthetic paste lands`
- PR: `fix(windows): delay clipboard restore after synthetic paste`
@@ -0,0 +1,76 @@
param(
[int]$ConsumerDelayMs = 250,
[int]$RestoreDelayMs = 150,
[string]$InsertedText = "OPENLESS_DICTATED_TEXT",
[string]$PreviousText = "OPENLESS_OLDER_CLIPBOARD"
)
$ErrorActionPreference = "Stop"
Add-Type -AssemblyName System.Windows.Forms
Add-Type -ReferencedAssemblies @("System.Windows.Forms") @"
using System;
using System.Threading;
using System.Windows.Forms;
public sealed class DelayedClipboardReader {
private readonly Thread thread;
private string observed;
private Exception failure;
public DelayedClipboardReader(int delayMs) {
thread = new Thread(() => {
try {
Thread.Sleep(delayMs);
if (Clipboard.ContainsText()) {
observed = Clipboard.GetText();
}
} catch (Exception ex) {
failure = ex;
}
});
thread.SetApartmentState(ApartmentState.STA);
}
public void Start() {
thread.Start();
}
public string JoinAndGetResult() {
thread.Join();
if (failure != null) {
throw failure;
}
return observed;
}
}
"@
function Restore-ClipboardValue($Value) {
if ($null -eq $Value) {
cmd /c "echo off | clip" | Out-Null
return
}
Set-Clipboard -Value $Value
}
$originalClipboard = Get-Clipboard -Raw -ErrorAction SilentlyContinue
try {
Set-Clipboard -Value $InsertedText
$reader = [DelayedClipboardReader]::new($ConsumerDelayMs)
$reader.Start()
Start-Sleep -Milliseconds $RestoreDelayMs
Restore-ClipboardValue $PreviousText
$observedText = $reader.JoinAndGetResult()
$result = [pscustomobject]@{
consumerDelayMs = $ConsumerDelayMs
restoreDelayMs = $RestoreDelayMs
insertedText = $InsertedText
previousText = $PreviousText
observedText = $observedText
matchedInserted = ($observedText -eq $InsertedText)
}
$result | ConvertTo-Json -Compress
} finally {
Restore-ClipboardValue $originalClipboard
}
@@ -0,0 +1,184 @@
param(
[ValidateSet("notepad", "powershell", "cmd", "wt")]
[string]$Target = "powershell",
[int]$RestoreDelayMs = 150,
[int]$PasteSettleDelayMs = 900
)
$ErrorActionPreference = "Stop"
Add-Type @"
using System;
using System.Runtime.InteropServices;
public static class ClipboardRestoreSmokeWin32 {
[DllImport("user32.dll")]
public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern void keybd_event(byte bVk, byte bScan, int dwFlags, UIntPtr dwExtraInfo);
public const int KEYEVENTF_EXTENDEDKEY = 0x0001;
public const int KEYEVENTF_KEYUP = 0x0002;
}
"@
function Send-KeyEdge($Vk, $KeyUp, $Extended = $false) {
$flags = 0
if ($Extended) {
$flags = $flags -bor [ClipboardRestoreSmokeWin32]::KEYEVENTF_EXTENDEDKEY
}
if ($KeyUp) {
$flags = $flags -bor [ClipboardRestoreSmokeWin32]::KEYEVENTF_KEYUP
}
$scanCode = if ($Vk -eq 0xA2 -or $Vk -eq 0xA3) { 0x1D } else { 0 }
[ClipboardRestoreSmokeWin32]::keybd_event([byte]$Vk, [byte]$scanCode, $flags, [UIntPtr]::Zero)
}
function Send-CtrlChord($Vk) {
Send-KeyEdge 0xA2 $false $false
Start-Sleep -Milliseconds 70
Send-KeyEdge $Vk $false $false
Start-Sleep -Milliseconds 70
Send-KeyEdge $Vk $true $false
Start-Sleep -Milliseconds 70
Send-KeyEdge 0xA2 $true $false
}
function Send-EnterKey {
Send-KeyEdge 0x0D $false $false
Start-Sleep -Milliseconds 70
Send-KeyEdge 0x0D $true $false
}
function Wait-ProcessWindow($ProcessName, $After, $TimeoutSeconds) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
$candidate = Get-Process $ProcessName -ErrorAction SilentlyContinue |
Where-Object { $_.StartTime -ge $After -and $_.MainWindowHandle -ne 0 } |
Sort-Object StartTime -Descending |
Select-Object -First 1
if ($null -ne $candidate) {
return $candidate
}
Start-Sleep -Milliseconds 300
}
return $null
}
function Focus-Window($Process) {
if ($null -eq $Process -or $Process.MainWindowHandle -eq 0) {
throw "Target window is unavailable."
}
[ClipboardRestoreSmokeWin32]::ShowWindow($Process.MainWindowHandle, 9) | Out-Null
[ClipboardRestoreSmokeWin32]::SetForegroundWindow($Process.MainWindowHandle) | Out-Null
Start-Sleep -Milliseconds 500
}
function Start-TargetWindow($TargetName) {
$startedAt = Get-Date
switch ($TargetName) {
"notepad" {
Start-Process notepad.exe | Out-Null
return Wait-ProcessWindow "notepad" $startedAt 15
}
"powershell" {
Start-Process powershell.exe -ArgumentList "-NoLogo" | Out-Null
return Wait-ProcessWindow "powershell" $startedAt 15
}
"cmd" {
Start-Process cmd.exe | Out-Null
return Wait-ProcessWindow "cmd" $startedAt 15
}
"wt" {
$wt = Get-Command wt.exe -ErrorAction SilentlyContinue
if ($null -eq $wt) {
throw "wt.exe was not found."
}
Start-Process $wt.Source -ArgumentList "new-tab", "powershell.exe", "-NoLogo" | Out-Null
return Wait-ProcessWindow "WindowsTerminal" $startedAt 20
}
}
}
function Wait-FileText($Path, $Expected, $TimeoutSeconds) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
if (Test-Path $Path) {
$content = Get-Content -Raw -Encoding UTF8 $Path
if ($content.Contains($Expected)) {
return $true
}
}
Start-Sleep -Milliseconds 200
}
return $false
}
function Restore-ClipboardValue($Value) {
if ($null -eq $Value) {
cmd /c "echo off | clip" | Out-Null
return
}
Set-Clipboard -Value $Value
}
$marker = "OPENLESS_CLIPBOARD_RESTORE_OK"
$outputPath = Join-Path $env:TEMP "openless-clipboard-restore-$Target.txt"
Remove-Item -LiteralPath $outputPath -Force -ErrorAction SilentlyContinue
$previousClipboard = Get-Clipboard -Raw -ErrorAction SilentlyContinue
switch ($Target) {
"notepad" {
$payload = $marker
}
"cmd" {
$payload = "echo $marker > `"$outputPath`""
}
default {
$payload = "Set-Content -Path `"$outputPath`" -Value `"$marker`""
}
}
$targetProcess = $null
try {
$targetProcess = Start-TargetWindow $Target
if ($null -eq $targetProcess) {
throw "Failed to start target window: $Target"
}
Focus-Window $targetProcess
Set-Clipboard -Value $payload
Start-Sleep -Milliseconds 150
Send-CtrlChord 0x56
Start-Sleep -Milliseconds $RestoreDelayMs
Restore-ClipboardValue $previousClipboard
if ($Target -eq "notepad") {
Start-Sleep -Milliseconds $PasteSettleDelayMs
Send-CtrlChord 0x41
Start-Sleep -Milliseconds 120
Send-CtrlChord 0x43
Start-Sleep -Milliseconds 200
$result = Get-Clipboard -Raw -ErrorAction SilentlyContinue
if (-not $result.Contains($marker)) {
throw "Notepad readback did not contain the pasted marker."
}
} else {
Start-Sleep -Milliseconds 120
Send-EnterKey
if (-not (Wait-FileText $outputPath $marker 6)) {
throw "Terminal target did not execute the pasted command before clipboard restore."
}
}
Write-Host "[ok] target=$Target restoreDelayMs=$RestoreDelayMs"
} finally {
Restore-ClipboardValue $previousClipboard
if ($null -ne $targetProcess) {
Stop-Process -Id $targetProcess.Id -Force -ErrorAction SilentlyContinue
}
}
+45 -4
View File
@@ -14,7 +14,10 @@ use std::time::Duration;
use crate::types::InsertStatus;
#[cfg(not(target_os = "macos"))]
#[cfg(target_os = "windows")]
const CLIPBOARD_RESTORE_DELAY: Duration = Duration::from_millis(750);
#[cfg(all(not(target_os = "macos"), not(target_os = "windows")))]
const CLIPBOARD_RESTORE_DELAY: Duration = Duration::from_millis(150);
pub struct TextInserter;
@@ -131,19 +134,24 @@ fn insert_with_clipboard_restore(text: &str, restore_clipboard_after_paste: bool
}
if restore_clipboard_after_paste {
maybe_restore_clipboard(restore_plan);
schedule_clipboard_restore(restore_plan);
}
// 关掉 → 听写文本留在剪贴板里,simulate_paste 没真正落地时用户能手动 Ctrl+V 找回。
insertion_success_status()
}
#[cfg(not(target_os = "macos"))]
fn maybe_restore_clipboard(plan: ClipboardRestorePlan) {
fn schedule_clipboard_restore(plan: ClipboardRestorePlan) {
std::thread::spawn(move || restore_clipboard_after_delay(plan, CLIPBOARD_RESTORE_DELAY));
}
#[cfg(not(target_os = "macos"))]
fn restore_clipboard_after_delay(plan: ClipboardRestorePlan, delay: Duration) {
if plan.previous_text.is_none() {
return;
}
std::thread::sleep(CLIPBOARD_RESTORE_DELAY);
std::thread::sleep(delay);
let mut clipboard = match arboard::Clipboard::new() {
Ok(clipboard) => clipboard,
@@ -303,6 +311,9 @@ mod macos {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;
#[test]
#[cfg(not(target_os = "macos"))]
@@ -317,4 +328,34 @@ mod tests {
));
assert!(!should_restore_clipboard(None, "dictated text"));
}
#[test]
#[cfg(target_os = "windows")]
fn delayed_terminal_paste_must_see_dictated_text_before_clipboard_restore() {
let inserted_text = "dictated text".to_string();
let previous_text = "older clipboard".to_string();
let clipboard = Arc::new(Mutex::new(inserted_text.clone()));
let pasted = Arc::new(Mutex::new(None::<String>));
let clipboard_for_paste = Arc::clone(&clipboard);
let pasted_for_paste = Arc::clone(&pasted);
let reader = thread::spawn(move || {
thread::sleep(Duration::from_millis(250));
let seen = clipboard_for_paste.lock().unwrap().clone();
*pasted_for_paste.lock().unwrap() = Some(seen);
});
thread::sleep(CLIPBOARD_RESTORE_DELAY);
let current_text = Some(clipboard.lock().unwrap().clone());
if should_restore_clipboard(current_text.as_deref(), &inserted_text) {
*clipboard.lock().unwrap() = previous_text;
}
reader.join().unwrap();
assert_eq!(
pasted.lock().unwrap().as_deref(),
Some(inserted_text.as_str())
);
}
}