diff --git a/CHANGELOG.md b/CHANGELOG.md index cbad273..509a360 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,26 @@ --- +## [0.1.8] - 2026-08-18 + +### 新增 +- **新增「上下文感知」（默认关闭）**：在设置 → 通用里打开后，口述时 SayIt 会通过 Windows UI Automation 读取当前文本框里光标前 500 字、后 300 字的内容（不截图、不用 OCR），作为上下文交给 AI 整理，让新说的话在语气、术语和格式上与前后文自然衔接；这一步仍使用你当前的 AI 整理 Prompt。如果先用鼠标选中一段文字再开口，说出的话会被当成对这段文字的编辑指令（翻译、精简、总结、改写或直接提问），处理完直接替换选中内容，这种情况下走的是设置页里可单独查看和编辑的另一份 Prompt。密码框会被自动跳过，不读取其内容。桌面端本地识别、云端语音识别与服务器模式均支持，网页版 Demo 不支持。 +- **服务器模式下可以提交识别纠错（实验功能）**：在历史记录里展开一条服务器模式的记录，点「纠正识别」，就能一边回放录音、一边把识别原文改成你实际说过的话，改了哪里会直接标在原文上（浅红是改掉的，浅绿是改后的）。提交后这次录音、识别原文、你写的正确内容和识别时用到的热词会发送给作者，用于实验性地微调语音识别模型，仅作技术验证、不用于商业用途；首次提交前会先把这些说清楚并请你确认，30 天内可以撤回，撤回后服务器上的录音和文本会被删除。AI 整理后的文本只作参照显示，不会发送。 +- **可以让短语音跳过 AI 整理**：设置 → AI 整理里新增「短语音不整理」，填一个秒数，短于它的录音就只做语音识别、不再交给 AI。默认是 0，也就是照旧每段都整理。「好」「知道了」这类一两个字的短句本来没什么可整理的，却要等一次 AI 往返，现在可以直接省掉。本地识别、云端 API 与服务器模式都生效。 +- **托盘右键菜单可以直接开关「AI 整理」**：不必先打开主窗口。菜单里那一行带一个开关，一眼能看出当前是开还是关，与标题栏的开关、AI 整理设置页始终一致；开机自启只停在托盘时也能用。高分屏下托盘图标也会换成更清晰的一档，不再糊。 +- **可以给「AI 整理」设一个快捷键，不打开窗口也能开关**：在设置 → AI 整理里设置，默认留空、不占用任何按键。支持组合键，也可以直接绑鼠标侧键或中键。按下后悬浮窗会显示一行「AI 整理已开启 / 已关闭」；录音进行中按下不弹这条提示，免得盖住波形。与「按住说话」「免提模式」「预设切换」撞键会当场提示，改哪一边都会互相检查。 +- **换了麦克风会在悬浮窗上提示这次用的是哪个设备**：换过麦克风、或改过设置里的音源之后，下一次录音时悬浮窗顶部会出现一行提示，写着「自动选择」或「指定设备」加设备名称，几秒后自动消失。同一个设备连续使用时不会重复提示。麦克风被系统静音时的检测现在也能准确识别到你选中的那一个特定设备，不再只对「系统默认麦克风」有效；持续静音满 3 秒会自动结束这次录音，不用再对着静音的麦克风干等。 + +### 优化 +- **应用 Prompt 规则支持拖动排序，启用哪条就把它排到最前**：多条规则自上而下依次匹配、命中的第一条生效，所以顺序就是优先级。此前顺序是固定的，看得见却改不了；而内置规则默认全部关闭，启用靠底部的一条（比如 QQ）之后，它仍夹在一堆未启用的规则中间。现在拖动左侧手柄即可调整顺序（也可聚焦手柄后用方向键），启用一条规则会自动把它移到列表最上面。 +- **规则的启用开关按下即生效**：此前按完还要再点一次「保存规则」才算数，开关看着已经打开、其实并未生效。附加提示词这类文本修改照旧点「保存规则」提交。 + +### 修复 +- **「录音保留期」设置没有真正生效**：如果你从没改过设置 → 通用里的录音保留期，界面显示的是「永久保留」，但每次启动 SayIt 仍会把 30 天前的录音文件删掉——历史记录里的文字还在，点播放却没有声音、点下载也没有反应。现在没改过设置就是真的永久保留，改成 7 / 30 / 90 天则照你选的执行。 +- **麦克风没有静音，却每次都先提示「麦克风已被静音」**：部分麦克风（常见于带静音按键的 USB 耳麦）会把系统里的静音标记留在打开状态，实际却仍在正常收音。此前只要读到这个标记就立刻弹出红色警告，并计划在 3 秒后结束这次录音；开口说话之后警告又自己消失，看起来像是无端报错。现在这个标记只作为线索，还要确认确实收不到任何声音才会提示，真被静音时的提醒和自动结束都不受影响。 + +--- + ## [0.1.7] - 2026-08-13 > 这一版有两件大事：整个界面多了一套英文；自动更新不再打断你说话。 diff --git a/README.md b/README.md index 7285368..2977556 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ Typing is often the slowest part of working with AI. SayIt turns speech into tex - **Voice typing anywhere** — dictate into editors, chat apps, browsers, and other Windows software. - **Editable AI cleanup** — remove filler words, repair recognition errors, format ideas, or keep a faithful transcript. Every prompt is yours to change. +- **Context-aware writing** (off by default) — reads the text around your cursor so new dictation matches its tone and terminology. Select text first and your speech becomes an editing instruction—translate, tighten, rewrite, or ask a question—replacing the selection directly. Password fields are skipped. - **Flexible speech recognition** — use a cloud ASR provider, run a local GGUF model on your own GPU, connect to the public trial server, or host your own backend. - **English and Chinese interface** — the UI follows your system language and can be switched at any time. - **Hotwords and per-app rules** — improve names and technical terms, then change cleanup behavior automatically for different apps. diff --git a/README.zh-CN.md b/README.zh-CN.md index d690ba2..4686e85 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -165,6 +165,7 @@ cd backend && uvicorn app.main:app --host 0.0.0.0 --port 8000 - **全局语音输入** — 在任何应用中按下快捷键即可口述，文字自动插入光标位置 - **AI 智能润色** — 口语自动转书面语，去口癖、纠错、分段，Prompt 完全可自定义 +- **上下文感知**（默认关闭）— 开启后会读取光标附近的文本，让新说的话在语气、术语和格式上与前后文衔接；先选中一段文字再开口，说出的话会被当成对它的编辑指令（翻译、精简、改写或提问），处理完直接替换选中内容。密码框会自动跳过 - **多种语音识别** — 云端可选豆包、千问、小米 MiMo、Groq Whisper；本地离线识别有 7 档 GGUF 模型（Parakeet Unified EN、SenseVoice、Fun-ASR Nano、Nemotron 3.5、Qwen3-ASR 三档），可用显卡加速 - **中英双语界面** — 默认跟随系统语言，也可以在设置里随时切换 - **热词增强** — 自定义专业术语词表；豆包、千问、服务器模式和本地 Qwen3-ASR 会参与 ASR 识别偏置，本地 SenseVoice 则通过 AI 整理/文本处理做后处理纠错 diff --git a/client/package.json b/client/package.json index b952916..265d4f3 100644 --- a/client/package.json +++ b/client/package.json @@ -1,6 +1,6 @@ { "name": "sayit-tauri", - "version": "0.1.7", + "version": "0.1.8", "description": "SayIt - Voice to Text Desktop App (Tauri)", "type": "module", "scripts": { diff --git a/client/src-tauri/Cargo.lock b/client/src-tauri/Cargo.lock index 2d20ca1..24073a7 100644 --- a/client/src-tauri/Cargo.lock +++ b/client/src-tauri/Cargo.lock @@ -4032,7 +4032,7 @@ dependencies = [ [[package]] name = "sayit" -version = "0.1.7" +version = "0.1.8" dependencies = [ "base64 0.22.1", "bzip2", diff --git a/client/src-tauri/Cargo.toml b/client/src-tauri/Cargo.toml index b3afa1f..68102ea 100644 --- a/client/src-tauri/Cargo.toml +++ b/client/src-tauri/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "sayit" -version = "0.1.7" +version = "0.1.8" authors = ["Liu Qianglong", "Claude"] edition = "2021" @@ -69,7 +69,11 @@ windows = { version = "0.58", features = [ "Win32_System_Memory", "Win32_System_SystemInformation", "Win32_System_Com", + "Win32_System_Com_StructuredStorage", + "Win32_System_Variant", "Win32_Graphics_Gdi", + "Win32_Devices_FunctionDiscovery", + "Win32_UI_Shell_PropertiesSystem", "Win32_Media_Audio", "Win32_Media_Audio_Endpoints", ] } diff --git a/client/src-tauri/capabilities/default.json b/client/src-tauri/capabilities/default.json index f4a0b1f..33a7fda 100644 --- a/client/src-tauri/capabilities/default.json +++ b/client/src-tauri/capabilities/default.json @@ -1,7 +1,7 @@ { "identifier": "default", "description": "Default capabilities for SayIt", - "windows": ["main", "overlay"], + "windows": ["main", "overlay", "tray-menu"], "permissions": [ "core:default", "core:window:default", diff --git a/client/src-tauri/gen/schemas/capabilities.json b/client/src-tauri/gen/schemas/capabilities.json index 3585752..6d8b168 100644 --- a/client/src-tauri/gen/schemas/capabilities.json +++ b/client/src-tauri/gen/schemas/capabilities.json @@ -1 +1 @@ -{"default":{"identifier":"default","description":"Default capabilities for SayIt","local":true,"windows":["main","overlay"],"permissions":["core:default","core:window:default","core:window:allow-create","core:window:allow-show","core:window:allow-hide","core:window:allow-close","core:window:allow-minimize","core:window:allow-toggle-maximize","core:window:allow-set-size","core:window:allow-set-position","core:window:allow-set-always-on-top","core:window:allow-set-ignore-cursor-events","core:window:allow-set-focus","core:webview:default","core:event:default","core:event:allow-emit","core:event:allow-emit-to","core:event:allow-listen","shell:allow-open","clipboard-manager:default","dialog:default","fs:default","fs:allow-write-file","autostart:default","global-shortcut:default"]}} \ No newline at end of file +{"default":{"identifier":"default","description":"Default capabilities for SayIt","local":true,"windows":["main","overlay","tray-menu"],"permissions":["core:default","core:window:default","core:window:allow-create","core:window:allow-show","core:window:allow-hide","core:window:allow-close","core:window:allow-minimize","core:window:allow-toggle-maximize","core:window:allow-set-size","core:window:allow-set-position","core:window:allow-set-always-on-top","core:window:allow-set-ignore-cursor-events","core:window:allow-set-focus","core:webview:default","core:event:default","core:event:allow-emit","core:event:allow-emit-to","core:event:allow-listen","shell:allow-open","clipboard-manager:default","dialog:default","fs:default","fs:allow-write-file","autostart:default","global-shortcut:default"]}} \ No newline at end of file diff --git a/client/src-tauri/icons/tray-16.png b/client/src-tauri/icons/tray-16.png new file mode 100644 index 0000000..b3ae92b Binary files /dev/null and b/client/src-tauri/icons/tray-16.png differ diff --git a/client/src-tauri/icons/tray-20.png b/client/src-tauri/icons/tray-20.png new file mode 100644 index 0000000..00d608b Binary files /dev/null and b/client/src-tauri/icons/tray-20.png differ diff --git a/client/src-tauri/icons/tray-24.png b/client/src-tauri/icons/tray-24.png new file mode 100644 index 0000000..deffb30 Binary files /dev/null and b/client/src-tauri/icons/tray-24.png differ diff --git a/client/src-tauri/icons/tray-32.png b/client/src-tauri/icons/tray-32.png new file mode 100644 index 0000000..6648825 Binary files /dev/null and b/client/src-tauri/icons/tray-32.png differ diff --git a/client/src-tauri/src/commands/audio.rs b/client/src-tauri/src/commands/audio.rs index aff759d..82d9dc8 100644 --- a/client/src-tauri/src/commands/audio.rs +++ b/client/src-tauri/src/commands/audio.rs @@ -53,6 +53,16 @@ pub fn save_pcm_as_wav(id: String, pcm_base64: String, sample_rate: Option) Ok(path.to_string_lossy().to_string()) } +/// 录音文件还在不在。 +/// +/// 单独一个命令而不是复用 `read_audio_file`：后者会把整份 WAV 读出来再 base64 +/// （五分钟录音约 13MB 的字符串），只为判断"文件存不存在"的话代价太大 —— +/// 历史列表里每条记录都要判一次。 +#[tauri::command] +pub fn audio_file_exists(file_path: String) -> bool { + !file_path.is_empty() && PathBuf::from(&file_path).is_file() +} + #[tauri::command] pub fn read_audio_file(file_path: String) -> Result, String> { let path = PathBuf::from(&file_path); diff --git a/client/src-tauri/src/commands/audio_mute.rs b/client/src-tauri/src/commands/audio_mute.rs index 40aaee6..68bfebc 100644 --- a/client/src-tauri/src/commands/audio_mute.rs +++ b/client/src-tauri/src/commands/audio_mute.rs @@ -140,21 +140,74 @@ pub fn restore_system_output() -> Result { // 麦克风静音状态查询 // ───────────────────────────────────────────────────────────────────────── +/// 浏览器会给默认设备名加上 Default/Communications 前缀；Windows 端点友好名没有。 +/// 这里只做名字规范化以定位本次实际打开的端点，不参与静音与否的判断。 +fn normalize_mic_label(label: &str) -> String { + let trimmed = label.trim(); + let lower = trimmed.to_lowercase(); + let without_route_prefix = ["default", "communications", "默认值", "默认", "通信"] + .iter() + .find_map(|prefix| { + let prefix_lower = prefix.to_lowercase(); + lower.starts_with(&prefix_lower).then(|| { + trimmed[prefix.len()..].trim_start_matches(|c: char| { + c.is_whitespace() || matches!(c, '-' | '–' | '—' | ':') + }) + }) + }) + .unwrap_or(trimmed); + + without_route_prefix + .split_whitespace() + .collect::>() + .join(" ") + .to_lowercase() +} + +#[cfg(windows)] +unsafe fn get_device_friendly_name( + device: &windows::Win32::Media::Audio::IMMDevice, +) -> Result { + use windows::Win32::Devices::FunctionDiscovery::PKEY_Device_FriendlyName; + use windows::Win32::System::Com::StructuredStorage::{ + PropVariantClear, PropVariantToStringAlloc, + }; + use windows::Win32::System::Com::{CoTaskMemFree, STGM_READ}; + + let store = device + .OpenPropertyStore(STGM_READ) + .map_err(|e| format!("OpenPropertyStore: {}", e))?; + let mut value = store + .GetValue(&PKEY_Device_FriendlyName) + .map_err(|e| format!("GetValue(PKEY_Device_FriendlyName): {}", e))?; + + let name_result = (|| -> Result { + let ptr = PropVariantToStringAlloc(&value) + .map_err(|e| format!("PropVariantToStringAlloc: {}", e))?; + let text = ptr + .to_string() + .map_err(|e| format!("FriendlyName to_string: {}", e)); + CoTaskMemFree(Some(ptr.0.cast())); + text + })(); + let clear_result = PropVariantClear(&mut value) + .map_err(|e| format!("PropVariantClear: {}", e)); + clear_result?; + name_result +} + /// 查询采集设备的静音状态。 /// -/// 当前实现只可靠支持「系统默认麦克风」（`label` 为 None/空）：直接查默认采集端点的静音开关。 -/// 若指定了特定设备名（`label` 非空），由于浏览器 deviceId 与系统设备不易可靠对应（按友好名 -/// 匹配在 windows-rs 0.58 上读取属性受限，且同名设备会有歧义），这里返回 `matched=false`， -/// 交由前端基于音频信号的检测兜底，避免误报。后续可在此扩展按名字匹配。 +/// 前端传入 getUserMedia 实际打开的 MediaStreamTrack.label。这里枚举 Windows 采集端点， +/// 按规范化后的友好名唯一匹配，再读取该端点真实的 IAudioEndpointVolume::GetMute 状态。 +/// 找不到或出现同名多个端点时返回 matched=false，绝不猜测静音。 #[cfg(windows)] unsafe fn query_mic_mute(label: Option<&str>) -> Result { - // 指定了特定设备 → 暂不做匹配，交给信号检测 - if label.map(str::trim).filter(|s| !s.is_empty()).is_some() { - return Ok(MicMuteState { matched: false, muted: false }); - } - use windows::Win32::Media::Audio::Endpoints::IAudioEndpointVolume; - use windows::Win32::Media::Audio::{eCapture, eConsole, IMMDeviceEnumerator, MMDeviceEnumerator}; + use windows::Win32::Media::Audio::{ + eCapture, eConsole, IMMDevice, IMMDeviceEnumerator, MMDeviceEnumerator, + DEVICE_STATE_ACTIVE, + }; use windows::Win32::System::Com::{ CoCreateInstance, CoInitializeEx, CoUninitialize, CLSCTX_ALL, CLSCTX_INPROC_SERVER, COINIT_APARTMENTTHREADED, @@ -167,9 +220,55 @@ unsafe fn query_mic_mute(label: Option<&str>) -> Result { let enumerator: IMMDeviceEnumerator = CoCreateInstance(&MMDeviceEnumerator, None, CLSCTX_INPROC_SERVER) .map_err(|e| format!("CoCreateInstance MMDeviceEnumerator: {}", e))?; - let device = enumerator - .GetDefaultAudioEndpoint(eCapture, eConsole) - .map_err(|e| format!("GetDefaultAudioEndpoint(eCapture): {}", e))?; + + let device: IMMDevice = if let Some(target_label) = + label.map(str::trim).filter(|s| !s.is_empty()) + { + let target = normalize_mic_label(target_label); + let devices = enumerator + .EnumAudioEndpoints(eCapture, DEVICE_STATE_ACTIVE) + .map_err(|e| format!("EnumAudioEndpoints(eCapture): {}", e))?; + let count = devices + .GetCount() + .map_err(|e| format!("IMMDeviceCollection.GetCount: {}", e))?; + let mut matched: Option = None; + + for index in 0..count { + let candidate = devices + .Item(index) + .map_err(|e| format!("IMMDeviceCollection.Item({}): {}", index, e))?; + let friendly_name = match get_device_friendly_name(&candidate) { + Ok(name) => name, + Err(_) => continue, + }; + if normalize_mic_label(&friendly_name) != target { + continue; + } + if matched.is_some() { + // 同名设备无法可靠区分；宁可回退信号检测，也不查询错端点。 + return Ok(MicMuteState { + matched: false, + muted: false, + }); + } + matched = Some(candidate); + } + + match matched { + Some(device) => device, + None => { + return Ok(MicMuteState { + matched: false, + muted: false, + }) + } + } + } else { + enumerator + .GetDefaultAudioEndpoint(eCapture, eConsole) + .map_err(|e| format!("GetDefaultAudioEndpoint(eCapture): {}", e))? + }; + let endpoint: IAudioEndpointVolume = device .Activate(CLSCTX_ALL, None) .map_err(|e| format!("Activate IAudioEndpointVolume: {}", e))?; @@ -188,7 +287,7 @@ unsafe fn query_mic_mute(label: Option<&str>) -> Result { } /// 查询麦克风是否被系统静音。 -/// `device_label`：前端选中麦克风的名字（enumerateDevices 的 label）；传 None/空表示系统默认麦克风。 +/// `device_label`：本次 MediaStreamTrack 实际打开的设备名；传 None/空才回退查询系统默认麦克风。 /// 返回 `matched=false` 时表示无法可靠定位设备（如选了特定设备但同名多个），前端应退回信号检测。 #[tauri::command] pub fn get_mic_mute_state(device_label: Option) -> MicMuteState { @@ -211,3 +310,25 @@ pub fn get_mic_mute_state(device_label: Option) -> MicMuteState { MicMuteState { matched: false, muted: false } } } + +#[cfg(test)] +mod tests { + use super::normalize_mic_label; + + #[test] + fn browser_route_prefix_does_not_change_endpoint_identity() { + assert_eq!( + normalize_mic_label("Default - Headset Microphone (Plantronics Blackwire 5220 Series)"), + normalize_mic_label("Headset Microphone (Plantronics Blackwire 5220 Series)") + ); + assert_eq!( + normalize_mic_label("Communications: USB Microphone"), + "usb microphone" + ); + } + + #[test] + fn ordinary_device_name_is_preserved_for_matching() { + assert_eq!(normalize_mic_label(" Studio Mic "), "studio mic"); + } +} diff --git a/client/src-tauri/src/commands/diagnostics.rs b/client/src-tauri/src/commands/diagnostics.rs index 0f9f416..622f838 100644 --- a/client/src-tauri/src/commands/diagnostics.rs +++ b/client/src-tauri/src/commands/diagnostics.rs @@ -9,7 +9,8 @@ use crate::storage::Storage; #[tauri::command] pub fn collect_settings(storage: State) -> Result { let keys = [ - "shortcutPTT", "shortcutPTTCombo", "shortcutHandsFree", + "shortcutPTT", "shortcutPTTCombo", "shortcutHandsFree", "shortcutToggleAi", + "aiEnabled", "aiMinDurationSec", "autoLaunch", "selectedMic", "hotwords", "builtinHotwordSets", "stats", "activePresetId", "audioRetentionDays", "serverUrl", "language", diff --git a/client/src-tauri/src/commands/mod.rs b/client/src-tauri/src/commands/mod.rs index d04c8ed..e826169 100644 --- a/client/src-tauri/src/commands/mod.rs +++ b/client/src-tauri/src/commands/mod.rs @@ -7,4 +7,5 @@ pub mod paste; pub mod shortcuts; pub mod storage; pub mod system; +pub mod tray; pub mod window; diff --git a/client/src-tauri/src/commands/paste.rs b/client/src-tauri/src/commands/paste.rs index b02a6fb..bc94555 100644 --- a/client/src-tauri/src/commands/paste.rs +++ b/client/src-tauri/src/commands/paste.rs @@ -1,9 +1,9 @@ -use serde::Serialize; -use serde_json::Value; -use tauri::{AppHandle, State}; use crate::context::{AppContext, ContextDetector}; use crate::inject; use crate::window::WindowState; +use serde::Serialize; +use serde_json::Value; +use tauri::{AppHandle, State}; #[derive(Serialize)] pub struct PasteResult { @@ -36,13 +36,16 @@ pub fn paste_text( .as_deref() .and_then(|f| f.parse::().ok()) .unwrap_or(0); - crate::commands::system::write_log_line( - &format!("[RUST] [paste] pre-probed hwnd target={} focus={}", target_val, focus_val) - ); + crate::commands::system::write_log_line(&format!( + "[RUST] [paste] pre-probed hwnd target={} focus={}", + target_val, focus_val + )); inject::inject_text_to_hwnd(&text, target_val, focus_val, restore_clipboard) } _ => { - crate::commands::system::write_log_line("[RUST] [paste] no pre-probed hwnd, fallback to inject_text"); + crate::commands::system::write_log_line( + "[RUST] [paste] no pre-probed hwnd, fallback to inject_text", + ); inject::inject_text(&text, restore_clipboard) } }; @@ -83,9 +86,12 @@ pub fn get_probe_result(detector: State) -> Result) -> Result { +pub fn get_recording_context( + include_text_context: Option, + detector: State, +) -> Result { let started_at = chrono::Utc::now().timestamp_millis(); - let ctx = detector.capture("recording_start"); + let ctx = detector.capture_recording("recording_start", include_text_context.unwrap_or(false)); let completed_at = chrono::Utc::now().timestamp_millis(); let app_context = serde_json::to_value(&ctx).map_err(|e| e.to_string())?; let probe = probe_result_for_context(&ctx, started_at, completed_at); @@ -96,12 +102,7 @@ pub fn get_recording_context(detector: State) -> Result Value { - +fn probe_result_for_context(ctx: &AppContext, started_at: i64, completed_at: i64) -> Value { // Determine editability using the same heuristic as inject let editable = crate::inject::is_likely_editable_pub(ctx); @@ -136,6 +137,8 @@ fn probe_result_for_context( "controlType": &ctx.control_type, "automationId": &ctx.automation_id, "isValuePatternAvailable": ctx.is_value_pattern_available, + "isTextPatternAvailable": ctx.is_text_pattern_available, + "isTextPattern2Available": ctx.is_text_pattern2_available, "isKeyboardFocusable": ctx.is_keyboard_focusable, "isEnabled": ctx.is_enabled, "isReadOnly": ctx.is_read_only, diff --git a/client/src-tauri/src/commands/shortcuts.rs b/client/src-tauri/src/commands/shortcuts.rs index a34893c..d85717e 100644 --- a/client/src-tauri/src/commands/shortcuts.rs +++ b/client/src-tauri/src/commands/shortcuts.rs @@ -21,15 +21,17 @@ pub fn shortcuts_changed( let ptt_str = ptt_setting.as_str().unwrap_or("ControlRight"); let hf_val = storage.get("shortcutHandsFree", None); let hf_key = hf_val.as_str().unwrap_or("AltRight"); + let ai_toggle_val = storage.get("shortcutToggleAi", None); + let ai_toggle_key = ai_toggle_val.as_str().unwrap_or(""); - // Reconfigure PTT + hands-free keyboard hook - hook.reconfigure(&app, ptt_str, hf_key); + // Reconfigure PTT + hands-free + AI cleanup single-key/mouse hook + hook.reconfigure(&app, ptt_str, hf_key, ai_toggle_key); // 重新注册所有 global_shortcut（免提组合键 + 各润色模式切换快捷键） register_all_global_shortcuts(&app, storage.inner()); } -/// 集中注册所有 global_shortcut：免提组合键 + 各润色模式切换快捷键。 +/// 集中注册所有 global_shortcut：免提组合键、AI 整理开关和各润色模式切换快捷键。 /// /// `unregister_all` 会清空全部已注册的 global_shortcut，因此这两类必须在同一处 /// 一次性重注册，否则会互相覆盖。单键（PTT/免提单键）不走这里，由底层键盘钩子处理。 @@ -57,6 +59,22 @@ pub fn register_all_global_shortcuts(app: &AppHandle, storage: &Storage) { } } + // AI 整理开关快捷键。留空即不注册，避免默认占用用户已有组合键。 + let ai_toggle = storage.get("shortcutToggleAi", None); + let ai_toggle_key = ai_toggle.as_str().unwrap_or("").to_string(); + if !ai_toggle_key.is_empty() && ai_toggle_key.contains('+') { + let ai_toggle_for_log = ai_toggle_key.clone(); + if let Err(e) = gs.on_shortcut(ai_toggle_key.as_str(), move |app, _shortcut, event| { + if event.state == ShortcutState::Pressed { + let _ = app.emit("toggle-ai-cleanup", serde_json::json!({ "source": "globalShortcut" })); + } + }) { + log::warn!("Failed to register AI cleanup shortcut '{}': {}", ai_toggle_for_log, e); + } else { + log::info!("Registered AI cleanup shortcut: {}", ai_toggle_for_log); + } + } + // 润色模式切换快捷键：presetShortcuts = { presetId: 组合键 } let preset_map = storage.get("presetShortcuts", None); if let Some(obj) = preset_map.as_object() { diff --git a/client/src-tauri/src/commands/tray.rs b/client/src-tauri/src/commands/tray.rs new file mode 100644 index 0000000..b3776ee --- /dev/null +++ b/client/src-tauri/src/commands/tray.rs @@ -0,0 +1,192 @@ +//! SayIt 自定义托盘菜单窗口。 +//! +//! Windows 原生菜单有系统级最小宽度，缩短文案也不会继续变窄；这里改用一个预创建、 +//! 隐藏的轻量 WebView 窗口。右键时只定位并显示，避免每次创建 WebView 的延迟。 + +use crate::storage::Storage; +use serde_json::json; +use tauri::{ + AppHandle, Emitter, LogicalSize, Manager, PhysicalPosition, Position, Size, State, + WebviewUrl, WebviewWindowBuilder, WindowEvent, +}; + +pub const TRAY_MENU_LABEL: &str = "tray-menu"; +const TRAY_MENU_WIDTH: f64 = 140.0; +const TRAY_MENU_HEIGHT: f64 = 112.0; + +pub fn create_tray_menu_window(app: &AppHandle) -> tauri::Result<()> { + if app.get_webview_window(TRAY_MENU_LABEL).is_some() { + return Ok(()); + } + + let window = WebviewWindowBuilder::new( + app, + TRAY_MENU_LABEL, + WebviewUrl::App("tray-menu.html".into()), + ) + .title("SayIt") + .inner_size(TRAY_MENU_WIDTH, TRAY_MENU_HEIGHT) + .decorations(false) + .transparent(true) + .shadow(false) + .always_on_top(true) + .skip_taskbar(true) + .resizable(false) + .focused(false) + .visible(false) + .build()?; + + // 原生菜单点击外部会自动收起；自定义窗口用失焦还原同样的交互。 + let window_to_hide = window.clone(); + window.on_window_event(move |event| { + if matches!(event, WindowEvent::Focused(false)) { + let _ = window_to_hide.hide(); + } + }); + + Ok(()) +} + +/// 在托盘图标旁边显示菜单。`position` 是 Tauri 给出的全局物理坐标。 +pub fn show_tray_menu(app: &AppHandle, position: PhysicalPosition) { + let Some(window) = app.get_webview_window(TRAY_MENU_LABEL) else { + log::warn!("[tray-menu] window is missing"); + return; + }; + + let monitor = app + .available_monitors() + .unwrap_or_default() + .into_iter() + .find(|monitor| { + let origin = monitor.position(); + let size = monitor.size(); + position.x >= origin.x as f64 + && position.x < (origin.x as f64 + size.width as f64) + && position.y >= origin.y as f64 + && position.y < (origin.y as f64 + size.height as f64) + }) + .or_else(|| app.primary_monitor().ok().flatten()); + + let (left, top, right, bottom, scale) = monitor + .as_ref() + .map(|monitor| { + let origin = monitor.position(); + let size = monitor.size(); + ( + origin.x as f64, + origin.y as f64, + origin.x as f64 + size.width as f64, + origin.y as f64 + size.height as f64, + monitor.scale_factor(), + ) + }) + .unwrap_or((0.0, 0.0, 1920.0, 1080.0, 1.0)); + + let width = TRAY_MENU_WIDTH * scale; + let height = TRAY_MENU_HEIGHT * scale; + let gap = 8.0 * scale; + let edge = 8.0 * scale; + let monitor_width = right - left; + let monitor_height = bottom - top; + + // 兼容任务栏停靠在上、下、左、右。常见的底部任务栏让菜单右边缘稍微越过 + // 光标中心，与 Windows 原生菜单从托盘图标向左展开的视觉锚点一致。 + let (mut x, mut y) = if position.y >= top + monitor_height * 0.75 { + (position.x - width + 22.0 * scale, position.y - height - gap) + } else if position.y <= top + monitor_height * 0.25 { + (position.x - width + 22.0 * scale, position.y + gap) + } else if position.x <= left + monitor_width * 0.25 { + (position.x + gap, position.y - height / 2.0) + } else { + (position.x - width - gap, position.y - height / 2.0) + }; + x = x.clamp(left + edge, (right - width - edge).max(left + edge)); + y = y.clamp(top + edge, (bottom - height - edge).max(top + edge)); + + let _ = window.set_size(Size::Logical(LogicalSize::new( + TRAY_MENU_WIDTH, + TRAY_MENU_HEIGHT, + ))); + let _ = window.set_position(Position::Physical(PhysicalPosition::new( + x.round() as i32, + y.round() as i32, + ))); + let _ = window.show(); + let _ = window.set_always_on_top(true); + let _ = window.set_focus(); + + let enabled = current_ai_enabled(app.state::().inner()); + let _ = window.emit("tray-menu-open", json!({ "enabled": enabled })); +} + +fn current_ai_enabled(storage: &Storage) -> bool { + storage.get("aiEnabled", None).as_bool().unwrap_or(true) +} + +fn hide_menu(app: &AppHandle) { + if let Some(window) = app.get_webview_window(TRAY_MENU_LABEL) { + let _ = window.hide(); + } +} + +fn show_main(app: &AppHandle) { + hide_menu(app); + if let Some(window) = app.get_webview_window("main") { + let _ = window.show(); + let _ = window.unminimize(); + let _ = window.set_focus(); + } +} + +#[tauri::command] +pub fn set_tray_ai_enabled(app: AppHandle, enabled: bool) -> Result<(), String> { + // 主窗口里的开关已经负责落库；这里只把最新状态推给独立的托盘菜单 WebView。 + let _ = app.emit_to( + TRAY_MENU_LABEL, + "tray-ai-state", + json!({ "enabled": enabled }), + ); + Ok(()) +} + +#[tauri::command] +pub fn get_tray_ai_enabled(storage: State<'_, Storage>) -> bool { + current_ai_enabled(storage.inner()) +} + +#[tauri::command] +pub fn toggle_tray_ai_enabled( + app: AppHandle, + storage: State<'_, Storage>, +) -> Result { + let next = !current_ai_enabled(storage.inner()); + storage + .set("aiEnabled", &json!(next)) + .map_err(|error| error.to_string())?; + + // 主窗口 store 会同步内存与录音器缓存；托盘窗口也立即刷新自己的图标。 + let _ = app.emit("ai-cleanup-changed", json!({ "enabled": next })); + let _ = app.emit_to( + TRAY_MENU_LABEL, + "tray-ai-state", + json!({ "enabled": next }), + ); + log::info!("[tray-menu] AI cleanup toggled to {next}"); + Ok(next) +} + +#[tauri::command] +pub fn show_main_from_tray(app: AppHandle) { + show_main(&app); +} + +#[tauri::command] +pub fn hide_tray_menu(app: AppHandle) { + hide_menu(&app); +} + +#[tauri::command] +pub fn quit_from_tray(app: AppHandle) { + app.exit(0); +} diff --git a/client/src-tauri/src/context/mod.rs b/client/src-tauri/src/context/mod.rs index 68a0acc..762cb27 100644 --- a/client/src-tauri/src/context/mod.rs +++ b/client/src-tauri/src/context/mod.rs @@ -16,10 +16,26 @@ use windows::Win32::Graphics::Gdi::{ }; #[cfg(windows)] use windows::Win32::UI::WindowsAndMessaging::{ - GetClassNameW, GetForegroundWindow, GetGUIThreadInfo, GetWindowTextW, - GetWindowThreadProcessId, GUITHREADINFO, + GetClassNameW, GetForegroundWindow, GetGUIThreadInfo, GetWindowTextW, GetWindowThreadProcessId, + GUITHREADINFO, }; +#[derive(Debug, Clone, Serialize, Default)] +#[serde(rename_all = "camelCase")] +pub struct TextContext { + /// UI Automation pattern used to obtain the text. Useful for diagnostics only. + pub source: String, + /// Text immediately before the caret/selection. Capped locally before serialization. + pub text_before: String, + /// Selected text. Empty means this is ordinary dictation rather than an edit command. + pub selected_text: String, + /// Text immediately after the caret/selection. Capped locally before serialization. + pub text_after: String, + /// A large selection is deliberately not sent to AI, because replacing a partial selection + /// would risk data loss. + pub selection_truncated: bool, +} + #[derive(Debug, Clone, Serialize, Default)] pub struct AppContext { pub reason: String, @@ -47,12 +63,22 @@ pub struct AppContext { pub automation_id: String, #[serde(rename = "isValuePatternAvailable")] pub is_value_pattern_available: bool, + #[serde(rename = "isTextPatternAvailable")] + pub is_text_pattern_available: bool, + #[serde(rename = "isTextPattern2Available")] + pub is_text_pattern2_available: bool, #[serde(rename = "isKeyboardFocusable")] pub is_keyboard_focusable: bool, #[serde(rename = "isEnabled")] pub is_enabled: bool, #[serde(rename = "isReadOnly")] pub is_read_only: Option, + #[serde(rename = "isPassword")] + pub is_password: bool, + /// Present only when the user explicitly enabled context-aware writing. Ordinary foreground + /// tracking and diagnostics never read or retain editor text. + #[serde(rename = "textContext", skip_serializing_if = "Option::is_none")] + pub text_context: Option, } /// 前台窗口所在显示器的物理工作区，已排除任务栏。 @@ -80,7 +106,16 @@ impl ContextDetector { /// Capture the current foreground window context pub fn capture(&self, reason: &str) -> AppContext { - let ctx = capture_context(reason); + let ctx = capture_context_with_text(reason, false); + *self.cached_context.lock().unwrap() = Some(ctx.clone()); + ctx + } + + /// Capture the recording target, optionally including a bounded slice of editor text. + /// This separate entry point is intentional: WinEvent hooks must never start reading text + /// merely because the feature exists. + pub fn capture_recording(&self, reason: &str, include_text_context: bool) -> AppContext { + let ctx = capture_context_with_text(reason, include_text_context); *self.cached_context.lock().unwrap() = Some(ctx.clone()); ctx } @@ -104,11 +139,9 @@ impl ContextDetector { let cached_for_cb = cached.clone(); thread::spawn(move || { + use windows::Win32::UI::Accessibility::{SetWinEventHook, HWINEVENTHOOK}; use windows::Win32::UI::WindowsAndMessaging::{ - GetMessageW, TranslateMessage, DispatchMessageW, MSG, - }; - use windows::Win32::UI::Accessibility::{ - SetWinEventHook, HWINEVENTHOOK, + DispatchMessageW, GetMessageW, TranslateMessage, MSG, }; // WinEvent constants @@ -162,7 +195,8 @@ impl ContextDetector { EVENT_SYSTEM_FOREGROUND, None, Some(winevent_proc), - 0, 0, + 0, + 0, WINEVENT_OUTOFCONTEXT, ); @@ -171,7 +205,8 @@ impl ContextDetector { EVENT_OBJECT_FOCUS, None, Some(winevent_proc), - 0, 0, + 0, + 0, WINEVENT_OUTOFCONTEXT, ); @@ -192,6 +227,11 @@ impl ContextDetector { /// Standalone capture function (no &self needed, usable from callbacks) #[cfg(windows)] pub fn capture_context(reason: &str) -> AppContext { + capture_context_with_text(reason, false) +} + +#[cfg(windows)] +fn capture_context_with_text(reason: &str, include_text_context: bool) -> AppContext { let mut ctx = AppContext { reason: reason.to_string(), timestamp: chrono::Utc::now().timestamp_millis(), @@ -234,7 +274,7 @@ pub fn capture_context(reason: &str) -> AppContext { // UI Automation: get focused element's ControlType, ValuePattern, etc. // This is critical for Chromium-class windows where GetGUIThreadInfo // doesn't report caret/focus reliably. - populate_uia_fields(&mut ctx); + populate_uia_fields(&mut ctx, include_text_context); } ctx } @@ -276,29 +316,27 @@ pub fn capture_foreground_monitor() -> Option { /// Populates control_type, automation_id, is_value_pattern_available, /// is_keyboard_focusable, is_enabled, is_read_only in AppContext. #[cfg(windows)] -unsafe fn populate_uia_fields(ctx: &mut AppContext) { +unsafe fn populate_uia_fields(ctx: &mut AppContext, include_text_context: bool) { use windows::Win32::System::Com::{ - CoCreateInstance, CoInitializeEx, CoUninitialize, - CLSCTX_INPROC_SERVER, COINIT_APARTMENTTHREADED, + CoCreateInstance, CoInitializeEx, CoUninitialize, CLSCTX_INPROC_SERVER, + COINIT_APARTMENTTHREADED, }; use windows::Win32::UI::Accessibility::{ - CUIAutomation, IUIAutomation, - UIA_ControlTypePropertyId, UIA_AutomationIdPropertyId, - UIA_IsValuePatternAvailablePropertyId, UIA_IsKeyboardFocusablePropertyId, - UIA_IsEnabledPropertyId, UIA_ValueIsReadOnlyPropertyId, + CUIAutomation, IUIAutomation, UIA_AutomationIdPropertyId, UIA_ControlTypePropertyId, + UIA_IsEnabledPropertyId, UIA_IsKeyboardFocusablePropertyId, UIA_IsPasswordPropertyId, + UIA_IsTextPattern2AvailablePropertyId, UIA_IsTextPatternAvailablePropertyId, + UIA_IsValuePatternAvailablePropertyId, UIA_ValueIsReadOnlyPropertyId, }; // CoInitialize for this thread (may already be initialized — that's ok) let co_init_result = CoInitializeEx(None, COINIT_APARTMENTTHREADED); let need_co_uninit = co_init_result.is_ok(); let result = (|| -> Result<(), String> { - let uia: IUIAutomation = CoCreateInstance( - &CUIAutomation, - None, - CLSCTX_INPROC_SERVER, - ).map_err(|e| format!("CoCreateInstance CUIAutomation: {}", e))?; + let uia: IUIAutomation = CoCreateInstance(&CUIAutomation, None, CLSCTX_INPROC_SERVER) + .map_err(|e| format!("CoCreateInstance CUIAutomation: {}", e))?; - let focused = uia.GetFocusedElement() + let focused = uia + .GetFocusedElement() .map_err(|e| format!("GetFocusedElement: {}", e))?; // ControlType (int → string name) @@ -325,6 +363,29 @@ unsafe fn populate_uia_fields(ctx: &mut AppContext) { } } + let tp_val = focused.GetCurrentPropertyValue(UIA_IsTextPatternAvailablePropertyId); + if let Ok(v) = tp_val { + if let Ok(b) = bool::try_from(&v) { + ctx.is_text_pattern_available = b; + } + } + + let tp2_val = focused.GetCurrentPropertyValue(UIA_IsTextPattern2AvailablePropertyId); + if let Ok(v) = tp2_val { + if let Ok(b) = bool::try_from(&v) { + ctx.is_text_pattern2_available = b; + } + } + + // Password controls are a hard boundary. Never ask a pattern for their value or text, + // even when the user enabled context-aware writing globally. + let password_val = focused.GetCurrentPropertyValue(UIA_IsPasswordPropertyId); + if let Ok(v) = password_val { + if let Ok(b) = bool::try_from(&v) { + ctx.is_password = b; + } + } + // IsKeyboardFocusable let kf_val = focused.GetCurrentPropertyValue(UIA_IsKeyboardFocusablePropertyId); if let Ok(v) = kf_val { @@ -351,6 +412,10 @@ unsafe fn populate_uia_fields(ctx: &mut AppContext) { } } + if include_text_context && !ctx.is_password && ctx.is_enabled { + ctx.text_context = capture_uia_text_context(&uia, &focused); + } + Ok(()) })(); @@ -363,6 +428,206 @@ unsafe fn populate_uia_fields(ctx: &mut AppContext) { } } +#[cfg(windows)] +const TEXT_BEFORE_LIMIT: i32 = 500; +#[cfg(windows)] +const TEXT_AFTER_LIMIT: i32 = 300; +#[cfg(windows)] +const SELECTED_TEXT_LIMIT: i32 = 6000; + +/// Read a small, caret-relative slice rather than the whole document. TextPattern is attempted on +/// the focused element and a few ancestors because Chromium/Office often put it on the enclosing +/// Document element. ValuePattern is a conservative fallback for simple inputs and is treated as +/// text-before-caret only (most such controls expose no selection/caret API through UIA). +#[cfg(windows)] +unsafe fn capture_uia_text_context( + uia: &windows::Win32::UI::Accessibility::IUIAutomation, + focused: &windows::Win32::UI::Accessibility::IUIAutomationElement, +) -> Option { + use windows::core::Interface; + use windows::Win32::UI::Accessibility::{ + IUIAutomationElement, IUIAutomationTextPattern, IUIAutomationTextPattern2, + IUIAutomationValuePattern, UIA_TextPattern2Id, UIA_TextPatternId, UIA_ValuePatternId, + }; + + let mut element: IUIAutomationElement = focused.clone(); + let walker = uia.RawViewWalker().ok(); + let value_fallback = focused + .GetCurrentPatternAs::(UIA_ValuePatternId) + .ok() + .and_then(|pattern| pattern.CurrentValue().ok()) + .map(|value| sanitize_uia_text(&value.to_string())) + .filter(|value| !value.is_empty()); + + for _depth in 0..=4 { + if let Ok(pattern2) = + element.GetCurrentPatternAs::(UIA_TextPattern2Id) + { + let mut active = windows::Win32::Foundation::BOOL::default(); + let caret = pattern2 + .GetCaretRange(&mut active) + .ok() + .filter(|_| active.as_bool()); + let base: IUIAutomationTextPattern = pattern2.cast().ok()?; + if let Some(context) = text_context_from_pattern(&base, caret, "text_pattern2") { + return Some(context); + } + } + + if let Ok(pattern) = + element.GetCurrentPatternAs::(UIA_TextPatternId) + { + if let Some(context) = text_context_from_pattern(&pattern, None, "text_pattern") { + return Some(context); + } + } + + let Some(ref tree_walker) = walker else { break }; + let Ok(parent) = tree_walker.GetParentElement(&element) else { + break; + }; + element = parent; + } + + value_fallback.map(|value| TextContext { + source: "value_pattern_tail".to_string(), + text_before: tail_chars(&value, TEXT_BEFORE_LIMIT as usize), + ..Default::default() + }) +} + +#[cfg(windows)] +unsafe fn text_context_from_pattern( + pattern: &windows::Win32::UI::Accessibility::IUIAutomationTextPattern, + caret: Option, + source: &str, +) -> Option { + use windows::Win32::UI::Accessibility::IUIAutomationTextRange; + + let selection_range = pattern.GetSelection().ok().and_then(|ranges| { + (ranges.Length().ok().unwrap_or(0) > 0) + .then(|| ranges.GetElement(0).ok()) + .flatten() + }); + + let mut selected_text = String::new(); + let mut selection_truncated = false; + if let Some(ref range) = selection_range { + if let Ok(text) = range.GetText(SELECTED_TEXT_LIMIT + 1) { + let raw = sanitize_uia_text(&text.to_string()); + if raw.chars().count() > SELECTED_TEXT_LIMIT as usize { + // Do not send a partial selection to AI: its output would replace the complete + // selection and could silently discard content it never saw. + selection_truncated = true; + } else if !raw.trim().is_empty() { + selected_text = raw; + } + } + } + + let anchor: IUIAutomationTextRange = if !selected_text.is_empty() { + selection_range.clone()? + } else { + caret.or(selection_range)? + }; + + let text_before = text_before_range(&anchor).unwrap_or_default(); + let text_after = text_after_range(&anchor).unwrap_or_default(); + if text_before.is_empty() + && selected_text.is_empty() + && text_after.is_empty() + && !selection_truncated + { + return None; + } + + Some(TextContext { + source: source.to_string(), + text_before, + selected_text, + text_after, + selection_truncated, + }) +} + +#[cfg(windows)] +unsafe fn text_before_range( + anchor: &windows::Win32::UI::Accessibility::IUIAutomationTextRange, +) -> Option { + use windows::Win32::UI::Accessibility::{ + TextPatternRangeEndpoint_End, TextPatternRangeEndpoint_Start, TextUnit_Character, + }; + let range = anchor.Clone().ok()?; + range + .MoveEndpointByRange( + TextPatternRangeEndpoint_End, + anchor, + TextPatternRangeEndpoint_Start, + ) + .ok()?; + range + .MoveEndpointByRange( + TextPatternRangeEndpoint_Start, + anchor, + TextPatternRangeEndpoint_Start, + ) + .ok()?; + let _ = range.MoveEndpointByUnit( + TextPatternRangeEndpoint_Start, + TextUnit_Character, + -TEXT_BEFORE_LIMIT, + ); + range + .GetText(TEXT_BEFORE_LIMIT) + .ok() + .map(|v| sanitize_uia_text(&v.to_string())) +} + +#[cfg(windows)] +unsafe fn text_after_range( + anchor: &windows::Win32::UI::Accessibility::IUIAutomationTextRange, +) -> Option { + use windows::Win32::UI::Accessibility::{ + TextPatternRangeEndpoint_End, TextPatternRangeEndpoint_Start, TextUnit_Character, + }; + let range = anchor.Clone().ok()?; + range + .MoveEndpointByRange( + TextPatternRangeEndpoint_Start, + anchor, + TextPatternRangeEndpoint_End, + ) + .ok()?; + range + .MoveEndpointByRange( + TextPatternRangeEndpoint_End, + anchor, + TextPatternRangeEndpoint_End, + ) + .ok()?; + let _ = range.MoveEndpointByUnit( + TextPatternRangeEndpoint_End, + TextUnit_Character, + TEXT_AFTER_LIMIT, + ); + range + .GetText(TEXT_AFTER_LIMIT) + .ok() + .map(|v| sanitize_uia_text(&v.to_string())) +} + +fn sanitize_uia_text(value: &str) -> String { + value + .replace('\0', "") + .replace("\r\n", "\n") + .replace('\r', "\n") +} + +fn tail_chars(value: &str, limit: usize) -> String { + let count = value.chars().count(); + value.chars().skip(count.saturating_sub(limit)).collect() +} + /// Map UIA ControlType ID to a human-readable name. #[cfg(windows)] fn uia_control_type_name(id: i32) -> String { @@ -430,22 +695,29 @@ pub fn capture_foreground_monitor() -> Option { pub unsafe fn read_window_text(hwnd: HWND) -> String { let mut buf = [0u16; 512]; let len = GetWindowTextW(hwnd, &mut buf); - if len > 0 { String::from_utf16_lossy(&buf[..len as usize]) } else { String::new() } + if len > 0 { + String::from_utf16_lossy(&buf[..len as usize]) + } else { + String::new() + } } #[cfg(windows)] pub unsafe fn read_class_name(hwnd: HWND) -> String { let mut buf = [0u16; 256]; let len = GetClassNameW(hwnd, &mut buf); - if len > 0 { String::from_utf16_lossy(&buf[..len as usize]) } else { String::new() } + if len > 0 { + String::from_utf16_lossy(&buf[..len as usize]) + } else { + String::new() + } } - /// pid → 可执行文件名（不含路径）。inject 模块用它指认「是谁占着剪贴板」。 #[cfg(windows)] pub(crate) fn get_process_name(pid: u32) -> String { - use windows::Win32::System::Threading::{OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION}; use windows::Win32::Foundation::CloseHandle; + use windows::Win32::System::Threading::{OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION}; unsafe { match OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, false, pid) { Ok(h) => { @@ -460,8 +732,8 @@ pub(crate) fn get_process_name(pid: u32) -> String { #[cfg(windows)] fn get_process_path(pid: u32) -> String { - use windows::Win32::System::Threading::{OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION}; use windows::Win32::Foundation::CloseHandle; + use windows::Win32::System::Threading::{OpenProcess, PROCESS_QUERY_LIMITED_INFORMATION}; unsafe { match OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, false, pid) { Ok(h) => { diff --git a/client/src-tauri/src/keyboard/mod.rs b/client/src-tauri/src/keyboard/mod.rs index 8d803eb..c3592cb 100644 --- a/client/src-tauri/src/keyboard/mod.rs +++ b/client/src-tauri/src/keyboard/mod.rs @@ -400,6 +400,10 @@ struct HookSharedState { /// VK codes for hands-free toggle key (empty = not using hook for hands-free) hf_vk_codes: Vec, hf_setting: String, + /// AI 整理开关支持单键与鼠标按键；组合键仍由 global_shortcut 处理。 + ai_toggle_key_down: AtomicBool, + ai_toggle_vk_codes: Vec, + ai_toggle_setting: String, app_handle: AppHandle, } @@ -410,6 +414,7 @@ enum HookAction { PttDown { vk: u32, gen: u64 }, PttUp { vk: u32, gen: u64, reason: &'static str }, HfToggle { vk: u32 }, + AiToggle { vk: u32 }, Escape { mode: u32, token: u64 }, Diag { vk: u32, msg_name: &'static str, flags: u32, scan_code: u32 }, Shutdown, @@ -1021,11 +1026,11 @@ impl KeyboardHookManager { } } - /// Start the keyboard hook with the given PTT setting and optional hands-free setting - pub fn start(&self, app: &AppHandle, ptt_setting: &str, hf_setting: &str) { + /// Start the keyboard hook with PTT, hands-free, and AI-cleanup single-key settings. + pub fn start(&self, app: &AppHandle, ptt_setting: &str, hf_setting: &str, ai_toggle_setting: &str) { crate::commands::system::write_log_line(&format!( - "[ptt-lifecycle] start() called ptt_setting={} hf_setting={}", - ptt_setting, hf_setting, + "[ptt-lifecycle] start() called ptt_setting={} hf_setting={} ai_toggle_setting={}", + ptt_setting, hf_setting, ai_toggle_setting, )); let has_hook_thread = self.hook_thread.lock().unwrap().is_some(); if self.running.load(Ordering::SeqCst) || has_hook_thread { @@ -1038,6 +1043,11 @@ impl KeyboardHookManager { } else { vec![] // combo key — handled by global_shortcut, not hook }; + let ai_toggle_vk_codes = if is_single_key_setting(ai_toggle_setting) { + vk_codes_for_setting(ai_toggle_setting) + } else { + vec![] // combo key — handled by global_shortcut, not hook + }; let ptt_full_mask = if ptt_config.vk_codes.is_empty() { 0 } else { @@ -1062,6 +1072,9 @@ impl KeyboardHookManager { hf_key_down: AtomicBool::new(false), hf_vk_codes, hf_setting: hf_setting.to_string(), + ai_toggle_key_down: AtomicBool::new(false), + ai_toggle_vk_codes, + ai_toggle_setting: ai_toggle_setting.to_string(), app_handle: app.clone(), }); @@ -1157,16 +1170,16 @@ impl KeyboardHookManager { *self.shared_state.lock().unwrap() = None; } - /// Reconfigure with new PTT and hands-free settings - pub fn reconfigure(&self, app: &AppHandle, ptt_setting: &str, hf_setting: &str) { + /// Reconfigure with new PTT, hands-free, and AI-cleanup settings. + pub fn reconfigure(&self, app: &AppHandle, ptt_setting: &str, hf_setting: &str, ai_toggle_setting: &str) { let count = RECONFIGURE_COUNT.fetch_add(1, Ordering::SeqCst) + 1; crate::commands::system::write_log_line(&format!( - "[ptt-lifecycle] reconfigure() #{} ptt_setting={} hf_setting={} — waiting for old hook shutdown", - count, ptt_setting, hf_setting, + "[ptt-lifecycle] reconfigure() #{} ptt_setting={} hf_setting={} ai_toggle_setting={} — waiting for old hook shutdown", + count, ptt_setting, hf_setting, ai_toggle_setting, )); self.stop(); // stop() 已 join 旧 hook 与 dispatcher，可直接启动新实例。 - self.start(app, ptt_setting, hf_setting); + self.start(app, ptt_setting, hf_setting, ai_toggle_setting); } /// Set hands-free mode active (suppresses PTT up events temporarily) @@ -1251,6 +1264,15 @@ impl KeyboardHookManager { "vk": vk, })); } + HookAction::AiToggle { vk } => { + crate::commands::system::write_log_line( + &format!("[RUST] [ai-toggle] vk={} setting={}", vk, dispatch_state.ai_toggle_setting) + ); + let _ = dispatch_state.app_handle.emit("toggle-ai-cleanup", serde_json::json!({ + "source": "rust_hook", + "vk": vk, + })); + } HookAction::Escape { mode, token } => { let mode_name = escape_action_mode_name(mode); crate::commands::system::write_log_line( @@ -1494,6 +1516,10 @@ unsafe extern "system" fn low_level_mouse_proc( let is_hf_key = !state.hf_vk_codes.is_empty() && state.hf_vk_codes.contains(&vk) && !is_ptt_key; // 同一键时 PTT 优先 + let is_ai_toggle_key = !state.ai_toggle_vk_codes.is_empty() + && state.ai_toggle_vk_codes.contains(&vk) + && !is_ptt_key + && !is_hf_key; // PTT / 免提高于 AI 开关 if let Some(member_index) = ptt_member { // 鼠标按键只允许旧单键格式；按下和抬起都吞掉，避免误触发其它程序导航。 @@ -1560,6 +1586,24 @@ unsafe extern "system" fn low_level_mouse_proc( consumed = true; } } + + if is_ai_toggle_key { + if is_down { + consumed = true; + if begin_hf_press(&state.ai_toggle_key_down) { + HOOK_ACTION_TX.with(|tx| { + if let Some(sender) = tx.borrow().as_ref() { + if sender.send(HookAction::AiToggle { vk }).is_err() { + TRY_SEND_FAIL_COUNT.fetch_add(1, Ordering::Relaxed); + } + } + }); + } + } + if is_up && end_hf_press(&state.ai_toggle_key_down) { + consumed = true; + } + } } }); @@ -1745,6 +1789,10 @@ unsafe extern "system" fn low_level_keyboard_proc( let is_hf_key = !state.hf_vk_codes.is_empty() && state.hf_vk_codes.contains(&vk) && !is_ptt_key; // PTT takes priority if same key + let is_ai_toggle_key = !state.ai_toggle_vk_codes.is_empty() + && state.ai_toggle_vk_codes.contains(&vk) + && !is_ptt_key + && !is_hf_key; // PTT / hands-free take priority if misconfigured // Alt 键（VK_LMENU=0xA4 / VK_RMENU=0xA5）作为单键热键时，keyup 也必须吞掉。 // 否则被放行的 Alt 抬起会被 Windows 判为「单击了一下 Alt」→ 激活前台程序的 @@ -1893,6 +1941,26 @@ unsafe extern "system" fn low_level_keyboard_proc( } } } + + if is_ai_toggle_key { + let is_up = msg == WM_KEYUP || msg == WM_SYSKEYUP; + let is_down = msg == WM_KEYDOWN || msg == WM_SYSKEYDOWN; + if is_down { + consumed = true; + if begin_hf_press(&state.ai_toggle_key_down) { + HOOK_ACTION_TX.with(|tx| { + if let Some(sender) = tx.borrow().as_ref() { + if sender.send(HookAction::AiToggle { vk }).is_err() { + TRY_SEND_FAIL_COUNT.fetch_add(1, Ordering::Relaxed); + } + } + }); + } + } + if is_up && (end_hf_press(&state.ai_toggle_key_down) || is_alt_key) { + consumed = true; + } + } } }); diff --git a/client/src-tauri/src/locale.rs b/client/src-tauri/src/locale.rs index 753e8a9..7913b6e 100644 --- a/client/src-tauri/src/locale.rs +++ b/client/src-tauri/src/locale.rs @@ -13,8 +13,6 @@ //! 两者可以不一致 —— 英文版 Windows 把区域设成中国是很常见的组合。 //! 界面语言只能跟显示语言，跟区域格式会把英文用户判成中文。 -use crate::storage::Storage; - /// 受支持的界面语言。取值与前端 `Locale` 一一对应。 #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Lang { @@ -66,38 +64,6 @@ pub fn system_ui_lang() -> Lang { } } -/// 当前实际生效的界面语言：`ui.language` 为 `auto`（或缺失/脏值）时按系统显示语言定。 -pub fn effective_lang(storage: &Storage) -> Lang { - match storage.get("ui.language", None).as_str() { - Some("zh-CN") => Lang::ZhCn, - Some("en") => Lang::En, - _ => system_ui_lang(), - } -} - -/// 托盘文案。刻意写死在 Rust 里而不去读前端的 locale JSON： -/// 只有三条，且托盘在前端加载之前就要建好，为它引入一条读文件的启动路径不值。 -pub struct TrayStrings { - pub show: &'static str, - pub quit: &'static str, - pub tooltip: &'static str, -} - -pub fn tray_strings(lang: Lang) -> TrayStrings { - match lang { - Lang::ZhCn => TrayStrings { - show: "显示主窗口", - quit: "退出 SayIt", - tooltip: "SayIt — 随口说，出色写", - }, - Lang::En => TrayStrings { - show: "Show main window", - quit: "Quit SayIt", - tooltip: "SayIt — Speak freely, write well", - }, - } -} - #[cfg(test)] mod tests { use super::*; diff --git a/client/src-tauri/src/main.rs b/client/src-tauri/src/main.rs index 665abd2..a14df87 100644 --- a/client/src-tauri/src/main.rs +++ b/client/src-tauri/src/main.rs @@ -18,7 +18,6 @@ use keyboard::KeyboardHookManager; use context::ContextDetector; use tauri::{Manager, Emitter}; use tauri::tray::{TrayIconBuilder, MouseButton, MouseButtonState, TrayIconEvent}; -use tauri::menu::{MenuBuilder, MenuItemBuilder}; use std::thread; /// Environment-level WebView2 flags must be identical for every webview that shares @@ -47,8 +46,11 @@ fn browser_args_fingerprint(value: &str) -> String { /// Clean up expired audio files based on retention setting. fn cleanup_expired_audio(storage: &Storage) { - let retention_val = storage.get("audioRetentionDays", Some(&serde_json::json!(30))); - let retention_days = retention_val.as_i64().unwrap_or(30); + // 默认必须和前端 defaults.ts 的 audioRetentionDays 一致（-1 = 永久保留）。 + // 曾经这里写死 30：用户从没动过这个设置时，界面显示"永久"，实际 30 天就被删掉了 —— + // 不报错、没有提示，只是录音悄悄消失（"纠正识别"入口大面积置灰就是这么来的）。 + let retention_val = storage.get("audioRetentionDays", Some(&serde_json::json!(-1))); + let retention_days = retention_val.as_i64().unwrap_or(-1); if retention_days < 0 { return; // -1 = keep forever } @@ -253,6 +255,8 @@ fn main() { let ptt_str = ptt_setting_val.as_str().unwrap_or("ControlRight").to_string(); let hf_setting_val = storage.get("shortcutHandsFree", None); let hf_str = hf_setting_val.as_str().unwrap_or("AltRight").to_string(); + let ai_toggle_setting_val = storage.get("shortcutToggleAi", None); + let ai_toggle_str = ai_toggle_setting_val.as_str().unwrap_or("").to_string(); log::info!("PTT setting from DB: raw={:?} parsed={:?}", ptt_setting_val, ptt_str); log::info!("HF setting from DB: raw={:?} parsed={:?}", hf_setting_val, hf_str); @@ -349,7 +353,7 @@ fn main() { } let hook: tauri::State = app.state(); - hook.start(app.handle(), &ptt_str, &hf_str); + hook.start(app.handle(), &ptt_str, &hf_str, &ai_toggle_str); // 每 60s 记录一次键盘钩子健康快照（[ptt-watchdog] 日志行）， // 用于排查"过一段时间后 Alt 说话没反应"这类间歇性问题： @@ -440,51 +444,51 @@ fn main() { // 系统托盘图标 { - // 语言在建菜单之前就要定下来：托盘是开机即可见的 UI， - // 等前端起来再改文案会先闪一遍错误语言。 - let tray_lang = { - let storage: tauri::State = app.state(); - locale::effective_lang(storage.inner()) - }; - let tray_text = locale::tray_strings(tray_lang); - let show_item = MenuItemBuilder::with_id("show", tray_text.show).build(app)?; - let quit_item = MenuItemBuilder::with_id("quit", tray_text.quit).build(app)?; - let tray_menu = MenuBuilder::new(app) - .item(&show_item) - .separator() - .item(&quit_item) - .build()?; + // 自定义菜单窗口在启动时预创建并隐藏；右键时只定位、显示，没有首开延迟。 + commands::tray::create_tray_menu_window(app.handle())?; - let ico_bytes = include_bytes!("../icons/icon.ico"); - let icon = tauri::image::Image::from_bytes(ico_bytes) + // 每张托盘图都由 256px 原图直接生成，避免 256 → 32 → 系统实际尺寸的 + // 二次缩放。按窗口所在屏幕的缩放比例选最接近 Windows 托盘槽位的尺寸； + // Logo 的轮廓和完整 SayIt 字样都不变，只对小图做了轻微锐化。 + let tray_scale = app + .get_webview_window("main") + .and_then(|window| window.scale_factor().ok()) + .unwrap_or(1.0); + let (tray_icon_bytes, tray_icon_size): (&[u8], u32) = if tray_scale >= 1.75 { + (include_bytes!("../icons/tray-32.png"), 32) + } else if tray_scale >= 1.375 { + (include_bytes!("../icons/tray-24.png"), 24) + } else if tray_scale >= 1.125 { + (include_bytes!("../icons/tray-20.png"), 20) + } else { + (include_bytes!("../icons/tray-16.png"), 16) + }; + log::info!( + "[tray] icon size={}px for scale factor {:.2}", + tray_icon_size, + tray_scale + ); + let icon = tauri::image::Image::from_bytes(tray_icon_bytes) .expect("failed to load tray icon"); let _tray = TrayIconBuilder::new() .icon(icon) - .tooltip(tray_text.tooltip) - .menu(&tray_menu) - .on_menu_event(|app, event| { - match event.id().as_ref() { - "show" => { - if let Some(w) = app.get_webview_window("main") { - let _ = w.show(); - let _ = w.unminimize(); - let _ = w.set_focus(); - } - } - "quit" => { - app.exit(0); - } - _ => {} - } - }) + .tooltip("SayIt") .on_tray_icon_event(|tray, event| { - // 左键单击托盘图标 → 显示主窗口 - if let TrayIconEvent::Click { button: MouseButton::Left, button_state: MouseButtonState::Up, .. } = event { - if let Some(w) = tray.app_handle().get_webview_window("main") { - let _ = w.show(); - let _ = w.unminimize(); - let _ = w.set_focus(); + if let TrayIconEvent::Click { button, button_state: MouseButtonState::Up, position, .. } = event { + match button { + MouseButton::Left => { + commands::tray::hide_tray_menu(tray.app_handle().clone()); + if let Some(w) = tray.app_handle().get_webview_window("main") { + let _ = w.show(); + let _ = w.unminimize(); + let _ = w.set_focus(); + } + } + MouseButton::Right => { + commands::tray::show_tray_menu(tray.app_handle(), position); + } + _ => {} } } }) @@ -596,10 +600,18 @@ fn main() { commands::system::save_audio_to_downloads, commands::system::reveal_file_in_folder, commands::system::open_folder, + // Tray + commands::tray::set_tray_ai_enabled, + commands::tray::get_tray_ai_enabled, + commands::tray::toggle_tray_ai_enabled, + commands::tray::show_main_from_tray, + commands::tray::hide_tray_menu, + commands::tray::quit_from_tray, // Audio commands::audio::save_audio_file, commands::audio::save_pcm_as_wav, commands::audio::read_audio_file, + commands::audio::audio_file_exists, commands::audio::delete_audio_file, // Audio output mute (录音期间静音系统输出，防回采) commands::audio_mute::mute_system_output, diff --git a/client/src-tauri/src/providers/ai_ollama.rs b/client/src-tauri/src/providers/ai_ollama.rs index c47bdc6..751c08a 100644 --- a/client/src-tauri/src/providers/ai_ollama.rs +++ b/client/src-tauri/src/providers/ai_ollama.rs @@ -3,7 +3,7 @@ use super::diag; use super::prompt::wrap_user_text; -use super::types::{AiProviderConfig, AiResult, TestResult}; +use super::types::{AiProviderConfig, AiResult, TestResult, TextContext}; use std::time::Instant; const SCOPE: &str = "ai/ollama"; @@ -13,6 +13,7 @@ pub async fn polish( text: &str, config: &AiProviderConfig, system_prompt: Option<&str>, + text_context: Option<&TextContext>, ) -> Result { if text.trim().is_empty() { return Ok(AiResult { @@ -25,7 +26,7 @@ pub async fn polish( let sys_prompt = system_prompt.unwrap_or("你是语音转文本的校对助手。"); // Ollama /api/generate 是单一 prompt 字符串，没有 system/user 角色区分， // 这里手动拼接：system prompt 在前，user 消息（中性标签包裹）在后。 - let combined = format!("{}\n\n{}", sys_prompt, wrap_user_text(text)); + let combined = format!("{}\n\n{}", sys_prompt, wrap_user_text(text, text_context)); let model = if config.model.is_empty() { "qwen2.5:7b" @@ -75,10 +76,13 @@ pub async fn polish( )); } - let body_text = resp - .text() - .await - .map_err(|e| diag::fail(SCOPE, "read_body", format!("Failed to read response: {}", e)))?; + let body_text = resp.text().await.map_err(|e| { + diag::fail( + SCOPE, + "read_body", + format!("Failed to read response: {}", e), + ) + })?; let data: serde_json::Value = serde_json::from_str(&body_text).map_err(|e| { diag::fail( SCOPE, @@ -114,7 +118,11 @@ pub async fn polish( } Ok(AiResult { - text: if result_text.is_empty() { text.to_string() } else { result_text }, + text: if result_text.is_empty() { + text.to_string() + } else { + result_text + }, elapsed_ms, }) } @@ -161,7 +169,9 @@ pub async fn test_connection(config: &AiProviderConfig) -> TestResult { .to_string(); let detail = format!( "Elapsed: {}ms\nModel: {}\nSent: \"{}\"\nReply: {}", - elapsed_ms, model, prompt, + elapsed_ms, + model, + prompt, if reply.is_empty() { "(empty)" } else { &reply } ); TestResult { @@ -191,7 +201,11 @@ pub async fn test_connection(config: &AiProviderConfig) -> TestResult { } Err(e) => TestResult { ok: false, - message: diag::fail("ai/ollama-test", "http_send", format!("Connection failed: {}", e)), + message: diag::fail( + "ai/ollama-test", + "http_send", + format!("Connection failed: {}", e), + ), elapsed_ms, detail: format!("Model: {}\nRequest URL: {}", model, url), }, diff --git a/client/src-tauri/src/providers/ai_openai_compat.rs b/client/src-tauri/src/providers/ai_openai_compat.rs index b96ab52..25411ee 100644 --- a/client/src-tauri/src/providers/ai_openai_compat.rs +++ b/client/src-tauri/src/providers/ai_openai_compat.rs @@ -3,7 +3,7 @@ use super::diag; use super::prompt::wrap_user_text; -use super::types::{AiProviderConfig, AiResult, TestResult}; +use super::types::{AiProviderConfig, AiResult, TestResult, TextContext}; use std::time::Instant; const SCOPE: &str = "ai/openai-compat"; @@ -55,6 +55,7 @@ pub async fn polish( text: &str, config: &AiProviderConfig, system_prompt: Option<&str>, + text_context: Option<&TextContext>, ) -> Result { if text.trim().is_empty() { return Ok(AiResult { @@ -67,7 +68,7 @@ pub async fn polish( let url = format!("{}/chat/completions", base_url); let sys_prompt = system_prompt.unwrap_or("你是语音转文本的校对助手。"); - let user_content = wrap_user_text(text); + let user_content = wrap_user_text(text, text_context); let mut body = serde_json::json!({ "model": config.model, @@ -134,10 +135,13 @@ pub async fn polish( )); } - let body_text = resp - .text() - .await - .map_err(|e| diag::fail(SCOPE, "read_body", format!("Failed to read response: {}", e)))?; + let body_text = resp.text().await.map_err(|e| { + diag::fail( + SCOPE, + "read_body", + format!("Failed to read response: {}", e), + ) + })?; let data: serde_json::Value = serde_json::from_str(&body_text).map_err(|e| { diag::fail( SCOPE, @@ -187,7 +191,11 @@ pub async fn polish( } Ok(AiResult { - text: if cleaned.is_empty() { text.to_string() } else { cleaned }, + text: if cleaned.is_empty() { + text.to_string() + } else { + cleaned + }, elapsed_ms, }) } @@ -250,7 +258,10 @@ pub async fn test_connection(config: &AiProviderConfig) -> TestResult { let reply = strip_thinking(&raw_reply); let detail = format!( "Elapsed: {}ms\nModel: {}\nSent: system=\"{}\" user=\"{}\"\nReply: {}", - elapsed_ms, config.model, system_prompt, user_prompt, + elapsed_ms, + config.model, + system_prompt, + user_prompt, if reply.is_empty() { "(empty)" } else { &reply } ); TestResult { @@ -303,15 +314,24 @@ fn describe_reqwest_error(e: &reqwest::Error) -> String { // 尝试区分 DNS / TLS / 连接拒绝 let lower = raw.to_lowercase(); if lower.contains("dns") || lower.contains("resolve") || lower.contains("getaddrinfo") { - return format!("DNS lookup failed; the host may not exist or the network may be unavailable: {}", raw); + return format!( + "DNS lookup failed; the host may not exist or the network may be unavailable: {}", + raw + ); } - if lower.contains("ssl") || lower.contains("tls") || lower.contains("certificate") - || lower.contains("handshake") || lower.contains("schannel") + if lower.contains("ssl") + || lower.contains("tls") + || lower.contains("certificate") + || lower.contains("handshake") + || lower.contains("schannel") { return format!("TLS/SSL handshake failed; check the certificate: {}", raw); } if lower.contains("refused") { - return format!("Connection refused; the service may not be running: {}", raw); + return format!( + "Connection refused; the service may not be running: {}", + raw + ); } return format!("Could not connect to the server: {}", raw); } @@ -325,9 +345,7 @@ fn normalize_base_url(url: &str) -> String { .rsplit('/') .next() .and_then(|segment| segment.strip_prefix('v')) - .is_some_and(|version| { - !version.is_empty() && version.chars().all(|c| c.is_ascii_digit()) - }); + .is_some_and(|version| !version.is_empty() && version.chars().all(|c| c.is_ascii_digit())); // 已经以 /v1、/v3、/v4 等版本路径结尾时直接使用 if has_version_suffix { diff --git a/client/src-tauri/src/providers/prompt.rs b/client/src-tauri/src/providers/prompt.rs index 6aadbcf..6e26ec1 100644 --- a/client/src-tauri/src/providers/prompt.rs +++ b/client/src-tauri/src/providers/prompt.rs @@ -9,6 +9,82 @@ // 现在改为与服务器模式完全一致的中性标签包裹，不含任何动词/祈使句，只标出 // 文本边界，prompt 的全部措辞（风格、约束）仍 100% 由 system_prompt（用户的 // 预设）决定，这里不再"额外说话"。三种模式（服务器 / 云 API / 本地）保持统一。 -pub fn wrap_user_text(text: &str) -> String { - format!("\n{}\n", text) +use super::types::TextContext; + +const BEFORE_LIMIT: usize = 500; +const SELECTED_LIMIT: usize = 6000; +const AFTER_LIMIT: usize = 300; + +pub fn wrap_user_text(text: &str, context: Option<&TextContext>) -> String { + let asr = escape_xml(text); + let Some(context) = context.filter(|context| !context.selection_truncated) else { + return format!("\n{}\n", asr); + }; + + let before = clip_tail(&context.text_before, BEFORE_LIMIT); + let selected = clip_head(&context.selected_text, SELECTED_LIMIT); + let after = clip_head(&context.text_after, AFTER_LIMIT); + if before.is_empty() && selected.is_empty() && after.is_empty() { + return format!("\n{}\n", asr); + } + + format!( + "\n{}\n{}\n{}\n\n\n{}\n", + escape_xml(&clip_head(&context.source, 64)), + escape_xml(&before), + escape_xml(&selected), + escape_xml(&after), + asr, + ) +} + +fn escape_xml(value: &str) -> String { + value + .replace('&', "&") + .replace('<', "<") + .replace('>', ">") + .replace('"', """) + .replace('\'', "'") +} + +fn clip_head(value: &str, limit: usize) -> String { + value.chars().take(limit).collect() +} + +fn clip_tail(value: &str, limit: usize) -> String { + let count = value.chars().count(); + value.chars().skip(count.saturating_sub(limit)).collect() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn wraps_bounded_context_and_escapes_fake_tags() { + let context = TextContext { + source: "text_pattern2".into(), + text_before: "before ".into(), + selected_text: "原文".into(), + text_after: "after".into(), + selection_truncated: false, + }; + let wrapped = wrap_user_text("翻译成英文", Some(&context)); + assert!(wrapped.contains("原文")); + assert!(wrapped.contains("before </text_before>")); + assert!(wrapped.contains("\n翻译成英文\n")); + } + + #[test] + fn ignores_a_truncated_selection() { + let context = TextContext { + selected_text: "partial".into(), + selection_truncated: true, + ..Default::default() + }; + assert_eq!( + wrap_user_text("replacement", Some(&context)), + "\nreplacement\n" + ); + } } diff --git a/client/src-tauri/src/providers/registry.rs b/client/src-tauri/src/providers/registry.rs index 428d6fb..de13a1e 100644 --- a/client/src-tauri/src/providers/registry.rs +++ b/client/src-tauri/src/providers/registry.rs @@ -1,7 +1,10 @@ // 供应商注册表 — Tauri commands 入口 use super::types::*; -use super::{ai_openai_compat, ai_ollama, asr_doubao, asr_doubao_stream, asr_qwen, asr_qwen_omni, asr_mimo, asr_groq}; +use super::{ + ai_ollama, ai_openai_compat, asr_doubao, asr_doubao_stream, asr_groq, asr_mimo, asr_qwen, + asr_qwen_omni, +}; use crate::error_protocol; /// 云端 AI 校对（Tauri command） @@ -16,6 +19,7 @@ pub async fn cloud_polish(request: CloudPolishRequest) -> Result Result Err(error_protocol::encode("connect_failed", format!("Unknown AI provider: {}", other))), + other => Err(error_protocol::encode( + "connect_failed", + format!("Unknown AI provider: {}", other), + )), } } @@ -38,10 +46,11 @@ pub async fn test_ai_connection(config: AiProviderConfig) -> Result { Ok(ai_openai_compat::test_connection(&config).await) } - "ollama" => { - Ok(ai_ollama::test_connection(&config).await) - } - other => Err(error_protocol::encode("connect_failed", format!("Unknown AI provider: {}", other))), + "ollama" => Ok(ai_ollama::test_connection(&config).await), + other => Err(error_protocol::encode( + "connect_failed", + format!("Unknown AI provider: {}", other), + )), } } @@ -104,7 +113,10 @@ pub async fn cloud_transcribe(request: CloudTranscribeRequest) -> Result Err(error_protocol::encode("connect_failed", format!("ASR provider \"{}\" is not implemented", other))), + other => Err(error_protocol::encode( + "connect_failed", + format!("ASR provider \"{}\" is not implemented", other), + )), } } @@ -118,6 +130,9 @@ pub async fn test_asr_connection(config: AsrProviderConfig) -> Result Ok(asr_qwen_omni::test_connection(&config).await), "mimo" => Ok(asr_mimo::test_connection(&config).await), "groq_whisper" => Ok(asr_groq::test_connection(&config).await), - other => Err(error_protocol::encode("connect_failed", format!("ASR provider \"{}\" is not implemented", other))), + other => Err(error_protocol::encode( + "connect_failed", + format!("ASR provider \"{}\" is not implemented", other), + )), } } diff --git a/client/src-tauri/src/providers/types.rs b/client/src-tauri/src/providers/types.rs index 05be7d0..d47ac65 100644 --- a/client/src-tauri/src/providers/types.rs +++ b/client/src-tauri/src/providers/types.rs @@ -76,4 +76,21 @@ pub struct CloudPolishRequest { pub ai_config: AiProviderConfig, #[serde(default)] pub system_prompt: Option, + #[serde(default)] + pub text_context: Option, +} + +/// Ephemeral editor context. It is accepted only for this one AI request and is never persisted. +#[derive(Debug, Clone, Serialize, Deserialize, Default)] +pub struct TextContext { + #[serde(default)] + pub source: String, + #[serde(default, alias = "textBefore")] + pub text_before: String, + #[serde(default, alias = "selectedText")] + pub selected_text: String, + #[serde(default, alias = "textAfter")] + pub text_after: String, + #[serde(default, alias = "selectionTruncated")] + pub selection_truncated: bool, } diff --git a/client/src-tauri/src/storage/mod.rs b/client/src-tauri/src/storage/mod.rs index 333db52..eec7e03 100644 --- a/client/src-tauri/src/storage/mod.rs +++ b/client/src-tauri/src/storage/mod.rs @@ -12,6 +12,7 @@ const DEFAULT_SETTINGS: &[(&str, &str)] = &[ ("shortcutPTT", r#""ControlRight""#), ("shortcutPTTCombo", r#""Alt+Q""#), ("shortcutHandsFree", r#""AltRight""#), + ("shortcutToggleAi", r#""""#), ("autoLaunch", "true"), ("selectedMic", r#""""#), ("hotwords", "[]"), diff --git a/client/src-tauri/src/window/mod.rs b/client/src-tauri/src/window/mod.rs index 052309a..64bf1a8 100644 --- a/client/src-tauri/src/window/mod.rs +++ b/client/src-tauri/src/window/mod.rs @@ -16,6 +16,10 @@ const OVERLAY_FALLBACK_HEIGHT: f64 = 224.0; // 流式实时显示：录音气泡 + 波形条堆叠，需要更宽更高的窗口 const OVERLAY_STREAMING_WIDTH: f64 = 480.0; const OVERLAY_STREAMING_HEIGHT: f64 = 200.0; +// 输入源状态条显示在胶囊上方；窗口仍锚定底部，因此只向上扩展。 +const OVERLAY_MIC_HINT_WIDTH: f64 = 480.0; +const OVERLAY_MIC_HINT_HEIGHT: f64 = 104.0; +const OVERLAY_STREAMING_MIC_HINT_HEIGHT: f64 = 244.0; const OVERLAY_SCREEN_MARGIN: f64 = 8.0; const ACK_FIRST_TIMEOUT_MS: u64 = 1_200; const ACK_SECOND_TIMEOUT_MS: u64 = 700; @@ -36,9 +40,11 @@ fn now_ms() -> i64 { #[derive(Debug, Clone, PartialEq)] enum OverlayLayout { Base, + BaseWithMicHint, Fallback, /// 流式实时显示：气泡 + 波形，窗口更大且非交互 Streaming, + StreamingWithMicHint, } impl OverlayLayout { @@ -51,8 +57,16 @@ impl OverlayLayout { fn dimensions(&self, base_width: f64) -> (f64, f64) { match self { OverlayLayout::Base => (base_width, OVERLAY_BASE_HEIGHT), + OverlayLayout::BaseWithMicHint => ( + base_width.max(OVERLAY_MIC_HINT_WIDTH), + OVERLAY_MIC_HINT_HEIGHT, + ), OverlayLayout::Fallback => (OVERLAY_FALLBACK_WIDTH, OVERLAY_FALLBACK_HEIGHT), OverlayLayout::Streaming => (OVERLAY_STREAMING_WIDTH, OVERLAY_STREAMING_HEIGHT), + OverlayLayout::StreamingWithMicHint => ( + OVERLAY_STREAMING_WIDTH, + OVERLAY_STREAMING_MIC_HINT_HEIGHT, + ), } } } @@ -396,11 +410,20 @@ impl WindowState { .and_then(Value::as_str) .map(|s| !s.trim().is_empty()) .unwrap_or(false); + let mic_hint_on = data + .get("micSourceLabel") + .and_then(Value::as_str) + .map(|s| !s.trim().is_empty()) + .unwrap_or(false); let next_layout = if state == Some("fallback") { OverlayLayout::Fallback + } else if state == Some("listening") && streaming_on && mic_hint_on { + OverlayLayout::StreamingWithMicHint } else if state == Some("listening") && streaming_on { // 录音中开启了实时显示 → 放大窗口容纳气泡（整段录音保持该尺寸，中途不再缩放） OverlayLayout::Streaming + } else if matches!(state, Some("listening") | Some("thinking")) && mic_hint_on { + OverlayLayout::BaseWithMicHint } else { OverlayLayout::Base }; diff --git a/client/src-tauri/tauri.conf.json b/client/src-tauri/tauri.conf.json index d06ac2a..bc024e3 100644 --- a/client/src-tauri/tauri.conf.json +++ b/client/src-tauri/tauri.conf.json @@ -1,6 +1,6 @@ { "productName": "SayIt", - "version": "0.1.7", + "version": "0.1.8", "identifier": "com.sayit.app", "build": { "frontendDist": "../dist", diff --git a/client/src/components/history/AsrCorrectionDialog.tsx b/client/src/components/history/AsrCorrectionDialog.tsx new file mode 100644 index 0000000..292bcc0 --- /dev/null +++ b/client/src/components/history/AsrCorrectionDialog.tsx @@ -0,0 +1,273 @@ +import { useCallback, useEffect, useRef, useState } from 'react' +import { AlertTriangle, Check, Info, Loader2, Pause, Play, RotateCcw } from 'lucide-react' +import { Modal } from '@/components/ui/modal' +import { Button } from '@/components/ui/button' +import * as bridge from '@/services/bridge' +import { type HistoryRecord } from '@/services/store' +import { + ASR_CORRECTION_WITHDRAW_DAYS, + grantAsrCorrectionConsent, + hasAsrCorrectionConsent, + submitAsrCorrection, + withdrawAsrCorrection, +} from '@/services/asrCorrection' +import { normalizeForDiff } from '@/lib/asrDiff' +import { t } from '@/i18n' +import { useT } from '@/i18n/useT' +import { useRecordingPlayback } from './useRecordingPlayback' +import { AudioProgressBar } from './AudioProgressBar' +import { AsrDiffPreview } from './AsrDiffPreview' + +/** + * 「纠正识别」面板。 + * + * 三个刻意的设计，改之前先看 docs/decisions.md： + * 1. 可编辑的那一栏初值是 **record.asrText（识别原文）**，绝不是 llmText。 + * 用户平时看到的是 AI 整理后的文本，如果照着它改（补标点、分段、改书面语）， + * 拿到的标注对 ASR 训练是噪声，因此纠正面板不展示 AI 整理后的文本。 + * 2. 音频读不到就不让提交。没有音频的样本对识别改进没有用。 + * 3. 首次提交必须过一次明确同意（不是一个信息图标）。上传的是用户的录音， + * 比文本敏感得多。 + */ +/** + * 可编辑框的高度上限（px）。上面那块「识别原文」是 max-h-40（160px），这里给到 200px： + * 两块的行高、字号、内边距完全一致，所以同样的文字在两块里占一样的高度， + * 只是"要动手改"的那块允许多长一点。超过之后内部滚动，而不是把弹窗撑到屏幕外。 + */ +const EDITOR_MAX_HEIGHT_PX = 200 + +export function AsrCorrectionDialog({ + record, + onClose, + onSubmitted, +}: { + record: HistoryRecord + onClose: () => void + onSubmitted: (patch: Partial) => void +}) { + useT() + const originalText = record.asrText || '' + const [correctedText, setCorrectedText] = useState(record.asrCorrectedText || originalText) + const [audioState, setAudioState] = useState<'checking' | 'ok' | 'missing'>('checking') + const [consentNeeded, setConsentNeeded] = useState(false) + const [consentChecked, setConsentChecked] = useState(false) + const [submitting, setSubmitting] = useState(false) + const [withdrawing, setWithdrawing] = useState(false) + const [error, setError] = useState('') + const editorRef = useRef(null) + // 用记录里存的音频时长做初值，这样没点播放也能看到总长（audioDurationSec 是音频真实 + // 长度，durationSec 是按住快捷键的时长，只在老记录缺前者时兜底） + const playback = useRecordingPlayback( + record.audioFilePath, + record.audioDurationSec || record.durationSec || 0, + ) + + const submitted = Boolean(record.asrCorrectionId) + + useEffect(() => { + let alive = true + void (async () => { + const path = record.audioFilePath + if (!path) { + if (alive) setAudioState('missing') + return + } + const exists = await bridge.audioFileExists(path).catch(() => false) + if (alive) setAudioState(exists ? 'ok' : 'missing') + })() + void (async () => { + const granted = await hasAsrCorrectionConsent().catch(() => false) + if (alive) setConsentNeeded(!granted) + })() + return () => { alive = false } + }, [record.audioFilePath]) + + /** + * 让可编辑框跟着内容长高，行为对齐上面那块只读的「识别原文」：一行内容就是一行高， + * 到上限后内部滚动。 + * + * 不用 `rows={行数}` 那种算法：`rows` 只数换行符，一段没有换行但会折行的长文本 + * 仍然只算一行，框会比内容矮一大截。所以按 scrollHeight 实测。 + * + * `+ extra` 是上下边框：box-sizing 是 border-box，height 含边框而 scrollHeight 不含， + * 少了它每次都会矮 2px，于是内容一进来就立刻出现滚动条。 + */ + useEffect(() => { + const el = editorRef.current + if (!el) return + el.style.height = 'auto' + const extra = el.offsetHeight - el.clientHeight + el.style.height = `${Math.min(el.scrollHeight + extra, EDITOR_MAX_HEIGHT_PX)}px` + }, [correctedText]) + + const changed = normalizeForDiff(originalText) !== normalizeForDiff(correctedText) + const canSubmit = + !submitted && + audioState === 'ok' && + changed && + !submitting && + (!consentNeeded || consentChecked) + + const handleSubmit = useCallback(async () => { + setError('') + setSubmitting(true) + try { + if (consentNeeded) await grantAsrCorrectionConsent() + const result = await submitAsrCorrection(record, correctedText) + if (!result.ok) { + setError(result.message) + return + } + setConsentNeeded(false) + onSubmitted({ + asrCorrectionId: result.correctionId, + asrCorrectionSubmittedAt: Date.now(), + asrCorrectedText: normalizeForDiff(correctedText), + }) + } finally { + setSubmitting(false) + } + }, [consentNeeded, correctedText, onSubmitted, record]) + + const handleWithdraw = useCallback(async () => { + if (!record.asrCorrectionId) return + setError('') + setWithdrawing(true) + try { + const ok = await withdrawAsrCorrection(record.asrCorrectionId) + if (!ok) { + setError(t('asrCorrection.errorNetwork')) + return + } + // 本地也清掉，入口回到"可提交"。 + // 必须写 null 而不是 undefined：patch 要过 JSON.stringify，undefined 的键会被丢掉， + // 结果是字段没被清、界面一直显示"已提交"（见 HistoryRecord 上的注释）。 + onSubmitted({ + asrCorrectionId: null, + asrCorrectionSubmittedAt: null, + asrCorrectedText: null, + }) + } finally { + setWithdrawing(false) + } + }, [onSubmitted, record.asrCorrectionId]) + + return ( + +
+ {/* 录音 */} + {audioState === 'missing' ? ( +
+ + {t('asrCorrection.audioMissing')} +
+ ) : ( +
+ + +
+ )} + + {/* 识别原文 + 改动标记合成一块。 + 原本是"只读原文"和"改动预览"两个框，但 diff 本身就包含完整原文（删掉的部分 + 带删除线），两个框摆在一起是同一段话看两遍 —— 连上可编辑框一共三个框， + 光是找"我该在哪儿打字"就要费一下。合成一块之后只剩「看」和「改」两块。 */} + + + {/* 正确文本（可编辑） */} +
+
+ + {/* 改乱了想重来时不用手抄一遍原文 */} + +
+ {/* 内边距/字号/行高与「识别原文」那块保持一致，两块才对得齐； + 高度交给上面那个 effect 算，所以关掉手动拖拽（拖完一打字就会被算回去）。 */} +