diff --git a/docs/remote-input-certificates.md b/docs/remote-input-certificates.md index 229019bf..0631c3db 100644 --- a/docs/remote-input-certificates.md +++ b/docs/remote-input-certificates.md @@ -5,42 +5,65 @@ private certificate authority (CA) on each computer and a separate server certificate covering that computer's LAN addresses. The phone installs the public CA certificate. The CA private key stays in the computer's user data. -## iPhone and iPad +## 首次信任前的一次性核验 -1. Enable remote input on the computer. Use the address shown in settings, on - the same network as the computer. Settings also has a **Copy iPhone - certificate link** button for each address. -2. Open the address in Safari. On the initial certificate warning, check the - address against the computer, then use **Show Details → Visit This Website** - to reach your own computer's setup page. This exception is only a bootstrap - step, not the persistent trust setup. -3. Expand **First-time setup: trust this computer** and choose **iPhone: - download profile**. Alternatively, open the copied `/cert.mobileconfig` link - directly in Safari. -4. Install the downloaded profile in **Settings → General → VPN & Device - Management**. -5. In **Settings → General → About → Certificate Trust Settings**, enable full - trust for **OpenLess Remote Input CA**. Return to Safari and reload the page. -6. Enter the pairing code and allow microphone access when Safari asks. +根据 [#1037 的审核建议](https://github.com/Open-Less/openless/pull/1037#discussion_r3964815714), +在信任根证书前,通过电脑本地设置与手机系统证书详情核对身份。 -Installing a profile and enabling full SSL trust are separate steps. Apple -requires the latter for profiles downloaded from a website; a desktop app -cannot silently approve it on a personal iPhone. See -[Apple's certificate trust instructions](https://support.apple.com/en-gb/102390). -This setup removes certificate warnings after trust is established; browser -microphone permissions and the pairing code remain separate controls. +电脑设置中的 **本机根证书 SHA-256** 来自正在运行的监听器所用的公开根证书, +经本地接口传给界面;Linux 原生界面也显示同一来源的完整指纹。 +指纹共 64 个十六进制字符,可忽略空格、冒号和大小写,但不能只核对开头几位。 +对比对象必须是将要信任的根证书,不能使用会随 IP 变化而重新签发的服务器证书。 -Only install a CA from your own computer. A CA can issue certificates, so its -private key is sensitive. Remove the OpenLess profile from the phone when you -no longer use it. The certificate fingerprint in each profile identifier keeps -profiles for different computers from replacing one another. +手机端必须从**系统证书详情**取得实际证书的 SHA-256。 +网页、描述文件名称、标识和描述文字都可以被替换,不能作为校验依据。 +描述文件名称末尾的短指纹仅用于区分电脑,不代表已经验证身份。 +本功能提供人工核验依据,不会自动确认手机是否正确核对,也不会代替系统开启信任。 -## Android +### iPhone 和 iPad -Download `/cert.cer` using the **Android: download CA** link. Install it through -the system's **Install a certificate → CA certificate** settings, then return -to the browser. Menu names and browser support for user-installed CAs vary by -device. The download contains only the public root certificate. +1. 在电脑启用远程输入,保留本地设置中的完整 SHA-256。指纹不可用时停止安装。 +2. 在可信网络中,用 Safari 打开电脑显示的地址或复制的证书链接。 + 首次 TLS 警告说明身份尚未验证;即使地址与电脑相同,也不能据此认定证书可信。 + 只有准备执行下面的独立核验时,才继续下载描述文件;否则使用已有的可信文件传输渠道。 +3. 在“设置 → 通用 → VPN 与设备管理”打开下载的描述文件。 + 先检查它**只包含一张根证书**;若有额外证书、VPN 或设备管理配置,不要安装。 +4. 在“更多详细信息”中打开根证书,查找系统显示的 SHA-256, + 与电脑本地设置中的全部 64 个字符逐一核对。 + 若当前 iOS 只能在安装后显示完整详情,安装前仍需确认只有一张根证书, + 且安装后先保持“完全信任”关闭,核对完毕再开启。 +5. 若指纹不一致、看不全或找不到 SHA-256,停止操作,删除下载的描述文件; + 已安装的则移除。不要用名称、网页上的值或配对码代替核验。 +6. 只有全部一致,才到“通用 → 关于本机 → 证书信任设置”为这张根证书开启完全信任。 + 返回 Safari 刷新,输入配对码并允许麦克风访问。 + +安装描述文件和开启完全信任是两个步骤,见[苹果说明](https://support.apple.com/en-us/102390)。 +不同 iOS 版本的菜单和完整指纹入口需要真机确认;无法独立查看指纹的设备不能声称通过了本流程。 +根证书可签发其他证书,私钥应始终保留在自己的电脑上。不再使用远程输入时请移除手机上的根证书。 + +### Android + +通过“安卓:下载 CA 证书”取得 `/cert.cer`,在系统证书预览中核对完整 SHA-256, +一致后再安装。部分设备安装 CA 即代表信任,因此必须在安装前完成核验。 +若系统无法在信任前显示完整指纹,请停止从网页安装,改用已有的可信文件传输渠道。 +菜单名称和用户 CA 支持情况因设备而异。 + +### 真机验收与截图 + +自动化测试可验证指纹来源、替换证书时的差异及生命周期,不能代替以下真机操作: + +| 检查 | 需要记录的结果 | +| --- | --- | +| 首次安装 | 电脑完整指纹、手机系统完整指纹一致;描述文件只有一张根证书 | +| 信任与录音 | 核对后开启完全信任,录音能正常传到电脑 | +| 重启 | 电脑重启后指纹不变,手机无需重新安装证书,仍可录音 | +| IP 变化 | 使用新地址重新连接后指纹不变,仍可录音 | +| 替换证书 | 在独立测试环境用另一台电脑生成的同名证书或替换描述文件,系统指纹不同,用户能按引导停止安装/信任 | + +记录设备型号、系统版本、测试提交号,以及每项通过或失败。 +截图至少包含电脑指纹、手机系统指纹和描述文件内容;录音、重启和 IP 变化可以用简短录屏或文字结果补充。 +替换测试只核对差异,不要开启对测试证书的完全信任;结束后移除测试描述文件。 +Linux 原生界面的显示与录音也需要在 Linux 上实际验收。 ## What OpenLess automates diff --git a/docs/remote-input-recording-recovery.md b/docs/remote-input-recording-recovery.md new file mode 100644 index 00000000..39163b09 --- /dev/null +++ b/docs/remote-input-recording-recovery.md @@ -0,0 +1,41 @@ +# 远程输入:息屏与录音恢复 + +手机录音页新增「录音时保持亮屏」,默认开启,并在当前浏览器记住选择。只在前台录音期间申请屏幕唤醒锁,结束、取消、断线或离开页面时释放。浏览器不支持,或系统因省电等原因拒绝时,页面会提示,录音仍可使用。 + +```text + [ 点击开始 / 点击结束 ] + 录音状态 + +录音时保持亮屏 [ 开 ] +息屏会结束本段录音,电脑继续处理已收到的部分。 + +电脑落字 [ 开 ] +``` + +## 息屏后会发生什么 + +- 自动息屏、手动锁屏、切到后台或麦克风被系统中断时,手机尽可能发送结束录音,电脑继续识别已经收到的音频。 +- 如果手机来不及通知,电脑在断线时收尾;连接仍在但连续 15 秒没有有效音频时,也会结束采集并开始识别。 +- 识别已经开始后,手机断线不会取消电脑上的识别与历史记录写入。主动取消、关闭远程输入、重置 PIN 仍会撤销会话。 +- 回到页面后重新认证并查询上次会话,找回结果或提示到电脑历史中重试。不会自动重新打开麦克风。 + +这不能让浏览器在锁屏后持续采集,也不能补回尚未传到电脑的音频。电脑退出、崩溃或磁盘不可写时,不能保证自动完成识别。 + +## 录音与结果保留 + +已收到的远程音频使用现有录音目录和保留设置写入 WAV。识别失败时,音频可供现有自动重试及桌面历史重新转录使用;识别成功后,除非开启保留成功录音,否则按现有策略删除音频。文字仍按历史记录设置保留。 + +手机只保存上次会话 ID 和单独生成的随机恢复凭据。恢复查询要求有效配对,不提供历史列表;仅凭公开会话 ID 无法读取结果。服务内最多保留最近 64 个会话的恢复权限,最长 24 小时,取消会话、重启服务或重置 PIN 后失效。此后请在电脑历史记录中查看。 + +## 真机验收(待补) + +记录手机型号、系统与浏览器版本、电脑平台、识别模式及测试 commit;分别检查 iPhone Safari 和 Android Chrome。 + +- 开关默认开启;关闭、刷新后仍关闭;录音中切换立即生效,结束后恢复系统自动息屏。 +- 点按和按住模式各录音 1–2 分钟:允许自动息屏、手动锁屏、切后台后,电脑保留已收到的部分并完成识别;回到手机能看到结果。 +- 识别过程中断开网络再重连;检查电脑历史以及手机找回的文字。 +- 模拟识别失败,检查 WAV 保留与历史重新转录;检查保留天数和录音数量上限。 +- 低电量或浏览器拒绝保持亮屏时,提示明确且能开始录音。 +- 识别中主动取消或重置 PIN,确认没有迟到落字,旧恢复凭据失效。 + +自动化测试覆盖模拟浏览器事件、两分钟 PCM、归档与失败重试、恢复鉴权和权限撤销;这些检查不能代替手机锁屏与省电行为的真机验收。 diff --git a/openless-all/app/crates/openless-core/src/api.rs b/openless-all/app/crates/openless-core/src/api.rs index 169cc8d7..e7433ef6 100644 --- a/openless-all/app/crates/openless-core/src/api.rs +++ b/openless-all/app/crates/openless-core/src/api.rs @@ -9446,6 +9446,58 @@ mod tests { let _ = std::fs::remove_dir_all(data_dir); } + #[tokio::test] + async fn external_audio_saves_failed_recordings_for_history_retry_and_prunes_successful_audio() + { + for fail in [false, true] { + let data_dir = std::env::temp_dir() + .join(format!("openless-remote-history-{}", uuid::Uuid::new_v4())); + let transcription = if fail { + crate::testing::FixtureTranscriptionEngine::failing(BackendError::new( + BackendErrorCode::Provider, + "fixture ASR failure", + )) + } else { + crate::testing::FixtureTranscriptionEngine::successful("received speech", 120_000) + }; + let engine = crate::PipelineDictationEngine::new( + Arc::new(crate::ExternalAudioRecorder::with_recordings_directory( + data_dir.join("recordings"), + )), + Arc::new(transcription.clone()), + Arc::new(crate::testing::FixtureTextPolisher::successful( + "complete transcription", + )), + ); + let backend = backend_with_dictation_engine(data_dir.clone(), Arc::new(engine)); + backend.start().await.unwrap(); + let session = backend.start_external_dictation().await.unwrap(); + let path = data_dir.join("recordings").join(format!("{session}.wav")); + for second in 0..120 { + backend + .feed_external_pcm(session, &vec![second; 32_000]) + .unwrap(); + } + let expected = transcription.pcm(); + assert_eq!(&std::fs::read(&path).unwrap()[44..], expected); + let result = backend.stop_dictation_session(session).await; + assert_eq!(result.is_err(), fail); + let history = backend.list_history().unwrap(); + assert_eq!(history.len(), 1); + assert_eq!(history[0].id, session.to_string()); + assert_eq!(history[0].has_audio_recording, Some(fail)); + if fail { + assert_eq!(history[0].error_code.as_deref(), Some("transcribeFailed")); + assert_eq!(&std::fs::read(path).unwrap()[44..], expected); + } else { + assert_eq!(history[0].final_text, "complete transcription"); + assert!(!path.exists()); + } + backend.shutdown().await.unwrap(); + let _ = std::fs::remove_dir_all(data_dir); + } + } + #[tokio::test] async fn dictation_freezes_channel_identity_protocol_and_model_for_the_session() { let data_dir = std::env::temp_dir().join(format!( diff --git a/openless-all/app/crates/openless-core/src/domains.rs b/openless-all/app/crates/openless-core/src/domains.rs index cf8600b7..4b73d88e 100644 --- a/openless-all/app/crates/openless-core/src/domains.rs +++ b/openless-all/app/crates/openless-core/src/domains.rs @@ -1000,6 +1000,9 @@ pub struct RemoteInputStatus { pub port: u16, pub urls: Vec, pub urls_stale: bool, + /// 由宿主提供,取自正在运行的监听器所使用的公开根证书。 + #[serde(default, skip_serializing_if = "Option::is_none")] + pub ca_fingerprint_sha256: Option, pub locale: String, pub connection_count: usize, #[serde(default, skip_serializing_if = "Option::is_none")] @@ -1030,6 +1033,7 @@ pub struct RemoteInputServerBinding { pub port: u16, pub urls: Vec, pub urls_stale: bool, + pub ca_fingerprint_sha256: Option, } /// Native transport and shared-dictation bridge. TLS, sockets, H5 assets and @@ -1065,6 +1069,27 @@ pub trait RemoteInputRuntimeAdapter: Send + Sync { &self, session_id: SessionId, ) -> BoxFuture<'static, Result<(), BackendError>>; + /// 只读取指定会话;由 Core 校验手机持有的恢复凭据。 + fn read_audio_history( + &self, + _session_id: SessionId, + ) -> BoxFuture<'static, Result, BackendError>> { + Box::pin(async { Ok(None) }) + } +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "camelCase")] +pub enum RemoteInputRecovery { + Pending, + Completed { + text: String, + }, + Failed { + #[serde(rename = "hasAudioRecording")] + has_audio_recording: bool, + }, + Unavailable, } pub trait RemoteInputApi: Send + Sync { @@ -1100,6 +1125,24 @@ pub trait RemoteInputApi: Send + Sync { ) -> BoxFuture<'static, Result<(), BackendError>> { unsupported("remote input") } + fn recover_stream( + &self, + _connection_id: SessionId, + _session_id: SessionId, + _recovery_key: crate::credentials::SecretValue, + ) -> BoxFuture<'static, Result> { + unsupported("remote input") + } + fn recovery_key( + &self, + _connection_id: SessionId, + _session_id: SessionId, + ) -> Result { + Err(BackendError::new( + BackendErrorCode::Unsupported, + "remote recovery is unavailable", + )) + } fn start_stream( &self, _connection_id: SessionId, @@ -2032,6 +2075,7 @@ mod tests { port: 18989, urls: vec!["https://192.168.1.2:18989".into()], urls_stale: false, + ca_fingerprint_sha256: None, locale: "zh-CN".into(), connection_count: 1, active_session_id: Some(SessionId::new()), diff --git a/openless-all/app/crates/openless-core/src/external_audio.rs b/openless-all/app/crates/openless-core/src/external_audio.rs index 793791fb..24d0cbf8 100644 --- a/openless-all/app/crates/openless-core/src/external_audio.rs +++ b/openless-all/app/crates/openless-core/src/external_audio.rs @@ -5,22 +5,30 @@ //! little-endian PCM contract and routes it to the active pipeline session. use std::collections::HashMap; +use std::path::PathBuf; use std::sync::{Arc, Mutex}; use futures_util::future::BoxFuture; use crate::dictation_context::{DictationAudioSource, DictationContext}; use crate::errors::{BackendError, BackendErrorCode}; -use crate::ports::{ActiveRecording, AudioConsumer, AudioRecorder, RecordingProgressSink}; +use crate::ports::{ + ActiveRecording, AudioConsumer, AudioRecorder, RecordingArchive, RecordingProgressSink, +}; use crate::types::SessionId; +mod archive; +use archive::ExternalRecordingArchive; + #[derive(Clone, Default)] pub struct ExternalAudioRecorder { sessions: Arc>>>, + recordings_dir: Option, } struct ExternalRecordingSession { state: Mutex, + archive: Option>, } struct ExternalRecordingState { @@ -37,6 +45,14 @@ struct ExternalActiveRecording { } impl ExternalAudioRecorder { + /// 与电脑历史记录共用 WAV 目录及保留策略,失败的远程录音也可重新转录。 + pub fn with_recordings_directory(directory: PathBuf) -> Self { + Self { + recordings_dir: Some(directory), + ..Self::default() + } + } + fn release( &self, session_id: SessionId, @@ -60,6 +76,9 @@ impl ExternalAudioRecorder { .lock() .expect("external audio state lock poisoned") .active = false; + if let Some(archive) = &expected.archive { + archive.finish(); + } sessions.remove(&session_id); Ok(()) } @@ -81,6 +100,29 @@ impl AudioRecorder for ExternalAudioRecorder { )) }); } + // 在归档创建/清理之前检查重复会话,避免误删仍在录音的文件。 + let mut sessions = self + .sessions + .lock() + .expect("external audio session lock poisoned"); + if sessions.contains_key(&session_id) { + return Box::pin(async { + Err(BackendError::new( + BackendErrorCode::Busy, + "external audio session already exists", + )) + }); + } + let archive = self + .recordings_dir + .as_ref() + .filter(|_| context.recording.archive_enabled) + .and_then(|directory| { + ExternalRecordingArchive::create(directory, session_id, &context.recording) + .map(Arc::new) + .map_err(|error| log::warn!("[remote-input] 录音归档不可用:{error}")) + .ok() + }); let session = Arc::new(ExternalRecordingSession { state: Mutex::new(ExternalRecordingState { active: true, @@ -88,22 +130,10 @@ impl AudioRecorder for ExternalAudioRecorder { consumer, progress, }), + archive, }); - { - let mut sessions = self - .sessions - .lock() - .expect("external audio session lock poisoned"); - if sessions.contains_key(&session_id) { - return Box::pin(async { - Err(BackendError::new( - BackendErrorCode::Busy, - "external audio session already exists", - )) - }); - } - sessions.insert(session_id, Arc::clone(&session)); - } + sessions.insert(session_id, Arc::clone(&session)); + drop(sessions); let recording = ExternalActiveRecording { recorder: self.clone(), session_id, @@ -141,6 +171,9 @@ impl AudioRecorder for ExternalAudioRecorder { "external audio session is not active", )); } + if let Some(archive) = &session.archive { + archive.append(pcm); + } state.consumer.consume_pcm_chunk(pcm); state.bytes_received = state.bytes_received.saturating_add(pcm.len() as u64); let elapsed_ms = state.bytes_received.saturating_mul(1_000) / 32_000; @@ -151,6 +184,13 @@ impl AudioRecorder for ExternalAudioRecorder { } impl ActiveRecording for ExternalActiveRecording { + fn archive(&self) -> Option> { + self.session + .archive + .as_ref() + .map(|archive| Arc::clone(archive) as Arc) + } + fn stop(self: Box) -> BoxFuture<'static, Result<(), BackendError>> { Box::pin(async move { self.recorder.release(self.session_id, &self.session) }) } @@ -253,6 +293,125 @@ mod tests { } } + #[tokio::test] + async fn remote_archive_keeps_two_minutes_of_pcm_and_survives_recording_stop() { + let directory = + std::env::temp_dir().join(format!("openless-remote-archive-{}", uuid::Uuid::new_v4())); + let recorder = ExternalAudioRecorder::with_recordings_directory(directory.clone()); + let id = SessionId::new(); + let mut context = DictationContext::default(); + context.audio_source = DictationAudioSource::External; + context.recording.archive_enabled = true; + let consumer = Arc::new(RecordingConsumer::default()); + let recording = recorder + .start( + id, + Arc::new(context.clone()), + consumer.clone(), + Arc::new(RecordingProgress::default()), + ) + .await + .unwrap(); + let archive = recording.archive().unwrap(); + assert!(!archive.is_available()); + for second in 0..120 { + recorder.feed_pcm(id, &vec![second; 32_000]).unwrap(); + } + let expected = consumer.0.lock().unwrap().clone(); + let path = directory.join(format!("{id}.wav")); + let wav = std::fs::read(&path).unwrap(); + assert_eq!(&wav[44..], expected); + assert_eq!( + u32::from_le_bytes(wav[40..44].try_into().unwrap()) as usize, + 120 * 32_000 + ); + assert!(recorder + .start( + id, + Arc::new(context), + consumer, + Arc::new(RecordingProgress::default()) + ) + .await + .is_err()); + assert_eq!(std::fs::read(&path).unwrap(), wav); + recording.stop().await.unwrap(); + assert!(archive.is_available()); + assert_eq!(archive.read_pcm().await.unwrap(), expected); + archive.discard().await.unwrap(); + assert!(!archive.is_available()); + assert!(!path.exists()); + let _ = std::fs::remove_dir_all(directory); + } + + #[tokio::test] + async fn disabled_or_unavailable_archive_does_not_block_remote_capture() { + let directory = std::env::temp_dir().join(format!( + "openless-remote-no-archive-{}", + uuid::Uuid::new_v4() + )); + for enabled in [false, true] { + if enabled { + std::fs::write(&directory, b"not a directory").unwrap(); + } + let recorder = ExternalAudioRecorder::with_recordings_directory(directory.clone()); + let id = SessionId::new(); + let mut context = DictationContext::default(); + context.audio_source = DictationAudioSource::External; + context.recording.archive_enabled = enabled; + let consumer = Arc::new(RecordingConsumer::default()); + let recording = recorder + .start( + id, + Arc::new(context), + consumer.clone(), + Arc::new(RecordingProgress::default()), + ) + .await + .unwrap(); + assert!(recording.archive().is_none()); + recorder.feed_pcm(id, &[1, 0, 2, 0]).unwrap(); + assert_eq!(*consumer.0.lock().unwrap(), vec![1, 0, 2, 0]); + recording.stop().await.unwrap(); + if !enabled { + assert!(!directory.exists()); + } + } + let _ = std::fs::remove_file(directory); + } + + #[tokio::test] + async fn remote_archive_applies_the_existing_count_limit_without_deleting_other_files() { + let directory = std::env::temp_dir().join(format!( + "openless-remote-retention-{}", + uuid::Uuid::new_v4() + )); + std::fs::create_dir_all(&directory).unwrap(); + std::fs::write(directory.join("user.wav"), b"keep").unwrap(); + let recorder = ExternalAudioRecorder::with_recordings_directory(directory.clone()); + let mut context = DictationContext::default(); + context.audio_source = DictationAudioSource::External; + context.recording.archive_enabled = true; + context.recording.max_entries = Some(2); + for _ in 0..4 { + let id = SessionId::new(); + let recording = recorder + .start( + id, + Arc::new(context.clone()), + Arc::new(RecordingConsumer::default()), + Arc::new(RecordingProgress::default()), + ) + .await + .unwrap(); + recorder.feed_pcm(id, &[1, 0]).unwrap(); + recording.stop().await.unwrap(); + } + assert_eq!(std::fs::read_dir(&directory).unwrap().count(), 3); + assert_eq!(std::fs::read(directory.join("user.wav")).unwrap(), b"keep"); + let _ = std::fs::remove_dir_all(directory); + } + #[tokio::test] async fn external_sessions_route_pcm_progress_and_reject_late_or_wrong_frames() { let microphone_starts = Arc::new(AtomicUsize::new(0)); diff --git a/openless-all/app/crates/openless-core/src/external_audio/archive.rs b/openless-all/app/crates/openless-core/src/external_audio/archive.rs new file mode 100644 index 00000000..14a78619 --- /dev/null +++ b/openless-all/app/crates/openless-core/src/external_audio/archive.rs @@ -0,0 +1,166 @@ +use std::io::{Seek, SeekFrom, Write}; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Mutex; + +use futures_util::future::BoxFuture; + +use crate::{BackendError, BackendErrorCode, RecordingArchive, RecordingPlan, SessionId}; + +pub(super) struct ExternalRecordingArchive { + path: PathBuf, + writer: Mutex<(Option, u32)>, + available: AtomicBool, +} + +impl ExternalRecordingArchive { + pub(super) fn create( + directory: &Path, + session_id: SessionId, + plan: &RecordingPlan, + ) -> std::io::Result { + std::fs::create_dir_all(directory)?; + // 只清理本应用生成的 UUID.wav,且为本次录音预留一个名额。 + let mut entries = Vec::new(); + for entry in std::fs::read_dir(directory)?.flatten() { + let path = entry.path(); + if path.extension().and_then(|ext| ext.to_str()) != Some("wav") + || path + .file_stem() + .and_then(|stem| stem.to_str()) + .and_then(|stem| uuid::Uuid::parse_str(stem).ok()) + .is_none() + { + continue; + } + let Ok(metadata) = entry.metadata() else { + continue; + }; + if !metadata.is_file() { + continue; + } + let modified = metadata.modified().unwrap_or(std::time::UNIX_EPOCH); + if plan.retention_days > 0 + && modified + .elapsed() + .is_ok_and(|age| age.as_secs() > u64::from(plan.retention_days) * 86_400) + { + let _ = std::fs::remove_file(path); + } else { + entries.push((path, modified)); + } + } + entries.sort_by(|left, right| right.1.cmp(&left.1)); + let cap = plan + .max_entries + .map(|count| (count as usize).clamp(1, crate::history::HISTORY_CAP)) + .unwrap_or(crate::history::HISTORY_CAP); + for (path, _) in entries.into_iter().skip(cap - 1) { + let _ = std::fs::remove_file(path); + } + let path = directory.join(format!("{session_id}.wav")); + let mut options = std::fs::OpenOptions::new(); + options.write(true).create_new(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600); + } + let mut file = options.open(&path)?; + file.write_all(&crate::audio::encode_dictation_wav(&[]).expect("empty PCM is valid"))?; + Ok(Self { + path, + writer: Mutex::new((Some(file), 0)), + available: AtomicBool::new(false), + }) + } + + pub(super) fn append(&self, pcm: &[u8]) { + let mut writer = self.writer.lock().expect("external archive lock poisoned"); + if writer.0.is_none() { + return; + } + let result = (|| -> std::io::Result<()> { + let size = u32::try_from(pcm.len()) + .ok() + .and_then(|size| writer.1.checked_add(size)) + .filter(|size| *size <= u32::MAX - 36) + .ok_or_else(|| std::io::Error::other("remote recording exceeds WAV size limit"))?; + let file = writer.0.as_mut().unwrap(); + file.write_all(pcm)?; + // 每帧修正标准头;无需等手机发送 stop,即可读取磁盘上已收到的音频。 + file.seek(SeekFrom::Start(4))?; + file.write_all(&(36 + size).to_le_bytes())?; + file.seek(SeekFrom::Start(40))?; + file.write_all(&size.to_le_bytes())?; + file.seek(SeekFrom::End(0))?; + writer.1 = size; + Ok(()) + })(); + if let Err(error) = result { + log::warn!("[remote-input] 写入录音归档失败:{error}"); + writer.0 = None; + self.available.store(false, Ordering::Release); + } else { + self.available.store(true, Ordering::Release); + } + } + + pub(super) fn finish(&self) { + let mut writer = self.writer.lock().expect("external archive lock poisoned"); + if let Some(file) = writer.0.take() { + if let Err(error) = file.sync_data() { + log::warn!("[remote-input] 同步录音归档失败:{error}"); + self.available.store(false, Ordering::Release); + } + } + if writer.1 == 0 { + let _ = std::fs::remove_file(&self.path); + } + } +} + +impl RecordingArchive for ExternalRecordingArchive { + fn is_available(&self) -> bool { + self.available.load(Ordering::Acquire) + } + + fn read_pcm(&self) -> BoxFuture<'static, Result, BackendError>> { + let path = self.path.clone(); + Box::pin(async move { + let wav = std::fs::read(path).map_err(|error| { + BackendError::new(BackendErrorCode::Persistence, error.to_string()) + })?; + if wav.len() <= 44 + || &wav[..4] != b"RIFF" + || &wav[8..12] != b"WAVE" + || !(wav.len() - 44).is_multiple_of(2) + { + return Err(BackendError::new( + BackendErrorCode::Persistence, + "remote recording WAV is empty or invalid", + )); + } + Ok(wav[44..].to_vec()) + }) + } + + fn discard(&self) -> BoxFuture<'static, Result<(), BackendError>> { + self.writer + .lock() + .expect("external archive lock poisoned") + .0 = None; + let result = match std::fs::remove_file(&self.path) { + Ok(()) => Ok(()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(error) => Err(BackendError::new( + BackendErrorCode::Persistence, + error.to_string(), + )), + }; + if result.is_ok() { + self.available.store(false, Ordering::Release); + } + Box::pin(async move { result }) + } +} diff --git a/openless-all/app/crates/openless-core/src/lib.rs b/openless-all/app/crates/openless-core/src/lib.rs index 98edf187..94f66b7c 100644 --- a/openless-all/app/crates/openless-core/src/lib.rs +++ b/openless-all/app/crates/openless-core/src/lib.rs @@ -308,8 +308,9 @@ pub use providers::{ }; pub use qa_service::QaService; pub use remote_input_service::{ - constant_time_eq, validate_pairing_pin, RemoteFrameCodec, RemoteInputService, - RemoteStreamSequence, REMOTE_INPUT_MAX_PCM_FRAME_BYTES, REMOTE_INPUT_PAIRING_PIN_LEN, + constant_time_eq, finish_remote_input_connection, validate_pairing_pin, RemoteFrameCodec, + RemoteInputService, RemoteStreamSequence, REMOTE_INPUT_MAX_PCM_FRAME_BYTES, + REMOTE_INPUT_PAIRING_PIN_LEN, }; pub use selection_voice_intent::SelectionVoiceIntent; pub use settings::*; diff --git a/openless-all/app/crates/openless-core/src/remote_input_service.rs b/openless-all/app/crates/openless-core/src/remote_input_service.rs index da98b3bf..9eab415a 100644 --- a/openless-all/app/crates/openless-core/src/remote_input_service.rs +++ b/openless-all/app/crates/openless-core/src/remote_input_service.rs @@ -1,12 +1,12 @@ -use std::collections::HashMap; +use std::collections::{HashMap, VecDeque}; use std::sync::{Arc, Mutex}; use futures_util::future::BoxFuture; use crate::credentials::SecretValue; use crate::domains::{ - RemoteAuthResult, RemoteInputApi, RemoteInputConfig, RemoteInputRuntimeAdapter, - RemoteInputServerConfig, RemoteInputStatus, + RemoteAuthResult, RemoteInputApi, RemoteInputConfig, RemoteInputRecovery, + RemoteInputRuntimeAdapter, RemoteInputServerConfig, RemoteInputStatus, }; use crate::errors::{BackendError, BackendErrorCode}; use crate::events::{ @@ -24,6 +24,27 @@ const PIN_LOCK_SECS: u64 = 60; const PIN_FAILS_MAX_ENTRIES: usize = 256; const PIN_GLOBAL_MAX_FAILS: u32 = 20; const PIN_GLOBAL_WINDOW_SECS: u64 = 60; +const RECOVERY_SESSION_CAP: usize = 64; +const RECOVERY_TTL: std::time::Duration = std::time::Duration::from_secs(24 * 60 * 60); + +/// 意外断线后由宿主继续轮询原来的 stop,避免丢弃识别和历史记录收尾。 +/// 主动取消及撤销配对权限仍直接调用 disconnect,不经过此路径。 +pub async fn finish_remote_input_connection( + remote: &dyn RemoteInputApi, + connection_id: SessionId, + session_id: Option, + pending_stop: Option>>, +) -> Result<(), BackendError> { + let result = match pending_stop { + Some(stopping) => stopping.await, + None => match session_id { + Some(session_id) => remote.stop_stream(connection_id, session_id).await, + None => Ok(()), + }, + }; + let cleanup = remote.disconnect(connection_id).await; + result.and(cleanup) +} pub fn validate_pairing_pin(pin: &str) -> bool { pin.len() == REMOTE_INPUT_PAIRING_PIN_LEN && pin.bytes().all(|byte| byte.is_ascii_digit()) @@ -118,9 +139,12 @@ struct RemoteInputState { port: u16, urls: Vec, urls_stale: bool, + ca_fingerprint_sha256: Option, locale: String, pairing_pin: Option, connections: HashMap, + // 恢复凭据与会话 ID 分离,不能凭公开状态中的会话 ID 读取文字。 + recoverable_sessions: VecDeque<(SessionId, SecretValue, std::time::Instant)>, pin_fails: HashMap)>, global_pin_fails: (u32, std::time::Instant), } @@ -165,9 +189,11 @@ impl RemoteInputService { port, urls: Vec::new(), urls_stale: false, + ca_fingerprint_sha256: None, locale, pairing_pin: None, connections: HashMap::new(), + recoverable_sessions: VecDeque::new(), pin_fails: HashMap::new(), global_pin_fails: (0, std::time::Instant::now()), })), @@ -217,10 +243,12 @@ impl RemoteInputService { .filter_map(|connection| connection.stream.take().map(|stream| stream.session_id)) .collect::>(); state.connections.clear(); + state.recoverable_sessions.clear(); state.running = false; state.starting = false; state.urls.clear(); state.urls_stale = false; + state.ca_fingerprint_sha256 = None; sessions }; let mut first_error = None; @@ -247,6 +275,7 @@ impl RemoteInputService { state.running = false; state.urls.clear(); state.urls_stale = false; + state.ca_fingerprint_sha256 = None; } self.publish_status(); match self @@ -261,6 +290,7 @@ impl RemoteInputService { state.port = binding.port; state.urls = binding.urls; state.urls_stale = binding.urls_stale; + state.ca_fingerprint_sha256 = binding.ca_fingerprint_sha256; drop(state); self.publish_status(); Ok(()) @@ -272,6 +302,7 @@ impl RemoteInputService { state.running = false; state.urls.clear(); state.urls_stale = false; + state.ca_fingerprint_sha256 = None; } let public = public_remote_error(&error); self.event_publisher().publish( @@ -484,6 +515,18 @@ impl RemoteInputService { sequence: RemoteStreamSequence::new(session_id), finishing: false, }); + let now = std::time::Instant::now(); + state + .recoverable_sessions + .retain(|(_, _, created)| now.duration_since(*created) < RECOVERY_TTL); + state.recoverable_sessions.push_back(( + session_id, + SecretValue::new(uuid::Uuid::new_v4().to_string()), + now, + )); + while state.recoverable_sessions.len() > RECOVERY_SESSION_CAP { + state.recoverable_sessions.pop_front(); + } true } _ => false, @@ -538,6 +581,9 @@ impl RemoteInputService { let stream = ensure_remote_stream_mut(&mut state, connection_id, session_id)?; if cancel { state.connections.get_mut(&connection_id).unwrap().stream = None; + state + .recoverable_sessions + .retain(|(id, _, _)| *id != session_id); } else { if stream.finishing { return Err(BackendError::new( @@ -600,6 +646,7 @@ impl RemoteInputApi for RemoteInputService { port: state.port, urls: state.urls.clone(), urls_stale: state.urls_stale, + ca_fingerprint_sha256: state.ca_fingerprint_sha256.clone(), locale: state.locale.clone(), connection_count: state.connections.len(), active_session_id: state @@ -666,6 +713,108 @@ impl RemoteInputApi for RemoteInputService { Box::pin(async move { service.disconnect_inner(connection_id).await }) } + fn recover_stream( + &self, + connection_id: SessionId, + session_id: SessionId, + recovery_key: SecretValue, + ) -> BoxFuture<'static, Result> { + let service = self.clone(); + Box::pin(async move { + let allowed = |state: &RemoteInputState| { + state.running + && state.connections.contains_key(&connection_id) + && state.recoverable_sessions.iter().any(|(id, key, created)| { + *id == session_id + && created.elapsed() < RECOVERY_TTL + && constant_time_eq( + key.expose_secret().as_bytes(), + recovery_key.expose_secret().as_bytes(), + ) + }) + }; + let was_pending = { + let state = service + .state + .lock() + .expect("remote input state lock poisoned"); + if !allowed(&state) { + return Ok(RemoteInputRecovery::Unavailable); + } + state.connections.values().any(|connection| { + connection + .stream + .as_ref() + .is_some_and(|stream| stream.session_id == session_id) + }) + }; + let history = service.runtime.read_audio_history(session_id).await?; + let state = service + .state + .lock() + .expect("remote input state lock poisoned"); + // 查询过程中取消、关闭服务或重置 PIN,必须立即撤销恢复权限。 + if !allowed(&state) { + return Ok(RemoteInputRecovery::Unavailable); + } + if let Some(entry) = history { + let text = if entry.final_text.trim().is_empty() { + entry.raw_transcript + } else { + entry.final_text + }; + return Ok(if text.trim().is_empty() { + RemoteInputRecovery::Failed { + has_audio_recording: entry.has_audio_recording == Some(true), + } + } else { + RemoteInputRecovery::Completed { text } + }); + } + Ok( + if was_pending + || state.connections.values().any(|connection| { + connection + .stream + .as_ref() + .is_some_and(|stream| stream.session_id == session_id) + }) + { + RemoteInputRecovery::Pending + } else { + RemoteInputRecovery::Unavailable + }, + ) + }) + } + + fn recovery_key( + &self, + connection_id: SessionId, + session_id: SessionId, + ) -> Result { + let state = self.state.lock().expect("remote input state lock poisoned"); + let owns_stream = state.running + && state + .connections + .get(&connection_id) + .and_then(|connection| connection.stream.as_ref()) + .is_some_and(|stream| stream.session_id == session_id); + if owns_stream { + if let Some((_, key, _)) = state + .recoverable_sessions + .iter() + .find(|(id, _, _)| *id == session_id) + { + return Ok(key.clone()); + } + } + Err(BackendError::new( + BackendErrorCode::Cancelled, + "remote recovery is unavailable", + )) + } + fn start_stream( &self, connection_id: SessionId, diff --git a/openless-all/app/crates/openless-core/src/testing.rs b/openless-all/app/crates/openless-core/src/testing.rs index 2ec695b7..8820b952 100644 --- a/openless-all/app/crates/openless-core/src/testing.rs +++ b/openless-all/app/crates/openless-core/src/testing.rs @@ -188,6 +188,7 @@ impl RemoteInputRuntimeAdapter for RecordingRemoteInputRuntime { port: config.port, urls: vec![format!("https://127.0.0.1:{}", config.port)], urls_stale: false, + ca_fingerprint_sha256: None, }) }) } diff --git a/openless-all/app/crates/openless-core/tests/remote_input_contract.rs b/openless-all/app/crates/openless-core/tests/remote_input_contract.rs index 86fd2a8f..2fa5dc53 100644 --- a/openless-all/app/crates/openless-core/tests/remote_input_contract.rs +++ b/openless-all/app/crates/openless-core/tests/remote_input_contract.rs @@ -12,6 +12,7 @@ use openless_core::{ #[derive(Default)] struct FixtureRemoteRuntime { persisted_pin: Mutex>, + ca_fingerprint_sha256: Mutex>, persist_count: AtomicUsize, reject_persist: AtomicBool, start_count: AtomicUsize, @@ -20,6 +21,8 @@ struct FixtureRemoteRuntime { audio_start_count: AtomicUsize, audio_stop_count: AtomicUsize, audio_cancel_count: AtomicUsize, + history_reads: AtomicUsize, + history: Mutex>, frames: Mutex)>>, insert_preferences: Mutex>, stop_started: Option>, @@ -72,6 +75,7 @@ impl RemoteInputRuntimeAdapter for FixtureRemoteRuntime { ) -> BoxFuture<'static, Result> { self.start_count.fetch_add(1, Ordering::AcqRel); let fail = self.fail_start.load(Ordering::Acquire); + let ca_fingerprint_sha256 = self.ca_fingerprint_sha256.lock().unwrap().clone(); Box::pin(async move { if fail { return Err(BackendError::new(BackendErrorCode::Platform, "port-in-use")); @@ -80,6 +84,7 @@ impl RemoteInputRuntimeAdapter for FixtureRemoteRuntime { port: config.port, urls: vec![format!("https://192.168.1.2:{}", config.port)], urls_stale: false, + ca_fingerprint_sha256, }) }) } @@ -136,6 +141,21 @@ impl RemoteInputRuntimeAdapter for FixtureRemoteRuntime { self.audio_cancel_count.fetch_add(1, Ordering::AcqRel); Box::pin(async { Ok(()) }) } + + fn read_audio_history( + &self, + session_id: SessionId, + ) -> BoxFuture<'static, Result, BackendError>> { + self.history_reads.fetch_add(1, Ordering::AcqRel); + let entry = self + .history + .lock() + .unwrap() + .iter() + .find(|entry| entry.id == session_id.to_string()) + .cloned(); + Box::pin(async move { Ok(entry) }) + } } fn backend(runtime: Arc) -> (OpenLessBackend, std::path::PathBuf) { @@ -280,6 +300,79 @@ fn contract_2_audio_frames_reject_invalid_headers_and_pcm() { ); } +#[tokio::test] +async fn ca_fingerprint_tracks_the_running_listener_and_clears_on_stop_or_failure() { + let runtime = Arc::new(FixtureRemoteRuntime::default()); + let first = "ab".repeat(32); + let replacement = "cd".repeat(32); + *runtime.ca_fingerprint_sha256.lock().unwrap() = Some(first.clone()); + let (backend, data_dir) = backend(Arc::clone(&runtime)); + let remote = &backend.services().remote_input; + assert_eq!(remote.status().unwrap().ca_fingerprint_sha256, None); + + remote + .configure(RemoteInputConfig { + enabled: true, + port: 8443, + }) + .await + .unwrap(); + assert_eq!( + remote.status().unwrap().ca_fingerprint_sha256, + Some(first.clone()) + ); + assert_eq!( + serde_json::to_value(remote.status().unwrap()).unwrap()["caFingerprintSha256"], + first + ); + + remote + .configure(RemoteInputConfig { + enabled: false, + port: 8443, + }) + .await + .unwrap(); + assert_eq!(remote.status().unwrap().ca_fingerprint_sha256, None); + assert!(serde_json::to_value(remote.status().unwrap()) + .unwrap() + .get("caFingerprintSha256") + .is_none()); + + // 启动失败时不能继续展示上一次监听器的指纹。 + *runtime.ca_fingerprint_sha256.lock().unwrap() = Some(replacement.clone()); + runtime.fail_start.store(true, Ordering::Release); + assert!(remote + .configure(RemoteInputConfig { + enabled: true, + port: 9443, + }) + .await + .is_err()); + assert_eq!(remote.status().unwrap().ca_fingerprint_sha256, None); + + runtime.fail_start.store(false, Ordering::Release); + remote + .configure(RemoteInputConfig { + enabled: true, + port: 9443, + }) + .await + .unwrap(); + assert_eq!( + remote.status().unwrap().ca_fingerprint_sha256, + Some(replacement) + ); + remote + .configure(RemoteInputConfig { + enabled: false, + port: 9443, + }) + .await + .unwrap(); + let _ = std::fs::remove_dir_all(data_dir); +} + #[tokio::test] async fn pairing_pin_is_explicit_persisted_and_absent_from_public_surfaces() { let runtime = Arc::new(FixtureRemoteRuntime::default()); @@ -474,6 +567,215 @@ async fn stream_association_validates_frames_and_rejects_duplicates_and_late_pcm let _ = std::fs::remove_dir_all(data_dir); } +#[tokio::test] +async fn interrupted_connection_finishes_two_minutes_of_received_pcm_without_cancelling() { + let runtime = Arc::new(FixtureRemoteRuntime::default()); + let (backend, data_dir) = backend(Arc::clone(&runtime)); + let remote = &backend.services().remote_input; + remote + .configure(RemoteInputConfig { + enabled: true, + port: 8443, + }) + .await + .unwrap(); + let connection = SessionId::new(); + authenticate(remote.as_ref(), connection).await; + let session = remote.start_stream(connection).await.unwrap(); + for second in 0..120 { + remote + .feed_pcm(connection, session, second, vec![second as u8; 32_000]) + .await + .unwrap(); + } + openless_core::finish_remote_input_connection(remote.as_ref(), connection, Some(session), None) + .await + .unwrap(); + assert_eq!(runtime.audio_stop_count.load(Ordering::Acquire), 1); + assert_eq!(runtime.audio_cancel_count.load(Ordering::Acquire), 0); + assert_eq!( + runtime + .frames + .lock() + .unwrap() + .iter() + .map(|(_, pcm)| pcm.len()) + .sum::(), + 120 * 32_000 + ); + assert_eq!(remote.status().unwrap().connection_count, 0); + assert!(remote + .feed_pcm(connection, session, 120, vec![1, 0]) + .await + .is_err()); + let _ = std::fs::remove_dir_all(data_dir); +} + +#[tokio::test] +async fn disconnect_keeps_an_inflight_stop_but_pin_rotation_can_still_revoke_it() { + for revoke in [false, true] { + let started = Arc::new(tokio::sync::Notify::new()); + let release = Arc::new(tokio::sync::Notify::new()); + let runtime = Arc::new(FixtureRemoteRuntime { + stop_started: Some(started), + release_stop: Some(Arc::clone(&release)), + ..Default::default() + }); + let (backend, data_dir) = backend(Arc::clone(&runtime)); + let remote = Arc::clone(&backend.services().remote_input); + remote + .configure(RemoteInputConfig { + enabled: true, + port: 8443, + }) + .await + .unwrap(); + let connection = SessionId::new(); + authenticate(remote.as_ref(), connection).await; + let session = remote.start_stream(connection).await.unwrap(); + let mut stopping = remote.stop_stream(connection, session); + assert!(futures_util::poll!(stopping.as_mut()).is_pending()); + let finishing = { + let remote = Arc::clone(&remote); + tokio::spawn(async move { + openless_core::finish_remote_input_connection( + remote.as_ref(), + connection, + Some(session), + Some(stopping), + ) + .await + }) + }; + tokio::task::yield_now().await; + assert_eq!(runtime.audio_stop_count.load(Ordering::Acquire), 1); + assert_eq!(runtime.audio_cancel_count.load(Ordering::Acquire), 0); + if revoke { + remote.regenerate_pairing_pin().await.unwrap(); + } + release.notify_one(); + let result = finishing.await.unwrap(); + if revoke { + assert_eq!(result.unwrap_err().code, BackendErrorCode::Cancelled); + } else { + result.unwrap(); + } + assert_eq!( + runtime.audio_cancel_count.load(Ordering::Acquire), + usize::from(revoke) + ); + assert_eq!(remote.status().unwrap().connection_count, 0); + let _ = std::fs::remove_dir_all(data_dir); + } +} + +#[tokio::test] +async fn recovery_requires_authentication_and_a_separate_key_and_obeys_revocation() { + use openless_core::RemoteInputRecovery; + let runtime = Arc::new(FixtureRemoteRuntime::default()); + let (backend, data_dir) = backend(Arc::clone(&runtime)); + let remote = &backend.services().remote_input; + remote + .configure(RemoteInputConfig { + enabled: true, + port: 8443, + }) + .await + .unwrap(); + let connection = SessionId::new(); + authenticate(remote.as_ref(), connection).await; + let session = remote.start_stream(connection).await.unwrap(); + let key = remote.recovery_key(connection, session).unwrap(); + let stranger = SessionId::new(); + assert_eq!( + remote + .recover_stream(stranger, session, key.clone()) + .await + .unwrap(), + RemoteInputRecovery::Unavailable + ); + authenticate(remote.as_ref(), stranger).await; + assert!(remote.recovery_key(stranger, session).is_err()); + assert_eq!( + remote + .recover_stream(stranger, session, SecretValue::new(session.to_string())) + .await + .unwrap(), + RemoteInputRecovery::Unavailable + ); + assert_eq!( + remote + .recover_stream(stranger, SessionId::new(), key.clone()) + .await + .unwrap(), + RemoteInputRecovery::Unavailable + ); + assert_eq!(runtime.history_reads.load(Ordering::Acquire), 0); + assert_eq!( + remote + .recover_stream(stranger, session, key.clone()) + .await + .unwrap(), + RemoteInputRecovery::Pending + ); + let public = format!( + "{} {}", + serde_json::to_string(&remote.status().unwrap()).unwrap(), + serde_json::to_string(&backend.replay_events_after(0)).unwrap() + ); + assert!(!public.contains(key.expose_secret())); + openless_core::finish_remote_input_connection(remote.as_ref(), connection, Some(session), None) + .await + .unwrap(); + runtime.history.lock().unwrap().push(serde_json::from_value(serde_json::json!({ + "id": session.to_string(), "createdAt":"2026-09-10T00:00:00Z", "rawTranscript":"received audio", + "finalText":"recovered text", "mode":"raw", "insertStatus":"notRequested", "hasAudioRecording":true, + "appName":"private desktop metadata" + })).unwrap()); + assert_eq!( + remote + .recover_stream(stranger, session, key.clone()) + .await + .unwrap(), + RemoteInputRecovery::Completed { + text: "recovered text".into() + } + ); + { + let mut history = runtime.history.lock().unwrap(); + history[0].raw_transcript.clear(); + history[0].final_text.clear(); + history[0].error_code = Some("transcribeFailed".into()); + } + assert_eq!( + remote + .recover_stream(stranger, session, key.clone()) + .await + .unwrap(), + RemoteInputRecovery::Failed { + has_audio_recording: true + } + ); + remote.regenerate_pairing_pin().await.unwrap(); + authenticate(remote.as_ref(), stranger).await; + assert_eq!( + remote.recover_stream(stranger, session, key).await.unwrap(), + RemoteInputRecovery::Unavailable + ); + let cancelled = remote.start_stream(stranger).await.unwrap(); + let cancelled_key = remote.recovery_key(stranger, cancelled).unwrap(); + remote.cancel_stream(stranger, cancelled).await.unwrap(); + assert_eq!( + remote + .recover_stream(stranger, cancelled, cancelled_key) + .await + .unwrap(), + RemoteInputRecovery::Unavailable + ); + remote.disconnect(stranger).await.unwrap(); + let _ = std::fs::remove_dir_all(data_dir); +} + #[tokio::test] async fn slow_finalization_remains_cancellable_and_cannot_clear_a_new_stream() { for disconnect in [false, true] { diff --git a/openless-all/app/linux-egui/src/backend.rs b/openless-all/app/linux-egui/src/backend.rs index 8775b896..7c1ff6a7 100644 --- a/openless-all/app/linux-egui/src/backend.rs +++ b/openless-all/app/linux-egui/src/backend.rs @@ -879,6 +879,12 @@ impl LinuxBackendBuilder { let recorder = self .recorder .unwrap_or_else(|| Arc::new(LinuxCpalRecorder::new(None)) as Arc); + let recorder: Arc = Arc::new(openless_core::AudioRecorderRouter::new( + recorder, + openless_core::ExternalAudioRecorder::with_recordings_directory( + self.config.data_dir.join("recordings"), + ), + )); let text_inserter = self .text_inserter .unwrap_or_else(|| Arc::new(Fcitx5TextInserter::new(true)) as Arc); diff --git a/openless-all/app/linux-egui/src/main.rs b/openless-all/app/linux-egui/src/main.rs index bc71c24f..2aff6bae 100644 --- a/openless-all/app/linux-egui/src/main.rs +++ b/openless-all/app/linux-egui/src/main.rs @@ -1628,6 +1628,27 @@ mod linux_app { ); } if remote.enabled && remote.running && !remote.urls_stale { + // 首次信任前必须核对根证书指纹:网页与描述文件名称不能证明身份。 + ui.label("本机根证书 SHA-256"); + match remote.ca_fingerprint_sha256.as_ref().filter(|value| { + value.len() == 64 && value.bytes().all(|byte| byte.is_ascii_hexdigit()) + }) { + Some(fingerprint) => { + let display = fingerprint + .as_bytes() + .chunks(2) + .map(|pair| std::str::from_utf8(pair).unwrap().to_ascii_uppercase()) + .collect::>() + .join(" "); + ui.add(egui::Label::new(egui::RichText::new(&display).monospace()).wrap()); + if ui.button("复制完整指纹").clicked() { + ui.ctx().copy_text(display); + } + } + None => ui.label("完整指纹不可用。请勿安装或信任下载的证书。"), + } + ui.label("安装或开启完全信任前,在手机系统的证书详情中核对全部 SHA-256 字符,必须与此处一致。网页、描述文件名称和标识不能证明证书身份。若不一致或无法查看,请停止并移除已下载或安装的描述文件。"); + ui.label("描述文件应只包含一张根证书。若有其他证书、VPN 或设备管理配置,请勿安装。首次下载仍可能被局域网攻击者替换;核验后再信任。根证书可签发其他证书,不再使用时请移除。"); ui.monospace(format!("PIN:{pin}")); for url in &remote.urls { ui.monospace(url); diff --git a/openless-all/app/linux-egui/src/remote_input.rs b/openless-all/app/linux-egui/src/remote_input.rs index 070579d9..8dced627 100644 --- a/openless-all/app/linux-egui/src/remote_input.rs +++ b/openless-all/app/linux-egui/src/remote_input.rs @@ -108,6 +108,7 @@ impl RemoteInputRuntimeAdapter for LinuxRemoteInputRuntime { port: handle.bound_port, urls: access_urls(handle.bound_port), urls_stale: false, + ca_fingerprint_sha256: Some(handle.ca_fingerprint_sha256.clone()), }; *server.lock().await = Some(handle); Ok(binding) @@ -172,6 +173,19 @@ impl RemoteInputRuntimeAdapter for LinuxRemoteInputRuntime { let backend = self.backend(); Box::pin(async move { backend?.cancel_dictation(Some(session_id)).await }) } + + fn read_audio_history( + &self, + session_id: SessionId, + ) -> BoxFuture<'static, Result, BackendError>> { + let backend = self.backend(); + Box::pin(async move { + Ok(backend? + .list_history()? + .into_iter() + .find(|entry| entry.id == session_id.to_string())) + }) + } } #[cfg(target_os = "linux")] @@ -208,6 +222,8 @@ mod assets { const KEEPALIVE_PING_SECS: u64 = 30; #[cfg(target_os = "linux")] const IDLE_TIMEOUT_SECS: u64 = 90; +#[cfg(target_os = "linux")] +const AUDIO_IDLE_TIMEOUT_SECS: u64 = 15; #[cfg(target_os = "linux")] struct LinuxRemoteServerHandle { @@ -217,6 +233,7 @@ struct LinuxRemoteServerHandle { connections_shutdown: tokio::sync::watch::Sender, join: tokio::task::JoinHandle<()>, bound_port: u16, + ca_fingerprint_sha256: String, } #[cfg(target_os = "linux")] @@ -233,6 +250,7 @@ impl LinuxRemoteServerHandle { #[cfg(not(target_os = "linux"))] struct LinuxRemoteServerHandle { bound_port: u16, + ca_fingerprint_sha256: String, } #[cfg(not(target_os = "linux"))] @@ -401,6 +419,7 @@ async fn start_server( sans.extend(local_lan_ipv4s()); let identity = tls_identity::load_or_create(&data_dir.join("remote-input"), &sans) .map_err(remote_platform_error)?; + let ca_fingerprint_sha256 = identity.ca_fingerprint_sha256; let cert_der = identity.trust_cert; let acceptor = TlsAcceptor::from(identity.server_config); let listener = TcpListener::bind(SocketAddr::from(([0, 0, 0, 0], port))) @@ -443,6 +462,7 @@ async fn start_server( connections_shutdown, join, bound_port, + ca_fingerprint_sha256, }) } @@ -498,18 +518,27 @@ async fn websocket_session(mut socket: WebSocket, state: Arc, peer: Ip let mut keepalive = tokio::time::interval(Duration::from_secs(KEEPALIVE_PING_SECS)); keepalive.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay); let mut last_received = Instant::now(); + let mut last_audio = Instant::now(); + let mut audio_watchdog = tokio::time::interval(Duration::from_secs(2)); + audio_watchdog.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay); + let mut preserve_audio = true; + let mut receiving_audio = false; 'connection: loop { tokio::select! { incoming = socket.recv() => { last_received = Instant::now(); match incoming { Some(Ok(Message::Binary(frame))) => { + if !receiving_audio { continue; } if let Ok((session_id, sequence, pcm)) = openless_core::RemoteFrameCodec::decode(&frame) { - let _ = state.backend.services().remote_input - .feed_pcm(connection_id, session_id, sequence, pcm).await; + if state.backend.services().remote_input + .feed_pcm(connection_id, session_id, sequence, pcm).await.is_ok() { + last_audio = Instant::now(); + } } } Some(Ok(Message::Text(text))) => { + let previous_session = remote_session; if let Some(reply) = apply_control( &text, state.backend.services().remote_input.as_ref(), @@ -519,6 +548,11 @@ async fn websocket_session(mut socket: WebSocket, state: Arc, peer: Ip ).await { if socket.send(json_message(reply)).await.is_err() { break; } } + if remote_session != previous_session { + last_audio = Instant::now(); + receiving_audio = remote_session.is_some(); + } + if pending_stop.is_some() { receiving_audio = false; } } Some(Ok(Message::Close(_))) | Some(Err(_)) | None => break, _ => {} @@ -533,9 +567,7 @@ async fn websocket_session(mut socket: WebSocket, state: Arc, peer: Ip phase: openless_core::DictationPhase::Cancelled | openless_core::DictationPhase::Failed, .. })); for reply in remote_event(event.kind) { - // A failed outbound send means this socket no longer owns - // a usable transport. Leave the outer loop immediately so - // Core disconnect cancels any active external-audio lease. + // 下行失败也保留已收到的录音,由外层继续完成识别和持久化。 if socket.send(json_message(reply)).await.is_err() { break 'connection; } @@ -561,11 +593,37 @@ async fn websocket_session(mut socket: WebSocket, state: Arc, peer: Ip break; } } + _ = audio_watchdog.tick(), if receiving_audio && pending_stop.is_none() && remote_session.is_some() => { + if last_audio.elapsed() >= Duration::from_secs(AUDIO_IDLE_TIMEOUT_SECS) { + receiving_audio = false; + let session_id = remote_session.unwrap(); + pending_stop = Some((session_id, state.backend.services().remote_input.stop_stream(connection_id, session_id))); + } + } changed = shutdown.changed() => { - if changed.is_err() || *shutdown.borrow() { break; } + if changed.is_err() || *shutdown.borrow() { preserve_audio = false; break; } } } } + drop(socket); + if preserve_audio && !*shutdown.borrow() { + let finishing = openless_core::finish_remote_input_connection( + state.backend.services().remote_input.as_ref(), + connection_id, + remote_session, + pending_stop.take().map(|(_, future)| future), + ); + tokio::pin!(finishing); + tokio::select! { + result = &mut finishing => { + if let Err(error) = result { log::warn!("[remote-input] 断线录音收尾:{error}"); } + } + _ = shutdown.changed() => { + let _ = state.backend.services().remote_input.disconnect(connection_id).await; + } + } + return; + } let _ = state .backend .services() @@ -610,7 +668,10 @@ async fn apply_control( "start" => match remote.start_stream(connection_id).await { Ok(session_id) => { *remote_session = Some(session_id); - Some(serde_json::json!({"type":"started", "sessionId":session_id.to_string()})) + Some( + serde_json::json!({"type":"started", "sessionId":session_id.to_string(), + "recoveryKey":remote.recovery_key(connection_id, session_id).ok().map(|key| key.into_exposed())}), + ) } Err(error) => Some(serde_json::json!({"type":"busy", "reason":error.to_string()})), }, @@ -636,6 +697,32 @@ async fn apply_control( } None } + "recover" => { + let session_id = value + .get("sessionId") + .and_then(|value| value.as_str()) + .and_then(|value| uuid::Uuid::parse_str(value).ok()) + .map(SessionId::from_uuid); + let recovery = match session_id { + Some(session_id) => remote + .recover_stream( + connection_id, + session_id, + SecretValue::new( + value + .get("recoveryKey") + .and_then(|value| value.as_str()) + .unwrap_or_default(), + ), + ) + .await + .unwrap_or(openless_core::RemoteInputRecovery::Unavailable), + None => openless_core::RemoteInputRecovery::Unavailable, + }; + Some( + serde_json::json!({"type":"recovery", "sessionId":session_id, "recovery":recovery}), + ) + } "set_insert" => { let insert = value .get("value") diff --git a/openless-all/app/scripts/remote-input-audio-queue.test.mjs b/openless-all/app/scripts/remote-input-audio-queue.test.mjs index 99f4017e..5a155456 100644 --- a/openless-all/app/scripts/remote-input-audio-queue.test.mjs +++ b/openless-all/app/scripts/remote-input-audio-queue.test.mjs @@ -37,12 +37,31 @@ function fakeElement() { }; } -async function openRemotePage({ defaultMode, savedMode } = {}) { +const fixtureRecoveryKey = '40112233-4455-4677-8899-aabbccddeeff'; +async function openRemotePage({ + defaultMode, + savedMode, + savedWakeLock, + savedRecovery, + wakeLockMode = 'supported', +} = {}) { const elements = new Map(); const documentListeners = {}; const sent = []; let socket; let worklet; + let audioContext; + const windowListeners = {}; + const wakeLocks = []; + const tracks = []; + const wakeResolvers = []; + let wakeRequests = 0; + const timers = new Map(); + let now = 0; + let timerId = 0; + const flush = async () => { + for (let i = 0; i < 12; i += 1) await Promise.resolve(); + }; const element = (id) => { if (!elements.has(id)) elements.set(id, fakeElement()); @@ -81,11 +100,13 @@ async function openRemotePage({ defaultMode, savedMode } = {}) { class FakeAudioContext { constructor() { + audioContext = this; this.state = 'running'; this.sampleRate = 48_000; this.audioWorklet = { addModule: () => Promise.resolve() }; } resume() { + this.state = 'running'; return Promise.resolve(); } suspend() { @@ -121,24 +142,78 @@ async function openRemotePage({ defaultMode, savedMode } = {}) { AudioContext: FakeAudioContext, AudioWorkletNode: FakeAudioWorkletNode, WebSocket: FakeWebSocket, - clearTimeout, + clearTimeout: (id) => timers.delete(id), console, document, isNaN, localStorage: storage([ ['ol_remote_pin', '123456'], ...(savedMode === undefined ? [] : [['ol_remote_mode', savedMode]]), + ...(savedWakeLock === undefined ? [] : [['ol_remote_wake_lock', savedWakeLock]]), + ...(savedRecovery === undefined + ? [] + : [ + [ + 'ol_remote_recovery_session', + JSON.stringify({ sessionId: savedRecovery, key: fixtureRecoveryKey }), + ], + ]), ]), location: { host: 'localhost:8443', origin: 'https://localhost:8443', reload() {} }, navigator: { language: 'zh-CN', + wakeLock: + wakeLockMode === 'unsupported' + ? undefined + : { + request: (type) => { + assert.equal(type, 'screen'); + wakeRequests++; + if (wakeLockMode === 'rejected') return Promise.reject(new Error('system denied')); + const sentinel = { + released: false, + releaseCount: 0, + listener: null, + addEventListener(type, listener) { + assert.equal(type, 'release'); + this.listener = listener; + }, + release() { + this.released = true; + this.releaseCount++; + this.listener?.(); + return Promise.resolve(); + }, + }; + wakeLocks.push(sentinel); + return wakeLockMode === 'deferred' + ? new Promise((resolve) => wakeResolvers.push(() => resolve(sentinel))) + : Promise.resolve(sentinel); + }, + }, mediaDevices: { - getUserMedia: () => Promise.resolve({ getTracks: () => [{ stop() {} }] }), + getUserMedia: () => { + const track = { + stopped: false, + stop() { + this.stopped = true; + }, + }; + tracks.push(track); + return Promise.resolve({ getTracks: () => [track] }); + }, }, }, performance: { now: () => 100 }, sessionStorage: storage([['ol_reloaded_once', '1']]), - setTimeout, + setTimeout: (callback, delay) => { + const id = ++timerId; + timers.set(id, { callback, at: now + delay }); + return id; + }, + addEventListener: (type, listener) => { + windowListeners[type] = listener; + }, }; context.window = context; // Exercise the embedded HTML script too, so a missing template variable cannot @@ -158,11 +233,35 @@ async function openRemotePage({ defaultMode, savedMode } = {}) { documentListeners, element, sent, - socket, + get socket() { + return socket; + }, + get audioContext() { + return audioContext; + }, + get wakeRequests() { + return wakeRequests; + }, + wakeLocks, + tracks, + wakeResolvers, + windowListeners, + flush, + async advance(ms) { + now += ms; + for (const [id, timer] of [...timers]) { + if (timer.at <= now && timers.delete(id)) timer.callback(); + } + await flush(); + }, storage: context.localStorage, async start() { - element('btn-record').listeners.click(); - for (let i = 0; i < 8; i += 1) await Promise.resolve(); + if (element('btn-record').style.touchAction === 'none') { + element('btn-record').listeners.pointerdown({ preventDefault() {} }); + } else { + element('btn-record').listeners.click(); + } + await flush(); assert.ok(worklet?.port.onmessage, 'audio capture must be running'); }, pcm(bytes) { @@ -259,8 +358,9 @@ for (const terminal of ['cancel', 'busy', 'disconnect']) { await page.start(); page.pcm([6, 0]); if (terminal === 'cancel') { - page.document.hidden = true; - page.documentListeners.visibilitychange(); + page.element('mode-switch').listeners.click({ + target: { closest: () => ({ getAttribute: () => 'hold' }) }, + }); } else if (terminal === 'busy') { page.socket.onmessage({ data: JSON.stringify({ type: 'busy', reason: 'test' }) }); } else { @@ -289,4 +389,166 @@ for (const terminal of ['cancel', 'busy', 'disconnect']) { assert.equal(binaryFrames(page.sent).length, 0); } +const controls = (page, type) => + page.sent + .filter((value) => typeof value === 'string') + .map((value) => JSON.parse(value)) + .filter((value) => value.type === type); +const recordingId = '30112233-4455-4677-8899-aabbccddeeff'; +const acknowledge = (page) => + page.socket.onmessage({ + data: JSON.stringify({ + type: 'started', + sessionId: recordingId, + recoveryKey: fixtureRecoveryKey, + }), + }); + +// 息屏结束两分钟录音,已发送的帧仍在 stop 之前;重复生命周期事件不会重复结束。 +for (const defaultMode of ['toggle', 'hold']) { + const page = await openRemotePage({ defaultMode }); + assert.equal(page.element('wake-lock-switch').checked, true); + assert.equal(page.storage.getItem('ol_remote_wake_lock'), null); + await page.start(); + acknowledge(page); + for (let second = 0; second < 120; second++) page.pcm(new Uint8Array(32_000).fill(second)); + page.document.hidden = true; + page.documentListeners.visibilitychange(); + page.windowListeners.pagehide(); + assert.equal(controls(page, 'stop').length, 1); + assert.equal(controls(page, 'cancel').length, 0); + assert.equal( + binaryFrames(page.sent).reduce((bytes, frame) => bytes + frame.byteLength - 28, 0), + 120 * 32_000, + ); + assert.equal(JSON.parse(page.sent.at(-1)).type, 'stop'); + assert.equal(page.wakeLocks[0].released, true); + assert.deepEqual(JSON.parse(page.storage.getItem('ol_remote_recovery_session')), { + sessionId: recordingId, + key: fixtureRecoveryKey, + }); + page.document.hidden = false; + page.documentListeners.visibilitychange(); + assert.equal(controls(page, 'start').length, 1, '恢复前台不会自动打开麦克风'); + assert.equal(page.wakeRequests, 1); +} + +{ + const page = await openRemotePage(); + await page.start(); + page.pcm([7, 0, 8, 0]); + page.document.hidden = true; + page.documentListeners.visibilitychange(); + assert.equal(controls(page, 'cancel').length, 0); + acknowledge(page); + assert.deepEqual(payload(binaryFrames(page.sent)[0]), [7, 0, 8, 0]); + assert.equal(JSON.parse(page.sent.at(-1)).type, 'stop', 'ACK 迟到时保留首段音频并按序结束'); +} + +{ + const page = await openRemotePage({ savedWakeLock: '0' }); + assert.equal(page.element('wake-lock-switch').checked, false); + await page.start(); + assert.equal(page.wakeRequests, 0); + const toggle = page.element('wake-lock-switch'); + toggle.checked = true; + toggle.listeners.change(); + await page.flush(); + assert.equal(page.wakeRequests, 1); + assert.equal(page.storage.getItem('ol_remote_wake_lock'), '1'); + toggle.checked = false; + toggle.listeners.change(); + assert.equal(page.wakeLocks[0].releaseCount, 1); + assert.equal(page.storage.getItem('ol_remote_wake_lock'), '0'); +} + +for (const wakeLockMode of ['unsupported', 'rejected']) { + const page = await openRemotePage({ wakeLockMode }); + await page.start(); + assert.equal(controls(page, 'start').length, 1, '保持亮屏失败不阻止录音'); + assert.match(page.element('wake-lock-hint').textContent, /未允许/); + acknowledge(page); + page.audioContext.state = 'interrupted'; + page.audioContext.onstatechange(); + assert.equal(controls(page, 'stop').length, 1); + assert.equal(controls(page, 'cancel').length, 0); +} + +{ + const page = await openRemotePage(); + await page.start(); + acknowledge(page); + page.tracks[0].onended(); + assert.equal(controls(page, 'stop').length, 1); + assert.equal(page.tracks[0].stopped, true); + page.socket.onmessage({ data: JSON.stringify({ type: 'status', kind: 'done' }) }); + await page.start(); + assert.equal(page.tracks.length, 2, '系统中断后的下一次录音必须重新获取麦克风'); + assert.equal(page.tracks[1].stopped, false); +} + +{ + const page = await openRemotePage({ wakeLockMode: 'deferred' }); + await page.start(); + acknowledge(page); + page.element('btn-record').listeners.click(); + page.wakeResolvers[0](); + await page.flush(); + assert.equal(page.wakeLocks[0].releaseCount, 1, '录音结束后迟到的请求必须释放'); +} + +{ + const page = await openRemotePage(); + await page.start(); + await page.wakeLocks[0].release(); + assert.match(page.element('wake-lock-hint').textContent, /未允许/); + assert.equal(page.wakeRequests, 1, '系统释放后不能循环申请'); + acknowledge(page); + page.socket.readyState = 3; + page.socket.onclose(); + page.documentListeners.visibilitychange(); + page.socket.onopen(); + page.socket.onmessage({ data: JSON.stringify({ type: 'auth', ok: true }) }); + assert.equal(controls(page, 'recover').at(-1).sessionId, recordingId); + assert.equal(controls(page, 'recover').at(-1).recoveryKey, fixtureRecoveryKey); + page.socket.onmessage({ + data: JSON.stringify({ + type: 'recovery', + sessionId: recordingId, + recovery: { kind: 'pending' }, + }), + }); + assert.equal(page.element('btn-record').disabled, true); + await page.advance(1500); + assert.equal(controls(page, 'recover').length, 2); + page.socket.onmessage({ + data: JSON.stringify({ + type: 'recovery', + sessionId: recordingId, + recovery: { kind: 'completed', text: '保留下来的两分钟录音' }, + }), + }); + assert.equal(page.element('result-text').textContent, '保留下来的两分钟录音'); + assert.equal(page.element('btn-record').disabled, false); +} + +{ + const page = await openRemotePage({ savedRecovery: recordingId }); + const oldSocket = page.socket; + await page.advance(8000); + assert.notEqual(page.socket, oldSocket, '半开连接恢复超时后重新认证'); + assert.equal(oldSocket.readyState, 3); +} + +for (const recovery of [{ kind: 'failed', hasAudioRecording: true }, { kind: 'unavailable' }]) { + const page = await openRemotePage({ savedRecovery: recordingId }); + page.socket.onmessage({ + data: JSON.stringify({ type: 'recovery', sessionId: recordingId, recovery }), + }); + assert.match(page.element('status-text').textContent, /历史记录/); + assert.equal(page.element('btn-record').disabled, false); + if (recovery.kind === 'unavailable') + assert.equal(page.storage.getItem('ol_remote_recovery_session'), null); +} + console.log('remote-input-audio-queue.test.mjs passed'); diff --git a/openless-all/app/src-tauri/src/commands/remote_input.rs b/openless-all/app/src-tauri/src/commands/remote_input.rs index 6837b0e5..43843fbb 100644 --- a/openless-all/app/src-tauri/src/commands/remote_input.rs +++ b/openless-all/app/src-tauri/src/commands/remote_input.rs @@ -36,6 +36,7 @@ fn map_remote_input_status( pin: pin.into_exposed(), urls: status.urls, urls_stale: status.urls_stale, + ca_fingerprint_sha256: status.ca_fingerprint_sha256, } } @@ -100,6 +101,7 @@ mod tests { port: 9443, urls: vec!["https://192.168.1.2:9443".into()], urls_stale: false, + ca_fingerprint_sha256: Some("ab".repeat(32)), locale: "zh-CN".into(), connection_count: 2, active_session_id: Some(openless_core::SessionId::new()), @@ -115,7 +117,8 @@ mod tests { "port": 9443, "pin": "123456", "urls": ["https://192.168.1.2:9443"], - "urlsStale": false + "urlsStale": false, + "caFingerprintSha256": "ab".repeat(32) }) ); } diff --git a/openless-all/app/src-tauri/src/core_adapters.rs b/openless-all/app/src-tauri/src/core_adapters.rs index de649537..2ef82fa7 100644 --- a/openless-all/app/src-tauri/src/core_adapters.rs +++ b/openless-all/app/src-tauri/src/core_adapters.rs @@ -193,16 +193,27 @@ pub(crate) fn backend_dependencies( app: Arc::clone(&app), backend: Arc::clone(&backend), }); - let recorder = - AudioRecorderRouter::new(Arc::clone(&host_recorder), ExternalAudioRecorder::default()); + let external = match crate::persistence::data_dir() { + Ok(directory) => { + ExternalAudioRecorder::with_recordings_directory(directory.join("recordings")) + } + Err(error) => { + log::warn!("[remote-input] 无法定位录音目录:{error}"); + ExternalAudioRecorder::default() + } + }; + let recorder: Arc = Arc::new(AudioRecorderRouter::new( + Arc::clone(&host_recorder), + external, + )); let traditional = Arc::new(openless_core::PipelineDictationEngine::new( - Arc::new(recorder), + Arc::clone(&recorder), transcription, Arc::clone(&polisher), )); let dictation = Arc::new(openless_core::DictationEngineRouter::new(traditional)); let production_omni: Arc = Arc::new( - openless_core::SharedOmniDictationEngine::new(Arc::clone(&credential_store), host_recorder), + openless_core::SharedOmniDictationEngine::new(Arc::clone(&credential_store), recorder), ); for provider_type in openless_core::SHARED_OMNI_PROVIDER_TYPES { dictation @@ -1280,6 +1291,7 @@ impl openless_core::RemoteInputRuntimeAdapter for TauriRemoteInputRuntimeAdapter port: handle.bound_port, urls: crate::remote_server::access_urls(handle.bound_port), urls_stale: false, + ca_fingerprint_sha256: Some(handle.ca_fingerprint_sha256.clone()), }; *server.lock().await = Some(handle); Ok(binding) @@ -1353,6 +1365,19 @@ impl openless_core::RemoteInputRuntimeAdapter for TauriRemoteInputRuntimeAdapter let backend = self.backend(); Box::pin(async move { backend?.cancel_dictation(Some(session_id)).await }) } + + fn read_audio_history( + &self, + session_id: SessionId, + ) -> BoxFuture<'static, Result, BackendError>> { + let backend = self.backend(); + Box::pin(async move { + Ok(backend? + .list_history()? + .into_iter() + .find(|entry| entry.id == session_id.to_string())) + }) + } } struct TauriPlatformApi { diff --git a/openless-all/app/src-tauri/src/remote_server/assets/app.js b/openless-all/app/src-tauri/src/remote_server/assets/app.js index abd692e8..d3ed74b0 100644 --- a/openless-all/app/src-tauri/src/remote_server/assets/app.js +++ b/openless-all/app/src-tauri/src/remote_server/assets/app.js @@ -14,6 +14,16 @@ // ============================================================ var I18N = { 'zh-CN': { + wakeLockLabel: '录音时保持亮屏', + wakeLockHint: '息屏会结束本段录音,电脑继续处理已收到的部分。', + wakeLockActive: '屏幕保持亮起,录音结束后允许自动息屏。', + wakeLockUnavailable: '浏览器或系统未允许保持亮屏;息屏后电脑会处理已收到的录音。', + interrupted: '录音已中断,电脑继续识别已收到的部分…', + offlineRecording: '连接已断开,电脑会继续处理已收到的录音。重连后可查看结果。', + recovering: '电脑正在处理上次录音…', + recovered: '已找回上次识别结果', + recoveryRetry: '识别未完成,录音已保存在电脑历史记录中,可重新转录。', + recoveryUnavailable: '暂未找到结果,请到电脑的历史记录中查看。', title: 'OpenLess 远程输入', brandTitle: 'OpenLess 远程输入', brandSub: '在手机上录音,实时输入到电脑', @@ -26,7 +36,8 @@ offlineTitle: '连接已断开', offlineSub: '与电脑的连接已中断。', btnReconnect: '重新连接', - certTip: "如遇证书提示,请核对地址与电脑显示一致,再按“首次设置:信任此电脑”完成安装和完全信任。", + certTip: + '信任前,请将手机系统证书详情中的完整 SHA-256 与电脑 OpenLess 设置核对;仅核对 IP 或名称不够。', tipToggle: '点击大按钮开始录音,再次点击结束并识别。', tipHold: '按住大按钮说话,松开结束并识别。', labelToggleIdle: '点击开始', @@ -57,18 +68,33 @@ pcmQueueOverflow: '❌ 音频缓存已满,请重试。', micUnknown: '❌ 无法启动录音{name}。', errGeneric: '发生错误', - helpTitle: "首次设置:信任此电脑", - helpAndroid: "安卓:下载 CA 证书,在系统设置的“安装证书 → CA 证书”中安装,再回到本页。菜单名称因设备而异。", - helpIos: "iPhone / iPad:下载描述文件,打开“设置 → 通用 → VPN 与设备管理”安装;再到“通用 → 关于本机 → 证书信任设置”,为 OpenLess Remote Input CA 打开“完全信任”,返回 Safari 刷新。每台电脑只需设置一次;旧版证书需要重新安装。", - helpDownloadCert: "↓ iPhone:下载描述文件", - helpDownloadAndroid: "↓ 安卓:下载 CA 证书", - helpTrustWarning: "首次证书下载无法验证电脑身份,恶意局域网设备可能通过中间人攻击替换根证书。仅在可信的家庭或私人网络中安装,勿在公共或共享网络操作。根证书具备签发能力,私钥保存在这台电脑;不再使用时请从手机移除。", + helpTitle: '首次设置:信任此电脑', + helpAndroid: + '安卓:下载 CA 证书,在系统证书预览中核对完整 SHA-256 后再安装。若系统无法在信任前显示指纹,请勿从此页面安装,改用已有的可信文件传输渠道。菜单名称因设备而异。', + helpVerify: + '先打开电脑上的 OpenLess 远程输入设置,保留“本机根证书 SHA-256”。在手机系统证书详情中核对全部 64 个字符;不要使用本网页、描述文件名称或标识里的值作为证明。不一致或无法查看时,请停止并移除描述文件。描述文件必须只含一张根证书,不得有其他证书、VPN 或管理配置。', + helpIos: + 'iPhone / iPad:下载描述文件,在“设置 → 通用 → VPN 与设备管理”中打开它,选择“更多详细信息”中的证书并核对指纹。确认一致且没有额外配置后再安装,并到“通用 → 关于本机 → 证书信任设置”开启完全信任。若只能在安装后查看详情,先保持完全信任关闭,核对后再开启。完成后返回 Safari 刷新。', + helpDownloadCert: '↓ iPhone:下载描述文件', + helpDownloadAndroid: '↓ 安卓:下载 CA 证书', + helpTrustWarning: + '首次证书下载无法验证电脑身份,恶意局域网设备可能通过中间人攻击替换根证书。仅在可信的家庭或私人网络中安装,勿在公共或共享网络操作。根证书具备签发能力,私钥保存在这台电脑;不再使用时请从手机移除。', helpCopyLink: '⧉ 复制链接', helpCopied: '已复制 ✓', copy: '复制', copied: '已复制 ✓', }, 'zh-TW': { + wakeLockLabel: '錄音時保持螢幕開啟', + wakeLockHint: '螢幕關閉會結束本段錄音,電腦繼續處理已收到的部分。', + wakeLockActive: '螢幕保持開啟,錄音結束後允許自動關閉螢幕。', + wakeLockUnavailable: '瀏覽器或系統未允許保持螢幕開啟;電腦會處理已收到的錄音。', + interrupted: '錄音已中斷,電腦繼續辨識已收到的部分…', + offlineRecording: '連線已中斷,電腦會繼續處理已收到的錄音。重新連線後可查看結果。', + recovering: '電腦正在處理上次錄音…', + recovered: '已找回上次辨識結果', + recoveryRetry: '辨識未完成,錄音已保存在電腦歷史記錄中,可重新轉錄。', + recoveryUnavailable: '暫未找到結果,請到電腦的歷史記錄中查看。', title: 'OpenLess 遠端輸入', brandTitle: 'OpenLess 遠端輸入', brandSub: '在手機上錄音,即時輸入到電腦', @@ -81,7 +107,8 @@ offlineTitle: '連線已中斷', offlineSub: '與電腦的連線已中斷。', btnReconnect: '重新連線', - certTip: "如遇憑證提示,請核對位址與電腦顯示一致,再依「首次設定:信任這台電腦」完成安裝與完全信任。", + certTip: + '信任前,請將手機系統憑證詳細資訊中的完整 SHA-256 與電腦 OpenLess 設定核對;僅核對 IP 或名稱不足以驗證。', tipToggle: '點擊大按鈕開始錄音,再次點擊結束並辨識。', tipHold: '按住大按鈕說話,放開結束並辨識。', labelToggleIdle: '點擊開始', @@ -112,18 +139,37 @@ pcmQueueOverflow: '❌ 音訊暫存已滿,請重試。', micUnknown: '❌ 無法啟動錄音{name}。', errGeneric: '發生錯誤', - helpTitle: "首次設定:信任這台電腦", - helpAndroid: "Android:下載 CA 憑證,在系統設定的「安裝憑證 → CA 憑證」中安裝,再返回本頁。選單名稱依裝置而異。", - helpIos: "iPhone / iPad:下載描述檔,開啟「設定 → 一般 → VPN 與裝置管理」安裝;再到「一般 → 關於本機 → 憑證信任設定」,為 OpenLess Remote Input CA 開啟「完全信任」,返回 Safari 重新整理。每台電腦只需設定一次;舊版憑證需要重新安裝。", - helpDownloadCert: "↓ iPhone:下載描述檔", - helpDownloadAndroid: "↓ Android:下載 CA 憑證", - helpTrustWarning: "首次憑證下載無法驗證電腦身分,惡意區域網路裝置可能透過中間人攻擊替換根憑證。僅在可信任的家庭或私人網路中安裝,請勿在公共或共享網路操作。根憑證能簽發憑證,私密金鑰保存在這台電腦;不再使用時請從手機移除。", + helpTitle: '首次設定:信任這台電腦', + helpAndroid: + 'Android:下載 CA 憑證,在系統憑證預覽中核對完整 SHA-256 後再安裝。若系統無法在信任前顯示指紋,請勿從此頁安裝,改用既有的可信任檔案傳輸管道。選單名稱依裝置而異。', + helpVerify: + '先開啟電腦的 OpenLess 遠端輸入設定,保留「本機根憑證 SHA-256」。在手機系統憑證詳細資訊中核對全部 64 個字元;不要使用本網頁、描述檔名稱或識別碼中的值作為證明。不一致或無法查看時,請停止並移除描述檔。描述檔必須只含一張根憑證,不得有其他憑證、VPN 或管理設定。', + helpIos: + 'iPhone / iPad:下載描述檔,在「設定 → 一般 → VPN 與裝置管理」中開啟,選擇「更多詳細資訊」中的憑證並核對指紋。確認一致且沒有額外設定後再安裝,並到「一般 → 關於本機 → 憑證信任設定」開啟完全信任。若只能在安裝後查看詳細資訊,請先保持完全信任關閉,核對後再開啟。完成後返回 Safari 重新整理。', + helpDownloadCert: '↓ iPhone:下載描述檔', + helpDownloadAndroid: '↓ Android:下載 CA 憑證', + helpTrustWarning: + '首次憑證下載無法驗證電腦身分,惡意區域網路裝置可能透過中間人攻擊替換根憑證。僅在可信任的家庭或私人網路中安裝,請勿在公共或共享網路操作。根憑證能簽發憑證,私密金鑰保存在這台電腦;不再使用時請從手機移除。', helpCopyLink: '⧉ 複製連結', helpCopied: '已複製 ✓', copy: '複製', copied: '已複製 ✓', }, en: { + wakeLockLabel: 'Keep screen awake while recording', + wakeLockHint: + 'Screen lock ends this recording. The computer processes the audio already received.', + wakeLockActive: 'Screen stays awake until recording ends.', + wakeLockUnavailable: + 'The browser or system did not allow screen wake lock. Received audio will still be processed.', + interrupted: 'Recording interrupted. The computer is processing the audio received…', + offlineRecording: + 'Disconnected. The computer continues processing received audio. Reconnect to see the result.', + recovering: 'The computer is processing your last recording…', + recovered: 'Last transcription recovered', + recoveryRetry: + 'Transcription failed. The recording is saved in computer history and can be retried.', + recoveryUnavailable: 'Result unavailable. Please check history on the computer.', title: 'OpenLess Remote Input', brandTitle: 'OpenLess Remote Input', brandSub: 'Record on your phone, type to your computer in real time', @@ -136,7 +182,8 @@ offlineTitle: 'Disconnected', offlineSub: 'The connection to your computer was lost.', btnReconnect: 'Reconnect', - certTip: "If a certificate warning appears, check that the address matches your computer, then follow “First-time setup: trust this computer” to install and fully trust the certificate.", + certTip: + "Before trusting, compare the full SHA-256 in the phone's system certificate details with OpenLess settings on the computer. An IP address or name alone is not enough.", tipToggle: 'Tap the big button to start recording, tap again to finish and transcribe.', tipHold: 'Hold the big button to talk, release to finish and transcribe.', labelToggleIdle: 'Tap to start', @@ -169,12 +216,17 @@ pcmQueueOverflow: '❌ Audio buffer is full. Please try again.', micUnknown: '❌ Could not start recording{name}.', errGeneric: 'An error occurred', - helpTitle: "First-time setup: trust this computer", - helpAndroid: "Android: download the CA certificate, install it under system Settings → Install a certificate → CA certificate, then return here. Menu names vary by device.", - helpIos: "iPhone / iPad: download the profile and install it in Settings → General → VPN & Device Management. Then enable full trust for OpenLess Remote Input CA in General → About → Certificate Trust Settings and reload Safari. Set up once per computer; certificates from older versions need this one-time replacement.", - helpDownloadCert: "↓ iPhone: download profile", - helpDownloadAndroid: "↓ Android: download CA", - helpTrustWarning: "The initial certificate download cannot verify the computer's identity; a malicious device on the LAN could replace the root certificate in a man-in-the-middle attack. Install it only on a trusted home or private network, never on a public or shared network. The root CA can issue certificates and its private key stays on this computer; remove it from your phone when no longer needed.", + helpTitle: 'First-time setup: trust this computer', + helpAndroid: + 'Android: download the CA and verify its full SHA-256 in the system certificate preview before installing it. If your device cannot show the fingerprint before trust, do not install from this page; use an existing authenticated file-transfer channel instead. Menu names vary by device.', + helpVerify: + "Open Remote Input settings in OpenLess on the computer and keep its root CA SHA-256 visible. Compare all 64 characters in the phone's system certificate details; do not use a value from this page, a profile name or identifier as proof. If it differs or cannot be viewed, stop and remove the profile. Expect only one root certificate, with no additional certificates, VPN or management settings.", + helpIos: + 'iPhone / iPad: download the profile, open it in Settings → General → VPN & Device Management, then open More Details → certificate and verify its fingerprint. Only after it matches and no extra settings are present, install and enable full trust in General → About → Certificate Trust Settings. If details are available only after installation, leave full trust off until verified. Reload Safari afterwards.', + helpDownloadCert: '↓ iPhone: download profile', + helpDownloadAndroid: '↓ Android: download CA', + helpTrustWarning: + "The initial certificate download cannot verify the computer's identity; a malicious device on the LAN could replace the root certificate in a man-in-the-middle attack. Install it only on a trusted home or private network, never on a public or shared network. The root CA can issue certificates and its private key stays on this computer; remove it from your phone when no longer needed.", helpCopyLink: '⧉ Copy link', helpCopied: 'Copied ✓', copy: 'Copy', @@ -227,11 +279,26 @@ pcmQueueOverflow: '❌ El búfer de audio está lleno. Inténtalo de nuevo.', micUnknown: '❌ No se pudo iniciar la grabación{name}.', errGeneric: 'Se ha producido un error', + wakeLockLabel: 'Mantener la pantalla encendida al grabar', + wakeLockHint: + 'El bloqueo de pantalla termina esta grabación; el ordenador procesa el audio ya recibido.', + wakeLockActive: 'La pantalla permanece encendida hasta que termine la grabación.', + wakeLockUnavailable: + 'El navegador o el sistema no permitió mantener la pantalla encendida; el audio recibido se procesará igualmente.', + recovering: 'El ordenador está procesando tu última grabación…', + interrupted: 'Grabación interrumpida. El ordenador procesa el audio ya recibido…', + offlineRecording: 'Conexión perdida; el ordenador seguirá procesando la grabación recibida. Reconecta para ver el resultado.', + recovered: 'Última transcripción recuperada', + recoveryRetry: + 'La transcripción falló. La grabación está guardada en el historial del ordenador y puede reintentarse.', + recoveryUnavailable: 'Resultado no disponible. Consulta el historial en el ordenador.', helpTitle: 'Configuración inicial: confiar en este ordenador', + helpVerify: + 'Abre los ajustes de entrada remota en OpenLess del ordenador y mantén visible el SHA-256 de la CA raíz. Compara los 64 caracteres en los detalles del certificado del sistema del teléfono; no uses como prueba un valor de esta página, un nombre de perfil o un identificador. Si difiere o no se puede ver, detente y elimina el perfil. Debe haber exactamente un certificado raíz, sin certificados adicionales, VPN ni ajustes de gestión.', helpAndroid: - 'Android: descarga el certificado CA e instálalo en Ajustes del sistema → Instalar un certificado → Certificado CA, y vuelve aquí. Los nombres de los menús varían según el dispositivo.', + 'Android: descarga el certificado CA y verifica su SHA-256 completo en la vista previa de certificados del sistema antes de instalarlo. Si tu dispositivo no puede mostrar la huella antes de confiar, no lo instales desde esta página; usa un canal de transferencia de archivos ya autenticado. Los nombres de los menús varían según el dispositivo.', helpIos: - 'iPhone / iPad: descarga el perfil e instálalo en Ajustes → General → VPN y gestión de dispositivos; luego activa la confianza completa para OpenLess Remote Input CA en General → Información → Ajustes de confianza de certificados y recarga Safari. Solo hay que hacerlo una vez por ordenador; los certificados de versiones anteriores deben reinstalarse.', + 'iPhone / iPad: descarga el perfil, ábrelo en Ajustes → General → VPN y gestión de dispositivos y revisa su huella en Más detalles → certificado. Solo cuando coincida y no haya ajustes extra, instálalo y activa la confianza completa en General → Información → Ajustes de confianza de certificados. Si los detalles solo se ven tras instalar, deja la confianza completa desactivada hasta verificarlo. Después recarga Safari.', helpDownloadCert: '↓ iPhone: descargar perfil', helpDownloadAndroid: '↓ Android: descargar certificado CA', helpTrustWarning: @@ -291,10 +358,25 @@ micUnknown: '❌ Impossible de démarrer l’enregistrement{name}.', errGeneric: 'Une erreur est survenue', helpTitle: 'Réglage initial : faire confiance à cet ordinateur', + wakeLockLabel: 'Garder l’écran allumé pendant l’enregistrement', + wakeLockHint: + 'Le verrouillage de l’écran met fin à cet enregistrement ; l’ordinateur traite l’audio déjà reçu.', + wakeLockActive: 'L’écran reste allumé jusqu’à la fin de l’enregistrement.', + wakeLockUnavailable: + 'Le navigateur ou le système n’a pas permis de garder l’écran allumé ; l’audio déjà reçu sera quand même traité.', + recovering: 'L’ordinateur traite votre dernier enregistrement…', + interrupted: 'Enregistrement interrompu. L’ordinateur traite l’audio déjà reçu…', + offlineRecording: 'Connexion perdue ; l’ordinateur continuera de traiter l’enregistrement reçu. Reconnectez-vous pour voir le résultat.', + recovered: 'Dernière transcription récupérée', + recoveryRetry: + 'La transcription a échoué. L’enregistrement est conservé dans l’historique de l’ordinateur et peut être relancé.', + recoveryUnavailable: 'Résultat indisponible. Consultez l’historique sur l’ordinateur.', + helpVerify: + 'Ouvrez les réglages de saisie à distance dans OpenLess sur l’ordinateur et gardez visible le SHA-256 de la CA racine. Comparez les 64 caractères dans les détails du certificat du système du téléphone ; n’utilisez jamais comme preuve une valeur de cette page, un nom de profil ou un identifiant. En cas de différence ou si l’affichage est impossible, arrêtez et supprimez le profil. Il ne doit y avoir exactement qu’un certificat racine, sans certificats, VPN ou réglages de gestion supplémentaires.', helpAndroid: - 'Android : téléchargez le certificat CA et installez-le dans Réglages système → Installer un certificat → Certificat CA, puis revenez ici. Les noms des menus varient selon l’appareil.', + 'Android : téléchargez le certificat CA et vérifiez son SHA-256 complet dans l’aperçu des certificats du système avant de l’installer. Si votre appareil ne peut pas afficher l’empreinte avant l’approbation, ne l’installez pas depuis cette page ; utilisez un canal de transfert de fichiers déjà authentifié. Les noms des menus varient selon l’appareil.', helpIos: - 'iPhone / iPad : téléchargez le profil et installez-le dans Réglages → Général → VPN et gestion des appareils ; activez ensuite la confiance complète pour OpenLess Remote Input CA dans Général → Informations → Réglages de confiance des certificats et rechargez Safari. À faire une seule fois par ordinateur ; les certificats des anciennes versions doivent être réinstallés.', + 'iPhone / iPad : téléchargez le profil, ouvrez-le dans Réglages → Général → VPN et gestion des appareils et vérifiez son empreinte dans Plus de détails → certificat. Installez-le et activez la confiance complète dans Général → Informations → Réglages de confiance des certificats uniquement si l’empreinte correspond et sans réglages supplémentaires. Si les détails ne sont visibles qu’après l’installation, laissez la confiance complète désactivée jusqu’à la vérification. Rechargez ensuite Safari.', helpDownloadCert: '↓ iPhone : télécharger le profil', helpDownloadAndroid: '↓ Android : télécharger le certificat CA', helpTrustWarning: @@ -353,11 +435,26 @@ pcmQueueOverflow: '❌ Der Audiopuffer ist voll. Versuche es erneut.', micUnknown: '❌ Die Aufnahme konnte nicht gestartet werden{name}.', errGeneric: 'Ein Fehler ist aufgetreten', + wakeLockLabel: 'Bildschirm bei Aufnahme eingeschaltet lassen', + wakeLockHint: + 'Der Sperrbildschirm beendet diese Aufnahme; der Computer verarbeitet die bereits empfangenen Daten.', + wakeLockActive: 'Der Bildschirm bleibt eingeschaltet, bis die Aufnahme endet.', + wakeLockUnavailable: + 'Browser oder System erlaubten das Wachhalten nicht; die empfangenen Audiodaten werden dennoch verarbeitet.', + recovering: 'Der Computer verarbeitet deine letzte Aufnahme…', + interrupted: 'Aufnahme unterbrochen. Der Computer verarbeitet die bereits empfangenen Daten…', + offlineRecording: 'Verbindung getrennt; der Computer verarbeitet die empfangene Aufnahme weiter. Zum Ansehen des Ergebnisses neu verbinden.', + recovered: 'Letzte Transkription wiederhergestellt', + recoveryRetry: + 'Die Transkription ist fehlgeschlagen. Die Aufnahme liegt im Verlauf des Computers und kann erneut versucht werden.', + recoveryUnavailable: 'Ergebnis nicht verfügbar. Prüfe den Verlauf am Computer.', helpTitle: 'Erstmalige Einrichtung: diesem Computer vertrauen', + helpVerify: + 'Öffne die Ferneingabe-Einstellungen in OpenLess am Computer und halte den SHA-256 der Root-CA sichtbar. Vergleiche alle 64 Zeichen in den Zertifikatdetails des Telefons; verwende niemals einen Wert dieser Seite, einen Profilnamen oder Bezeichner als Nachweis. Bei Abweichung oder wenn nichts angezeigt werden kann, brich ab und entferne das Profil. Es darf genau ein Root-Zertifikat enthalten sein, ohne zusätzliche Zertifikate, VPN- oder Verwaltungsprofile.', helpAndroid: - 'Android: Lade das CA-Zertifikat herunter und installiere es unter Systemeinstellungen → Zertifikat installieren → CA-Zertifikat, kehre dann hierher zurück. Die Menünamen unterscheiden sich je nach Gerät.', + 'Android: Lade das CA-Zertifikat herunter und prüfe vor der Installation den vollständigen SHA-256 in der System-Zertifikatsvorschau. Kann dein Gerät den Fingerabdruck vor dem Vertrauen nicht anzeigen, installiere nichts von dieser Seite; nutze stattdessen einen bereits authentifizierten Übertragungsweg. Die Menünamen unterscheiden sich je nach Gerät.', helpIos: - 'iPhone / iPad: Lade das Konfigurationsprofil herunter und installiere es unter Einstellungen → Allgemein → VPN & Geräteverwaltung; aktiviere danach die volle Vertrauensstellung für OpenLess Remote Input CA unter Allgemein → Info → Zertifikatsvertrauenseinstellungen und lade Safari neu. Nur einmal pro Computer nötig; Zertifikate älterer Versionen müssen neu installiert werden.', + 'iPhone / iPad: Lade das Konfigurationsprofil herunter, öffne es unter Einstellungen → Allgemein → VPN & Geräteverwaltung und prüfe seinen Fingerabdruck unter „Mehr Details“ → Zertifikat. Installiere es und aktiviere die volle Vertrauensstellung unter Allgemein → Info → Zertifikatsvertrauenseinstellungen erst, wenn er übereinstimmt und keine zusätzlichen Einstellungen vorhanden sind. Sind Details erst nach der Installation sichtbar, lasse die volle Vertrauensstellung bis zur Prüfung aus. Lade Safari danach neu.', helpDownloadCert: '↓ iPhone: Profil herunterladen', helpDownloadAndroid: '↓ Android: CA-Zertifikat herunterladen', helpTrustWarning: @@ -368,6 +465,19 @@ copied: 'Kopiert ✓', }, ja: { + wakeLockLabel: '録音中は画面をオンにする', + wakeLockHint: '画面をロックすると録音を終了し、受信済みの音声をパソコンで処理します。', + wakeLockActive: '録音が終わるまで画面をオンに保ちます。', + wakeLockUnavailable: + 'ブラウザーまたはシステムが画面の維持を許可しませんでした。受信済みの音声は処理されます。', + interrupted: '録音が中断されました。受信済みの音声をパソコンで処理しています…', + offlineRecording: + '接続が切れました。受信済みの音声の処理は続きます。再接続すると結果を確認できます。', + recovering: '前回の録音をパソコンで処理しています…', + recovered: '前回の文字起こし結果を復元しました', + recoveryRetry: + '文字起こしが完了しませんでした。録音はパソコンの履歴に保存され、再試行できます。', + recoveryUnavailable: '結果が見つかりません。パソコンの履歴を確認してください。', title: 'OpenLess リモート入力', brandTitle: 'OpenLess リモート入力', brandSub: 'スマホで録音し、リアルタイムでパソコンに入力', @@ -380,7 +490,8 @@ offlineTitle: '接続が切断されました', offlineSub: 'パソコンとの接続が切断されました。', btnReconnect: '再接続', - certTip: "証明書の警告が表示されたら、アドレスがコンピュータの表示と一致することを確認し、「初回設定:このコンピュータを信頼」の手順でインストールと完全な信頼を行ってください。", + certTip: + '信頼する前に、スマートフォンのシステム証明書詳細にある SHA-256 全体をコンピューターの OpenLess 設定と照合してください。IP や名前だけでは確認できません。', tipToggle: '大きいボタンをタップして録音開始、もう一度タップで終了して認識します。', tipHold: '大きいボタンを長押しして話し、離すと終了して認識します。', labelToggleIdle: 'タップで開始', @@ -413,18 +524,36 @@ pcmQueueOverflow: '❌ 音声バッファがいっぱいです。もう一度お試しください。', micUnknown: '❌ 録音を開始できませんでした{name}。', errGeneric: 'エラーが発生しました', - helpTitle: "初回設定:このコンピュータを信頼", - helpAndroid: "Android:CA 証明書をダウンロードし、システム設定の「証明書のインストール → CA 証明書」でインストールしてから戻ってください。項目名は端末により異なります。", - helpIos: "iPhone / iPad:プロファイルをダウンロードし、「設定 → 一般 → VPN とデバイス管理」でインストールします。次に「一般 → 情報 → 証明書信頼設定」で OpenLess Remote Input CA を完全に信頼し、Safari を再読み込みしてください。各コンピュータで一度だけ必要です。旧バージョンの証明書は一度入れ替えてください。", - helpDownloadCert: "↓ iPhone:プロファイルをダウンロード", - helpDownloadAndroid: "↓ Android:CA をダウンロード", - helpTrustWarning: "初回の証明書ダウンロードではコンピューターの身元を確認できず、LAN 上の悪意あるデバイスが中間者攻撃でルート証明書を置き換える可能性があります。信頼できる家庭内またはプライベートネットワークでのみインストールし、公共または共有ネットワークでは操作しないでください。ルート CA は証明書を発行でき、秘密鍵はこのコンピューターに保存されます。不要になったらスマートフォンから削除してください。", + helpTitle: '初回設定:このコンピュータを信頼', + helpAndroid: + 'Android:CA をダウンロードし、システムの証明書プレビューで SHA-256 全体を確認してからインストールします。信頼する前に指紋を表示できない端末では、このページからインストールせず、既存の認証済みファイル転送手段を使用してください。項目名は端末によって異なります。', + helpVerify: + 'コンピューターの OpenLess でリモート入力設定を開き、ルート CA の SHA-256 を表示したままにします。スマートフォンのシステム証明書詳細で全 64 文字を照合してください。このページ、プロファイル名や識別子の値は証明に使えません。一致しない場合や表示できない場合は中止し、プロファイルを削除してください。含まれるのはルート証明書 1 枚のみで、追加の証明書、VPN、管理設定がないことも確認してください。', + helpIos: + 'iPhone / iPad:プロファイルをダウンロードし、「設定 → 一般 → VPN とデバイス管理」で開き、「詳細情報」の証明書で指紋を照合します。一致し、余分な設定がないことを確認してからインストールし、「一般 → 情報 → 証明書信頼設定」で完全に信頼してください。インストール後にしか詳細を表示できない場合は、確認が終わるまで完全な信頼をオフにしてください。その後 Safari を再読み込みします。', + helpDownloadCert: '↓ iPhone:プロファイルをダウンロード', + helpDownloadAndroid: '↓ Android:CA をダウンロード', + helpTrustWarning: + '初回の証明書ダウンロードではコンピューターの身元を確認できず、LAN 上の悪意あるデバイスが中間者攻撃でルート証明書を置き換える可能性があります。信頼できる家庭内またはプライベートネットワークでのみインストールし、公共または共有ネットワークでは操作しないでください。ルート CA は証明書を発行でき、秘密鍵はこのコンピューターに保存されます。不要になったらスマートフォンから削除してください。', helpCopyLink: '⧉ リンクをコピー', helpCopied: 'コピーしました ✓', copy: 'コピー', copied: 'コピー済み ✓', }, ko: { + wakeLockLabel: '녹음 중 화면 켜짐 유지', + wakeLockHint: '화면을 잠그면 녹음이 끝나고 컴퓨터가 이미 받은 오디오를 처리합니다.', + wakeLockActive: '녹음이 끝날 때까지 화면을 켜진 상태로 유지합니다.', + wakeLockUnavailable: + '브라우저 또는 시스템이 화면 켜짐 유지를 허용하지 않았습니다. 수신한 오디오는 계속 처리됩니다.', + interrupted: '녹음이 중단되었습니다. 컴퓨터가 받은 오디오를 처리하고 있습니다…', + offlineRecording: + '연결이 끊겼습니다. 받은 오디오는 계속 처리됩니다. 다시 연결하면 결과를 볼 수 있습니다.', + recovering: '컴퓨터가 마지막 녹음을 처리하고 있습니다…', + recovered: '마지막 음성 인식 결과를 복구했습니다', + recoveryRetry: + '음성 인식을 완료하지 못했습니다. 녹음은 컴퓨터 기록에 저장되며 다시 시도할 수 있습니다.', + recoveryUnavailable: '결과를 찾을 수 없습니다. 컴퓨터의 기록을 확인해 주세요.', title: 'OpenLess 원격 입력', brandTitle: 'OpenLess 원격 입력', brandSub: '휴대폰으로 녹음하여 실시간으로 컴퓨터에 입력', @@ -437,7 +566,8 @@ offlineTitle: '연결이 끊겼습니다', offlineSub: '컴퓨터와의 연결이 끊겼습니다.', btnReconnect: '다시 연결', - certTip: "인증서 경고가 나타나면 주소가 컴퓨터에 표시된 주소와 일치하는지 확인한 뒤 “최초 설정: 이 컴퓨터 신뢰”에 따라 인증서를 설치하고 완전히 신뢰하세요.", + certTip: + '신뢰하기 전에 휴대폰 시스템의 인증서 상세 정보에 있는 전체 SHA-256을 컴퓨터의 OpenLess 설정과 비교하세요. IP나 이름만 확인해서는 충분하지 않습니다.', tipToggle: '큰 버튼을 탭하여 녹음을 시작하고, 다시 탭하면 종료 후 인식합니다.', tipHold: '큰 버튼을 길게 눌러 말하고, 떼면 종료 후 인식합니다.', labelToggleIdle: '탭하여 시작', @@ -470,12 +600,17 @@ pcmQueueOverflow: '❌ 오디오 버퍼가 가득 찼습니다. 다시 시도하세요.', micUnknown: '❌ 녹음을 시작할 수 없습니다{name}.', errGeneric: '오류가 발생했습니다', - helpTitle: "최초 설정: 이 컴퓨터 신뢰", - helpAndroid: "Android: CA 인증서를 다운로드하고 시스템 설정 → 인증서 설치 → CA 인증서에서 설치한 뒤 돌아오세요. 메뉴 이름은 기기마다 다릅니다.", - helpIos: "iPhone / iPad: 프로파일을 다운로드하고 설정 → 일반 → VPN 및 기기 관리에서 설치하세요. 일반 → 정보 → 인증서 신뢰 설정에서 OpenLess Remote Input CA를 완전히 신뢰한 뒤 Safari를 새로고침하세요. 컴퓨터마다 한 번만 설정하면 됩니다. 이전 버전의 인증서는 한 번 교체해야 합니다.", - helpDownloadCert: "↓ iPhone: 프로파일 다운로드", - helpDownloadAndroid: "↓ Android: CA 다운로드", - helpTrustWarning: "최초 인증서 다운로드에서는 컴퓨터의 신원을 확인할 수 없으며, LAN의 악성 기기가 중간자 공격으로 루트 인증서를 바꿀 수 있습니다. 신뢰할 수 있는 가정용 또는 사설 네트워크에서만 설치하고 공용 또는 공유 네트워크에서는 진행하지 마세요. 루트 CA는 인증서를 발급할 수 있고 개인 키는 이 컴퓨터에 저장됩니다. 더 이상 사용하지 않으면 휴대폰에서 제거하세요.", + helpTitle: '최초 설정: 이 컴퓨터 신뢰', + helpAndroid: + 'Android: CA를 다운로드하고 시스템 인증서 미리보기에서 전체 SHA-256을 확인한 뒤 설치하세요. 신뢰하기 전에 지문을 볼 수 없는 기기에서는 이 페이지에서 설치하지 말고 기존의 인증된 파일 전송 수단을 사용하세요. 메뉴 이름은 기기마다 다릅니다.', + helpVerify: + '컴퓨터의 OpenLess 원격 입력 설정에서 루트 CA SHA-256을 표시해 두세요. 휴대폰 시스템의 인증서 상세 정보에서 64자 전체를 비교하세요. 이 웹 페이지, 프로파일 이름이나 식별자의 값은 증명으로 사용할 수 없습니다. 일치하지 않거나 볼 수 없으면 중단하고 프로파일을 제거하세요. 루트 인증서 한 개만 있고 추가 인증서, VPN 또는 관리 설정이 없는지도 확인하세요.', + helpIos: + 'iPhone / iPad: 프로파일을 다운로드하고 설정 → 일반 → VPN 및 기기 관리에서 여세요. 추가 세부사항의 인증서에서 지문을 확인하세요. 일치하고 추가 설정이 없는 경우에만 설치한 뒤 일반 → 정보 → 인증서 신뢰 설정에서 완전한 신뢰를 켜세요. 설치 후에만 상세 정보를 볼 수 있다면 확인이 끝날 때까지 완전한 신뢰를 꺼 두세요. 이후 Safari를 새로고침하세요.', + helpDownloadCert: '↓ iPhone: 프로파일 다운로드', + helpDownloadAndroid: '↓ Android: CA 다운로드', + helpTrustWarning: + '최초 인증서 다운로드에서는 컴퓨터의 신원을 확인할 수 없으며, LAN의 악성 기기가 중간자 공격으로 루트 인증서를 바꿀 수 있습니다. 신뢰할 수 있는 가정용 또는 사설 네트워크에서만 설치하고 공용 또는 공유 네트워크에서는 진행하지 마세요. 루트 CA는 인증서를 발급할 수 있고 개인 키는 이 컴퓨터에 저장됩니다. 더 이상 사용하지 않으면 휴대폰에서 제거하세요.', helpCopyLink: '⧉ 링크 복사', helpCopied: '복사됨 ✓', copy: '복사', @@ -528,6 +663,8 @@ var MODE_KEY = 'ol_remote_mode'; // localStorage 键:录音方式 var PIN_KEY = 'ol_remote_pin'; // localStorage 键:上次成功的配对码 var INSERT_KEY = 'ol_remote_insert'; // localStorage 键:电脑落字开关(默认开) + var WAKE_LOCK_KEY = 'ol_remote_wake_lock'; + var RECOVERY_KEY = 'ol_remote_recovery_session'; var MIC_PREP_TIMEOUT_MS = 10000; // 麦克风准备超时:超过则判失败让用户重试,避免无限卡"准备中" var PCM_QUEUE_MAX_BYTES = 128 * 1024; @@ -556,6 +693,8 @@ var recTip = $('rec-tip'); var modeSwitch = $('mode-switch'); var insertSwitch = $('insert-switch'); + var wakeLockSwitch = $('wake-lock-switch'); + var wakeLockHint = $('wake-lock-hint'); var btnReconnect = $('btn-reconnect'); var offlineReason = $('offline-reason'); @@ -574,6 +713,14 @@ var finishAfterStarted = ''; // ACK 前松手/取消:'stop' | 'cancel' | '' var pendingPcm = []; var pendingPcmBytes = 0; + var awaitingResult = false; + var savedRecovery = readRecoverySession(); + var recoverySessionId = savedRecovery.sessionId; + var recoveryKey = savedRecovery.key; + var recoveryTimer = null; + var wakeLock = null; + var wakeLockGeneration = 0; + var wakeLockPending = null; // 音频相关 var audioCtx = null; @@ -610,6 +757,132 @@ try { localStorage.removeItem(PIN_KEY); } catch (e) {} + saveRecoverySession(''); + } + + // 恢复凭据仅用于本次随机会话;不请求电脑历史记录列表。 + function readRecoverySession() { + try { + var saved = JSON.parse(localStorage.getItem(RECOVERY_KEY) || 'null'); + if (saved && validSessionId(saved.sessionId) && validSessionId(saved.key)) return saved; + } catch (e) {} + return { sessionId: '', key: '' }; + } + function validSessionId(id) { + return ( + typeof id === 'string' && + /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(id) + ); + } + function saveRecoverySession(id, key) { + recoverySessionId = validSessionId(id) && validSessionId(key) ? id : ''; + recoveryKey = recoverySessionId ? key : ''; + try { + if (recoverySessionId) + localStorage.setItem( + RECOVERY_KEY, + JSON.stringify({ sessionId: recoverySessionId, key: recoveryKey }), + ); + else localStorage.removeItem(RECOVERY_KEY); + } catch (e) {} + } + function clearRecoveryTimer() { + if (recoveryTimer) { + clearTimeout(recoveryTimer); + recoveryTimer = null; + } + } + function requestRecovery() { + clearRecoveryTimer(); + if (!authed || document.hidden || recording || startSent || !recoverySessionId) return; + wsSendJSON({ type: 'recover', sessionId: recoverySessionId, recoveryKey: recoveryKey }); + // 唤醒后的旧连接可能仍显示 OPEN,却再也收不到数据;超时重新认证。 + recoveryTimer = setTimeout(function () { + recoveryTimer = null; + if (!recording && authed && !document.hidden) { + var pin = readPin(); + if (pin) connect(pin); + } + }, 8000); + } + function handleRecovery(msg) { + if (msg.sessionId !== recoverySessionId || recording || startSent) return; + clearRecoveryTimer(); + clearWorkTimeout(); + var recovery = msg.recovery || {}; + awaitingResult = recovery.kind === 'pending'; + updateRecordBtnUI(); + if (recovery.kind === 'pending') { + setStatus(L.recovering, 'work'); + if (!document.hidden) recoveryTimer = setTimeout(requestRecovery, 1500); + } else if (recovery.kind === 'completed') { + showResult(recovery.text); + setStatus(L.recovered, 'ok'); + } else if (recovery.kind === 'failed') { + setStatus(recovery.hasAudioRecording ? L.recoveryRetry : L.recoveryUnavailable, 'error'); + } else { + saveRecoverySession(''); + setStatus(L.recoveryUnavailable, 'error'); + } + } + + function shouldKeepAwake() { + return recording && !document.hidden && wakeLockSwitch && wakeLockSwitch.checked; + } + function releaseWakeLock() { + wakeLockGeneration++; + wakeLockPending = null; + var previous = wakeLock; + wakeLock = null; + if (previous) previous.release().catch(function () {}); + } + function acquireWakeLock() { + if (!shouldKeepAwake() || wakeLock || wakeLockPending !== null) return; + if (!navigator.wakeLock || !navigator.wakeLock.request) { + if (wakeLockHint) wakeLockHint.textContent = L.wakeLockUnavailable; + return; + } + var generation = wakeLockGeneration; + wakeLockPending = generation; + navigator.wakeLock + .request('screen') + .then(function (sentinel) { + if (wakeLockPending === generation) wakeLockPending = null; + if (generation !== wakeLockGeneration || !shouldKeepAwake()) { + sentinel.release().catch(function () {}); + return; + } + wakeLock = sentinel; + if (wakeLockHint) wakeLockHint.textContent = L.wakeLockActive; + sentinel.addEventListener('release', function () { + if (wakeLock !== sentinel) return; + wakeLock = null; + if (shouldKeepAwake() && wakeLockHint) wakeLockHint.textContent = L.wakeLockUnavailable; + }); + }) + .catch(function () { + if (wakeLockPending === generation) wakeLockPending = null; + if (generation === wakeLockGeneration && shouldKeepAwake() && wakeLockHint) { + wakeLockHint.textContent = L.wakeLockUnavailable; + } + }); + } + function initWakeLockSwitch() { + if (!wakeLockSwitch) return; + try { + wakeLockSwitch.checked = localStorage.getItem(WAKE_LOCK_KEY) !== '0'; + } catch (e) { + wakeLockSwitch.checked = true; + } + if (wakeLockHint) wakeLockHint.textContent = L.wakeLockHint; + wakeLockSwitch.addEventListener('change', function () { + try { + localStorage.setItem(WAKE_LOCK_KEY, wakeLockSwitch.checked ? '1' : '0'); + } catch (e) {} + if (wakeLockHint) wakeLockHint.textContent = L.wakeLockHint; + if (wakeLockSwitch.checked) acquireWakeLock(); + else releaseWakeLock(); + }); } // ============================================================ @@ -771,7 +1044,10 @@ workTimer = null; if (!recording && authed) { if (startSent) failRecording('❌ ' + L.errGeneric, true); + else if (recoverySessionId) requestRecovery(); else { + awaitingResult = false; + updateRecordBtnUI(); setStatus('❌ ' + L.errGeneric, 'error'); setLevel(0); } @@ -825,6 +1101,7 @@ closeWS(); // 清理旧连接 authed = false; busy = false; + awaitingResult = false; var url = 'wss://' + location.host + '/ws'; try { @@ -861,6 +1138,7 @@ clearConnectTimeout(); clearReadyTimer(); clearWorkTimeout(); + clearRecoveryTimer(); if (busyTimer) { clearTimeout(busyTimer); busyTimer = null; @@ -868,12 +1146,13 @@ var wasAuthed = authed; authed = false; recording = false; + awaitingResult = false; detachHoldEnd(); resetRemoteStreamState(); teardownAudio(); if (wasAuthed) { // 已进入录音屏后断开 → 断线屏 - offlineReason.textContent = L.offlineSub; + offlineReason.textContent = recoverySessionId ? L.offlineRecording : L.offlineSub; showScreen('offline'); } else { // 未认证就关闭(握手被拒/证书不受信任/网络中断)。无论当前是否在配对屏都给出 @@ -890,11 +1169,13 @@ clearConnectTimeout(); clearReadyTimer(); clearWorkTimeout(); + clearRecoveryTimer(); if (busyTimer) { clearTimeout(busyTimer); busyTimer = null; } recording = false; + awaitingResult = false; detachHoldEnd(); resetRemoteStreamState(); teardownAudio(); @@ -918,6 +1199,7 @@ clearConnectTimeout(); writePin(lastPin); // 配对成功 → 记住配对码,刷新后免重输 enterRecScreen(); + requestRecovery(); } else { authed = false; clearPin(); // 配对码失效(错误/锁定)→ 清除,避免下次自动重连又失败 @@ -934,7 +1216,11 @@ break; case 'started': - handleStarted(msg.sessionId); + handleStarted(msg.sessionId, msg.recoveryKey); + break; + + case 'recovery': + handleRecovery(msg); break; case 'level': @@ -945,6 +1231,7 @@ clearWorkTimeout(); busy = true; recording = false; + awaitingResult = false; resetRemoteStreamState(); teardownAudioCapture(); // 停止采集但保留 ctx updateRecordBtnUI(); @@ -962,6 +1249,9 @@ case 'result': // 电脑落字完成后回传的最终文字,显示给手机用户看本次识别结果。 showResult(msg.text); + awaitingResult = false; + clearRecoveryTimer(); + updateRecordBtnUI(); break; } } @@ -979,6 +1269,14 @@ setStatus(stripLeadingIcon(L.statusRecording), 'work'); break; case 'transcribing': + if (recording) { + recording = false; + detachHoldEnd(); + resetRemoteStreamState(); + teardownAudioCapture(); + } + awaitingResult = true; + updateRecordBtnUI(); setStatus(stripLeadingIcon(L.statusTranscribing), 'work'); if (statusDots) statusDots.hidden = false; // 识别中:三点加载动效 armWorkTimeout(); // 工作状态续上兜底超时,防止服务端中途无响应卡死 @@ -988,6 +1286,8 @@ armWorkTimeout(); // 同上 break; case 'done': + awaitingResult = false; + updateRecordBtnUI(); clearWorkTimeout(); // 正常收尾,解除兜底超时 var n = typeof msg.insertedChars === 'number' ? msg.insertedChars : 0; setStatus(stripLeadingIcon(fmt(L.statusDone, { n: n })), 'ok'); @@ -999,6 +1299,8 @@ scheduleReady(); break; case 'error': + awaitingResult = false; + updateRecordBtnUI(); clearWorkTimeout(); // 服务端已明确报错,解除兜底超时 if (recording || startSent) failRecording('❌ ' + (msg.message || L.errGeneric), true); else { @@ -1143,7 +1445,8 @@ // ============================================================ function updateRecordBtnUI() { recordBtn.classList.toggle('recording', recording); - recordBtn.classList.toggle('busy', busy && !recording); + recordBtn.classList.toggle('busy', (busy || awaitingResult) && !recording); + recordBtn.disabled = (busy || awaitingResult) && !recording; if (recording) { recordLabel.textContent = mode === 'hold' ? L.labelHoldRec : L.labelToggleRec; } else { @@ -1154,7 +1457,7 @@ // toggle 模式:click 切换 recordBtn.addEventListener('click', function () { if (mode !== 'toggle') return; - if (!authed || busy) return; + if (!authed || busy || awaitingResult) return; if (recording) stopRecording(); else startRecording(); }); @@ -1184,7 +1487,7 @@ recordBtn.addEventListener('pointerdown', function (e) { if (mode !== 'hold') return; - if (!authed || busy) return; + if (!authed || busy || awaitingResult) return; e.preventDefault(); attachHoldEnd(); if (!recording) startRecording(); @@ -1218,13 +1521,15 @@ } function startRecording() { - if (recording || startSent) return; + if (recording || startSent || awaitingResult) return; if (!ws || ws.readyState !== 1) { setStatus(L.connLost, 'error'); return; } // 先乐观置态,保证 iOS 在手势同步栈内 resume() recording = true; + clearRecoveryTimer(); + acquireWakeLock(); resetRemoteStreamState(); clearReadyTimer(); // 防止上一次 done 的回 ready 定时器迟到覆盖本次状态 clearWorkTimeout(); // 新一次录音开始,作废上一轮的识别兜底超时 @@ -1270,6 +1575,8 @@ setLevel(0); return; } + awaitingResult = true; + updateRecordBtnUI(); if (remoteSessionId) { wsSendJSON({ type: 'stop' }); resetRemoteStreamState(); @@ -1285,6 +1592,9 @@ function cancelRecording() { detachHoldEnd(); + awaitingResult = false; + clearRecoveryTimer(); + saveRecoverySession(''); if (!recording && !startSent) { teardownAudioCapture(); resetRemoteStreamState(); @@ -1341,6 +1651,11 @@ return Promise.reject(new Error('UNSUPPORTED:浏览器不支持录音,请升级或换浏览器')); } audioCtx = new AC(); + audioCtx.onstatechange = function () { + if (audioCtx && recording && startSent && audioCtx.state !== 'running') { + interruptRecording(); + } + }; } // 注意:iOS Safari 来电/Siri 后 ctx 处于私有的 'interrupted' 状态,只判 'suspended' @@ -1375,6 +1690,11 @@ return null; // 交给下一步判空直接放弃 } mediaStream = stream; + stream.getTracks().forEach(function (track) { + track.onended = function () { + if (recording) interruptRecording(); + }; + }); return stream; }); }) @@ -1577,6 +1897,7 @@ function failRecording(message, notifyBackend) { var waitingForAck = startSent && !remoteSessionId; recording = false; + awaitingResult = false; detachHoldEnd(); teardownAudioCapture(); if (notifyBackend && startSent) wsSendJSON({ type: 'cancel' }); @@ -1611,7 +1932,7 @@ return true; } - function handleStarted(sessionId) { + function handleStarted(sessionId, key) { if (!startSent) { clearPendingPcm(); return; @@ -1633,9 +1954,10 @@ } remoteSessionId = sessionId; + saveRecoverySession(sessionId, key); remoteSequence = 0; if (!flushPendingPcm()) { - failRecording(L.connLost, true); + interruptRecording(); return; } if (finishAfterStarted === 'stop') { @@ -1654,11 +1976,11 @@ function sendAudio(buf) { if (!recording || !buf || !buf.byteLength) return; if (!ws || ws.readyState !== 1) { - failRecording(L.connLost, false); + interruptRecording(); return; } if (remoteSessionId) { - if (!sendRemoteFrame(buf)) failRecording(L.connLost, true); + if (!sendRemoteFrame(buf)) interruptRecording(); else updateLocalLevel(buf); return; } @@ -1713,6 +2035,8 @@ // ============================================================ // 仅停止"采集/推流"(断开节点),保留 audioCtx & mediaStream 以便快速重启。 function teardownAudioCapture() { + releaseWakeLock(); + if (wakeLockHint) wakeLockHint.textContent = L.wakeLockHint; try { if (workletNode) { workletNode.port.onmessage = null; @@ -1787,12 +2111,40 @@ } // ============================================================ - // 页面可见性:切后台时若在 hold 录音则取消,避免半截音频 + // 息屏和切后台结束本段录音,保留电脑已收到的部分。 // ============================================================ - document.addEventListener('visibilitychange', function () { - if (document.hidden && recording) { - cancelRecording(); + function interruptRecording() { + var hadStarted = startSent; + if (recording) stopRecording(); + else if (startSent && remoteSessionId) { + wsSendJSON({ type: 'stop' }); + resetRemoteStreamState(); + awaitingResult = true; + teardownAudioCapture(); + updateRecordBtnUI(); } + // 系统中断后释放旧轨道,下一次由用户开始录音时重新获取麦克风。 + teardownAudio(); + if (hadStarted) setStatus(L.interrupted, 'work'); + } + document.addEventListener('visibilitychange', function () { + if (document.hidden) { + if (recording) interruptRecording(); + releaseWakeLock(); + clearRecoveryTimer(); + clearWorkTimeout(); + } else { + if (recording) acquireWakeLock(); + if (authed) requestRecovery(); + else if (!ws || ws.readyState > 1) { + var pin = readPin(); + if (pin) connect(pin); + } + } + }); + window.addEventListener('pagehide', function () { + if (recording) interruptRecording(); + releaseWakeLock(); }); // ============================================================ @@ -1825,6 +2177,7 @@ applyStaticI18n(); syncModeUI(); initInsertSwitch(); + initWakeLockSwitch(); showScreen('pin'); showPinError(''); // 上次成功的配对码 → 自动填充并重连,刷新/重开页面免再输一次 diff --git a/openless-all/app/src-tauri/src/remote_server/assets/index.html b/openless-all/app/src-tauri/src/remote_server/assets/index.html index 16d044ed..d9add7ca 100644 --- a/openless-all/app/src-tauri/src/remote_server/assets/index.html +++ b/openless-all/app/src-tauri/src/remote_server/assets/index.html @@ -45,20 +45,28 @@ + 重启 / 换 IP 都不再失效(持久本地 CA,见 tls_identity.rs)。安装前必须 + 在手机系统证书详情里核对电脑端显示的 SHA-256 指纹。 -->
首次设置:信任此电脑 +

+ 先打开电脑上的 OpenLess 远程输入设置,保留「本机根证书 + SHA-256」。在手机系统证书详情中核对全部 64 + 个字符;不要使用本网页、描述文件名称或标识里的值作为证明。不一致或无法查看时,请停止并移除描述文件。描述文件必须只含一张根证书,不得有其他证书、VPN + 或管理配置。 +

- iPhone / iPad:下载描述文件,打开“设置 → 通用 → VPN 与设备管理”安装;再到“通用 → 关于本机 → - 证书信任设置”,为 OpenLess Remote Input CA 打开“完全信任”,返回 Safari - 刷新。每台电脑只需设置一次;旧版证书需要重新安装。 + iPhone / iPad:下载描述文件,在“设置 → 通用 → VPN 与设备管理”中打开它,选择“更多详细信息”中的证书并核对指纹。确认一致且没有额外配置后再安装,并到“通用 + → 关于本机 → 证书信任设置”开启完全信任。若只能在安装后查看详情,先保持完全信任关闭,核对后再开启。完成后返回 + Safari 刷新。

- 安卓:下载 CA 证书,在系统设置的“安装证书 → CA 证书”中安装,再回到本页。菜单名称因设备而异。 + 安卓:下载 CA + 证书,在系统证书预览中核对完整 SHA-256 + 后再安装。若系统无法在信任前显示指纹,请勿从此页面安装,改用已有的可信文件传输渠道。菜单名称因设备而异。

- 只安装你自己电脑提供的 OpenLess - 根证书。它具备签发证书的能力,私钥保存在这台电脑;不再使用时请从手机移除。 + 首次证书下载无法验证电脑身份,恶意局域网设备可能通过中间人攻击替换根证书。仅在可信的家庭或私人网络中安装,勿在公共或共享网络操作。根证书具备签发能力,私钥保存在这台电脑;不再使用时请从手机移除。

@@ -124,6 +132,20 @@

点击大按钮开始录音,再次点击结束并识别。

+ + +

+