This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections. This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work. ## Validation - Full local Web check: 79 files / 1,317 tests, TypeScript and production build. - Full Go checks, platform builds and Windows native capture regressions/probe. - Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup. - Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI. - Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved. ## Release notes ### 中文 这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。 #### 新增与改进 - 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。 - 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。 - 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。 - 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。 - 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。 - App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。 #### 修复 - 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。 - 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。 - 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。 - 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。 - 无站点密码时,连接恢复不再误引导用户输入站点密码。 - 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。 - 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。 #### 使用说明 下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。 ### English This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged. #### New and improved - Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs. - Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them. - Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date. - Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts. - Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues. - The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source. #### Fixes - Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling. - Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session. - Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections. - Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests. - Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase. - Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip. - Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449). #### Updating Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome. ## Attribution Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
251 lines
12 KiB
TypeScript
251 lines
12 KiB
TypeScript
import { afterEach, expect, test, vi } from "vitest";
|
|
import { HostAudio } from "../src/client/media/host-audio";
|
|
import { captureBrowserSource, QUALITY_PROFILES } from "../src/client/media/quality";
|
|
|
|
class Track {
|
|
enabled = true;
|
|
readyState = "live";
|
|
onended: (() => void) | null = null;
|
|
constructor(readonly kind: string) {}
|
|
stop = vi.fn(() => { this.readyState = "ended"; });
|
|
}
|
|
class Stream {
|
|
constructor(private tracks: Track[]) {}
|
|
getTracks() { return this.tracks; }
|
|
getAudioTracks() { return this.tracks.filter((track) => track.kind === "audio"); }
|
|
getVideoTracks() { return this.tracks.filter((track) => track.kind === "video"); }
|
|
}
|
|
const media = (...tracks: Track[]) => new Stream(tracks) as unknown as MediaStream;
|
|
|
|
function setup() {
|
|
const contexts: Context[] = [];
|
|
class Context {
|
|
output = new Stream([new Track("audio")]);
|
|
inputs: Stream[] = [];
|
|
nodes: { connect: ReturnType<typeof vi.fn>; disconnect: ReturnType<typeof vi.fn> }[] = [];
|
|
currentTime = 0;
|
|
gain = { gain: { value: 1, setTargetAtTime: vi.fn() }, connect: vi.fn(), disconnect: vi.fn() };
|
|
constructor() { contexts.push(this); }
|
|
createMediaStreamDestination() { return { stream: this.output }; }
|
|
createGain() { return this.gain; }
|
|
createMediaStreamSource(stream: Stream) {
|
|
this.inputs.push(stream);
|
|
const node = { connect: vi.fn(), disconnect: vi.fn() };
|
|
this.nodes.push(node);
|
|
return node;
|
|
}
|
|
resume = vi.fn(async () => {});
|
|
close = vi.fn(async () => {});
|
|
}
|
|
vi.stubGlobal("MediaStream", Stream);
|
|
vi.stubGlobal("AudioContext", Context);
|
|
const getUserMedia = vi.fn();
|
|
vi.stubGlobal("navigator", { mediaDevices: { getUserMedia } });
|
|
return { contexts, getUserMedia };
|
|
}
|
|
afterEach(() => vi.unstubAllGlobals());
|
|
|
|
test("camera is a bounded video source and never implicitly requests a microphone", async () => {
|
|
const { getUserMedia } = setup();
|
|
const video = new Track("video");
|
|
const source = media(video);
|
|
getUserMedia.mockResolvedValue(source);
|
|
expect(await captureBrowserSource(QUALITY_PROFILES["1080p30"], "camera")).toBe(source);
|
|
expect(getUserMedia).toHaveBeenCalledWith({ audio: false, video: {
|
|
facingMode: { ideal: "environment" }, width: { ideal: 1920 }, height: { ideal: 1080 },
|
|
frameRate: { ideal: 30, max: 30 },
|
|
} });
|
|
});
|
|
|
|
test("mixing and source changes retain one audio track, microphone intent and the capture-owned video", async () => {
|
|
const { contexts, getUserMedia } = setup();
|
|
const video = new Track("video"), sound = new Track("audio"), microphone = new Track("audio");
|
|
const change = vi.fn();
|
|
const audio = new HostAudio(media(video, sound), change);
|
|
getUserMedia.mockResolvedValue(media(microphone));
|
|
const mixed = (await audio.toggleMicrophone())!;
|
|
expect(mixed.getVideoTracks()).toEqual([video]);
|
|
expect(mixed.getAudioTracks()).toHaveLength(1);
|
|
await audio.toggleMicrophone();
|
|
expect(microphone.enabled).toBe(false);
|
|
expect(change).toHaveBeenLastCalledWith(false);
|
|
const camera = new Track("video");
|
|
const replaced = audio.attach(media(camera));
|
|
expect(audio.sourceStream.getAudioTracks()).toHaveLength(0);
|
|
expect(replaced.getAudioTracks()).toEqual(mixed.getAudioTracks());
|
|
expect(replaced.getVideoTracks()).toEqual([camera]);
|
|
expect(sound.stop).toHaveBeenCalledOnce();
|
|
expect(video.stop).not.toHaveBeenCalled(); // Host owns picture retirement.
|
|
await audio.toggleMicrophone();
|
|
microphone.stop(); microphone.onended!();
|
|
expect(getUserMedia).toHaveBeenCalledOnce(); // Mute/unmute must not open a second microphone.
|
|
expect(change).toHaveBeenLastCalledWith(false);
|
|
expect(mixed.getAudioTracks()[0].readyState).toBe("live");
|
|
expect(camera.stop).not.toHaveBeenCalled();
|
|
audio.dispose();
|
|
expect(contexts[0].close).toHaveBeenCalledOnce();
|
|
expect(mixed.getAudioTracks()[0].readyState).toBe("ended");
|
|
});
|
|
|
|
test("source audio facts are independent of mixed voice, mute, volume and share pause", async () => {
|
|
const { getUserMedia } = setup();
|
|
const camera = new Track("video"), microphone = new Track("audio");
|
|
const source = media(camera);
|
|
const audio = new HostAudio(source, vi.fn());
|
|
getUserMedia.mockResolvedValue(media(microphone));
|
|
const mixed = (await audio.toggleMicrophone())!;
|
|
expect(mixed.getAudioTracks()).toHaveLength(1);
|
|
expect(audio.sourceStream).toBe(source);
|
|
expect(audio.sourceStream.getAudioTracks()).toHaveLength(0);
|
|
await audio.toggleMicrophone();
|
|
audio.setMicrophoneVolume(0);
|
|
mixed.getTracks().forEach(track => { track.enabled = false; });
|
|
expect(audio.sourceStream.getAudioTracks()).toHaveLength(0);
|
|
const sourceSound = new Track("audio");
|
|
sourceSound.enabled = false;
|
|
const nextSource = media(new Track("video"), sourceSound);
|
|
audio.attach(nextSource);
|
|
expect(audio.sourceStream).toBe(nextSource);
|
|
expect(audio.sourceStream.getAudioTracks()).toHaveLength(1);
|
|
expect(microphone.enabled).toBe(false);
|
|
audio.dispose();
|
|
expect(sourceSound.stop).toHaveBeenCalledOnce();
|
|
});
|
|
|
|
test("microphone denial leaves source audio/video alone; late permission cannot revive a retired share", async () => {
|
|
const { contexts, getUserMedia } = setup();
|
|
const video = new Track("video"), sound = new Track("audio");
|
|
const changed = vi.fn();
|
|
const audio = new HostAudio(media(video, sound), changed);
|
|
getUserMedia.mockRejectedValueOnce(new DOMException("Denied", "NotAllowedError"));
|
|
await expect(audio.toggleMicrophone()).rejects.toThrow("Denied");
|
|
expect(contexts[0].close).toHaveBeenCalledOnce();
|
|
expect(video.stop).not.toHaveBeenCalled();
|
|
expect(sound.stop).not.toHaveBeenCalled();
|
|
let grant!: (stream: MediaStream) => void;
|
|
getUserMedia.mockImplementationOnce(() => new Promise<MediaStream>((resolve) => { grant = resolve; }));
|
|
const request = audio.toggleMicrophone();
|
|
audio.dispose();
|
|
const lateMicrophone = new Track("audio");
|
|
grant(media(lateMicrophone));
|
|
expect(await request).toBeNull();
|
|
expect(lateMicrophone.stop).toHaveBeenCalledOnce();
|
|
expect(changed).not.toHaveBeenCalled();
|
|
expect(contexts[1].close).toHaveBeenCalledOnce();
|
|
});
|
|
|
|
test("microphone volume changes only the microphone input without replacing the shared output", async () => {
|
|
const { contexts, getUserMedia } = setup();
|
|
const sound = new Track("audio"), microphone = new Track("audio");
|
|
const audio = new HostAudio(media(new Track("video"), sound), vi.fn());
|
|
audio.setMicrophoneVolume(0.5);
|
|
getUserMedia.mockResolvedValue(media(microphone));
|
|
const mixed = (await audio.toggleMicrophone())!;
|
|
const context = contexts[0];
|
|
expect(context.gain.gain.value).toBe(0.5);
|
|
expect(context.nodes[0].connect).not.toHaveBeenCalledWith(context.gain);
|
|
expect(context.nodes[1].connect).toHaveBeenCalledWith(context.gain);
|
|
audio.setMicrophoneVolume(1.5);
|
|
expect(context.gain.gain.setTargetAtTime).toHaveBeenLastCalledWith(1.5, 0, 0.01);
|
|
audio.setMicrophoneVolume(Number.NaN);
|
|
expect(context.gain.gain.setTargetAtTime).toHaveBeenCalledOnce();
|
|
await audio.toggleMicrophone();
|
|
const changed = audio.attach(media(new Track("video")));
|
|
expect(changed.getAudioTracks()).toEqual(mixed.getAudioTracks());
|
|
expect(microphone.enabled).toBe(false);
|
|
expect(getUserMedia).toHaveBeenCalledOnce();
|
|
audio.dispose();
|
|
expect(context.gain.disconnect).toHaveBeenCalledOnce();
|
|
});
|
|
|
|
test("switching microphone preserves output and leaves the old input on failure", async () => {
|
|
const { getUserMedia } = setup();
|
|
const video = new Track("video"), first = new Track("audio"), second = new Track("audio");
|
|
const audio = new HostAudio(media(video), vi.fn());
|
|
getUserMedia.mockResolvedValueOnce(media(first));
|
|
const output = (await audio.setMicrophone(true, "first"))!;
|
|
getUserMedia.mockRejectedValueOnce(new DOMException("missing", "NotFoundError"));
|
|
await expect(audio.setMicrophone(true, "missing")).rejects.toThrow("missing");
|
|
expect(first.stop).not.toHaveBeenCalled();
|
|
expect(video.stop).not.toHaveBeenCalled();
|
|
getUserMedia.mockResolvedValueOnce(media(second));
|
|
const replacement = await audio.setMicrophone(true, "second");
|
|
expect(getUserMedia).toHaveBeenLastCalledWith({ audio: expect.objectContaining({ deviceId: { exact: "second" } }) });
|
|
expect(replacement).toBeNull(); // No transport update for an input-only change.
|
|
expect(output.getAudioTracks()[0].readyState).toBe("live");
|
|
expect(first.stop).toHaveBeenCalledOnce();
|
|
first.onended!(); // Retired input cannot mute its replacement.
|
|
expect(second.enabled).toBe(true);
|
|
await audio.setMicrophone(false, "first");
|
|
expect(second.stop).toHaveBeenCalledOnce();
|
|
expect(getUserMedia).toHaveBeenCalledTimes(3); // Selecting while muted never captures.
|
|
audio.dispose();
|
|
});
|
|
|
|
test("camera selection requests one exact device without microphone permission", async () => {
|
|
const { getUserMedia } = setup();
|
|
getUserMedia.mockResolvedValue(media(new Track("video")));
|
|
await captureBrowserSource(QUALITY_PROFILES["1080p30"], "camera", "front-camera");
|
|
expect(getUserMedia).toHaveBeenCalledWith({ audio: false, video: expect.objectContaining({ deviceId: { exact: "front-camera" } }) });
|
|
expect(getUserMedia.mock.calls[0][0].video).not.toHaveProperty("facingMode");
|
|
});
|
|
|
|
test("voice processing replaces only the input and commits only after successful capture", async () => {
|
|
const { getUserMedia } = setup();
|
|
const video = new Track("video"), first = new Track("audio"), raw = new Track("audio");
|
|
const audio = new HostAudio(media(video), vi.fn(), "camera");
|
|
getUserMedia.mockResolvedValueOnce(media(first));
|
|
const output = (await audio.setMicrophone(true, "virtual-input"))!;
|
|
expect(getUserMedia).toHaveBeenLastCalledWith({ audio: {
|
|
echoCancellation: true, noiseSuppression: true, autoGainControl: true,
|
|
deviceId: { exact: "virtual-input" },
|
|
} });
|
|
getUserMedia.mockRejectedValueOnce(new DOMException("Device busy", "NotReadableError"));
|
|
await expect(audio.setMicrophone(true, "virtual-input", false)).rejects.toThrow("Device busy");
|
|
expect(first.stop).not.toHaveBeenCalled();
|
|
await audio.setMicrophone(true, "virtual-input"); // Failure must retain the previous configuration.
|
|
expect(getUserMedia).toHaveBeenCalledTimes(2);
|
|
|
|
getUserMedia.mockResolvedValueOnce(media(raw));
|
|
expect(await audio.setMicrophone(true, "virtual-input", false)).toBeNull();
|
|
expect(getUserMedia).toHaveBeenLastCalledWith({ audio: {
|
|
echoCancellation: false, noiseSuppression: false, autoGainControl: false,
|
|
voiceIsolation: false, channelCount: { ideal: 2 }, deviceId: { exact: "virtual-input" },
|
|
} });
|
|
expect(first.stop).toHaveBeenCalledOnce();
|
|
expect(video.stop).not.toHaveBeenCalled();
|
|
expect(output.getAudioTracks()[0].readyState).toBe("live");
|
|
expect(audio.attach(media(video)).getAudioTracks()).toEqual(output.getAudioTracks());
|
|
await audio.setMicrophone(false, "virtual-input");
|
|
await audio.toggleMicrophone();
|
|
expect(raw.enabled).toBe(true);
|
|
expect(getUserMedia).toHaveBeenCalledTimes(3); // Mute and source replacement retain processing intent.
|
|
|
|
await audio.setMicrophone(false, "virtual-input", true);
|
|
expect(raw.stop).toHaveBeenCalledOnce();
|
|
expect(getUserMedia).toHaveBeenCalledTimes(3); // A settings change while muted never opens the input.
|
|
getUserMedia.mockResolvedValueOnce(media(new Track("audio")));
|
|
await audio.toggleMicrophone();
|
|
expect(getUserMedia).toHaveBeenLastCalledWith({ audio: expect.objectContaining({ echoCancellation: true }) });
|
|
audio.dispose();
|
|
});
|
|
|
|
test("share retirement also cancels a pending processing change", async () => {
|
|
const { getUserMedia } = setup();
|
|
const first = new Track("audio"), late = new Track("audio");
|
|
const changed = vi.fn();
|
|
const audio = new HostAudio(media(new Track("video")), changed);
|
|
getUserMedia.mockResolvedValueOnce(media(first));
|
|
const output = (await audio.setMicrophone(true, ""))!;
|
|
let complete!: (stream: MediaStream) => void;
|
|
getUserMedia.mockImplementationOnce(() => new Promise<MediaStream>(resolve => { complete = resolve; }));
|
|
const replacing = audio.setMicrophone(true, "", false);
|
|
audio.dispose();
|
|
complete(media(late));
|
|
expect(await replacing).toBeNull();
|
|
expect(first.stop).toHaveBeenCalledOnce();
|
|
expect(late.stop).toHaveBeenCalledOnce();
|
|
expect(output.getAudioTracks()[0].readyState).toBe("ended");
|
|
expect(changed).toHaveBeenCalledOnce();
|
|
});
|