Files
Piik/native/capture/windows/output_worker.test.cpp
AAAandPedro Antonio c18502fd18 feat: improve microphone, chat and platform support
This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections.

This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work.

## Validation

- Full local Web check: 79 files / 1,317 tests, TypeScript and production build.
- Full Go checks, platform builds and Windows native capture regressions/probe.
- Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup.
- Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI.
- Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved.

## Release notes

### 中文

这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。

#### 新增与改进

- 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。
- 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。
- 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。
- 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。
- 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。
- App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。

#### 修复

- 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。
- 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。
- 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。
- 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。
- 无站点密码时,连接恢复不再误引导用户输入站点密码。
- 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。
- 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。

#### 使用说明

下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。

### English

This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged.

#### New and improved

- Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs.
- Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them.
- Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date.
- Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts.
- Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues.
- The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source.

#### Fixes

- Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling.
- Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session.
- Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections.
- Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests.
- Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase.
- Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip.
- Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449).

#### Updating

Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome.

## Attribution

Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
2026-10-04 16:21:01 +08:00

306 lines
13 KiB
C++

// Exercise the production worker with a synthetic codec at its existing link
// boundary. WARP supplies only the device-status API; no screen or audio is used.
#define wmain capture_main
#include "main.cpp"
#undef wmain
#include <cassert>
#include <future>
namespace piik::capture::windows {
class AdaptiveEncoder::Impl final {
public:
explicit Impl(Factory create, VideoProfile profile) : encoder(create(profile)) {}
std::unique_ptr<VideoEncoder> encoder;
};
AdaptiveEncoder::AdaptiveEncoder(OutputKind, VideoProfile profile, ID3D11Device*,
Factory create, std::unique_ptr<VideoEncoder>, int)
: impl_(std::make_unique<Impl>(std::move(create), profile)) {}
AdaptiveEncoder::~AdaptiveEncoder() = default;
std::optional<AdaptiveAccessUnit> AdaptiveEncoder::Encode(FrameProducer,
UINT32 width, UINT32 height, UINT64 timestamp, bool key, UINT32 bitrate,
EncoderClock::time_point deadline) {
impl_->encoder->SetBitrate(bitrate);
return AdaptiveAccessUnit{impl_->encoder->Encode(nullptr, timestamp, key, deadline),
width, height, 333'333};
}
} // namespace piik::capture::windows
class FixtureEncoder final : public VideoEncoder {
public:
FixtureEncoder(std::promise<void>* entered, std::shared_future<void> released)
: VideoEncoder(OutputKind::h264, "fixture", "fixture"),
entered_(entered), released_(std::move(released)) {}
EncodedAccessUnit Encode(ID3D11Texture2D*, UINT64 timestamp, bool key,
EncoderClock::time_point) override {
if (entered_) {
entered_->set_value();
released_.wait();
throw GateFailure("fixture-encode", "synthetic encoder failed", E_FAIL);
}
return {timestamp, key, {1, 2, 3}};
}
void SetBitrate(UINT32) override {}
private:
std::promise<void>* entered_;
std::shared_future<void> released_;
};
void CheckWorkerFailureGeneration(ID3D11Device* device, bool replace) {
std::promise<void> entered, release, output, failure;
auto began = entered.get_future();
auto released = release.get_future().share();
auto delivered = output.get_future();
auto failed = failure.get_future();
std::atomic<unsigned> created = 0;
HANDLE sink = CreateFileW(L"NUL", GENERIC_WRITE, 0, nullptr, OPEN_EXISTING, 0, nullptr);
assert(sink != INVALID_HANDLE_VALUE);
ProtocolWriter writer(sink);
OutputWorker worker(0, VideoProfile{}, OutputKind::h264, device, writer,
[&](const VideoProfile&) -> std::unique_ptr<VideoEncoder> {
return std::make_unique<FixtureEncoder>(created++ == 0 ? &entered : nullptr, released);
}, nullptr, true, [&](const EncodedAccessUnit&) { output.set_value(); },
[&](std::exception_ptr error) {
try { std::rethrow_exception(error); }
catch (const GateFailure& actual) { assert(actual.result() == E_FAIL); }
failure.set_value();
});
auto input = std::make_shared<CaptureInput>();
input->width = 1280;
input->height = 720;
input->timestamp = 333'333;
worker.Submit(input);
assert(began.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
if (replace) {
worker.SetActive(false);
assert(worker.SetActive(true));
worker.Submit(input);
}
release.set_value();
if (replace) {
assert(delivered.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
assert(failed.wait_for(std::chrono::milliseconds(0)) == std::future_status::timeout);
assert(created == 2);
} else {
assert(failed.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
assert(delivered.wait_for(std::chrono::milliseconds(0)) == std::future_status::timeout);
assert(created == 1);
assert(!worker.SetActive(false) && !worker.SetActive(true));
}
worker.Stop();
worker.Join();
CloseHandle(sink);
}
void CheckPrimaryFailureDetails() {
std::ostringstream output;
auto previous = std::cerr.rdbuf(output.rdbuf());
assert(OutputFailureDetail(3, std::make_exception_ptr(
GateFailure("fixture-output", "original cause", E_ACCESSDENIED))) == "fixture-output");
std::cerr.rdbuf(previous);
const auto text = output.str();
assert(text.find("layer=3") != std::string::npos);
assert(text.find("original cause") != std::string::npos);
assert(text.find("hresult=0x80070005") != std::string::npos);
// With no COM apartment, the real decoder fails at its first API boundary.
bool rejected = false;
try { piik::capture::H264Decoder decoder(nullptr); }
catch (const GateFailure& error) {
rejected = error.stage() == "h264-decoder-create" && error.result() == CO_E_NOTINITIALIZED;
}
assert(rejected);
}
void CheckH264ProfileCompatibility() {
const VideoProfile profile;
for (const auto* value : {"42c01f", "42e01f", "42E01F", "4d801f", "58c01f"})
assert(profile.accepts_h264_profile_level_id(value));
for (const auto* value : {"42001f", "4d001f", "64001f", "42e11f", "42e01", "42e01fg",
"42e0gg", "42e034", "42e01e", " 42c01"})
assert(!profile.accepts_h264_profile_level_id(value));
HANDLE read = nullptr, write = nullptr;
assert(CreatePipe(&read, &write, nullptr, 4096));
ProtocolWriter writer(write);
ProductArguments arguments;
arguments.outputs = {profile};
EncodedAccessUnit output{0, true, {0, 0, 0, 1, 0x67, 0x42, 0xe0, 0x1f}};
WriteVideoActive(writer, arguments, true, output);
std::array<char, 4096> bytes{};
DWORD count = 0;
assert(ReadFile(read, bytes.data(), static_cast<DWORD>(bytes.size()), &count, nullptr));
const std::string status(bytes.data(), count);
assert(status.find("\"profileLevelId\":\"42e01f\"") != std::string::npos);
assert(status.find("42c01f") == std::string::npos);
CloseHandle(write);
CloseHandle(read);
}
void CheckEncoderCandidates() {
const std::vector<EncoderCandidate> candidates{{0, 0}, {1, 0}, {1, 1}, {2, 0}};
const EncoderCandidate preferred{1, 0};
std::vector<EncoderCandidate> tried;
const auto deadline = EncoderClock::now() + std::chrono::seconds(1);
std::ostringstream diagnostic;
auto previous = std::cerr.rdbuf(diagnostic.rdbuf());
const auto chosen = SelectEncoderCandidate(candidates, preferred, deadline,
[&](EncoderCandidate candidate) {
tried.push_back(candidate);
if (candidate == preferred)
throw GateFailure("input-texture-create", "fixture device loss", DXGI_ERROR_DEVICE_REMOVED);
if (candidate != EncoderCandidate{1, 1})
throw GateFailure("fixture-activation", "candidate is not usable", E_FAIL);
});
std::cerr.rdbuf(previous);
assert((chosen == EncoderCandidate{1, 1}));
assert((tried == std::vector<EncoderCandidate>{{1, 0}, {0, 0}, {1, 1}}));
assert(diagnostic.str().find("stage=input-texture-create") != std::string::npos);
assert(diagnostic.str().find("fixture device loss") != std::string::npos);
assert(diagnostic.str().find("hresult=0x887a0005") != std::string::npos);
tried.clear();
try {
SelectEncoderCandidate(candidates, chosen, deadline, [&](EncoderCandidate candidate) {
tried.push_back(candidate);
throw GateFailure("fixture-all-failed", "no working candidate", E_FAIL);
});
assert(false);
} catch (const GateFailure& error) {
assert(error.stage() == "fixture-all-failed");
}
assert(tried.size() == candidates.size() && tried.front() == chosen);
tried.clear();
try {
SelectEncoderCandidate(candidates, preferred, EncoderClock::now(),
[&](EncoderCandidate candidate) { tried.push_back(candidate); });
assert(false);
} catch (const GateFailure& error) {
assert(error.stage() == "codec-probe-timeout");
}
assert(tried.empty());
}
class CadenceFixtureEncoder final : public VideoEncoder {
public:
CadenceFixtureEncoder(std::chrono::milliseconds latency, bool dropped, bool cold_only)
: VideoEncoder(OutputKind::h264, "cadence-fixture", "cadence-fixture"),
latency_(latency), dropped_(dropped), cold_only_(cold_only) {}
EncodedAccessUnit Encode(ID3D11Texture2D*, UINT64 timestamp, bool key,
EncoderClock::time_point deadline) override {
if (!cold_only_ || calls_ == 0) std::this_thread::sleep_for(latency_);
++calls_;
RequireEncoderTime(deadline);
// An empty unit stands for an input the pipeline did not encode.
return dropped_ ? EncodedAccessUnit{timestamp, key, {}} : EncodedAccessUnit{timestamp, key, {1, 2, 3}};
}
void SetBitrate(UINT32) override {}
private:
std::chrono::milliseconds latency_;
bool dropped_;
bool cold_only_;
unsigned calls_ = 0;
};
void CheckAutoCadenceProbe(ID3D11Device* device) {
// The Browser probe allows one 100 ms poll of lag at each target rate.
assert(CadenceLagFrames(5) == 1 && CadenceLagFrames(15) == 2 &&
CadenceLagFrames(30) == 3 && CadenceLagFrames(60) == 6);
VideoProfile profile;
profile.width = 64;
profile.height = 64;
auto probe = [&](std::chrono::milliseconds latency, bool dropped,
EncoderClock::time_point deadline, bool cold_only = false) {
return SustainsEncodedCadence([=](const VideoProfile&) -> std::unique_ptr<VideoEncoder> {
return std::make_unique<CadenceFixtureEncoder>(latency, dropped, cold_only);
}, device, profile, deadline);
};
const auto deadline = EncoderClock::now() + std::chrono::seconds(30);
// Valid output at a fraction of the source rate is the driver defect Auto rejects.
assert(!probe(std::chrono::milliseconds(0), true, deadline));
// Cold startup must not reject an encoder that sustains the measured cadence.
assert(probe(std::chrono::milliseconds(220), false, deadline, true));
// Sustained slow encoding still fails after warmup.
assert(!probe(std::chrono::milliseconds(300), false, deadline));
bool expired = false;
try {
probe(std::chrono::milliseconds(0), false, EncoderClock::time_point::min());
} catch (const GateFailure& error) {
expired = error.stage() == "codec-probe-timeout";
}
assert(expired);
}
void CheckOutputSampleAlignment() {
Runtime runtime;
for (DWORD alignment : {0u, 1u, 16u, 64u, 512u}) {
MFT_OUTPUT_STREAM_INFO info{};
info.cbSize = 4096;
info.cbAlignment = alignment;
auto sample = CreateCallerOutputSample(info);
ComPtr<IMFMediaBuffer> buffer;
Check(sample->GetBufferByIndex(0, &buffer), "fixture-output-buffer");
BYTE* data = nullptr;
DWORD maximum = 0;
Check(buffer->Lock(&data, &maximum, nullptr), "fixture-output-lock");
const auto address = reinterpret_cast<uintptr_t>(data);
Check(buffer->Unlock(), "fixture-output-unlock");
assert(maximum >= info.cbSize);
assert(alignment == 0 || address % alignment == 0);
}
}
void CheckSurfaceSample(ID3D11Device* device) {
Runtime runtime;
const VideoProfile profile{64, 64, 30, 3'000'000};
auto texture = CreateSyntheticTexture(device, 0, profile);
auto sample = CreateSurfaceSample(texture.Get(), 1'000'000, 333'333);
ComPtr<IMFMediaBuffer> buffer;
Check(sample->GetBufferByIndex(0, &buffer), "fixture-sample-buffer");
DWORD current = 0, maximum = 0, total = 0;
Check(buffer->GetCurrentLength(&current), "fixture-buffer-length");
Check(buffer->GetMaxLength(&maximum), "fixture-buffer-capacity");
Check(sample->GetTotalLength(&total), "fixture-sample-length");
assert(current == profile.width * profile.height * 3 / 2);
assert(total == current && maximum == current);
ComPtr<IMFDXGIBuffer> surface;
Check(buffer.As(&surface), "fixture-surface-buffer");
ComPtr<ID3D11Texture2D> retained;
Check(surface->GetResource(IID_PPV_ARGS(&retained)), "fixture-surface-resource");
assert(retained.Get() == texture.Get());
LONGLONG time = 0, duration = 0;
Check(sample->GetSampleTime(&time), "fixture-sample-time");
Check(sample->GetSampleDuration(&duration), "fixture-sample-duration");
assert(time == 1'000'000 && duration == 333'333);
}
int main() {
VideoProfile profile{2560, 1440, 60, 12'000'000};
ValidateVideoProfile(profile);
assert(profile.h264_level() == 51 && profile.profile_level_id() == "42c033");
profile = VideoProfile{1920, 1080, 60, 8'000'000};
assert(profile.h264_level() == 42);
profile = VideoProfile{3840, 2160, 30, 12'000'000};
try {
ValidateVideoProfile(profile);
assert(false);
} catch (const GateFailure& error) {
assert(error.stage() == "argument-profile");
}
CheckEncoderCandidates();
CheckH264ProfileCompatibility();
CheckPrimaryFailureDetails();
CheckOutputSampleAlignment();
ComPtr<ID3D11Device> device;
Check(D3D11CreateDevice(nullptr, D3D_DRIVER_TYPE_WARP, nullptr, 0, nullptr, 0,
D3D11_SDK_VERSION, &device, nullptr, nullptr), "fixture-warp");
CheckSurfaceSample(device.Get());
CheckAutoCadenceProbe(device.Get());
CheckWorkerFailureGeneration(device.Get(), false);
CheckWorkerFailureGeneration(device.Get(), true);
}