This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections. This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work. ## Validation - Full local Web check: 79 files / 1,317 tests, TypeScript and production build. - Full Go checks, platform builds and Windows native capture regressions/probe. - Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup. - Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI. - Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved. ## Release notes ### 中文 这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。 #### 新增与改进 - 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。 - 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。 - 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。 - 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。 - 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。 - App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。 #### 修复 - 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。 - 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。 - 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。 - 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。 - 无站点密码时,连接恢复不再误引导用户输入站点密码。 - 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。 - 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。 #### 使用说明 下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。 ### English This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged. #### New and improved - Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs. - Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them. - Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date. - Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts. - Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues. - The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source. #### Fixes - Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling. - Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session. - Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections. - Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests. - Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase. - Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip. - Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449). #### Updating Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome. ## Attribution Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
306 lines
13 KiB
C++
306 lines
13 KiB
C++
// Exercise the production worker with a synthetic codec at its existing link
|
|
// boundary. WARP supplies only the device-status API; no screen or audio is used.
|
|
#define wmain capture_main
|
|
#include "main.cpp"
|
|
#undef wmain
|
|
|
|
#include <cassert>
|
|
#include <future>
|
|
|
|
namespace piik::capture::windows {
|
|
class AdaptiveEncoder::Impl final {
|
|
public:
|
|
explicit Impl(Factory create, VideoProfile profile) : encoder(create(profile)) {}
|
|
std::unique_ptr<VideoEncoder> encoder;
|
|
};
|
|
|
|
AdaptiveEncoder::AdaptiveEncoder(OutputKind, VideoProfile profile, ID3D11Device*,
|
|
Factory create, std::unique_ptr<VideoEncoder>, int)
|
|
: impl_(std::make_unique<Impl>(std::move(create), profile)) {}
|
|
AdaptiveEncoder::~AdaptiveEncoder() = default;
|
|
|
|
std::optional<AdaptiveAccessUnit> AdaptiveEncoder::Encode(FrameProducer,
|
|
UINT32 width, UINT32 height, UINT64 timestamp, bool key, UINT32 bitrate,
|
|
EncoderClock::time_point deadline) {
|
|
impl_->encoder->SetBitrate(bitrate);
|
|
return AdaptiveAccessUnit{impl_->encoder->Encode(nullptr, timestamp, key, deadline),
|
|
width, height, 333'333};
|
|
}
|
|
} // namespace piik::capture::windows
|
|
|
|
class FixtureEncoder final : public VideoEncoder {
|
|
public:
|
|
FixtureEncoder(std::promise<void>* entered, std::shared_future<void> released)
|
|
: VideoEncoder(OutputKind::h264, "fixture", "fixture"),
|
|
entered_(entered), released_(std::move(released)) {}
|
|
|
|
EncodedAccessUnit Encode(ID3D11Texture2D*, UINT64 timestamp, bool key,
|
|
EncoderClock::time_point) override {
|
|
if (entered_) {
|
|
entered_->set_value();
|
|
released_.wait();
|
|
throw GateFailure("fixture-encode", "synthetic encoder failed", E_FAIL);
|
|
}
|
|
return {timestamp, key, {1, 2, 3}};
|
|
}
|
|
void SetBitrate(UINT32) override {}
|
|
|
|
private:
|
|
std::promise<void>* entered_;
|
|
std::shared_future<void> released_;
|
|
};
|
|
|
|
void CheckWorkerFailureGeneration(ID3D11Device* device, bool replace) {
|
|
std::promise<void> entered, release, output, failure;
|
|
auto began = entered.get_future();
|
|
auto released = release.get_future().share();
|
|
auto delivered = output.get_future();
|
|
auto failed = failure.get_future();
|
|
std::atomic<unsigned> created = 0;
|
|
HANDLE sink = CreateFileW(L"NUL", GENERIC_WRITE, 0, nullptr, OPEN_EXISTING, 0, nullptr);
|
|
assert(sink != INVALID_HANDLE_VALUE);
|
|
ProtocolWriter writer(sink);
|
|
OutputWorker worker(0, VideoProfile{}, OutputKind::h264, device, writer,
|
|
[&](const VideoProfile&) -> std::unique_ptr<VideoEncoder> {
|
|
return std::make_unique<FixtureEncoder>(created++ == 0 ? &entered : nullptr, released);
|
|
}, nullptr, true, [&](const EncodedAccessUnit&) { output.set_value(); },
|
|
[&](std::exception_ptr error) {
|
|
try { std::rethrow_exception(error); }
|
|
catch (const GateFailure& actual) { assert(actual.result() == E_FAIL); }
|
|
failure.set_value();
|
|
});
|
|
auto input = std::make_shared<CaptureInput>();
|
|
input->width = 1280;
|
|
input->height = 720;
|
|
input->timestamp = 333'333;
|
|
worker.Submit(input);
|
|
assert(began.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
|
|
if (replace) {
|
|
worker.SetActive(false);
|
|
assert(worker.SetActive(true));
|
|
worker.Submit(input);
|
|
}
|
|
release.set_value();
|
|
if (replace) {
|
|
assert(delivered.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
|
|
assert(failed.wait_for(std::chrono::milliseconds(0)) == std::future_status::timeout);
|
|
assert(created == 2);
|
|
} else {
|
|
assert(failed.wait_for(std::chrono::seconds(2)) == std::future_status::ready);
|
|
assert(delivered.wait_for(std::chrono::milliseconds(0)) == std::future_status::timeout);
|
|
assert(created == 1);
|
|
assert(!worker.SetActive(false) && !worker.SetActive(true));
|
|
}
|
|
worker.Stop();
|
|
worker.Join();
|
|
CloseHandle(sink);
|
|
}
|
|
|
|
void CheckPrimaryFailureDetails() {
|
|
std::ostringstream output;
|
|
auto previous = std::cerr.rdbuf(output.rdbuf());
|
|
assert(OutputFailureDetail(3, std::make_exception_ptr(
|
|
GateFailure("fixture-output", "original cause", E_ACCESSDENIED))) == "fixture-output");
|
|
std::cerr.rdbuf(previous);
|
|
const auto text = output.str();
|
|
assert(text.find("layer=3") != std::string::npos);
|
|
assert(text.find("original cause") != std::string::npos);
|
|
assert(text.find("hresult=0x80070005") != std::string::npos);
|
|
// With no COM apartment, the real decoder fails at its first API boundary.
|
|
bool rejected = false;
|
|
try { piik::capture::H264Decoder decoder(nullptr); }
|
|
catch (const GateFailure& error) {
|
|
rejected = error.stage() == "h264-decoder-create" && error.result() == CO_E_NOTINITIALIZED;
|
|
}
|
|
assert(rejected);
|
|
}
|
|
|
|
void CheckH264ProfileCompatibility() {
|
|
const VideoProfile profile;
|
|
for (const auto* value : {"42c01f", "42e01f", "42E01F", "4d801f", "58c01f"})
|
|
assert(profile.accepts_h264_profile_level_id(value));
|
|
for (const auto* value : {"42001f", "4d001f", "64001f", "42e11f", "42e01", "42e01fg",
|
|
"42e0gg", "42e034", "42e01e", " 42c01"})
|
|
assert(!profile.accepts_h264_profile_level_id(value));
|
|
|
|
HANDLE read = nullptr, write = nullptr;
|
|
assert(CreatePipe(&read, &write, nullptr, 4096));
|
|
ProtocolWriter writer(write);
|
|
ProductArguments arguments;
|
|
arguments.outputs = {profile};
|
|
EncodedAccessUnit output{0, true, {0, 0, 0, 1, 0x67, 0x42, 0xe0, 0x1f}};
|
|
WriteVideoActive(writer, arguments, true, output);
|
|
std::array<char, 4096> bytes{};
|
|
DWORD count = 0;
|
|
assert(ReadFile(read, bytes.data(), static_cast<DWORD>(bytes.size()), &count, nullptr));
|
|
const std::string status(bytes.data(), count);
|
|
assert(status.find("\"profileLevelId\":\"42e01f\"") != std::string::npos);
|
|
assert(status.find("42c01f") == std::string::npos);
|
|
CloseHandle(write);
|
|
CloseHandle(read);
|
|
}
|
|
|
|
void CheckEncoderCandidates() {
|
|
const std::vector<EncoderCandidate> candidates{{0, 0}, {1, 0}, {1, 1}, {2, 0}};
|
|
const EncoderCandidate preferred{1, 0};
|
|
std::vector<EncoderCandidate> tried;
|
|
const auto deadline = EncoderClock::now() + std::chrono::seconds(1);
|
|
std::ostringstream diagnostic;
|
|
auto previous = std::cerr.rdbuf(diagnostic.rdbuf());
|
|
const auto chosen = SelectEncoderCandidate(candidates, preferred, deadline,
|
|
[&](EncoderCandidate candidate) {
|
|
tried.push_back(candidate);
|
|
if (candidate == preferred)
|
|
throw GateFailure("input-texture-create", "fixture device loss", DXGI_ERROR_DEVICE_REMOVED);
|
|
if (candidate != EncoderCandidate{1, 1})
|
|
throw GateFailure("fixture-activation", "candidate is not usable", E_FAIL);
|
|
});
|
|
std::cerr.rdbuf(previous);
|
|
assert((chosen == EncoderCandidate{1, 1}));
|
|
assert((tried == std::vector<EncoderCandidate>{{1, 0}, {0, 0}, {1, 1}}));
|
|
assert(diagnostic.str().find("stage=input-texture-create") != std::string::npos);
|
|
assert(diagnostic.str().find("fixture device loss") != std::string::npos);
|
|
assert(diagnostic.str().find("hresult=0x887a0005") != std::string::npos);
|
|
tried.clear();
|
|
try {
|
|
SelectEncoderCandidate(candidates, chosen, deadline, [&](EncoderCandidate candidate) {
|
|
tried.push_back(candidate);
|
|
throw GateFailure("fixture-all-failed", "no working candidate", E_FAIL);
|
|
});
|
|
assert(false);
|
|
} catch (const GateFailure& error) {
|
|
assert(error.stage() == "fixture-all-failed");
|
|
}
|
|
assert(tried.size() == candidates.size() && tried.front() == chosen);
|
|
tried.clear();
|
|
try {
|
|
SelectEncoderCandidate(candidates, preferred, EncoderClock::now(),
|
|
[&](EncoderCandidate candidate) { tried.push_back(candidate); });
|
|
assert(false);
|
|
} catch (const GateFailure& error) {
|
|
assert(error.stage() == "codec-probe-timeout");
|
|
}
|
|
assert(tried.empty());
|
|
}
|
|
|
|
class CadenceFixtureEncoder final : public VideoEncoder {
|
|
public:
|
|
CadenceFixtureEncoder(std::chrono::milliseconds latency, bool dropped, bool cold_only)
|
|
: VideoEncoder(OutputKind::h264, "cadence-fixture", "cadence-fixture"),
|
|
latency_(latency), dropped_(dropped), cold_only_(cold_only) {}
|
|
|
|
EncodedAccessUnit Encode(ID3D11Texture2D*, UINT64 timestamp, bool key,
|
|
EncoderClock::time_point deadline) override {
|
|
if (!cold_only_ || calls_ == 0) std::this_thread::sleep_for(latency_);
|
|
++calls_;
|
|
RequireEncoderTime(deadline);
|
|
// An empty unit stands for an input the pipeline did not encode.
|
|
return dropped_ ? EncodedAccessUnit{timestamp, key, {}} : EncodedAccessUnit{timestamp, key, {1, 2, 3}};
|
|
}
|
|
void SetBitrate(UINT32) override {}
|
|
|
|
private:
|
|
std::chrono::milliseconds latency_;
|
|
bool dropped_;
|
|
bool cold_only_;
|
|
unsigned calls_ = 0;
|
|
};
|
|
|
|
void CheckAutoCadenceProbe(ID3D11Device* device) {
|
|
// The Browser probe allows one 100 ms poll of lag at each target rate.
|
|
assert(CadenceLagFrames(5) == 1 && CadenceLagFrames(15) == 2 &&
|
|
CadenceLagFrames(30) == 3 && CadenceLagFrames(60) == 6);
|
|
VideoProfile profile;
|
|
profile.width = 64;
|
|
profile.height = 64;
|
|
auto probe = [&](std::chrono::milliseconds latency, bool dropped,
|
|
EncoderClock::time_point deadline, bool cold_only = false) {
|
|
return SustainsEncodedCadence([=](const VideoProfile&) -> std::unique_ptr<VideoEncoder> {
|
|
return std::make_unique<CadenceFixtureEncoder>(latency, dropped, cold_only);
|
|
}, device, profile, deadline);
|
|
};
|
|
const auto deadline = EncoderClock::now() + std::chrono::seconds(30);
|
|
// Valid output at a fraction of the source rate is the driver defect Auto rejects.
|
|
assert(!probe(std::chrono::milliseconds(0), true, deadline));
|
|
// Cold startup must not reject an encoder that sustains the measured cadence.
|
|
assert(probe(std::chrono::milliseconds(220), false, deadline, true));
|
|
// Sustained slow encoding still fails after warmup.
|
|
assert(!probe(std::chrono::milliseconds(300), false, deadline));
|
|
bool expired = false;
|
|
try {
|
|
probe(std::chrono::milliseconds(0), false, EncoderClock::time_point::min());
|
|
} catch (const GateFailure& error) {
|
|
expired = error.stage() == "codec-probe-timeout";
|
|
}
|
|
assert(expired);
|
|
}
|
|
|
|
void CheckOutputSampleAlignment() {
|
|
Runtime runtime;
|
|
for (DWORD alignment : {0u, 1u, 16u, 64u, 512u}) {
|
|
MFT_OUTPUT_STREAM_INFO info{};
|
|
info.cbSize = 4096;
|
|
info.cbAlignment = alignment;
|
|
auto sample = CreateCallerOutputSample(info);
|
|
ComPtr<IMFMediaBuffer> buffer;
|
|
Check(sample->GetBufferByIndex(0, &buffer), "fixture-output-buffer");
|
|
BYTE* data = nullptr;
|
|
DWORD maximum = 0;
|
|
Check(buffer->Lock(&data, &maximum, nullptr), "fixture-output-lock");
|
|
const auto address = reinterpret_cast<uintptr_t>(data);
|
|
Check(buffer->Unlock(), "fixture-output-unlock");
|
|
assert(maximum >= info.cbSize);
|
|
assert(alignment == 0 || address % alignment == 0);
|
|
}
|
|
}
|
|
|
|
void CheckSurfaceSample(ID3D11Device* device) {
|
|
Runtime runtime;
|
|
const VideoProfile profile{64, 64, 30, 3'000'000};
|
|
auto texture = CreateSyntheticTexture(device, 0, profile);
|
|
auto sample = CreateSurfaceSample(texture.Get(), 1'000'000, 333'333);
|
|
ComPtr<IMFMediaBuffer> buffer;
|
|
Check(sample->GetBufferByIndex(0, &buffer), "fixture-sample-buffer");
|
|
DWORD current = 0, maximum = 0, total = 0;
|
|
Check(buffer->GetCurrentLength(¤t), "fixture-buffer-length");
|
|
Check(buffer->GetMaxLength(&maximum), "fixture-buffer-capacity");
|
|
Check(sample->GetTotalLength(&total), "fixture-sample-length");
|
|
assert(current == profile.width * profile.height * 3 / 2);
|
|
assert(total == current && maximum == current);
|
|
ComPtr<IMFDXGIBuffer> surface;
|
|
Check(buffer.As(&surface), "fixture-surface-buffer");
|
|
ComPtr<ID3D11Texture2D> retained;
|
|
Check(surface->GetResource(IID_PPV_ARGS(&retained)), "fixture-surface-resource");
|
|
assert(retained.Get() == texture.Get());
|
|
LONGLONG time = 0, duration = 0;
|
|
Check(sample->GetSampleTime(&time), "fixture-sample-time");
|
|
Check(sample->GetSampleDuration(&duration), "fixture-sample-duration");
|
|
assert(time == 1'000'000 && duration == 333'333);
|
|
}
|
|
|
|
int main() {
|
|
VideoProfile profile{2560, 1440, 60, 12'000'000};
|
|
ValidateVideoProfile(profile);
|
|
assert(profile.h264_level() == 51 && profile.profile_level_id() == "42c033");
|
|
profile = VideoProfile{1920, 1080, 60, 8'000'000};
|
|
assert(profile.h264_level() == 42);
|
|
profile = VideoProfile{3840, 2160, 30, 12'000'000};
|
|
try {
|
|
ValidateVideoProfile(profile);
|
|
assert(false);
|
|
} catch (const GateFailure& error) {
|
|
assert(error.stage() == "argument-profile");
|
|
}
|
|
CheckEncoderCandidates();
|
|
CheckH264ProfileCompatibility();
|
|
CheckPrimaryFailureDetails();
|
|
CheckOutputSampleAlignment();
|
|
ComPtr<ID3D11Device> device;
|
|
Check(D3D11CreateDevice(nullptr, D3D_DRIVER_TYPE_WARP, nullptr, 0, nullptr, 0,
|
|
D3D11_SDK_VERSION, &device, nullptr, nullptr), "fixture-warp");
|
|
CheckSurfaceSample(device.Get());
|
|
CheckAutoCadenceProbe(device.Get());
|
|
CheckWorkerFailureGeneration(device.Get(), false);
|
|
CheckWorkerFailureGeneration(device.Get(), true);
|
|
}
|