This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections. This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work. ## Validation - Full local Web check: 79 files / 1,317 tests, TypeScript and production build. - Full Go checks, platform builds and Windows native capture regressions/probe. - Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup. - Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI. - Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved. ## Release notes ### 中文 这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。 #### 新增与改进 - 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。 - 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。 - 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。 - 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。 - 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。 - App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。 #### 修复 - 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。 - 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。 - 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。 - 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。 - 无站点密码时,连接恢复不再误引导用户输入站点密码。 - 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。 - 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。 #### 使用说明 下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。 ### English This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged. #### New and improved - Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs. - Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them. - Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date. - Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts. - Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues. - The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source. #### Fixes - Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling. - Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session. - Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections. - Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests. - Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase. - Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip. - Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449). #### Updating Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome. ## Attribution Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
310 lines
11 KiB
Go
310 lines
11 KiB
Go
package mediaedge
|
|
|
|
import (
|
|
"errors"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
|
|
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
|
|
"github.com/livekit/livekit-server/pkg/sfu"
|
|
"github.com/pion/interceptor"
|
|
"github.com/pion/rtp"
|
|
"github.com/pion/webrtc/v4"
|
|
)
|
|
|
|
func TestPublicationOwnsOneReservationAndSourceLifetime(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
defer engine.Close()
|
|
source, err := engine.NewSource("vp8", 3, 2, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
defer source.Close()
|
|
for index := 0; index < 2; index++ {
|
|
if err = source.SetFormat(index, 320, 180); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
if err = source.ConfigureOutputs([]uint32{90_000, 300_000}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
publication, err := engine.NewPublication(source, EdgeOptions{ConnectionID: "publication"})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
candidate, err := engine.NewPublication(source, EdgeOptions{ConnectionID: "publication"})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = publication.transport.SetConnected(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = candidate.transport.SetConnected(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if len(source.media.GetDownTracks()) != 4 {
|
|
t.Fatalf("candidate overwrote current outputs: tracks=%d current=%s candidate=%s",
|
|
len(source.media.GetDownTracks()), publication.transport.PC.ID(), candidate.transport.PC.ID())
|
|
}
|
|
if _, err = engine.NewPublication(source, EdgeOptions{ConnectionID: "third"}); err == nil {
|
|
t.Fatal("third publication was admitted")
|
|
}
|
|
if source.reservations != 2 || len(engine.sockets) != 2 {
|
|
t.Fatal("failed publication leaked its reservation or socket")
|
|
}
|
|
edge, err := engine.NewEdge(source, EdgeOptions{ConnectionID: "sibling"})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
defer edge.Close()
|
|
if _, err = engine.NewEdge(source, EdgeOptions{ConnectionID: "overflow"}); !errors.Is(err, ErrSourceCapacity) {
|
|
t.Fatal("publication did not consume exactly one outbound copy")
|
|
}
|
|
if err = candidate.Close(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if source.reservations != 1 || len(source.publications) != 1 || publication.signaling.connection.ConnectionState() == webrtc.PeerConnectionStateClosed {
|
|
t.Fatal("candidate rollback retired the current publication")
|
|
}
|
|
replacement, err := engine.NewPublication(source, EdgeOptions{ConnectionID: "replacement"})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
retire, err := source.MarkLayerUnavailable(0)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
retire()
|
|
if _, err = engine.NewPublication(source, EdgeOptions{ConnectionID: "failed-output"}); err == nil {
|
|
t.Fatal("failed output was admitted into publication metadata")
|
|
}
|
|
if source.reservations != 0 || len(source.publications) != 0 || len(engine.sockets) != 1 {
|
|
t.Fatal("failed publication demand retained a reservation")
|
|
}
|
|
if err = source.ConfigureOutputs([]uint32{90_000, 300_000}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
replacement, err = engine.NewPublication(source, EdgeOptions{ConnectionID: "restored-output"})
|
|
if err != nil {
|
|
t.Fatalf("validated replacement did not restore publication admission: %v", err)
|
|
}
|
|
if err = source.Close(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if replacement.signaling.connection.ConnectionState() != webrtc.PeerConnectionStateClosed || len(engine.publications) != 0 || source.reservations != 0 || len(engine.sockets) != 0 {
|
|
t.Fatal("source retirement did not release the publication")
|
|
}
|
|
}
|
|
|
|
func TestPublicationOverlapToSharedSFUSocket(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("vp8", 2, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = source.Close() })
|
|
if err = source.SetFormat(0, 8, 8); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = source.ConfigureOutputs([]uint32{90_000}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
server, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = server.Close() })
|
|
// The embedded SFU deliberately keeps its shared listener. Native publication
|
|
// replacement must work against that unchanged, single-port remote endpoint.
|
|
sfuSocket := testICESocket(t, server)
|
|
for _, uri := range []string{"urn:ietf:params:rtp-hdrext:sdes:mid", "urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id"} {
|
|
if err = server.media.RegisterHeaderExtension(webrtc.RTPHeaderExtensionCapability{URI: uri}, webrtc.RTPCodecTypeVideo); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
connect := func(id string) (*Publication, <-chan *rtp.Packet) {
|
|
t.Helper()
|
|
publication, err := engine.NewPublication(source, EdgeOptions{ConnectionID: id})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = publication.Close() })
|
|
receiver, err := sfuSocket.newPeerConnection()
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = receiver.Close() })
|
|
packets := make(chan *rtp.Packet, 8)
|
|
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
|
|
for {
|
|
packet, _, err := track.ReadRTP()
|
|
if err != nil {
|
|
return
|
|
}
|
|
select {
|
|
case packets <- packet:
|
|
default:
|
|
}
|
|
}
|
|
})
|
|
connectEdgeToReceiver(t, &publication.signaling, receiver)
|
|
return publication, packets
|
|
}
|
|
pts := time.Duration(0)
|
|
delivery := func(streams ...<-chan *rtp.Packet) {
|
|
t.Helper()
|
|
for _, packets := range streams {
|
|
for len(packets) > 0 {
|
|
<-packets
|
|
}
|
|
}
|
|
for range 20 {
|
|
pts += time.Second / 30
|
|
if err = writeSourceFrame(source, sfu.VP8KeyFrame8x8, pts, time.Second/30); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
time.Sleep(time.Second / 30)
|
|
}
|
|
for _, packets := range streams {
|
|
_ = waitPacket(t, packets)
|
|
}
|
|
}
|
|
_, current := connect("current")
|
|
delivery(current)
|
|
candidate, pending := connect("candidate")
|
|
delivery(current, pending)
|
|
_ = candidate.Close()
|
|
delivery(current)
|
|
}
|
|
|
|
func TestPublicationAdmissionIgnoresUnrelatedFailedOutputs(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
defer engine.Close()
|
|
source, err := engine.NewSource("vp8", 2, 2, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
defer source.Close()
|
|
publication, err := engine.NewPublication(source, EdgeOptions{ConnectionID: "initial"})
|
|
if err != nil {
|
|
t.Fatalf("unconfigured outputs prevented initial publication: %v", err)
|
|
}
|
|
defer publication.Close()
|
|
if err = source.ConfigureOutputs([]uint32{90_000, 300_000, 90_000, 90_000}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
for _, slot := range []int{2, 3, 0, 1} {
|
|
retire, err := source.MarkLayerUnavailable(slot)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
retire()
|
|
candidate, createErr := engine.NewPublication(source, EdgeOptions{ConnectionID: "replacement"})
|
|
if slot >= 2 {
|
|
if createErr != nil {
|
|
t.Fatalf("unrelated failed output %d prevented publication: %v", slot, createErr)
|
|
}
|
|
if len(candidate.Media().Layers) != 2 || len(source.publications) != 2 {
|
|
t.Fatal("independent output failure changed the base publication")
|
|
}
|
|
if err = candidate.Close(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
} else if createErr == nil {
|
|
_ = candidate.Close()
|
|
t.Fatalf("failed base output %d was admitted", slot)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestPublicationQualityUsesHighestSentLayerCadence(t *testing.T) {
|
|
check := func(err error) {
|
|
t.Helper()
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
check(err)
|
|
defer engine.Close()
|
|
source, err := engine.NewSource("vp8", 1, 2, nil)
|
|
check(err)
|
|
defer source.Close()
|
|
check(source.SetFormat(0, 320, 180))
|
|
check(source.SetFormat(1, 640, 360))
|
|
check(source.ConfigureOutputs([]uint32{90_000, 300_000}))
|
|
publication, err := engine.NewPublication(source, EdgeOptions{ConnectionID: "publication_quality"})
|
|
check(err)
|
|
mediaEngine := &webrtc.MediaEngine{}
|
|
check(mediaEngine.RegisterDefaultCodecs())
|
|
for _, uri := range []string{"urn:ietf:params:rtp-hdrext:sdes:mid", "urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id"} {
|
|
check(mediaEngine.RegisterHeaderExtension(webrtc.RTPHeaderExtensionCapability{URI: uri}, webrtc.RTPCodecTypeVideo))
|
|
}
|
|
registry := &interceptor.Registry{}
|
|
check(webrtc.RegisterDefaultInterceptors(mediaEngine, registry))
|
|
settings := webrtc.SettingEngine{}
|
|
settings.SetIncludeLoopbackCandidate(true)
|
|
receiver, err := webrtc.NewAPI(webrtc.WithMediaEngine(mediaEngine), webrtc.WithInterceptorRegistry(registry), webrtc.WithSettingEngine(settings)).NewPeerConnection(webrtc.Configuration{})
|
|
check(err)
|
|
defer receiver.Close()
|
|
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
|
|
for {
|
|
if _, _, readErr := track.ReadRTP(); readErr != nil {
|
|
return
|
|
}
|
|
}
|
|
})
|
|
connectEdgeToReceiver(t, &publication.signaling, receiver)
|
|
assigned, err := forwarding.NewEncodedSource(forwarding.SourceOptions{
|
|
ID: string(source.media.TrackID()), StreamID: source.media.StreamID(), Codec: source.media.Codec(), MaxPackets: encoded.MaxPacketWindow,
|
|
Formats: []forwarding.LayerFormat{{Width: 8, Height: 8, Bitrate: 45_000}, {Width: 8, Height: 8, Bitrate: 150_000}},
|
|
})
|
|
check(err)
|
|
defer assigned.Close()
|
|
check(assigned.BeginFrame(0, time.Now()))
|
|
for layer := range 2 {
|
|
check(assigned.WriteFrame(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, Duration: time.Second / 30, Recovery: true}))
|
|
}
|
|
check(publication.transport.ReplaceSource(assigned.Source))
|
|
metadata := publication.Media()
|
|
if len(metadata.Layers) != 2 || metadata.Layers[0].Width != 8 || metadata.Layers[0].Height != 8 ||
|
|
metadata.Layers[0].Bitrate != 45_000 || metadata.Layers[1].Bitrate != 150_000 {
|
|
t.Fatalf("publication descriptor ignored assigned source: %+v", metadata)
|
|
}
|
|
now := time.Now()
|
|
if _, ok := publication.QualitySample(now); ok {
|
|
t.Fatal("first sample invented a window")
|
|
}
|
|
for frame := range 60 {
|
|
pts := time.Duration(frame+1) * time.Second / 30
|
|
check(assigned.BeginFrame(pts, time.Now()))
|
|
check(assigned.WriteFrame(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: time.Second / 30, Recovery: true}))
|
|
if frame%2 == 0 {
|
|
check(assigned.WriteFrame(1, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: pts, Duration: time.Second / 15, Recovery: true}))
|
|
}
|
|
time.Sleep(time.Second / 30)
|
|
}
|
|
sample, ok := publication.QualitySample(now.Add(2 * time.Second))
|
|
if !ok || sample.RID != "h" || sample.ActiveVideoEncodingCount != 2 || sample.Width != 8 || sample.Height != 8 {
|
|
t.Fatalf("actual publication dimensions or active outputs missing: %+v counters=%+v", sample, publication.transport.Counters())
|
|
}
|
|
if sample.IntervalFramesSent == 0 || sample.IntervalFramesSent != publication.qualityCounters.Frames[1] ||
|
|
sample.FramesPerSecond != float64(publication.qualityCounters.Frames[1])/2 {
|
|
t.Fatalf("publication summed layer cadence instead of highest sent output: %+v", sample)
|
|
}
|
|
if sample.BitrateKbps != float64(publication.qualityCounters.VideoBytes*8)/2/1000 {
|
|
t.Fatal("publication did not aggregate actual written layer bytes")
|
|
}
|
|
}
|