This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections. This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work. ## Validation - Full local Web check: 79 files / 1,317 tests, TypeScript and production build. - Full Go checks, platform builds and Windows native capture regressions/probe. - Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup. - Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI. - Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved. ## Release notes ### 中文 这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。 #### 新增与改进 - 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。 - 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。 - 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。 - 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。 - 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。 - App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。 #### 修复 - 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。 - 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。 - 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。 - 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。 - 无站点密码时,连接恢复不再误引导用户输入站点密码。 - 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。 - 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。 #### 使用说明 下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。 ### English This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged. #### New and improved - Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs. - Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them. - Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date. - Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts. - Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues. - The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source. #### Fixes - Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling. - Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session. - Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections. - Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests. - Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase. - Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip. - Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449). #### Updating Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome. ## Attribution Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
889 lines
28 KiB
Go
889 lines
28 KiB
Go
package mediaedge
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"net"
|
|
"slices"
|
|
"strings"
|
|
"sync"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/TNTcraftHIM/Piik/internal/app/nativeaudio"
|
|
"github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
|
|
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
|
|
"github.com/livekit/livekit-server/pkg/sfu"
|
|
"github.com/livekit/livekit-server/pkg/sfu/streamtracker"
|
|
"github.com/pion/interceptor"
|
|
"github.com/pion/logging"
|
|
"github.com/pion/rtcp"
|
|
"github.com/pion/rtp"
|
|
"github.com/pion/rtp/codecs"
|
|
"github.com/pion/webrtc/v4"
|
|
)
|
|
|
|
func writeSourceFrame(source *Source, data []byte, timestamp, duration time.Duration) error {
|
|
if _, err := source.BeginFrame(timestamp); err != nil {
|
|
return err
|
|
}
|
|
return source.WriteVideo(0, encoded.Frame{Data: data, PTS: timestamp, Duration: duration, Recovery: true})
|
|
}
|
|
|
|
func TestOutputPlanRetiresOnlyFailedLayerConsumers(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
wake := make(chan struct{}, 8)
|
|
source, err := engine.NewSource("vp8", 2, 2, func([]int) { wake <- struct{}{} })
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = source.ConfigureOutputs([]uint32{150_000, 600_000}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
high, _, _ := connectedReceiver(t, engine, source, "high")
|
|
low, _, _ := connectedReceiver(t, engine, source, "low")
|
|
if _, err = source.BeginFrame(0); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
for len(wake) > 0 {
|
|
<-wake
|
|
}
|
|
if err = low.SetTargetLayer(0); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
waitSignal(t, wake, "quiet source target-change wakeup")
|
|
plan, err := source.BeginFrame(time.Second / 30)
|
|
if err != nil || !slices.Equal(plan.Active, []bool{true, true}) || len(plan.Bitrates) != 2 {
|
|
t.Fatalf("output plan = %+v, %v", plan, err)
|
|
}
|
|
plan.Bitrates[0] = 1
|
|
trackers := source.media.StreamTrackerManager()
|
|
lowerTracker, higherTracker := trackers.AddTracker(0), trackers.AddTracker(1)
|
|
lowerTracker.Observe(0, 1200, 1188, true, 90000, nil)
|
|
higherTracker.Observe(0, 1200, 1188, true, 90000, nil)
|
|
retire, err := source.MarkLayerUnavailable(0)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
retire()
|
|
if low.State() != webrtc.PeerConnectionStateClosed || high.State() != webrtc.PeerConnectionStateConnected {
|
|
t.Fatalf("failed/sibling state = %s/%s", low.State(), high.State())
|
|
}
|
|
if err = high.SetTargetLayer(0); err == nil {
|
|
t.Fatal("failed output accepted a new demand")
|
|
}
|
|
lowerTracker.Observe(0, 1200, 1188, true, 93000, nil)
|
|
if lowerTracker.Status() != streamtracker.StreamStatusStopped || higherTracker.Status() != streamtracker.StreamStatusActive {
|
|
t.Fatal("failed tracker revived or reset a healthy sibling")
|
|
}
|
|
plan, err = source.BeginFrame(2 * time.Second / 30)
|
|
if err != nil || !slices.Equal(plan.Active, []bool{false, true}) || plan.Bitrates[0] != 0 || plan.Bitrates[1] != 600_000 {
|
|
t.Fatalf("surviving output plan = %+v, %v", plan, err)
|
|
}
|
|
if err = source.WriteVideo(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: 2 * time.Second / 30, Duration: time.Second / 30}); err == nil {
|
|
t.Fatal("late output revived a failed encoder slot")
|
|
}
|
|
for index, input := range []string{"capture", "relay"} {
|
|
candidate, _, _ := connectedReceiver(t, engine, source, "failed-demand-"+input)
|
|
// Exercise a framework request independently of the explicit Native
|
|
// setter. This is admission metadata, not simulated network evidence.
|
|
candidate.transport.OnTransportCCFeedback(nil, &rtcp.TransportLayerCC{})
|
|
candidate.transport.Output.SetMaxSpatialLayer(0)
|
|
candidate.transport.Output.SetBudget(80_000)
|
|
if input == "capture" {
|
|
plan, err = source.BeginFrame(time.Duration(index+3) * time.Second / 30)
|
|
if err != nil || !slices.Equal(plan.Active, []bool{false, true}) || plan.Bitrates[0] != 0 {
|
|
t.Fatalf("failed demand changed the capture plan: %+v, %v", plan, err)
|
|
}
|
|
} else {
|
|
err = source.WriteRTP(&rtp.Packet{Header: rtp.Header{Version: 2, SSRC: 42, PayloadType: 96,
|
|
SequenceNumber: 1, Timestamp: 90000, Marker: true}, Payload: append([]byte{0x10}, sfu.VP8KeyFrame8x8...)})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
if candidate.State() != webrtc.PeerConnectionStateClosed || high.State() != webrtc.PeerConnectionStateConnected {
|
|
t.Fatalf("%s failed/sibling state = %s/%s", input, candidate.State(), high.State())
|
|
}
|
|
}
|
|
if err = source.ConfigureOutputs([]uint32{150_000, 600_000}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
lowerTracker.Observe(0, 1200, 1188, true, 96000, nil)
|
|
if lowerTracker.Status() != streamtracker.StreamStatusActive || high.SetTargetLayer(0) != nil {
|
|
t.Fatal("validated capture output did not restore the failed slot")
|
|
}
|
|
}
|
|
|
|
func TestRelayFailureCannotRetireReconfiguredProfile(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("vp8", 1, 2, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = source.Close() })
|
|
source.relay = &relayDerivation{source: source}
|
|
_ = source.SetFormat(1, 640, 360)
|
|
profile := nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced"}
|
|
if err = source.SetRelayProfile(profile); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
low, _, _ := connectedReceiver(t, engine, source, "relay-failure")
|
|
if err = low.SetTargetLayer(0); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
ctx, cancel := context.WithCancelCause(t.Context())
|
|
defer cancel(nil)
|
|
done := make(chan struct{})
|
|
close(done)
|
|
old := &relayRun{owner: source.relay, ctx: ctx, cancel: cancel, done: done,
|
|
plan: relayPlan{profile: source.relayProfile, format: source.formats[1].Load()}}
|
|
source.relay.run = old
|
|
cancel(errors.New("old decoder failed"))
|
|
profile.Preference = "maintain-resolution"
|
|
if err = source.SetRelayProfile(profile); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
profile.Preference = "balanced"
|
|
if err = source.SetRelayProfile(profile); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if source.relayProfile == old.plan.profile || *source.relayProfile != *old.plan.profile {
|
|
t.Fatal("fixture did not return to equal settings with a new owner")
|
|
}
|
|
retire, err := source.markLayerUnavailable(0, old)
|
|
if err != nil || retire != nil || source.outputBitrates[0] == 0 || low.State() != webrtc.PeerConnectionStateConnected {
|
|
t.Fatal("late failed decoder retired reconfigured output")
|
|
}
|
|
current := &relayRun{owner: source.relay, ctx: ctx, cancel: cancel, done: done,
|
|
plan: relayPlan{profile: source.relayProfile, format: source.formats[1].Load()}}
|
|
source.relay.run = current
|
|
retire, err = source.markLayerUnavailable(0, current)
|
|
if err != nil || retire == nil || low.State() != webrtc.PeerConnectionStateConnected {
|
|
t.Fatal("current failure did not defer transport retirement until after owner locks")
|
|
}
|
|
retire()
|
|
if source.outputBitrates[0] != 0 || low.State() != webrtc.PeerConnectionStateClosed {
|
|
t.Fatal("current decoder failure was not retired")
|
|
}
|
|
}
|
|
|
|
func TestRetiredRelayFailureCannotDisableLaterDemand(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("vp8", 2, 2, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = source.Close() })
|
|
source.relay = &relayDerivation{source: source}
|
|
if err = source.SetFormat(1, 640, 360); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = source.SetRelayProfile(nativecapture.VideoProfile{
|
|
Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced",
|
|
}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
high, _, _ := connectedReceiver(t, engine, source, "retiring-relay-high")
|
|
low, _, _ := connectedReceiver(t, engine, source, "retiring-relay-low")
|
|
if err = low.SetTargetLayer(0); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
ctx, cancel := context.WithCancelCause(t.Context())
|
|
defer cancel(nil)
|
|
done := make(chan struct{})
|
|
t.Cleanup(func() { close(done) })
|
|
run := &relayRun{owner: source.relay, ctx: ctx, cancel: cancel, done: done,
|
|
plan: relayPlan{profile: source.relayProfile, format: source.formats[1].Load()}}
|
|
source.relay.run = run
|
|
if err = low.Close(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if !errors.Is(context.Cause(ctx), context.Canceled) {
|
|
t.Fatal("last lower consumer did not retire the derivation normally")
|
|
}
|
|
// A failure already read from the old process can reach the source lock
|
|
// after cancellation, before the process exits and the next run starts.
|
|
retire, err := source.markLayerUnavailable(0, run)
|
|
if err != nil || retire != nil || source.outputBitrates[0] == 0 {
|
|
t.Fatal("normal retirement disabled a reusable output slot")
|
|
}
|
|
rejoined, _, _ := connectedReceiver(t, engine, source, "retiring-relay-rejoined")
|
|
if err = rejoined.SetTargetLayer(0); err != nil {
|
|
t.Fatalf("later lower demand was rejected: %v", err)
|
|
}
|
|
if rejoined.State() != webrtc.PeerConnectionStateConnected || high.State() != webrtc.PeerConnectionStateConnected {
|
|
t.Fatal("late failure retired a current consumer or healthy sibling")
|
|
}
|
|
}
|
|
|
|
func TestShutdownClosesSiblingTransportsBeforeWaitingForWrites(t *testing.T) {
|
|
for _, owner := range []string{"source", "engine"} {
|
|
t.Run(owner, func(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("vp8", 2, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
closed := []chan struct{}{make(chan struct{}, 1), make(chan struct{}, 1)}
|
|
for index := range closed {
|
|
_, err := engine.NewEdge(source, EdgeOptions{
|
|
ConnectionID: "shutdown-" + owner + string(rune('a'+index)),
|
|
Events: EdgeEvents{ConnectionState: func(state webrtc.PeerConnectionState, _ *SelectedPair) {
|
|
if state == webrtc.PeerConnectionStateClosed {
|
|
closed[index] <- struct{}{}
|
|
}
|
|
}},
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
source.writeMu.Lock()
|
|
unlock := sync.OnceFunc(source.writeMu.Unlock)
|
|
done := make(chan struct{})
|
|
t.Cleanup(func() { unlock(); waitSignal(t, done, "shutdown cleanup") })
|
|
go func() {
|
|
if owner == "source" {
|
|
_ = source.Close()
|
|
} else {
|
|
_ = engine.Close()
|
|
}
|
|
close(done)
|
|
}()
|
|
for _, transport := range closed {
|
|
waitSignal(t, transport, "sibling transport close")
|
|
}
|
|
unlock()
|
|
waitSignal(t, done, "shutdown completion")
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestPortMappingCandidateUsesObservedPublicAddress(t *testing.T) {
|
|
var emitted []*webrtc.ICECandidateInit
|
|
gathering := &localCandidateGathering{
|
|
socket: &iceSocket{localPort: 43210},
|
|
mappedPort: 43211,
|
|
mappedCandidates: make(map[string]struct{}),
|
|
emit: func(candidate *webrtc.ICECandidateInit) {
|
|
emitted = append(emitted, candidate)
|
|
},
|
|
}
|
|
mapped := mappedAddress{address: "203.0.113.7", port: 41000}
|
|
gathering.emitMappedCandidate(mapped)
|
|
gathering.emitMappedCandidate(mapped)
|
|
|
|
if len(emitted) != 1 {
|
|
t.Fatalf("emitted candidates = %d, want one mapped", len(emitted))
|
|
}
|
|
fields := strings.Fields(emitted[0].Candidate)
|
|
if len(fields) < 8 || fields[0] != "candidate:mp1" ||
|
|
fields[3] != "1694498559" ||
|
|
fields[4] != "203.0.113.7" || fields[5] != "43211" ||
|
|
fields[7] != "srflx" {
|
|
t.Fatalf("mapped candidate = %q", emitted[0].Candidate)
|
|
}
|
|
}
|
|
|
|
func TestOneEncodedSourceFeedsTwoIndependentEdges(t *testing.T) {
|
|
for _, codec := range []string{"h264", "vp8"} {
|
|
t.Run(codec, func(t *testing.T) { testEncodedSourceFanout(t, codec) })
|
|
}
|
|
}
|
|
|
|
func testEncodedSourceFanout(t *testing.T, codec string) {
|
|
engine, err := NewEngine(EngineOptions{
|
|
BindAddress: "127.0.0.1:0",
|
|
IncludeLoopback: true,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
keyFrames := make(chan struct{}, 1)
|
|
source, err := engine.NewSource(codec, 2, 1, func([]int) {
|
|
select {
|
|
case keyFrames <- struct{}{}:
|
|
default:
|
|
}
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = source.Close() })
|
|
|
|
edgeA, receiverA, packetA := connectedReceiver(t, engine, source, "edge-a")
|
|
edgeB, receiverB, packetB := connectedReceiver(t, engine, source, "edge-b")
|
|
t.Cleanup(func() { _ = receiverA.Close() })
|
|
t.Cleanup(func() { _ = receiverB.Close() })
|
|
if edgeA.connection == edgeB.connection || edgeA.socket == edgeB.socket || edgeA.socket.localPort == edgeB.socket.localPort {
|
|
t.Fatal("media edges did not keep independent transports")
|
|
}
|
|
if edgeA.transport == edgeB.transport {
|
|
t.Fatal("media edges shared one bandwidth estimator")
|
|
}
|
|
if _, err = engine.NewEdge(source, EdgeOptions{ConnectionID: "edge-c"}); !errors.Is(err, ErrSourceCapacity) {
|
|
t.Fatalf("third edge error = %v", err)
|
|
}
|
|
select {
|
|
case <-keyFrames:
|
|
case <-time.After(3 * time.Second):
|
|
t.Fatal("connected edge did not request a recovery frame")
|
|
}
|
|
|
|
accessUnit := []byte{
|
|
0, 0, 0, 1, 0x67, 0x42, 0xc0, 0x1f, 0x96, 0x54, 0x05, 0x01,
|
|
0, 0, 0, 1, 0x68, 0xce, 0x3c, 0x80,
|
|
0, 0, 0, 1, 0x65, 0x88, 0x84, 0x00,
|
|
}
|
|
if codec == "vp8" {
|
|
accessUnit = []byte{0x10, 0, 0, 0x9d, 0x01, 0x2a, 0x80, 0x02, 0xe0, 0x01, 0}
|
|
}
|
|
if err = writeSourceFrame(source, accessUnit, time.Second, time.Second/30); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
first := waitPacket(t, packetA)
|
|
_ = waitPacket(t, packetB)
|
|
if codec == "vp8" {
|
|
packet := &codecs.VP8Packet{}
|
|
payload, parseErr := packet.Unmarshal(first.Payload)
|
|
if parseErr != nil || string(payload) != string(accessUnit) || packet.S != 1 {
|
|
t.Fatalf("VP8 packetizer changed its encoded source: %v", parseErr)
|
|
}
|
|
}
|
|
// Verify feedback delivery independently of the upstream bootstrap throttle.
|
|
source.media.GetAllBuffers()[0].SetPLIThrottle(0)
|
|
for len(keyFrames) > 0 {
|
|
<-keyFrames
|
|
}
|
|
if err = receiverA.WriteRTCP([]rtcp.Packet{
|
|
&rtcp.PictureLossIndication{MediaSSRC: first.SSRC},
|
|
}); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
select {
|
|
case <-keyFrames:
|
|
case <-time.After(3 * time.Second):
|
|
t.Fatal("PLI did not reach the shared source")
|
|
}
|
|
|
|
if err = edgeA.Close(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
edgeC, err := engine.NewEdge(source, EdgeOptions{ConnectionID: "edge-c"})
|
|
if err != nil {
|
|
t.Fatalf("released source slot was not reusable: %v", err)
|
|
}
|
|
_ = edgeC.Close()
|
|
}
|
|
|
|
func TestRemoteCandidatesAreBoundedUntilTheAnswer(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{
|
|
BindAddress: "127.0.0.1:0",
|
|
IncludeLoopback: true,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("h264", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
edge, err := engine.NewEdge(source, EdgeOptions{ConnectionID: "edge"})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = edge.AddRemoteCandidate(&webrtc.ICECandidateInit{
|
|
Candidate: "candidate:garbage",
|
|
}); err != nil {
|
|
t.Fatalf("malformed candidate escaped the edge boundary: %v", err)
|
|
}
|
|
if len(edge.pendingCandidates) != 0 {
|
|
t.Fatalf("malformed candidate was queued: %d", len(edge.pendingCandidates))
|
|
}
|
|
for index := 0; index < maxPendingCandidates; index++ {
|
|
if err = edge.AddRemoteCandidate(&webrtc.ICECandidateInit{
|
|
Candidate: "candidate:1 1 udp 1 127.0.0.1 9 typ host",
|
|
}); err != nil {
|
|
t.Fatalf("candidate %d: %v", index, err)
|
|
}
|
|
}
|
|
if err = edge.AddRemoteCandidate(&webrtc.ICECandidateInit{
|
|
Candidate: "candidate:1 1 udp 1 127.0.0.1 9 typ host",
|
|
}); err != nil {
|
|
t.Fatalf("excess candidate was not discarded: %v", err)
|
|
}
|
|
if len(edge.pendingCandidates) != maxPendingCandidates {
|
|
t.Fatalf("candidate queue grew beyond the bound: %d", len(edge.pendingCandidates))
|
|
}
|
|
}
|
|
|
|
func TestSelectedNatTraversalPathKeepsOnlyAnonymousProvenance(t *testing.T) {
|
|
if got := selectedNatTraversalPath("ordinary", "sp3"); got != "predicted" {
|
|
t.Fatalf("predicted path = %q", got)
|
|
}
|
|
if got := selectedNatTraversalPath("ordinary", "remote"); got != "ordinary" {
|
|
t.Fatalf("ordinary path = %q", got)
|
|
}
|
|
if got := selectedNatTraversalPath("", ""); got != "unknown" {
|
|
t.Fatalf("unknown path = %q", got)
|
|
}
|
|
}
|
|
|
|
func TestRepeatedAnswerIsIdempotent(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{
|
|
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("h264", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = source.Close() })
|
|
edge, receiver, _ := connectedReceiver(t, engine, source, "repeat-answer")
|
|
t.Cleanup(func() { _ = receiver.Close() })
|
|
answer := receiver.LocalDescription()
|
|
if answer == nil {
|
|
t.Fatal("receiver did not retain its answer")
|
|
}
|
|
if err = edge.SetAnswer(*answer); err != nil {
|
|
t.Fatalf("repeated answer was not idempotent: %v", err)
|
|
}
|
|
}
|
|
|
|
func TestCaptureTimestampsDriveTheRTPClock(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{
|
|
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("vp8", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = source.Close() })
|
|
_, receiver, packets := connectedReceiver(t, engine, source, "timed-edge")
|
|
t.Cleanup(func() { _ = receiver.Close() })
|
|
accessUnit := sfu.VP8KeyFrame8x8
|
|
start := 10 * time.Second
|
|
frameDuration := time.Second / 30
|
|
|
|
if err = writeSourceFrame(source, accessUnit, start, frameDuration); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
first := waitPacket(t, packets)
|
|
if err = writeSourceFrame(source, accessUnit, start+frameDuration, frameDuration); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
second := waitPacket(t, packets)
|
|
if got := second.Timestamp - first.Timestamp; got < 2_999 || got > 3_001 {
|
|
t.Fatalf("steady timestamp delta = %d", got)
|
|
}
|
|
if err = writeSourceFrame(source, accessUnit, start+5*time.Second, frameDuration); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
third := waitPacket(t, packets)
|
|
if got := third.Timestamp - second.Timestamp; got < 446_999 || got > 447_001 {
|
|
t.Fatalf("sparse timestamp delta = %d", got)
|
|
}
|
|
}
|
|
|
|
func TestNewCaptureGenerationKeepsTheRTPClockContinuous(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("vp8", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
_, receiver, packets := connectedReceiver(t, engine, source, "generation-edge")
|
|
t.Cleanup(func() { _ = receiver.Close() })
|
|
accessUnit := sfu.VP8KeyFrame8x8
|
|
frameDuration := time.Second / 30
|
|
if err = writeSourceFrame(source, accessUnit, 50*time.Second, frameDuration); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
first := waitPacket(t, packets)
|
|
source.BeginGeneration()
|
|
if err = writeSourceFrame(source, accessUnit, 100*time.Millisecond, frameDuration); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
second := waitPacket(t, packets)
|
|
if got := second.Timestamp - first.Timestamp; got < 2_999 || got > 3_001 {
|
|
t.Fatalf("generation timestamp delta = %d", got)
|
|
}
|
|
}
|
|
|
|
func TestOneLocalBridgeDoesNotConsumeRouteCapacity(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{
|
|
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("h264", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
audio, err := engine.NewAudioSource(1, 128_000)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = audio.Close() })
|
|
local, err := engine.NewEdge(source, EdgeOptions{
|
|
ConnectionID: "local-preview", Local: true, Audio: audio,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
route, err := engine.NewEdge(source, EdgeOptions{
|
|
ConnectionID: "route-edge", Audio: audio,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if _, err = engine.NewEdge(source, EdgeOptions{
|
|
ConnectionID: "second-local", Local: true,
|
|
}); !errors.Is(err, ErrSourceCapacity) {
|
|
t.Fatalf("second local edge error = %v", err)
|
|
}
|
|
if _, err = engine.NewEdge(source, EdgeOptions{
|
|
ConnectionID: "second-route",
|
|
}); !errors.Is(err, ErrSourceCapacity) {
|
|
t.Fatalf("second route edge error = %v", err)
|
|
}
|
|
_ = local.Close()
|
|
_ = route.Close()
|
|
if replacement, replaceErr := engine.NewEdge(source, EdgeOptions{
|
|
ConnectionID: "replacement-local", Local: true, Audio: audio,
|
|
}); replaceErr != nil {
|
|
t.Fatalf("released local slot was not reusable: %v", replaceErr)
|
|
} else {
|
|
_ = replacement.Close()
|
|
}
|
|
}
|
|
|
|
func TestBandwidthObserverReceivesTransportFeedbackWithoutPacing(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{
|
|
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
source, err := engine.NewSource("h264", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
edge, receiver, packets := connectedReceiver(t, engine, source, "quality-edge")
|
|
t.Cleanup(func() { _ = receiver.Close() })
|
|
if !strings.Contains(edge.connection.LocalDescription().SDP, "transport-cc") {
|
|
t.Fatal("native media offer did not negotiate transport feedback")
|
|
}
|
|
source.SetFormat(0, 1280, 720)
|
|
started := time.Now()
|
|
if _, ok := edge.QualitySample(started); ok {
|
|
t.Fatal("first quality sample did not establish a baseline")
|
|
}
|
|
|
|
accessUnit := []byte{
|
|
0, 0, 0, 1, 0x67, 0x42, 0xc0, 0x1f, 0x96, 0x54, 0x05, 0x01,
|
|
0, 0, 0, 1, 0x68, 0xce, 0x3c, 0x80,
|
|
0, 0, 0, 1, 0x65, 0x88, 0x84, 0x00,
|
|
}
|
|
deadline := time.Now().Add(5 * time.Second)
|
|
captureTimestamp := time.Second
|
|
for time.Now().Before(deadline) {
|
|
if err = writeSourceFrame(source,
|
|
accessUnit,
|
|
captureTimestamp,
|
|
time.Second/30,
|
|
); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
captureTimestamp += time.Second / 30
|
|
select {
|
|
case <-packets:
|
|
default:
|
|
}
|
|
if bitrate, observed := edge.targetBitrate(); observed && time.Since(started) >= time.Second {
|
|
if bitrate <= 0 {
|
|
t.Fatalf("observed target bitrate = %d", bitrate)
|
|
}
|
|
sample, ok := edge.QualitySample(time.Now())
|
|
if !ok || sample.State != "healthy" || sample.Reason == nil ||
|
|
*sample.Reason != "none" || sample.IntervalFramesEncoded == 0 ||
|
|
sample.RTPStatsID == "" || sample.TrackIdentifier == "" {
|
|
t.Fatalf("quality sample = %+v, %v", sample, ok)
|
|
}
|
|
return
|
|
}
|
|
time.Sleep(time.Second / 30)
|
|
}
|
|
t.Fatal("transport feedback did not produce a bandwidth estimate")
|
|
}
|
|
|
|
func TestAudioUsesTheSamePeerConnectionAndCapacityAsVideo(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
video, err := engine.NewSource("h264", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
audio, err := engine.NewAudioSource(1, 128_000)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = video.Close(); _ = audio.Close() })
|
|
edge, err := engine.NewEdge(video, EdgeOptions{
|
|
ConnectionID: "audio-edge",
|
|
Audio: audio,
|
|
})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = edge.Close() })
|
|
if edge.audioSender == nil {
|
|
t.Fatal("audio sender was not attached")
|
|
}
|
|
if _, err = engine.NewEdge(video, EdgeOptions{ConnectionID: "video-only"}); !errors.Is(err, ErrSourceCapacity) {
|
|
t.Fatalf("video capacity was not shared: %v", err)
|
|
}
|
|
if _, err = engine.NewEdge(video, EdgeOptions{ConnectionID: "audio-only", Audio: audio}); !errors.Is(err, ErrSourceCapacity) {
|
|
t.Fatalf("audio capacity was not shared: %v", err)
|
|
}
|
|
offer, err := edge.CreateOffer()
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if !strings.Contains(offer.SDP, "m=audio") || !strings.Contains(offer.SDP, "opus/48000") {
|
|
t.Fatalf("same connection offer has no Opus section: %s", offer.SDP)
|
|
}
|
|
}
|
|
|
|
func TestAudioSourceDeliversOpusOnTheVideoPeerConnection(t *testing.T) {
|
|
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = engine.Close() })
|
|
video, err := engine.NewSource("h264", 1, 1, nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
audio, err := engine.NewAudioSource(1, 128_000)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
edge, err := engine.NewEdge(video, EdgeOptions{ConnectionID: "audio-edge", Audio: audio})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = edge.Close(); _ = video.Close(); _ = audio.Close() })
|
|
receiver := newAudioReceiver(t)
|
|
t.Cleanup(func() { _ = receiver.Close() })
|
|
audioPackets := make(chan *rtp.Packet, 1)
|
|
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
|
|
if track.Kind() != webrtc.RTPCodecTypeAudio {
|
|
return
|
|
}
|
|
packet, _, readErr := track.ReadRTP()
|
|
if readErr == nil {
|
|
audioPackets <- packet
|
|
}
|
|
})
|
|
connectEdgeToReceiver(t, edge, receiver)
|
|
if err = audio.WritePCM(make([]byte, nativeaudio.FrameBytes), 20*time.Millisecond); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
select {
|
|
case packet := <-audioPackets:
|
|
if packet.PayloadType != 111 || len(packet.Payload) == 0 {
|
|
t.Fatalf("unexpected Opus RTP packet: payload type %d, %d bytes", packet.PayloadType, len(packet.Payload))
|
|
}
|
|
case <-time.After(5 * time.Second):
|
|
t.Fatal("timed out waiting for Opus RTP")
|
|
}
|
|
}
|
|
|
|
func connectedReceiver(
|
|
t *testing.T,
|
|
engine *Engine,
|
|
source *Source,
|
|
connectionID string,
|
|
) (*Edge, *webrtc.PeerConnection, <-chan *rtp.Packet) {
|
|
t.Helper()
|
|
edge, err := engine.NewEdge(source, EdgeOptions{ConnectionID: connectionID})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
t.Cleanup(func() { _ = edge.Close() })
|
|
receiver := newReceiver(t)
|
|
packets := make(chan *rtp.Packet, 64)
|
|
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
|
|
go func() {
|
|
for {
|
|
packet, _, readErr := track.ReadRTP()
|
|
if readErr != nil {
|
|
return
|
|
}
|
|
select {
|
|
case packets <- packet:
|
|
default:
|
|
}
|
|
}
|
|
}()
|
|
})
|
|
|
|
connectEdgeToReceiver(t, edge, receiver)
|
|
return edge, receiver, packets
|
|
}
|
|
|
|
func connectEdgeToReceiver(
|
|
t *testing.T,
|
|
edge *Edge,
|
|
receiver *webrtc.PeerConnection,
|
|
) {
|
|
t.Helper()
|
|
gatherOffer := webrtc.GatheringCompletePromise(edge.connection)
|
|
if _, err := edge.CreateOffer(); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
waitSignal(t, gatherOffer, "offer ICE gathering")
|
|
if err := receiver.SetRemoteDescription(*edge.connection.LocalDescription()); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
gatherAnswer := webrtc.GatheringCompletePromise(receiver)
|
|
answer, err := receiver.CreateAnswer(nil)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err = receiver.SetLocalDescription(answer); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
waitSignal(t, gatherAnswer, "answer ICE gathering")
|
|
if err = edge.SetAnswer(*receiver.LocalDescription()); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
deadline := time.Now().Add(5 * time.Second)
|
|
for edge.State() != webrtc.PeerConnectionStateConnected {
|
|
if time.Now().After(deadline) {
|
|
t.Fatalf("edge state = %s", edge.State())
|
|
}
|
|
time.Sleep(10 * time.Millisecond)
|
|
}
|
|
}
|
|
|
|
func newReceiver(t *testing.T) *webrtc.PeerConnection {
|
|
return newReceiverWithAudio(t, false, "127.0.0.1:0")
|
|
}
|
|
|
|
func newAudioReceiver(t *testing.T) *webrtc.PeerConnection {
|
|
return newReceiverWithAudio(t, true, "127.0.0.1:0")
|
|
}
|
|
|
|
func newReceiverWithAudio(t *testing.T, includeAudio bool, bindAddress string) *webrtc.PeerConnection {
|
|
t.Helper()
|
|
mediaEngine := &webrtc.MediaEngine{}
|
|
for _, codec := range []string{"h264", "vp8"} {
|
|
if err := mediaEngine.RegisterCodec(videoCodecs[codec], webrtc.RTPCodecTypeVideo); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
if includeAudio {
|
|
if err := mediaEngine.RegisterCodec(webrtc.RTPCodecParameters{
|
|
RTPCodecCapability: opusCapability,
|
|
PayloadType: 111,
|
|
}, webrtc.RTPCodecTypeAudio); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
registry := &interceptor.Registry{}
|
|
if err := webrtc.ConfigureTWCCSender(mediaEngine, registry); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if err := webrtc.RegisterDefaultInterceptors(mediaEngine, registry); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
settings := webrtc.SettingEngine{}
|
|
settings.SetIncludeLoopbackCandidate(true)
|
|
address, err := net.ResolveUDPAddr("udp", bindAddress)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
udp, err := net.ListenUDP("udp", address)
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
mux := webrtc.NewICEUDPMux(logging.NewDefaultLoggerFactory().NewLogger("mediaedge-test"), udp)
|
|
settings.SetICEUDPMux(mux)
|
|
t.Cleanup(func() { _ = mux.Close() })
|
|
api := webrtc.NewAPI(
|
|
webrtc.WithMediaEngine(mediaEngine),
|
|
webrtc.WithInterceptorRegistry(registry),
|
|
webrtc.WithSettingEngine(settings),
|
|
)
|
|
connection, err := api.NewPeerConnection(webrtc.Configuration{})
|
|
if err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if includeAudio {
|
|
if _, err = connection.AddTransceiverFromKind(
|
|
webrtc.RTPCodecTypeAudio,
|
|
webrtc.RTPTransceiverInit{Direction: webrtc.RTPTransceiverDirectionRecvonly},
|
|
); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
}
|
|
return connection
|
|
}
|
|
|
|
func waitSignal(t *testing.T, signal <-chan struct{}, label string) {
|
|
t.Helper()
|
|
select {
|
|
case <-signal:
|
|
case <-time.After(5 * time.Second):
|
|
t.Fatalf("timed out waiting for %s", label)
|
|
}
|
|
}
|
|
|
|
func waitPacket(t *testing.T, packets <-chan *rtp.Packet) *rtp.Packet {
|
|
t.Helper()
|
|
select {
|
|
case packet := <-packets:
|
|
return packet
|
|
case <-time.After(5 * time.Second):
|
|
t.Fatal("timed out waiting for RTP")
|
|
return nil
|
|
}
|
|
}
|