Files
Piik/internal/app/mediaedge/edge_test.go
T
AAAandPedro Antonio c18502fd18 feat: improve microphone, chat and platform support
This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections.

This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work.

## Validation

- Full local Web check: 79 files / 1,317 tests, TypeScript and production build.
- Full Go checks, platform builds and Windows native capture regressions/probe.
- Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup.
- Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI.
- Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved.

## Release notes

### 中文

这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。

#### 新增与改进

- 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。
- 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。
- 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。
- 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。
- 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。
- App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。

#### 修复

- 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。
- 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。
- 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。
- 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。
- 无站点密码时,连接恢复不再误引导用户输入站点密码。
- 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。
- 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。

#### 使用说明

下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。

### English

This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged.

#### New and improved

- Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs.
- Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them.
- Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date.
- Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts.
- Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues.
- The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source.

#### Fixes

- Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling.
- Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session.
- Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections.
- Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests.
- Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase.
- Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip.
- Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449).

#### Updating

Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome.

## Attribution

Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
2026-10-04 16:21:01 +08:00

889 lines
28 KiB
Go

package mediaedge
import (
"context"
"errors"
"net"
"slices"
"strings"
"sync"
"testing"
"time"
"github.com/TNTcraftHIM/Piik/internal/app/nativeaudio"
"github.com/TNTcraftHIM/Piik/internal/app/nativecapture"
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
"github.com/livekit/livekit-server/pkg/sfu"
"github.com/livekit/livekit-server/pkg/sfu/streamtracker"
"github.com/pion/interceptor"
"github.com/pion/logging"
"github.com/pion/rtcp"
"github.com/pion/rtp"
"github.com/pion/rtp/codecs"
"github.com/pion/webrtc/v4"
)
func writeSourceFrame(source *Source, data []byte, timestamp, duration time.Duration) error {
if _, err := source.BeginFrame(timestamp); err != nil {
return err
}
return source.WriteVideo(0, encoded.Frame{Data: data, PTS: timestamp, Duration: duration, Recovery: true})
}
func TestOutputPlanRetiresOnlyFailedLayerConsumers(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
wake := make(chan struct{}, 8)
source, err := engine.NewSource("vp8", 2, 2, func([]int) { wake <- struct{}{} })
if err != nil {
t.Fatal(err)
}
if err = source.ConfigureOutputs([]uint32{150_000, 600_000}); err != nil {
t.Fatal(err)
}
high, _, _ := connectedReceiver(t, engine, source, "high")
low, _, _ := connectedReceiver(t, engine, source, "low")
if _, err = source.BeginFrame(0); err != nil {
t.Fatal(err)
}
for len(wake) > 0 {
<-wake
}
if err = low.SetTargetLayer(0); err != nil {
t.Fatal(err)
}
waitSignal(t, wake, "quiet source target-change wakeup")
plan, err := source.BeginFrame(time.Second / 30)
if err != nil || !slices.Equal(plan.Active, []bool{true, true}) || len(plan.Bitrates) != 2 {
t.Fatalf("output plan = %+v, %v", plan, err)
}
plan.Bitrates[0] = 1
trackers := source.media.StreamTrackerManager()
lowerTracker, higherTracker := trackers.AddTracker(0), trackers.AddTracker(1)
lowerTracker.Observe(0, 1200, 1188, true, 90000, nil)
higherTracker.Observe(0, 1200, 1188, true, 90000, nil)
retire, err := source.MarkLayerUnavailable(0)
if err != nil {
t.Fatal(err)
}
retire()
if low.State() != webrtc.PeerConnectionStateClosed || high.State() != webrtc.PeerConnectionStateConnected {
t.Fatalf("failed/sibling state = %s/%s", low.State(), high.State())
}
if err = high.SetTargetLayer(0); err == nil {
t.Fatal("failed output accepted a new demand")
}
lowerTracker.Observe(0, 1200, 1188, true, 93000, nil)
if lowerTracker.Status() != streamtracker.StreamStatusStopped || higherTracker.Status() != streamtracker.StreamStatusActive {
t.Fatal("failed tracker revived or reset a healthy sibling")
}
plan, err = source.BeginFrame(2 * time.Second / 30)
if err != nil || !slices.Equal(plan.Active, []bool{false, true}) || plan.Bitrates[0] != 0 || plan.Bitrates[1] != 600_000 {
t.Fatalf("surviving output plan = %+v, %v", plan, err)
}
if err = source.WriteVideo(0, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: 2 * time.Second / 30, Duration: time.Second / 30}); err == nil {
t.Fatal("late output revived a failed encoder slot")
}
for index, input := range []string{"capture", "relay"} {
candidate, _, _ := connectedReceiver(t, engine, source, "failed-demand-"+input)
// Exercise a framework request independently of the explicit Native
// setter. This is admission metadata, not simulated network evidence.
candidate.transport.OnTransportCCFeedback(nil, &rtcp.TransportLayerCC{})
candidate.transport.Output.SetMaxSpatialLayer(0)
candidate.transport.Output.SetBudget(80_000)
if input == "capture" {
plan, err = source.BeginFrame(time.Duration(index+3) * time.Second / 30)
if err != nil || !slices.Equal(plan.Active, []bool{false, true}) || plan.Bitrates[0] != 0 {
t.Fatalf("failed demand changed the capture plan: %+v, %v", plan, err)
}
} else {
err = source.WriteRTP(&rtp.Packet{Header: rtp.Header{Version: 2, SSRC: 42, PayloadType: 96,
SequenceNumber: 1, Timestamp: 90000, Marker: true}, Payload: append([]byte{0x10}, sfu.VP8KeyFrame8x8...)})
if err != nil {
t.Fatal(err)
}
}
if candidate.State() != webrtc.PeerConnectionStateClosed || high.State() != webrtc.PeerConnectionStateConnected {
t.Fatalf("%s failed/sibling state = %s/%s", input, candidate.State(), high.State())
}
}
if err = source.ConfigureOutputs([]uint32{150_000, 600_000}); err != nil {
t.Fatal(err)
}
lowerTracker.Observe(0, 1200, 1188, true, 96000, nil)
if lowerTracker.Status() != streamtracker.StreamStatusActive || high.SetTargetLayer(0) != nil {
t.Fatal("validated capture output did not restore the failed slot")
}
}
func TestRelayFailureCannotRetireReconfiguredProfile(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("vp8", 1, 2, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = source.Close() })
source.relay = &relayDerivation{source: source}
_ = source.SetFormat(1, 640, 360)
profile := nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced"}
if err = source.SetRelayProfile(profile); err != nil {
t.Fatal(err)
}
low, _, _ := connectedReceiver(t, engine, source, "relay-failure")
if err = low.SetTargetLayer(0); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(nil)
done := make(chan struct{})
close(done)
old := &relayRun{owner: source.relay, ctx: ctx, cancel: cancel, done: done,
plan: relayPlan{profile: source.relayProfile, format: source.formats[1].Load()}}
source.relay.run = old
cancel(errors.New("old decoder failed"))
profile.Preference = "maintain-resolution"
if err = source.SetRelayProfile(profile); err != nil {
t.Fatal(err)
}
profile.Preference = "balanced"
if err = source.SetRelayProfile(profile); err != nil {
t.Fatal(err)
}
if source.relayProfile == old.plan.profile || *source.relayProfile != *old.plan.profile {
t.Fatal("fixture did not return to equal settings with a new owner")
}
retire, err := source.markLayerUnavailable(0, old)
if err != nil || retire != nil || source.outputBitrates[0] == 0 || low.State() != webrtc.PeerConnectionStateConnected {
t.Fatal("late failed decoder retired reconfigured output")
}
current := &relayRun{owner: source.relay, ctx: ctx, cancel: cancel, done: done,
plan: relayPlan{profile: source.relayProfile, format: source.formats[1].Load()}}
source.relay.run = current
retire, err = source.markLayerUnavailable(0, current)
if err != nil || retire == nil || low.State() != webrtc.PeerConnectionStateConnected {
t.Fatal("current failure did not defer transport retirement until after owner locks")
}
retire()
if source.outputBitrates[0] != 0 || low.State() != webrtc.PeerConnectionStateClosed {
t.Fatal("current decoder failure was not retired")
}
}
func TestRetiredRelayFailureCannotDisableLaterDemand(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("vp8", 2, 2, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = source.Close() })
source.relay = &relayDerivation{source: source}
if err = source.SetFormat(1, 640, 360); err != nil {
t.Fatal(err)
}
if err = source.SetRelayProfile(nativecapture.VideoProfile{
Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced",
}); err != nil {
t.Fatal(err)
}
high, _, _ := connectedReceiver(t, engine, source, "retiring-relay-high")
low, _, _ := connectedReceiver(t, engine, source, "retiring-relay-low")
if err = low.SetTargetLayer(0); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(nil)
done := make(chan struct{})
t.Cleanup(func() { close(done) })
run := &relayRun{owner: source.relay, ctx: ctx, cancel: cancel, done: done,
plan: relayPlan{profile: source.relayProfile, format: source.formats[1].Load()}}
source.relay.run = run
if err = low.Close(); err != nil {
t.Fatal(err)
}
if !errors.Is(context.Cause(ctx), context.Canceled) {
t.Fatal("last lower consumer did not retire the derivation normally")
}
// A failure already read from the old process can reach the source lock
// after cancellation, before the process exits and the next run starts.
retire, err := source.markLayerUnavailable(0, run)
if err != nil || retire != nil || source.outputBitrates[0] == 0 {
t.Fatal("normal retirement disabled a reusable output slot")
}
rejoined, _, _ := connectedReceiver(t, engine, source, "retiring-relay-rejoined")
if err = rejoined.SetTargetLayer(0); err != nil {
t.Fatalf("later lower demand was rejected: %v", err)
}
if rejoined.State() != webrtc.PeerConnectionStateConnected || high.State() != webrtc.PeerConnectionStateConnected {
t.Fatal("late failure retired a current consumer or healthy sibling")
}
}
func TestShutdownClosesSiblingTransportsBeforeWaitingForWrites(t *testing.T) {
for _, owner := range []string{"source", "engine"} {
t.Run(owner, func(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("vp8", 2, 1, nil)
if err != nil {
t.Fatal(err)
}
closed := []chan struct{}{make(chan struct{}, 1), make(chan struct{}, 1)}
for index := range closed {
_, err := engine.NewEdge(source, EdgeOptions{
ConnectionID: "shutdown-" + owner + string(rune('a'+index)),
Events: EdgeEvents{ConnectionState: func(state webrtc.PeerConnectionState, _ *SelectedPair) {
if state == webrtc.PeerConnectionStateClosed {
closed[index] <- struct{}{}
}
}},
})
if err != nil {
t.Fatal(err)
}
}
source.writeMu.Lock()
unlock := sync.OnceFunc(source.writeMu.Unlock)
done := make(chan struct{})
t.Cleanup(func() { unlock(); waitSignal(t, done, "shutdown cleanup") })
go func() {
if owner == "source" {
_ = source.Close()
} else {
_ = engine.Close()
}
close(done)
}()
for _, transport := range closed {
waitSignal(t, transport, "sibling transport close")
}
unlock()
waitSignal(t, done, "shutdown completion")
})
}
}
func TestPortMappingCandidateUsesObservedPublicAddress(t *testing.T) {
var emitted []*webrtc.ICECandidateInit
gathering := &localCandidateGathering{
socket: &iceSocket{localPort: 43210},
mappedPort: 43211,
mappedCandidates: make(map[string]struct{}),
emit: func(candidate *webrtc.ICECandidateInit) {
emitted = append(emitted, candidate)
},
}
mapped := mappedAddress{address: "203.0.113.7", port: 41000}
gathering.emitMappedCandidate(mapped)
gathering.emitMappedCandidate(mapped)
if len(emitted) != 1 {
t.Fatalf("emitted candidates = %d, want one mapped", len(emitted))
}
fields := strings.Fields(emitted[0].Candidate)
if len(fields) < 8 || fields[0] != "candidate:mp1" ||
fields[3] != "1694498559" ||
fields[4] != "203.0.113.7" || fields[5] != "43211" ||
fields[7] != "srflx" {
t.Fatalf("mapped candidate = %q", emitted[0].Candidate)
}
}
func TestOneEncodedSourceFeedsTwoIndependentEdges(t *testing.T) {
for _, codec := range []string{"h264", "vp8"} {
t.Run(codec, func(t *testing.T) { testEncodedSourceFanout(t, codec) })
}
}
func testEncodedSourceFanout(t *testing.T, codec string) {
engine, err := NewEngine(EngineOptions{
BindAddress: "127.0.0.1:0",
IncludeLoopback: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
keyFrames := make(chan struct{}, 1)
source, err := engine.NewSource(codec, 2, 1, func([]int) {
select {
case keyFrames <- struct{}{}:
default:
}
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = source.Close() })
edgeA, receiverA, packetA := connectedReceiver(t, engine, source, "edge-a")
edgeB, receiverB, packetB := connectedReceiver(t, engine, source, "edge-b")
t.Cleanup(func() { _ = receiverA.Close() })
t.Cleanup(func() { _ = receiverB.Close() })
if edgeA.connection == edgeB.connection || edgeA.socket == edgeB.socket || edgeA.socket.localPort == edgeB.socket.localPort {
t.Fatal("media edges did not keep independent transports")
}
if edgeA.transport == edgeB.transport {
t.Fatal("media edges shared one bandwidth estimator")
}
if _, err = engine.NewEdge(source, EdgeOptions{ConnectionID: "edge-c"}); !errors.Is(err, ErrSourceCapacity) {
t.Fatalf("third edge error = %v", err)
}
select {
case <-keyFrames:
case <-time.After(3 * time.Second):
t.Fatal("connected edge did not request a recovery frame")
}
accessUnit := []byte{
0, 0, 0, 1, 0x67, 0x42, 0xc0, 0x1f, 0x96, 0x54, 0x05, 0x01,
0, 0, 0, 1, 0x68, 0xce, 0x3c, 0x80,
0, 0, 0, 1, 0x65, 0x88, 0x84, 0x00,
}
if codec == "vp8" {
accessUnit = []byte{0x10, 0, 0, 0x9d, 0x01, 0x2a, 0x80, 0x02, 0xe0, 0x01, 0}
}
if err = writeSourceFrame(source, accessUnit, time.Second, time.Second/30); err != nil {
t.Fatal(err)
}
first := waitPacket(t, packetA)
_ = waitPacket(t, packetB)
if codec == "vp8" {
packet := &codecs.VP8Packet{}
payload, parseErr := packet.Unmarshal(first.Payload)
if parseErr != nil || string(payload) != string(accessUnit) || packet.S != 1 {
t.Fatalf("VP8 packetizer changed its encoded source: %v", parseErr)
}
}
// Verify feedback delivery independently of the upstream bootstrap throttle.
source.media.GetAllBuffers()[0].SetPLIThrottle(0)
for len(keyFrames) > 0 {
<-keyFrames
}
if err = receiverA.WriteRTCP([]rtcp.Packet{
&rtcp.PictureLossIndication{MediaSSRC: first.SSRC},
}); err != nil {
t.Fatal(err)
}
select {
case <-keyFrames:
case <-time.After(3 * time.Second):
t.Fatal("PLI did not reach the shared source")
}
if err = edgeA.Close(); err != nil {
t.Fatal(err)
}
edgeC, err := engine.NewEdge(source, EdgeOptions{ConnectionID: "edge-c"})
if err != nil {
t.Fatalf("released source slot was not reusable: %v", err)
}
_ = edgeC.Close()
}
func TestRemoteCandidatesAreBoundedUntilTheAnswer(t *testing.T) {
engine, err := NewEngine(EngineOptions{
BindAddress: "127.0.0.1:0",
IncludeLoopback: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("h264", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
edge, err := engine.NewEdge(source, EdgeOptions{ConnectionID: "edge"})
if err != nil {
t.Fatal(err)
}
if err = edge.AddRemoteCandidate(&webrtc.ICECandidateInit{
Candidate: "candidate:garbage",
}); err != nil {
t.Fatalf("malformed candidate escaped the edge boundary: %v", err)
}
if len(edge.pendingCandidates) != 0 {
t.Fatalf("malformed candidate was queued: %d", len(edge.pendingCandidates))
}
for index := 0; index < maxPendingCandidates; index++ {
if err = edge.AddRemoteCandidate(&webrtc.ICECandidateInit{
Candidate: "candidate:1 1 udp 1 127.0.0.1 9 typ host",
}); err != nil {
t.Fatalf("candidate %d: %v", index, err)
}
}
if err = edge.AddRemoteCandidate(&webrtc.ICECandidateInit{
Candidate: "candidate:1 1 udp 1 127.0.0.1 9 typ host",
}); err != nil {
t.Fatalf("excess candidate was not discarded: %v", err)
}
if len(edge.pendingCandidates) != maxPendingCandidates {
t.Fatalf("candidate queue grew beyond the bound: %d", len(edge.pendingCandidates))
}
}
func TestSelectedNatTraversalPathKeepsOnlyAnonymousProvenance(t *testing.T) {
if got := selectedNatTraversalPath("ordinary", "sp3"); got != "predicted" {
t.Fatalf("predicted path = %q", got)
}
if got := selectedNatTraversalPath("ordinary", "remote"); got != "ordinary" {
t.Fatalf("ordinary path = %q", got)
}
if got := selectedNatTraversalPath("", ""); got != "unknown" {
t.Fatalf("unknown path = %q", got)
}
}
func TestRepeatedAnswerIsIdempotent(t *testing.T) {
engine, err := NewEngine(EngineOptions{
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("h264", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = source.Close() })
edge, receiver, _ := connectedReceiver(t, engine, source, "repeat-answer")
t.Cleanup(func() { _ = receiver.Close() })
answer := receiver.LocalDescription()
if answer == nil {
t.Fatal("receiver did not retain its answer")
}
if err = edge.SetAnswer(*answer); err != nil {
t.Fatalf("repeated answer was not idempotent: %v", err)
}
}
func TestCaptureTimestampsDriveTheRTPClock(t *testing.T) {
engine, err := NewEngine(EngineOptions{
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("vp8", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = source.Close() })
_, receiver, packets := connectedReceiver(t, engine, source, "timed-edge")
t.Cleanup(func() { _ = receiver.Close() })
accessUnit := sfu.VP8KeyFrame8x8
start := 10 * time.Second
frameDuration := time.Second / 30
if err = writeSourceFrame(source, accessUnit, start, frameDuration); err != nil {
t.Fatal(err)
}
first := waitPacket(t, packets)
if err = writeSourceFrame(source, accessUnit, start+frameDuration, frameDuration); err != nil {
t.Fatal(err)
}
second := waitPacket(t, packets)
if got := second.Timestamp - first.Timestamp; got < 2_999 || got > 3_001 {
t.Fatalf("steady timestamp delta = %d", got)
}
if err = writeSourceFrame(source, accessUnit, start+5*time.Second, frameDuration); err != nil {
t.Fatal(err)
}
third := waitPacket(t, packets)
if got := third.Timestamp - second.Timestamp; got < 446_999 || got > 447_001 {
t.Fatalf("sparse timestamp delta = %d", got)
}
}
func TestNewCaptureGenerationKeepsTheRTPClockContinuous(t *testing.T) {
engine, err := NewEngine(EngineOptions{})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("vp8", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
_, receiver, packets := connectedReceiver(t, engine, source, "generation-edge")
t.Cleanup(func() { _ = receiver.Close() })
accessUnit := sfu.VP8KeyFrame8x8
frameDuration := time.Second / 30
if err = writeSourceFrame(source, accessUnit, 50*time.Second, frameDuration); err != nil {
t.Fatal(err)
}
first := waitPacket(t, packets)
source.BeginGeneration()
if err = writeSourceFrame(source, accessUnit, 100*time.Millisecond, frameDuration); err != nil {
t.Fatal(err)
}
second := waitPacket(t, packets)
if got := second.Timestamp - first.Timestamp; got < 2_999 || got > 3_001 {
t.Fatalf("generation timestamp delta = %d", got)
}
}
func TestOneLocalBridgeDoesNotConsumeRouteCapacity(t *testing.T) {
engine, err := NewEngine(EngineOptions{
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("h264", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
audio, err := engine.NewAudioSource(1, 128_000)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = audio.Close() })
local, err := engine.NewEdge(source, EdgeOptions{
ConnectionID: "local-preview", Local: true, Audio: audio,
})
if err != nil {
t.Fatal(err)
}
route, err := engine.NewEdge(source, EdgeOptions{
ConnectionID: "route-edge", Audio: audio,
})
if err != nil {
t.Fatal(err)
}
if _, err = engine.NewEdge(source, EdgeOptions{
ConnectionID: "second-local", Local: true,
}); !errors.Is(err, ErrSourceCapacity) {
t.Fatalf("second local edge error = %v", err)
}
if _, err = engine.NewEdge(source, EdgeOptions{
ConnectionID: "second-route",
}); !errors.Is(err, ErrSourceCapacity) {
t.Fatalf("second route edge error = %v", err)
}
_ = local.Close()
_ = route.Close()
if replacement, replaceErr := engine.NewEdge(source, EdgeOptions{
ConnectionID: "replacement-local", Local: true, Audio: audio,
}); replaceErr != nil {
t.Fatalf("released local slot was not reusable: %v", replaceErr)
} else {
_ = replacement.Close()
}
}
func TestBandwidthObserverReceivesTransportFeedbackWithoutPacing(t *testing.T) {
engine, err := NewEngine(EngineOptions{
BindAddress: "127.0.0.1:0", IncludeLoopback: true,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
source, err := engine.NewSource("h264", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
edge, receiver, packets := connectedReceiver(t, engine, source, "quality-edge")
t.Cleanup(func() { _ = receiver.Close() })
if !strings.Contains(edge.connection.LocalDescription().SDP, "transport-cc") {
t.Fatal("native media offer did not negotiate transport feedback")
}
source.SetFormat(0, 1280, 720)
started := time.Now()
if _, ok := edge.QualitySample(started); ok {
t.Fatal("first quality sample did not establish a baseline")
}
accessUnit := []byte{
0, 0, 0, 1, 0x67, 0x42, 0xc0, 0x1f, 0x96, 0x54, 0x05, 0x01,
0, 0, 0, 1, 0x68, 0xce, 0x3c, 0x80,
0, 0, 0, 1, 0x65, 0x88, 0x84, 0x00,
}
deadline := time.Now().Add(5 * time.Second)
captureTimestamp := time.Second
for time.Now().Before(deadline) {
if err = writeSourceFrame(source,
accessUnit,
captureTimestamp,
time.Second/30,
); err != nil {
t.Fatal(err)
}
captureTimestamp += time.Second / 30
select {
case <-packets:
default:
}
if bitrate, observed := edge.targetBitrate(); observed && time.Since(started) >= time.Second {
if bitrate <= 0 {
t.Fatalf("observed target bitrate = %d", bitrate)
}
sample, ok := edge.QualitySample(time.Now())
if !ok || sample.State != "healthy" || sample.Reason == nil ||
*sample.Reason != "none" || sample.IntervalFramesEncoded == 0 ||
sample.RTPStatsID == "" || sample.TrackIdentifier == "" {
t.Fatalf("quality sample = %+v, %v", sample, ok)
}
return
}
time.Sleep(time.Second / 30)
}
t.Fatal("transport feedback did not produce a bandwidth estimate")
}
func TestAudioUsesTheSamePeerConnectionAndCapacityAsVideo(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
video, err := engine.NewSource("h264", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
audio, err := engine.NewAudioSource(1, 128_000)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = video.Close(); _ = audio.Close() })
edge, err := engine.NewEdge(video, EdgeOptions{
ConnectionID: "audio-edge",
Audio: audio,
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = edge.Close() })
if edge.audioSender == nil {
t.Fatal("audio sender was not attached")
}
if _, err = engine.NewEdge(video, EdgeOptions{ConnectionID: "video-only"}); !errors.Is(err, ErrSourceCapacity) {
t.Fatalf("video capacity was not shared: %v", err)
}
if _, err = engine.NewEdge(video, EdgeOptions{ConnectionID: "audio-only", Audio: audio}); !errors.Is(err, ErrSourceCapacity) {
t.Fatalf("audio capacity was not shared: %v", err)
}
offer, err := edge.CreateOffer()
if err != nil {
t.Fatal(err)
}
if !strings.Contains(offer.SDP, "m=audio") || !strings.Contains(offer.SDP, "opus/48000") {
t.Fatalf("same connection offer has no Opus section: %s", offer.SDP)
}
}
func TestAudioSourceDeliversOpusOnTheVideoPeerConnection(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = engine.Close() })
video, err := engine.NewSource("h264", 1, 1, nil)
if err != nil {
t.Fatal(err)
}
audio, err := engine.NewAudioSource(1, 128_000)
if err != nil {
t.Fatal(err)
}
edge, err := engine.NewEdge(video, EdgeOptions{ConnectionID: "audio-edge", Audio: audio})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = edge.Close(); _ = video.Close(); _ = audio.Close() })
receiver := newAudioReceiver(t)
t.Cleanup(func() { _ = receiver.Close() })
audioPackets := make(chan *rtp.Packet, 1)
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
if track.Kind() != webrtc.RTPCodecTypeAudio {
return
}
packet, _, readErr := track.ReadRTP()
if readErr == nil {
audioPackets <- packet
}
})
connectEdgeToReceiver(t, edge, receiver)
if err = audio.WritePCM(make([]byte, nativeaudio.FrameBytes), 20*time.Millisecond); err != nil {
t.Fatal(err)
}
select {
case packet := <-audioPackets:
if packet.PayloadType != 111 || len(packet.Payload) == 0 {
t.Fatalf("unexpected Opus RTP packet: payload type %d, %d bytes", packet.PayloadType, len(packet.Payload))
}
case <-time.After(5 * time.Second):
t.Fatal("timed out waiting for Opus RTP")
}
}
func connectedReceiver(
t *testing.T,
engine *Engine,
source *Source,
connectionID string,
) (*Edge, *webrtc.PeerConnection, <-chan *rtp.Packet) {
t.Helper()
edge, err := engine.NewEdge(source, EdgeOptions{ConnectionID: connectionID})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = edge.Close() })
receiver := newReceiver(t)
packets := make(chan *rtp.Packet, 64)
receiver.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
go func() {
for {
packet, _, readErr := track.ReadRTP()
if readErr != nil {
return
}
select {
case packets <- packet:
default:
}
}
}()
})
connectEdgeToReceiver(t, edge, receiver)
return edge, receiver, packets
}
func connectEdgeToReceiver(
t *testing.T,
edge *Edge,
receiver *webrtc.PeerConnection,
) {
t.Helper()
gatherOffer := webrtc.GatheringCompletePromise(edge.connection)
if _, err := edge.CreateOffer(); err != nil {
t.Fatal(err)
}
waitSignal(t, gatherOffer, "offer ICE gathering")
if err := receiver.SetRemoteDescription(*edge.connection.LocalDescription()); err != nil {
t.Fatal(err)
}
gatherAnswer := webrtc.GatheringCompletePromise(receiver)
answer, err := receiver.CreateAnswer(nil)
if err != nil {
t.Fatal(err)
}
if err = receiver.SetLocalDescription(answer); err != nil {
t.Fatal(err)
}
waitSignal(t, gatherAnswer, "answer ICE gathering")
if err = edge.SetAnswer(*receiver.LocalDescription()); err != nil {
t.Fatal(err)
}
deadline := time.Now().Add(5 * time.Second)
for edge.State() != webrtc.PeerConnectionStateConnected {
if time.Now().After(deadline) {
t.Fatalf("edge state = %s", edge.State())
}
time.Sleep(10 * time.Millisecond)
}
}
func newReceiver(t *testing.T) *webrtc.PeerConnection {
return newReceiverWithAudio(t, false, "127.0.0.1:0")
}
func newAudioReceiver(t *testing.T) *webrtc.PeerConnection {
return newReceiverWithAudio(t, true, "127.0.0.1:0")
}
func newReceiverWithAudio(t *testing.T, includeAudio bool, bindAddress string) *webrtc.PeerConnection {
t.Helper()
mediaEngine := &webrtc.MediaEngine{}
for _, codec := range []string{"h264", "vp8"} {
if err := mediaEngine.RegisterCodec(videoCodecs[codec], webrtc.RTPCodecTypeVideo); err != nil {
t.Fatal(err)
}
}
if includeAudio {
if err := mediaEngine.RegisterCodec(webrtc.RTPCodecParameters{
RTPCodecCapability: opusCapability,
PayloadType: 111,
}, webrtc.RTPCodecTypeAudio); err != nil {
t.Fatal(err)
}
}
registry := &interceptor.Registry{}
if err := webrtc.ConfigureTWCCSender(mediaEngine, registry); err != nil {
t.Fatal(err)
}
if err := webrtc.RegisterDefaultInterceptors(mediaEngine, registry); err != nil {
t.Fatal(err)
}
settings := webrtc.SettingEngine{}
settings.SetIncludeLoopbackCandidate(true)
address, err := net.ResolveUDPAddr("udp", bindAddress)
if err != nil {
t.Fatal(err)
}
udp, err := net.ListenUDP("udp", address)
if err != nil {
t.Fatal(err)
}
mux := webrtc.NewICEUDPMux(logging.NewDefaultLoggerFactory().NewLogger("mediaedge-test"), udp)
settings.SetICEUDPMux(mux)
t.Cleanup(func() { _ = mux.Close() })
api := webrtc.NewAPI(
webrtc.WithMediaEngine(mediaEngine),
webrtc.WithInterceptorRegistry(registry),
webrtc.WithSettingEngine(settings),
)
connection, err := api.NewPeerConnection(webrtc.Configuration{})
if err != nil {
t.Fatal(err)
}
if includeAudio {
if _, err = connection.AddTransceiverFromKind(
webrtc.RTPCodecTypeAudio,
webrtc.RTPTransceiverInit{Direction: webrtc.RTPTransceiverDirectionRecvonly},
); err != nil {
t.Fatal(err)
}
}
return connection
}
func waitSignal(t *testing.T, signal <-chan struct{}, label string) {
t.Helper()
select {
case <-signal:
case <-time.After(5 * time.Second):
t.Fatalf("timed out waiting for %s", label)
}
}
func waitPacket(t *testing.T, packets <-chan *rtp.Packet) *rtp.Packet {
t.Helper()
select {
case packet := <-packets:
return packet
case <-time.After(5 * time.Second):
t.Fatal("timed out waiting for RTP")
return nil
}
}