This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections. This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work. ## Validation - Full local Web check: 79 files / 1,317 tests, TypeScript and production build. - Full Go checks, platform builds and Windows native capture regressions/probe. - Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup. - Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI. - Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved. ## Release notes ### 中文 这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。 #### 新增与改进 - 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。 - 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。 - 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。 - 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。 - 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。 - App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。 #### 修复 - 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。 - 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。 - 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。 - 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。 - 无站点密码时,连接恢复不再误引导用户输入站点密码。 - 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。 - 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。 #### 使用说明 下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。 ### English This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged. #### New and improved - Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs. - Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them. - Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date. - Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts. - Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues. - The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source. #### Fixes - Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling. - Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session. - Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections. - Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests. - Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase. - Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip. - Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449). #### Updating Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome. ## Attribution Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
230 lines
6.6 KiB
Go
230 lines
6.6 KiB
Go
package mediaedge
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"net"
|
|
"strings"
|
|
"sync"
|
|
"sync/atomic"
|
|
|
|
"github.com/TNTcraftHIM/Piik/internal/diagnostics"
|
|
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
|
|
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
|
|
"github.com/pion/interceptor"
|
|
"github.com/pion/rtcp"
|
|
"github.com/pion/webrtc/v4"
|
|
)
|
|
|
|
// Use the browser-standard spelling of Constrained Baseline in SDP. Pion's
|
|
// codec ordering prefers literal profile matches over equivalent constraint bits.
|
|
const H264ProfileLevelID = "42e033"
|
|
|
|
var h264Capability = webrtc.RTPCodecCapability{
|
|
MimeType: webrtc.MimeTypeH264,
|
|
ClockRate: 90_000,
|
|
SDPFmtpLine: "level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=" + H264ProfileLevelID,
|
|
RTCPFeedback: []webrtc.RTCPFeedback{
|
|
{Type: "goog-remb"},
|
|
{Type: webrtc.TypeRTCPFBTransportCC},
|
|
{Type: "ccm", Parameter: "fir"},
|
|
{Type: "nack"},
|
|
{Type: "nack", Parameter: "pli"},
|
|
},
|
|
}
|
|
|
|
var vp8Capability = webrtc.RTPCodecCapability{
|
|
MimeType: webrtc.MimeTypeVP8,
|
|
ClockRate: videoClockRate,
|
|
RTCPFeedback: h264Capability.RTCPFeedback,
|
|
}
|
|
|
|
var videoCodecs = map[string]webrtc.RTPCodecParameters{
|
|
"h264": {RTPCodecCapability: h264Capability, PayloadType: h264PayloadType},
|
|
"vp8": {RTPCodecCapability: vp8Capability, PayloadType: vp8PayloadType},
|
|
}
|
|
|
|
type EngineOptions struct {
|
|
BindAddress string
|
|
IncludeLoopback bool
|
|
PortMapping bool
|
|
InitialBitrate int
|
|
}
|
|
|
|
type Engine struct {
|
|
media *webrtc.MediaEngine
|
|
interceptors *interceptor.Registry
|
|
settings webrtc.SettingEngine
|
|
bindAddress string
|
|
portMapping bool
|
|
listenPacket func(string, string) (net.PacketConn, error)
|
|
initialBitrate int
|
|
ctx context.Context
|
|
cancel context.CancelFunc
|
|
|
|
mu sync.Mutex
|
|
edges map[*Edge]struct{}
|
|
publications map[*Publication]struct{}
|
|
sockets map[*iceSocket]struct{}
|
|
closed bool
|
|
}
|
|
|
|
func NewEngine(options EngineOptions) (*Engine, error) {
|
|
bindAddress := options.BindAddress
|
|
if bindAddress == "" {
|
|
bindAddress = ":0"
|
|
}
|
|
if _, err := net.ResolveUDPAddr("udp", bindAddress); err != nil {
|
|
return nil, errors.New("native media UDP bind address is invalid")
|
|
}
|
|
loggerFactory := diagnostics.PionLoggerFactory()
|
|
settingEngine := webrtc.SettingEngine{LoggerFactory: loggerFactory}
|
|
settingEngine.SetIncludeLoopbackCandidate(options.IncludeLoopback)
|
|
|
|
mediaEngine := &webrtc.MediaEngine{}
|
|
for _, codec := range []string{"h264", "vp8"} {
|
|
if err := mediaEngine.RegisterCodec(videoCodecs[codec], webrtc.RTPCodecTypeVideo); err != nil {
|
|
return nil, err
|
|
}
|
|
}
|
|
if err := mediaEngine.RegisterCodec(webrtc.RTPCodecParameters{
|
|
RTPCodecCapability: opusCapability,
|
|
PayloadType: 111,
|
|
}, webrtc.RTPCodecTypeAudio); err != nil {
|
|
return nil, err
|
|
}
|
|
registry := &interceptor.Registry{}
|
|
if err := webrtc.RegisterDefaultInterceptors(mediaEngine, registry); err != nil {
|
|
return nil, err
|
|
}
|
|
ctx, cancel := context.WithCancel(context.Background())
|
|
if options.InitialBitrate <= 0 {
|
|
options.InitialBitrate = defaultNativeSourceInitialBitrate
|
|
}
|
|
engine := &Engine{
|
|
media: mediaEngine,
|
|
interceptors: registry,
|
|
settings: settingEngine,
|
|
bindAddress: bindAddress,
|
|
listenPacket: net.ListenPacket,
|
|
portMapping: options.PortMapping,
|
|
initialBitrate: options.InitialBitrate,
|
|
ctx: ctx,
|
|
cancel: cancel,
|
|
edges: make(map[*Edge]struct{}),
|
|
publications: make(map[*Publication]struct{}),
|
|
sockets: make(map[*iceSocket]struct{}),
|
|
}
|
|
return engine, nil
|
|
}
|
|
|
|
func stunServers(servers []webrtc.ICEServer) []webrtc.ICEServer {
|
|
result := make([]webrtc.ICEServer, 0, len(servers))
|
|
for _, server := range servers {
|
|
urls := make([]string, 0, len(server.URLs))
|
|
for _, rawURL := range server.URLs {
|
|
if strings.HasPrefix(strings.ToLower(rawURL), "stun:") {
|
|
urls = append(urls, rawURL)
|
|
}
|
|
}
|
|
if len(urls) > 0 {
|
|
result = append(result, webrtc.ICEServer{URLs: urls})
|
|
}
|
|
}
|
|
return result
|
|
}
|
|
|
|
// NewSource coalesces recovery requests by physical output slot; [-1] requests
|
|
// all outputs. The callback runs outside the source lock and forwarding path.
|
|
func (engine *Engine) NewSource(codec string, capacity, layers int, requestKeyFrame func([]int)) (*Source, error) {
|
|
_, supported := videoCodecs[codec]
|
|
if !supported {
|
|
return nil, errors.New("native video codec is unsupported")
|
|
}
|
|
if capacity < 1 || capacity > 4 {
|
|
return nil, errors.New("native media source capacity is outside the route bound")
|
|
}
|
|
if layers < 1 || layers > 3 {
|
|
return nil, errors.New("native media source output count is invalid")
|
|
}
|
|
engine.mu.Lock()
|
|
defer engine.mu.Unlock()
|
|
if engine.closed {
|
|
return nil, errors.New("native media engine is closed")
|
|
}
|
|
source := &Source{
|
|
engine: engine, codec: codec, formats: make([]atomic.Uint64, layers),
|
|
capacity: capacity, requestKeyFrame: requestKeyFrame, edges: make(map[*Edge]bool),
|
|
publications: make(map[*Publication]struct{}),
|
|
}
|
|
media, err := forwarding.NewEncodedSource(forwarding.SourceOptions{
|
|
ID: "screen", StreamID: "piik-native", Codec: videoCodecs[codec],
|
|
Formats: make([]forwarding.LayerFormat, layers), MaxPackets: encoded.MaxPacketWindow,
|
|
OnRTCP: func(layer int, packets []rtcp.Packet) {
|
|
for _, packet := range packets {
|
|
switch packet.(type) {
|
|
case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest:
|
|
source.requestLayerKeyFrame(layer)
|
|
}
|
|
}
|
|
},
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
source.media = media
|
|
if requestKeyFrame != nil {
|
|
source.recoveryRequests = make(chan struct{}, 1)
|
|
source.recoveryStopped = make(chan struct{})
|
|
go source.runRecovery()
|
|
}
|
|
return source, nil
|
|
}
|
|
|
|
func (engine *Engine) register(edge *Edge) error {
|
|
engine.mu.Lock()
|
|
defer engine.mu.Unlock()
|
|
if engine.closed {
|
|
return errors.New("native media engine is closed")
|
|
}
|
|
engine.edges[edge] = struct{}{}
|
|
return nil
|
|
}
|
|
|
|
func (engine *Engine) remove(edge *Edge) {
|
|
engine.mu.Lock()
|
|
delete(engine.edges, edge)
|
|
engine.mu.Unlock()
|
|
}
|
|
|
|
func (engine *Engine) Close() error {
|
|
engine.mu.Lock()
|
|
if engine.closed {
|
|
engine.mu.Unlock()
|
|
return nil
|
|
}
|
|
engine.closed = true
|
|
engine.cancel()
|
|
edges := make([]*Edge, 0, len(engine.edges))
|
|
for edge := range engine.edges {
|
|
edges = append(edges, edge)
|
|
}
|
|
publications := make([]*Publication, 0, len(engine.publications))
|
|
for publication := range engine.publications {
|
|
publications = append(publications, publication)
|
|
}
|
|
sockets := make([]*iceSocket, 0, len(engine.sockets))
|
|
for socket := range engine.sockets {
|
|
sockets = append(sockets, socket)
|
|
}
|
|
engine.mu.Unlock()
|
|
closeEdges(edges)
|
|
for _, publication := range publications {
|
|
_ = publication.Close()
|
|
}
|
|
for _, socket := range sockets {
|
|
socket.close()
|
|
}
|
|
return nil
|
|
}
|