This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections. This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work. ## Validation - Full local Web check: 79 files / 1,317 tests, TypeScript and production build. - Full Go checks, platform builds and Windows native capture regressions/probe. - Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup. - Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI. - Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved. ## Release notes ### 中文 这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。 #### 新增与改进 - 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。 - 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。 - 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。 - 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。 - 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。 - App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。 #### 修复 - 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。 - 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。 - 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。 - 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。 - 无站点密码时,连接恢复不再误引导用户输入站点密码。 - 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。 - 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。 #### 使用说明 下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。 ### English This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged. #### New and improved - Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs. - Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them. - Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date. - Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts. - Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues. - The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source. #### Fixes - Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling. - Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session. - Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections. - Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests. - Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase. - Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip. - Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449). #### Updating Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome. ## Attribution Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
320 lines
10 KiB
Go
320 lines
10 KiB
Go
package mediaedge
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"log/slog"
|
|
"time"
|
|
|
|
"github.com/TNTcraftHIM/Piik/internal/diagnostics"
|
|
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
|
|
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
|
|
"github.com/pion/rtcp"
|
|
"github.com/pion/webrtc/v4"
|
|
)
|
|
|
|
// Transport owns current attachment; this map owns only requested membership.
|
|
// An independent keyframe installs a requested source without replacing its PC.
|
|
type groupConsumer interface {
|
|
CurrentSource() *forwarding.Source
|
|
ReplaceSource(*forwarding.Source) error
|
|
}
|
|
|
|
type outputGroup struct {
|
|
slot int
|
|
media *forwarding.EncodedSource
|
|
budget uint32
|
|
active bool
|
|
start time.Duration
|
|
}
|
|
|
|
type outputDemand struct {
|
|
consumer groupConsumer
|
|
budget uint32
|
|
active bool
|
|
lower bool
|
|
original bool
|
|
}
|
|
|
|
func (source *Source) groupForMedia(media *forwarding.Source) *outputGroup {
|
|
for _, group := range source.groups {
|
|
if group.media.Source == media {
|
|
return group
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (source *Source) collectDemands(edges []*Edge, publications []*Publication) []outputDemand {
|
|
demands := make([]outputDemand, 0, len(edges)+len(publications))
|
|
ceiling := source.media.TrackInfo().Layers[0].Bitrate
|
|
for _, edge := range edges {
|
|
state := edge.transport.Output.State()
|
|
_, observed := edge.transport.TargetBitrate()
|
|
active := edge.State() == webrtc.PeerConnectionStateConnected && state.VideoBudget > 0
|
|
lower := active && !edge.local && observed &&
|
|
(state.Target == 0 || state.Current == 0 && !state.Paused || state.Paused && state.Prepare == 0)
|
|
demands = append(demands, outputDemand{consumer: edge.transport, active: active, lower: lower,
|
|
original: edge.transport.RequiredActiveCount() > 1,
|
|
budget: uint32(max(1000, min(state.VideoBudget, int64(ceiling))))})
|
|
}
|
|
for _, publication := range publications {
|
|
required := publication.RequiredActiveCount()
|
|
active := required > 0
|
|
budget := publication.LowestLayerBudget()
|
|
if budget <= 0 {
|
|
budget = int64(ceiling)
|
|
}
|
|
demands = append(demands, outputDemand{consumer: publication.transport, active: active, lower: active,
|
|
original: required > 1,
|
|
budget: uint32(max(1000, min(budget, int64(ceiling))))})
|
|
}
|
|
return demands
|
|
}
|
|
|
|
// Equal connection-level requests share one correction/adaptation owner. The
|
|
// group's RTP-to-codec conversion is applied once, not used as a membership key.
|
|
func (source *Source) planGroups(demands []outputDemand) (OutputPlan, error) {
|
|
plan := OutputPlan{Active: make([]bool, len(source.outputBitrates)),
|
|
Bitrates: append([]uint32(nil), source.outputBitrates...)}
|
|
if len(source.formats) != 2 || len(plan.Bitrates) < 2 {
|
|
return plan, errors.New("shared output groups need an original and derived slot")
|
|
}
|
|
if source.groups == nil {
|
|
source.groups = map[int]*outputGroup{0: {slot: 0, media: source.media, budget: source.outputBitrates[0]}}
|
|
source.memberships = make(map[groupConsumer]*outputGroup)
|
|
}
|
|
present := make(map[groupConsumer]bool, len(demands))
|
|
hasLowerDemand := false
|
|
for _, demand := range demands {
|
|
present[demand.consumer] = true
|
|
hasLowerDemand = hasLowerDemand || demand.active && demand.lower
|
|
plan.Active[1] = plan.Active[1] || demand.active && demand.original
|
|
}
|
|
for consumer := range source.memberships {
|
|
if !present[consumer] {
|
|
delete(source.memberships, consumer)
|
|
}
|
|
}
|
|
compatible := func(group *outputGroup, demand outputDemand) bool {
|
|
for _, other := range demands {
|
|
if other.active && other.lower && other.budget != demand.budget &&
|
|
(other.consumer.CurrentSource() == group.media.Source || source.memberships[other.consumer] == group) {
|
|
return false
|
|
}
|
|
}
|
|
return true
|
|
}
|
|
// Revalidate pending joins before they can prevent an incumbent from
|
|
// adjusting in place. Membership order must not give stale demand priority.
|
|
for _, demand := range demands {
|
|
group := source.memberships[demand.consumer]
|
|
if demand.active && demand.lower && group != nil && demand.consumer.CurrentSource() != group.media.Source && !compatible(group, demand) {
|
|
delete(source.memberships, demand.consumer)
|
|
}
|
|
}
|
|
// The previous frame's active bit alone may outlive its last handoff.
|
|
inUse := func(group *outputGroup) bool {
|
|
if group == nil || !group.active {
|
|
return false
|
|
}
|
|
for _, demand := range demands {
|
|
if demand.active && (demand.consumer.CurrentSource() == group.media.Source || source.memberships[demand.consumer] == group) {
|
|
return true
|
|
}
|
|
}
|
|
return false
|
|
}
|
|
for _, demand := range demands {
|
|
if !demand.active || !demand.lower {
|
|
if source.memberships[demand.consumer] != nil {
|
|
source.groupRecovery.Or(2)
|
|
}
|
|
source.memberships[demand.consumer] = nil
|
|
continue
|
|
}
|
|
var selected *outputGroup
|
|
for slot := 0; slot < len(source.outputBitrates); slot++ {
|
|
group := source.groups[slot]
|
|
if group != nil && source.outputBitrates[slot] > 0 && group.budget == demand.budget {
|
|
if demand.consumer.CurrentSource() != group.media.Source && !compatible(group, demand) {
|
|
continue
|
|
}
|
|
if selected == nil || inUse(group) {
|
|
selected = group
|
|
}
|
|
if inUse(group) {
|
|
break
|
|
}
|
|
}
|
|
}
|
|
if selected == nil || !inUse(selected) {
|
|
// A group's rates can change in place only when no other current or
|
|
// requested member still needs a different budget.
|
|
owned := []*outputGroup{source.memberships[demand.consumer], source.groupForMedia(demand.consumer.CurrentSource())}
|
|
var ordered []*outputGroup
|
|
for _, group := range owned {
|
|
if inUse(group) {
|
|
ordered = append(ordered, group)
|
|
}
|
|
}
|
|
// Keep an owned live pipeline before reviving an inactive cached match.
|
|
ordered = append(ordered, selected)
|
|
ordered = append(ordered, owned...)
|
|
for slot := 0; slot < len(source.outputBitrates); slot++ {
|
|
if group := source.groups[slot]; group != nil {
|
|
ordered = append(ordered, group)
|
|
}
|
|
}
|
|
for _, group := range ordered {
|
|
if group == nil {
|
|
continue
|
|
}
|
|
if source.outputBitrates[group.slot] == 0 {
|
|
continue
|
|
}
|
|
if compatible(group, demand) {
|
|
selected = group
|
|
break
|
|
}
|
|
}
|
|
}
|
|
if selected == nil {
|
|
for slot := 2; slot < len(source.outputBitrates); slot++ {
|
|
if source.groups[slot] == nil && source.outputBitrates[slot] > 0 {
|
|
var err error
|
|
selected, err = source.newGroup(slot)
|
|
if err != nil {
|
|
return plan, err
|
|
}
|
|
break
|
|
}
|
|
}
|
|
}
|
|
if selected == nil {
|
|
// Existing delivery remains while a prepared split releases its old
|
|
// membership. No timer, retry queue or temporary capacity exemption.
|
|
continue
|
|
}
|
|
if selected.budget != demand.budget || source.memberships[demand.consumer] != selected {
|
|
if slog.Default().Enabled(source.engine.ctx, slog.LevelDebug) {
|
|
slog.Debug("encoding-group", "event", "demand", "slot", selected.slot, "budget", demand.budget,
|
|
"sourceId", diagnostics.ID(fmt.Sprintf("%p", source)), "rtcPeerId", groupConsumerID(demand.consumer))
|
|
}
|
|
}
|
|
selected.budget = demand.budget
|
|
if !selected.active {
|
|
selected.start = source.inputPTS
|
|
}
|
|
if source.memberships[demand.consumer] != selected && demand.consumer.CurrentSource() != selected.media.Source {
|
|
source.groupRecovery.Or(uint32(1) << selected.slot)
|
|
}
|
|
source.memberships[demand.consumer] = selected
|
|
}
|
|
for _, group := range source.groups {
|
|
active := group.slot == 0 && plan.Active[1] && !hasLowerDemand
|
|
for _, demand := range demands {
|
|
if !demand.active {
|
|
continue
|
|
}
|
|
if source.memberships[demand.consumer] == group ||
|
|
demand.consumer.CurrentSource() == group.media.Source && (demand.lower || !demand.original || group.slot != 0) {
|
|
active = true
|
|
}
|
|
}
|
|
nextActive := active && source.outputBitrates[group.slot] > 0
|
|
if group.active != nextActive {
|
|
slog.Debug("encoding-group", "event", "activity", "slot", group.slot, "active", nextActive, "sourceId", diagnostics.ID(fmt.Sprintf("%p", source)))
|
|
}
|
|
group.active = nextActive
|
|
plan.Active[group.slot] = group.active
|
|
if group.active {
|
|
plan.Bitrates[group.slot] = uint32(max(1000, min(group.media.CodecBudget(0, int64(group.budget)), int64(source.outputBitrates[group.slot]))))
|
|
}
|
|
}
|
|
return plan, nil
|
|
}
|
|
|
|
func (source *Source) newGroup(slot int) (*outputGroup, error) {
|
|
info := source.media.TrackInfo()
|
|
formats := make([]forwarding.LayerFormat, len(info.Layers))
|
|
for layer, format := range info.Layers {
|
|
formats[layer] = forwarding.LayerFormat{Width: format.Width, Height: format.Height, Bitrate: format.Bitrate}
|
|
}
|
|
formats[0].Bitrate = source.outputBitrates[slot]
|
|
media, err := forwarding.NewEncodedSource(forwarding.SourceOptions{
|
|
ID: string(source.media.TrackID()), StreamID: source.media.StreamID(), Codec: videoCodecs[source.codec],
|
|
Formats: formats, MaxPackets: encoded.MaxPacketWindow,
|
|
OnRTCP: func(layer int, packets []rtcp.Packet) {
|
|
for _, packet := range packets {
|
|
switch packet.(type) {
|
|
case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest:
|
|
if layer == 0 {
|
|
source.requestLayerKeyFrame(slot)
|
|
} else {
|
|
source.requestLayerKeyFrame(1)
|
|
}
|
|
}
|
|
}
|
|
},
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
media.SetLowestLayerRateControlled(true)
|
|
group := &outputGroup{slot: slot, media: media, start: source.inputPTS}
|
|
if source.hasInput {
|
|
if err = media.BeginFrame(source.inputPTS, source.inputAt); err != nil {
|
|
media.Close()
|
|
return nil, err
|
|
}
|
|
}
|
|
source.groups[slot] = group
|
|
return group, nil
|
|
}
|
|
|
|
func (source *Source) installGroup(group *outputGroup, original bool) {
|
|
next := source.media.Source
|
|
if group != nil {
|
|
next = group.media.Source
|
|
}
|
|
for consumer, wanted := range source.memberships {
|
|
if wanted == group && (original || group != nil) && consumer.CurrentSource() != next {
|
|
err := consumer.ReplaceSource(next)
|
|
if slog.Default().Enabled(source.engine.ctx, slog.LevelDebug) {
|
|
slot := -1
|
|
if group != nil {
|
|
slot = group.slot
|
|
}
|
|
slog.Debug("encoding-group", "event", "attachment", "slot", slot, "original", original,
|
|
"sourceId", diagnostics.ID(fmt.Sprintf("%p", source)), "rtcPeerId", groupConsumerID(consumer), diagnostics.Error(err))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
func groupConsumerID(consumer groupConsumer) string {
|
|
switch consumer := consumer.(type) {
|
|
case *forwarding.Transport:
|
|
return diagnostics.ID(consumer.PC.ID())
|
|
case *forwarding.Publication:
|
|
return diagnostics.ID(consumer.PC.ID())
|
|
}
|
|
return ""
|
|
}
|
|
|
|
func (source *Source) beginGroupFrame(pts time.Duration, at time.Time) error {
|
|
if err := source.media.BeginFrame(pts, at); err != nil {
|
|
return err
|
|
}
|
|
source.inputPTS, source.inputAt, source.hasInput = pts, at, true
|
|
for slot, group := range source.groups {
|
|
if slot != 0 {
|
|
if err := group.media.BeginFrame(pts, at); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
}
|
|
return nil
|
|
}
|