Files
Piik/internal/app/mediaedge/groups.go
T
AAAandPedro Antonio c18502fd18 feat: improve microphone, chat and platform support
This release adds microphone processing controls and local chat controls, expands platform packages, and synchronizes the website and guides. It also fixes native H.264 input/format handling, signaling recovery, fresh STUN discovery, and UDP ownership during overlapping connections.

This is **v1.8.0**. Browser/Server v23, Native control v9 and capture v7 remain unchanged, with the existing 1440p ceiling. Additional resolutions, capability negotiation and Magicsock are separate follow-up work.

## Validation

- Full local Web check: 79 files / 1,317 tests, TypeScript and production build.
- Full Go checks, platform builds and Windows native capture regressions/probe.
- Native H.264 Host lifecycle, Browser-to-App fanout and App-exit recovery, and Native H.264 through embedded SFU: all passed with owned process/port/profile cleanup.
- Website build, documentation and repository hygiene passed. Existing UI acceptance covers narrow screens, both languages/themes, architecture downloads and all 23 film checks; final packaging and Go race checks run on the merged revision in CI.
- Physical macOS/Linux capture, representative public networks and unmatched field reports retain their documented evidence limits. No claim that every blur/disconnection report is resolved.

## Release notes

### 中文

这是一次功能与兼容性更新:麦克风和聊天增加了实用的小选项,App 与 Server 支持更多设备,同时修复了原生分享和连接恢复中的若干问题。现有 1.x 协议保持不变。

#### 新增与改进

- 浏览器麦克风新增「降噪与回声消除」开关,默认开启回声消除、降噪和自动增益;分享音乐或使用虚拟音频输入时,可按需关闭。
- 可调整弹幕字号和透明度,并将当前保留的聊天记录导出为 TXT。当前会话最多保留 1,000 条文字消息,刷新页面后清空;需要保存时请先导出。
- 新增 Intel Mac、Linux ARM64 App,以及 Linux ARM64 Server 和容器镜像。官网下载区可直接选择对应架构,并显示最新版本和发布时间。
- 更新官网演示、README 截图和中英文教程,明确浏览器分享也可选择窗口或屏幕;开始分享前显示房间号占位,保持底栏控件位置稳定。
- 官网新增直达问题排查的入口,补充浏览器 WebRTC 限制、校园网和 Windows 显卡相关的排查步骤。
- App 模式选择页显示免费开源说明、作者与贡献者署名,以及官方网站和源码链接。

#### 修复

- 修正部分 H.264 原生分享的格式兼容性校验,以及 Windows 编码器输入长度和输出缓冲区对齐问题。
- 清理已失效的重连任务和 Native 接收信令,避免旧连接继续影响当前会话。
- 修复 App 重连时可能复用过期公网地址的问题;重新探测使用当前地址,并处理探测丢包和过期响应,保留其他正常连接。
- 修复 App 新旧媒体连接重叠时共用端口造成的冲突,避免准备或撤销新连接打断仍在使用的画面;关闭连接时及时取消未完成的端口映射请求。
- 无站点密码时,连接恢复不再误引导用户输入站点密码。
- 房间失效或会话终止时,开播前后的原因提示统一显示在电视状态图标中,避免额外提示条。
- 修正 Linux 原生采集的时间戳与关键帧时间基准。感谢 Pedro Antonio 在 [#449](https://github.com/TNTcraftHIM/Piik/pull/449) 的贡献。

#### 使用说明

下载对应平台的完整压缩包,退出旧 App 后解压使用。Windows 继续提供 x64;macOS 提供 Apple 芯片与 Intel 版本,原生采集需要 macOS 13 或更新版本。Linux 提供 x64 与 ARM64,原生采集还需安装[系统组件](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md)。macOS 和 Linux 的实机覆盖仍有限,欢迎反馈。

### English

This update adds useful microphone and chat controls, brings Piik to more devices, and fixes several native-sharing and connection-recovery issues. The existing 1.x protocols remain unchanged.

#### New and improved

- Browser microphones now have a “Noise & echo reduction” toggle. Echo cancellation, noise suppression and automatic gain control are on by default; turn them off when appropriate for music or virtual audio inputs.
- Adjust chat-overlay text size and opacity, and export retained messages as TXT. The current session keeps up to 1,000 text messages and clears on page refresh; export first if you want to keep them.
- Added Intel Mac and Linux ARM64 App packages, plus Linux ARM64 Server and container images. The website provides architecture selectors and shows the latest release and publication date.
- Updated the website demonstration, README screenshots and bilingual guides, clarifying that browser capture can also select windows or screens. A room-code placeholder keeps the bottom bar stable before sharing starts.
- Added direct website links to troubleshooting, with steps for browser WebRTC restrictions, campus networks and Windows graphics issues.
- The App mode-selection page now shows its free and open-source status, author and contributor credit, and links to the official website and source.

#### Fixes

- Corrected H.264 format compatibility checks and Windows encoder input-length and output-buffer alignment handling.
- Retired stale reconnect tasks and Native receive signals so they cannot continue affecting the current session.
- Fixed App reconnection reusing an expired public address. Fresh discovery handles lost probes and stale responses while preserving other working connections.
- Fixed UDP socket conflicts between overlapping App media connections so preparing or canceling a replacement preserves the active picture. Closing a connection now cancels unfinished gateway mapping requests.
- Connection recovery on passwordless sites no longer incorrectly asks for a site passphrase.
- Expired-room and session-termination reasons now use the same TV status indicator before and during sharing, without an extra notice strip.
- Corrected Linux native capture timestamps and keyframe time bases. Thanks to Pedro Antonio for [#449](https://github.com/TNTcraftHIM/Piik/pull/449).

#### Updating

Download the complete archive for your platform, quit the old App, and extract the new package. Windows remains x64-only. macOS packages support Apple silicon and Intel; native capture requires macOS 13 or newer. Linux supports x64 and ARM64 and requires additional [system components](https://github.com/TNTcraftHIM/Piik/blob/main/native/capture/linux/README.md) for native capture. Physical-device coverage on macOS and Linux remains limited; feedback is welcome.

## Attribution

Co-authored-by: Pedro Antonio <wrenchwastaken@gmail.com>
2026-10-04 16:21:01 +08:00

320 lines
10 KiB
Go

package mediaedge
import (
"errors"
"fmt"
"log/slog"
"time"
"github.com/TNTcraftHIM/Piik/internal/diagnostics"
"github.com/TNTcraftHIM/Piik/internal/media/encoded"
"github.com/TNTcraftHIM/Piik/internal/media/forwarding"
"github.com/pion/rtcp"
"github.com/pion/webrtc/v4"
)
// Transport owns current attachment; this map owns only requested membership.
// An independent keyframe installs a requested source without replacing its PC.
type groupConsumer interface {
CurrentSource() *forwarding.Source
ReplaceSource(*forwarding.Source) error
}
type outputGroup struct {
slot int
media *forwarding.EncodedSource
budget uint32
active bool
start time.Duration
}
type outputDemand struct {
consumer groupConsumer
budget uint32
active bool
lower bool
original bool
}
func (source *Source) groupForMedia(media *forwarding.Source) *outputGroup {
for _, group := range source.groups {
if group.media.Source == media {
return group
}
}
return nil
}
func (source *Source) collectDemands(edges []*Edge, publications []*Publication) []outputDemand {
demands := make([]outputDemand, 0, len(edges)+len(publications))
ceiling := source.media.TrackInfo().Layers[0].Bitrate
for _, edge := range edges {
state := edge.transport.Output.State()
_, observed := edge.transport.TargetBitrate()
active := edge.State() == webrtc.PeerConnectionStateConnected && state.VideoBudget > 0
lower := active && !edge.local && observed &&
(state.Target == 0 || state.Current == 0 && !state.Paused || state.Paused && state.Prepare == 0)
demands = append(demands, outputDemand{consumer: edge.transport, active: active, lower: lower,
original: edge.transport.RequiredActiveCount() > 1,
budget: uint32(max(1000, min(state.VideoBudget, int64(ceiling))))})
}
for _, publication := range publications {
required := publication.RequiredActiveCount()
active := required > 0
budget := publication.LowestLayerBudget()
if budget <= 0 {
budget = int64(ceiling)
}
demands = append(demands, outputDemand{consumer: publication.transport, active: active, lower: active,
original: required > 1,
budget: uint32(max(1000, min(budget, int64(ceiling))))})
}
return demands
}
// Equal connection-level requests share one correction/adaptation owner. The
// group's RTP-to-codec conversion is applied once, not used as a membership key.
func (source *Source) planGroups(demands []outputDemand) (OutputPlan, error) {
plan := OutputPlan{Active: make([]bool, len(source.outputBitrates)),
Bitrates: append([]uint32(nil), source.outputBitrates...)}
if len(source.formats) != 2 || len(plan.Bitrates) < 2 {
return plan, errors.New("shared output groups need an original and derived slot")
}
if source.groups == nil {
source.groups = map[int]*outputGroup{0: {slot: 0, media: source.media, budget: source.outputBitrates[0]}}
source.memberships = make(map[groupConsumer]*outputGroup)
}
present := make(map[groupConsumer]bool, len(demands))
hasLowerDemand := false
for _, demand := range demands {
present[demand.consumer] = true
hasLowerDemand = hasLowerDemand || demand.active && demand.lower
plan.Active[1] = plan.Active[1] || demand.active && demand.original
}
for consumer := range source.memberships {
if !present[consumer] {
delete(source.memberships, consumer)
}
}
compatible := func(group *outputGroup, demand outputDemand) bool {
for _, other := range demands {
if other.active && other.lower && other.budget != demand.budget &&
(other.consumer.CurrentSource() == group.media.Source || source.memberships[other.consumer] == group) {
return false
}
}
return true
}
// Revalidate pending joins before they can prevent an incumbent from
// adjusting in place. Membership order must not give stale demand priority.
for _, demand := range demands {
group := source.memberships[demand.consumer]
if demand.active && demand.lower && group != nil && demand.consumer.CurrentSource() != group.media.Source && !compatible(group, demand) {
delete(source.memberships, demand.consumer)
}
}
// The previous frame's active bit alone may outlive its last handoff.
inUse := func(group *outputGroup) bool {
if group == nil || !group.active {
return false
}
for _, demand := range demands {
if demand.active && (demand.consumer.CurrentSource() == group.media.Source || source.memberships[demand.consumer] == group) {
return true
}
}
return false
}
for _, demand := range demands {
if !demand.active || !demand.lower {
if source.memberships[demand.consumer] != nil {
source.groupRecovery.Or(2)
}
source.memberships[demand.consumer] = nil
continue
}
var selected *outputGroup
for slot := 0; slot < len(source.outputBitrates); slot++ {
group := source.groups[slot]
if group != nil && source.outputBitrates[slot] > 0 && group.budget == demand.budget {
if demand.consumer.CurrentSource() != group.media.Source && !compatible(group, demand) {
continue
}
if selected == nil || inUse(group) {
selected = group
}
if inUse(group) {
break
}
}
}
if selected == nil || !inUse(selected) {
// A group's rates can change in place only when no other current or
// requested member still needs a different budget.
owned := []*outputGroup{source.memberships[demand.consumer], source.groupForMedia(demand.consumer.CurrentSource())}
var ordered []*outputGroup
for _, group := range owned {
if inUse(group) {
ordered = append(ordered, group)
}
}
// Keep an owned live pipeline before reviving an inactive cached match.
ordered = append(ordered, selected)
ordered = append(ordered, owned...)
for slot := 0; slot < len(source.outputBitrates); slot++ {
if group := source.groups[slot]; group != nil {
ordered = append(ordered, group)
}
}
for _, group := range ordered {
if group == nil {
continue
}
if source.outputBitrates[group.slot] == 0 {
continue
}
if compatible(group, demand) {
selected = group
break
}
}
}
if selected == nil {
for slot := 2; slot < len(source.outputBitrates); slot++ {
if source.groups[slot] == nil && source.outputBitrates[slot] > 0 {
var err error
selected, err = source.newGroup(slot)
if err != nil {
return plan, err
}
break
}
}
}
if selected == nil {
// Existing delivery remains while a prepared split releases its old
// membership. No timer, retry queue or temporary capacity exemption.
continue
}
if selected.budget != demand.budget || source.memberships[demand.consumer] != selected {
if slog.Default().Enabled(source.engine.ctx, slog.LevelDebug) {
slog.Debug("encoding-group", "event", "demand", "slot", selected.slot, "budget", demand.budget,
"sourceId", diagnostics.ID(fmt.Sprintf("%p", source)), "rtcPeerId", groupConsumerID(demand.consumer))
}
}
selected.budget = demand.budget
if !selected.active {
selected.start = source.inputPTS
}
if source.memberships[demand.consumer] != selected && demand.consumer.CurrentSource() != selected.media.Source {
source.groupRecovery.Or(uint32(1) << selected.slot)
}
source.memberships[demand.consumer] = selected
}
for _, group := range source.groups {
active := group.slot == 0 && plan.Active[1] && !hasLowerDemand
for _, demand := range demands {
if !demand.active {
continue
}
if source.memberships[demand.consumer] == group ||
demand.consumer.CurrentSource() == group.media.Source && (demand.lower || !demand.original || group.slot != 0) {
active = true
}
}
nextActive := active && source.outputBitrates[group.slot] > 0
if group.active != nextActive {
slog.Debug("encoding-group", "event", "activity", "slot", group.slot, "active", nextActive, "sourceId", diagnostics.ID(fmt.Sprintf("%p", source)))
}
group.active = nextActive
plan.Active[group.slot] = group.active
if group.active {
plan.Bitrates[group.slot] = uint32(max(1000, min(group.media.CodecBudget(0, int64(group.budget)), int64(source.outputBitrates[group.slot]))))
}
}
return plan, nil
}
func (source *Source) newGroup(slot int) (*outputGroup, error) {
info := source.media.TrackInfo()
formats := make([]forwarding.LayerFormat, len(info.Layers))
for layer, format := range info.Layers {
formats[layer] = forwarding.LayerFormat{Width: format.Width, Height: format.Height, Bitrate: format.Bitrate}
}
formats[0].Bitrate = source.outputBitrates[slot]
media, err := forwarding.NewEncodedSource(forwarding.SourceOptions{
ID: string(source.media.TrackID()), StreamID: source.media.StreamID(), Codec: videoCodecs[source.codec],
Formats: formats, MaxPackets: encoded.MaxPacketWindow,
OnRTCP: func(layer int, packets []rtcp.Packet) {
for _, packet := range packets {
switch packet.(type) {
case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest:
if layer == 0 {
source.requestLayerKeyFrame(slot)
} else {
source.requestLayerKeyFrame(1)
}
}
}
},
})
if err != nil {
return nil, err
}
media.SetLowestLayerRateControlled(true)
group := &outputGroup{slot: slot, media: media, start: source.inputPTS}
if source.hasInput {
if err = media.BeginFrame(source.inputPTS, source.inputAt); err != nil {
media.Close()
return nil, err
}
}
source.groups[slot] = group
return group, nil
}
func (source *Source) installGroup(group *outputGroup, original bool) {
next := source.media.Source
if group != nil {
next = group.media.Source
}
for consumer, wanted := range source.memberships {
if wanted == group && (original || group != nil) && consumer.CurrentSource() != next {
err := consumer.ReplaceSource(next)
if slog.Default().Enabled(source.engine.ctx, slog.LevelDebug) {
slot := -1
if group != nil {
slot = group.slot
}
slog.Debug("encoding-group", "event", "attachment", "slot", slot, "original", original,
"sourceId", diagnostics.ID(fmt.Sprintf("%p", source)), "rtcPeerId", groupConsumerID(consumer), diagnostics.Error(err))
}
}
}
}
func groupConsumerID(consumer groupConsumer) string {
switch consumer := consumer.(type) {
case *forwarding.Transport:
return diagnostics.ID(consumer.PC.ID())
case *forwarding.Publication:
return diagnostics.ID(consumer.PC.ID())
}
return ""
}
func (source *Source) beginGroupFrame(pts time.Duration, at time.Time) error {
if err := source.media.BeginFrame(pts, at); err != nil {
return err
}
source.inputPTS, source.inputAt, source.hasInput = pts, at, true
for slot, group := range source.groups {
if slot != 0 {
if err := group.media.BeginFrame(pts, at); err != nil {
return err
}
}
}
return nil
}