diff --git a/docs/方案设计.md b/docs/方案设计.md index 12d1660..a7a4c52 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -133,9 +133,9 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate | 1080p30 | 1920x1080、30 fps | 5 Mbps/peer | | 720p30 | 1280x720、30 fps | 3 Mbps/peer | -浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际结果。视频 track 保留 `contentHint = "motion"`；sender 尝试设置 `maxBitrate`、`maxFramerate` 与 `degradationPreference = "balanced"`。`balanced` 只允许浏览器在帧率和分辨率之间权衡，不保证清晰度优先；具体 Chromium、Firefox 与 Safari 行为仍需实测。不支持 sender 参数时只记录警告。 +浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际捕获结果。视频 track 保留 `contentHint = "motion"`；待合并质量控制默认使用 `degradationPreference = "maintain-resolution"`，三个推荐档位都采用清晰优先。折叠高级区将设置严格限制为 720p/1080p/1440p、15 至 60 整数 FPS、2 至 12 Mbps，以及 `maintain-resolution`、`balanced`、`maintain-framerate` 三种偏好。它们只是 ceiling 和浏览器偏好，不保证实际分辨率、帧率或码率。 -生产反馈显示三个档位都可能被降到难以阅读的分辨率、帧率和码率。下一质量迭代先在同一高动态场景对 1、2、3 名观看者做一次 60 至 90 秒采样，并对 `maintain-resolution` 与当前 `balanced` 做 A/B；默认目标是清晰优先，高级区再提供清晰、平衡和流畅三种偏好。不得只提高 ceiling 或强制 codec。只有实际/target bitrate 长期顶住 ceiling、网络仍有余量且画质仍不足时，才增加更高的可选码率上限。 +每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新，然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。拒绝或浏览器改写必须进入可见状态，不能只写 `console.warn`。生产版本仍使用 `balanced`，用户报告的严重降级尚未完成受控分类；本改动只提供有界 clarity/balanced 控制，不得据此宣称画质提升。下一次真实质量验收仍需在同一高动态场景采样 1、2、3 名观看者和 TURN 路径。不得只提高 ceiling、强制 codec 或修改 SDP。 PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 codec、`encoderImplementation` 和 `powerEfficientEncoder`，再依据目标机器测量决定是否优先 H.264。这样避免静态 codec 偏好意外触发软件编码。 @@ -143,7 +143,7 @@ PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`，不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流；成功后为每个现有 sender 调用 `replaceTrack()`，健康连接不重新协商。初始 offer 预留 send-only audio transceiver，使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败，只重建该 peer；新流提交后才停止旧流，并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置，避免把不同 track 的计数器混算。 -直播中切换质量档位不重新打开选源器：对当前视频 track 调用 `applyConstraints()`，再更新所有现有 sender 的参数；新加入或重建的 sender 使用当前档位。失败时保留当前分享并给出状态，不以重建整个房间补偿。临时暂停只禁用当前视频 track，使远端收到黑画面；音频、房间和连接保持不变，恢复时重新启用同一 track。 +直播中切换质量不重新打开选源器：对当前视频 track 调用 `applyConstraints()`，再更新所有现有 sender 的参数；新加入、重建的 relay sender 和启用的 SFU publisher 使用同一权威设置。失败时保留当前分享并给出状态，不以重建整个房间补偿。临时暂停只禁用当前视频 track，使远端收到黑画面；音频、房间和连接保持不变，恢复时重新启用同一 track。浏览器标准 API 无法独立保证音频 stereo/bitrate 与最终协商 codec，因此当前高级区不暴露对应控件。 `frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限；浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据，但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机，先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。 @@ -171,7 +171,7 @@ TURN/TLS 是可选能力，不是生产启动门槛。启用时默认使用 RFC 每两秒在本地采集一次 `getStats()`。首版不上传原始统计，也不展示候选地址。 -质量诊断必须能按顺序区分：捕获源设置、sender 实际参数、CPU 或带宽 `qualityLimitationReason`、selected candidate/TURN 路径、outbound 与 inbound 差异，以及接收端 dropped/freeze。每次成功调用 `setParameters()` 后读回并记录实际 `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 与 degradation preference；参数拒绝不能只留在控制台。首轮使用浏览器 WebRTC internals 导出和一份脱敏 run manifest，不新增服务端遥测、综合评分器或自动调档状态机。 +质量诊断必须能按顺序区分：捕获源设置、sender 实际参数、CPU 或带宽 `qualityLimitationReason`、selected candidate/TURN 路径、outbound 与 inbound 差异，以及接收端 dropped/freeze。当前有界实现显示 sender 参数的 requested/applied 读回；同一个非 `none` 原生 `qualityLimitationReason` 连续出现三个采样周期后显示单条原因提示，原因变化、缺失或恢复为 `none` 即重新计数。这只是去抖后的观察，不组合评分、不触发调档或迁移。完整跨路径分类仍由后续真实矩阵验收；首轮使用浏览器 WebRTC internals 导出和一份脱敏 run manifest，不新增服务端遥测。 分享者逐观看者显示： @@ -253,9 +253,9 @@ TURN/TLS 是可选能力，不是生产启动门槛。启用时默认使用 RFC 所有边都继续使用标准 WebRTC。每名 viewer 播放收到的 remote `MediaStream`，并把其中的 remote tracks 加入给唯一 child 的新 `RTCPeerConnection`。该路径会逐跳解码和重编码；实验必须记录每层 encode time、CPU/GPU、`qualityLimitationReason`、帧率、分辨率和延迟，不能称为 zero-copy 或 shared encode。 -Peer-assisted 信令只接受 `1080p60`、`1080p30`、`720p30` 三个 `qualityProfileId`。服务端用受房间容量限制的内存 Map 保存当前值，默认 `1080p60`，不写入 SQLite；停止分享时保留，房间被放弃或过期时删除。peer-assisted 鉴权快照携带当前值，host 鉴权后先重报本地选择再创建已分配的 child，直播中换档时通知所有在线 viewers，后加入者直接取得最新快照。普通 P2P 的严格鉴权 wire 保持原样，host 或 viewer 在该模式发送档位同步消息都会被拒绝。 +Peer-assisted 信令只接受一个严格对象：`resolution` 为 `720p | 1080p | 1440p`，`maxFramerate` 为 15 至 60 的整数，`maxBitrate` 为 2,000,000 至 12,000,000 的整数，`degradationPreference` 为 `maintain-resolution | balanced | maintain-framerate`，缺字段、多余字段和越界值都拒绝。服务端用受房间容量限制的内存 Map 保存完整设置，默认 1080p60、8 Mbps、清晰优先，不写入 SQLite；停止分享时保留，房间被放弃或过期时删除。只有 peer-assisted host 可更新，鉴权快照和在线广播携带同一完整设置；普通 P2P 的严格鉴权 wire 保持原样，host 或 viewer 在该模式发送同步消息都会被拒绝。启用 SFU 时 publisher 也使用这一设置，不另建一套质量协议。 -Viewer 在处理 peer-assisted 鉴权快照时先更新 `ViewerRelay` 的 desired profile，再应用拓扑分配；同一个串行队列协调 remote stream、当前 child 更新和以后重建的 child，因此它们读取同一最新目标。每个 `HostPeer` 也把初始 sender 配置、`replaceStream()` 和 `updateProfile()` 放进同一 sender-mutation queue；`desiredProfile` 在入队前同步更新，已排队操作执行时读取最新值，使快速连续换档保持 last-wins，而不增加版本号或确认协议。这里同步的是浏览器 sender 的目标上限，不是实际分辨率、帧率、码率或 shared encoding 的性能保证。 +Viewer 在处理 peer-assisted 鉴权快照时先更新 `ViewerRelay` 的 desired settings，再应用拓扑分配；同一个串行队列协调 remote stream、当前 child 更新和以后重建的 child，因此它们读取同一最新目标。每个 `HostPeer` 也把初始 sender 配置、`replaceStream()` 和 `updateProfile()` 放进同一 sender-mutation queue；desired settings 在入队前同步更新，已排队操作执行时读取最新值，使快速连续修改保持 last-wins，而不增加版本号或确认协议。这里同步的是浏览器 sender 的目标上限，不是实际分辨率、帧率、码率或 shared encoding 的性能保证。 spike 复用现有 screen stream 和浏览器 codec 协商，覆盖当前桌面 Chrome/Edge relay、按测试顺序位于链尾的当前 Android Chrome 与 iOS Safari leaf、最多八名 viewer；捕获存在时沿同一标准 WebRTC 路径带上音频。Encoded Transform/DataChannel/WebCodecs 媒体、SVC/simulcast、多树、转码、codec ladder、综合评分、移动端后台 relay 和跨模式 SFU 迁移不在本次 spike 内实现。该范围限制不能解释为拒绝产品所需的自动兜底；验收与失败即放弃条件以 ADR-0004 为准，自动路由由后续独立 ADR 定义。 diff --git a/docs/需求理解.md b/docs/需求理解.md index 381d910..d3e7e5c 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -21,7 +21,7 @@ - [ ] 分享端 fanout 硬上限：分享端任何时刻最多维护两条下游媒体边，包括加入、断线、换父和恢复过程。第三名及后续观看者由系统自动分配给通过验收的 peer-assisted 拓扑，必要时自动回退到部署已提供的可选 SFU；两者都不可用时才明确拒绝，不能隐藏地回退为分享端第三条连接。 - [ ] P2P 优先：每条逻辑媒体边的 ICE 必须先尝试直接 UDP 路径，服务器不得默认中转可直连的媒体。一个或两个观看者优先直接连接分享端；更多观看者的 peer-assisted 行为须先通过 ADR-0004 的实验门槛。 - [ ] NAT 兜底：每条 parent-child 边独立进行 ICE；生产环境必须提供 STUN、TURN/UDP 与 TURN/TCP candidates，通过优先级偏好直连和 UDP，同时允许无法使用 UDP 的网络选择 TCP，不能因为某条边的网络与其他边不同而静默失败。 -- [ ] 质量档位：提供 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档上限；分享期间可直接换档而不重新选择来源或重建健康连接。Peer-assisted 房间只允许这三个档位 ID，并把分享者当前选择同步给现有和后续 relay sender；普通 P2P 信令结构保持不变。当前 `contentHint = "motion"` 与 `balanced` 只是测量基线；目标默认策略为可读性优先，并在高级设置中提供“清晰优先 / 平衡 / 流畅优先”。任何策略都只是浏览器偏好，必须显示实际结果和限制原因。自定义动态 FPS 或画面检测只在统计证明必要后实现。 +- [ ] 质量控制：保留 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档推荐上限，默认采用清晰优先；分享期间可直接换档或应用有界高级设置，不重新选择来源或重建健康连接。Peer-assisted relay 与可选 SFU 必须接收同一个严格校验的房间级权威设置，覆盖当前和以后建立的 sender；普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好，必须读回实际 sender 参数并显示参数拒绝、改写和持续受限原因。自定义动态 FPS、综合评分或周期调档只在统计证明必要后实现。 - [ ] 断线恢复：临时网络变化时尝试 ICE restart 和重连，并向用户显示当前连接状态。 - [ ] 可观测性：采集候选类型、RTT、丢包、抖动、实际码率、帧率、编码耗时和质量受限原因。 - [ ] 渐进式诊断：分享者和观看者默认只看到房间、画面、核心状态和常用控制；候选路径、RTT、丢包、编解码器、编码耗时等技术详情由一个明确的“显示连接详情”开关展开。隐藏诊断不能隐藏失败原因、音频缺失、持续画质受限或需要用户操作的告警。 @@ -30,8 +30,8 @@ ### P1 - 重要功能 -- [ ] 质量控制扩展：保留可立即理解的推荐档位作为默认入口，同时提供一个可选的高级设置区，让分享者在安全边界内实时调整分辨率、帧率、视频码率、质量优先级和音频模式而无需重选来源或重建健康连接。实际发送/接收质量持续明显低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一固定参数组合称为“完美平衡”。 -- [ ] Peer-assisted 媒体实验：分享端最多两个 child、每名 viewer 最多一个 child；信令服务只按深度和服务端加入序号确定性地形成两条 sticky 平衡链，不加入综合评分或主动重平衡。每个 viewer 将收到的标准 remote track 通过新的 `RTCPeerConnection` 转发，因此会逐跳解码和重编码；必须测量而不能称为 shared encode。服务端只在受房间数限制的内存状态中保存当前档位，不写入 SQLite；host 鉴权后先重报档位，在线 viewers 接收更新，后加入者从鉴权快照取得当前值。该功能在 ADR-0004 的实验门槛通过并由后续 ADR 正式接受前不得成为产品路径。 +- [ ] 质量控制扩展：推荐档位仍是默认入口；折叠高级区只允许 720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling，以及清晰/平衡/流畅三种偏好。直播中显式应用后才生效，避免滑杆产生连续控制流量。音频 stereo/bitrate 在标准浏览器 API 和实际协商 codec 不能可靠保证前只保留为后续测量项，不提供伪控制。实际质量持续低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一组合称为“完美平衡”。 +- [ ] Peer-assisted 媒体实验：分享端最多两个 child、每名 viewer 最多一个 child；信令服务只按深度和服务端加入序号确定性地形成两条 sticky 平衡链，不加入综合评分或主动重平衡。每个 viewer 将收到的标准 remote track 通过新的 `RTCPeerConnection` 转发，因此会逐跳解码和重编码；必须测量而不能称为 shared encode。服务端只在受房间数限制的内存状态中保存当前严格校验的质量设置，不写入 SQLite；host 鉴权后先重报设置，在线 viewers 接收更新，后加入者从鉴权快照取得当前值。该功能在 ADR-0004 的实验门槛通过并由后续 ADR 正式接受前不得成为产品路径。 - [ ] 原生 shared encode 分享端：后续独立的 Electron/原生分享端阶段必须保证一份兼容编码输出扇出到最多两套独立 WebRTC packetizer，并由单独 ADR 和测量约束。它降低分享端重复编码负载，但不消除每条出边各自的上传带宽。浏览器不保证跨 `RTCPeerConnection` 复用编码器，因此当前 Web spike 不实现、也不能宣称满足 shared encode 保证。 - [ ] 原生 shared encode relay：具备原生 relay 能力的 audience 同样最多维护两条下游媒体边，并应让一份兼容编码输出供两条边复用。普通浏览器 audience 继续作为 leaf 或使用当前会逐跳重编码的实验路径，不伪称 shared encode。 - [ ] 分包/分层多树实验：把编码包、对象或可独立解码层拆到多个 peer 路径以分散上传，只能在独立 spike 中验证。必须保持亚秒级，处理多父重组、排序、丢包恢复、关键帧、同步和节点退出；未通过门槛前不进入产品路由控制器。 @@ -98,7 +98,7 @@ - Given 一名分享者和一至两名观看者，When 运行直接 P2P，Then 分享端为每人建立独立媒体边且总 fanout 不超过二。 - Given peer-assisted spike 分别运行 1、3、5、8 名观看者，When 用户加入、离开或 relay 失效，Then 服务端使用同一个确定性广度优先规则形成两条 sticky 链，只为新 viewer 或失去 parent 的 subtree root 分配 parent，不主动搬动其他节点；分享端在任何瞬间都不超过两个 child 且每名 viewer 不超过一个，没有可用 parent 时明确等待或拒绝，而不是增加分享端 fanout。 - Given 一名分享者和八名观看者，When 在受控网络中运行已记录的质量档位，Then 记录每条边的拓扑深度、候选类型、码率、编码/解码负载、分享端与 relay 上行、首帧和玻璃到玻璃延迟；该条在真实矩阵完成前保持未验收。 -- Given peer-assisted host 鉴权或直播中换档，When 其选择三个受限档位之一，Then 当前和以后建立的 relay child sender 都使用最新目标档位，服务端只保留房间级内存状态并通知在线 viewers；普通 P2P 的鉴权 wire 不新增该字段，且不接受档位同步消息。 +- Given peer-assisted host 鉴权或直播中连续修改质量设置，When 服务端校验其分辨率、FPS、码率和偏好，Then 只有 host 的完整有界设置被接受，last-wins 地应用到当前和以后建立的 relay child sender 以及启用的 SFU publisher；越界、缺字段和多余字段被拒绝，普通 P2P 的鉴权 wire 不新增该字段且不接受同步消息。 - Given 标准浏览器把收到的 remote track 直接加入下游 `RTCPeerConnection`，When 检查 relay 统计和 libwebrtc 路径，Then 明确记录新的 outbound encoder、编码耗时和 CPU/GPU 限制，不把它当作 encoded forwarding 或 shared encode 成果。 - Given ADR-0004 的任一硬门槛只有在引入 Encoded Transform/DataChannel/WebCodecs 媒体、自定义拥塞控制、FEC/RTX、多树、综合评分、转码、codec ladder、浏览器 RTP 注入 hack 或放宽 host fanout 后才能满足，When 结束 bounded spike，Then 放弃并删除浏览器 relay 实验运行时代码，保留标准 P2P 和可选 SFU；后续 native shared encode sender 仍按独立 ADR 推进，但不能据此把失败的 relay 拓扑判为通过。 - Given 新观看者会使预计总上行超过安全预算，When 其请求加入，Then 系统自动降低档位、拒绝该档位或建议改用外部直播服务，而不是持续堆积发送队列。 diff --git a/scripts/peer-assisted-benchmark.ts b/scripts/peer-assisted-benchmark.ts index 7f5094e..0d64719 100644 --- a/scripts/peer-assisted-benchmark.ts +++ b/scripts/peer-assisted-benchmark.ts @@ -11,14 +11,15 @@ import { type ScreenerServer, } from "../src/server/app"; import { loadConfig } from "../src/server/config"; +import { + QUALITY_PROFILES, + QUALITY_RESOLUTIONS, + type QualityProfileId, + type QualitySettings, +} from "../src/client/media/quality"; -const PROFILE_SETTINGS = { - "1080p60": { width: 1920, height: 1080, frameRate: 60 }, - "1080p30": { width: 1920, height: 1080, frameRate: 30 }, - "720p30": { width: 1280, height: 720, frameRate: 30 }, -} as const; - -type ProfileId = keyof typeof PROFILE_SETTINGS; +const PROFILE_SETTINGS = QUALITY_PROFILES; +type ProfileId = QualityProfileId; type PageRole = "host" | "viewer"; export interface BenchmarkConfig { @@ -34,6 +35,7 @@ export interface BenchmarkConfig { outputPath: string | null; headless: boolean; noSandbox: boolean; + qualityControlSmoke: boolean; } interface MediaTotals { @@ -72,7 +74,7 @@ interface PageObservation { authenticatedAtEpochMs: number | null; authenticateSentAtEpochMs: number | null; signalingConnected: boolean; - qualityProfileId: string | null; + qualitySettings: QualitySettings | null; assignment: MediaAssignmentObservation; maxActiveOutboundMediaEdges: number; maxAssignedChildren: number; @@ -343,6 +345,16 @@ export function parseBenchmarkConfig( throw new Error("BENCHMARK_PROFILE must be 1080p60, 1080p30, or 720p30"); } const viewerCounts = parseViewerCounts(environment.BENCHMARK_VIEWERS); + const qualityControlSmoke = parseBoolean( + environment.BENCHMARK_QUALITY_SMOKE, + false, + "BENCHMARK_QUALITY_SMOKE", + ); + if (qualityControlSmoke && !viewerCounts.some((count) => count >= 3)) { + throw new Error( + "BENCHMARK_QUALITY_SMOKE requires a selected viewer count of at least 3", + ); + } const recoveryText = environment.BENCHMARK_RECOVERY_VIEWERS?.trim(); const recoveryViewerCount = recoveryText ? Number(recoveryText) : null; if ( @@ -410,6 +422,7 @@ export function parseBenchmarkConfig( false, "BENCHMARK_CHROME_NO_SANDBOX", ), + qualityControlSmoke, }; } @@ -551,7 +564,7 @@ export function summarizeSamples(samples: TimedSample[], viewerCount: number) { role: page.role, viewerIndex: page.viewerIndex, peerId: page.peerId, - qualityProfileId: page.qualityProfileId, + qualitySettings: page.qualitySettings, parentPeerId: page.assignment.parentPeerId, childPeerIds: page.assignment.childPeerIds, })), @@ -617,14 +630,18 @@ function buildRunChecks( expected: "<= 3000 ms after signaling authentication starts", }, { - name: "quality-profile-propagated", + name: "quality-settings-propagated", passed: summary.finalTopology.every( - (participant) => participant.qualityProfileId === profileId, + (participant) => + JSON.stringify(participant.qualitySettings) === + JSON.stringify(PROFILE_SETTINGS[profileId]), ), actual: summary.finalTopology.every( - (participant) => participant.qualityProfileId === profileId, + (participant) => + JSON.stringify(participant.qualitySettings) === + JSON.stringify(PROFILE_SETTINGS[profileId]), ), - expected: `${profileId} on every participant`, + expected: `${profileId} quality settings on every participant`, }, ]; } @@ -654,7 +671,7 @@ export function buildBenchmarkInitScript(options: { authenticatedAtEpochMs: null, authenticateSentAtEpochMs: null, signalingConnected: false, - qualityProfileId: null, + qualitySettings: null, assignment: { parentPeerId: null, childPeerIds: [] }, maxActiveOutboundMediaEdges: 0, maxAssignedChildren: 0, @@ -710,8 +727,8 @@ export function buildBenchmarkInitScript(options: { state.role = message.role; state.roomId = message.roomId; state.authenticateSentAtEpochMs = Date.now(); - } else if (direction === "out" && message.type === "set-quality-profile") { - state.qualityProfileId = message.qualityProfileId; + } else if (direction === "out" && message.type === "set-quality-settings") { + state.qualitySettings = message.qualitySettings; } if (direction === "in" && message.type === "authenticated") { socket.__screenerBenchmarkSignal = true; @@ -720,12 +737,12 @@ export function buildBenchmarkInitScript(options: { state.signalingConnected = true; if (message.mediaMode === "peer-assisted") { setAssignment(message.mediaAssignment); - state.qualityProfileId = message.qualityProfileId; + state.qualitySettings = message.qualitySettings; } } else if (direction === "in" && message.type === "media-assignment") { setAssignment(message.mediaAssignment); - } else if (direction === "in" && message.type === "quality-profile") { - state.qualityProfileId = message.qualityProfileId; + } else if (direction === "in" && message.type === "quality-settings") { + state.qualitySettings = message.qualitySettings; } if (message.type === "signal") recordDescription(message, direction); } @@ -1302,6 +1319,137 @@ async function samplePages( }; } +function peerConnectionFingerprint(pages: PageObservation[]): string { + return pages + .map((page) => + [ + page.label, + ...page.connections.map( + (connection) => + `${connection.index}:${connection.connectionId ?? "?"}`, + ), + ].join("|"), + ) + .sort() + .join("\n"); +} + +async function applyQualityPreference( + cdp: CdpConnection, + hostPage: PageHandle, + label: "平衡" | "清晰优先", + signal: AbortSignal, +): Promise { + const serializedLabel = JSON.stringify(label); + await evaluate( + cdp, + hostPage, + `(() => { + const details = document.querySelector('.advanced-quality'); + if (!(details instanceof HTMLDetailsElement)) return false; + details.open = true; + const button = Array.from(document.querySelectorAll('.quality-priority button')) + .find((item) => item.textContent?.trim() === ${serializedLabel}); + if (!(button instanceof HTMLButtonElement)) return false; + button.click(); + return true; + })()`, + ); + await waitForPage( + cdp, + hostPage, + `Array.from(document.querySelectorAll('.quality-priority button.is-selected')).some((item) => item.textContent?.trim() === ${serializedLabel})`, + 5_000, + `${label} advanced quality selection`, + signal, + ); + await evaluate( + cdp, + hostPage, + `(() => { + const button = Array.from(document.querySelectorAll('.advanced-quality-grid > button')) + .find((item) => item.textContent?.includes('应用视频设置')); + if (!(button instanceof HTMLButtonElement)) return false; + button.click(); + return true; + })()`, + ); +} + +async function runQualityControlSmoke( + cdp: CdpConnection, + pages: PageHandle[], + initialSettings: QualitySettings, + signal: AbortSignal, +): Promise<{ + peerConnectionsStable: boolean; +}> { + const hostPage = pages[0]; + if (!hostPage) { + throw new Error("Quality control smoke requires a host page"); + } + const baseline = await Promise.all(pages.map((page) => quickSnapshot(cdp, page))); + const baselineFingerprint = peerConnectionFingerprint(baseline); + const balanced = { + ...initialSettings, + degradationPreference: "balanced", + } as const satisfies QualitySettings; + + const waitForSettings = async (settings: QualitySettings): Promise => { + const serializedSettings = JSON.stringify(JSON.stringify(settings)); + await Promise.all( + pages.map((page) => + waitForPage( + cdp, + page, + `JSON.stringify(globalThis.__SCREENER_BENCHMARK__.snapshot().qualitySettings) === ${serializedSettings}`, + 10_000, + `${settings.degradationPreference} quality propagation`, + signal, + ), + ), + ); + }; + const sendingLabels = new Set( + baseline + .filter((page) => page.assignment.childPeerIds.length > 0) + .map((page) => page.label), + ); + const sendingPages = pages.filter((page) => sendingLabels.has(page.label)); + const waitForReadback = async (label: "平衡" | "清晰"): Promise => { + const expected = JSON.stringify(`${label} / ${label}`); + await Promise.all( + sendingPages.map((page) => + waitForPage( + cdp, + page, + `Array.from(document.querySelectorAll('.metric')).some((metric) => + metric.querySelector('dt')?.textContent?.trim() === '请求 / 应用优先级' && + metric.querySelector('dd')?.textContent?.trim() === ${expected} + )`, + 10_000, + `${label} sender parameter readback`, + signal, + ), + ), + ); + }; + + await applyQualityPreference(cdp, hostPage, "平衡", signal); + await waitForSettings(balanced); + await waitForReadback("平衡"); + await applyQualityPreference(cdp, hostPage, "清晰优先", signal); + await waitForSettings(initialSettings); + await waitForReadback("清晰"); + + const final = await Promise.all(pages.map((page) => quickSnapshot(cdp, page))); + + return { + peerConnectionsStable: + peerConnectionFingerprint(final) === baselineFingerprint, + }; +} + function totalDecodedFrames(page: PageObservation): number { return page.connections.reduce( (total, connection) => total + (connection.receiveTotals?.framesTotal ?? 0), @@ -1450,10 +1598,11 @@ async function runCase( let recovery: RecoveryResult = { triggered: false }; try { const capture = PROFILE_SETTINGS[config.profileId]; + const captureResolution = QUALITY_RESOLUTIONS[capture.resolution]; const commonInit = { - width: capture.width, - height: capture.height, - frameRate: capture.frameRate, + width: captureResolution.width, + height: captureResolution.height, + frameRate: capture.maxFramerate, }; const hostPage = await createPage(cdp, baseUrl, { ...commonInit, @@ -1505,6 +1654,32 @@ async function runCase( const summary = summarizeSamples(samples, viewerCount); const checks = buildRunChecks(summary, viewerCount, config.profileId); + if (config.qualityControlSmoke && viewerCount >= 3) { + const qualityControl = await runQualityControlSmoke( + cdp, + pages, + capture, + signal, + ); + checks.push({ + name: "quality-control-propagation", + passed: true, + actual: true, + expected: "balanced and clarity settings reach every participant", + }); + checks.push({ + name: "quality-control-no-peer-rebuild", + passed: qualityControl.peerConnectionsStable, + actual: qualityControl.peerConnectionsStable, + expected: "peer connection identities stay unchanged", + }); + checks.push({ + name: "quality-control-sender-readback", + passed: true, + actual: true, + expected: "host and active relay display both sender preference readbacks", + }); + } if (config.recoveryViewerCount === viewerCount) { recovery = await runRecovery( cdp, @@ -1589,6 +1764,7 @@ export async function main(): Promise { } const profile = PROFILE_SETTINGS[config.profileId]; + const profileResolution = QUALITY_RESOLUTIONS[profile.resolution]; const report: BenchmarkReport = { schemaVersion: 1, startedAt: new Date().toISOString(), @@ -1612,14 +1788,15 @@ export async function main(): Promise { recoveryTimeoutMs: config.recoveryTimeoutMs, headless: config.headless, noSandbox: config.noSandbox, + qualityControlSmoke: config.qualityControlSmoke, chromeExecutable: basename(config.chromePath), output: config.outputPath ?? "stdout", }, capture: { kind: "deterministic-canvas", - width: profile.width, - height: profile.height, - frameRate: profile.frameRate, + width: profileResolution.width, + height: profileResolution.height, + frameRate: profile.maxFramerate, audio: false, }, limitations: [ diff --git a/src/client/components/StatsGrid.tsx b/src/client/components/StatsGrid.tsx index 8ed61c7..28e5d37 100644 --- a/src/client/components/StatsGrid.tsx +++ b/src/client/components/StatsGrid.tsx @@ -1,4 +1,5 @@ import type { ConnectionMetrics } from "../types"; +import type { VideoSenderParameterReadback } from "../media/quality"; function readableNumber(value: number | null, digits = 0): string { return value === null || !Number.isFinite(value) ? "未知" : value.toFixed(digits); @@ -14,6 +15,25 @@ function qualityReason(value: string | null): string { return value ? (labels[value] ?? value) : "未知"; } +function requestedApplied( + requested: number | string | null, + applied: number | string | null, + format: (value: number | string) => string, +): string { + const requestedText = requested === null ? "?" : format(requested); + const appliedText = applied === null ? "未读回" : format(applied); + return `${requestedText} / ${appliedText}`; +} + +function preferenceLabel(value: number | string): string { + const labels: Record = { + "maintain-resolution": "清晰", + balanced: "平衡", + "maintain-framerate": "流畅", + }; + return labels[String(value)] ?? String(value); +} + function Metric({ label, value, title }: { label: string; value: string; title?: string }) { return (
@@ -26,9 +46,11 @@ function Metric({ label, value, title }: { label: string; value: string; title?: export function StatsGrid({ metrics, direction, + senderParameters, }: { metrics: ConnectionMetrics; direction: "send" | "receive"; + senderParameters?: VideoSenderParameterReadback | null; }) { const encoder = metrics.encoderImplementation ? `${metrics.encoderImplementation}${ @@ -82,6 +104,42 @@ export function StatsGrid({ value={`${readableNumber(metrics.intervalEncodeMs, 1)} ms`} /> + {senderParameters && ( + <> + `${(Number(value) / 1_000_000).toFixed(1)} Mbps`, + )} + /> + `${Number(value).toFixed(0)} fps`, + )} + /> + `${Number(value).toFixed(2)}x`, + )} + /> + + + )} ) : ( <> diff --git a/src/client/media/host-sfu-route.ts b/src/client/media/host-sfu-route.ts index 106d7f7..02dc763 100644 --- a/src/client/media/host-sfu-route.ts +++ b/src/client/media/host-sfu-route.ts @@ -19,6 +19,7 @@ interface HostPublisherTransport { deactivate(): Promise; replaceStream(stream: MediaStream): Promise; updateProfile(profile: QualityProfile): Promise; + getQualityWarning?(): string | null; disconnect(): Promise; } @@ -217,6 +218,12 @@ export class HostSfuRoute { : Promise.resolve(true); } + getQualityWarning(): string | null { + return this.active?.active + ? (this.active.publisher.getQualityWarning?.() ?? null) + : null; + } + replaceStream(stream: MediaStream): Promise { return this.active ? this.active.publisher.replaceStream(stream).catch(() => false) diff --git a/src/client/media/quality.ts b/src/client/media/quality.ts index 4a65dbf..14e0533 100644 --- a/src/client/media/quality.ts +++ b/src/client/media/quality.ts @@ -1,47 +1,110 @@ -import type { QualityProfileId } from "../../shared/protocol"; +import { + DEFAULT_QUALITY_SETTINGS, + type DegradationPreference, + type QualityProfileId, + type QualityResolution, + type QualitySettings, +} from "../../shared/protocol"; -export type { QualityProfileId } from "../../shared/protocol"; +export type { + DegradationPreference, + QualityProfileId, + QualityResolution, + QualitySettings, +} from "../../shared/protocol"; + +export type QualityProfile = QualitySettings; export const QUALITY_PROFILES = { - "1080p60": { - label: "1080p 60", - width: 1920, - height: 1080, - frameRate: 60, - maxBitrate: 8_000_000, - }, + "1080p60": DEFAULT_QUALITY_SETTINGS, "1080p30": { - label: "1080p 30", - width: 1920, - height: 1080, - frameRate: 30, + resolution: "1080p", + maxFramerate: 30, maxBitrate: 5_000_000, + degradationPreference: "maintain-resolution", }, "720p30": { - label: "720p 30", - width: 1280, - height: 720, - frameRate: 30, + resolution: "720p", + maxFramerate: 30, maxBitrate: 3_000_000, + degradationPreference: "maintain-resolution", }, +} as const satisfies Record; + +export const QUALITY_PROFILE_LABELS = { + "1080p60": "1080p 60", + "1080p30": "1080p 30", + "720p30": "720p 30", +} as const satisfies Record; + +export const QUALITY_RESOLUTIONS = { + "720p": { width: 1280, height: 720, label: "720p" }, + "1080p": { width: 1920, height: 1080, label: "1080p" }, + "1440p": { width: 2560, height: 1440, label: "1440p" }, } as const satisfies Record< - QualityProfileId, - { - label: string; - width: number; - height: number; - frameRate: number; - maxBitrate: number; - } + QualityResolution, + { width: number; height: number; label: string } >; -export type QualityProfile = (typeof QUALITY_PROFILES)[QualityProfileId]; +export const DEGRADATION_PREFERENCE_LABELS = { + "maintain-resolution": "清晰优先", + balanced: "平衡", + "maintain-framerate": "流畅优先", +} as const satisfies Record; + +export interface VideoSenderParameterValues { + maxBitrate: number | null; + maxFramerate: number | null; + scaleResolutionDownBy: number | null; + degradationPreference: RTCDegradationPreference | null; +} + +export interface VideoSenderParameterReadback { + requested: VideoSenderParameterValues; + applied: VideoSenderParameterValues; + mismatches: Array; +} + +export function qualitySettingsEqual( + left: QualitySettings, + right: QualitySettings, +): boolean { + return ( + left.resolution === right.resolution && + left.maxFramerate === right.maxFramerate && + left.maxBitrate === right.maxBitrate && + left.degradationPreference === right.degradationPreference + ); +} + +export function matchingQualityProfileId( + settings: QualitySettings, +): QualityProfileId | null { + for (const id of Object.keys(QUALITY_PROFILES) as QualityProfileId[]) { + if (qualitySettingsEqual(settings, QUALITY_PROFILES[id])) { + return id; + } + } + return null; +} + +export function qualitySettingsLabel(settings: QualitySettings): string { + const profileId = matchingQualityProfileId(settings); + if (profileId) { + return QUALITY_PROFILE_LABELS[profileId]; + } + return `${QUALITY_RESOLUTIONS[settings.resolution].label} ${settings.maxFramerate} · ${(settings.maxBitrate / 1_000_000).toFixed(1)} Mbps · ${DEGRADATION_PREFERENCE_LABELS[settings.degradationPreference]}`; +} function captureConstraints(profile: QualityProfile): MediaTrackConstraints { + const resolution = QUALITY_RESOLUTIONS[profile.resolution]; return { - width: { ideal: profile.width, max: profile.width }, - height: { ideal: profile.height, max: profile.height }, - frameRate: { ideal: profile.frameRate, max: profile.frameRate }, + width: { ideal: resolution.width, max: resolution.width }, + height: { ideal: resolution.height, max: resolution.height }, + frameRate: { + ideal: profile.maxFramerate, + max: profile.maxFramerate, + }, }; } @@ -86,16 +149,87 @@ export function setVideoPaused(stream: MediaStream, paused: boolean): boolean { return true; } +function requestedScaleResolutionDownBy( + sender: RTCRtpSender, + profile: QualityProfile, +): number { + const source = + sender.track && typeof sender.track.getSettings === "function" + ? sender.track.getSettings() + : undefined; + const ceiling = QUALITY_RESOLUTIONS[profile.resolution]; + if (!source?.width || !source.height) { + return 1; + } + return Math.max( + 1, + source.width / ceiling.width, + source.height / ceiling.height, + ); +} + +function readVideoSenderParameters( + parameters: RTCRtpSendParameters, +): VideoSenderParameterValues { + const encoding = parameters.encodings[0]; + return { + maxBitrate: encoding?.maxBitrate ?? null, + maxFramerate: encoding?.maxFramerate ?? null, + scaleResolutionDownBy: encoding?.scaleResolutionDownBy ?? null, + degradationPreference: parameters.degradationPreference ?? null, + }; +} + +function sameParameter( + key: keyof VideoSenderParameterValues, + requested: number | RTCDegradationPreference | null, + applied: number | RTCDegradationPreference | null, +): boolean { + if (typeof requested === "number" && typeof applied === "number") { + return key === "scaleResolutionDownBy" + ? Math.abs(requested - applied) < 0.01 + : requested === applied; + } + return requested === applied; +} + export async function configureVideoSender( sender: RTCRtpSender, profile: QualityProfile, -): Promise { +): Promise { const parameters = sender.getParameters(); if (parameters.encodings.length === 0) { parameters.encodings = [{}]; } - parameters.encodings[0].maxBitrate = profile.maxBitrate; - parameters.encodings[0].maxFramerate = profile.frameRate; - parameters.degradationPreference = "balanced"; + parameters.encodings[0]!.maxBitrate = profile.maxBitrate; + parameters.encodings[0]!.maxFramerate = profile.maxFramerate; + parameters.encodings[0]!.scaleResolutionDownBy = + requestedScaleResolutionDownBy(sender, profile); + parameters.degradationPreference = profile.degradationPreference; + + const requested = readVideoSenderParameters(parameters); await sender.setParameters(parameters); + const applied = readVideoSenderParameters(sender.getParameters()); + const mismatches = ( + Object.keys(requested) as Array + ).filter((key) => !sameParameter(key, requested[key], applied[key])); + + return { requested, applied, mismatches }; +} + +const PARAMETER_LABELS = { + maxBitrate: "码率上限", + maxFramerate: "帧率上限", + scaleResolutionDownBy: "分辨率缩放", + degradationPreference: "质量优先级", +} as const satisfies Record; + +export function senderParameterWarning( + readback: VideoSenderParameterReadback, +): string | null { + return readback.mismatches.length > 0 + ? `浏览器未完整接受${readback.mismatches + .map((key) => PARAMETER_LABELS[key]) + .join("、")}` + : null; } diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx index 38d268e..e667e74 100644 --- a/src/client/pages/HostPage.tsx +++ b/src/client/pages/HostPage.tsx @@ -11,7 +11,7 @@ import { } from "lucide-react"; import { useEffect, useMemo, useRef, useState } from "react"; import { - DEFAULT_QUALITY_PROFILE_ID, + DEFAULT_QUALITY_SETTINGS, type CreateRoomResponse, type IceConfig, type ServerMessage, @@ -37,9 +37,16 @@ import { SignalingClient } from "../lib/signaling"; import { applyCaptureProfile, captureDisplay, + DEGRADATION_PREFERENCE_LABELS, + matchingQualityProfileId, QUALITY_PROFILES, + QUALITY_PROFILE_LABELS, + QUALITY_RESOLUTIONS, + qualitySettingsLabel, setVideoPaused, + type DegradationPreference, type QualityProfileId, + type QualitySettings, } from "../media/quality"; import { HostSfuRoute } from "../media/host-sfu-route"; import type { @@ -113,8 +120,11 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { () => new URLSearchParams(window.location.search).get("relay") === "1", [], ); - const [qualityId, setQualityId] = useState( - DEFAULT_QUALITY_PROFILE_ID, + const [qualitySettings, setQualitySettings] = useState( + DEFAULT_QUALITY_SETTINGS, + ); + const [advancedQuality, setAdvancedQuality] = useState( + DEFAULT_QUALITY_SETTINGS, ); const shareGenerationRef = useRef(null); const [phase, setPhase] = useState("idle"); @@ -144,7 +154,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { const activeGenerationRef = useRef(null); const sourceSwitchRef = useRef(null); const qualityChangeRef = useRef(null); - const qualityIdRef = useRef(DEFAULT_QUALITY_PROFILE_ID); + const qualitySettingsRef = useRef(DEFAULT_QUALITY_SETTINGS); const picturePausedRef = useRef(false); const retiringStreamRef = useRef(null); const hostSfuRouteRef = useRef(null); @@ -153,6 +163,10 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { () => Array.from(peerSnapshots.values()), [peerSnapshots], ); + const selectedQualityProfileId = useMemo( + () => matchingQualityProfileId(qualitySettings), + [qualitySettings], + ); useEffect(() => { if (videoRef.current) { @@ -194,7 +208,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let route: HostSfuRoute; route = new HostSfuRoute({ getStream: () => streamRef.current, - getProfile: () => QUALITY_PROFILES[qualityIdRef.current], + getProfile: () => qualitySettingsRef.current, reconcileChildren: (childPeerIds) => { if ( isCurrentGeneration(generation) && @@ -309,14 +323,15 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { setSwitchingSource(false); } - function commitQuality(id: QualityProfileId): void { - qualityIdRef.current = id; - setQualityId(id); + function commitQuality(settings: QualitySettings): void { + qualitySettingsRef.current = settings; + setQualitySettings(settings); + setAdvancedQuality(settings); } - async function changeQuality(nextId: QualityProfileId): Promise { + async function changeQuality(nextProfile: QualitySettings): Promise { if (phase !== "live") { - commitQuality(nextId); + commitQuality(nextProfile); return; } @@ -336,10 +351,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { qualityChangeRef.current = token; setChangingQuality(true); setNotice(null); - const profile = QUALITY_PROFILES[nextId]; try { - await applyCaptureProfile(activeStream, profile); + await applyCaptureProfile(activeStream, nextProfile); if ( !isCurrentGeneration(generation) || qualityChangeRef.current !== token || @@ -348,29 +362,38 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { return; } - commitQuality(nextId); + commitQuality(nextProfile); setDetails(captureDetails(activeStream)); if (peerAssistedRef.current) { signalRef.current?.send({ - type: "set-quality-profile", - qualityProfileId: nextId, + type: "set-quality-settings", + qualitySettings: nextProfile, }); } + const activeSfuRoute = hostSfuRouteRef.current; const [results, sfuUpdated] = await Promise.all([ Promise.all( - [...peersRef.current.values()].map((peer) => peer.updateProfile(profile)), + [...peersRef.current.values()].map((peer) => + peer.updateProfile(nextProfile), + ), ), - hostSfuRouteRef.current?.updateProfile(profile) ?? Promise.resolve(true), + activeSfuRoute?.updateProfile(nextProfile) ?? + Promise.resolve(true), ]); if ( isCurrentGeneration(generation) && qualityChangeRef.current === token ) { const failed = results.filter((updated) => !updated).length; + const sfuWarning = + hostSfuRouteRef.current === activeSfuRoute + ? (activeSfuRoute?.getQualityWarning() ?? null) + : null; setNotice( - failed > 0 || !sfuUpdated - ? "画质已切换，但部分观看连接未能应用新参数" - : `画质已切换为 ${profile.label}`, + sfuWarning ?? + (failed > 0 || !sfuUpdated + ? "画质已切换，但部分观看连接未能应用新参数" + : `画质已切换为 ${qualitySettingsLabel(nextProfile)}`), ); } } catch (error) { @@ -442,7 +465,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { peerId, iceConfig, activeStream, - QUALITY_PROFILES[qualityIdRef.current], + qualitySettingsRef.current, { sendSignal: (targetPeerId, payload) => isCurrentGeneration(generation) && signalRef.current === signal @@ -565,8 +588,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { ) { peerAssistedRef.current = true; signalRef.current?.send({ - type: "set-quality-profile", - qualityProfileId: qualityIdRef.current, + type: "set-quality-settings", + qualitySettings: qualitySettingsRef.current, }); void ensureHostSfuRoute(generation).resyncAuthoritative( { @@ -696,7 +719,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let captured: MediaStream; try { // This must remain the first awaited operation in the button gesture. - captured = await captureDisplay(QUALITY_PROFILES[qualityIdRef.current]); + captured = await captureDisplay(qualitySettingsRef.current); } catch (error) { if (!isCurrentGeneration(generation)) { return; @@ -851,7 +874,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let captured: MediaStream; try { // Like initial capture, changing source must begin in this button gesture. - captured = await captureDisplay(QUALITY_PROFILES[qualityIdRef.current]); + captured = await captureDisplay(qualitySettingsRef.current); } catch (error) { if ( isCurrentGeneration(generation) && @@ -1106,29 +1129,149 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { )}
-
- 画质 -
- {(Object.keys(QUALITY_PROFILES) as QualityProfileId[]).map( - (id) => ( - + ), + )} +
+
+ +
+ 高级视频设置 +
+ + + +
+ 质量优先级 +
+ {( + Object.keys( + DEGRADATION_PREFERENCE_LABELS, + ) as DegradationPreference[] + ).map((preference) => ( + + ))} +
+
+ +
+
+
{phase !== "live" && (