diff --git a/docs/adr/0007-path-isolated-representation-quality.md b/docs/adr/0007-path-isolated-representation-quality.md index aac3c2d..aa4a209 100644 --- a/docs/adr/0007-path-isolated-representation-quality.md +++ b/docs/adr/0007-path-isolated-representation-quality.md @@ -23,7 +23,8 @@ it does not need a second bitrate, resolution, FPS, or layer-control system. ## Decision -1. Browser video makes one internal, sender-scoped codec decision. A bounded +1. Browser video makes one internal, share-generation-scoped codec decision per + sending endpoint. A bounded actual-`RTCPeerConnection` preflight measures H.264 encoded progress against a deterministic moving probe track at the current share's effective target dimensions and frame rate. The real capture supplies those target settings, @@ -43,7 +44,7 @@ it does not need a second bitrate, resolution, FPS, or layer-control system. After accepting an answer, the Host reapplies the current video profile to the negotiated sender because the browser may replace or rewrite encoding parameters during negotiation. -5. Direct and Browser-relay offers with a proved source order H.264 before VP8 +5. Direct and Browser-relay offers with a proved share source order H.264 before VP8 in one standard codec-preference list; both endpoints select their first common codec without a parallel media connection. A failed source gate offers VP8 only. Existing edges are not renegotiated when a later relay gate diff --git a/docs/project-memory.md b/docs/project-memory.md index a01ec53..e56fdd2 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -87,7 +87,7 @@ Last updated: 2026-08-26 - WebRTC owns direct/peer media adaptation. The Host reapplies the selected video profile after answer negotiation; no periodic application controller exists. - The production SFU publisher uses the Host's H.264/VP8 decision and selected - ceiling but no custom simulcast layers. Pinned LiveKit defaults own + ceiling but no backup codec or custom simulcast layers. Pinned LiveKit defaults own representations, Dynacast owns aggregate demand, and server send-side BWE owns subscriber forwarding. AdaptiveStream stays disabled because any Viewer may relay its received track. @@ -118,8 +118,10 @@ Last updated: 2026-08-26 ## Current Source And Production -- Production and the matching canonical application/runtime tree use strict - `screener-v12`; exact identity is indexed by [status](./status.md). +- Production and canonical source use strict `screener-v12`, but the pending + application candidate adds the internal H.264/VP8 decision while production + remains on its prior fixed-VP8 artifact. Exact identities are indexed by + [status](./status.md). - Production uses random memory rooms, 20-Viewer admission, endpoint cap `2`, fixed 9,000-publication ingress and `9000 * 20` subscription egress admission, STUN UDP 3478, LiveKit media UDP 7882, and Web TCP 80/443. Node 8787 and diff --git a/docs/方案设计.md b/docs/方案设计.md index e7f7396..c2e3542 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -2,7 +2,7 @@ ## 2026-08-18 14:45 -状态:production 与 canonical application/runtime tree 使用严格 `screener-v12`。Browser 视频固定 VP8 并使用 `contentHint = "motion"`,音频使用 `music`;codec UI/state/wire、自定义 SFU layer list与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配;screen audio 使用 96/128/192 kbps ceiling、stereo、DTX off 与 RED off。Viewer 页面恢复、原生控件和当前线路重连已进入 source 与 production,精确执行状态由 `docs/status.md` 持有。 +状态:production 与 canonical source 使用严格 `screener-v12`。Canonical source 已实现每分享代次的 H.264 实际 sender gate 与 VP8 fallback,production 仍是待切换的固定 VP8 artifact;两者都使用 video `contentHint = "motion"` 和 audio `music`,且 codec UI/state/wire、自定义 SFU layer list与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配;screen audio 使用 96/128/192 kbps ceiling、stereo、DTX off 与 RED off。精确执行状态由 `docs/status.md` 持有。 ## 目标与边界 @@ -162,7 +162,7 @@ accepted 的唯一 wire 是 `screener-v12`,不做版本协商、旧 parser、 每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级,实际结果由 stats 观测;拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。 -Accepted `screener-v12` Web 视频使用页面内 sender-scoped H.264/VP8 决策。初始 Host capture 在取得 room/route authority 前,以一对无 ICE server 的本地标准 `RTCPeerConnection` 运行一个有界 H.264-only sender probe。Probe 使用按当前分享 effective target dimensions/frame rate 生成的短时确定性运动 Canvas track,并以 `motion` 表达同一内容意图;真实 capture/upstream track 只提供目标规格,其当前内容是否变化不参与 encoder capability 判断。Gate 只用实际协商 codec、同一身份的 probe media-source/encoded progress 与 bounded completion 判断,不读取 UA/GPU/MFT allowlist,也不单独相信 capability 或 `powerEfficientEncoder`;标准 Canvas capture 不可用、失败或 inconclusive 时使用 VP8。成功结果让后续 P2P/browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与 RTX/RED/FEC repair capabilities;失败或 inconclusive 只列 VP8 与 repair。Relay 收到新 upstream stream 后在后台运行同一 gate,完成前建立的 child 使用 VP8,完成后只影响未来 child。无法应用 codec preferences 或缺少 VP8 时明确失败。SFU publication 使用 Host source 当时的同一 H.264/VP8 决定并关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling;当 desired degradation preference 允许分辨率下降时,启动阶段的 effective preference 暂为 `maintain-resolution`,当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset,不增加周期重写、质量评分或应用层调档器。UI 和 quality state/wire 没有 codec 字段;gate 结果不持久化、不改写已连接 edge,实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。 +Accepted `screener-v12` Web 视频使用页面内 share-generation-scoped H.264/VP8 决策。初始 Host capture 在取得 room/route authority 前,以一对无 ICE server 的本地标准 `RTCPeerConnection` 运行一个有界 H.264-only sender probe。Probe 使用按当前分享 effective target dimensions/frame rate 生成的短时确定性运动 Canvas track,并以 `motion` 表达同一内容意图;真实 capture/upstream track 只提供目标规格,其当前内容是否变化不参与 encoder capability 判断。Gate 只用实际协商 codec、同一身份的 probe media-source/encoded progress 与 bounded completion 判断,不读取 UA/GPU/MFT allowlist,也不单独相信 capability 或 `powerEfficientEncoder`;标准 Canvas capture 不可用、失败或 inconclusive 时使用 VP8。成功结果让后续 P2P/browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与 RTX/RED/FEC repair capabilities;失败或 inconclusive 只列 VP8 与 repair。Relay 在该分享代首次收到 upstream video 且没有 active/prepared child 时运行同一 gate;若 child prepare 先到,则等待已经在运行的 bounded gate 结束后只创建一份真实 child sender。质量设置、换源和换父不重复 gate。无法应用 codec preferences 或缺少 VP8 时明确失败。SFU publication 使用 Host share generation 的同一 H.264/VP8 决定并关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling;当 desired degradation preference 允许分辨率下降时,启动阶段的 effective preference 暂为 `maintain-resolution`,当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset,不增加周期重写、质量评分或应用层调档器。UI 和 quality state/wire 没有 codec 字段;gate 结果不持久化、不改写已连接 edge,实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。 Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`autoGainControl` 和 `voiceIsolation`,并以 `channelCount: { ideal: 2 }` 偏好双声道;这是当前 Chrome/Edge Host 基线的实现契约,用于避开网页 `getDisplayMedia()` 默认的语音处理和单声道 APM 候选,不宣称跨浏览器保证,也不固定采样率。所有值都是非强制 post-selection preference;不使用 `exact`、`min` 或 `advanced`。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`:窗口选项请求该窗口/应用音频,整个屏幕选项允许用户勾选系统音频;标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些约束或 picker hints,且没有对应的标准来源类型读回,因此 UI 不额外推断或标注来源。系统音频由用户显式选择,可能包含语音或通知;video-only 始终可用。 @@ -172,7 +172,7 @@ Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、` 直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`,不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流;成功后为每个现有 sender 调用 `replaceTrack()`,健康连接不重新协商。初始 offer 预留 send-only audio transceiver,使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败,只重建该 peer;新流提交后才停止旧流,并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置,避免把不同 track 的计数器混算。 -直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;quality mutation 不重新运行 codec gate,新加入或重建的 Browser sender 使用当前 source 的内存结果与最新其他权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停禁用当前 capture stream 的全部音视频 tracks;Host 通过当前 session 与 `shareGeneration` 发送一个 paused boolean,服务端同步 `host-status`、route pause 与 `authenticated` 快照,Viewer 在舞台显示“分享者已暂停”。Resume 使用同一状态更新并重新启用同一批 tracks,不另设 authorization、source acknowledgement 或 proof。晚加入和信令重连从快照恢复,恢复、停止、新分享或会话替换清除旧状态;不得把 track mute、黑帧或网络停顿推断成主动暂停。房间与连接保持不变。 +直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;quality mutation 不重新运行 codec gate,新加入或重建的 Browser sender 使用当前分享代次的内存结果与最新其他权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停禁用当前 capture stream 的全部音视频 tracks;Host 通过当前 session 与 `shareGeneration` 发送一个 paused boolean,服务端同步 `host-status`、route pause 与 `authenticated` 快照,Viewer 在舞台显示“分享者已暂停”。Resume 使用同一状态更新并重新启用同一批 tracks,不另设 authorization、source acknowledgement 或 proof。晚加入和信令重连从快照恢复,恢复、停止、新分享或会话替换清除旧状态;不得把 track mute、黑帧或网络停顿推断成主动暂停。房间与连接保持不变。 Host 本地预览直接绑定现有 capture stream,不创建 viewer、`RTCPeerConnection` 或额外媒体边,也不产生 SFU/应用服务器出口。预览仍可能有本地合成、缩放和 GPU 显示成本;在同一动态场景对预览开/关采集 Host CPU、GPU、encode time 和实际 FPS。后台性能诊断必须把 Host 页面前台/遮挡/最小化、被捕获 source 前台/遮挡/最小化以及 preview 播放/暂停分成独立变量,并关联 document visibility/focus、track mute/readyState 与 capture/outbound/inbound stats。页面 timer、伪活跃和 Screen Wake Lock 都不构成屏幕捕获保活方案;若标准浏览器无法维持目标行为,应记录平台能力边界而不是制造第二套 lifecycle。只有稳定收益且捕获、编码和远端画面不变时,才把直播后的默认动态预览改为静止缩略图或隐藏并允许按需展开;隐藏预览绝不停止 capture track 或 sender encode。 @@ -363,7 +363,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio ## 当前交付边界 -- Production 与 canonical application/runtime tree 使用严格单一 `screener-v12` wire。当前实现包含短 room-lived grant、明确 missing-room 结果、产品诊断下载移除、完整短 URL 邀请 UI、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性,固定 Browser VP8、video `contentHint = "motion"`、audio `music`,且不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list或分享生命周期 codec 分支。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off;精确执行状态由 `docs/status.md` 持有。 +- Production 与 canonical source 使用严格单一 `screener-v12` wire。Canonical source 的每分享代次 actual-sender gate 在证据通过时让未来 direct/relay 和 Host 单一 SFU publication 使用 H.264,否则使用 VP8;production 仍是固定 VP8 artifact。两者都使用 video `contentHint = "motion"`、audio `music`,不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list、backup publication或健康 edge codec 切换。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off;精确执行状态由 `docs/status.md` 持有。 - Pause/Resume 由当前 Host session 和 `shareGeneration` 约束。Host 后台性能和 VP8 hardware acceleration 分别按受控根因证据诊断;标准 ICE/SFU 按真实异构网络验收。 - 当前发布面只包含 Web Host、Viewer 和 browser relay。Native sender、capture helper、shared-encode executable 与两个分发包仍由各自后续决策和物理证据驱动,不属于当前 Browser release。 diff --git a/docs/需求理解.md b/docs/需求理解.md index 1096f58..0bbb194 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -27,7 +27,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - 端点发送容量:所有非服务器端点使用同一个服务端权威稳态 outbound media-copy capacity,由唯一部署项 `ENDPOINT_MEDIA_COPY_CAPACITY` 配置,默认 `2` 且只接受 `1`、`2` 或 `3`;不读取旧键或保留第二套 tier。peer child 或 Host 的唯一 SFU publication 各消耗一份,上游接收不消耗该预算。客户端只能声明不高于部署值的可用容量,角色、浏览器、UA、设备类型和页面可见性不产生不同上限。服务端在每次 `authenticated` 权威快照中返回最终 `endpointMediaCopyCapacity`;普通非 peer-assisted 模式也只授权稳定加入顺序中的至多 `C` 个 Host child,超额 Viewer 保持入房等待并在槽位释放后提升,信令不能绕过该集合。Browser Host 与 Viewer 严格校验该值并用它约束稳态 assignment;并行产生媒体的 candidate 必须持有唯一 `TransitionOverlapSlot`,active+pending 物理 copy 只可达到 `min(C + 1, 3)` 并在 commit/abort 后释放。`C=3` 满载时必须先收到 active bounded-gap retirement 再 prepare,绝不产生第四份 copy。SFU subscriber egress 由独立 server admission 记账。当前不兼容房间合同以单一 release 原子升级;缺失/非法 cap、旧 wire 与 executable sender都在取得room/route authority前fail closed。 - P2P 优先:每条普通逻辑媒体边先尝试 direct/peer UDP,服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布;每名 Viewer 同时至多一个 active upstream,失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate;Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction,并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时,同一 controller 用有限 carrier cursor 建立唯一 publication;carrier 必须能在既有 overlap 上限内完成,不能先切断健康分支。终态归原始等待需求,publication 成功后该需求先取得 SFU 画面,剩余 direct candidate 再在后台收敛;完整事务由 ADR-0005 定义。 - NAT 与传输边界:STUN 必须可用,所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`,保留 LiveKit signaling 交付的授权 SFU UDP candidates,并由标准 ICE 选择 nominated non-relay pair;不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC,也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威;Screener 不合成 remote candidate、不预测端口、不分类 NAT,也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity;成功即停止,失败、超时或绑定失效即撤销,direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP,全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers,再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media;当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输,必须把它作为 LiveKit 内部能力重新决策,不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target;HTTPS/WSS 继续独立使用 TLS/TCP。 -- 质量控制:推荐档位始终只有 720p30(3 Mbps)、1080p30(5 Mbps)、1080p60(8 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`(均衡),`maintain-resolution`(清晰)、`maintain-framerate`(流畅)保留为显式选择。Browser 对每个新分享 source 运行一次有界实际 sender 预检;H.264 源帧与编码帧进度通过时,未来 direct/relay offer 原生排序 H.264 后接 VP8,Host 的唯一 SFU publication 也使用 H.264;不支持、证据不足、超时或表现不达标时只使用 VP8。界面、质量状态和 wire 不提供 codec 选择,结果不持久化,也不切换已连接线路。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling,音频默认选择中档 128 kbps,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由 WebRTC/LiveKit 决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。 +- 质量控制:推荐档位始终只有 720p30(3 Mbps)、1080p30(5 Mbps)、1080p60(8 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`(均衡),`maintain-resolution`(清晰)、`maintain-framerate`(流畅)保留为显式选择。Browser Host 在每个新分享代次运行一次有界实际 sender 预检,Relay 在该代首次取得 upstream source 且尚无 child 时运行同一预检;H.264 源帧与编码帧进度通过时,未来 direct/relay offer 原生排序 H.264 后接 VP8,Host 的唯一 SFU publication 也使用 H.264;不支持、证据不足、超时或表现不达标时只使用 VP8。界面、质量状态和 wire 不提供 codec 选择,结果不持久化,也不因质量设置、换源或换父重复探测或切换已连接线路。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling,音频默认选择中档 128 kbps,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由 WebRTC/LiveKit 决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。 - 路径隔离表示:direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC,不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 不设置 `screenShareSimulcastLayers`,由锁定的 LiveKit client 默认行为拥有当前 H.264 或 VP8 source decision 对应的 screen-share representations、source replacement 与 republish;LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 负责需求聚合与每个 subscriber 的实际下行层。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。所有当前 Viewer 都可能承担 peer children,因此 subscriber 保持 `adaptiveStream: false`,不得让本地元素尺寸或可见性控制 relay ingress。表示选择不改变拓扑、端点容量或路由失效权威。 - 路由失效输入:应用不推断 parent-wide 质量,也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`,或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时,使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`,route/connection变化或新帧重置、暂停抑制,同一 edge只报告一次,不增加轮询器。Active SFU route 暂时没有 video track、stats report 为空或 stats 读取失败时,现有采样 cadence 仍把该次作为无 decoded progress 推进 deadline;它不得停表、伪造连接指标或立即判定 edge 失效,只有 route 停用或 teardown 才停止该 cadence。该 operation 把失败的 exact direct parent 在当前 fact version 视为已尝试;foreground direct window结束后用SFU取得可用画面,剩余direct candidate只在画面后方收敛,新的外部事实才允许本轮失败tuple重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时,它作为child换父并保留subtree;disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。 - 断线恢复:临时网络变化先交给 WebRTC/LiveKit 的 ICE、consent、重连和流状态。Direct edge 可在 exact current parent/child sessions 内采用 framework rebuild 的新 connection ID而不改变topology。应用只在上述exact edge失效后运行一次有界reconcile;任一步成功即停止。候选耗尽时hard-invalid、confirmed-departed或overflow edge被退休并进入明确waiting,healthy bootstrap edge保留。所有异步结果受authenticated session、share generation、base/pending route revision与candidate connection identity约束,未受影响分支保持sticky。权威暂停abort未提交candidate、保留active graph、抑制stall,并在resume后重新reconcile。 @@ -161,7 +161,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - Given 同一时刻采集、host outbound 与 viewer inbound 证据,When 进行根因分类,Then 关联派生后的实际协商视频 codec/profile/parameters、适用时的 `scalabilityMode` 与真实解码表现,依次区分 Capture 低、Capture 高但 outbound 因 CPU 低、Capture 高但 outbound 因 bandwidth 低、outbound 正常但 inbound 低、以及 inbound 正常但肉眼模糊五类;不用 UA 或设备型号代替运行时证据,也不上传 raw SDP。 - Given Viewer 上报当前路径质量证据,When 报告未鉴权、过频、重复、过期或没有匹配的 sender/viewer 统计,Then 报告被丢弃;有效报告也只能用于诊断,不能改变 Host 表示或路径状态。 - Given 受控 RTT/丢包测试条件,When 连续分享高动态游戏画面 30 分钟,Then 延迟、帧率与丢帧达到上述目标,且没有不可恢复的冻结。 -- Given Windows Chrome/Edge Browser Host 开始或继续分享,When 浏览器返回 display video track,Then 应用设置 `contentHint = "motion"`,并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 新 source 的有界 H.264 预检证明实际 `video/H264` sender 的源帧/编码帧进度达标,Then 后续 direct 或 browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与修复格式;双方不支持 H.264 时同一次协商选择 VP8。Given 预检不支持、证据不足、超时、失败或不达标,Then offer 只列 VP8 与修复格式。缺少 VP8、`setCodecPreferences()` 或应用失败仍在创建 offer 前明确失败。Given 建立 SFU publication,Then 使用 Host source 的同一 H.264/VP8 决定、禁用 backup codec,且 Screener 不设置自定义 simulcast layer list。 +- Given Windows Chrome/Edge Browser Host 开始或继续分享,When 浏览器返回 display video track,Then 应用设置 `contentHint = "motion"`,并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 当前分享代次的有界 H.264 预检证明实际 `video/H264` sender 的源帧/编码帧进度达标,Then 后续 direct 或 browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与修复格式;双方不支持 H.264 时同一次协商选择 VP8。Given 预检不支持、证据不足、超时、失败或不达标,Then offer 只列 VP8 与修复格式。缺少 VP8、`setCodecPreferences()` 或应用失败仍在创建 offer 前明确失败。Given 建立 SFU publication,Then 使用 Host share generation 的同一 H.264/VP8 决定、禁用 backup codec,且 Screener 不设置自定义 simulcast layer list。 - Given 分享过程中切换 Wi-Fi/有线网络,When ICE 连接失败,Then 客户端触发 ICE restart,并在成功或超时后给出明确状态。 - Given 当前路径仍可播放且控制器准备迁移 logical parent 或替换 transport,When 候选路径拥有模型授权的 overlap 资源,Then 旧路径保持权威,直到 exact candidate child 解码出第一帧新画面后才原子提交;ICE `connected` 单独不能触发切换。 - Given 候选没有准入资源,或 provisional 路径失败、超时、代次过期、参与者变化、房间停止、权限撤销或 session 被替换,When 本次机会结束,Then 原路径和未受影响 branches 继续工作,临时资源被销毁,重试遵守模型定义的有界策略。 @@ -171,7 +171,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - Given Web Host 或浏览器 relay 建立或替换分享音轨,When 对实际音频 sender 应用发送参数,Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 64000 | 128000 | 192000`、调用 `setParameters()` 并立即读回,档位缺失时按 128000 处理;Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer,Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=192000`,并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用;空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续,不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增,其他 fmtp/FEC/session/media 字段逐项保留。 - Given 通过 SFU 发布分享音轨,When 构建 LiveKit 2.22.0 publish options,Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、192 kbps 使用同等结构的显式 sender ceiling,三档都设置 `forceStereo = true`、`dtx = false` 与 `red = false`;验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率,不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。 - Given 分享正在直播或暂停且 Host 选择新的 64/128/192 kbps 音频档位,When 当前 profile mutation 串行执行,Then 服务端以 last-wins 保存并广播最新 desired profile,每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback,过期 completion 不得覆盖新值;以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试,不得谎称全房已收敛。当前 applied readback 保持 endpoint-local;没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation,也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge;其他浏览器只有实际 sender readback 成功时才算支持。 -- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质,When 查看 UI 或校验消息,Then 不存在 codec selector 或 codec quality-state/wire 字段。source-scoped H.264 预检只产生当前页面内存中的内部发送偏好;结果不持久化、不重新协商健康 edge,实际协商 codec 与 encoder 只作为只读诊断。 +- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质,When 查看 UI 或校验消息,Then 不存在 codec selector 或 codec quality-state/wire 字段。share-generation-scoped H.264 预检只产生当前页面内存中的内部发送偏好;结果不持久化、不重新协商健康 edge,实际协商 codec 与 encoder 只作为只读诊断。 - Given 浏览器返回屏幕音轨,When 产品展示音频状态或未来连接详情,Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实;字段缺失必须显示为未知,且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。 - Given Web 或后续 native sender 分享电影/视频音频,When 进行产品验收,Then 必须用独立 A/B/C 证据关联 capture audio track/settings、实际 negotiated codec/fmtp、outbound/inbound 实际码率、loss、jitter、concealment、jitter-buffer 与 A/V sync;P2P、浏览器 relay 与 SFU route switch 保持同一持久 Viewer `MediaStream`/媒体元素和 stereo 协商,不新增独立 audio element 或 Web Audio 时钟边界。native sender 必须通过该音质和同步矩阵,且不得把 audio `maxBitrate` ceiling 当作提质证明。 - Given 分享者同时运行语音软件,When 选择仅分享游戏声音,Then supported Windows 11 native sender 只保留目标游戏及其子进程的渲染音频,观众、通知和其他应用声音不得进入分享;浏览器 system audio 无法满足时必须允许 video-only 或明确告知其可能包含语音/通知,不能用 AEC、降噪或来源标签伪称已隔离。 diff --git a/src/client/components/StatsGrid.tsx b/src/client/components/StatsGrid.tsx index 74bf6eb..a9013a3 100644 --- a/src/client/components/StatsGrid.tsx +++ b/src/client/components/StatsGrid.tsx @@ -48,8 +48,13 @@ function codecName(codec: string): string { export function codecContractWarnings(metrics: ConnectionMetrics): string[] { const warnings: string[] = []; - if (metrics.codec && metrics.codec.toLowerCase() !== "video/vp8") { - warnings.push(`视频编码为 ${codecName(metrics.codec)},预期 VP8`); + if ( + metrics.codec && + !["video/h264", "video/vp8"].includes(metrics.codec.toLowerCase()) + ) { + warnings.push( + `视频编码为 ${codecName(metrics.codec)},预期 H264 或 VP8`, + ); } if (metrics.audioCodec && metrics.audioCodec.toLowerCase() !== "audio/opus") { warnings.push(`音频编码为 ${codecName(metrics.audioCodec)},预期 Opus`); diff --git a/src/client/media/host-provisional-child.ts b/src/client/media/host-provisional-child.ts index 7f6246b..81c8d99 100644 --- a/src/client/media/host-provisional-child.ts +++ b/src/client/media/host-provisional-child.ts @@ -6,6 +6,7 @@ import type { } from "../../shared/protocol"; import { countEndpointMediaCopies } from "../../shared/media-copy-accounting"; import { HostPeer } from "../webrtc/host-peer"; +import type { BrowserVideoCodec } from "../webrtc/video-codec"; import type { PeerSnapshot } from "../types"; import type { QualityProfile } from "./quality"; @@ -21,6 +22,7 @@ interface HostProvisionalPrepareInput extends HostProvisionalInput { iceConfig: IceConfig; stream: MediaStream; profile: QualityProfile; + videoCodec: BrowserVideoCodec; } interface HostProvisionalChildEvents { @@ -86,7 +88,10 @@ export class HostProvisionalChild { private startPrepared( revision: number, candidate: PreparedRouteCandidate, - input: Pick, + input: Pick< + HostProvisionalPrepareInput, + "iceConfig" | "stream" | "profile" | "videoCodec" + >, ): void { let peer: HostPeer; peer = new HostPeer( @@ -113,6 +118,7 @@ export class HostProvisionalChild { } }, }, + input.videoCodec, candidate.connectionId, ); this.signalingPeer = peer; diff --git a/src/client/media/host-sfu-route.ts b/src/client/media/host-sfu-route.ts index 5f38d62..d658380 100644 --- a/src/client/media/host-sfu-route.ts +++ b/src/client/media/host-sfu-route.ts @@ -10,6 +10,7 @@ import { type SfuPublisherFailureStage, } from "../sfu/publisher"; import type { ConnectionMetrics } from "../types"; +import type { BrowserVideoCodec } from "../webrtc/video-codec"; import type { AudioSenderParameterReadback, QualityProfile, @@ -25,7 +26,11 @@ import { interface HostPublisherTransport { connect(config: SfuConnectionConfig): Promise; - activate(stream: MediaStream, profile: QualityProfile): Promise; + activate( + stream: MediaStream, + profile: QualityProfile, + videoCodec?: BrowserVideoCodec, + ): Promise; deactivate(): Promise; replaceStream(stream: MediaStream): Promise; updateProfile(profile: QualityProfile): Promise; @@ -54,6 +59,7 @@ export interface HostSfuPublisherSnapshot { interface HostSfuRouteEvents { getStream: () => MediaStream | null; getProfile: () => QualityProfile; + getVideoCodec: () => BrowserVideoCodec; reconcileChildren: (childPeerIds: string[]) => void; send: (message: ClientMessage) => boolean; onPublisherUpdate?: (snapshot: HostSfuPublisherSnapshot | null) => void; @@ -481,12 +487,13 @@ export class HostSfuRoute { private async preparePublisher(slot: HostPublisherSlot): Promise { const stream = this.events.getStream(); const profile = this.events.getProfile(); + const videoCodec = this.events.getVideoCodec(); if (!stream) { this.handleFailure(slot); return false; } try { - if (!(await slot.publisher.activate(stream, profile))) { + if (!(await slot.publisher.activate(stream, profile, videoCodec))) { this.handleFailure(slot); await disconnectPublisher(slot.publisher); return false; diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx index 54c1103..ec85d18 100644 --- a/src/client/pages/HostPage.tsx +++ b/src/client/pages/HostPage.tsx @@ -120,6 +120,8 @@ import { MAX_ENDPOINT_MEDIA_CHILDREN, reconcileBoundedMediaChildren, } from "../webrtc/media-assignment"; +import type { BrowserVideoCodec } from "../webrtc/video-codec"; +import { preferredVideoCodecForTrack } from "../webrtc/video-codec-preflight"; import { hostActionErrorNotice, hostServerErrorNotice, @@ -321,6 +323,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { const pendingQualityChangeRef = useRef(null); const qualitySettingsRef = useRef(DEFAULT_QUALITY_SETTINGS); const advancedQualityRef = useRef(advancedQuality); + const videoCodecRef = useRef("vp8"); + const codecProbeAbortRef = useRef(null); const sharingPausedRef = useRef(false); const retiringStreamRef = useRef(null); const hostSfuRouteRef = useRef(null); @@ -398,6 +402,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { sourceSwitchRef.current = null; qualityChangeRef.current = null; pendingQualityChangeRef.current = null; + codecProbeAbortRef.current?.abort(); + codecProbeAbortRef.current = null; stopPreferredRoomRenewal(); signalRef.current?.stop(); peersRef.current.forEach((peer) => peer.dispose()); @@ -432,6 +438,28 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { ); } + async function probeStreamVideoCodec( + stream: MediaStream, + ): Promise { + codecProbeAbortRef.current?.abort(); + const controller = new AbortController(); + codecProbeAbortRef.current = controller; + const track = stream.getVideoTracks()[0]; + try { + return track + ? await preferredVideoCodecForTrack( + track, + qualitySettingsRef.current, + controller.signal, + ) + : "vp8"; + } finally { + if (codecProbeAbortRef.current === controller) { + codecProbeAbortRef.current = null; + } + } + } + function stopPreferredRoomRenewal(): void { if (preferredRoomRenewalTimerRef.current !== null) { window.clearInterval(preferredRoomRenewalTimerRef.current); @@ -469,6 +497,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { route = new HostSfuRoute({ getStream: () => streamRef.current, getProfile: () => qualitySettingsRef.current, + getVideoCodec: () => videoCodecRef.current, reconcileChildren: (childPeerIds) => { if ( isCurrentGeneration(generation) && @@ -522,6 +551,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { sourceSwitchRef.current = null; qualityChangeRef.current = null; pendingQualityChangeRef.current = null; + codecProbeAbortRef.current?.abort(); + codecProbeAbortRef.current = null; + videoCodecRef.current = "vp8"; const signal = signalRef.current; if (signal) { if (notifyServer) { @@ -1002,6 +1034,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { iceConfig, stream, profile: qualitySettingsRef.current, + videoCodec: videoCodecRef.current, }); } @@ -1074,6 +1107,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { } }, }, + videoCodecRef.current, ); peersRef.current.set(peerId, peer); let started: boolean; @@ -1448,6 +1482,12 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { setDetails(captureDetails(captured)); watchCaptureEnd(captured, generation); + videoCodecRef.current = await probeStreamVideoCodec(captured); + if (!isCurrentGeneration(generation)) { + captured.getTracks().forEach((track) => track.stop()); + return; + } + let createdRoom: HostRoomState | null = null; let claimedRoom = false; try { diff --git a/src/client/sfu/publisher.ts b/src/client/sfu/publisher.ts index 1a42dda..51bd2dc 100644 --- a/src/client/sfu/publisher.ts +++ b/src/client/sfu/publisher.ts @@ -23,6 +23,7 @@ import { type VideoSenderParameterReadback, } from "../media/quality"; import type { ConnectionMetrics } from "../types"; +import type { BrowserVideoCodec } from "../webrtc/video-codec"; import { captureMetrics, collectConnectionMetricsFromReport, @@ -169,7 +170,11 @@ export class SfuPublisher { } } - activate(stream: MediaStream, profile: QualityProfile): Promise { + activate( + stream: MediaStream, + profile: QualityProfile, + videoCodec: BrowserVideoCodec = "vp8", + ): Promise { this.desiredProfile = profile; this.startupVideoProfilePending = needsStartupVideoProfile(profile); const effectiveVideoProfile = startupVideoProfile(profile); @@ -195,7 +200,7 @@ export class SfuPublisher { room, videoTrack, sdk.Track.Source.ScreenShare, - videoPublishOptions(effectiveVideoProfile), + videoPublishOptions(effectiveVideoProfile, videoCodec), ); if (!this.owns(room, generation)) { return false; @@ -1165,7 +1170,7 @@ async function configurePublishedVideo( const retainedPublishOptions = { ...published.publication.options, ...videoTrack.publishOptions, - ...videoPublishOptions(profile), + ...videoPublishOptions(profile, publishedVideoCodec(published)), }; published.publication.options = retainedPublishOptions; videoTrack.publishOptions = retainedPublishOptions; @@ -1248,10 +1253,19 @@ function mergeQualityWarnings(...warnings: Array): string | null return present.length > 0 ? present.join(";") : null; } -function videoPublishOptions(profile: QualityProfile): TrackPublishOptions { +function publishedVideoCodec(published: PublishedTrack): BrowserVideoCodec { + return published.publication.options?.videoCodec === "h264" + ? "h264" + : "vp8"; +} + +function videoPublishOptions( + profile: QualityProfile, + videoCodec: BrowserVideoCodec, +): TrackPublishOptions { return { backupCodec: false, - videoCodec: "vp8", + videoCodec, screenShareEncoding: { maxBitrate: profile.maxBitrate, maxFramerate: profile.maxFramerate, diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts index be2bad5..3d2e45f 100644 --- a/src/client/webrtc/host-peer.ts +++ b/src/client/webrtc/host-peer.ts @@ -24,7 +24,10 @@ import { collectConnectionMetrics, createStatsAccumulator, } from "./stats"; -import { applyVp8Codec } from "./vp8-codec"; +import { + applyVideoCodecPreference, + type BrowserVideoCodec, +} from "./video-codec"; const MAX_PENDING_CANDIDATES = 64; type PeerIceConfig = Pick; @@ -73,6 +76,7 @@ export class HostPeer { private stream: MediaStream, private desiredProfile: QualityProfile, private readonly events: HostPeerEvents, + private readonly videoCodec: BrowserVideoCodec = "vp8", connectionId = createOpaqueId(), ) { this.connectionId = connectionId; @@ -105,8 +109,8 @@ export class HostPeer { direction: "sendonly", streams: [this.stream], }); - if (!applyVp8Codec(videoTransceiver)) { - this.setError(null, "当前浏览器无法使用 VP8 视频编码"); + if (!applyVideoCodecPreference(videoTransceiver, this.videoCodec)) { + this.setError(null, "当前浏览器无法使用支持的视频编码"); return false; } this.videoSender = videoTransceiver.sender; diff --git a/src/client/webrtc/video-codec-preflight.ts b/src/client/webrtc/video-codec-preflight.ts new file mode 100644 index 0000000..3ef5989 --- /dev/null +++ b/src/client/webrtc/video-codec-preflight.ts @@ -0,0 +1,276 @@ +import { + configureVideoSender, + startupVideoProfile, + type QualityProfile, +} from "../media/quality"; +import { + applyH264ProbeCodec, + type BrowserVideoCodec, +} from "./video-codec"; + +const PREFLIGHT_DEADLINE_MS = 4_000; +const PREFLIGHT_POLL_MS = 100; +const PREFLIGHT_MIN_SOURCE_FRAMES = 15; +const PREFLIGHT_MAX_FRAME_DEFICIT = 2; +const PREFLIGHT_FPS_WINDOW_MS = 1_000; + +export interface H264ProbeSample { + outboundId: string; + sourceId: string; + timestamp: number; + codec: string; + framesEncoded: number | null; + sourceFrames: number | null; + encodedFramesPerSecond: number | null; + sourceFramesPerSecond: number | null; + qualityLimitationReason: string | null; +} + +type StatsRecord = RTCStats & Record; + +function finiteNumber(value: unknown): number | null { + return typeof value === "number" && Number.isFinite(value) ? value : null; +} + +function stringValue(value: unknown): string | null { + return typeof value === "string" ? value : null; +} + +function recordById( + report: RTCStatsReport, + id: unknown, +): StatsRecord | null { + if (typeof id !== "string") { + return null; + } + return (report.get(id) as StatsRecord | undefined) ?? null; +} + +function readH264ProbeSample(report: RTCStatsReport): H264ProbeSample | null { + const candidates: StatsRecord[] = []; + report.forEach((raw) => { + const record = raw as StatsRecord; + if ( + record.type === "outbound-rtp" && + record.kind === "video" && + record.isRemote !== true && + finiteNumber(record.framesEncoded) !== null + ) { + candidates.push(record); + } + }); + if (candidates.length !== 1) { + return null; + } + const outbound = candidates[0]!; + const codec = recordById(report, outbound.codecId); + const source = recordById(report, outbound.mediaSourceId); + const mimeType = stringValue(codec?.mimeType); + if ( + codec?.type !== "codec" || + source?.type !== "media-source" || + mimeType?.toLowerCase() !== "video/h264" + ) { + return null; + } + return { + outboundId: outbound.id, + sourceId: source.id, + timestamp: finiteNumber(outbound.timestamp) ?? 0, + codec: mimeType, + framesEncoded: finiteNumber(outbound.framesEncoded), + sourceFrames: finiteNumber(source.frames), + encodedFramesPerSecond: finiteNumber(outbound.framesPerSecond), + sourceFramesPerSecond: finiteNumber(source.framesPerSecond), + qualityLimitationReason: stringValue(outbound.qualityLimitationReason), + }; +} + +export function h264ProbeSustainsSource( + baseline: H264ProbeSample, + current: H264ProbeSample, +): boolean | null { + if ( + baseline.outboundId !== current.outboundId || + baseline.sourceId !== current.sourceId || + current.timestamp <= baseline.timestamp || + current.codec.toLowerCase() !== "video/h264" || + current.qualityLimitationReason === "cpu" + ) { + return false; + } + if ( + baseline.framesEncoded !== null && + current.framesEncoded !== null && + baseline.sourceFrames !== null && + current.sourceFrames !== null && + current.framesEncoded >= baseline.framesEncoded && + current.sourceFrames >= baseline.sourceFrames + ) { + const encoded = current.framesEncoded - baseline.framesEncoded; + const source = current.sourceFrames - baseline.sourceFrames; + if (source < PREFLIGHT_MIN_SOURCE_FRAMES) { + return null; + } + return encoded + PREFLIGHT_MAX_FRAME_DEFICIT >= source; + } + if ( + current.timestamp - baseline.timestamp < PREFLIGHT_FPS_WINDOW_MS || + current.encodedFramesPerSecond === null || + current.sourceFramesPerSecond === null || + current.sourceFramesPerSecond <= 0 + ) { + return null; + } + return ( + current.encodedFramesPerSecond + PREFLIGHT_MAX_FRAME_DEFICIT >= + current.sourceFramesPerSecond + ); +} + +function hasProbeWindow( + baseline: H264ProbeSample, + current: H264ProbeSample, +): boolean { + if ( + baseline.outboundId !== current.outboundId || + baseline.sourceId !== current.sourceId || + current.timestamp <= baseline.timestamp + ) { + return false; + } + if ( + baseline.sourceFrames !== null && + current.sourceFrames !== null && + current.sourceFrames >= baseline.sourceFrames + ) { + return ( + current.sourceFrames - baseline.sourceFrames >= + PREFLIGHT_MIN_SOURCE_FRAMES + ); + } + return current.timestamp - baseline.timestamp >= PREFLIGHT_FPS_WINDOW_MS; +} + +function delay(milliseconds: number, signal?: AbortSignal): Promise { + return new Promise((resolve, reject) => { + if (signal?.aborted) { + reject(signal.reason); + return; + } + const onAbort = () => { + window.clearTimeout(timer); + reject(signal?.reason); + }; + const timer = window.setTimeout(() => { + signal?.removeEventListener("abort", onAbort); + resolve(); + }, milliseconds); + signal?.addEventListener("abort", onAbort, { once: true }); + }); +} + +async function waitForIceGathering( + connection: RTCPeerConnection, + deadline: number, + signal?: AbortSignal, +): Promise { + while (connection.iceGatheringState !== "complete") { + if (Date.now() >= deadline) { + throw new Error("H.264 preflight timed out"); + } + await delay(PREFLIGHT_POLL_MS, signal); + } +} + +async function runH264Probe( + track: MediaStreamTrack, + profile: QualityProfile, + signal?: AbortSignal, +): Promise { + const senderConnection = new RTCPeerConnection({ iceServers: [] }); + const receiverConnection = new RTCPeerConnection({ iceServers: [] }); + const abort = () => { + senderConnection.close(); + receiverConnection.close(); + }; + signal?.addEventListener("abort", abort, { once: true }); + try { + const transceiver = senderConnection.addTransceiver(track, { + direction: "sendonly", + }); + if (!applyH264ProbeCodec(transceiver)) { + return false; + } + await configureVideoSender( + transceiver.sender, + startupVideoProfile(profile), + ); + const deadline = Date.now() + PREFLIGHT_DEADLINE_MS; + await senderConnection.setLocalDescription( + await senderConnection.createOffer(), + ); + await waitForIceGathering(senderConnection, deadline, signal); + await receiverConnection.setRemoteDescription( + senderConnection.localDescription!, + ); + await receiverConnection.setLocalDescription( + await receiverConnection.createAnswer(), + ); + await waitForIceGathering(receiverConnection, deadline, signal); + await senderConnection.setRemoteDescription( + receiverConnection.localDescription!, + ); + + let warmupBaseline: H264ProbeSample | null = null; + let measurementBaseline: H264ProbeSample | null = null; + while (Date.now() < deadline) { + if (signal?.aborted || track.readyState === "ended") { + return false; + } + const sample = readH264ProbeSample( + await transceiver.sender.getStats(), + ); + if (sample) { + warmupBaseline ??= sample; + if ( + !measurementBaseline && + sample !== warmupBaseline && + hasProbeWindow(warmupBaseline, sample) + ) { + measurementBaseline = sample; + } else if (measurementBaseline && sample !== measurementBaseline) { + const result = h264ProbeSustainsSource(measurementBaseline, sample); + if (result !== null) { + return result; + } + } + } + await delay(PREFLIGHT_POLL_MS, signal); + } + return false; + } finally { + signal?.removeEventListener("abort", abort); + senderConnection.close(); + receiverConnection.close(); + } +} + +export async function preferredVideoCodecForTrack( + track: MediaStreamTrack, + profile: QualityProfile, + signal?: AbortSignal, +): Promise { + let codec: BrowserVideoCodec = "vp8"; + if (track.kind !== "video" || track.readyState === "ended") { + console.debug("[Screener] video codec preflight", codec); + return codec; + } + try { + codec = (await runH264Probe(track, profile, signal)) ? "h264" : "vp8"; + } catch {} + if (!signal?.aborted) { + console.debug("[Screener] video codec preflight", codec); + } + return codec; +} diff --git a/src/client/webrtc/video-codec.ts b/src/client/webrtc/video-codec.ts new file mode 100644 index 0000000..c370b3f --- /dev/null +++ b/src/client/webrtc/video-codec.ts @@ -0,0 +1,83 @@ +export type BrowserVideoCodec = "h264" | "vp8"; + +const REPAIR_CODEC_MIME_TYPES = new Set([ + "video/rtx", + "video/red", + "video/ulpfec", + "video/flexfec-03", +]); + +function videoCapabilities(): RTCRtpCodec[] { + return RTCRtpSender.getCapabilities?.("video")?.codecs ?? []; +} + +function vp8Codecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] { + return codecs.filter( + ({ mimeType }) => mimeType.toLowerCase() === "video/vp8", + ); +} + +function h264ModeOneCodecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] { + return codecs.filter(({ mimeType, sdpFmtpLine }) => { + if (mimeType.toLowerCase() !== "video/h264") { + return false; + } + return ( + !sdpFmtpLine || + /(?:^|;)\s*packetization-mode=1(?:;|$)/i.test(sdpFmtpLine) + ); + }); +} + +function repairCodecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] { + return codecs.filter(({ mimeType }) => + REPAIR_CODEC_MIME_TYPES.has(mimeType.toLowerCase()), + ); +} + +function applyCodecs( + transceiver: RTCRtpTransceiver, + codecs: RTCRtpCodec[], +): boolean { + if (typeof transceiver.setCodecPreferences !== "function") { + return false; + } + try { + transceiver.setCodecPreferences(codecs); + return true; + } catch { + return false; + } +} + +export function applyVideoCodecPreference( + transceiver: RTCRtpTransceiver, + preference: BrowserVideoCodec, +): boolean { + if (typeof RTCRtpSender === "undefined") { + return false; + } + const codecs = videoCapabilities(); + const vp8 = vp8Codecs(codecs); + if (vp8.length === 0) { + return false; + } + const h264 = preference === "h264" ? h264ModeOneCodecs(codecs) : []; + return applyCodecs(transceiver, [ + ...h264, + ...vp8, + ...repairCodecs(codecs), + ]); +} + +export function applyH264ProbeCodec( + transceiver: RTCRtpTransceiver, +): boolean { + if (typeof RTCRtpSender === "undefined") { + return false; + } + const codecs = videoCapabilities(); + const h264 = h264ModeOneCodecs(codecs); + return h264.length > 0 && + applyCodecs(transceiver, [...h264, ...repairCodecs(codecs)]); +} diff --git a/src/client/webrtc/viewer-relay.ts b/src/client/webrtc/viewer-relay.ts index 5b89c4c..38ddd5d 100644 --- a/src/client/webrtc/viewer-relay.ts +++ b/src/client/webrtc/viewer-relay.ts @@ -7,6 +7,8 @@ import type { QualityProfile } from "../media/quality"; import type { PeerSnapshot } from "../types"; import { HostPeer } from "./host-peer"; import { MAX_ENDPOINT_MEDIA_CHILDREN } from "./media-assignment"; +import type { BrowserVideoCodec } from "./video-codec"; +import { preferredVideoCodecForTrack } from "./video-codec-preflight"; interface ViewerRelayEvents { sendSignal: (peerId: string, payload: SignalPayload) => boolean; @@ -31,6 +33,11 @@ export class ViewerRelay { private preparedCandidate: PreparedRouteCandidate | null = null; private plannedChildPeerIds: string[] = []; private syncQueue = Promise.resolve(); + private videoCodec: BrowserVideoCodec = "vp8"; + private codecProbeTrack: MediaStreamTrack | null = null; + private codecProbeAbort: AbortController | null = null; + private codecProbePromise: Promise | null = null; + private codecProbeSettled = false; private disposed = false; constructor( @@ -84,7 +91,14 @@ export class ViewerRelay { this.preparedCandidate = { ...candidate }; this.plannedChildPeerIds = planned; if (candidate.transport === "direct") { - this.startPreparedChild(revision, candidate, stream); + const pendingProbe = this.codecProbePromise; + if (pendingProbe) { + void pendingProbe.then(() => { + this.startPreparedChildIfCurrent(revision, candidate, stream); + }); + } else { + this.startPreparedChildIfCurrent(revision, candidate, stream); + } } return true; } @@ -126,6 +140,7 @@ export class ViewerRelay { this.preparedCandidate = null; this.plannedChildPeerIds = []; this.discardPreparedPeer(); + this.startCodecProbe(); } private discardPreparedPeer(): void { @@ -155,6 +170,9 @@ export class ViewerRelay { } } this.childPeerIds = nextChildPeerIds; + if (this.childPeerIds.length > 0) { + this.cancelCodecProbe(); + } for (const childPeerId of this.childPeerIds) { const peer = this.peers.get(childPeerId); if ( @@ -166,17 +184,27 @@ export class ViewerRelay { this.startPeer(childPeerId, this.stream); } } + this.startCodecProbe(); } setStream(stream: MediaStream): void { if (this.disposed) { return; } + const nextVideoTrack = stream.getVideoTracks()[0] ?? null; + if (nextVideoTrack !== this.codecProbeTrack) { + this.cancelCodecProbe(); + this.codecProbeTrack = nextVideoTrack; + if (!this.codecProbeSettled) { + this.videoCodec = "vp8"; + } + } const preparedPeer = this.preparedChild?.peer ?? null; this.stream = stream; this.syncQueue = this.syncQueue .then(() => this.syncStream(stream, preparedPeer)) .catch(() => undefined); + this.startCodecProbe(); } updateProfile(profile: QualityProfile): Promise { @@ -271,6 +299,10 @@ export class ViewerRelay { } stop(): void { + this.cancelCodecProbe(); + this.codecProbeTrack = null; + this.codecProbeSettled = false; + this.videoCodec = "vp8"; this.stream = null; this.discardPreparedChild(); this.disposePeers(); @@ -281,6 +313,8 @@ export class ViewerRelay { return; } this.disposed = true; + this.cancelCodecProbe(); + this.codecProbeTrack = null; this.stream = null; this.discardPreparedChild(); this.childPeerIds = []; @@ -316,6 +350,23 @@ export class ViewerRelay { }); } + private startPreparedChildIfCurrent( + revision: number, + candidate: PreparedRouteCandidate, + stream: MediaStream, + ): void { + if ( + this.disposed || + this.stream !== stream || + this.preparedChild || + this.preparedRevision !== revision || + !sameCandidate(this.preparedCandidate, candidate) + ) { + return; + } + this.startPreparedChild(revision, candidate, stream); + } + private async syncStream( stream: MediaStream, preparedPeer: HostPeer | null, @@ -475,11 +526,61 @@ export class ViewerRelay { } }, }, + this.videoCodec, connectionId, ); return peer; } + private startCodecProbe(): void { + const track = this.codecProbeTrack; + if ( + this.disposed || + this.codecProbeSettled || + this.codecProbeAbort || + this.codecProbePromise || + !track || + track.readyState === "ended" || + this.childPeerIds.length > 0 || + this.preparedChild + ) { + return; + } + const controller = new AbortController(); + this.codecProbeAbort = controller; + const promise = preferredVideoCodecForTrack( + track, + this.desiredProfile, + controller.signal, + ) + .then((codec) => { + if ( + this.codecProbeAbort !== controller || + controller.signal.aborted || + this.codecProbeTrack !== track || + this.disposed + ) { + return; + } + this.videoCodec = codec; + this.codecProbeSettled = true; + this.codecProbeAbort = null; + }) + .finally(() => { + if (this.codecProbePromise === promise) { + this.codecProbePromise = null; + this.startCodecProbe(); + } + }); + this.codecProbePromise = promise; + } + + private cancelCodecProbe(): void { + const controller = this.codecProbeAbort; + this.codecProbeAbort = null; + controller?.abort(); + } + private failPreparedChild(peer: HostPeer): void { const prepared = this.preparedChild; if (prepared?.peer !== peer) { diff --git a/src/client/webrtc/vp8-codec.ts b/src/client/webrtc/vp8-codec.ts deleted file mode 100644 index 6d0956a..0000000 --- a/src/client/webrtc/vp8-codec.ts +++ /dev/null @@ -1,35 +0,0 @@ -const REPAIR_CODEC_MIME_TYPES = new Set([ - "video/rtx", - "video/red", - "video/ulpfec", - "video/flexfec-03", -]); - -export function applyVp8Codec(transceiver: RTCRtpTransceiver): boolean { - if ( - typeof RTCRtpSender === "undefined" || - typeof transceiver.setCodecPreferences !== "function" - ) { - return false; - } - - try { - const codecs = RTCRtpSender.getCapabilities?.("video")?.codecs ?? []; - const vp8Codecs = codecs.filter( - ({ mimeType }) => mimeType.toLowerCase() === "video/vp8", - ); - if (vp8Codecs.length === 0) { - return false; - } - - transceiver.setCodecPreferences([ - ...vp8Codecs, - ...codecs.filter(({ mimeType }) => - REPAIR_CODEC_MIME_TYPES.has(mimeType.toLowerCase()), - ), - ]); - return true; - } catch { - return false; - } -} diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts index 2b8515d..efd8424 100644 --- a/tests/host-peer-tracks.test.ts +++ b/tests/host-peer-tracks.test.ts @@ -7,12 +7,21 @@ import { import { HostProvisionalChild } from "../src/client/media/host-provisional-child.ts"; import type { PeerSnapshot } from "../src/client/types.ts"; import { HostPeer } from "../src/client/webrtc/host-peer.ts"; +import type { BrowserVideoCodec } from "../src/client/webrtc/video-codec.ts"; import { ViewerRelay } from "../src/client/webrtc/viewer-relay.ts"; import type { IceConfig, ParticipantRouteAssignment, } from "../src/shared/protocol.ts"; +const codecPreflight = vi.hoisted(() => ({ + probe: vi.fn(async (): Promise<"h264" | "vp8"> => "vp8"), +})); + +vi.mock("../src/client/webrtc/video-codec-preflight.ts", () => ({ + preferredVideoCodecForTrack: codecPreflight.probe, +})); + const statsCallbacks: Array<() => void> = []; class FakeSender { @@ -387,6 +396,7 @@ function createPeer( onUpdate: (snapshot: PeerSnapshot) => void = () => undefined, iceConfig: IceConfig = { iceServers: [] }, profile: QualityProfile = QUALITY_PROFILES["720p30"], + videoCodec: BrowserVideoCodec = "vp8", ): HostPeer { return new HostPeer( "viewer-peer", @@ -397,6 +407,7 @@ function createPeer( sendSignal: () => true, onUpdate, }, + videoCodec, ); } @@ -434,6 +445,8 @@ async function completeVideoStartup( } beforeEach(() => { + codecPreflight.probe.mockReset(); + codecPreflight.probe.mockResolvedValue("vp8"); FakePeerConnection.latest = null; FakePeerConnection.instances = []; FakePeerConnection.activeCount = 0; @@ -519,6 +532,50 @@ describe("HostPeer source replacement", () => { expect(connection.createOfferCallCount).toBe(1); }); + it("prefers native H264 mode 1 and retains VP8 fallback", async () => { + vi.stubGlobal("RTCRtpSender", { + getCapabilities: () => ({ + codecs: [ + { + mimeType: "video/H264", + clockRate: 90_000, + sdpFmtpLine: "packetization-mode=0;profile-level-id=42001f", + }, + { + mimeType: "video/H264", + clockRate: 90_000, + sdpFmtpLine: "packetization-mode=1;profile-level-id=42001f", + }, + { mimeType: "video/VP8", clockRate: 90_000 }, + { mimeType: "video/rtx", clockRate: 90_000 }, + { mimeType: "video/RED", clockRate: 90_000 }, + ], + headerExtensions: [], + }), + }); + const peer = createPeer( + createStream(createTrack("video", "video"), null), + () => undefined, + { iceServers: [] }, + QUALITY_PROFILES["720p30"], + "h264", + ); + + await expect(peer.start()).resolves.toBe(true); + + expect( + FakePeerConnection.latest!.codecPreferenceCalls[0]?.map((codec) => [ + codec.mimeType.toLowerCase(), + codec.sdpFmtpLine ?? null, + ]), + ).toEqual([ + ["video/h264", "packetization-mode=1;profile-level-id=42001f"], + ["video/vp8", null], + ["video/rtx", null], + ["video/red", null], + ]); + }); + it("fails before creating an offer when codec preferences are unavailable", async () => { FakePeerConnection.omitCodecPreferenceSetter = true; const updates: PeerSnapshot[] = []; @@ -531,7 +588,7 @@ describe("HostPeer source replacement", () => { expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0); expect(FakePeerConnection.latest!.localDescription).toBeNull(); - expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码"); + expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码"); }); it("fails before creating an offer when VP8 is unavailable", async () => { @@ -554,7 +611,7 @@ describe("HostPeer source replacement", () => { expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0); expect(FakePeerConnection.latest!.localDescription).toBeNull(); - expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码"); + expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码"); }); it("fails before creating an offer when codec preference setup fails", async () => { @@ -569,7 +626,7 @@ describe("HostPeer source replacement", () => { expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0); expect(FakePeerConnection.latest!.localDescription).toBeNull(); - expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码"); + expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码"); }); it("applies STUN-only ICE configuration at creation and update", () => { @@ -1384,6 +1441,7 @@ function hostProvisionalInput( iceConfig: { iceServers: [] }, stream, profile: QUALITY_PROFILES["720p30"], + videoCodec: "vp8" as const, }; } @@ -1516,6 +1574,29 @@ describe("ViewerRelay downstream ownership", () => { transport: "direct" as const, }); + it("uses a completed H264 probe only for a future child", async () => { + codecPreflight.probe.mockResolvedValueOnce("h264"); + const relay = new ViewerRelay( + { iceServers: [] }, + QUALITY_PROFILES["720p30"], + { sendSignal: () => true }, + ); + relay.setStream(createStream(createTrack("video", "h264-source"), null)); + await vi.waitFor(() => expect(codecPreflight.probe).toHaveBeenCalledOnce()); + await Promise.resolve(); + relay.setChildren(["h264-child"]); + await vi.waitFor(() => + expect(FakePeerConnection.latest?.codecPreferenceCalls).toHaveLength(1), + ); + + expect( + FakePeerConnection.latest!.codecPreferenceCalls[0]?.map(({ mimeType }) => + mimeType.toLowerCase(), + ), + ).toEqual(["video/h264", "video/vp8"]); + relay.dispose(); + }); + it("promotes the exact prepared child connection within the current Viewer cap", async () => { const signals: Array<{ peerId: string; connectionId: string }> = []; const relay = new ViewerRelay( @@ -1593,6 +1674,7 @@ describe("ViewerRelay downstream ownership", () => { ); relay.setStream(createStream(createTrack("video", "rollback-video"), null)); expect(relay.prepareChild(7, routeCandidate(7, "first-probe"), ["first-probe"])).toBe(true); + await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull()); const firstProbe = FakePeerConnection.latest!; expect(relay.prepareChild(8, routeCandidate(8, "second-probe"), ["second-probe"])).toBe(true); const secondProbe = FakePeerConnection.latest!; @@ -1675,6 +1757,7 @@ describe("ViewerRelay downstream ownership", () => { ["prepared-child"], ), ).toBe(true); + await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull()); const preparedConnection = FakePeerConnection.latest!; expect(preparedConnection.senders[1]?.track).toBeNull(); @@ -1701,6 +1784,7 @@ describe("ViewerRelay downstream ownership", () => { ); relay.setStream(createStream(createTrack("video", "cap-video"), null)); expect(relay.prepareChild(1, routeCandidate(1, "child-0"), ["child-0"])).toBe(true); + await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull()); const preparedConnection = FakePeerConnection.latest!; preparedConnection.connectionState = "connected"; relay.activateChildren(1, ["child-0"]); diff --git a/tests/route-transition.test.ts b/tests/route-transition.test.ts index 31afb92..ff44c4f 100644 --- a/tests/route-transition.test.ts +++ b/tests/route-transition.test.ts @@ -259,6 +259,7 @@ describe("minimal route transition contracts", () => { const route = new HostSfuRoute({ getStream: () => ({}) as MediaStream, getProfile: () => QUALITY_PROFILES["720p30"], + getVideoCodec: () => "h264", reconcileChildren: () => undefined, send: (message) => { messages.push(message); @@ -281,6 +282,11 @@ describe("minimal route transition contracts", () => { }); await route.acceptConfig(sfuConfig(1)); expect(publishers[0]?.activate).toHaveBeenCalledOnce(); + expect(publishers[0]?.activate).toHaveBeenCalledWith( + expect.anything(), + QUALITY_PROFILES["720p30"], + "h264", + ); route.accept({ revision: 2, @@ -311,6 +317,7 @@ describe("minimal route transition contracts", () => { const route = new HostSfuRoute({ getStream: () => ({}) as MediaStream, getProfile: () => QUALITY_PROFILES["720p30"], + getVideoCodec: () => "vp8", reconcileChildren: (children) => reconciledChildren.push(children), send: () => true, createPublisher: () => { @@ -365,6 +372,7 @@ describe("minimal route transition contracts", () => { const route = new HostSfuRoute({ getStream: () => ({}) as MediaStream, getProfile: () => QUALITY_PROFILES["720p30"], + getVideoCodec: () => "vp8", reconcileChildren: () => undefined, send: () => true, createPublisher: () => { diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts index 9636888..6d5bcb1 100644 --- a/tests/sfu-client.test.ts +++ b/tests/sfu-client.test.ts @@ -1,5 +1,6 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { QUALITY_PROFILES } from "../src/client/media/quality.ts"; import { SfuPublisher } from "../src/client/sfu/publisher.ts"; import { SfuSubscriber } from "../src/client/sfu/subscriber.ts"; import type { ConnectionMetrics } from "../src/client/types.ts"; @@ -878,6 +879,27 @@ describe("SfuPublisher", () => { }); }); + it("publishes one H264 source without a backup codec", async () => { + const publisher = new SfuPublisher(); + const video = track("video", "video-h264"); + await publisher.connect(connection); + + await expect( + publisher.activate(stream(video), qualityProfile, "h264"), + ).resolves.toBe(true); + + const publication = livekit.state.rooms[0].localParticipant.publications[0]; + expect(publication.options).toMatchObject({ + source: Track.Source.ScreenShare, + backupCodec: false, + videoCodec: "h264", + }); + await expect( + publisher.updateProfile(QUALITY_PROFILES["720p30"]), + ).resolves.toBe(true); + expect(publication.options.videoCodec).toBe("h264"); + }); + it("keeps media and prior audio readback when a live update fails", async () => { const publisher = new SfuPublisher(); await publisher.connect(connection); diff --git a/tests/stats-grid.test.ts b/tests/stats-grid.test.ts index ab0f300..59ec191 100644 --- a/tests/stats-grid.test.ts +++ b/tests/stats-grid.test.ts @@ -107,14 +107,14 @@ describe("StatsGrid progressive disclosure", () => { createElement(StatsGrid, { metrics: { ...metrics, - codec: "video/H264", + codec: "video/VP9", audioCodec: "audio/PCMU", }, direction: "receive", }), ); - expect(html).toContain("视频编码为 H264,预期 VP8"); + expect(html).toContain("视频编码为 VP9,预期 H264 或 VP8"); expect(html).toContain("音频编码为 PCMU,预期 Opus"); expect(html).not.toContain("视频 Codec"); expect(html).not.toContain("音频 Codec"); diff --git a/tests/video-codec-preflight.test.ts b/tests/video-codec-preflight.test.ts new file mode 100644 index 0000000..9c677ab --- /dev/null +++ b/tests/video-codec-preflight.test.ts @@ -0,0 +1,128 @@ +import { describe, expect, it } from "vitest"; + +import { + h264ProbeSustainsSource, + type H264ProbeSample, +} from "../src/client/webrtc/video-codec-preflight.ts"; + +function sample( + overrides: Partial = {}, +): H264ProbeSample { + return { + outboundId: "outbound", + sourceId: "source", + timestamp: 1_000, + codec: "video/H264", + framesEncoded: 10, + sourceFrames: 10, + encodedFramesPerSecond: 30, + sourceFramesPerSecond: 30, + qualityLimitationReason: "none", + ...overrides, + }; +} + +describe("H264 sender preflight", () => { + it("accepts bounded pipeline lag against the same source", () => { + const baseline = sample(); + const current = sample({ + timestamp: 2_000, + framesEncoded: 39, + sourceFrames: 40, + }); + + expect(h264ProbeSustainsSource(baseline, current)).toBe(true); + }); + + it("rejects sustained encoder frame dropping", () => { + const baseline = sample(); + const current = sample({ + timestamp: 2_000, + framesEncoded: 22, + sourceFrames: 40, + encodedFramesPerSecond: 12, + sourceFramesPerSecond: 30, + }); + + expect(h264ProbeSustainsSource(baseline, current)).toBe(false); + }); + + it("waits for enough source progress before deciding", () => { + const baseline = sample(); + const current = sample({ + timestamp: 1_400, + framesEncoded: 20, + sourceFrames: 20, + }); + + expect(h264ProbeSustainsSource(baseline, current)).toBeNull(); + }); + + it("rejects CPU-limited or identity-mismatched samples", () => { + const baseline = sample(); + + expect( + h264ProbeSustainsSource( + baseline, + sample({ + timestamp: 2_000, + framesEncoded: 40, + sourceFrames: 40, + qualityLimitationReason: "cpu", + }), + ), + ).toBe(false); + expect( + h264ProbeSustainsSource( + baseline, + sample({ + outboundId: "replacement", + timestamp: 2_000, + framesEncoded: 40, + sourceFrames: 40, + }), + ), + ).toBe(false); + }); + + it("uses reported cadence when cumulative source frames are unavailable", () => { + const baseline = sample({ framesEncoded: null, sourceFrames: null }); + + expect( + h264ProbeSustainsSource( + baseline, + sample({ + timestamp: 2_000, + framesEncoded: null, + sourceFrames: null, + encodedFramesPerSecond: 29, + sourceFramesPerSecond: 30, + }), + ), + ).toBe(true); + expect( + h264ProbeSustainsSource( + baseline, + sample({ + timestamp: 2_000, + framesEncoded: null, + sourceFrames: null, + encodedFramesPerSecond: 12, + sourceFramesPerSecond: 30, + }), + ), + ).toBe(false); + expect( + h264ProbeSustainsSource( + baseline, + sample({ + timestamp: 1_500, + framesEncoded: null, + sourceFrames: null, + encodedFramesPerSecond: 30, + sourceFramesPerSecond: 30, + }), + ), + ).toBeNull(); + }); +});