feat: prefer proven H264 browser senders
This commit is contained in:
1 parent
6abd5c8030
commit
6b3c3de8a0
19 files changed
+812
-66
No files matched your search
@@ -23,7 +23,8 @@ it does not need a second bitrate, resolution, FPS, or layer-control system.
|
||||
|
||||
## Decision
|
||||
|
||||
1. Browser video makes one internal, sender-scoped codec decision. A bounded
|
||||
1. Browser video makes one internal, share-generation-scoped codec decision per
|
||||
sending endpoint. A bounded
|
||||
actual-`RTCPeerConnection` preflight measures H.264 encoded progress against
|
||||
a deterministic moving probe track at the current share's effective target
|
||||
dimensions and frame rate. The real capture supplies those target settings,
|
||||
@@ -43,7 +44,7 @@ it does not need a second bitrate, resolution, FPS, or layer-control system.
|
||||
After accepting an answer, the Host reapplies the current video profile to
|
||||
the negotiated sender because the browser may replace or rewrite encoding
|
||||
parameters during negotiation.
|
||||
5. Direct and Browser-relay offers with a proved source order H.264 before VP8
|
||||
5. Direct and Browser-relay offers with a proved share source order H.264 before VP8
|
||||
in one standard codec-preference list; both endpoints select their first
|
||||
common codec without a parallel media connection. A failed source gate
|
||||
offers VP8 only. Existing edges are not renegotiated when a later relay gate
|
||||
|
||||
@@ -87,7 +87,7 @@ Last updated: 2026-08-26
|
||||
- WebRTC owns direct/peer media adaptation. The Host reapplies the selected video
|
||||
profile after answer negotiation; no periodic application controller exists.
|
||||
- The production SFU publisher uses the Host's H.264/VP8 decision and selected
|
||||
ceiling but no custom simulcast layers. Pinned LiveKit defaults own
|
||||
ceiling but no backup codec or custom simulcast layers. Pinned LiveKit defaults own
|
||||
representations, Dynacast owns aggregate
|
||||
demand, and server send-side BWE owns subscriber forwarding. AdaptiveStream
|
||||
stays disabled because any Viewer may relay its received track.
|
||||
@@ -118,8 +118,10 @@ Last updated: 2026-08-26
|
||||
|
||||
## Current Source And Production
|
||||
|
||||
- Production and the matching canonical application/runtime tree use strict
|
||||
`screener-v12`; exact identity is indexed by [status](./status.md).
|
||||
- Production and canonical source use strict `screener-v12`, but the pending
|
||||
application candidate adds the internal H.264/VP8 decision while production
|
||||
remains on its prior fixed-VP8 artifact. Exact identities are indexed by
|
||||
[status](./status.md).
|
||||
- Production uses random memory rooms, 20-Viewer admission, endpoint cap `2`,
|
||||
fixed 9,000-publication ingress and `9000 * 20` subscription egress admission,
|
||||
STUN UDP 3478, LiveKit media UDP 7882, and Web TCP 80/443. Node 8787 and
|
||||
|
||||
+4
-4
@@ -2,7 +2,7 @@
|
||||
|
||||
## 2026-08-18 14:45
|
||||
|
||||
状态:production 与 canonical application/runtime tree 使用严格 `screener-v12`。Browser 视频固定 VP8 并使用 `contentHint = "motion"`,音频使用 `music`;codec UI/state/wire、自定义 SFU layer list与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配;screen audio 使用 96/128/192 kbps ceiling、stereo、DTX off 与 RED off。Viewer 页面恢复、原生控件和当前线路重连已进入 source 与 production,精确执行状态由 `docs/status.md` 持有。
|
||||
状态:production 与 canonical source 使用严格 `screener-v12`。Canonical source 已实现每分享代次的 H.264 实际 sender gate 与 VP8 fallback,production 仍是待切换的固定 VP8 artifact;两者都使用 video `contentHint = "motion"` 和 audio `music`,且 codec UI/state/wire、自定义 SFU layer list与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配;screen audio 使用 96/128/192 kbps ceiling、stereo、DTX off 与 RED off。精确执行状态由 `docs/status.md` 持有。
|
||||
|
||||
## 目标与边界
|
||||
|
||||
@@ -162,7 +162,7 @@ accepted 的唯一 wire 是 `screener-v12`,不做版本协商、旧 parser、
|
||||
|
||||
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级,实际结果由 stats 观测;拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。
|
||||
|
||||
Accepted `screener-v12` Web 视频使用页面内 sender-scoped H.264/VP8 决策。初始 Host capture 在取得 room/route authority 前,以一对无 ICE server 的本地标准 `RTCPeerConnection` 运行一个有界 H.264-only sender probe。Probe 使用按当前分享 effective target dimensions/frame rate 生成的短时确定性运动 Canvas track,并以 `motion` 表达同一内容意图;真实 capture/upstream track 只提供目标规格,其当前内容是否变化不参与 encoder capability 判断。Gate 只用实际协商 codec、同一身份的 probe media-source/encoded progress 与 bounded completion 判断,不读取 UA/GPU/MFT allowlist,也不单独相信 capability 或 `powerEfficientEncoder`;标准 Canvas capture 不可用、失败或 inconclusive 时使用 VP8。成功结果让后续 P2P/browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与 RTX/RED/FEC repair capabilities;失败或 inconclusive 只列 VP8 与 repair。Relay 收到新 upstream stream 后在后台运行同一 gate,完成前建立的 child 使用 VP8,完成后只影响未来 child。无法应用 codec preferences 或缺少 VP8 时明确失败。SFU publication 使用 Host source 当时的同一 H.264/VP8 决定并关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling;当 desired degradation preference 允许分辨率下降时,启动阶段的 effective preference 暂为 `maintain-resolution`,当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset,不增加周期重写、质量评分或应用层调档器。UI 和 quality state/wire 没有 codec 字段;gate 结果不持久化、不改写已连接 edge,实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。
|
||||
Accepted `screener-v12` Web 视频使用页面内 share-generation-scoped H.264/VP8 决策。初始 Host capture 在取得 room/route authority 前,以一对无 ICE server 的本地标准 `RTCPeerConnection` 运行一个有界 H.264-only sender probe。Probe 使用按当前分享 effective target dimensions/frame rate 生成的短时确定性运动 Canvas track,并以 `motion` 表达同一内容意图;真实 capture/upstream track 只提供目标规格,其当前内容是否变化不参与 encoder capability 判断。Gate 只用实际协商 codec、同一身份的 probe media-source/encoded progress 与 bounded completion 判断,不读取 UA/GPU/MFT allowlist,也不单独相信 capability 或 `powerEfficientEncoder`;标准 Canvas capture 不可用、失败或 inconclusive 时使用 VP8。成功结果让后续 P2P/browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与 RTX/RED/FEC repair capabilities;失败或 inconclusive 只列 VP8 与 repair。Relay 在该分享代首次收到 upstream video 且没有 active/prepared child 时运行同一 gate;若 child prepare 先到,则等待已经在运行的 bounded gate 结束后只创建一份真实 child sender。质量设置、换源和换父不重复 gate。无法应用 codec preferences 或缺少 VP8 时明确失败。SFU publication 使用 Host share generation 的同一 H.264/VP8 决定并关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling;当 desired degradation preference 允许分辨率下降时,启动阶段的 effective preference 暂为 `maintain-resolution`,当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset,不增加周期重写、质量评分或应用层调档器。UI 和 quality state/wire 没有 codec 字段;gate 结果不持久化、不改写已连接 edge,实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。
|
||||
|
||||
Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`autoGainControl` 和 `voiceIsolation`,并以 `channelCount: { ideal: 2 }` 偏好双声道;这是当前 Chrome/Edge Host 基线的实现契约,用于避开网页 `getDisplayMedia()` 默认的语音处理和单声道 APM 候选,不宣称跨浏览器保证,也不固定采样率。所有值都是非强制 post-selection preference;不使用 `exact`、`min` 或 `advanced`。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`:窗口选项请求该窗口/应用音频,整个屏幕选项允许用户勾选系统音频;标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些约束或 picker hints,且没有对应的标准来源类型读回,因此 UI 不额外推断或标注来源。系统音频由用户显式选择,可能包含语音或通知;video-only 始终可用。
|
||||
|
||||
@@ -172,7 +172,7 @@ Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`
|
||||
|
||||
直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`,不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流;成功后为每个现有 sender 调用 `replaceTrack()`,健康连接不重新协商。初始 offer 预留 send-only audio transceiver,使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败,只重建该 peer;新流提交后才停止旧流,并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置,避免把不同 track 的计数器混算。
|
||||
|
||||
直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;quality mutation 不重新运行 codec gate,新加入或重建的 Browser sender 使用当前 source 的内存结果与最新其他权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停禁用当前 capture stream 的全部音视频 tracks;Host 通过当前 session 与 `shareGeneration` 发送一个 paused boolean,服务端同步 `host-status`、route pause 与 `authenticated` 快照,Viewer 在舞台显示“分享者已暂停”。Resume 使用同一状态更新并重新启用同一批 tracks,不另设 authorization、source acknowledgement 或 proof。晚加入和信令重连从快照恢复,恢复、停止、新分享或会话替换清除旧状态;不得把 track mute、黑帧或网络停顿推断成主动暂停。房间与连接保持不变。
|
||||
直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;quality mutation 不重新运行 codec gate,新加入或重建的 Browser sender 使用当前分享代次的内存结果与最新其他权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停禁用当前 capture stream 的全部音视频 tracks;Host 通过当前 session 与 `shareGeneration` 发送一个 paused boolean,服务端同步 `host-status`、route pause 与 `authenticated` 快照,Viewer 在舞台显示“分享者已暂停”。Resume 使用同一状态更新并重新启用同一批 tracks,不另设 authorization、source acknowledgement 或 proof。晚加入和信令重连从快照恢复,恢复、停止、新分享或会话替换清除旧状态;不得把 track mute、黑帧或网络停顿推断成主动暂停。房间与连接保持不变。
|
||||
|
||||
Host 本地预览直接绑定现有 capture stream,不创建 viewer、`RTCPeerConnection` 或额外媒体边,也不产生 SFU/应用服务器出口。预览仍可能有本地合成、缩放和 GPU 显示成本;在同一动态场景对预览开/关采集 Host CPU、GPU、encode time 和实际 FPS。后台性能诊断必须把 Host 页面前台/遮挡/最小化、被捕获 source 前台/遮挡/最小化以及 preview 播放/暂停分成独立变量,并关联 document visibility/focus、track mute/readyState 与 capture/outbound/inbound stats。页面 timer、伪活跃和 Screen Wake Lock 都不构成屏幕捕获保活方案;若标准浏览器无法维持目标行为,应记录平台能力边界而不是制造第二套 lifecycle。只有稳定收益且捕获、编码和远端画面不变时,才把直播后的默认动态预览改为静止缩略图或隐藏并允许按需展开;隐藏预览绝不停止 capture track 或 sender encode。
|
||||
|
||||
@@ -363,7 +363,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio
|
||||
|
||||
## 当前交付边界
|
||||
|
||||
- Production 与 canonical application/runtime tree 使用严格单一 `screener-v12` wire。当前实现包含短 room-lived grant、明确 missing-room 结果、产品诊断下载移除、完整短 URL 邀请 UI、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性,固定 Browser VP8、video `contentHint = "motion"`、audio `music`,且不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list或分享生命周期 codec 分支。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off;精确执行状态由 `docs/status.md` 持有。
|
||||
- Production 与 canonical source 使用严格单一 `screener-v12` wire。Canonical source 的每分享代次 actual-sender gate 在证据通过时让未来 direct/relay 和 Host 单一 SFU publication 使用 H.264,否则使用 VP8;production 仍是固定 VP8 artifact。两者都使用 video `contentHint = "motion"`、audio `music`,不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list、backup publication或健康 edge codec 切换。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off;精确执行状态由 `docs/status.md` 持有。
|
||||
- Pause/Resume 由当前 Host session 和 `shareGeneration` 约束。Host 后台性能和 VP8 hardware acceleration 分别按受控根因证据诊断;标准 ICE/SFU 按真实异构网络验收。
|
||||
- 当前发布面只包含 Web Host、Viewer 和 browser relay。Native sender、capture helper、shared-encode executable 与两个分发包仍由各自后续决策和物理证据驱动,不属于当前 Browser release。
|
||||
|
||||
|
||||
+3
-3
@@ -27,7 +27,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- 端点发送容量:所有非服务器端点使用同一个服务端权威稳态 outbound media-copy capacity,由唯一部署项 `ENDPOINT_MEDIA_COPY_CAPACITY` 配置,默认 `2` 且只接受 `1`、`2` 或 `3`;不读取旧键或保留第二套 tier。peer child 或 Host 的唯一 SFU publication 各消耗一份,上游接收不消耗该预算。客户端只能声明不高于部署值的可用容量,角色、浏览器、UA、设备类型和页面可见性不产生不同上限。服务端在每次 `authenticated` 权威快照中返回最终 `endpointMediaCopyCapacity`;普通非 peer-assisted 模式也只授权稳定加入顺序中的至多 `C` 个 Host child,超额 Viewer 保持入房等待并在槽位释放后提升,信令不能绕过该集合。Browser Host 与 Viewer 严格校验该值并用它约束稳态 assignment;并行产生媒体的 candidate 必须持有唯一 `TransitionOverlapSlot`,active+pending 物理 copy 只可达到 `min(C + 1, 3)` 并在 commit/abort 后释放。`C=3` 满载时必须先收到 active bounded-gap retirement 再 prepare,绝不产生第四份 copy。SFU subscriber egress 由独立 server admission 记账。当前不兼容房间合同以单一 release 原子升级;缺失/非法 cap、旧 wire 与 executable sender都在取得room/route authority前fail closed。
|
||||
- P2P 优先:每条普通逻辑媒体边先尝试 direct/peer UDP,服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布;每名 Viewer 同时至多一个 active upstream,失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate;Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction,并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时,同一 controller 用有限 carrier cursor 建立唯一 publication;carrier 必须能在既有 overlap 上限内完成,不能先切断健康分支。终态归原始等待需求,publication 成功后该需求先取得 SFU 画面,剩余 direct candidate 再在后台收敛;完整事务由 ADR-0005 定义。
|
||||
- NAT 与传输边界:STUN 必须可用,所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`,保留 LiveKit signaling 交付的授权 SFU UDP candidates,并由标准 ICE 选择 nominated non-relay pair;不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC,也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威;Screener 不合成 remote candidate、不预测端口、不分类 NAT,也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity;成功即停止,失败、超时或绑定失效即撤销,direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP,全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers,再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media;当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输,必须把它作为 LiveKit 内部能力重新决策,不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target;HTTPS/WSS 继续独立使用 TLS/TCP。
|
||||
- 质量控制:推荐档位始终只有 720p30(3 Mbps)、1080p30(5 Mbps)、1080p60(8 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`(均衡),`maintain-resolution`(清晰)、`maintain-framerate`(流畅)保留为显式选择。Browser 对每个新分享 source 运行一次有界实际 sender 预检;H.264 源帧与编码帧进度通过时,未来 direct/relay offer 原生排序 H.264 后接 VP8,Host 的唯一 SFU publication 也使用 H.264;不支持、证据不足、超时或表现不达标时只使用 VP8。界面、质量状态和 wire 不提供 codec 选择,结果不持久化,也不切换已连接线路。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling,音频默认选择中档 128 kbps,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由 WebRTC/LiveKit 决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。
|
||||
- 质量控制:推荐档位始终只有 720p30(3 Mbps)、1080p30(5 Mbps)、1080p60(8 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`(均衡),`maintain-resolution`(清晰)、`maintain-framerate`(流畅)保留为显式选择。Browser Host 在每个新分享代次运行一次有界实际 sender 预检,Relay 在该代首次取得 upstream source 且尚无 child 时运行同一预检;H.264 源帧与编码帧进度通过时,未来 direct/relay offer 原生排序 H.264 后接 VP8,Host 的唯一 SFU publication 也使用 H.264;不支持、证据不足、超时或表现不达标时只使用 VP8。界面、质量状态和 wire 不提供 codec 选择,结果不持久化,也不因质量设置、换源或换父重复探测或切换已连接线路。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling,音频默认选择中档 128 kbps,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由 WebRTC/LiveKit 决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。
|
||||
- 路径隔离表示:direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC,不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 不设置 `screenShareSimulcastLayers`,由锁定的 LiveKit client 默认行为拥有当前 H.264 或 VP8 source decision 对应的 screen-share representations、source replacement 与 republish;LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 负责需求聚合与每个 subscriber 的实际下行层。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。所有当前 Viewer 都可能承担 peer children,因此 subscriber 保持 `adaptiveStream: false`,不得让本地元素尺寸或可见性控制 relay ingress。表示选择不改变拓扑、端点容量或路由失效权威。
|
||||
- 路由失效输入:应用不推断 parent-wide 质量,也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`,或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时,使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`,route/connection变化或新帧重置、暂停抑制,同一 edge只报告一次,不增加轮询器。Active SFU route 暂时没有 video track、stats report 为空或 stats 读取失败时,现有采样 cadence 仍把该次作为无 decoded progress 推进 deadline;它不得停表、伪造连接指标或立即判定 edge 失效,只有 route 停用或 teardown 才停止该 cadence。该 operation 把失败的 exact direct parent 在当前 fact version 视为已尝试;foreground direct window结束后用SFU取得可用画面,剩余direct candidate只在画面后方收敛,新的外部事实才允许本轮失败tuple重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时,它作为child换父并保留subtree;disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。
|
||||
- 断线恢复:临时网络变化先交给 WebRTC/LiveKit 的 ICE、consent、重连和流状态。Direct edge 可在 exact current parent/child sessions 内采用 framework rebuild 的新 connection ID而不改变topology。应用只在上述exact edge失效后运行一次有界reconcile;任一步成功即停止。候选耗尽时hard-invalid、confirmed-departed或overflow edge被退休并进入明确waiting,healthy bootstrap edge保留。所有异步结果受authenticated session、share generation、base/pending route revision与candidate connection identity约束,未受影响分支保持sticky。权威暂停abort未提交candidate、保留active graph、抑制stall,并在resume后重新reconcile。
|
||||
@@ -161,7 +161,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- Given 同一时刻采集、host outbound 与 viewer inbound 证据,When 进行根因分类,Then 关联派生后的实际协商视频 codec/profile/parameters、适用时的 `scalabilityMode` 与真实解码表现,依次区分 Capture 低、Capture 高但 outbound 因 CPU 低、Capture 高但 outbound 因 bandwidth 低、outbound 正常但 inbound 低、以及 inbound 正常但肉眼模糊五类;不用 UA 或设备型号代替运行时证据,也不上传 raw SDP。
|
||||
- Given Viewer 上报当前路径质量证据,When 报告未鉴权、过频、重复、过期或没有匹配的 sender/viewer 统计,Then 报告被丢弃;有效报告也只能用于诊断,不能改变 Host 表示或路径状态。
|
||||
- Given 受控 RTT/丢包测试条件,When 连续分享高动态游戏画面 30 分钟,Then 延迟、帧率与丢帧达到上述目标,且没有不可恢复的冻结。
|
||||
- Given Windows Chrome/Edge Browser Host 开始或继续分享,When 浏览器返回 display video track,Then 应用设置 `contentHint = "motion"`,并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 新 source 的有界 H.264 预检证明实际 `video/H264` sender 的源帧/编码帧进度达标,Then 后续 direct 或 browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与修复格式;双方不支持 H.264 时同一次协商选择 VP8。Given 预检不支持、证据不足、超时、失败或不达标,Then offer 只列 VP8 与修复格式。缺少 VP8、`setCodecPreferences()` 或应用失败仍在创建 offer 前明确失败。Given 建立 SFU publication,Then 使用 Host source 的同一 H.264/VP8 决定、禁用 backup codec,且 Screener 不设置自定义 simulcast layer list。
|
||||
- Given Windows Chrome/Edge Browser Host 开始或继续分享,When 浏览器返回 display video track,Then 应用设置 `contentHint = "motion"`,并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 当前分享代次的有界 H.264 预检证明实际 `video/H264` sender 的源帧/编码帧进度达标,Then 后续 direct 或 browser-relay offer 按浏览器原生 H.264 capability 顺序列出 H.264,再列 VP8 与修复格式;双方不支持 H.264 时同一次协商选择 VP8。Given 预检不支持、证据不足、超时、失败或不达标,Then offer 只列 VP8 与修复格式。缺少 VP8、`setCodecPreferences()` 或应用失败仍在创建 offer 前明确失败。Given 建立 SFU publication,Then 使用 Host share generation 的同一 H.264/VP8 决定、禁用 backup codec,且 Screener 不设置自定义 simulcast layer list。
|
||||
- Given 分享过程中切换 Wi-Fi/有线网络,When ICE 连接失败,Then 客户端触发 ICE restart,并在成功或超时后给出明确状态。
|
||||
- Given 当前路径仍可播放且控制器准备迁移 logical parent 或替换 transport,When 候选路径拥有模型授权的 overlap 资源,Then 旧路径保持权威,直到 exact candidate child 解码出第一帧新画面后才原子提交;ICE `connected` 单独不能触发切换。
|
||||
- Given 候选没有准入资源,或 provisional 路径失败、超时、代次过期、参与者变化、房间停止、权限撤销或 session 被替换,When 本次机会结束,Then 原路径和未受影响 branches 继续工作,临时资源被销毁,重试遵守模型定义的有界策略。
|
||||
@@ -171,7 +171,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- Given Web Host 或浏览器 relay 建立或替换分享音轨,When 对实际音频 sender 应用发送参数,Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 64000 | 128000 | 192000`、调用 `setParameters()` 并立即读回,档位缺失时按 128000 处理;Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer,Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=192000`,并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用;空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续,不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增,其他 fmtp/FEC/session/media 字段逐项保留。
|
||||
- Given 通过 SFU 发布分享音轨,When 构建 LiveKit 2.22.0 publish options,Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、192 kbps 使用同等结构的显式 sender ceiling,三档都设置 `forceStereo = true`、`dtx = false` 与 `red = false`;验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率,不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。
|
||||
- Given 分享正在直播或暂停且 Host 选择新的 64/128/192 kbps 音频档位,When 当前 profile mutation 串行执行,Then 服务端以 last-wins 保存并广播最新 desired profile,每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback,过期 completion 不得覆盖新值;以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试,不得谎称全房已收敛。当前 applied readback 保持 endpoint-local;没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation,也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge;其他浏览器只有实际 sender readback 成功时才算支持。
|
||||
- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质,When 查看 UI 或校验消息,Then 不存在 codec selector 或 codec quality-state/wire 字段。source-scoped H.264 预检只产生当前页面内存中的内部发送偏好;结果不持久化、不重新协商健康 edge,实际协商 codec 与 encoder 只作为只读诊断。
|
||||
- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质,When 查看 UI 或校验消息,Then 不存在 codec selector 或 codec quality-state/wire 字段。share-generation-scoped H.264 预检只产生当前页面内存中的内部发送偏好;结果不持久化、不重新协商健康 edge,实际协商 codec 与 encoder 只作为只读诊断。
|
||||
- Given 浏览器返回屏幕音轨,When 产品展示音频状态或未来连接详情,Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实;字段缺失必须显示为未知,且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。
|
||||
- Given Web 或后续 native sender 分享电影/视频音频,When 进行产品验收,Then 必须用独立 A/B/C 证据关联 capture audio track/settings、实际 negotiated codec/fmtp、outbound/inbound 实际码率、loss、jitter、concealment、jitter-buffer 与 A/V sync;P2P、浏览器 relay 与 SFU route switch 保持同一持久 Viewer `MediaStream`/媒体元素和 stereo 协商,不新增独立 audio element 或 Web Audio 时钟边界。native sender 必须通过该音质和同步矩阵,且不得把 audio `maxBitrate` ceiling 当作提质证明。
|
||||
- Given 分享者同时运行语音软件,When 选择仅分享游戏声音,Then supported Windows 11 native sender 只保留目标游戏及其子进程的渲染音频,观众、通知和其他应用声音不得进入分享;浏览器 system audio 无法满足时必须允许 video-only 或明确告知其可能包含语音/通知,不能用 AEC、降噪或来源标签伪称已隔离。
|
||||
|
||||
@@ -48,8 +48,13 @@ function codecName(codec: string): string {
|
||||
|
||||
export function codecContractWarnings(metrics: ConnectionMetrics): string[] {
|
||||
const warnings: string[] = [];
|
||||
if (metrics.codec && metrics.codec.toLowerCase() !== "video/vp8") {
|
||||
warnings.push(`视频编码为 ${codecName(metrics.codec)},预期 VP8`);
|
||||
if (
|
||||
metrics.codec &&
|
||||
!["video/h264", "video/vp8"].includes(metrics.codec.toLowerCase())
|
||||
) {
|
||||
warnings.push(
|
||||
`视频编码为 ${codecName(metrics.codec)},预期 H264 或 VP8`,
|
||||
);
|
||||
}
|
||||
if (metrics.audioCodec && metrics.audioCodec.toLowerCase() !== "audio/opus") {
|
||||
warnings.push(`音频编码为 ${codecName(metrics.audioCodec)},预期 Opus`);
|
||||
|
||||
@@ -6,6 +6,7 @@ import type {
|
||||
} from "../../shared/protocol";
|
||||
import { countEndpointMediaCopies } from "../../shared/media-copy-accounting";
|
||||
import { HostPeer } from "../webrtc/host-peer";
|
||||
import type { BrowserVideoCodec } from "../webrtc/video-codec";
|
||||
import type { PeerSnapshot } from "../types";
|
||||
import type { QualityProfile } from "./quality";
|
||||
|
||||
@@ -21,6 +22,7 @@ interface HostProvisionalPrepareInput extends HostProvisionalInput {
|
||||
iceConfig: IceConfig;
|
||||
stream: MediaStream;
|
||||
profile: QualityProfile;
|
||||
videoCodec: BrowserVideoCodec;
|
||||
}
|
||||
|
||||
interface HostProvisionalChildEvents {
|
||||
@@ -86,7 +88,10 @@ export class HostProvisionalChild {
|
||||
private startPrepared(
|
||||
revision: number,
|
||||
candidate: PreparedRouteCandidate,
|
||||
input: Pick<HostProvisionalPrepareInput, "iceConfig" | "stream" | "profile">,
|
||||
input: Pick<
|
||||
HostProvisionalPrepareInput,
|
||||
"iceConfig" | "stream" | "profile" | "videoCodec"
|
||||
>,
|
||||
): void {
|
||||
let peer: HostPeer;
|
||||
peer = new HostPeer(
|
||||
@@ -113,6 +118,7 @@ export class HostProvisionalChild {
|
||||
}
|
||||
},
|
||||
},
|
||||
input.videoCodec,
|
||||
candidate.connectionId,
|
||||
);
|
||||
this.signalingPeer = peer;
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
type SfuPublisherFailureStage,
|
||||
} from "../sfu/publisher";
|
||||
import type { ConnectionMetrics } from "../types";
|
||||
import type { BrowserVideoCodec } from "../webrtc/video-codec";
|
||||
import type {
|
||||
AudioSenderParameterReadback,
|
||||
QualityProfile,
|
||||
@@ -25,7 +26,11 @@ import {
|
||||
|
||||
interface HostPublisherTransport {
|
||||
connect(config: SfuConnectionConfig): Promise<boolean>;
|
||||
activate(stream: MediaStream, profile: QualityProfile): Promise<boolean>;
|
||||
activate(
|
||||
stream: MediaStream,
|
||||
profile: QualityProfile,
|
||||
videoCodec?: BrowserVideoCodec,
|
||||
): Promise<boolean>;
|
||||
deactivate(): Promise<boolean>;
|
||||
replaceStream(stream: MediaStream): Promise<boolean>;
|
||||
updateProfile(profile: QualityProfile): Promise<boolean>;
|
||||
@@ -54,6 +59,7 @@ export interface HostSfuPublisherSnapshot {
|
||||
interface HostSfuRouteEvents {
|
||||
getStream: () => MediaStream | null;
|
||||
getProfile: () => QualityProfile;
|
||||
getVideoCodec: () => BrowserVideoCodec;
|
||||
reconcileChildren: (childPeerIds: string[]) => void;
|
||||
send: (message: ClientMessage) => boolean;
|
||||
onPublisherUpdate?: (snapshot: HostSfuPublisherSnapshot | null) => void;
|
||||
@@ -481,12 +487,13 @@ export class HostSfuRoute {
|
||||
private async preparePublisher(slot: HostPublisherSlot): Promise<boolean> {
|
||||
const stream = this.events.getStream();
|
||||
const profile = this.events.getProfile();
|
||||
const videoCodec = this.events.getVideoCodec();
|
||||
if (!stream) {
|
||||
this.handleFailure(slot);
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
if (!(await slot.publisher.activate(stream, profile))) {
|
||||
if (!(await slot.publisher.activate(stream, profile, videoCodec))) {
|
||||
this.handleFailure(slot);
|
||||
await disconnectPublisher(slot.publisher);
|
||||
return false;
|
||||
|
||||
@@ -120,6 +120,8 @@ import {
|
||||
MAX_ENDPOINT_MEDIA_CHILDREN,
|
||||
reconcileBoundedMediaChildren,
|
||||
} from "../webrtc/media-assignment";
|
||||
import type { BrowserVideoCodec } from "../webrtc/video-codec";
|
||||
import { preferredVideoCodecForTrack } from "../webrtc/video-codec-preflight";
|
||||
import {
|
||||
hostActionErrorNotice,
|
||||
hostServerErrorNotice,
|
||||
@@ -321,6 +323,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
const pendingQualityChangeRef = useRef<QualitySettings | null>(null);
|
||||
const qualitySettingsRef = useRef<QualitySettings>(DEFAULT_QUALITY_SETTINGS);
|
||||
const advancedQualityRef = useRef<QualitySettings>(advancedQuality);
|
||||
const videoCodecRef = useRef<BrowserVideoCodec>("vp8");
|
||||
const codecProbeAbortRef = useRef<AbortController | null>(null);
|
||||
const sharingPausedRef = useRef(false);
|
||||
const retiringStreamRef = useRef<MediaStream | null>(null);
|
||||
const hostSfuRouteRef = useRef<HostSfuRoute | null>(null);
|
||||
@@ -398,6 +402,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
sourceSwitchRef.current = null;
|
||||
qualityChangeRef.current = null;
|
||||
pendingQualityChangeRef.current = null;
|
||||
codecProbeAbortRef.current?.abort();
|
||||
codecProbeAbortRef.current = null;
|
||||
stopPreferredRoomRenewal();
|
||||
signalRef.current?.stop();
|
||||
peersRef.current.forEach((peer) => peer.dispose());
|
||||
@@ -432,6 +438,28 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
);
|
||||
}
|
||||
|
||||
async function probeStreamVideoCodec(
|
||||
stream: MediaStream,
|
||||
): Promise<BrowserVideoCodec> {
|
||||
codecProbeAbortRef.current?.abort();
|
||||
const controller = new AbortController();
|
||||
codecProbeAbortRef.current = controller;
|
||||
const track = stream.getVideoTracks()[0];
|
||||
try {
|
||||
return track
|
||||
? await preferredVideoCodecForTrack(
|
||||
track,
|
||||
qualitySettingsRef.current,
|
||||
controller.signal,
|
||||
)
|
||||
: "vp8";
|
||||
} finally {
|
||||
if (codecProbeAbortRef.current === controller) {
|
||||
codecProbeAbortRef.current = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function stopPreferredRoomRenewal(): void {
|
||||
if (preferredRoomRenewalTimerRef.current !== null) {
|
||||
window.clearInterval(preferredRoomRenewalTimerRef.current);
|
||||
@@ -469,6 +497,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
route = new HostSfuRoute({
|
||||
getStream: () => streamRef.current,
|
||||
getProfile: () => qualitySettingsRef.current,
|
||||
getVideoCodec: () => videoCodecRef.current,
|
||||
reconcileChildren: (childPeerIds) => {
|
||||
if (
|
||||
isCurrentGeneration(generation) &&
|
||||
@@ -522,6 +551,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
sourceSwitchRef.current = null;
|
||||
qualityChangeRef.current = null;
|
||||
pendingQualityChangeRef.current = null;
|
||||
codecProbeAbortRef.current?.abort();
|
||||
codecProbeAbortRef.current = null;
|
||||
videoCodecRef.current = "vp8";
|
||||
const signal = signalRef.current;
|
||||
if (signal) {
|
||||
if (notifyServer) {
|
||||
@@ -1002,6 +1034,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
iceConfig,
|
||||
stream,
|
||||
profile: qualitySettingsRef.current,
|
||||
videoCodec: videoCodecRef.current,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1074,6 +1107,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
}
|
||||
},
|
||||
},
|
||||
videoCodecRef.current,
|
||||
);
|
||||
peersRef.current.set(peerId, peer);
|
||||
let started: boolean;
|
||||
@@ -1448,6 +1482,12 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
setDetails(captureDetails(captured));
|
||||
watchCaptureEnd(captured, generation);
|
||||
|
||||
videoCodecRef.current = await probeStreamVideoCodec(captured);
|
||||
if (!isCurrentGeneration(generation)) {
|
||||
captured.getTracks().forEach((track) => track.stop());
|
||||
return;
|
||||
}
|
||||
|
||||
let createdRoom: HostRoomState | null = null;
|
||||
let claimedRoom = false;
|
||||
try {
|
||||
|
||||
@@ -23,6 +23,7 @@ import {
|
||||
type VideoSenderParameterReadback,
|
||||
} from "../media/quality";
|
||||
import type { ConnectionMetrics } from "../types";
|
||||
import type { BrowserVideoCodec } from "../webrtc/video-codec";
|
||||
import {
|
||||
captureMetrics,
|
||||
collectConnectionMetricsFromReport,
|
||||
@@ -169,7 +170,11 @@ export class SfuPublisher {
|
||||
}
|
||||
}
|
||||
|
||||
activate(stream: MediaStream, profile: QualityProfile): Promise<boolean> {
|
||||
activate(
|
||||
stream: MediaStream,
|
||||
profile: QualityProfile,
|
||||
videoCodec: BrowserVideoCodec = "vp8",
|
||||
): Promise<boolean> {
|
||||
this.desiredProfile = profile;
|
||||
this.startupVideoProfilePending = needsStartupVideoProfile(profile);
|
||||
const effectiveVideoProfile = startupVideoProfile(profile);
|
||||
@@ -195,7 +200,7 @@ export class SfuPublisher {
|
||||
room,
|
||||
videoTrack,
|
||||
sdk.Track.Source.ScreenShare,
|
||||
videoPublishOptions(effectiveVideoProfile),
|
||||
videoPublishOptions(effectiveVideoProfile, videoCodec),
|
||||
);
|
||||
if (!this.owns(room, generation)) {
|
||||
return false;
|
||||
@@ -1165,7 +1170,7 @@ async function configurePublishedVideo(
|
||||
const retainedPublishOptions = {
|
||||
...published.publication.options,
|
||||
...videoTrack.publishOptions,
|
||||
...videoPublishOptions(profile),
|
||||
...videoPublishOptions(profile, publishedVideoCodec(published)),
|
||||
};
|
||||
published.publication.options = retainedPublishOptions;
|
||||
videoTrack.publishOptions = retainedPublishOptions;
|
||||
@@ -1248,10 +1253,19 @@ function mergeQualityWarnings(...warnings: Array<string | null>): string | null
|
||||
return present.length > 0 ? present.join(";") : null;
|
||||
}
|
||||
|
||||
function videoPublishOptions(profile: QualityProfile): TrackPublishOptions {
|
||||
function publishedVideoCodec(published: PublishedTrack): BrowserVideoCodec {
|
||||
return published.publication.options?.videoCodec === "h264"
|
||||
? "h264"
|
||||
: "vp8";
|
||||
}
|
||||
|
||||
function videoPublishOptions(
|
||||
profile: QualityProfile,
|
||||
videoCodec: BrowserVideoCodec,
|
||||
): TrackPublishOptions {
|
||||
return {
|
||||
backupCodec: false,
|
||||
videoCodec: "vp8",
|
||||
videoCodec,
|
||||
screenShareEncoding: {
|
||||
maxBitrate: profile.maxBitrate,
|
||||
maxFramerate: profile.maxFramerate,
|
||||
|
||||
@@ -24,7 +24,10 @@ import {
|
||||
collectConnectionMetrics,
|
||||
createStatsAccumulator,
|
||||
} from "./stats";
|
||||
import { applyVp8Codec } from "./vp8-codec";
|
||||
import {
|
||||
applyVideoCodecPreference,
|
||||
type BrowserVideoCodec,
|
||||
} from "./video-codec";
|
||||
|
||||
const MAX_PENDING_CANDIDATES = 64;
|
||||
type PeerIceConfig = Pick<RTCConfiguration, "iceServers">;
|
||||
@@ -73,6 +76,7 @@ export class HostPeer {
|
||||
private stream: MediaStream,
|
||||
private desiredProfile: QualityProfile,
|
||||
private readonly events: HostPeerEvents,
|
||||
private readonly videoCodec: BrowserVideoCodec = "vp8",
|
||||
connectionId = createOpaqueId(),
|
||||
) {
|
||||
this.connectionId = connectionId;
|
||||
@@ -105,8 +109,8 @@ export class HostPeer {
|
||||
direction: "sendonly",
|
||||
streams: [this.stream],
|
||||
});
|
||||
if (!applyVp8Codec(videoTransceiver)) {
|
||||
this.setError(null, "当前浏览器无法使用 VP8 视频编码");
|
||||
if (!applyVideoCodecPreference(videoTransceiver, this.videoCodec)) {
|
||||
this.setError(null, "当前浏览器无法使用支持的视频编码");
|
||||
return false;
|
||||
}
|
||||
this.videoSender = videoTransceiver.sender;
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
import {
|
||||
configureVideoSender,
|
||||
startupVideoProfile,
|
||||
type QualityProfile,
|
||||
} from "../media/quality";
|
||||
import {
|
||||
applyH264ProbeCodec,
|
||||
type BrowserVideoCodec,
|
||||
} from "./video-codec";
|
||||
|
||||
const PREFLIGHT_DEADLINE_MS = 4_000;
|
||||
const PREFLIGHT_POLL_MS = 100;
|
||||
const PREFLIGHT_MIN_SOURCE_FRAMES = 15;
|
||||
const PREFLIGHT_MAX_FRAME_DEFICIT = 2;
|
||||
const PREFLIGHT_FPS_WINDOW_MS = 1_000;
|
||||
|
||||
export interface H264ProbeSample {
|
||||
outboundId: string;
|
||||
sourceId: string;
|
||||
timestamp: number;
|
||||
codec: string;
|
||||
framesEncoded: number | null;
|
||||
sourceFrames: number | null;
|
||||
encodedFramesPerSecond: number | null;
|
||||
sourceFramesPerSecond: number | null;
|
||||
qualityLimitationReason: string | null;
|
||||
}
|
||||
|
||||
type StatsRecord = RTCStats & Record<string, unknown>;
|
||||
|
||||
function finiteNumber(value: unknown): number | null {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : null;
|
||||
}
|
||||
|
||||
function stringValue(value: unknown): string | null {
|
||||
return typeof value === "string" ? value : null;
|
||||
}
|
||||
|
||||
function recordById(
|
||||
report: RTCStatsReport,
|
||||
id: unknown,
|
||||
): StatsRecord | null {
|
||||
if (typeof id !== "string") {
|
||||
return null;
|
||||
}
|
||||
return (report.get(id) as StatsRecord | undefined) ?? null;
|
||||
}
|
||||
|
||||
function readH264ProbeSample(report: RTCStatsReport): H264ProbeSample | null {
|
||||
const candidates: StatsRecord[] = [];
|
||||
report.forEach((raw) => {
|
||||
const record = raw as StatsRecord;
|
||||
if (
|
||||
record.type === "outbound-rtp" &&
|
||||
record.kind === "video" &&
|
||||
record.isRemote !== true &&
|
||||
finiteNumber(record.framesEncoded) !== null
|
||||
) {
|
||||
candidates.push(record);
|
||||
}
|
||||
});
|
||||
if (candidates.length !== 1) {
|
||||
return null;
|
||||
}
|
||||
const outbound = candidates[0]!;
|
||||
const codec = recordById(report, outbound.codecId);
|
||||
const source = recordById(report, outbound.mediaSourceId);
|
||||
const mimeType = stringValue(codec?.mimeType);
|
||||
if (
|
||||
codec?.type !== "codec" ||
|
||||
source?.type !== "media-source" ||
|
||||
mimeType?.toLowerCase() !== "video/h264"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
outboundId: outbound.id,
|
||||
sourceId: source.id,
|
||||
timestamp: finiteNumber(outbound.timestamp) ?? 0,
|
||||
codec: mimeType,
|
||||
framesEncoded: finiteNumber(outbound.framesEncoded),
|
||||
sourceFrames: finiteNumber(source.frames),
|
||||
encodedFramesPerSecond: finiteNumber(outbound.framesPerSecond),
|
||||
sourceFramesPerSecond: finiteNumber(source.framesPerSecond),
|
||||
qualityLimitationReason: stringValue(outbound.qualityLimitationReason),
|
||||
};
|
||||
}
|
||||
|
||||
export function h264ProbeSustainsSource(
|
||||
baseline: H264ProbeSample,
|
||||
current: H264ProbeSample,
|
||||
): boolean | null {
|
||||
if (
|
||||
baseline.outboundId !== current.outboundId ||
|
||||
baseline.sourceId !== current.sourceId ||
|
||||
current.timestamp <= baseline.timestamp ||
|
||||
current.codec.toLowerCase() !== "video/h264" ||
|
||||
current.qualityLimitationReason === "cpu"
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
baseline.framesEncoded !== null &&
|
||||
current.framesEncoded !== null &&
|
||||
baseline.sourceFrames !== null &&
|
||||
current.sourceFrames !== null &&
|
||||
current.framesEncoded >= baseline.framesEncoded &&
|
||||
current.sourceFrames >= baseline.sourceFrames
|
||||
) {
|
||||
const encoded = current.framesEncoded - baseline.framesEncoded;
|
||||
const source = current.sourceFrames - baseline.sourceFrames;
|
||||
if (source < PREFLIGHT_MIN_SOURCE_FRAMES) {
|
||||
return null;
|
||||
}
|
||||
return encoded + PREFLIGHT_MAX_FRAME_DEFICIT >= source;
|
||||
}
|
||||
if (
|
||||
current.timestamp - baseline.timestamp < PREFLIGHT_FPS_WINDOW_MS ||
|
||||
current.encodedFramesPerSecond === null ||
|
||||
current.sourceFramesPerSecond === null ||
|
||||
current.sourceFramesPerSecond <= 0
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return (
|
||||
current.encodedFramesPerSecond + PREFLIGHT_MAX_FRAME_DEFICIT >=
|
||||
current.sourceFramesPerSecond
|
||||
);
|
||||
}
|
||||
|
||||
function hasProbeWindow(
|
||||
baseline: H264ProbeSample,
|
||||
current: H264ProbeSample,
|
||||
): boolean {
|
||||
if (
|
||||
baseline.outboundId !== current.outboundId ||
|
||||
baseline.sourceId !== current.sourceId ||
|
||||
current.timestamp <= baseline.timestamp
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
baseline.sourceFrames !== null &&
|
||||
current.sourceFrames !== null &&
|
||||
current.sourceFrames >= baseline.sourceFrames
|
||||
) {
|
||||
return (
|
||||
current.sourceFrames - baseline.sourceFrames >=
|
||||
PREFLIGHT_MIN_SOURCE_FRAMES
|
||||
);
|
||||
}
|
||||
return current.timestamp - baseline.timestamp >= PREFLIGHT_FPS_WINDOW_MS;
|
||||
}
|
||||
|
||||
function delay(milliseconds: number, signal?: AbortSignal): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (signal?.aborted) {
|
||||
reject(signal.reason);
|
||||
return;
|
||||
}
|
||||
const onAbort = () => {
|
||||
window.clearTimeout(timer);
|
||||
reject(signal?.reason);
|
||||
};
|
||||
const timer = window.setTimeout(() => {
|
||||
signal?.removeEventListener("abort", onAbort);
|
||||
resolve();
|
||||
}, milliseconds);
|
||||
signal?.addEventListener("abort", onAbort, { once: true });
|
||||
});
|
||||
}
|
||||
|
||||
async function waitForIceGathering(
|
||||
connection: RTCPeerConnection,
|
||||
deadline: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
while (connection.iceGatheringState !== "complete") {
|
||||
if (Date.now() >= deadline) {
|
||||
throw new Error("H.264 preflight timed out");
|
||||
}
|
||||
await delay(PREFLIGHT_POLL_MS, signal);
|
||||
}
|
||||
}
|
||||
|
||||
async function runH264Probe(
|
||||
track: MediaStreamTrack,
|
||||
profile: QualityProfile,
|
||||
signal?: AbortSignal,
|
||||
): Promise<boolean> {
|
||||
const senderConnection = new RTCPeerConnection({ iceServers: [] });
|
||||
const receiverConnection = new RTCPeerConnection({ iceServers: [] });
|
||||
const abort = () => {
|
||||
senderConnection.close();
|
||||
receiverConnection.close();
|
||||
};
|
||||
signal?.addEventListener("abort", abort, { once: true });
|
||||
try {
|
||||
const transceiver = senderConnection.addTransceiver(track, {
|
||||
direction: "sendonly",
|
||||
});
|
||||
if (!applyH264ProbeCodec(transceiver)) {
|
||||
return false;
|
||||
}
|
||||
await configureVideoSender(
|
||||
transceiver.sender,
|
||||
startupVideoProfile(profile),
|
||||
);
|
||||
const deadline = Date.now() + PREFLIGHT_DEADLINE_MS;
|
||||
await senderConnection.setLocalDescription(
|
||||
await senderConnection.createOffer(),
|
||||
);
|
||||
await waitForIceGathering(senderConnection, deadline, signal);
|
||||
await receiverConnection.setRemoteDescription(
|
||||
senderConnection.localDescription!,
|
||||
);
|
||||
await receiverConnection.setLocalDescription(
|
||||
await receiverConnection.createAnswer(),
|
||||
);
|
||||
await waitForIceGathering(receiverConnection, deadline, signal);
|
||||
await senderConnection.setRemoteDescription(
|
||||
receiverConnection.localDescription!,
|
||||
);
|
||||
|
||||
let warmupBaseline: H264ProbeSample | null = null;
|
||||
let measurementBaseline: H264ProbeSample | null = null;
|
||||
while (Date.now() < deadline) {
|
||||
if (signal?.aborted || track.readyState === "ended") {
|
||||
return false;
|
||||
}
|
||||
const sample = readH264ProbeSample(
|
||||
await transceiver.sender.getStats(),
|
||||
);
|
||||
if (sample) {
|
||||
warmupBaseline ??= sample;
|
||||
if (
|
||||
!measurementBaseline &&
|
||||
sample !== warmupBaseline &&
|
||||
hasProbeWindow(warmupBaseline, sample)
|
||||
) {
|
||||
measurementBaseline = sample;
|
||||
} else if (measurementBaseline && sample !== measurementBaseline) {
|
||||
const result = h264ProbeSustainsSource(measurementBaseline, sample);
|
||||
if (result !== null) {
|
||||
return result;
|
||||
}
|
||||
}
|
||||
}
|
||||
await delay(PREFLIGHT_POLL_MS, signal);
|
||||
}
|
||||
return false;
|
||||
} finally {
|
||||
signal?.removeEventListener("abort", abort);
|
||||
senderConnection.close();
|
||||
receiverConnection.close();
|
||||
}
|
||||
}
|
||||
|
||||
export async function preferredVideoCodecForTrack(
|
||||
track: MediaStreamTrack,
|
||||
profile: QualityProfile,
|
||||
signal?: AbortSignal,
|
||||
): Promise<BrowserVideoCodec> {
|
||||
let codec: BrowserVideoCodec = "vp8";
|
||||
if (track.kind !== "video" || track.readyState === "ended") {
|
||||
console.debug("[Screener] video codec preflight", codec);
|
||||
return codec;
|
||||
}
|
||||
try {
|
||||
codec = (await runH264Probe(track, profile, signal)) ? "h264" : "vp8";
|
||||
} catch {}
|
||||
if (!signal?.aborted) {
|
||||
console.debug("[Screener] video codec preflight", codec);
|
||||
}
|
||||
return codec;
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
export type BrowserVideoCodec = "h264" | "vp8";
|
||||
|
||||
const REPAIR_CODEC_MIME_TYPES = new Set([
|
||||
"video/rtx",
|
||||
"video/red",
|
||||
"video/ulpfec",
|
||||
"video/flexfec-03",
|
||||
]);
|
||||
|
||||
function videoCapabilities(): RTCRtpCodec[] {
|
||||
return RTCRtpSender.getCapabilities?.("video")?.codecs ?? [];
|
||||
}
|
||||
|
||||
function vp8Codecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] {
|
||||
return codecs.filter(
|
||||
({ mimeType }) => mimeType.toLowerCase() === "video/vp8",
|
||||
);
|
||||
}
|
||||
|
||||
function h264ModeOneCodecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] {
|
||||
return codecs.filter(({ mimeType, sdpFmtpLine }) => {
|
||||
if (mimeType.toLowerCase() !== "video/h264") {
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
!sdpFmtpLine ||
|
||||
/(?:^|;)\s*packetization-mode=1(?:;|$)/i.test(sdpFmtpLine)
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function repairCodecs(codecs: readonly RTCRtpCodec[]): RTCRtpCodec[] {
|
||||
return codecs.filter(({ mimeType }) =>
|
||||
REPAIR_CODEC_MIME_TYPES.has(mimeType.toLowerCase()),
|
||||
);
|
||||
}
|
||||
|
||||
function applyCodecs(
|
||||
transceiver: RTCRtpTransceiver,
|
||||
codecs: RTCRtpCodec[],
|
||||
): boolean {
|
||||
if (typeof transceiver.setCodecPreferences !== "function") {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
transceiver.setCodecPreferences(codecs);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function applyVideoCodecPreference(
|
||||
transceiver: RTCRtpTransceiver,
|
||||
preference: BrowserVideoCodec,
|
||||
): boolean {
|
||||
if (typeof RTCRtpSender === "undefined") {
|
||||
return false;
|
||||
}
|
||||
const codecs = videoCapabilities();
|
||||
const vp8 = vp8Codecs(codecs);
|
||||
if (vp8.length === 0) {
|
||||
return false;
|
||||
}
|
||||
const h264 = preference === "h264" ? h264ModeOneCodecs(codecs) : [];
|
||||
return applyCodecs(transceiver, [
|
||||
...h264,
|
||||
...vp8,
|
||||
...repairCodecs(codecs),
|
||||
]);
|
||||
}
|
||||
|
||||
export function applyH264ProbeCodec(
|
||||
transceiver: RTCRtpTransceiver,
|
||||
): boolean {
|
||||
if (typeof RTCRtpSender === "undefined") {
|
||||
return false;
|
||||
}
|
||||
const codecs = videoCapabilities();
|
||||
const h264 = h264ModeOneCodecs(codecs);
|
||||
return h264.length > 0 &&
|
||||
applyCodecs(transceiver, [...h264, ...repairCodecs(codecs)]);
|
||||
}
|
||||
@@ -7,6 +7,8 @@ import type { QualityProfile } from "../media/quality";
|
||||
import type { PeerSnapshot } from "../types";
|
||||
import { HostPeer } from "./host-peer";
|
||||
import { MAX_ENDPOINT_MEDIA_CHILDREN } from "./media-assignment";
|
||||
import type { BrowserVideoCodec } from "./video-codec";
|
||||
import { preferredVideoCodecForTrack } from "./video-codec-preflight";
|
||||
|
||||
interface ViewerRelayEvents {
|
||||
sendSignal: (peerId: string, payload: SignalPayload) => boolean;
|
||||
@@ -31,6 +33,11 @@ export class ViewerRelay {
|
||||
private preparedCandidate: PreparedRouteCandidate | null = null;
|
||||
private plannedChildPeerIds: string[] = [];
|
||||
private syncQueue = Promise.resolve();
|
||||
private videoCodec: BrowserVideoCodec = "vp8";
|
||||
private codecProbeTrack: MediaStreamTrack | null = null;
|
||||
private codecProbeAbort: AbortController | null = null;
|
||||
private codecProbePromise: Promise<void> | null = null;
|
||||
private codecProbeSettled = false;
|
||||
private disposed = false;
|
||||
|
||||
constructor(
|
||||
@@ -84,7 +91,14 @@ export class ViewerRelay {
|
||||
this.preparedCandidate = { ...candidate };
|
||||
this.plannedChildPeerIds = planned;
|
||||
if (candidate.transport === "direct") {
|
||||
this.startPreparedChild(revision, candidate, stream);
|
||||
const pendingProbe = this.codecProbePromise;
|
||||
if (pendingProbe) {
|
||||
void pendingProbe.then(() => {
|
||||
this.startPreparedChildIfCurrent(revision, candidate, stream);
|
||||
});
|
||||
} else {
|
||||
this.startPreparedChildIfCurrent(revision, candidate, stream);
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
@@ -126,6 +140,7 @@ export class ViewerRelay {
|
||||
this.preparedCandidate = null;
|
||||
this.plannedChildPeerIds = [];
|
||||
this.discardPreparedPeer();
|
||||
this.startCodecProbe();
|
||||
}
|
||||
|
||||
private discardPreparedPeer(): void {
|
||||
@@ -155,6 +170,9 @@ export class ViewerRelay {
|
||||
}
|
||||
}
|
||||
this.childPeerIds = nextChildPeerIds;
|
||||
if (this.childPeerIds.length > 0) {
|
||||
this.cancelCodecProbe();
|
||||
}
|
||||
for (const childPeerId of this.childPeerIds) {
|
||||
const peer = this.peers.get(childPeerId);
|
||||
if (
|
||||
@@ -166,17 +184,27 @@ export class ViewerRelay {
|
||||
this.startPeer(childPeerId, this.stream);
|
||||
}
|
||||
}
|
||||
this.startCodecProbe();
|
||||
}
|
||||
|
||||
setStream(stream: MediaStream): void {
|
||||
if (this.disposed) {
|
||||
return;
|
||||
}
|
||||
const nextVideoTrack = stream.getVideoTracks()[0] ?? null;
|
||||
if (nextVideoTrack !== this.codecProbeTrack) {
|
||||
this.cancelCodecProbe();
|
||||
this.codecProbeTrack = nextVideoTrack;
|
||||
if (!this.codecProbeSettled) {
|
||||
this.videoCodec = "vp8";
|
||||
}
|
||||
}
|
||||
const preparedPeer = this.preparedChild?.peer ?? null;
|
||||
this.stream = stream;
|
||||
this.syncQueue = this.syncQueue
|
||||
.then(() => this.syncStream(stream, preparedPeer))
|
||||
.catch(() => undefined);
|
||||
this.startCodecProbe();
|
||||
}
|
||||
|
||||
updateProfile(profile: QualityProfile): Promise<boolean> {
|
||||
@@ -271,6 +299,10 @@ export class ViewerRelay {
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
this.cancelCodecProbe();
|
||||
this.codecProbeTrack = null;
|
||||
this.codecProbeSettled = false;
|
||||
this.videoCodec = "vp8";
|
||||
this.stream = null;
|
||||
this.discardPreparedChild();
|
||||
this.disposePeers();
|
||||
@@ -281,6 +313,8 @@ export class ViewerRelay {
|
||||
return;
|
||||
}
|
||||
this.disposed = true;
|
||||
this.cancelCodecProbe();
|
||||
this.codecProbeTrack = null;
|
||||
this.stream = null;
|
||||
this.discardPreparedChild();
|
||||
this.childPeerIds = [];
|
||||
@@ -316,6 +350,23 @@ export class ViewerRelay {
|
||||
});
|
||||
}
|
||||
|
||||
private startPreparedChildIfCurrent(
|
||||
revision: number,
|
||||
candidate: PreparedRouteCandidate,
|
||||
stream: MediaStream,
|
||||
): void {
|
||||
if (
|
||||
this.disposed ||
|
||||
this.stream !== stream ||
|
||||
this.preparedChild ||
|
||||
this.preparedRevision !== revision ||
|
||||
!sameCandidate(this.preparedCandidate, candidate)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
this.startPreparedChild(revision, candidate, stream);
|
||||
}
|
||||
|
||||
private async syncStream(
|
||||
stream: MediaStream,
|
||||
preparedPeer: HostPeer | null,
|
||||
@@ -475,11 +526,61 @@ export class ViewerRelay {
|
||||
}
|
||||
},
|
||||
},
|
||||
this.videoCodec,
|
||||
connectionId,
|
||||
);
|
||||
return peer;
|
||||
}
|
||||
|
||||
private startCodecProbe(): void {
|
||||
const track = this.codecProbeTrack;
|
||||
if (
|
||||
this.disposed ||
|
||||
this.codecProbeSettled ||
|
||||
this.codecProbeAbort ||
|
||||
this.codecProbePromise ||
|
||||
!track ||
|
||||
track.readyState === "ended" ||
|
||||
this.childPeerIds.length > 0 ||
|
||||
this.preparedChild
|
||||
) {
|
||||
return;
|
||||
}
|
||||
const controller = new AbortController();
|
||||
this.codecProbeAbort = controller;
|
||||
const promise = preferredVideoCodecForTrack(
|
||||
track,
|
||||
this.desiredProfile,
|
||||
controller.signal,
|
||||
)
|
||||
.then((codec) => {
|
||||
if (
|
||||
this.codecProbeAbort !== controller ||
|
||||
controller.signal.aborted ||
|
||||
this.codecProbeTrack !== track ||
|
||||
this.disposed
|
||||
) {
|
||||
return;
|
||||
}
|
||||
this.videoCodec = codec;
|
||||
this.codecProbeSettled = true;
|
||||
this.codecProbeAbort = null;
|
||||
})
|
||||
.finally(() => {
|
||||
if (this.codecProbePromise === promise) {
|
||||
this.codecProbePromise = null;
|
||||
this.startCodecProbe();
|
||||
}
|
||||
});
|
||||
this.codecProbePromise = promise;
|
||||
}
|
||||
|
||||
private cancelCodecProbe(): void {
|
||||
const controller = this.codecProbeAbort;
|
||||
this.codecProbeAbort = null;
|
||||
controller?.abort();
|
||||
}
|
||||
|
||||
private failPreparedChild(peer: HostPeer): void {
|
||||
const prepared = this.preparedChild;
|
||||
if (prepared?.peer !== peer) {
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
const REPAIR_CODEC_MIME_TYPES = new Set([
|
||||
"video/rtx",
|
||||
"video/red",
|
||||
"video/ulpfec",
|
||||
"video/flexfec-03",
|
||||
]);
|
||||
|
||||
export function applyVp8Codec(transceiver: RTCRtpTransceiver): boolean {
|
||||
if (
|
||||
typeof RTCRtpSender === "undefined" ||
|
||||
typeof transceiver.setCodecPreferences !== "function"
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
const codecs = RTCRtpSender.getCapabilities?.("video")?.codecs ?? [];
|
||||
const vp8Codecs = codecs.filter(
|
||||
({ mimeType }) => mimeType.toLowerCase() === "video/vp8",
|
||||
);
|
||||
if (vp8Codecs.length === 0) {
|
||||
return false;
|
||||
}
|
||||
|
||||
transceiver.setCodecPreferences([
|
||||
...vp8Codecs,
|
||||
...codecs.filter(({ mimeType }) =>
|
||||
REPAIR_CODEC_MIME_TYPES.has(mimeType.toLowerCase()),
|
||||
),
|
||||
]);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -7,12 +7,21 @@ import {
|
||||
import { HostProvisionalChild } from "../src/client/media/host-provisional-child.ts";
|
||||
import type { PeerSnapshot } from "../src/client/types.ts";
|
||||
import { HostPeer } from "../src/client/webrtc/host-peer.ts";
|
||||
import type { BrowserVideoCodec } from "../src/client/webrtc/video-codec.ts";
|
||||
import { ViewerRelay } from "../src/client/webrtc/viewer-relay.ts";
|
||||
import type {
|
||||
IceConfig,
|
||||
ParticipantRouteAssignment,
|
||||
} from "../src/shared/protocol.ts";
|
||||
|
||||
const codecPreflight = vi.hoisted(() => ({
|
||||
probe: vi.fn(async (): Promise<"h264" | "vp8"> => "vp8"),
|
||||
}));
|
||||
|
||||
vi.mock("../src/client/webrtc/video-codec-preflight.ts", () => ({
|
||||
preferredVideoCodecForTrack: codecPreflight.probe,
|
||||
}));
|
||||
|
||||
const statsCallbacks: Array<() => void> = [];
|
||||
|
||||
class FakeSender {
|
||||
@@ -387,6 +396,7 @@ function createPeer(
|
||||
onUpdate: (snapshot: PeerSnapshot) => void = () => undefined,
|
||||
iceConfig: IceConfig = { iceServers: [] },
|
||||
profile: QualityProfile = QUALITY_PROFILES["720p30"],
|
||||
videoCodec: BrowserVideoCodec = "vp8",
|
||||
): HostPeer {
|
||||
return new HostPeer(
|
||||
"viewer-peer",
|
||||
@@ -397,6 +407,7 @@ function createPeer(
|
||||
sendSignal: () => true,
|
||||
onUpdate,
|
||||
},
|
||||
videoCodec,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -434,6 +445,8 @@ async function completeVideoStartup(
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
codecPreflight.probe.mockReset();
|
||||
codecPreflight.probe.mockResolvedValue("vp8");
|
||||
FakePeerConnection.latest = null;
|
||||
FakePeerConnection.instances = [];
|
||||
FakePeerConnection.activeCount = 0;
|
||||
@@ -519,6 +532,50 @@ describe("HostPeer source replacement", () => {
|
||||
expect(connection.createOfferCallCount).toBe(1);
|
||||
});
|
||||
|
||||
it("prefers native H264 mode 1 and retains VP8 fallback", async () => {
|
||||
vi.stubGlobal("RTCRtpSender", {
|
||||
getCapabilities: () => ({
|
||||
codecs: [
|
||||
{
|
||||
mimeType: "video/H264",
|
||||
clockRate: 90_000,
|
||||
sdpFmtpLine: "packetization-mode=0;profile-level-id=42001f",
|
||||
},
|
||||
{
|
||||
mimeType: "video/H264",
|
||||
clockRate: 90_000,
|
||||
sdpFmtpLine: "packetization-mode=1;profile-level-id=42001f",
|
||||
},
|
||||
{ mimeType: "video/VP8", clockRate: 90_000 },
|
||||
{ mimeType: "video/rtx", clockRate: 90_000 },
|
||||
{ mimeType: "video/RED", clockRate: 90_000 },
|
||||
],
|
||||
headerExtensions: [],
|
||||
}),
|
||||
});
|
||||
const peer = createPeer(
|
||||
createStream(createTrack("video", "video"), null),
|
||||
() => undefined,
|
||||
{ iceServers: [] },
|
||||
QUALITY_PROFILES["720p30"],
|
||||
"h264",
|
||||
);
|
||||
|
||||
await expect(peer.start()).resolves.toBe(true);
|
||||
|
||||
expect(
|
||||
FakePeerConnection.latest!.codecPreferenceCalls[0]?.map((codec) => [
|
||||
codec.mimeType.toLowerCase(),
|
||||
codec.sdpFmtpLine ?? null,
|
||||
]),
|
||||
).toEqual([
|
||||
["video/h264", "packetization-mode=1;profile-level-id=42001f"],
|
||||
["video/vp8", null],
|
||||
["video/rtx", null],
|
||||
["video/red", null],
|
||||
]);
|
||||
});
|
||||
|
||||
it("fails before creating an offer when codec preferences are unavailable", async () => {
|
||||
FakePeerConnection.omitCodecPreferenceSetter = true;
|
||||
const updates: PeerSnapshot[] = [];
|
||||
@@ -531,7 +588,7 @@ describe("HostPeer source replacement", () => {
|
||||
|
||||
expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0);
|
||||
expect(FakePeerConnection.latest!.localDescription).toBeNull();
|
||||
expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码");
|
||||
expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码");
|
||||
});
|
||||
|
||||
it("fails before creating an offer when VP8 is unavailable", async () => {
|
||||
@@ -554,7 +611,7 @@ describe("HostPeer source replacement", () => {
|
||||
|
||||
expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0);
|
||||
expect(FakePeerConnection.latest!.localDescription).toBeNull();
|
||||
expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码");
|
||||
expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码");
|
||||
});
|
||||
|
||||
it("fails before creating an offer when codec preference setup fails", async () => {
|
||||
@@ -569,7 +626,7 @@ describe("HostPeer source replacement", () => {
|
||||
|
||||
expect(FakePeerConnection.latest!.createOfferCallCount).toBe(0);
|
||||
expect(FakePeerConnection.latest!.localDescription).toBeNull();
|
||||
expect(updates.at(-1)?.error).toBe("当前浏览器无法使用 VP8 视频编码");
|
||||
expect(updates.at(-1)?.error).toBe("当前浏览器无法使用支持的视频编码");
|
||||
});
|
||||
|
||||
it("applies STUN-only ICE configuration at creation and update", () => {
|
||||
@@ -1384,6 +1441,7 @@ function hostProvisionalInput(
|
||||
iceConfig: { iceServers: [] },
|
||||
stream,
|
||||
profile: QUALITY_PROFILES["720p30"],
|
||||
videoCodec: "vp8" as const,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1516,6 +1574,29 @@ describe("ViewerRelay downstream ownership", () => {
|
||||
transport: "direct" as const,
|
||||
});
|
||||
|
||||
it("uses a completed H264 probe only for a future child", async () => {
|
||||
codecPreflight.probe.mockResolvedValueOnce("h264");
|
||||
const relay = new ViewerRelay(
|
||||
{ iceServers: [] },
|
||||
QUALITY_PROFILES["720p30"],
|
||||
{ sendSignal: () => true },
|
||||
);
|
||||
relay.setStream(createStream(createTrack("video", "h264-source"), null));
|
||||
await vi.waitFor(() => expect(codecPreflight.probe).toHaveBeenCalledOnce());
|
||||
await Promise.resolve();
|
||||
relay.setChildren(["h264-child"]);
|
||||
await vi.waitFor(() =>
|
||||
expect(FakePeerConnection.latest?.codecPreferenceCalls).toHaveLength(1),
|
||||
);
|
||||
|
||||
expect(
|
||||
FakePeerConnection.latest!.codecPreferenceCalls[0]?.map(({ mimeType }) =>
|
||||
mimeType.toLowerCase(),
|
||||
),
|
||||
).toEqual(["video/h264", "video/vp8"]);
|
||||
relay.dispose();
|
||||
});
|
||||
|
||||
it("promotes the exact prepared child connection within the current Viewer cap", async () => {
|
||||
const signals: Array<{ peerId: string; connectionId: string }> = [];
|
||||
const relay = new ViewerRelay(
|
||||
@@ -1593,6 +1674,7 @@ describe("ViewerRelay downstream ownership", () => {
|
||||
);
|
||||
relay.setStream(createStream(createTrack("video", "rollback-video"), null));
|
||||
expect(relay.prepareChild(7, routeCandidate(7, "first-probe"), ["first-probe"])).toBe(true);
|
||||
await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull());
|
||||
const firstProbe = FakePeerConnection.latest!;
|
||||
expect(relay.prepareChild(8, routeCandidate(8, "second-probe"), ["second-probe"])).toBe(true);
|
||||
const secondProbe = FakePeerConnection.latest!;
|
||||
@@ -1675,6 +1757,7 @@ describe("ViewerRelay downstream ownership", () => {
|
||||
["prepared-child"],
|
||||
),
|
||||
).toBe(true);
|
||||
await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull());
|
||||
const preparedConnection = FakePeerConnection.latest!;
|
||||
expect(preparedConnection.senders[1]?.track).toBeNull();
|
||||
|
||||
@@ -1701,6 +1784,7 @@ describe("ViewerRelay downstream ownership", () => {
|
||||
);
|
||||
relay.setStream(createStream(createTrack("video", "cap-video"), null));
|
||||
expect(relay.prepareChild(1, routeCandidate(1, "child-0"), ["child-0"])).toBe(true);
|
||||
await vi.waitFor(() => expect(FakePeerConnection.latest).not.toBeNull());
|
||||
const preparedConnection = FakePeerConnection.latest!;
|
||||
preparedConnection.connectionState = "connected";
|
||||
relay.activateChildren(1, ["child-0"]);
|
||||
|
||||
@@ -259,6 +259,7 @@ describe("minimal route transition contracts", () => {
|
||||
const route = new HostSfuRoute({
|
||||
getStream: () => ({}) as MediaStream,
|
||||
getProfile: () => QUALITY_PROFILES["720p30"],
|
||||
getVideoCodec: () => "h264",
|
||||
reconcileChildren: () => undefined,
|
||||
send: (message) => {
|
||||
messages.push(message);
|
||||
@@ -281,6 +282,11 @@ describe("minimal route transition contracts", () => {
|
||||
});
|
||||
await route.acceptConfig(sfuConfig(1));
|
||||
expect(publishers[0]?.activate).toHaveBeenCalledOnce();
|
||||
expect(publishers[0]?.activate).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
QUALITY_PROFILES["720p30"],
|
||||
"h264",
|
||||
);
|
||||
|
||||
route.accept({
|
||||
revision: 2,
|
||||
@@ -311,6 +317,7 @@ describe("minimal route transition contracts", () => {
|
||||
const route = new HostSfuRoute({
|
||||
getStream: () => ({}) as MediaStream,
|
||||
getProfile: () => QUALITY_PROFILES["720p30"],
|
||||
getVideoCodec: () => "vp8",
|
||||
reconcileChildren: (children) => reconciledChildren.push(children),
|
||||
send: () => true,
|
||||
createPublisher: () => {
|
||||
@@ -365,6 +372,7 @@ describe("minimal route transition contracts", () => {
|
||||
const route = new HostSfuRoute({
|
||||
getStream: () => ({}) as MediaStream,
|
||||
getProfile: () => QUALITY_PROFILES["720p30"],
|
||||
getVideoCodec: () => "vp8",
|
||||
reconcileChildren: () => undefined,
|
||||
send: () => true,
|
||||
createPublisher: () => {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { QUALITY_PROFILES } from "../src/client/media/quality.ts";
|
||||
import { SfuPublisher } from "../src/client/sfu/publisher.ts";
|
||||
import { SfuSubscriber } from "../src/client/sfu/subscriber.ts";
|
||||
import type { ConnectionMetrics } from "../src/client/types.ts";
|
||||
@@ -878,6 +879,27 @@ describe("SfuPublisher", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("publishes one H264 source without a backup codec", async () => {
|
||||
const publisher = new SfuPublisher();
|
||||
const video = track("video", "video-h264");
|
||||
await publisher.connect(connection);
|
||||
|
||||
await expect(
|
||||
publisher.activate(stream(video), qualityProfile, "h264"),
|
||||
).resolves.toBe(true);
|
||||
|
||||
const publication = livekit.state.rooms[0].localParticipant.publications[0];
|
||||
expect(publication.options).toMatchObject({
|
||||
source: Track.Source.ScreenShare,
|
||||
backupCodec: false,
|
||||
videoCodec: "h264",
|
||||
});
|
||||
await expect(
|
||||
publisher.updateProfile(QUALITY_PROFILES["720p30"]),
|
||||
).resolves.toBe(true);
|
||||
expect(publication.options.videoCodec).toBe("h264");
|
||||
});
|
||||
|
||||
it("keeps media and prior audio readback when a live update fails", async () => {
|
||||
const publisher = new SfuPublisher();
|
||||
await publisher.connect(connection);
|
||||
|
||||
@@ -107,14 +107,14 @@ describe("StatsGrid progressive disclosure", () => {
|
||||
createElement(StatsGrid, {
|
||||
metrics: {
|
||||
...metrics,
|
||||
codec: "video/H264",
|
||||
codec: "video/VP9",
|
||||
audioCodec: "audio/PCMU",
|
||||
},
|
||||
direction: "receive",
|
||||
}),
|
||||
);
|
||||
|
||||
expect(html).toContain("视频编码为 H264,预期 VP8");
|
||||
expect(html).toContain("视频编码为 VP9,预期 H264 或 VP8");
|
||||
expect(html).toContain("音频编码为 PCMU,预期 Opus");
|
||||
expect(html).not.toContain("视频 Codec");
|
||||
expect(html).not.toContain("音频 Codec");
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
h264ProbeSustainsSource,
|
||||
type H264ProbeSample,
|
||||
} from "../src/client/webrtc/video-codec-preflight.ts";
|
||||
|
||||
function sample(
|
||||
overrides: Partial<H264ProbeSample> = {},
|
||||
): H264ProbeSample {
|
||||
return {
|
||||
outboundId: "outbound",
|
||||
sourceId: "source",
|
||||
timestamp: 1_000,
|
||||
codec: "video/H264",
|
||||
framesEncoded: 10,
|
||||
sourceFrames: 10,
|
||||
encodedFramesPerSecond: 30,
|
||||
sourceFramesPerSecond: 30,
|
||||
qualityLimitationReason: "none",
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe("H264 sender preflight", () => {
|
||||
it("accepts bounded pipeline lag against the same source", () => {
|
||||
const baseline = sample();
|
||||
const current = sample({
|
||||
timestamp: 2_000,
|
||||
framesEncoded: 39,
|
||||
sourceFrames: 40,
|
||||
});
|
||||
|
||||
expect(h264ProbeSustainsSource(baseline, current)).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects sustained encoder frame dropping", () => {
|
||||
const baseline = sample();
|
||||
const current = sample({
|
||||
timestamp: 2_000,
|
||||
framesEncoded: 22,
|
||||
sourceFrames: 40,
|
||||
encodedFramesPerSecond: 12,
|
||||
sourceFramesPerSecond: 30,
|
||||
});
|
||||
|
||||
expect(h264ProbeSustainsSource(baseline, current)).toBe(false);
|
||||
});
|
||||
|
||||
it("waits for enough source progress before deciding", () => {
|
||||
const baseline = sample();
|
||||
const current = sample({
|
||||
timestamp: 1_400,
|
||||
framesEncoded: 20,
|
||||
sourceFrames: 20,
|
||||
});
|
||||
|
||||
expect(h264ProbeSustainsSource(baseline, current)).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects CPU-limited or identity-mismatched samples", () => {
|
||||
const baseline = sample();
|
||||
|
||||
expect(
|
||||
h264ProbeSustainsSource(
|
||||
baseline,
|
||||
sample({
|
||||
timestamp: 2_000,
|
||||
framesEncoded: 40,
|
||||
sourceFrames: 40,
|
||||
qualityLimitationReason: "cpu",
|
||||
}),
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
h264ProbeSustainsSource(
|
||||
baseline,
|
||||
sample({
|
||||
outboundId: "replacement",
|
||||
timestamp: 2_000,
|
||||
framesEncoded: 40,
|
||||
sourceFrames: 40,
|
||||
}),
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("uses reported cadence when cumulative source frames are unavailable", () => {
|
||||
const baseline = sample({ framesEncoded: null, sourceFrames: null });
|
||||
|
||||
expect(
|
||||
h264ProbeSustainsSource(
|
||||
baseline,
|
||||
sample({
|
||||
timestamp: 2_000,
|
||||
framesEncoded: null,
|
||||
sourceFrames: null,
|
||||
encodedFramesPerSecond: 29,
|
||||
sourceFramesPerSecond: 30,
|
||||
}),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(
|
||||
h264ProbeSustainsSource(
|
||||
baseline,
|
||||
sample({
|
||||
timestamp: 2_000,
|
||||
framesEncoded: null,
|
||||
sourceFrames: null,
|
||||
encodedFramesPerSecond: 12,
|
||||
sourceFramesPerSecond: 30,
|
||||
}),
|
||||
),
|
||||
).toBe(false);
|
||||
expect(
|
||||
h264ProbeSustainsSource(
|
||||
baseline,
|
||||
sample({
|
||||
timestamp: 1_500,
|
||||
framesEncoded: null,
|
||||
sourceFrames: null,
|
||||
encodedFramesPerSecond: 30,
|
||||
sourceFramesPerSecond: 30,
|
||||
}),
|
||||
),
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user