diff --git a/docs/adr/0005-automatic-hybrid-media-routing.md b/docs/adr/0005-automatic-hybrid-media-routing.md index cbeeabf..1de18c6 100644 --- a/docs/adr/0005-automatic-hybrid-media-routing.md +++ b/docs/adr/0005-automatic-hybrid-media-routing.md @@ -276,10 +276,12 @@ second mutable graph. another no-progress sample can invalidate an edge. Frozen JavaScript wall time is not media-failure proof. A newly composited current-generation frame clears stale local recovery presentation but does not create route authority. -- Manual media reconnect remains on the current exact route: a peer rebuild uses - the same parent, and an SFU reconnect reconstructs the same subscription. - Neither action performs quality selection or active reparenting. Only actual - recovery exhaustion reports route failure and enters normal reconciliation. +- Manual media reconnect remains on the current exact route. P2P directly + rebuilds the connection to the existing parent; SFU + disconnects and reconnects only the current LiveKit subscriber with fresh + scoped configuration. Neither action restarts Screener signaling, changes the + route revision, performs quality selection, or reparents. Only actual recovery + exhaustion reports route failure and enters normal reconciliation. - Authoritative pause aborts the pending child operation, including its current candidate and reservations, keeps the active graph, suppresses decoded-frame- stall decisions, and leaves new participants waiting. Resume always wakes a diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md index 9a48c81..87314b3 100644 --- a/docs/research/realtime-quality-adaptation.md +++ b/docs/research/realtime-quality-adaptation.md @@ -39,10 +39,47 @@ quantization or frame delivery, so the earlier no-hint improvement was not free adaptation. Current policy follows the standard game-motion intent and leaves the resulting tradeoff to the browser. -The Host configures a sender before the first offer and reapplies the current -video profile after accepting an answer. The latter is necessary because -negotiation may replace or rewrite encoding parameters; it is not a periodic -controller and does not reset bandwidth estimation on a timer. +### Chrome 151 Startup Matrix + +On 2026-08-25, Chrome `151.0.7922.174` on Windows reproduced the production +symptom in a controlled local loopback with no external network. Every run used +the same fake `getDisplayMedia` monitor at 1920x1080@30, one P2P sender and +receiver, VP8 `libvpx` with `powerEfficientEncoder = false`, and sender ceilings +of 5 Mbps and 30 fps. Only the named startup input changed. + +| Startup input | About 1 second | About 4 seconds | About 8 seconds | Limitation | +| --- | --- | --- | --- | --- | +| `motion + balanced` | 480x270, 13 fps | 480x270, 15 fps | 480x270, 14 fps | `bandwidth` | +| `motion + balanced + x-google-start-bitrate=4500` | 480x270, 14 fps | 480x270, 15 fps | 480x270, 14 fps | `bandwidth` | +| no hint + balanced | 1920x1080, 18 fps | 1920x1080, 19 fps | 1920x1080, 20 fps | `none` | +| `detail + balanced` | 1920x1080, 18 fps | 1920x1080, 19 fps | 1920x1080, 20 fps | `none` | +| `motion + maintain-resolution` | 1920x1080, 17 fps | 1920x1080, 20 fps | 1920x1080, 20 fps | `none` | + +The transition matrix then started with `motion + maintain-resolution` and +changed only the sender degradation preference to balanced: + +| Balanced transition | Observed result | +| --- | --- | +| At connection, about 11 ms | Fell to 480x270 and remained bandwidth-limited | +| After the first encoded frame, about 80 ms | Retained 1280x720 but remained bandwidth-limited | +| After five encoded frames, about 442 ms | Retained 1920x1080 with no limitation through 8 seconds | +| After one or two seconds | Retained 1920x1080 with no limitation through 8 seconds | + +This isolates the failure from network, codec selection, hardware encoding, +capture constraints, sender readback, and the startup-bitrate SDP hint. +Chromium maps `motion` to non-screencast realtime video, where balanced permits +startup resolution restrictions; entering or leaving balanced clears those +restrictions. Five encoded frames is the first measured safe media fact and is +beyond libwebrtc's four-frame startup-drop bound, rather than an arbitrary wall +clock delay. + +Screener keeps `motion` because its later multilevel resolution adaptation is +required. Each new peer sender and SFU publication therefore starts with the +selected ceilings but an effective `maintain-resolution` preference. After the +current sender has encoded at least five frames, the existing stats path applies +the user's desired preference once. Connected was too early, and one encoded +frame retained only 720p in the controlled matrix; five frames retained 1080p. +There is no added timer, periodic rewrite, or application quality controller. ## LiveKit SFU Evidence @@ -52,6 +89,10 @@ representations, and that server send-side BWE can select a lower representation for a constrained subscriber while an unconstrained subscriber receives the highest available representation. +Pinned LiveKit retains its own codec and startup-bitrate behavior. Screener does +not add an SDP startup hint to raw peers; the controlled matrix showed that it +does not address this `motion + balanced` restriction. + An earlier exact-production A/B also showed that forcing an always-active lower encoding can consume the same Host-to-SFU congestion budget and reduce the highest encoding. That evidence rejects Screener mutation of active layers; it @@ -160,6 +201,12 @@ interval. Missing counters and identity changes remain unknown, not zero. - [WebRTC](https://www.w3.org/TR/webrtc/) - [WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) - [libwebrtc adaptation overview](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/adaptation.md) +- [Chromium content-hint capture mapping](https://chromium.googlesource.com/chromium/src/+/3468eea378284a9cc42d05532cf3e1ee1f716fa9/content/renderer/media/webrtc/webrtc_video_capturer_adapter.cc) +- [libwebrtc content-hint sender mapping](https://webrtc.googlesource.com/src/+/98c256dadcab7c69e45de78091da9932d244f2e3/pc/rtp_sender.cc) +- [libwebrtc balanced restriction reset](https://webrtc.googlesource.com/src/+/f20ebb8adbf4fa781830e4384c61f732bd28a217/call/adaptation/video_stream_adapter.cc) +- [libwebrtc startup frame dropper](https://webrtc.googlesource.com/src/+/f20ebb8adbf4fa781830e4384c61f732bd28a217/video/adaptation/video_stream_encoder_resource_manager.cc) +- [LiveKit initial-quality fix](https://github.com/livekit/client-sdk-js/pull/1987) +- [LiveKit initial-quality implementation](https://github.com/livekit/client-sdk-js/commit/5db17af) - [LiveKit screen-share encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts) - [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/) - [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/) diff --git a/docs/todo.md b/docs/todo.md index e3f6cb0..dece978 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -27,30 +27,37 @@ later topic is not implementation authority by itself. ## Accepted Later Roadmap -1. **Quality-based topology research.** After the real-network baseline, decide +1. **Preferred room reacquisition.** Keep the last assigned four-digit code and + its 24-hour preference window in the Host browser. When prior ownership is + no longer valid, the next explicit share requests that code only if the + preference is still current; the server grants it only while free and + otherwise uses normal random allocation. This does not reserve a dormant + code or restore the old room incarnation, Host token, Viewer grant, password + verifier, lease, participants, or routes. +2. **Quality-based topology research.** After the real-network baseline, decide whether measured user-visible quality justifies any active parent-selection mechanism. Current evidence cannot compare an active route with an unconnected alternative, so no quality score, all-pairs probing, periodic rebalancing, relay abdication threshold, or active parent switch is accepted. Prefer mature algorithms and one general model if this boundary is reopened. -2. **Platform output only when real.** Revisit AirPlay/Cast only when a target +3. **Platform output only when real.** Revisit AirPlay/Cast only when a target browser and physical receiver prove the live `MediaStream` contract. System mirroring remains external. -3. **Public-server one-click package.** After functional and real-network work, +4. **Public-server one-click package.** After functional and real-network work, package the exact application, STUN/SFU, reverse proxy, secrets, and health checks for a user-owned public server. Do not call a partial installer ready. -4. **Fully local one-click package.** Package Windows/macOS/Linux Host capture, +5. **Fully local one-click package.** Package Windows/macOS/Linux Host capture, application server, and local state without requiring source or Node. Report public-origin, TLS, gateway, NAT, and firewall limits honestly. -5. **Native Host and shared encode, Windows first.** Productize only after real +6. **Native Host and shared encode, Windows first.** Productize only after real capture, hardware-only encode, audio, identity, RTP/RTCP feedback, resource, packaging, and licensing gates pass. Browser Host/relay keeps standard per-`RTCPeerConnection` encoding. -6. **Whole-product UI and bilingual decision.** Once media and route behavior +7. **Whole-product UI and bilingual decision.** Once media and route behavior stabilizes, review copy, responsive hierarchy, visual consistency, restrained motion, bundle/rendering cost, and Chinese/English scope once as a whole. Ordinary screen-specific edits do not create parallel documentation. -7. **Repository simplification audit.** After the Browser, route, room, and +8. **Repository simplification audit.** After the Browser, route, room, and physical-media checkpoints settle, inventory components, configuration, migrations, timers, compatibility paths, tests, and truth duplication using the [maintenance review](./maintenance.md#机制减负审查). Begin read-only and do diff --git a/docs/方案设计.md b/docs/方案设计.md index a1365e1..ebeb806 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -162,7 +162,7 @@ accepted 的唯一 wire 是 `screener-v12`，不做版本协商、旧 parser、 每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新，然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级，实际结果由 stats 观测；拒绝或浏览器改写必须进入可见状态，不能只写 `console.warn`。 -Accepted `screener-v12` Web 视频固定使用 `VP8`。P2P 与 browser relay 在首次 offer 前只保留 VP8 media capabilities 及 RTX/RED/FEC repair capabilities；无法应用该约束时明确失败，不静默协商另一媒体 codec。SFU publication 显式使用 VP8 且关闭 backup codec。UI、quality state/wire 和分享生命周期均没有 codec 分支；实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。Viewer answer 的 Opus stereo receive preference 是当前唯一应用 SDP 变换边界。 +Accepted `screener-v12` Web 视频固定使用 `VP8`。P2P 与 browser relay 在首次 offer 前只保留 VP8 media capabilities 及 RTX/RED/FEC repair capabilities；无法应用该约束时明确失败，不静默协商另一媒体 codec。SFU publication 显式使用 VP8 且关闭 backup codec。所有新 Web peer sender 与 SFU publication 保留 `motion` 和用户选择的分辨率、帧率及码率 ceiling；当 desired degradation preference 允许分辨率下降时，启动阶段的 effective preference 暂为 `maintain-resolution`，当前 sender 累计编码至少五帧后由既有 stats 路径一次性应用 desired preference。该切换使用标准 `setParameters()` 进入或离开 balanced 时的原生 restriction reset，不增加 timer、周期重写、质量评分或应用层调档器。UI、quality state/wire 和分享生命周期均没有 codec 分支；实际 outbound codec/profile、encoder implementation 与 stats 是只读结果真相。应用 SDP 变换边界仍只有 Viewer answer 的 Opus stereo receive preference。 Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`autoGainControl` 和 `voiceIsolation`，并以 `channelCount: { ideal: 2 }` 偏好双声道；这是当前 Chrome/Edge Host 基线的实现契约，用于避开网页 `getDisplayMedia()` 默认的语音处理和单声道 APM 候选，不宣称跨浏览器保证，也不固定采样率。所有值都是非强制 post-selection preference；不使用 `exact`、`min` 或 `advanced`。若 `getAudioTracks()` 为空，分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`：窗口选项请求该窗口/应用音频，整个屏幕选项允许用户勾选系统音频；标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些约束或 picker hints，且没有对应的标准来源类型读回，因此 UI 不额外推断或标注来源。系统音频由用户显式选择，可能包含语音或通知；video-only 始终可用。 diff --git a/docs/需求理解.md b/docs/需求理解.md index cdfb5b7..7e5c5ad 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -27,7 +27,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - 端点发送容量：所有非服务器端点使用同一个服务端权威稳态 outbound media-copy capacity，由唯一部署项 `ENDPOINT_MEDIA_COPY_CAPACITY` 配置，默认 `2` 且只接受 `1`、`2` 或 `3`；不读取旧键或保留第二套 tier。peer child 或 Host 的唯一 SFU publication 各消耗一份，上游接收不消耗该预算。客户端只能声明不高于部署值的可用容量，角色、浏览器、UA、设备类型和页面可见性不产生不同上限。服务端在每次 `authenticated` 权威快照中返回最终 `endpointMediaCopyCapacity`；普通非 peer-assisted 模式也只授权稳定加入顺序中的至多 `C` 个 Host child，超额 Viewer 保持入房等待并在槽位释放后提升，信令不能绕过该集合。Browser Host 与 Viewer 严格校验该值并用它约束稳态 assignment；并行产生媒体的 candidate 必须持有唯一 `TransitionOverlapSlot`，active+pending 物理 copy 只可达到 `min(C + 1, 3)` 并在 commit/abort 后释放。`C=3` 满载时必须先收到 active bounded-gap retirement 再 prepare，绝不产生第四份 copy。SFU subscriber egress 由独立 server admission 记账。当前不兼容房间合同以单一 release 原子升级；缺失/非法 cap、旧 wire 与 executable sender都在取得room/route authority前fail closed。 - P2P 优先：每条普通逻辑媒体边先尝试 direct/peer UDP，服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布；每名 Viewer 同时至多一个 active upstream，失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate；Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction，并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时，同一 reconcile 将 affected Host direct child（若无则稳定顺序中最新的 connected Host direct child）作为本次 child operation，创建唯一 Host publication 并把该 child 变为首个 SFU subscriber，subtree 不动；publication 取代释放的 Host peer slot，下一次扫描再处理原受限 Viewer，不建立 bootstrap 状态机。 - NAT 与传输边界：STUN 必须可用，所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`，保留 LiveKit signaling 交付的授权 SFU UDP candidates，并由标准 ICE 选择 nominated non-relay pair；不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC，也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威；Screener 不合成 remote candidate、不预测端口、不分类 NAT，也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity；成功即停止，失败、超时或绑定失效即撤销，direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP，全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers，再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media；当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输，必须把它作为 LiveKit 内部能力重新决策，不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target；HTTPS/WSS 继续独立使用 TLS/TCP。 -- 质量控制：推荐档位始终只有 720p30（3 Mbps）、1080p30（5 Mbps）、1080p60（8 Mbps）三档，默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`（854x480），其 FPS 与码率仍由高级控件独立选择；不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。默认 `1080p30` 与高级设置初始值采用 `maintain-resolution`（清晰优先），1080p60、720p30 继续采用 `balanced`（平衡），`maintain-framerate`（流畅优先）保留为显式选择。Browser 视频固定使用 VP8；界面、质量状态和 wire 不提供 codec 选择或切换。分享期间可直接换视频档位和 96/128/192 kbps 音频 sender ceiling，不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置，覆盖当前和以后建立的 sender；普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好，具体降级由 WebRTC/LiveKit 决定；必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。 +- 质量控制：推荐档位始终只有 720p30（3 Mbps）、1080p30（5 Mbps）、1080p60（8 Mbps）三档，默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`（854x480），其 FPS 与码率仍由高级控件独立选择；不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用中档 `balanced`（均衡），`maintain-resolution`（清晰）、`maintain-framerate`（流畅）保留为显式选择。Browser 视频固定使用 VP8；界面、质量状态和 wire 不提供 codec 选择或切换。分享期间可直接换视频档位和 64/128/192 kbps 音频 sender ceiling，音频默认选择中档 128 kbps，不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置，覆盖当前和以后建立的 sender；普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好，具体降级由 WebRTC/LiveKit 决定；必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。 - 路径隔离表示：direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC，不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 不设置 `screenShareSimulcastLayers`，由锁定的 LiveKit client 默认行为拥有 VP8 screen-share representations、source replacement 与 republish；LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 负责需求聚合与每个 subscriber 的实际下行层。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。所有当前 Viewer 都可能承担 peer children，因此 subscriber 保持 `adaptiveStream: false`，不得让本地元素尺寸或可见性控制 relay ingress。表示选择不改变拓扑、端点容量或路由失效权威。 - 路由失效输入：应用不推断 parent-wide 质量，也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`，或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时，使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`，route/connection变化或新帧重置、暂停抑制，同一 edge只报告一次，不增加轮询器。Active SFU route 暂时没有 video track、stats report 为空或 stats 读取失败时，现有采样 cadence 仍把该次作为无 decoded progress 推进 deadline；它不得停表、伪造连接指标或立即判定 edge 失效，只有 route 停用或 teardown 才停止该 cadence。该 operation 把失败的 exact direct parent 在当前 fact version 直接视为已尝试，先选择其他 direct parent，再进入有界 SFU suffix；新的外部事实才允许原 direct tuple 重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时，它作为child换父并保留subtree；disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。 - 断线恢复：临时网络变化先交给 WebRTC/LiveKit 的 ICE、consent、重连和流状态。Direct edge 可在 exact current parent/child sessions 内采用 framework rebuild 的新 connection ID而不改变topology。应用只在上述exact edge失效后运行一次有界reconcile；任一步成功即停止。候选耗尽时hard-invalid、confirmed-departed或overflow edge被退休并进入明确waiting，healthy bootstrap edge保留。所有异步结果受authenticated session、share generation、base/pending route revision与candidate connection identity约束，未受影响分支保持sticky。权威暂停abort未提交candidate、保留active graph、抑制stall，并在resume后重新reconcile。 @@ -45,8 +45,8 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - 独立房间入口：合法 `/r/{code}#v={grant}` 邀请直接进入对应房间；没有 grant 的 `/join` 或 `/r/{code}` 先显示明确标为“站点访问”的整站验证，再提交 code-only 尝试。`open` 且可准入时直接进入；未分配、已回收或已过期的 code 明确显示“房间不存在或已过期”且不提供密码输入；现存房间的其他预期拒绝使用 `ROOM_ACCESS_DENIED`，不披露具体 policy。有效 site-access cookie 只跳过第一步，两类密码不得复用或互相替代。Host 可设置、轮换或移除房间观看密码；密码接受 1 至 64 个可见 ASCII 字符，异步 scrypt 全局最多执行 2 个、等待 16 个派生。Host 可把房间密码保存在同源本地创建偏好中；它不得进入 URL、cookie、日志或服务端持久存储。 - 统一自动路由：旗舰部署开启 `PEER_ASSISTED_MEDIA` 后，每个房间使用一个服务端 route controller，独占 committed graph、一个 pending child operation 和 event-driven reconcile。运行时只增加一个串行pump、一个operation deadline timer与typed resource lifecycle task，不维护另一份graph。operation持有candidate list/cursor、一个current candidate/资源预留、fact version和一个总deadline。committed graph必须无环、source-reachable，且每名Viewer同时至多一个active upstream。加入、容量变化、节点离开、exact edge `failed/closed`，以及非暂停decoded-frame stall都只唤醒同一个reconcile；其余健康分支保持sticky。候选按当前身份、source reachability、无环、effective capacity与sender transition硬过滤，再按结果深度、剩余sender slots、稳定加入顺序和peer identity排序；因此 Host root退出后新释放的Host槽位比已有root的更深槽位优先，但健康旧边不因周期性平衡而迁移。server admission在执行当前cursor时独立reserve。candidate child首帧ready触发`settleReady(commitAdmissionSync)`，成功prepare `P`提交active `P`；失败释放资源并用同一monotonic allocator广播`R>P` rollback，cursor推进但总deadline不重置。bounded-gap先preflight server resource，再retire producer、按新graph重算未尝试suffix，并在切断healthy edge时追加一个普通restore candidate。relay自身ingress失效时作为child换父并保留subtree；relay disconnect/cap降低让overflow children逐个走同一事务，childless后删除。房间admission和无环约束最大深度，不设独立hard cap。普通peer始终STUN-only，SFU使用独立admission。 -- 观看端连接反馈：Viewer 舞台从鉴权到首帧使用一个由现有权威状态推导的稳定阶段反馈，至少区分等待分享、分配线路、建立 P2P、连接 SFU、等待首帧、浏览器阻止自动播放、恢复连接和有界失败。阶段不是进度百分比，也不产生 route authority；只有当前 video element 实际提交首帧后才显示正在播放。Autoplay rejection 通过原生媒体 controls 恢复；连接中不显示虚假的播放动作。手动“重新连接媒体”只重建当前 exact route：P2P 使用同一 parent，SFU 使用同一 publication/subscription，不触发主动换父、质量选优或 P2P/SFU 重选；真实恢复耗尽后才进入既有 route-failure operation。make-before-break 或仍有可见旧帧时保留画面并叠加恢复状态，不为显示反馈主动清空健康 media element。页面从冻结/隐藏恢复时先重置 decoded-stall wall-clock baseline 并重新证明当前代画面，冻结时间本身不得成为立即换路依据。Host 离线本身不使仍健康的媒体失效；exact upstream 硬失败或 route terminal failure 必须使 current frame proof 失效，已证明画面至多降为 retained background，不能遮蔽离线或有界失败。已接受的 `screener-v12` 服务端补充消息是 strict、current-revision-fenced `route-status` union，且只允许 `waiting/sfu-admission` 与 `failed/route-exhausted` 两种配对；它不携带 raw text、身份、candidate 或 retry hint，prepare/active、access、Host、signal、media 与 autoplay 继续由各自权威输入派生。 -- 质量控制扩展：推荐档位仍是默认入口且只保留 720p30/1080p30/1080p60 三档，当前页面初始 desired state 选择清晰优先的 1080p30 且不写 `localStorage`；“分享高级设置”允许 480p（854x480）/720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling、清晰/平衡/流畅三种偏好，以及离散的 `96 kbps 普通默认 | 128 kbps 音乐 | 192 kbps 保真` 屏幕音频 sender ceiling；480p 不形成推荐档或固定 FPS/码率组合，音频初始值为 96 kbps。视频档位和音频档位可在直播中显式应用。所有当前及以后建立的 Web Host、Browser relay 与 SFU sender 都读取最新 desired 档位，并各自在 fresh sender parameters 上应用和读回 endpoint-local applied 值；实际码率仍由浏览器拥塞控制和 stats 决定。Peer 只在唯一明确的 active Opus audio 上做结构化、幂等的 Viewer answer `stereo=1;maxaveragebitrate=192000` 变换，使同一接收上限覆盖三档；非目标形态或 parser/writer 异常必须原样保留浏览器 answer，不得阻断可继续的 mono/video edge。不得用宽泛字符串/正则改 SDP，也不得改写浏览器协商的 FEC。SFU 将 96/128 kbps 映射到 pinned SDK music preset，192 kbps 使用显式 ceiling，统一设置 `forceStereo = true`、`dtx = false` 与 `red = false`。不得暴露任意码率、声道、采样率、audio codec、stereo、DTX、RED 或 FEC 控件；192 kbps 只是显式较高上限，不是无损、高保真或最低码率保证。实际质量持续低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一组合称为“完美平衡”。未来 microphone voice 若进入范围，必须是独立 track/处理链，不能把 AEC/降噪/DTX 套到电影或游戏音频。 +- 观看端连接反馈：Viewer 舞台从鉴权到首帧使用一个由现有权威状态推导的稳定阶段反馈，至少区分等待分享、分配线路、建立 P2P、连接 SFU、等待首帧、浏览器阻止自动播放、恢复连接和有界失败。阶段不是进度百分比，也不产生 route authority；只有当前 video element 实际提交首帧后才显示正在播放。Autoplay rejection 通过原生媒体 controls 恢复；连接中不显示虚假的播放动作。手动“重新连接媒体”只恢复当前 exact route：P2P 直接重建到同一 parent 的连接，SFU 只断开并重连当前 publication 的 subscriber；两者都保留 Screener WebSocket、房间会话和 route revision，不触发主动换父、质量选优或 P2P/SFU 重选。真实恢复耗尽后才进入既有 route-failure operation。仍有已证明旧帧时保留为背景并显示恢复状态，不为显示反馈主动清空健康 media element。页面从冻结/隐藏恢复时先重置 decoded-stall wall-clock baseline 并重新证明当前代画面，冻结时间本身不得成为立即换路依据。Host 离线本身不使仍健康的媒体失效；exact upstream 硬失败或 route terminal failure 必须使 current frame proof 失效，已证明画面至多降为 retained background，不能遮蔽离线或有界失败。已接受的 `screener-v12` 服务端补充消息是 strict、current-revision-fenced `route-status` union，且只允许 `waiting/sfu-admission` 与 `failed/route-exhausted` 两种配对；它不携带 raw text、身份、candidate 或 retry hint，prepare/active、access、Host、signal、media 与 autoplay 继续由各自权威输入派生。 +- 质量控制扩展：推荐档位仍是默认入口且只保留 720p30/1080p30/1080p60 三档，当前页面初始 desired state 选择均衡的 1080p30 且不写 `localStorage`；“分享高级设置”允许 480p（854x480）/720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling、清晰/均衡/流畅三种偏好，以及离散的 `64 kbps 普通 | 128 kbps 音乐默认 | 192 kbps 保真` 屏幕音频 sender ceiling；480p 不形成推荐档或固定 FPS/码率组合，音频初始值为 128 kbps。视频档位和音频档位可在直播中显式应用。所有当前及以后建立的 Web Host、Browser relay 与 SFU sender 都读取最新 desired 档位，并各自在 fresh sender parameters 上应用和读回 endpoint-local applied 值；实际码率仍由浏览器拥塞控制和 stats 决定。Peer 只在唯一明确的 active Opus audio 上做结构化、幂等的 Viewer answer `stereo=1;maxaveragebitrate=192000` 变换，使同一接收上限覆盖三档；非目标形态或 parser/writer 异常必须原样保留浏览器 answer，不得阻断可继续的 mono/video edge。不得用宽泛字符串/正则改 SDP，也不得改写浏览器协商的 FEC。SFU 将 64/128 kbps 映射到 pinned SDK stereo music preset，192 kbps 使用显式 ceiling，统一设置 `forceStereo = true`、`dtx = false` 与 `red = false`。不得暴露任意码率、声道、采样率、audio codec、stereo、DTX、RED 或 FEC 控件；192 kbps 只是显式较高上限，不是无损、高保真或最低码率保证。实际质量持续低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一组合称为“完美平衡”。未来 microphone voice 若进入范围，必须是独立 track/处理链，不能把 AEC/降噪/DTX 套到电影或游戏音频。 - SFU 质量设置生命周期：直播中修改档位时，浏览器 sender 的 applied readback、LiveKit `LocalVideoTrack` 保存的 degradation preference、track restart options 与 `LocalTrackPublication` 的 republish options 必须作为同一次串行操作更新；任一步失败恢复旧 sender 与旧 SDK 保存态，旧 publication generation 不得回写当前状态。LiveKit full reconnect/republish 不得恢复初次发布的旧偏好。该修复不改变 codec、码率上限、层数或路由，也不能单独解释即时低帧。 - 本地预览性能候选：在相同动态场景中 A/B 测量 Host 动态预览开/关的 CPU、GPU、encode time 和实际 FPS。只有稳定收益且捕获、编码和远端画面不受影响时，才允许直播开始后默认显示静止缩略图或隐藏动态预览并提供按需展开；未测前不宣称收益。 - Peer-assisted 媒体路径：所有非服务器 endpoint 都受同一个服务端权威稳态 outbound media-copy capacity 约束，默认 `2` 且可静态配置为 `1`、`2` 或 `3`；peer child 与 Host publication 都按实际发送副本计数。每名 Viewer 任一时刻只有一个 active upstream，活动媒体图保持无环；只在离散事件中做有界分配或换父。浏览器把收到的标准 remote track 加入独立下游 `RTCPeerConnection` 时会逐跳解码并分别重编码，必须测量而不能称为 shared encode。服务端只在有界房间内存中保存严格校验的质量设置，不写入持久存储。 @@ -167,9 +167,9 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - Given 浏览器未返回系统音轨，When 分享开始，Then 分享者在启播前看到“当前来源没有可共享音频”的提示。 - Given 分享者选择标签页或窗口，When 浏览器支持对应音频捕获，Then picker 提供该来源音频而不扩大为系统混音。Given 分享者选择整个屏幕，When 浏览器支持系统音频，Then picker 提供由用户决定的系统音频开关。应用同时请求 `windowAudio: "window"` 和 `systemAudio: "include"`，不根据 track 存在性伪造来源标签。 - Given 浏览器返回分享音轨，When 开始发布，Then 应用设置标准 `contentHint = "music"` 作为内容意图元数据；该值可读回，但不作为 Opus、码率、声道、采样率、FEC、DTX 或立体声的质量证据。 -- Given Web Host 或浏览器 relay 建立或替换分享音轨，When 对实际音频 sender 应用发送参数，Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 96000 | 128000 | 192000`、调用 `setParameters()` 并立即读回，档位缺失时按 96000 处理；Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer，Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=192000`，并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用；空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续，不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增，其他 fmtp/FEC/session/media 字段逐项保留。 -- Given 通过 SFU 发布分享音轨，When 构建 LiveKit 2.22.0 publish options，Then 96 kbps 使用 `AudioPresets.musicHighQuality`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、192 kbps 使用同等结构的显式 sender ceiling，三档都设置 `forceStereo = true`、`dtx = false` 与 `red = false`；验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率，不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。 -- Given 分享正在直播或暂停且 Host 选择新的 96/128/192 kbps 音频档位，When 当前 profile mutation 串行执行，Then 服务端以 last-wins 保存并广播最新 desired profile，每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback，过期 completion 不得覆盖新值；以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试，不得谎称全房已收敛。当前 applied readback 保持 endpoint-local；没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation，也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge；其他浏览器只有实际 sender readback 成功时才算支持。 +- Given Web Host 或浏览器 relay 建立或替换分享音轨，When 对实际音频 sender 应用发送参数，Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 64000 | 128000 | 192000`、调用 `setParameters()` 并立即读回，档位缺失时按 128000 处理；Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer，Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=192000`，并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用；空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续，不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增，其他 fmtp/FEC/session/media 字段逐项保留。 +- Given 通过 SFU 发布分享音轨，When 构建 LiveKit 2.22.0 publish options，Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、192 kbps 使用同等结构的显式 sender ceiling，三档都设置 `forceStereo = true`、`dtx = false` 与 `red = false`；验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率，不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。 +- Given 分享正在直播或暂停且 Host 选择新的 64/128/192 kbps 音频档位，When 当前 profile mutation 串行执行，Then 服务端以 last-wins 保存并广播最新 desired profile，每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback，过期 completion 不得覆盖新值；以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试，不得谎称全房已收敛。当前 applied readback 保持 endpoint-local；没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation，也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge；其他浏览器只有实际 sender readback 成功时才算支持。 - Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质，When 查看 UI 或校验消息，Then 不存在 codec selector、codec quality-state/wire 字段或分享生命周期 codec 分支；Browser direct、browser relay 与 SFU publication 始终使用固定 VP8。实际协商 codec 与 encoder 只作为只读诊断显示。 - Given 浏览器返回屏幕音轨，When 产品展示音频状态或未来连接详情，Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实；字段缺失必须显示为未知，且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。 - Given Web 或后续 native sender 分享电影/视频音频，When 进行产品验收，Then 必须用独立 A/B/C 证据关联 capture audio track/settings、实际 negotiated codec/fmtp、outbound/inbound 实际码率、loss、jitter、concealment、jitter-buffer 与 A/V sync；P2P、浏览器 relay 与 SFU route switch 保持同一持久 Viewer `MediaStream`/媒体元素和 stereo 协商，不新增独立 audio element 或 Web Audio 时钟边界。native sender 必须通过该音质和同步矩阵，且不得把 audio `maxBitrate` ceiling 当作提质证明。 diff --git a/src/client/lib/signaling.ts b/src/client/lib/signaling.ts index 2059696..3d97ee0 100644 --- a/src/client/lib/signaling.ts +++ b/src/client/lib/signaling.ts @@ -113,21 +113,6 @@ export class SignalingClient { this.events.onStatus("offline"); } - reconnect(): boolean { - const socket = this.socket; - if ( - this.stopped || - !this.authenticated || - !socket || - socket.readyState >= WebSocket.CLOSING - ) { - return false; - } - this.clearSignalingWatchdog(); - socket.close(SIGNAL_CLOSE_CODES.clientReconnect, "client reconnect"); - return true; - } - send(message: ClientMessage): boolean { if ( !this.authenticated || diff --git a/src/client/media/quality.ts b/src/client/media/quality.ts index 020fe3e..e8d0346 100644 --- a/src/client/media/quality.ts +++ b/src/client/media/quality.ts @@ -17,6 +17,7 @@ export type { } from "../../shared/protocol"; export type QualityProfile = QualitySettings; +export const STARTUP_VIDEO_ENCODED_FRAMES = 5; export const QUALITY_PROFILES = { "720p30": { @@ -24,7 +25,7 @@ export const QUALITY_PROFILES = { maxFramerate: 30, maxBitrate: 3_000_000, degradationPreference: "balanced", - screenAudioQuality: "saver", + screenAudioQuality: "music", }, "1080p30": DEFAULT_QUALITY_SETTINGS, "1080p60": { @@ -32,7 +33,7 @@ export const QUALITY_PROFILES = { maxFramerate: 60, maxBitrate: 8_000_000, degradationPreference: "balanced", - screenAudioQuality: "saver", + screenAudioQuality: "music", }, } as const satisfies Record; @@ -70,15 +71,27 @@ export const SCREEN_AUDIO_QUALITY_LABELS = { "very-high": "保真", } as const satisfies Record; -export const DEFAULT_SCREEN_AUDIO_QUALITY: ScreenAudioQuality = "saver"; +export const DEFAULT_SCREEN_AUDIO_QUALITY: ScreenAudioQuality = "music"; export const SCREEN_AUDIO_BITRATES = { - saver: 96_000, + saver: 64_000, music: 128_000, "very-high": 192_000, } as const satisfies Record; export const SCREEN_AUDIO_RECEIVE_MAX_BITRATE = SCREEN_AUDIO_BITRATES["very-high"]; +export function startupVideoProfile( + profile: QualityProfile, +): QualityProfile { + return profile.degradationPreference === "maintain-resolution" + ? profile + : { ...profile, degradationPreference: "maintain-resolution" }; +} + +export function needsStartupVideoProfile(profile: QualityProfile): boolean { + return profile.degradationPreference !== "maintain-resolution"; +} + export interface VideoSenderParameterValues { maxBitrate: number | null; maxFramerate: number | null; diff --git a/src/client/media/viewer-presentation.ts b/src/client/media/viewer-presentation.ts index 9f95ad2..9395120 100644 --- a/src/client/media/viewer-presentation.ts +++ b/src/client/media/viewer-presentation.ts @@ -103,6 +103,7 @@ export type ViewerPresentationAction = revision: number; phase: "prepare" | "active"; kind: ViewerRouteKind; + preserveMedia?: boolean; } | { type: "route-status"; @@ -197,6 +198,10 @@ export function reduceViewerPresentation( } const revisionChanged = state.revision !== action.revision; const currentFrame = hasCurrentFrame(state); + const preserveMedia = + revisionChanged && + action.preserveMedia === true && + state.media?.revision === state.revision; return { ...state, revision: action.revision, @@ -210,12 +215,17 @@ export function reduceViewerPresentation( state.routeStatus.state === "waiting" ? state.routeStatus : null, - autoplayBlockedGeneration: revisionChanged + media: preserveMedia + ? { ...state.media!, revision: action.revision } + : state.media, + autoplayBlockedGeneration: revisionChanged && !preserveMedia ? null : state.autoplayBlockedGeneration, retainedFrame: - state.retainedFrame || (revisionChanged && currentFrame), - connection: revisionChanged + preserveMedia + ? state.retainedFrame + : state.retainedFrame || (revisionChanged && currentFrame), + connection: revisionChanged && !preserveMedia ? action.kind === "none" ? "idle" : "connecting" diff --git a/src/client/media/viewer-sfu-route.ts b/src/client/media/viewer-sfu-route.ts index d7011e7..d87f04f 100644 --- a/src/client/media/viewer-sfu-route.ts +++ b/src/client/media/viewer-sfu-route.ts @@ -99,6 +99,7 @@ export class ViewerSfuRoute { private transitionTail: Promise = Promise.resolve(); private resyncGeneration = 0; private resyncing = false; + private manualReconnectRevision: number | null = null; private paused = false; private closed = false; @@ -138,6 +139,7 @@ export class ViewerSfuRoute { } if (previousRevision !== update.revision) { this.recovery = null; + this.manualReconnectRevision = null; if ( update.phase !== "prepare" && this.pendingPeerRevision !== null @@ -210,7 +212,39 @@ export class ViewerSfuRoute { } } + reconnectActive(): boolean { + const revision = this.route.getRevision(); + const assignment = this.route.getActiveAssignment(); + const active = this.active; + if ( + this.closed || + this.resyncing || + this.manualReconnectRevision !== null || + this.pending !== null || + this.paused || + revision === null || + this.route.getPhase() !== "active" || + assignment?.upstream.kind !== "sfu" || + !active || + active.failed || + !active.activated || + active.revision !== revision || + active.publicationGeneration !== assignment.sfuPublicationGeneration + ) { + return false; + } + + this.manualReconnectRevision = revision; + if (!this.events.send({ type: "refresh-sfu", revision })) { + this.manualReconnectRevision = null; + return false; + } + this.events.onSfuState?.("reconnecting", revision); + return true; + } + private discardPending(): void { + this.manualReconnectRevision = null; this.pendingPeerRevision = null; this.events.preparePeer?.(null); this.events.prepareChild?.(null); @@ -233,6 +267,7 @@ export class ViewerSfuRoute { const resyncGeneration = ++this.resyncGeneration; this.resyncing = true; + this.manualReconnectRevision = null; this.route.reset(); this.recovery = null; this.events.prepareChild?.(null); @@ -292,6 +327,8 @@ export class ViewerSfuRoute { const phase = this.route.getPhase(); const candidate = this.route.getPreparedCandidate(); const publicationGeneration = assignment?.sfuPublicationGeneration; + const manualReconnect = + phase === "active" && this.manualReconnectRevision === message.revision; if ( !token || !phase || @@ -307,7 +344,8 @@ export class ViewerSfuRoute { if ( phase === "active" && this.active?.publicationGeneration === publicationGeneration && - this.active.activated + this.active.activated && + !manualReconnect ) { if (phase === "active") { await this.queueActiveRoute(token, true, this.paused); @@ -420,6 +458,7 @@ export class ViewerSfuRoute { return; } this.closed = true; + this.manualReconnectRevision = null; this.resyncGeneration += 1; this.resyncing = false; this.route.reset(); @@ -550,10 +589,12 @@ export class ViewerSfuRoute { ): Promise { const assignment = this.route.getActiveAssignment(); const publicationGeneration = assignment?.sfuPublicationGeneration; + const manualReconnect = this.manualReconnectRevision === token.revision; if ( this.active?.revision === token.revision && this.active.publicationGeneration === publicationGeneration && - this.active.activated + this.active.activated && + !manualReconnect ) { return; } @@ -634,6 +675,7 @@ export class ViewerSfuRoute { slot.subscriber.stopDecodedFrameProof(); this.pending = null; const previous = this.active; + const manualReconnect = this.manualReconnectRevision === slot.revision; this.active = slot; if (!this.route.markMediaActive(token)) { this.active = previous; @@ -641,6 +683,9 @@ export class ViewerSfuRoute { return; } this.recovery = null; + if (manualReconnect) { + this.manualReconnectRevision = null; + } slot.mediaAvailable = true; this.events.onSfuStream(slot.stream, assignment, true, slot.revision); if (previous && previous !== slot) { @@ -651,6 +696,9 @@ export class ViewerSfuRoute { } await disconnectSubscriber(previous.subscriber); } + if (manualReconnect) { + this.events.onSfuState?.("connected", slot.revision); + } } private handleVideoAvailability( @@ -735,6 +783,12 @@ export class ViewerSfuRoute { return; } const wasActive = this.active === slot; + const failedManualReconnect = + this.pending === slot && + this.manualReconnectRevision === slot.revision && + this.active?.publicationGeneration === slot.publicationGeneration && + this.active.activated && + !this.active.failed; slot.failed = true; slot.subscriber.stopDecodedFrameProof(); if (this.pending === slot) { @@ -744,6 +798,11 @@ export class ViewerSfuRoute { this.active = null; this.events.onSfuUpdate?.(null, slot.revision); } + if (failedManualReconnect) { + this.manualReconnectRevision = null; + this.events.onSfuState?.("connected", slot.revision); + return; + } const assignment = this.route.getPlannedAssignment(); const revision = this.route.getRevision(); diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx index be7a4f1..2e02b34 100644 --- a/src/client/pages/HostPage.tsx +++ b/src/client/pages/HostPage.tsx @@ -2268,7 +2268,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { aria-pressed={ advancedQuality.degradationPreference === preference } - disabled={phase === "starting" || switchingSource} + disabled={phase !== "live" || switchingSource} onClick={() => changeAdvancedQuality({ degradationPreference: preference, diff --git a/src/client/pages/ViewerPage.tsx b/src/client/pages/ViewerPage.tsx index 8145a5a..e1bdc2a 100644 --- a/src/client/pages/ViewerPage.tsx +++ b/src/client/pages/ViewerPage.tsx @@ -189,6 +189,7 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { const videoRef = useRef(null); const peerRef = useRef(null); + const viewerSfuRouteRef = useRef(null); const signalRef = useRef(null); const displayNameRef = useRef(displayName); const remoteMediaRef = useRef(null); @@ -207,7 +208,9 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { peerConnectionIdentity, ); const reconnectAvailable = - signalStatus === "connected" && reconnectRoute !== null; + signalStatus === "connected" && + reconnectRoute !== null && + presentationState.connection !== "reconnecting"; const routeConnectionState = routePresentation.evidence?.connectionState ?? (hostOnline ? "routing" : "waiting"); @@ -280,7 +283,16 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { revision: number, upstream: ParticipantRouteAssignment["upstream"], phase: "prepare" | "active" = "active", + preserveMedia = false, ): void { + if (preserveMedia) { + const current = remoteMediaRef.current; + if (current && current.revision !== revision) { + const rebased = { ...current, revision }; + remoteMediaRef.current = rebased; + setRemoteMedia(rebased); + } + } setAssignedRoute((current) => current && revision < current.revision ? current @@ -291,6 +303,7 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { revision, phase, kind: routeKindFromAssignment(upstream), + preserveMedia, }); } @@ -822,6 +835,7 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { active && viewerSfuRoute === route ? signal.send(message) : false, }); viewerSfuRoute = route; + viewerSfuRouteRef.current = route; return route; } @@ -829,6 +843,9 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { const route = viewerSfuRoute; discardPendingPeer(); viewerSfuRoute = null; + if (viewerSfuRouteRef.current === route) { + viewerSfuRouteRef.current = null; + } setSfuUpstream(null); void route?.disconnect(); } @@ -1033,7 +1050,9 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { dispatchPresentation({ type: "connection", revision: currentRouteRevision, - connection: connectionFact(snapshot.connectionState), + connection: peer.isRecovering() + ? "reconnecting" + : connectionFact(snapshot.connectionState), }); } }, @@ -1192,11 +1211,13 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { revision: message.revision, connectionId: message.candidate.connectionId, }; + acceptAssignedRoute( + message.revision, + message.assignment.upstream, + "prepare", + ); } - if (result === "accepted" && message.phase === "active") { - if (message.revision !== currentRouteRevision) { - clearRelayChildEvidence(); - } + if (message.phase === "active") { const samePeerUpstream = currentRouteAssignment?.upstream.kind === "peer" && message.assignment.upstream.kind === "peer" && @@ -1207,23 +1228,41 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { message.assignment.upstream.kind === "sfu" && currentRouteAssignment.sfuPublicationGeneration === message.assignment.sfuPublicationGeneration; + const peerIdentity = peerRef.current?.getConnectionIdentity(); + const exactPeerUpstream = + samePeerUpstream && + currentRouteAssignment?.upstream.kind === "peer" && + currentRouteConnectionId !== null && + peerIdentity != null && + peerIdentity.parentPeerId === + currentRouteAssignment.upstream.peerId && + peerIdentity.connectionId === currentRouteConnectionId; + const preserveMedia = + remoteMediaRef.current !== null && + (sameSfuUpstream || exactPeerUpstream); const connectionId = pendingRouteConnection?.revision === message.revision ? pendingRouteConnection.connectionId : samePeerUpstream || sameSfuUpstream ? currentRouteConnectionId : null; - activateRouteIdentity( + if (result === "accepted") { + if (message.revision !== currentRouteRevision) { + clearRelayChildEvidence(); + } + activateRouteIdentity( + message.revision, + message.assignment, + connectionId, + ); + } + acceptAssignedRoute( message.revision, - message.assignment, - connectionId, + message.assignment.upstream, + "active", + preserveMedia, ); } - acceptAssignedRoute( - message.revision, - message.assignment.upstream, - message.phase, - ); } } return; @@ -1483,6 +1522,9 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { signalRef.current = null; } void viewerSfuRoute?.disconnect(); + if (viewerSfuRouteRef.current === viewerSfuRoute) { + viewerSfuRouteRef.current = null; + } viewerSfuRoute = null; preparedParentPeerId = null; preparedParentSignals = []; @@ -1571,17 +1613,16 @@ export function ViewerPage({ roomId, viewerGrant }: ViewerPageProps) { }, [presentation.overlay]); function retryConnection(): void { - const requested = - reconnectRoute === "sfu" - ? signalRef.current?.reconnect() === true - : reconnectRoute === "p2p" && - peerRef.current?.requestRecovery(true) === true; - if (requested) { - dispatchPresentation({ - type: "connection", - revision: presentationState.revision ?? assignedRoute?.revision ?? 0, - connection: "reconnecting", - }); + if (reconnectRoute === "sfu") { + viewerSfuRouteRef.current?.reconnectActive(); + } else if (reconnectRoute === "p2p") { + if (peerRef.current?.requestRecovery(true)) { + dispatchPresentation({ + type: "connection", + revision: presentationState.revision ?? assignedRoute?.revision ?? 0, + connection: "reconnecting", + }); + } } } diff --git a/src/client/sfu/publisher.ts b/src/client/sfu/publisher.ts index 5028fca..1a42dda 100644 --- a/src/client/sfu/publisher.ts +++ b/src/client/sfu/publisher.ts @@ -9,10 +9,13 @@ import { audioSenderParameterWarning, configureScreenAudioSender, configureVideoSender, + needsStartupVideoProfile, resolveScreenAudioQuality, screenAudioQualityEqual, screenAudioBitrate, senderParameterWarning, + STARTUP_VIDEO_ENCODED_FRAMES, + startupVideoProfile, videoQualitySettingsEqual, type AudioSenderParameterReadback, type QualityProfile, @@ -24,6 +27,7 @@ import { captureMetrics, collectConnectionMetricsFromReport, createStatsAccumulator, + maxEncodedVideoFrames, mergeStatsReports, type StatsAccumulator, } from "../webrtc/stats"; @@ -106,6 +110,7 @@ export class SfuPublisher { private statsIdentity: PublisherStatsIdentity | null = null; private statsInFlight: PublisherStatsIdentity | null = null; private statsTimer: ReturnType | null = null; + private startupVideoProfilePending = false; constructor(private readonly events: PublisherEvents = {}) {} @@ -166,6 +171,8 @@ export class SfuPublisher { activate(stream: MediaStream, profile: QualityProfile): Promise { this.desiredProfile = profile; + this.startupVideoProfilePending = needsStartupVideoProfile(profile); + const effectiveVideoProfile = startupVideoProfile(profile); ++this.profileRevision; const generation = this.generation; return this.enqueue(async () => { @@ -188,7 +195,7 @@ export class SfuPublisher { room, videoTrack, sdk.Track.Source.ScreenShare, - videoPublishOptions(profile), + videoPublishOptions(effectiveVideoProfile), ); if (!this.owns(room, generation)) { return false; @@ -196,7 +203,7 @@ export class SfuPublisher { failureStage = "sender-config"; const videoConfiguration = await configurePublishedVideo( video, - profile, + effectiveVideoProfile, () => this.owns(room, generation), ); if (!videoConfiguration || !this.owns(room, generation)) { @@ -240,7 +247,7 @@ export class SfuPublisher { this.video = video; this.audio = audio; - this.appliedVideoProfile = profile; + this.appliedVideoProfile = effectiveVideoProfile; this.retainSenderParameters( videoConfiguration, audioConfiguration?.readback ?? null, @@ -299,6 +306,7 @@ export class SfuPublisher { this.audio = null; this.desiredProfile = null; this.appliedVideoProfile = null; + this.startupVideoProfilePending = false; ++this.profileRevision; this.senderParameters = null; this.audioSenderParameters = null; @@ -324,6 +332,7 @@ export class SfuPublisher { if (!sdk || !previousVideo || !profile || !previousVideoProfile) { throw new Error("SFU publisher has no active video publication"); } + const effectiveVideoProfile = startupVideoProfile(profile); const nextVideoTrack = requiredVideoTrack(stream); const nextAudioTrack = stream.getAudioTracks()[0] ?? null; @@ -391,7 +400,7 @@ export class SfuPublisher { const videoConfiguration = await configurePublishedVideo( previousVideo, - profile, + effectiveVideoProfile, () => this.owns(room, generation), ); if (!videoConfiguration || !this.owns(room, generation)) { @@ -401,7 +410,8 @@ export class SfuPublisher { if (previousAudio && nextAudioTrack) { previousAudio.rawTrack = nextAudioTrack; } - this.appliedVideoProfile = profile; + this.startupVideoProfilePending = needsStartupVideoProfile(profile); + this.appliedVideoProfile = effectiveVideoProfile; this.retainSenderParameters( videoConfiguration, nextAudioTrack @@ -471,11 +481,20 @@ export class SfuPublisher { updateProfile(profile: QualityProfile): Promise { const previousDesiredProfile = this.desiredProfile; + if (!needsStartupVideoProfile(profile)) { + this.startupVideoProfilePending = false; + } + const effectiveVideoProfile = this.startupVideoProfilePending + ? startupVideoProfile(profile) + : profile; const requestedVideo = previousDesiredProfile === null || !videoQualitySettingsEqual(previousDesiredProfile, profile) || this.appliedVideoProfile === null || - !videoQualitySettingsEqual(this.appliedVideoProfile, profile); + !videoQualitySettingsEqual( + this.appliedVideoProfile, + effectiveVideoProfile, + ); const requestedAudio = previousDesiredProfile === null || !screenAudioQualityEqual(previousDesiredProfile, profile) || @@ -504,7 +523,10 @@ export class SfuPublisher { } const updateVideo = requestedVideo || - !videoQualitySettingsEqual(previousVideoProfile, profile); + !videoQualitySettingsEqual( + previousVideoProfile, + effectiveVideoProfile, + ); const currentAudioSender = audio?.publication.audioTrack?.sender ?? null; const updateAudio = audio !== null && @@ -530,7 +552,7 @@ export class SfuPublisher { try { const configured = await configurePublishedVideo( video, - profile, + effectiveVideoProfile, () => this.owns(room, generation) && requestedRevision === this.profileRevision && @@ -544,7 +566,7 @@ export class SfuPublisher { } senderParameters = configured.readback; videoWarning = configured.warning; - appliedVideoProfile = profile; + appliedVideoProfile = effectiveVideoProfile; } catch { if ( !this.owns(room, generation) || @@ -909,6 +931,7 @@ export class SfuPublisher { this.audio = null; this.desiredProfile = null; this.appliedVideoProfile = null; + this.startupVideoProfilePending = false; ++this.profileRevision; this.senderParameters = null; this.audioSenderParameters = null; @@ -1039,6 +1062,17 @@ export class SfuPublisher { ), ...captureMetrics(identity.videoTrack), }; + if ( + this.startupVideoProfilePending && + maxEncodedVideoFrames(report, identity.videoTrack.id) >= + STARTUP_VIDEO_ENCODED_FRAMES + ) { + this.startupVideoProfilePending = false; + const desiredProfile = this.desiredProfile; + if (desiredProfile) { + void this.updateProfile(desiredProfile); + } + } this.events.onStats?.(metrics); } catch { // Stats are observational and must never disrupt active SFU media. @@ -1233,7 +1267,7 @@ function audioPublishOptions( const resolvedQuality = resolveScreenAudioQuality(quality); const audioPreset = resolvedQuality === "saver" - ? sdk.AudioPresets.musicHighQuality + ? sdk.AudioPresets.musicStereo : resolvedQuality === "music" ? sdk.AudioPresets.musicHighQualityStereo : { maxBitrate: screenAudioBitrate(resolvedQuality) }; diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts index bca96c3..be2bad5 100644 --- a/src/client/webrtc/host-peer.ts +++ b/src/client/webrtc/host-peer.ts @@ -4,9 +4,12 @@ import { audioSenderParameterWarning, configureScreenAudioSender, configureVideoSender, + needsStartupVideoProfile, resolveScreenAudioQuality, screenAudioQualityEqual, senderParameterWarning, + STARTUP_VIDEO_ENCODED_FRAMES, + startupVideoProfile, videoQualitySettingsEqual, type AudioSenderParameterReadback, type QualityProfile, @@ -61,6 +64,7 @@ export class HostPeer { private ordinaryAnswerEpoch: number | null = null; private senderMutationTail: Promise = Promise.resolve(); private negotiationTail: Promise = Promise.resolve(); + private startupVideoProfilePending: boolean; private snapshot: PeerSnapshot; constructor( @@ -72,6 +76,7 @@ export class HostPeer { connectionId = createOpaqueId(), ) { this.connectionId = connectionId; + this.startupVideoProfilePending = needsStartupVideoProfile(desiredProfile); this.connection = new RTCPeerConnection({ iceServers: iceConfig.iceServers, }); @@ -113,7 +118,12 @@ export class HostPeer { if (this.disposed || !this.videoSender || !this.audioSender) { return false; } - return this.configureSender(this.videoSender, this.audioSender); + return this.configureSender(this.videoSender, this.audioSender, { + profile: startupVideoProfile(this.desiredProfile), + profileRevision: this.profileRevision, + video: true, + audio: true, + }); }); if (!(await this.createOffer(false)) || this.disposed) { return false; @@ -161,10 +171,18 @@ export class HostPeer { return false; } this.stream = nextStream; + this.startupVideoProfilePending = needsStartupVideoProfile( + this.desiredProfile, + ); this.limitationReason = null; this.limitationSamples = 0; this.snapshot = { ...this.snapshot, metrics: { ...EMPTY_METRICS } }; - await this.configureSender(videoSender, audioSender); + await this.configureSender(videoSender, audioSender, { + profile: startupVideoProfile(this.desiredProfile), + profileRevision: this.profileRevision, + video: this.connection.connectionState === "connected", + audio: true, + }); this.snapshot = { ...this.snapshot, error: null }; this.emit(); return true; @@ -179,10 +197,18 @@ export class HostPeer { if (this.disposed) { return Promise.resolve(false); } + if (!needsStartupVideoProfile(profile)) { + this.startupVideoProfilePending = false; + } + const effectiveVideoProfile = this.startupVideoProfilePending + ? startupVideoProfile(profile) + : profile; const requestedVideo = - !videoQualitySettingsEqual(this.desiredProfile, profile) || this.appliedVideoProfile === null || - !videoQualitySettingsEqual(this.appliedVideoProfile, profile); + !videoQualitySettingsEqual( + this.appliedVideoProfile, + effectiveVideoProfile, + ); const requestedAudio = !screenAudioQualityEqual(this.desiredProfile, profile) || this.appliedAudioQuality !== @@ -201,9 +227,13 @@ export class HostPeer { return false; } const updateVideo = - requestedVideo || - this.appliedVideoProfile === null || - !videoQualitySettingsEqual(this.appliedVideoProfile, profile); + this.connection.connectionState === "connected" && + (requestedVideo || + this.appliedVideoProfile === null || + !videoQualitySettingsEqual( + this.appliedVideoProfile, + effectiveVideoProfile, + )); const updateAudio = requestedAudio || this.appliedAudioQuality !== @@ -213,7 +243,7 @@ export class HostPeer { } if ( !(await this.configureSender(videoSender, audioSender, { - profile, + profile: effectiveVideoProfile, profileRevision: requestedRevision, video: updateVideo, audio: updateAudio, @@ -318,7 +348,9 @@ export class HostPeer { : null, }); }); - this.connection.addEventListener("connectionstatechange", () => this.emit()); + this.connection.addEventListener("connectionstatechange", () => { + this.emit(); + }); this.connection.addEventListener("iceconnectionstatechange", () => this.emit()); } @@ -419,19 +451,6 @@ export class HostPeer { return; } await this.flushCandidates(); - await this.enqueueSenderMutation(async () => { - const videoSender = this.videoSender; - const audioSender = this.audioSender; - if (this.disposed || !videoSender || !audioSender) { - return false; - } - return this.configureSender(videoSender, audioSender, { - profile: this.desiredProfile, - profileRevision: this.profileRevision, - video: true, - audio: false, - }); - }); if (this.ownsAnswer(epoch)) { this.ordinaryAnswerEpoch = null; } @@ -507,6 +526,13 @@ export class HostPeer { ) { return; } + if ( + this.startupVideoProfilePending && + (statsAccumulator.frames ?? 0) >= STARTUP_VIDEO_ENCODED_FRAMES + ) { + this.startupVideoProfilePending = false; + void this.updateProfile(this.desiredProfile); + } this.updateLimitationWarning(metrics.qualityLimitationReason); this.snapshot = { ...this.snapshot, diff --git a/src/client/webrtc/stats.ts b/src/client/webrtc/stats.ts index 8d16b59..c682880 100644 --- a/src/client/webrtc/stats.ts +++ b/src/client/webrtc/stats.ts @@ -278,6 +278,25 @@ export function decodedVideoFrames( return numberValue(mediaRecord(report, "receive", selector), "framesDecoded"); } +export function maxEncodedVideoFrames( + report: RTCStatsReport, + trackIdentifier: string, +): number { + let frames = 0; + report.forEach((raw) => { + const record = raw as StatsRecord; + if ( + record.type === "outbound-rtp" && + record.kind === "video" && + record.isRemote !== true && + mediaTrackIdentifier(report, record, "send") === trackIdentifier + ) { + frames = Math.max(frames, numberValue(record, "framesEncoded") ?? 0); + } + }); + return frames; +} + function transportRecord( report: RTCStatsReport, media: StatsRecord | null, diff --git a/src/client/webrtc/viewer-peer.ts b/src/client/webrtc/viewer-peer.ts index aefe5b4..05c7e37 100644 --- a/src/client/webrtc/viewer-peer.ts +++ b/src/client/webrtc/viewer-peer.ts @@ -113,7 +113,12 @@ export class ViewerPeer { } requestRecovery(rebuild = false): boolean { - if (!this.connectionId || !this.parentPeerId) { + if ( + this.recoveryTimer !== null || + !this.connection || + !this.connectionId || + !this.parentPeerId + ) { return false; } const sent = this.events.sendRestartRequest( @@ -132,6 +137,10 @@ export class ViewerPeer { return sent; } + isRecovering(): boolean { + return this.recoveryTimer !== null; + } + hasConnection(): boolean { return this.connection !== null; } @@ -429,6 +438,7 @@ export class ViewerPeer { if (this.connection?.connectionState !== "connected") { this.attemptAutomaticRecovery(); } + this.emit(); }, AUTOMATIC_RECOVERY_TIMEOUT_MS); } diff --git a/src/server/hybrid-media-router.ts b/src/server/hybrid-media-router.ts index c8e9b3a..424197c 100644 --- a/src/server/hybrid-media-router.ts +++ b/src/server/hybrid-media-router.ts @@ -78,7 +78,7 @@ interface RoomRuntime { requested: boolean; pump?: Promise; deadlineTimer?: NodeJS.Timeout; - refreshKeys: Set; + sfuRefreshesInFlight: Set; } interface PreparedCandidate { @@ -617,9 +617,11 @@ export class HybridMediaRouter { const snapshot = room?.controller?.snapshot(); if (!room || !snapshot || snapshot.revision !== revision) return; const key = `${revision}\u0000${participant.peerId}\u0000${participant.sessionId}`; - if (room.refreshKeys.has(key)) return; - room.refreshKeys.add(key); - void this.sendFreshSfuConfig(participant).catch(() => undefined); + if (room.sfuRefreshesInFlight.has(key)) return; + room.sfuRefreshesInFlight.add(key); + void this.sendFreshSfuConfig(participant) + .catch(() => undefined) + .finally(() => room.sfuRefreshesInFlight.delete(key)); } disconnectParticipant(roomId: string, peerId: string, sessionId: string): void { @@ -652,7 +654,7 @@ export class HybridMediaRouter { room = { advertisedCapacityByViewer: new Map(), requested: false, - refreshKeys: new Set(), + sfuRefreshesInFlight: new Set(), }; this.rooms.set(roomId, room); } diff --git a/src/shared/protocol.ts b/src/shared/protocol.ts index fdf57a8..23da06d 100644 --- a/src/shared/protocol.ts +++ b/src/shared/protocol.ts @@ -208,8 +208,8 @@ export const DEFAULT_QUALITY_SETTINGS = { resolution: "1080p", maxFramerate: 30, maxBitrate: 5_000_000, - degradationPreference: "maintain-resolution", - screenAudioQuality: "saver", + degradationPreference: "balanced", + screenAudioQuality: "music", } as const satisfies QualitySettings; export const relayDownstreamEdgesSchema = z diff --git a/tests/client-state.test.ts b/tests/client-state.test.ts index 5a99955..4787bd2 100644 --- a/tests/client-state.test.ts +++ b/tests/client-state.test.ts @@ -718,71 +718,6 @@ describe("client signaling recovery policy", () => { expect(sockets).toHaveLength(1); }); - it("restarts only an authenticated active signaling session on request", () => { - const sockets: FakeWebSocket[] = []; - class FakeWebSocket extends EventTarget { - static readonly CLOSING = 2; - readyState = 1; - readonly send = vi.fn(); - readonly close = vi.fn((code?: number, reason?: string) => { - void code; - void reason; - this.readyState = FakeWebSocket.CLOSING; - }); - - constructor(readonly url: string) { - super(); - sockets.push(this); - } - } - vi.stubGlobal("WebSocket", FakeWebSocket); - vi.stubGlobal("window", { - location: new URL("https://share.test/r/1234"), - setTimeout, - clearTimeout, - }); - const signal = new SignalingClient( - { - roomId: "1234", - role: "viewer", - clientId: "viewer-client", - }, - { - onMessage: () => undefined, - onStatus: () => undefined, - onTerminated: () => undefined, - onAccessRequired: () => undefined, - }, - ); - - signal.start(); - expect(signal.reconnect()).toBe(false); - sockets[0]!.dispatchEvent(new Event("open")); - const authenticated = new Event("message"); - Object.defineProperty(authenticated, "data", { - value: JSON.stringify({ - type: "authenticated", - protocol: "screener-v12", - role: "viewer", - peerId: "viewer_12345678", - roomExpiresAt: null, - maxViewers: 8, - endpointMediaCopyCapacity: 2, - hostOnline: true, - connectionId: null, - viewerPeerIds: [], - iceConfig: { iceServers: [] }, - codeEntryPolicy: "open", - viewerAuthorizationGeneration: "viewer_generation_12345678", - }), - }); - sockets[0]!.dispatchEvent(authenticated); - - expect(signal.reconnect()).toBe(true); - expect(sockets[0]!.close).toHaveBeenCalledWith(4002, "client reconnect"); - expect(signal.reconnect()).toBe(false); - }); - it("keeps a Host paused across reconnect until an explicit Resume is sent", () => { vi.useFakeTimers(); const sockets: FakeWebSocket[] = []; @@ -848,11 +783,10 @@ describe("client signaling recovery policy", () => { }); }; const reconnect = (socket: FakeWebSocket) => { - expect(signal.reconnect()).toBe(true); const close = new Event("close"); Object.defineProperties(close, { - code: { value: 4002 }, - reason: { value: "client reconnect" }, + code: { value: 1006 }, + reason: { value: "network interrupted" }, }); socket.dispatchEvent(close); vi.advanceTimersByTime(750); diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts index 3627ef4..2b8515d 100644 --- a/tests/host-peer-tracks.test.ts +++ b/tests/host-peer-tracks.test.ts @@ -100,6 +100,7 @@ class FakePeerConnection { remoteDescription: RTCSessionDescription | null = null; readonly addedIceCandidates: Array = []; readonly statsReports: Array> = []; + private readonly eventListeners = new Map void>>(); constructor(configuration?: RTCConfiguration) { FakePeerConnection.latest = this; @@ -145,7 +146,18 @@ class FakePeerConnection { } as unknown as RTCRtpTransceiver; } - addEventListener(): void {} + addEventListener(type: string, listener: () => void): void { + const listeners = this.eventListeners.get(type) ?? []; + listeners.push(listener); + this.eventListeners.set(type, listeners); + } + + dispatchEvent(event: Event): boolean { + for (const listener of this.eventListeners.get(event.type) ?? []) { + listener(); + } + return true; + } async createOffer(): Promise { this.createOfferCallCount += 1; @@ -388,6 +400,39 @@ function createPeer( ); } +async function acceptPeerAnswer(peer: HostPeer): Promise { + const connection = FakePeerConnection.latest!; + await peer.acceptSignal({ + kind: "description", + connectionId: peer.connectionId, + description: { type: "answer", sdp: "test-answer" }, + }); + connection.connectionState = "connected"; + connection.dispatchEvent(new Event("connectionstatechange")); + await completeVideoStartup(connection); +} + +async function completeVideoStartup( + connection: FakePeerConnection, + framesEncoded = 5, +): Promise { + const statsCalls = connection.getStats.mock.calls.length; + connection.statsReports.push( + sendStatsReport({ + bytesSent: 10_000, + framesEncoded, + timestamp: 1_000, + qualityLimitationReason: "none", + trackIdentifier: connection.senders[0]?.track?.id ?? "video", + }), + ); + statsCallbacks.at(-1)!(); + await vi.waitFor(() => + expect(connection.getStats).toHaveBeenCalledTimes(statsCalls + 1), + ); + await new Promise((resolve) => setImmediate(resolve)); +} + beforeEach(() => { FakePeerConnection.latest = null; FakePeerConnection.instances = []; @@ -572,6 +617,7 @@ describe("HostPeer source replacement", () => { ); await expect(peer.start()).resolves.toBe(true); + await acceptPeerAnswer(peer); const connection = FakePeerConnection.latest!; expect(connection.transceiverInputs).toHaveLength(2); expect(connection.transceiverInputs.map(({ init }) => init?.direction)).toEqual([ @@ -588,7 +634,7 @@ describe("HostPeer source replacement", () => { expect(connection.senders[0]?.track).toBe(nextVideo); expect(connection.senders[1]?.track).toBe(nextAudio); expect(connection.transceiverInputs).toHaveLength(2); - expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(3); expect(connection.senders[1]?.appliedMaxBitrates).toEqual([ 192_000, 192_000, @@ -613,7 +659,7 @@ describe("HostPeer source replacement", () => { ), ).resolves.toBe(true); expect(connection.senders[1]?.track).toBe(nextAudio); - expect(connection.senders[1]?.appliedMaxBitrates).toEqual([96_000]); + expect(connection.senders[1]?.appliedMaxBitrates).toEqual([128_000]); }); it("stops sending audio without renegotiating when the new source has none", async () => { @@ -633,7 +679,7 @@ describe("HostPeer source replacement", () => { ).resolves.toBe(true); expect(connection.senders[1]?.track).toBeNull(); - expect(connection.senders[1]?.appliedMaxBitrates).toEqual([96_000]); + expect(connection.senders[1]?.appliedMaxBitrates).toEqual([128_000]); }); it("updates quality parameters without replacing media tracks", async () => { @@ -642,6 +688,7 @@ describe("HostPeer source replacement", () => { const peer = createPeer(createStream(video, audio)); await expect(peer.start()).resolves.toBe(true); + await acceptPeerAnswer(peer); const connection = FakePeerConnection.latest!; await expect( peer.updateProfile(QUALITY_PROFILES["1080p60"]), @@ -649,7 +696,7 @@ describe("HostPeer source replacement", () => { expect(connection.senders[0]?.replaceTrack).not.toHaveBeenCalled(); expect(connection.senders[1]?.replaceTrack).not.toHaveBeenCalled(); - expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(3); expect( connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0], ).toMatchObject({ @@ -664,10 +711,10 @@ describe("HostPeer source replacement", () => { screenAudioQuality: "very-high", }), ).resolves.toBe(true); - expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(3); expect(connection.senders[1]?.setParameters).toHaveBeenCalledTimes(2); expect(connection.senders[1]?.appliedMaxBitrates).toEqual([ - 96_000, + 128_000, 192_000, ]); expect(peer.getSnapshot().audioSenderParameters).toEqual({ @@ -695,9 +742,9 @@ describe("HostPeer source replacement", () => { ).resolves.toBe(false); expect(audioSender.track).toBe(audio); - expect(audioSender.getParameters().encodings[0]?.maxBitrate).toBe(96_000); + expect(audioSender.getParameters().encodings[0]?.maxBitrate).toBe(128_000); expect(peer.getSnapshot().audioSenderParameters?.appliedMaxBitrate).toBe( - 96_000, + 128_000, ); expect(peer.getSnapshot().qualityWarning).toContain( "应用音频发送参数失败", @@ -721,9 +768,9 @@ describe("HostPeer source replacement", () => { const audioSender = FakePeerConnection.latest!.senders[1]!; audioSender.deferNextSetParameters = true; - const music = peer.updateProfile({ + const saver = peer.updateProfile({ ...QUALITY_PROFILES["720p30"], - screenAudioQuality: "music", + screenAudioQuality: "saver", }); await vi.waitFor(() => expect(audioSender.setParameters).toHaveBeenCalledTimes(2), @@ -734,11 +781,11 @@ describe("HostPeer source replacement", () => { }); audioSender.releaseDeferredSetParameters(); - await expect(music).resolves.toBe(false); + await expect(saver).resolves.toBe(false); await expect(veryHigh).resolves.toBe(true); expect(audioSender.appliedMaxBitrates).toEqual([ - 96_000, 128_000, + 64_000, 192_000, ]); expect(peer.getSnapshot().audioSenderParameters?.appliedMaxBitrate).toBe( @@ -746,12 +793,16 @@ describe("HostPeer source replacement", () => { ); }); - it("reapplies the selected video profile after accepting an answer", async () => { + it("enables the selected balanced profile after startup frames", async () => { const video = createTrack("video", "video"); const peer = createPeer(createStream(video, createTrack("audio", "audio"))); await expect(peer.start()).resolves.toBe(true); const connection = FakePeerConnection.latest!; + expect(connection.senders[0]?.setParameters).toHaveBeenCalledOnce(); + expect( + connection.senders[0]?.setParameters.mock.calls[0]?.[0], + ).toMatchObject({ degradationPreference: "maintain-resolution" }); const pendingCandidate = { candidate: "candidate-before-answer" }; await peer.acceptSignal({ kind: "candidate", @@ -768,6 +819,17 @@ describe("HostPeer source replacement", () => { expect(connection.remoteDescription?.type).toBe("answer"); expect(connection.addedIceCandidates).toEqual([pendingCandidate]); expect(connection.senders[0]?.track).toBe(video); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledOnce(); + + connection.connectionState = "connected"; + connection.dispatchEvent(new Event("connectionstatechange")); + await Promise.resolve(); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledOnce(); + + await completeVideoStartup(connection, 4); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledOnce(); + + await completeVideoStartup(connection, 5); expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2); expect( connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0], @@ -778,7 +840,7 @@ describe("HostPeer source replacement", () => { expect(connection.senders[1]?.setParameters).toHaveBeenCalledOnce(); }); - it("serializes initial sender configuration with a live profile update", async () => { + it("applies the latest pre-answer profile once negotiation completes", async () => { const peer = createPeer( createStream( createTrack("video", "video"), @@ -786,20 +848,19 @@ describe("HostPeer source replacement", () => { ), ); - const starting = peer.start(); + await expect(peer.start()).resolves.toBe(true); const videoSender = FakePeerConnection.latest!.senders[0]!; - videoSender.deferNextSetParameters = true; + expect(videoSender.setParameters).toHaveBeenCalledOnce(); + + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(true); + expect(videoSender.setParameters).toHaveBeenCalledOnce(); + + await acceptPeerAnswer(peer); await vi.waitFor(() => - expect(videoSender.setParameters).toHaveBeenCalledTimes(1), + expect(videoSender.setParameters).toHaveBeenCalledTimes(2), ); - - const updating = peer.updateProfile(QUALITY_PROFILES["1080p60"]); - await Promise.resolve(); - expect(videoSender.setParameters).toHaveBeenCalledTimes(1); - - videoSender.releaseDeferredSetParameters(); - await expect(starting).resolves.toBe(true); - await expect(updating).resolves.toBe(true); expect(videoSender.appliedMaxBitrates).toEqual([3_000_000, 8_000_000]); }); @@ -816,17 +877,19 @@ describe("HostPeer source replacement", () => { const starting = peer.start(); const videoSender = FakePeerConnection.latest!.senders[0]!; videoSender.failNextSetParameters = true; - const updating = peer.updateProfile(QUALITY_PROFILES["1080p60"]); - await expect(starting).resolves.toBe(true); + await acceptPeerAnswer(peer); + const failureWarning = updates.find((snapshot) => snapshot.qualityWarning?.startsWith("应用发送参数失败"), )?.qualityWarning; expect(failureWarning).toBe("应用发送参数失败"); expect(failureWarning).not.toContain("setParameters failed"); - await expect(updating).resolves.toBe(true); - expect(videoSender.setParameters).toHaveBeenCalledTimes(2); - expect(videoSender.appliedMaxBitrates).toEqual([8_000_000]); + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(true); + expect(videoSender.setParameters).toHaveBeenCalledTimes(3); + expect(videoSender.appliedMaxBitrates).toEqual([3_000_000, 8_000_000]); }); it("can retry the selected quality after a sender update fails", async () => { @@ -838,6 +901,7 @@ describe("HostPeer source replacement", () => { ); await expect(peer.start()).resolves.toBe(true); + await acceptPeerAnswer(peer); const videoSender = FakePeerConnection.latest!.senders[0]!; videoSender.failNextSetParameters = true; @@ -848,7 +912,7 @@ describe("HostPeer source replacement", () => { peer.updateProfile(QUALITY_PROFILES["1080p60"]), ).resolves.toBe(true); - expect(videoSender.setParameters).toHaveBeenCalledTimes(3); + expect(videoSender.setParameters).toHaveBeenCalledTimes(4); expect(videoSender.setParameters.mock.calls.at(-1)?.[0]).toMatchObject({ degradationPreference: "balanced", encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }], @@ -867,12 +931,13 @@ describe("HostPeer source replacement", () => { ); await expect(peer.start()).resolves.toBe(true); + await acceptPeerAnswer(peer); const connection = FakePeerConnection.latest!; connection.statsReports.push(oldStats); statsCallbacks[0]!(); await vi.waitFor(() => expect(connection.statsReports).toHaveLength(0)); statsCallbacks[0]!(); - expect(connection.getStats).toHaveBeenCalledOnce(); + expect(connection.getStats).toHaveBeenCalledTimes(2); await expect( peer.updateProfile(QUALITY_PROFILES["1080p60"]), @@ -1181,6 +1246,7 @@ describe("HostPeer source replacement", () => { (snapshot) => updates.push(snapshot), ); await expect(peer.start()).resolves.toBe(true); + await acceptPeerAnswer(peer); const connection = FakePeerConnection.latest!; const sender = connection.senders[0]!; connection.statsReports.push( @@ -1220,11 +1286,11 @@ describe("HostPeer source replacement", () => { expect(updates.at(-1)?.qualityWarning).toBeNull(); await sample(); expect(updates.at(-1)?.qualityWarning).toContain("持续受带宽限制"); - expect(sender.setParameters).toHaveBeenCalledOnce(); + expect(sender.setParameters).toHaveBeenCalledTimes(2); await sample(); expect(updates.at(-1)?.qualityWarning).toBeNull(); - expect(sender.setParameters).toHaveBeenCalledOnce(); + expect(sender.setParameters).toHaveBeenCalledTimes(2); }); it("does not expose an unknown browser quality-limitation value", async () => { @@ -1930,11 +1996,24 @@ describe("ViewerRelay downstream ownership", () => { relay.setChildren(["first-profile-child"]); relay.setStream(createStream(createTrack("video", "profile-video"), null)); await vi.waitFor(() => - expect( - FakePeerConnection.latest?.senders[0]?.setParameters, - ).toHaveBeenCalled(), + expect(relay.getSnapshot("first-profile-child")).not.toBeNull(), ); + const firstConnectionId = relay.getSnapshot("first-profile-child")!.connectionId; const firstConnection = FakePeerConnection.latest!; + await expect( + relay.acceptSignal( + "first-profile-child", + { + kind: "description", + connectionId: firstConnectionId, + description: { type: "answer", sdp: "first-answer" }, + }, + 0, + ), + ).resolves.toBe(true); + firstConnection.connectionState = "connected"; + firstConnection.dispatchEvent(new Event("connectionstatechange")); + await completeVideoStartup(firstConnection); await expect( relay.updateProfile(customSettings), @@ -1952,8 +2031,26 @@ describe("ViewerRelay downstream ownership", () => { ); const secondConnection = FakePeerConnection.latest!; await vi.waitFor(() => - expect(secondConnection.senders[0]?.setParameters).toHaveBeenCalled(), + expect(secondConnection.localDescription?.type).toBe("offer"), ); + await vi.waitFor(() => + expect(relay.getSnapshot("second-profile-child")).not.toBeNull(), + ); + const secondConnectionId = relay.getSnapshot("second-profile-child")!.connectionId; + await expect( + relay.acceptSignal( + "second-profile-child", + { + kind: "description", + connectionId: secondConnectionId, + description: { type: "answer", sdp: "second-answer" }, + }, + 0, + ), + ).resolves.toBe(true); + secondConnection.connectionState = "connected"; + secondConnection.dispatchEvent(new Event("connectionstatechange")); + await completeVideoStartup(secondConnection); expect( secondConnection.senders[0]?.setParameters.mock.calls[0]?.[0], ).toMatchObject({ @@ -1981,7 +2078,7 @@ describe("ViewerRelay downstream ownership", () => { ); await vi.waitFor(() => expect(FakePeerConnection.latest?.senders[1]?.appliedMaxBitrates).toEqual([ - 96_000, + 64_000, ]), ); const firstConnection = FakePeerConnection.latest!; @@ -1993,7 +2090,7 @@ describe("ViewerRelay downstream ownership", () => { }), ).resolves.toBe(true); expect(firstConnection.senders[1]?.appliedMaxBitrates).toEqual([ - 96_000, + 64_000, 192_000, ]); diff --git a/tests/hybrid-media-router.test.ts b/tests/hybrid-media-router.test.ts index 9f76cda..ee5171e 100644 --- a/tests/hybrid-media-router.test.ts +++ b/tests/hybrid-media-router.test.ts @@ -749,6 +749,31 @@ describe("HybridMediaRouter v9 runtime", () => { } }); + it("serializes fresh SFU configuration without making refresh one-shot", async () => { + const { store, sent, router } = harness(1, true); + try { + const room = await store.createRoom(); + const { first } = await establishSfuRoom(store, sent, router, room); + const active = router.resolveActiveViewerMediaEdge( + room.roomId, + first.peerId, + ); + expect(active?.upstream).toEqual({ kind: "sfu" }); + const configCount = () => + sent + .get(first.sessionId) + ?.filter((message) => message.type === "sfu-config").length ?? 0; + const initialCount = configCount(); + + router.refreshSfu(first, active!.revision); + await vi.waitFor(() => expect(configCount()).toBe(initialCount + 1)); + router.refreshSfu(first, active!.revision); + await vi.waitFor(() => expect(configCount()).toBe(initialCount + 2)); + } finally { + await router.close(); + } + }); + diff --git a/tests/quality.test.ts b/tests/quality.test.ts index 79abde6..d20187a 100644 --- a/tests/quality.test.ts +++ b/tests/quality.test.ts @@ -7,12 +7,14 @@ import { configureScreenAudioSender, configureVideoSender, matchingQualityProfileId, + needsStartupVideoProfile, QUALITY_PROFILES, QUALITY_RESOLUTIONS, qualitySettingsEqual, SCREEN_AUDIO_BITRATES, senderParameterWarning, setMediaPaused, + startupVideoProfile, } from "../src/client/media/quality.ts"; function createVideoStream() { @@ -109,26 +111,39 @@ describe("realtime quality controls", () => { }); }); - it("defaults 1080p30 to clarity without changing the other presets", () => { - expect(QUALITY_PROFILES["1080p30"].degradationPreference).toBe( - "maintain-resolution", - ); + it("defaults every recommended profile to balanced", () => { + expect(QUALITY_PROFILES["1080p30"].degradationPreference).toBe("balanced"); expect(QUALITY_PROFILES["1080p60"].degradationPreference).toBe("balanced"); expect(QUALITY_PROFILES["720p30"].degradationPreference).toBe("balanced"); }); - it("keeps audio selection orthogonal while defaulting old settings to saver", () => { - const music = { + it("holds resolution only while a scaling preference starts", () => { + expect(needsStartupVideoProfile(QUALITY_PROFILES["1080p30"])).toBe(true); + expect(startupVideoProfile(QUALITY_PROFILES["1080p30"])).toMatchObject({ + degradationPreference: "maintain-resolution", + maxBitrate: 5_000_000, + maxFramerate: 30, + }); + const clarity = { + ...QUALITY_PROFILES["1080p30"], + degradationPreference: "maintain-resolution", + } as const; + expect(needsStartupVideoProfile(clarity)).toBe(false); + expect(startupVideoProfile(clarity)).toBe(clarity); + }); + + it("keeps audio selection orthogonal while defaulting missing settings to music", () => { + const saver = { ...QUALITY_PROFILES["1080p60"], - screenAudioQuality: "music", + screenAudioQuality: "saver", } as const; const { screenAudioQuality: _screenAudioQuality, ...legacySettings } = QUALITY_PROFILES["1080p60"]; - expect(matchingQualityProfileId(music)).toBe("1080p60"); - expect(qualitySettingsEqual(music, QUALITY_PROFILES["1080p60"])).toBe(false); + expect(matchingQualityProfileId(saver)).toBe("1080p60"); + expect(qualitySettingsEqual(saver, QUALITY_PROFILES["1080p60"])).toBe(false); expect( qualitySettingsEqual(legacySettings, QUALITY_PROFILES["1080p60"]), ).toBe(true); @@ -224,7 +239,7 @@ describe("realtime quality controls", () => { }); it.each([ - ["saver", 96_000], + ["saver", 64_000], ["music", 128_000], ["very-high", 192_000], ] as const)("applies and reads back the %s audio ceiling", async (quality, bitrate) => { @@ -245,7 +260,7 @@ describe("realtime quality controls", () => { expect(SCREEN_AUDIO_BITRATES[quality]).toBe(bitrate); }); - it("uses the saver ceiling for a setting without an audio preset", async () => { + it("uses the music ceiling for a setting without an audio preset", async () => { let applied = { encodings: [] } as unknown as RTCRtpSendParameters; const sender = { getParameters: () => applied, @@ -255,8 +270,8 @@ describe("realtime quality controls", () => { } as unknown as RTCRtpSender; await expect(configureScreenAudioSender(sender)).resolves.toEqual({ - requestedMaxBitrate: 96_000, - appliedMaxBitrate: 96_000, + requestedMaxBitrate: 128_000, + appliedMaxBitrate: 128_000, mismatch: false, }); }); @@ -286,7 +301,7 @@ describe("realtime quality controls", () => { it("does not mutate prior audio parameters when setParameters fails", async () => { const applied = { - encodings: [{ maxBitrate: 96_000 }], + encodings: [{ maxBitrate: 128_000 }], } as RTCRtpSendParameters; const sender = { getParameters: () => applied, @@ -298,7 +313,7 @@ describe("realtime quality controls", () => { await expect( configureScreenAudioSender(sender, "very-high"), ).rejects.toThrow("rejected"); - expect(applied.encodings[0]?.maxBitrate).toBe(96_000); + expect(applied.encodings[0]?.maxBitrate).toBe(128_000); }); it("reports fields the browser does not retain", async () => { diff --git a/tests/route-transition.test.ts b/tests/route-transition.test.ts index bd7923e..31afb92 100644 --- a/tests/route-transition.test.ts +++ b/tests/route-transition.test.ts @@ -501,6 +501,65 @@ describe("minimal route transition contracts", () => { expect(subscribers[0]?.disconnect).toHaveBeenCalledOnce(); }); + it("reconnects only the active SFU subscriber on the current route", async () => { + const messages: ClientMessage[] = []; + const streams: MediaStream[] = []; + const subscribers: ReturnType[] = []; + const resetMedia = vi.fn(); + const route = new ViewerSfuRoute("viewer_12345678", { + activatePeer: () => true, + resetMedia, + reconcileSfuChildren: () => undefined, + onSfuStream: (stream) => streams.push(stream), + send: (message) => { + messages.push(message); + return true; + }, + createSubscriber: (events) => { + const subscriber = createFakeSubscriber( + events, + [], + `subscriber-${subscribers.length + 1}`, + ); + subscribers.push(subscriber); + return subscriber; + }, + }); + + route.accept({ + revision: 7, + phase: "active", + assignment: viewerSfuAssignment(), + }); + await route.acceptConfig(sfuConfig(7)); + const firstStream = {} as MediaStream; + subscribers[0]?.events.onStream(firstStream); + subscribers[0]?.events.onFirstDecodedFrame(); + await vi.waitFor(() => expect(streams).toEqual([firstStream])); + + expect(route.reconnectActive()).toBe(true); + expect(route.reconnectActive()).toBe(false); + await vi.waitFor(() => + expect(messages.at(-1)).toEqual({ type: "refresh-sfu", revision: 7 }), + ); + expect(resetMedia).not.toHaveBeenCalled(); + expect(subscribers[0]?.deactivate).not.toHaveBeenCalled(); + expect(subscribers[0]?.disconnect).not.toHaveBeenCalled(); + + await route.acceptConfig({ ...sfuConfig(7), token: "fresh-token" }); + const nextStream = {} as MediaStream; + subscribers[1]?.events.onStream(nextStream); + subscribers[1]?.events.onFirstDecodedFrame(); + await vi.waitFor(() => expect(streams).toEqual([firstStream, nextStream])); + expect(subscribers[1]?.connect).toHaveBeenCalledWith({ + url: "wss://sfu.example.test", + token: "fresh-token", + }); + expect(subscribers[0]?.deactivate).toHaveBeenCalledOnce(); + expect(subscribers[0]?.disconnect).toHaveBeenCalledOnce(); + await route.disconnect(); + }); + it("discards a paused pending subscriber and ignores its later evidence", async () => { const messages: ClientMessage[] = []; let subscriber!: ReturnType; diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts index 35e8ea1..592551e 100644 --- a/tests/sfu-client.test.ts +++ b/tests/sfu-client.test.ts @@ -248,7 +248,6 @@ const livekit = vi.hoisted(() => { const AudioPresets = { musicStereo: { maxBitrate: 64_000 }, - musicHighQuality: { maxBitrate: 96_000 }, musicHighQualityStereo: { maxBitrate: 128_000 }, } as const; @@ -481,6 +480,40 @@ afterEach(() => { }); describe("SfuPublisher", () => { + it("enables balanced SFU adaptation after startup frames", async () => { + vi.useFakeTimers(); + const publisher = new SfuPublisher(); + const video = track("video", "startup-video"); + const balancedProfile = { + ...qualityProfile, + degradationPreference: "balanced", + } as const; + + await publisher.connect(connection); + await publisher.activate(stream(video), balancedProfile); + const localTrack = livekit.state.rooms[0].localParticipant.publications[0] + .track; + const sender = localTrack.sender; + expect(localTrack.savedDegradationPreference).toBe("maintain-resolution"); + expect(sender.parameters.degradationPreference).toBe( + "maintain-resolution", + ); + + sender.getStats + .mockResolvedValueOnce(senderReport(video.id, 1_000, 10_000, 4)) + .mockResolvedValueOnce(senderReport(video.id, 2_000, 20_000, 5)); + await vi.advanceTimersByTimeAsync(2_000); + expect(localTrack.savedDegradationPreference).toBe("maintain-resolution"); + expect(sender.setParameters).toHaveBeenCalledOnce(); + + await vi.advanceTimersByTimeAsync(2_000); + await vi.waitFor(() => + expect(localTrack.savedDegradationPreference).toBe("balanced"), + ); + expect(sender.parameters.degradationPreference).toBe("balanced"); + expect(sender.setParameters).toHaveBeenCalledTimes(2); + }); + it("samples only the current published sender and clears replaced evidence", async () => { vi.useFakeTimers(); const updates: Array = []; @@ -675,7 +708,7 @@ describe("SfuPublisher", () => { ).not.toHaveProperty("simulcast"); expect(room.localParticipant.publishTrack).toHaveBeenNthCalledWith(2, audio, { source: Track.Source.ScreenShareAudio, - audioPreset: { maxBitrate: 96_000 }, + audioPreset: { maxBitrate: 128_000 }, forceStereo: true, dtx: false, red: false, @@ -713,7 +746,7 @@ describe("SfuPublisher", () => { }); it.each([ - ["saver", 96_000], + ["saver", 64_000], ["music", 128_000], ["very-high", 192_000], ] as const)("maps the %s audio preset to %i bps", async (screenAudioQuality, bitrate) => { diff --git a/tests/viewer-peer-generation.test.ts b/tests/viewer-peer-generation.test.ts index f851a49..f5c57bb 100644 --- a/tests/viewer-peer-generation.test.ts +++ b/tests/viewer-peer-generation.test.ts @@ -582,7 +582,7 @@ describe("ViewerPeer connection generations", () => { expect([...timeoutDelays.values()]).toEqual([3_000]); }); - it("targets answers and recovery requests at the current parent", async () => { + it("serializes recovery requests at the current parent", async () => { const signals: SignalPayload[] = []; const signalPeers: string[] = []; const restartRequests: Array<{ @@ -611,21 +611,22 @@ describe("ViewerPeer connection generations", () => { await peer.acceptSignal("parent-new", offer("connection-new")); expect(signalPeers).toEqual(["parent-old", "parent-new"]); - expect(peer.requestRecovery()).toBe(true); + expect(peer.requestRecovery(true)).toBe(true); + expect(peer.requestRecovery(true)).toBe(false); expect(restartRequests).toEqual([ { peerId: "parent-new", connectionId: "connection-new", - rebuild: false, + rebuild: true, }, ]); + const connection = FakePeerConnection.instances.at(-1)!; + connection.connectionState = "connected"; + [...timeoutCallbacks.values()].at(-1)!(); expect(peer.requestRecovery(true)).toBe(true); - expect(restartRequests.at(-1)).toEqual({ - peerId: "parent-new", - connectionId: "connection-new", - rebuild: true, - }); + expect(restartRequests).toHaveLength(2); + peer.dispose(); }); it("does not send an old answer or candidate after replacing the connection", async () => { diff --git a/tests/viewer-presentation.test.ts b/tests/viewer-presentation.test.ts index 2b58055..fced1e2 100644 --- a/tests/viewer-presentation.test.ts +++ b/tests/viewer-presentation.test.ts @@ -124,6 +124,38 @@ describe("Viewer presentation reducer", () => { expect(nextRoute.connection).toBe("connecting"); }); + it("rebases proven media when only the room graph revision changes", () => { + const playing = apply( + { type: "access", access: "ready" }, + { type: "signal", signal: "connected" }, + { type: "host", host: "online" }, + { type: "route", revision: 2, phase: "active", kind: "p2p" }, + { type: "connection", revision: 2, connection: "connected" }, + { type: "media-bound", generation: 4, revision: 2 }, + { type: "frame-presented", generation: 4, revision: 2 }, + ); + const rebased = reduceViewerPresentation(playing, { + type: "route", + revision: 3, + phase: "active", + kind: "p2p", + preserveMedia: true, + }); + + expect(rebased.media).toEqual({ + generation: 4, + revision: 3, + framePresented: true, + }); + expect(rebased.connection).toBe("connected"); + expect(deriveViewerPresentation(rebased)).toMatchObject({ + stage: "playing", + overlay: "none", + hasCurrentFrame: true, + hasRetainedFrame: false, + }); + }); + it("ignores a late playback failure from a replaced media binding", () => { const rebound = apply( { type: "access", access: "ready" },