diff --git a/docs/README.md b/docs/README.md index 7e4dbb4..0fde0e3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,10 +8,10 @@ - [Maintenance guide](./maintenance.md): Git workflow, context hygiene, document lifecycle, research policy, and automation. - [P2P WebRTC research](./research/webrtc-p2p-screen-sharing.md): evidence, bandwidth model, browser constraints, reference implementations, and feasibility assessment. - [Peer-assisted media research](./research/peer-assisted-media.md): browser shared-encode limits, standard relay re-encoding, deterministic sticky topology, runtime relay-capacity extension, and abandon gates. -- [Low-server-cost media routes](./research/low-server-media-routes.md): traffic accounting, route screening, token-free SFU standby prewarm, conditional native encoded-RTP relay, and automatic final fallback. +- [Low-server-cost media routes](./research/low-server-media-routes.md): per-edge TURN versus SFU-root topology, hop-level traffic accounting, privacy-safe ICE evidence, dual-TURN shadow gates, and failure-only final fallback. - [Advanced peer distribution](./research/advanced-peer-distribution.md): multi-tree/SVC, encoded-object relay, native RTP forwarding, FEC/network-coding, MoQ, and measurable go/no-go gates. - [Native shared-encode sender](./research/native-shared-encode-sender.md): consolidated Draft #16/#18/#22/#23/#25/#28 evidence ladder, current Pion/WebCodecs verdicts, and product stop line. -- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): correlated capture/send/receive diagnosis, demand-driven dual representations, SVC boundary, and measurement gates. +- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): correlated capture/send/receive diagnosis, production generation-rebuild gate, demand-driven dual representations, and bounded simulcast/Dynacast/SVC checks. - [Browser screen-audio quality](./research/browser-screen-audio-quality.md): capture/source compatibility, standard readback limits, Opus negotiation boundaries, and the retained no-control decision. - [Agent context governance research](./research/agent-context-governance.md): official Codex, Claude Code, Hermes Agent, and GitHub practices adopted by this repository. - [ADR-0001](./adr/0001-p2p-first-media-topology.md): accepted P2P-first media topology and its consequences. diff --git a/docs/adr/0005-automatic-hybrid-media-routing.md b/docs/adr/0005-automatic-hybrid-media-routing.md index e0b68bc..8aaecc4 100644 --- a/docs/adr/0005-automatic-hybrid-media-routing.md +++ b/docs/adr/0005-automatic-hybrid-media-routing.md @@ -23,6 +23,14 @@ TURN is not a separate room topology. The host and every future native/encoded relay have a downstream budget of at most two active media edges. The current browser relay remains stricter at one child. +TURN and SFU occupy different layers. TURN replaces transport for only the ICE +edge that selected a relay pair. An SFU is a virtual topology parent and may +feed only one or two necessary roots; after media reaches a reliable root, the +existing bounded peer subtree remains the preferred distribution path. A +zero-descendant SFU root is allowed only when no relay root satisfies +capability, depth, path, and recovery gates; it still counts against the same +one-or-two-root room budget. + ADR-0004 implements automatic peer assignment but not cross-mode fallback. Closed PR #12 proposed a mutually exclusive process-wide `p2p|sfu` mode; this ADR and merged PR #17 supersede that model because it cannot satisfy the route @@ -42,9 +50,11 @@ fallback: host -> remaining peer root -> ... host -> SFU -> selected fallback roots -> existing descendants ``` -Only fallback roots subscribe directly to the SFU. Each root can continue to -feed its existing peer child, so SFU egress is approximately `K*B`, where `K` -is the number of SFU roots, rather than necessarily `N*B` for all viewers. +Only one or two fallback roots subscribe directly to the SFU, and each can +continue to feed its bounded peer descendants. If no reliable relay root +exists, a necessary viewer may be an SFU root with no descendants under that +same total; this is the last central-fanout boundary, not a default whole-room +topology. The cost model is owned by the linked low-server research. Deployment decides whether SFU capacity exists by configuring the complete LiveKit endpoint/key/secret tuple. There is no user-facing topology selector and @@ -71,9 +81,12 @@ percentage framework, or second router. The implementation keeps the LiveKit dependency dormant unless the complete URL, API key, and API secret tuple is present together with `PEER_ASSISTED_MEDIA=true`. It issues short-lived room-, role-, peer-, and publication-generation-bound grants, -and only allowlisted branch roots may subscribe. The server first retries a -failed edge through the deterministic peer topology; only an exhausted peer -route can request an SFU branch root. +and only allowlisted branch roots may subscribe. A necessary viewer with no +descendants is still a root under the same authorization and total cap. The +server first retries a failed edge through the deterministic peer topology; +only an exhausted peer route can request an SFU branch root. A +healthy route does not migrate merely because one or both selected edge +transports use TURN. Every viewer starts with zero relay capacity for each authenticated session. A peer-assisted Web client explicitly advertises either zero or one downstream @@ -251,13 +264,39 @@ Sub-second failure recovery is a target after failure detection. The current plane needs a small 100-200 ms liveness/queue signal or an equivalent native transport event. This signal must be measured before its interval is fixed. +## Shadow-Only Dual-TURN Candidate + +Four cases define the decision boundary: retain direct/direct and direct/TURN +peer roots; treat stable TURN/TURN roots only as a shadow SFU-root candidate; +and use the current failure-only SFU path only after no reliable relay root +remains. A necessary zero-descendant viewer still consumes one of the same one +or two SFU-root slots. No stability threshold or new trigger is accepted. + +The bounded cost model, privacy-safe ICE fields, four-case measurement matrix, +and exact-room shadow/A/B sequence live in +[Low-Server-Cost Media Routes](../research/low-server-media-routes.md). One +important accounting invariant is retained here: host-to-SFU publisher, +SFU-to-root subscriber, and peer-descendant edges are independent ICE +connections and each may use TURN. Every host NIC, TURN ingress/egress, SFU +ingress/egress, and root NIC hop is real traffic and remains in service and +billing totals, even when the same logical payload traverses consecutive hops. + +Only measured benefit plus an amended ADR-0005 may authorize an automatic +exact-room canary. Until then, shadow observation changes no route, wire, +controller, or production trigger, and the implementation remains +failure-only. + ## Security And Privacy The standby URL is an origin, not a credential. LiveKit credentials remain short-lived, room-bound, role-bound, and memory-only. Only the host may publish screen tracks. Subscription permissions allow only -the current fallback-root identities. Ordinary SFU media is not end-to-end -encrypted from the SFU operator; the deployment and UI must not claim otherwise. +the current fallback-root identities, including any zero-descendant roots. +Ordinary SFU WebRTC transport encryption terminates at the SFU, so its operator +can access media. LiveKit supports application E2EE in which its server cannot +access media content, but signaling/API data remains visible and Screener has +not implemented the required key distribution. The shadow comparison must +record which boundary is actually configured and the UI must not claim E2EE. ## Implementation Order @@ -280,6 +319,9 @@ encrypted from the SFU operator; the deployment and UI must not claim otherwise. only by a proven native/encoded capability. - Only necessary fallback roots receive SFU media and the configured root/egress budget is never exceeded. +- A reliable SFU root continues to serve bounded peer descendants; only when no + reliable relay root exists may a necessary viewer be a zero-descendant root, + still within the same one-or-two-root cap. - Prepare failure leaves the old active route unchanged. - First new decodable picture arrives within one second after a route failure is detected in the reference regional network. @@ -334,4 +376,7 @@ Negative: - [LiveKit track subscription permissions](https://docs.livekit.io/transport/media/publish/#track-permissions) - [LiveKit client 2.22.0 `prepareConnection` source](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts) - [LiveKit JavaScript client usage](https://github.com/livekit/client-sdk-js#usage) +- [LiveKit end-to-end encryption](https://docs.livekit.io/transport/encryption/) +- [TURN, RFC 8656](https://www.rfc-editor.org/rfc/rfc8656.html) +- [WebRTC statistics](https://www.w3.org/TR/webrtc-stats/) - [WebRTC](https://w3c.github.io/webrtc-pc/) diff --git a/docs/adr/0007-demand-driven-dual-representation-quality.md b/docs/adr/0007-demand-driven-dual-representation-quality.md index b147f7a..0f8be49 100644 --- a/docs/adr/0007-demand-driven-dual-representation-quality.md +++ b/docs/adr/0007-demand-driven-dual-representation-quality.md @@ -104,6 +104,35 @@ selection. The existing mobile/iPad heuristic remains limited to conservative relay-capacity admission. Browser capability queries can guide a bounded probe, but runtime encode/decode behavior is authoritative. +## Deferred Capability Spikes + +These spikes start only after the A+B/C evidence contract and the native +fixed-`HIGH` path pass. They do not change the accepted two-state controller: + +1. Negotiate exactly two `HIGH`/`LOW` simulcast encodings on one sender in its + initial envelope, with `LOW` inactive. Verify requested/applied parameters, + per-RID traffic, restart behavior, and actual encoder/CPU/GPU release when + `LOW` is inactive. W3C `active=false` stops that encoding from being sent; + it does not guarantee that a physical encoder or GPU resource is released. + Separate direct PeerConnections have no portable shared-encode contract. +2. On the optional SFU path, publish at most two LiveKit simulcast + representations and let each one or two roots select independently. Treat + Dynacast as a bounded rejection/verification spike, not an assumed fit. The + pinned server 1.13.5 aggregates the maximum quality requested across all + subscribers/nodes and marks every quality `q <= maxQuality` enabled; client + 2.22.0 applies those flags to simulcast encoding `active`. Thus any `HIGH` + root is expected to keep `LOW` enabled. Firefox disabling is only a roughly + 10 bps, 2 fps, 4x-scale compatibility reduction. Accept this path only if + actual per-RID bytes/frames plus host CPU/GPU/encoder evidence prove that + `LOW` stops while `HIGH` continues; otherwise reject it for the exact + on-demand-`LOW` requirement. +3. Only for the strict-one-output requirement, request SVC and compare the + applied codec/`scalabilityMode` with Media Capabilities `powerEfficient` and + the game-performance matrix. Reject silent software fallback. + +None may reuse PR #28's minimum-of-two target, bypass a per-edge stock WebRTC +congestion controller, or expand the representation limit beyond two. + ## SVC Boundary Consider SVC only when a later accepted requirement demands one encoded output @@ -142,6 +171,8 @@ Negative: ## Product Stop Lines - Do not productize PR #28's minimum-of-two bitrate or frame-rate aggregation. +- Do not use simulcast, Dynacast, SVC, or an SFU as a semantic bypass around + PR #28's stock-GCC/RTX stop line or per-edge congestion control. - Do not lower `HIGH` because one viewer reports trouble. - Do not create one representation or encoder per viewer. - Do not use UA/device identity as a quality signal. @@ -175,3 +206,8 @@ Primary sources checked 2026-08-19: - [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/) - [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/) - [W3C WebCodecs](https://www.w3.org/TR/webcodecs/) +- [W3C WebRTC simulcast](https://www.w3.org/TR/webrtc/#simulcast-functionality) +- [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/) +- [LiveKit client 2.22.0 Dynacast layer control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalVideoTrack.ts) +- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go) +- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go) diff --git a/docs/project-memory.md b/docs/project-memory.md index 7e6ab31..61ea8ba 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -7,8 +7,8 @@ Last updated: 2026-08-19 - Build low-latency game screen sharing for one broadcaster and a small trusted group; public or large broadcasts belong on OBS/Twitch-class services. - Viewers join from normal desktop/mobile browsers. A later native sender may share encoding and improve capture/audio without changing the Web viewer requirement. - Use a small central service for access, rooms, signaling, deterministic topology, STUN/TURN, and observability while clients carry media whenever practical. -- Minimize server bandwidth. Automatic, viewer-transparent routing follows direct P2P, measured peer assistance, then enabled user-operated/central SFU fallback under deterministic budgets; earlier anti-migration text is superseded. -- Host and relay targets are at most two downstream edges; each independently uses direct ICE or authenticated TURN. A long-term native/object path should reuse one encode, while the browser experiment has one viewer child and re-encodes. +- Minimize server bandwidth. Automatic, viewer-transparent routing follows direct P2P, measured peer assistance, then enabled user-operated/central SFU fallback under deterministic budgets; TURN is transport chosen per edge, not a topology. +- Host/native roots have at most two downstream edges; browser relays have one. An SFU is only a virtual parent for one or two necessary roots, which retain bounded peer descendants; a zero-descendant viewer consumes the same root cap only when no reliable relay root exists. A long-term native/object path should reuse one encode, while browsers re-encode. - Two-tree packet/layer striping may reduce endpoint upload toward one stream bitrate, but needs a bounded multi-parent, loss, sync, churn, and latency experiment; it is not in the current full-stream chains. - Shared encoding reduces compute, not each viewer's last-hop copy. Automatic quality uses shared `HIGH` plus exactly one on-demand `LOW` whenever verified weak paths exist; all weak paths share it and recovery stops it. An unavailable efficient second encoder fails closed for that attempt while protecting `HIGH`, but a supported sender cohort that cannot start `LOW` reliably fails controller acceptance and must be optimized or marked unsupported. SVC has no software fallback. - Keep decisions, snapshots, research, code, `AGENTS.md`, and `.codex/` in Git; rewrite memory/status in place. Research current primary sources before material work and reject speculative machinery. @@ -20,7 +20,7 @@ Last updated: 2026-08-19 - Browser relays resend remote `MediaStreamTrack` values and re-encode at each hop; WebRTC does not guarantee a shared encoder across peer connections, so measure the cost. - Keep browser experiments bounded; spike native RTP relay, encoded-object striping, SVC, and FEC separately. - Plan a separate native-sender ADR regardless of browser relay results. One encode may feed at most two WebRTC packetizers, reducing encode work but not per-edge upload; it cannot rescue a failed browser gate. -- Optional single-node SFU capacity is final fallback only; operator egress does not justify Redis/multi-node infrastructure. PR #12's whole-room SFU mode is superseded. +- Optional single-node SFU capacity is failure-only final fallback; it never defaults to whole-room fanout. Stable dual-TURN host roots are only a future exact-room shadow comparison against the same SFU roots, with real hop/egress and E2EE/operator accounting, until evidence amends ADR-0005. PR #12's whole-room SFU mode is superseded. - Keep ADR-0005 routing default-off: discrete failures drive peer/SFU recovery, full host fanout uses break-before-make, and mobile/iPad clients stay leaves. PR #20 prewarm passes locally; public transport/load gates remain. - Local reparenting is a later candidate: start with unassigned relay admission rescue; do not block current work. - Prefer direct UDP, then TURN/UDP, with TURN/TCP required; optional TURN/TLS defaults to 5349, while 443 needs a dedicated address or validated L4/SNI routing. diff --git a/docs/research/advanced-peer-distribution.md b/docs/research/advanced-peer-distribution.md index 488c395..542fb25 100644 --- a/docs/research/advanced-peer-distribution.md +++ b/docs/research/advanced-peer-distribution.md @@ -30,16 +30,18 @@ experience automatic, and meets the latency and queue limits below. Let `B` be one complete stream bitrate, `N` the viewer count, and `r` repair overhead. Protocol headers, retransmission, and TURN overhead are additional. -| Route | Host upload | Central egress | Encode work | +| Route | Host upload | Central media traffic | Encode work | | --- | ---: | ---: | --- | | Current full-stream browser chains | at most `2B` | zero except TURN edges | host up to two encoders; every relay encodes again | | Native full-stream RTP relay | at most `2B` | zero except TURN edges | relay zero encode; native host can share one encode | | Two encoded-object stripe trees | about `(1+r)B` | zero except TURN edges | host one encode; relay zero encode | -| Central SFU or MoQ relay | about `B` | about `N*B` | host one encode; server forwards without transcoding | +| SFU virtual parent to bounded roots | about `B` | root-only; see low-server model | host one publication; roots keep peer descendants | +| Full central SFU/MoQ fanout | about `B` | ingress `B`, egress `N*B` | comparison class, not the accepted fallback shape | Useful last-hop traffic remains approximately `N*B`; these routes only decide -which nodes emit the copies. Every P2P edge still runs its own ICE process and -may independently select direct connectivity or TURN. +which nodes emit the copies. Exact `B_pub`/`B_i`, root, TURN-hop, and billing +accounting is owned by [Low-Server-Cost Media Routes](./low-server-media-routes.md). +TURN is edge transport; it is not a peer/SFU topology. ## Candidate 1: Native RTP Relay @@ -233,9 +235,21 @@ healthy TURN-to-direct optimization. ## Automatic Route Controller Boundary -The product order is: +The accepted product order is: -`direct -> peer full/native -> peer striped-object -> enabled SFU root -> enabled central fanout -> wait/fail` +`direct host P2P -> accepted peer-assisted capability -> enabled SFU roots -> wait/fail` + +Native full-stream relay and striped-object routes remain the bounded candidates +above; passing their own gates may change capability, but does not insert them +into the current product ladder. + +The enabled SFU is a virtual parent for at most `R=2` roots. Roots retain +bounded peer descendants; only if no reliable relay root exists may a necessary +viewer consume one of the same root slots with zero descendants. There is no +second whole-room central-fanout budget. The detailed direct/direct, +direct/TURN, TURN/TURN shadow, and no-root matrix lives in +[Low-Server-Cost Media Routes](./low-server-media-routes.md); it does not alter +the sticky local-reparenting candidate above or ADR-0005's failure-only trigger. The controller must be automatic and invisible. It uses explicit capability bits, route revisions, media generations, edge budgets, and discrete failures; @@ -278,6 +292,8 @@ Sources checked on 2026-08-19: [ICE, RFC 8445](https://www.rfc-editor.org/rfc/rfc8445.html), and [LiveKit SFU](https://docs.livekit.io/reference/internals/livekit-sfu/) - observable path boundaries and the centralized low-latency alternative. +- [TURN, RFC 8656](https://www.rfc-editor.org/rfc/rfc8656.html) - per-edge relay + transport rather than a room topology. - [MOQT draft](https://datatracker.ietf.org/doc/draft-ietf-moq-transport/) and [moq-dev/moq](https://github.com/moq-dev/moq) - core MIT/Apache-2.0; its OBS plugin is separately GPL-2.0-or-later and remains study-only. diff --git a/docs/research/low-server-media-routes.md b/docs/research/low-server-media-routes.md index fa4be90..7f7444f 100644 --- a/docs/research/low-server-media-routes.md +++ b/docs/research/low-server-media-routes.md @@ -3,9 +3,9 @@ - Research date: 2026-08-19 - Scope: one broadcaster, at most eight trusted viewers, low latency, and host media fanout at most two -- Status: route screening for ADR-0004/ADR-0005; automatic hybrid routing has a - Draft implementation, while every non-browser data-plane candidate remains a - separate experiment +- Status: route screening for ADR-0004/ADR-0005; automatic failure-only hybrid + routing has a Draft implementation, while dual-TURN optimization and every + non-browser data-plane candidate remain separate, unimplemented experiments ## Current Route Ladder @@ -24,9 +24,11 @@ The smallest current plan is: experiment add opt-in native volunteer encoded-RTP relays. 5. Keep a user-operated mini-SFU and a centrally operated single-node SFU as optional capacity. The current ADR-0005 Draft controller uses that capacity - automatically only after deterministic peer recovery is exhausted. - They transfer fanout bandwidth to their operator; they do not make it - disappear. + automatically only after deterministic peer recovery is exhausted. The SFU + is a virtual parent for only one or two necessary roots; those roots continue + bounded peer descendants. A necessary viewer may be a zero-descendant root + under that same total only when no reliable relay root exists. Server + capacity transfers fanout bandwidth to its operator; it does not disappear. Any browser-spike failure other than isolated relay re-encoding closes that browser-relay route. It does not cancel the separate native sender plan. Closed @@ -67,15 +69,124 @@ still require measurement. ## Traffic Conservation And The Impossible Triangle -This is an engineering accounting identity, not a named theorem. For `N` -viewers receiving bitrate `B`, useful last-hop delivery is approximately -`N * B`. Ignoring protocol overhead: +This is an engineering accounting identity, not a named theorem. Let `B` be +the measured useful media bitrate for one same-representation branch, `B_pub` +the actual SFU publication bitrate, `B_i` root `i`'s selected bitrate, `N` the +viewer count, `R` the one or two seed/root viewers, and `B_edge` a descendant +edge's actual bitrate. +Useful last-hop delivery is approximately `N*B`. Ignoring protocol overhead: `host last-hop copies + peer last-hop copies + server last-hop copies = N * B` -TURN, RTP/QUIC headers, retransmission, FEC, and redundant paths only add traffic. -Encoding once can reduce compute and memory bandwidth, but it cannot remove the -network copy delivered to each viewer. +| Full-stream shape | Host upload | Viewer-relay upload | Central media traffic | +| --- | ---: | ---: | ---: | +| Peer roots, direct edges | `R*B` | `(N-R)*B` | none | +| Peer roots, all `R` seed edges through TURN | `R*B` | `(N-R)*B` | TURN ingress `R*B` + egress `R*B` | +| SFU virtual parent to the same roots | `B_pub` | `(N-R)*B` | SFU ingress `B_pub` + egress `sum(B_i)` | +| Same-representation full-room SFU | `B` | `0` | SFU ingress `B` + egress `N*B` | + +The table's `R*B` and `(N-R)*B` rows are equal-representation screening cases. +With mixed root representations, peer host/seed-TURN traffic is `sum(B_i)` and +descendant upload is `sum(B_edge)`. SFU host upload and ingress are `B_pub`, the +sum of all actively published representations; if simulcast `HIGH` and `LOW` +both remain active, that may be `B_HIGH+B_LOW`, not one root bitrate. The +SFU-root row, not full-room SFU, is the retained fallback shape. For equal +representations, `B_pub = B` and `sum(B_i) = R*B`. Publisher-to-SFU and every +SFU-to-root subscriber transport are independent ICE connections and may +select deployment-supported ICE/UDP, ICE/TCP, or TURN/TLS; peer descendants +may also select TURN. If the publisher leg uses TURN, retain host upload +`B_pub`, TURN ingress `B_pub`, TURN egress `B_pub`, and SFU ingress `B_pub` as +distinct interface/service traffic. A relayed root leg likewise adds TURN +ingress and egress `B_i`; a relayed descendant edge adds both directions for +that edge's actual bitrate. They are one logical useful payload copy but real +physical hops, so NIC, service, and billing counters must never be folded. + +RTP/RTCP/SRTP, DTLS, ICE/TURN and IP headers, retransmission, FEC, and redundant +paths only add traffic. W3C candidate-pair byte counters exclude some transport +overhead, and a provider may bill ingress and egress differently, so formulas +screen candidates but never replace host NIC, TURN/SFU, and billing counters +from the same run. Encoding once can reduce compute and memory bandwidth, but +it cannot remove the network copy delivered to each viewer. + +## Transport And Topology Layers + +TURN is a per-edge ICE transport. Each host-root or peer-peer P2P edge and each +SFU publisher/subscriber connection can independently select direct or relay +candidates without changing the rest of the room. An SFU is a topology node. +Screener uses it as a virtual +parent for one or two roots, not as an automatic all-viewer fanout service. +After media reaches a root, the deterministic sticky subtree, host/root fanout +at most two, browser fanout one, maximum depth, and bounded failure radius still +apply. Only the absence of any reliable relay root permits a necessary viewer +to consume one of the same one or two root slots with zero descendants. + +This distinction follows TURN's allocation/relay role in RFC 8656 and the media +topology boundary in RFC 7667. A relay candidate proves transport for one edge; +it is not evidence that an SFU topology would be cheaper or faster. + +## Privacy-Safe ICE Evidence Candidate + +Start with locally retained, redacted `webrtc-internals` as manual ground truth. +Only then may one exact allowlisted room record, without changing its route: + +- selected local/remote candidate type and protocol plus local + `relayProtocol`; +- `iceGatheringState`, `iceConnectionState`, and `connectionState` transitions + with monotonic event-derived establishment duration; +- `selectedCandidatePairChanges` when implemented, reported only as a delta + from the current opaque connection/ICE-restart generation baseline; +- RTT, loss, and actual/available bitrate for the selected RTP-bound path; and +- `icecandidateerror.errorCode` bucketed as + `3xx|4xx|5xx|6xx|701|other`. + +Never retain raw `errorText`, address, port, URL, candidate strings, SDP, or +device/network identifiers. Stats members may be absent, and the pair-change +counter is transport-lifetime state, so events and generation baselines prevent +old connection or ICE-restart history from being attributed to a new route. +Use opaque generations only. This is a shadow diagnostic manifest, not a +backend telemetry schema or controller input. + +## Dual-TURN Shadow Candidate + +Use four explicit network scenarios; no threshold or optimizer is selected: + +1. **Direct/direct roots:** retain peer distribution. +2. **Direct/TURN roots:** retain the mixed peer distribution; one relayed edge + does not justify an SFU. +3. **TURN/TURN roots:** after both host seed edges remain selected relay pairs + across a future reviewed stability window, emit only a shadow candidate. + The active route does not move under ADR-0005. A necessary useful-payload + condition for any host-upload claim is measured `B_pub < sum(B_i)`; real + hop counters must still prove the total benefit. +4. **No reliable relay root:** after deterministic recovery is exhausted, use + the existing failure-only SFU route; necessary viewers may consume the same + one-or-two root slots with zero descendants, never a second fanout budget. + +The evidence sequence is deliberately narrow: + +1. Use local `webrtc-internals` as manual ground truth for selected pairs and + redact SDP, addresses, candidates, and identifiers before retention. +2. Enable privacy-safe diagnostics for one exact `PEER_ASSISTED_ROOM_IDS` room + in shadow mode. It records a candidate but changes no media or protocol. +3. Run matched, non-simultaneous exact-room A/B sessions for two TURN roots and + one SFU publisher feeding the same one or two roots. This avoids exceeding + the live host-edge budget merely to benchmark both routes concurrently. +4. Compare real host upload, TURN/SFU ingress and egress, selected protocols, + RTT/loss/bitrate, first picture, steady latency, recovery, host/root load, + and the actual E2EE/operator boundary. The host-to-SFU publisher is an + independent ICE connection that may use ICE/UDP, ICE/TCP, or deployed + TURN/TLS. Retain every transport and service hop separately, including TURN + on publisher, subscriber-root, or peer-descendant edges; distinguish only + physical traffic totals from logical useful-payload copies. Thresholds + remain unknown. +5. Only a measured win plus an amended ADR-0005 may authorize an automatic + exact-room canary; broad rollout remains default-off. + +Ordinary SFU transport encryption lets the SFU operator access media. LiveKit +offers application E2EE for media/data, but signaling and APIs remain visible +and Screener does not yet implement key distribution. Compare the configured +boundary instead of treating E2EE as a checkbox. This candidate adds no route +score, controller trigger, wire field, backend telemetry system, or code. On today's unicast Internet, a design cannot maximize all three of these for more than one viewer: @@ -97,8 +208,8 @@ lost. | Fixed two-chain browser relay | Host emits at most two copies; each relay emits at most one | Ordinary browser, but every relay decodes and re-encodes and adds a hop | Current default-off, maximum-eight-viewer spike | | Native shared-encode host | Host targets one encode for at most two standard WebRTC edges | libwebrtc public-API proxy risk spike, with Pion as fallback | Planned separate sender phase; still pays per-edge upload | | Native volunteer encoded-RTP relay | Each volunteer forwards one encoded copy | Native install, RTP/RTCP forwarding, packaging, and opt-in relay policy | Conditional experiment only if relay re-encoding is the sole browser-spike failure | -| User-operated mini-SFU | User's SFU emits viewer copies | Separate deployment and its egress bill; running it on the host does not reduce that host's uplink | Optional capacity; automatic final fallback when enabled | -| Central single-node SFU | Service SFU emits viewer copies | Lowest endpoint relay burden; service pays approximately `N * B` egress | Optional capacity; automatic final fallback through merged PR #17 when enabled | +| SFU virtual parent | SFU emits only one or two root copies; roots keep peer descendants | Service pays measured root egress; host sends one publication | Default retained SFU shape; automatic only after peer failure under ADR-0005 | +| Zero-descendant SFU roots | SFU emits necessary viewer copies | Same root budget; service egress is at most `sum(B_i)` | Last resort only when no reliable relay root exists | | SVC plus multiple trees | Peers emit striped layer copies across several trees | Layer scheduling, reassembly, redundancy, and more churn state | Separate conditional spike; target endpoint upload near `B` | | Network coding | Peers or servers emit coded blocks | Generations, buffering, decoding, integrity, and a custom media plane | Trace/FEC spike only; optimize loss recovery, not clean bandwidth | | MoQ | Publishers and MoQ relays emit object copies | New transport, packaging, player, relay, and auth stack | Optional central-fallback benchmark; still pays server egress | @@ -176,7 +287,10 @@ Primary sources checked on 2026-08-19: - [RTP Topologies, RFC 7667](https://www.rfc-editor.org/rfc/rfc7667.html) and [ICE, RFC 8445](https://www.rfc-editor.org/rfc/rfc8445.html) - IETF standards under IETF Trust terms. +- [TURN, RFC 8656](https://www.rfc-editor.org/rfc/rfc8656.html) - the relay is + transport for client/peer traffic, not a room distribution topology. - [WebRTC](https://www.w3.org/TR/webrtc/), + [WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/), [WebRTC Encoded Transform](https://w3c.github.io/webrtc-encoded-transform/#stream-processing), and [WebRTC SVC](https://www.w3.org/TR/webrtc-svc/) - W3C specifications; SVC is an encoding control, not a distribution topology. @@ -198,6 +312,9 @@ Primary sources checked on 2026-08-19: - IETF standards under IETF Trust terms. - [LiveKit](https://github.com/livekit/livekit) - Apache-2.0; the optional SFU reference, with no code copied into this research change. +- [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/) + and [end-to-end encryption](https://docs.livekit.io/transport/encryption/) - + official behavior and operator-boundary references. - [LiveKit client 2.22.0 `Room.prepareConnection`](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts) and [official usage](https://github.com/livekit/client-sdk-js#usage) - Apache-2.0; API behavior was inspected, with no source copied. diff --git a/docs/research/native-shared-encode-sender.md b/docs/research/native-shared-encode-sender.md index d992f00..8d701b1 100644 --- a/docs/research/native-shared-encode-sender.md +++ b/docs/research/native-shared-encode-sender.md @@ -170,11 +170,21 @@ re-encode them. An ordinary non-scalable representation cannot be forwarded into a second quality; the bounded choices are a temporary second encode, SVC, or transcoding. The default is the temporary second representation. SVC is considered only for a later strict-one-output requirement and only after the -exact codec/mode and a hardware or power-efficient implementation pass the game -performance matrix; software fallback is rejected rather than hidden. Future +exact applied codec/mode, available encoder-implementation/native evidence, and +the real game/power matrix establish the path. Media Capabilities or RTCStats +power-efficiency signals are admission evidence, not hardware proof; an +unestablished path stays disabled rather than silently using software. Future dual-tree/striped distribution may reduce two-copy host upload toward one copy plus redundancy, but does not block dual-representation work. +Simulcast does not change this native proof boundary: one sender may negotiate +`HIGH`/`LOW`, but separate direct PeerConnections have no portable shared-encode +contract and inactive API state is not physical resource proof. ADR-0007 and +[Realtime Quality Adaptation](./realtime-quality-adaptation.md) own the +post-A+B/C simulcast, pinned LiveKit +Dynacast, and SVC gates. This path still requires measured per-representation +traffic/resources and cannot bypass #28's stock-GCC/RTX stop line. + ## Product Stop Line The stock Pion GCC plus negotiated RFC 4588 RTX composition is @@ -216,6 +226,7 @@ evidence does not retain its policy. - [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) - [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/) - [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/) +- [W3C WebRTC simulcast](https://www.w3.org/TR/webrtc/#simulcast-functionality) - [Pion WebRTC v4.2.18](https://github.com/pion/webrtc/tree/v4.2.18) - [Pion Interceptor v0.1.47](https://github.com/pion/interceptor/tree/v0.1.47) - [RFC 4585 RTP/AVPF feedback](https://www.rfc-editor.org/rfc/rfc4585.html) diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md index 5f1fb16..9d45dc7 100644 --- a/docs/research/realtime-quality-adaptation.md +++ b/docs/research/realtime-quality-adaptation.md @@ -35,32 +35,52 @@ described as automatically detecting game motion and forcing a 720p cap. ## Production Degradation Report -On 2026-08-19 the deployed build was reported to deliver very low resolution, -bitrate, and frame rate across all three profiles. This is a user observation, -not yet an instrumented result. Draft PR #28's minimum-of-two feedback loop was -never deployed and cannot have caused it. The production `getStats()` probes are -read-only and do not trigger a downgrade. Peer/SFU code ships in `769de201f7cc`, -but its production configuration is absent, so relay re-encoding and LiveKit -cannot explain the earlier observation either. +On 2026-08-19 production `769de201f7cc` was reported to become severely blurry +while the host repeatedly showed a sustained native +`qualityLimitationReason=bandwidth`, despite one capable viewer on the same LAN +and the UI showing direct P2P. This is a user report, not yet an instrumented +root cause. The probe is read-only, requires three consecutive windows, and +does not change media. Same-LAN/direct does not exclude Wi-Fi loss or queuing, +per-PeerConnection GCC, aggregate host-uplink contention, or browser bandwidth +underestimation; isolated encoder pressure would more commonly report `cpu`. -Plausible causes remain: capture settings below the request; one independent -encoder pipeline per peer exhausting CPU/GPU; GCC reacting to host uplink or a -TURN path; treating `maxBitrate` as a target when it is only a ceiling; browser -rewriting or scaling sender parameters; or receive loss, jitter, decode, and -display scaling. None is selected as the root cause before correlated evidence. +Only a full Host refresh and re-share was reported to restore immediate +high-definition, fluid video; Viewer refresh did not. With a stable +session-storage `clientId` and reconnect inside the five-second grace, Viewer +reauth reuses its `peerId` and emits `peer-joined`, while `HostPage.startPeer` +retains an already-connected `HostPeer`; a new tab, storage failure, or expired +grace can instead allocate a new peer. The reported Viewer refresh therefore +did not create a new host PeerConnection/GCC generation. Host-page cleanup +disposes every peer, +stops capture, and creates all-new capture, sender, PC, and GCC generations. +This makes a stuck host sender/PC/GCC or shared-capture generation a priority +hypothesis, not a conclusion. A further report that quality may remain low +after the old viewer leaves and a new viewer joins must likewise be tested +against lifecycle evidence rather than assumed. -One controlled capture should classify the problem before changing constants: +Use this as the first A+B/C reproduction before the viewer controller or +`LOW` runtime: -1. Run the same high-motion scene for 60 to 90 seconds with one, two, then three - viewers, plus one TURN/UDP case. -2. Compare capture settings/media-source, outbound and inbound resolution/FPS, - actual and target bitrate, codec and encoder implementation, interval encode - cost, selected path, and `qualityLimitationReason`. -3. Treat source-low as capture, outbound-low with `cpu` as encoder pressure, - outbound-low with `bandwidth` as uplink/path pressure, and inbound-only loss - or dropped/frozen frames as network or receiver pressure. -4. Export `chrome://webrtc-internals` only as sensitive local evidence; remove - SDP, candidate addresses, and other identifiers before sharing or retaining. +1. Reproduce one wired LAN/direct viewer first, then Wi-Fi, two/three viewers, + and forced TURN, using the same high-motion scene for 60 to 90 seconds. +2. In one window record A capture; B actual/target/available outbound bitrate, + dimensions/FPS, codec, encode cost, loss/RTT/retransmission, selected path, + limitation, and PC/track generations; and C inbound decode/drop/freeze. +3. First retain capture and every healthy peer while rebuilding only the + affected `HostPeer`/`connectionId`. Separately compare capture/track + replacement with PCs retained, then a full Host rebuild; do not combine the + interventions. +4. For viewer churn and stable-`clientId` reconnect, retain old/new + `peerId`/`connectionId`, `peer-left`-to-dispose time, active sender count + inside/outside the five-second grace, old snapshot removal, whether a new PC + actually exists, capture settings, and new-path BWE/limitation. + +A single-edge rebuild passes only if the affected viewer recovers, unaffected +viewers neither migrate nor interrupt, and the host-edge cap holds. Only proven +generation-specific failure can justify a later automatic one-edge recovery +with sustained-bandwidth and healthy-counterpart evidence, cooldown, and a +generation guard. Do not add periodic reconnect, blindly raise a ceiling, or +change the current controller before this gate. Missing `scaleResolutionDownBy` is not itself a root cause for a single encoding because its effective default is 1.0. Likewise, the current 3/5/8 Mbps values @@ -171,6 +191,19 @@ defines `hardwareAcceleration` only as a hint the user agent may ignore. Therefore none is, by itself, proof of a particular hardware encoder, and a software SVC fallback must not be silent. +After trustworthy A+B/C and the native fixed-`HIGH` path, run three bounded +capability spikes. First, negotiate `HIGH`/`LOW` simulcast in one sender's +initial envelope with `LOW` inactive and prove applied parameters, per-RID +bytes/frames, and CPU/GPU/encoder release; separate PeerConnections have no +portable shared-encode contract. Second, test at most two independently +selecting LiveKit roots, but expect pinned server 1.13.5 to enable every layer +at or below the maximum requested quality, so a `HIGH` root also keeps `LOW` +enabled; client 2.22.0 then applies `active`, with Firefox using only a low-rate, +low-FPS, 4x-scale fallback. Reject Dynacast for the exact on-demand-`LOW` +requirement unless runtime and resource counters disprove that boundary. Third, +compare requested/applied SVC mode and Media Capabilities `powerEfficient`, +with no software fallback. None may bypass PR #28's stock-GCC/RTX stop line. + ## Why Offline Encoding Presets Do Not Transfer [Maruko Toolbox](https://github.com/wzxjohn/marukotoolbox) is an Apache-2.0 @@ -307,6 +340,9 @@ recovery, and that each supported sender cohort can start it reliably. - [libwebrtc removal of the automatic animation-detection experiment, 2024-05-22](https://webrtc.googlesource.com/src/+/1d7d0e6e2c5002815853be251ce43fe88779ac85) - [LiveKit screen-share presets](https://github.com/livekit/client-sdk-js/blob/main/src/room/track/options.ts) - [LiveKit degradation defaults](https://github.com/livekit/client-sdk-js/blob/main/src/room/participant/publishUtils.ts) +- [LiveKit client 2.22.0 Dynacast layer control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalVideoTrack.ts) +- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go) +- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go) - [Jitsi desktop degradation preference](https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/TraceablePeerConnection.ts) - [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology) - [Discord encoder-quality case study](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus) diff --git a/docs/status.md b/docs/status.md index 61153c6..5922903 100644 --- a/docs/status.md +++ b/docs/status.md @@ -14,7 +14,7 @@ unconfigured, so production remains one host connection per viewer and P2P/TURN. - Optional `ACCESS_PASSWORD` uses a stateless 12-hour HMAC HttpOnly Strict cookie; host auth stays internal and signaling role-bound. - Without `ROOM_DATABASE_PATH`, rooms are temporary. With it and the password, sequential links persist after stop; SQLite stores only room ID and host-token digest. - Direct ICE is preferred independently per media edge. Authenticated TURN/UDP and TURN/TCP are required production fallbacks; TURN/TLS is optional. -- Hidden routing is `direct P2P -> peer-assisted -> optional SFU`; exact-room canaries use `PEER_ASSISTED_ROOM_IDS`. Production is off and 22 files/290 tests cover both paths. +- Hidden routing is `direct P2P -> peer-assisted -> optional SFU roots`; TURN is selected per edge, and SFU roots retain bounded peer descendants. Exact-room canaries use `PEER_ASSISTED_ROOM_IDS`; production is off. - Configured fallback adds a non-secret standby URL and token-free DNS/TLS warmup; absent configuration adds no field, import, request, participant, or edge. - Each viewer advertises relay capacity 0/1; detected mobile/iPad clients are leaves. Withdrawal does not move a healthy edge. Browser relays stay one-child and host two-child. - Production has one memory-only video setting with manual ceilings/readback. Automatic per-path `HIGH`/`FALLBACK` and one on-demand shared `LOW` are accepted but not deployed; only the first local A+B foundation exists. Browser audio is request/presence-only. @@ -36,20 +36,29 @@ unconfigured, so production remains one host connection per viewer and P2P/TURN. - Android Chrome/iOS Safari leaves are unverified; runtime conservatively marks detected mobile/iPad clients as leaves. - Silent partitions can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; this remains unverified. - Real audio, heterogeneous clients, mobile lifecycle, quality/pause, room `1` republish, and sustained profiles are unverified. Poor movie/video audio is user-reported but unclassified; after video A+B, diagnose capture settings, codec/fmtp, actual bitrate, loss, jitter, concealment, jitter buffer, and A/V sync. -- Earlier production degraded across profiles. Controlled 1/2/3-viewer and TURN samples must isolate capture, CPU, path, and receiver limits before quality/controller claims. -- ADR-0007 remains incomplete. The first local A+B alignment foundation is implemented, but codec evidence is MIME-only and still lacks derived profile/parameters/`scalabilityMode`; the minimal authenticated C report, two-state controller, and `LOW` runtime do not exist. Verified weak paths must ultimately move to the one shared `LOW`, and supported sender cohorts that cannot start it reliably fail acceptance; fail closed protects `HIGH` only as exceptional damage containment. +- Production `769de201f7cc` reportedly sustains native `qualityLimitationReason=bandwidth` and severe blur with one capable LAN/direct viewer; only Host refresh restores quality. Within the five-second grace, stable-clientId Viewer refresh retains its `peerId`/`HostPeer`, while Host refresh rebuilds peers/capture; churn persistence is also reported. This prioritizes but does not prove sender/PC/GCC/capture generation over network or lifecycle causes. +- ADR-0007 is incomplete. Local A+B alignment exists, but codec evidence is MIME-only; derived profile/parameters/`scalabilityMode`, authenticated C, the controller, and `LOW` runtime are absent. Weak paths must ultimately share one `LOW`; fail closed protects `HIGH` only exceptionally, and an unreliable supported cohort fails acceptance. - Browser relays re-encode. The native ladder proves one WebCodecs object, not hardware encode. GCC+RTX is no-go; no-RTX weakens stats. Broader audio/loss/routes/reconnect/browser/product/striping gates remain. - The endpoint budget remains host/relay at most two downstream edges; the browser path is host two/viewer one. +- ADR-0005 permits SFU only after peer failure. Stable dual-TURN roots are shadow-only; exact-room TURN-root/SFU-root cost, recovery, and trust evidence is absent. +- Pinned LiveKit 1.13.5 Dynacast enables all qualities at or below the room's maximum request, so a `HIGH` root is expected to keep `LOW` enabled. It remains a bounded rejection/verification spike, not evidence that on-demand `LOW` can stop. - The corrected standby smoke is localhost/headless/video-only; public DNS/TLS reuse, transport, audio, shaping, load, mobile, endurance, and sub-25 ms overlap remain open. ## Next Milestone -Validate production controls on real game capture and finish video A+B codec -derivation. Then run the separate movie/video audio A/B/C and sync diagnosis in -`docs/research/browser-screen-audio-quality.md`; `maxBitrate` is a ceiling, not -a quality fix. Next add minimal authenticated video C, the two-state controller, -and on-demand shared `LOW`. Route enablement still needs one exact-room canary -across public transports, recovery, edge counts, egress, and load. +Finish video A+B codec derivation, then reproduce the same-LAN/direct case. +First rebuild one affected `HostPeer`/`connectionId` while retaining capture and +healthy peers; separately compare capture-only and full Host rebuilds with +A+B/C evidence and opaque generations. The affected viewer must recover without +moving healthy peers or breaking the host-edge cap. Only proven stuck +generations justify guarded/cooldown recovery; never reconnect periodically or +raise ceilings blindly. + +After video A+B, separately run the movie/video audio A/B/C and sync diagnosis +in `docs/research/browser-screen-audio-quality.md`; `maxBitrate` is not a quality +fix. Then add minimal authenticated video C, the two-state controller, and +on-demand shared `LOW`. Later route work starts with manual ICE ground truth and +shadow-only exact-room TURN-root/SFU-root comparisons, not automatic migration. ADR-0004 still requires a full-resolution 30-minute `1/3/5/8` network, resource, quality, latency, recovery, and browser/mobile-leaf matrix. diff --git a/docs/方案设计.md b/docs/方案设计.md index 4e036a1..6efc14e 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -145,6 +145,8 @@ PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 直播中切换质量不重新打开选源器：对当前视频 track 调用 `applyConstraints()`，再更新所有现有 sender 的参数；新加入、重建的 relay sender 和启用的 SFU publisher 使用同一权威设置。失败时保留当前分享并给出状态，不以重建整个房间补偿。临时暂停只禁用当前视频 track，使远端收到黑画面；音频、房间和连接保持不变，恢复时重新启用同一 track。 +Host 本地预览直接绑定现有 capture stream，不创建 viewer、`RTCPeerConnection` 或额外媒体边，也不产生 TURN/SFU/应用服务器出口。预览仍可能有本地合成、缩放和 GPU 显示成本；在同一动态场景对预览开/关采集 Host CPU、GPU、encode time 和实际 FPS。只有稳定收益且捕获、编码和远端画面不变时，才把直播后的默认动态预览改为静止缩略图或隐藏并允许按需展开；隐藏预览绝不停止 capture track 或 sender encode。 + 当前 Web 高级区不暴露音频质量控件。Screen Capture 规范未把 `channelCount`、`sampleRate` 列为 display-audio 可控属性；`track.getSettings()` 中偶尔出现的值只能作为捕获观测。标准 `RTCRtpSender.encodings[].maxBitrate` 可设置和读回，但只降低发送 ceiling，既不能要求编码器提高质量，也不等于 Opus `maxaveragebitrate`。Opus stereo、DTX 和 in-band FEC 属于协商的 codec/fmtp 行为，没有跨主流浏览器的标准 sender setter；`opus/48000/2` 也只是 RTP 信令约定，不能证明 48 kHz 源或实际 stereo。设计不为这些控件新增应用层 SDP munging，也不把 SDK 私有设置包装成浏览器保证。详细证据与复核门槛见 `docs/research/browser-screen-audio-quality.md`。 `frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限；浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据，但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机，先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。 @@ -171,6 +173,8 @@ Viewer 可通过已鉴权、绑定当前房间/路径代次的控制面请求 `L 质量控制不放宽媒体拓扑预算：host 仍最多两条下游媒体边；native relay 只转发选定表示的编码包，不解码重编码。后续双树或条带化可把 host 上传从约两份完整码率推向一份加必要冗余，但不阻塞按需双表示的实现。 +标准表示能力的顺序固定在完整 A+B/C 证据和原生 fixed-`HIGH` 通过之后：再依次做同 sender simulcast、最多两个 LiveKit root 的 Dynacast 验证/否决和 SVC spike。具体语义与资源门槛由 ADR-0007 和[实时质量调研](./research/realtime-quality-adaptation.md)持有，任何方案都不得复用 PR #28 的 minimum-of-two 或绕过 stock WebRTC congestion control。 + ## TURN 配置与轮换 应用服务在房间鉴权后生成 coturn REST 凭据： @@ -187,7 +191,9 @@ TURN/TLS 是可选能力，不是生产启动门槛。启用时默认使用 RFC 凭据过期前，客户端通过已认证 WebSocket 请求新配置，并对现有连接调用 `setConfiguration()`。已有 TURN allocation 通常不会因凭据时间点到达而立即终止，但后续 ICE restart 必须使用新凭据。 -候选由 ICE 按优先级和连通性检查选择，不承诺 JavaScript 严格串行执行 UDP、TCP 或可选 TLS 尝试。界面报告最终选中的 candidate type、ICE protocol，以及本地为 relay 时的 TURN protocol，而不是推测回退步骤。`turn:` over TCP 没有客户端至 TURN 的 TLS 封装，但被中继的 WebRTC 媒体仍由 DTLS-SRTP 保护。 +候选由 ICE 按优先级和连通性检查选择，不承诺 JavaScript 严格串行执行 UDP、TCP 或可选 TLS 尝试。界面报告最终选中的 candidate type、ICE protocol，以及本地为 relay 时的 TURN protocol，而不是推测回退步骤。`turn:` over TCP 没有客户端至 TURN 的 TLS 封装，但被中继的 WebRTC 媒体仍由 DTLS-SRTP 保护。TURN 是单条 ICE 边的 transport，peer-assisted 与 SFU 才是拓扑；一个 selected relay pair 不改变其他边的父子关系。 + +可选 SFU 只作为虚拟 parent 向总计一至两个必要 roots 发布；每个可靠 root 收到媒体后继续服务既有 sticky descendants。host/native root 最多两条下游媒体边，browser root/relay 最多一个 child；没有可靠 relay root 时，必要 viewer 只作为同一上限内的零 descendants root。host→SFU publisher 是独立 ICE 连接，所有 TURN/SFU/endpoint 物理 hop 都按[低服务器成本媒体路由](./research/low-server-media-routes.md)逐项计量。双 host seed 稳定 TURN 仍只允许精确房间 shadow；ADR-0005 保持 failure-only。 ## 可观测性 @@ -200,6 +206,7 @@ TURN/TLS 是可选能力，不是生产启动门槛。启用时默认使用 RFC - `connectionState` / `iceConnectionState`； - 选中候选对的 local/remote candidate type 与 ICE `protocol`，任一端为 `relay` 即标记中继； - 本地 relay candidate 的 `relayProtocol`；规范禁止远端 candidate 暴露该字段，因此不推断远端使用 TURN/UDP、TCP 还是 TLS； +- 后续双 TURN shadow 只按[低服务器成本媒体路由](./research/low-server-media-routes.md)的 generation-baselined、privacy-safe ICE manifest 记录；先以本地脱敏 `webrtc-internals` 建立 ground truth，不新增后台遥测； - `currentRoundTripTime`、发送码率和 `availableOutgoingBitrate`； - 实际帧率、分辨率和丢包； - codec、编码实现、最近采样区间内的每帧编解码耗时和 `qualityLimitationReason`；同一连接不并发采样，首个样本、无新帧、统计对象换代或计数器重置时显示未知并重新建立基线，不用连接生命周期均值掩盖后续过载。 @@ -278,7 +285,7 @@ room code 规则且不重复的 ID；非法配置启动即失败。未配置或 `HybridMediaRouter`、房间质量同步或 LiveKit 兜底，鉴权 wire、定向信令、停止、 重连和删除必须逐项保持普通 P2P 行为。该边界不增加 UI、用户选择或百分比框架。 -产品硬目标是分享端任何时刻最多两条下游媒体边。系统按一个或两个观看者直接 P2P、更多观看者 peer-assisted、部署已提供的 SFU 最后兜底的顺序自动分配和恢复，分享者与观看者不选择拓扑。当前实验只实现前两级；跨模式迁移由后续独立 ADR 定义。 +产品硬目标是分享端任何时刻最多两条下游媒体边。系统按一个或两个观看者直接 P2P、更多观看者 peer-assisted、部署已提供的 SFU 最后兜底的顺序自动分配和恢复，分享者与观看者不选择拓扑。TURN 只在每条逻辑边内独立兜底；SFU 作为虚拟 parent 只送总计一至两个必要 roots，roots 之后仍优先沿受限 peer descendants 扩散；没有可靠 relay root 时，必要 viewer 只作为同一总上限内的零 descendants root。`main` 已包含 default-off 的跨模式控制器，但生产未启用且 ADR-0004/0005 尚未验收。 实验只实现一个由服务端确定的广度优先拓扑：host 最多两个 child，每名 viewer 最多一个 child，因此形成两条按加入顺序平衡的 sticky 链。父节点只按深度和 `serverJoinSequence` 选择第一个空 slot。新 viewer 加入时不搬动已有节点；parent 离开时只把失去 parent 的 subtree root 重新分配到第一个空 slot，其 descendants 保持连接。该规则不采集或组合 RTT、带宽、CPU、地理位置与质量评分，也不主动重平衡。没有 parent 时明确等待或拒绝，不能创建分享端第三条边。 @@ -307,6 +314,7 @@ spike 复用现有 screen stream 和浏览器 codec 协商，覆盖当前桌面 - [Media Capture Content Hints](https://www.w3.org/TR/mst-content-hint/) - [RTCRtpSender.replaceTrack](https://www.w3.org/TR/webrtc/#dom-rtcrtpsender-replacetrack) - [WebRTC Stats](https://www.w3.org/TR/webrtc-stats/) +- [TURN, RFC 8656](https://www.rfc-editor.org/rfc/rfc8656.html) - [WebRTC SVC](https://www.w3.org/TR/webrtc-svc/) - [Media Capabilities](https://www.w3.org/TR/media-capabilities/) - [WebCodecs](https://www.w3.org/TR/webcodecs/) diff --git a/docs/需求理解.md b/docs/需求理解.md index ca2c5cc..08e68ff 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -15,17 +15,17 @@ - [ ] 首屏双入口：通过全站验证后，主工作区必须同等突出“开始分享”和“加入房间”两个主要动作，并用清楚的视觉分隔表达发送与接收；加入房间不能只藏在页头或次要链接中。点击后直接进入现有分享或房间码加入流程，不增加营销首页、模式向导或拓扑选择。 - [ ] 简单房间：分享者获得纯数字房间码和 `/r/{code}` 链接；观看者也可在 `/join` 只输入房间码加入，链接不携带额外观看 token 或 fragment。启用持久受保护房间时，房间码从 `1` 开始自增；其他模式继续使用随机临时房间码防止枚举。 - [ ] 免安装观看：桌面和手机观看者可直接通过浏览器链接进入、授权并播放，不依赖与分享端相同的桌面客户端。 -- [ ] 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止、临时暂停画面和切换分享来源。 +- [ ] 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止、临时暂停画面和切换分享来源。Host 本地预览必须直接使用 capture stream，不得为预览建立 viewer/媒体边或消耗 TURN、SFU、应用服务器 media egress；预览仍可能产生本地合成、缩放和 GPU 显示成本，不能把隐藏预览称为停止采集或编码。 - [ ] 游戏声音：Windows Chrome/Edge 目标环境可请求系统音频；没有返回音轨时必须明确提示，而不是静默无声。 - [ ] 一对多观看：一名分享者可向配置允许的多名观看者发送画面；默认上限为 8，可配置范围为 1 至 16。当前 PoC 的逐观看者连接是测量基线，不再是超过两名观看者时的目标拓扑。 -- [ ] 分享端 fanout 硬上限：分享端任何时刻最多维护两条下游媒体边，包括加入、断线、换父和恢复过程。第三名及后续观看者由系统自动分配给通过验收的 peer-assisted 拓扑，必要时自动回退到部署已提供的可选 SFU；两者都不可用时才明确拒绝，不能隐藏地回退为分享端第三条连接。 +- [ ] 分享端 fanout 硬上限：分享端任何时刻最多维护两条下游媒体边，包括加入、断线、换父和恢复过程。第三名及后续观看者由系统自动分配给通过验收的 peer-assisted 拓扑，必要时自动回退到部署已提供的可选 SFU；两者都不可用时才明确拒绝，不能隐藏地回退为分享端第三条连接。TURN 只替代无法直连的单条 ICE 边，不定义房间拓扑；SFU 只作为虚拟 parent 向一至两个必要 root 送流，媒体到达 root 后其余可行观看者仍优先沿受限 peer descendants 扩散。只有没有可靠 relay root 时，中央服务才可把必要 viewer 作为零 descendants 的 SFU root，仍计入同一个一至二 root 总上限，不能把 SFU fallback 默认为整房 fanout。 - [ ] P2P 优先：每条逻辑媒体边的 ICE 必须先尝试直接 UDP 路径，服务器不得默认中转可直连的媒体。一个或两个观看者优先直接连接分享端；更多观看者的 peer-assisted 行为须先通过 ADR-0004 的实验门槛。 -- [ ] NAT 兜底：每条 parent-child 边独立进行 ICE；生产环境必须提供 STUN、TURN/UDP 与 TURN/TCP candidates，通过优先级偏好直连和 UDP，同时允许无法使用 UDP 的网络选择 TCP，不能因为某条边的网络与其他边不同而静默失败。 +- [ ] NAT 兜底：每条 P2P 边以及 SFU publisher/subscriber connection 独立进行 ICE；生产环境必须提供 STUN、TURN/UDP 与 TURN/TCP candidates，通过优先级偏好直连和 UDP，同时允许无法使用 UDP 的网络选择 TCP。某条边选择 TURN 只改变该边的 transport，不能把其他边或整房静默改成服务器拓扑。 - [ ] 质量控制：保留 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档推荐上限，默认采用清晰优先；分享期间可直接换档或应用有界高级设置，不重新选择来源或重建健康连接。Peer-assisted relay 与可选 SFU 必须接收同一个严格校验的房间级权威设置，覆盖当前和以后建立的 sender；普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好，必须读回实际 sender 参数并显示参数拒绝、改写和持续受限原因。不做自定义动态 FPS、综合评分或周期性全房间调档。 - [ ] 按需双表示：健康房间只运行一套共享 `HIGH` 编码；第一条观看路径被同一时段的 sender 与 viewer 统计持续证明无法承受时，系统必须启动唯一一套由全部弱路径共用的 `LOW` 并迁移弱路径。健康路径继续使用 `HIGH`，全部弱路径稳定恢复后关闭 `LOW`。编码表示硬上限为二，禁止按观看者增长为 N 套，也禁止把最差路径的目标应用到整个房间。没有合格硬件/power-efficient 路径或第二编码造成不可接受负载时，当次必须 fail closed：保护 `HIGH` 并明确显示弱路径降级或不可用；这只是异常止损，supported sender cohort 若不能可靠按需启动 `LOW`，则自动质量控制验收失败，必须优化编码路径或明确标记不支持。 - [ ] 路径级质量状态：每条观看路径只使用 `HIGH` 与 `FALLBACK` 两态。进入 `FALLBACK` 需要多个连续窗口的带宽不足、冻结或解码跟不上证据，回到 `HIGH` 需要更长的稳定窗口；进入与退出使用不同门槛且不计算综合分数。Viewer 的低档请求必须经当前会话鉴权、限频和去重，并由 sender 的 GCC/Transport-CC、实际码率、RTT、丢包、重传、编码限制、派生后的实际协商视频 codec/profile/parameters（适用时含 `scalabilityMode`）与 viewer 的实际解码、丢帧、冻结证据交叉验证，不能仅凭自报生效；上报不得包含 raw SDP、candidate 地址或原始设备/网络标识，鉴权与关联只使用服务端签发的不透明路径/连接代次 ID。 - [ ] 断线恢复：临时网络变化时尝试 ICE restart 和重连，并向用户显示当前连接状态。 -- [ ] 可观测性：采集候选类型、RTT、丢包、抖动、实际码率、帧率、编码耗时和质量受限原因。 +- [ ] 可观测性：按目标 RTP transport 与不透明连接/ICE 代次关联 selected path、`iceGatheringState`/ICE/connection state、事件推导的建立时长、generation-baselined pair-change delta，以及 RTT、丢包、抖动、码率、帧率、编码耗时和受限原因。候选与错误仅使用[低服务器成本媒体路由](./research/low-server-media-routes.md)的脱敏字段和 code bucket，禁止 raw SDP、candidate、地址、URL、`errorText` 或网络标识。 - [ ] 渐进式诊断：分享者和观看者默认只看到房间、画面、核心状态和常用控制；候选路径、RTT、丢包、编解码器、编码耗时等技术详情由一个明确的“显示连接详情”开关展开。隐藏诊断不能隐藏失败原因、音频缺失、持续画质受限或需要用户操作的告警。 - [ ] 访问控制：部署者可配置一个全站 `ACCESS_PASSWORD`；为空时网站公开，非空值必须为 1 至 128 个可见 ASCII 字符。配置时分享者和观看者都必须先登录。登录成功签发 12 小时、无服务端会话状态的 HttpOnly、`SameSite=Strict` cookie，建房 HTTP 和 WebSocket 都只使用该 cookie，不提供建房 Bearer 旁路；不引入账号、JWT、服务端 session Map 或 logout。 - [ ] 持久受保护房间：`ROOM_DATABASE_PATH` 可选且必须与 `ACCESS_PASSWORD` 同时配置。两者都存在时，SQLite 只持久保存从 `1` 开始自增的房间 ID 与 host token 摘要；房间及链接不设过期时间，停止后分享端显示“已停止分享”，观看页显示“等待开始分享”。密码模式不配置数据库时仍使用临时房间；公开模式始终使用随机临时房间。 @@ -35,12 +35,13 @@ - [ ] 实验路由按房间灰度：部署开启 `PEER_ASSISTED_MEDIA` 时可用严格的房间 ID allowlist 只让指定房间进入 peer-assisted 与可选 SFU 路径；同一进程内未命中房间的鉴权 wire、P2P 信令、质量拒绝、停止、重连和删除行为必须保持不变。缺失或空 allowlist 为兼容已有部署而表示全部房间，因此隔离灰度必须显式填写至少一个合法 ID；非法或重复配置启动即失败。该能力仅由部署控制，不增加 UI、用户选择、百分比发布或第二套路由器。 - [ ] 质量控制扩展：推荐档位仍是默认入口；折叠高级区只允许 720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling，以及清晰/平衡/流畅三种偏好。直播中显式应用后才生效，避免滑杆产生连续控制流量。当前 Web 发送端不提供音频声道、采样率、Opus 码率、stereo、DTX 或 FEC 控件：display capture 未把声道和采样率定义为可移植控制，标准音频 `maxBitrate` 只是可能导致更低质量或停播的发送上限，其他项依赖 codec/fmtp 协商而没有可移植 sender setter。除非真实音频矩阵证明用户问题、标准 API 能应用并读回且实际结果可独立观测，否则不得为这些控件新增应用层 SDP munging 或用默认值制造伪控制。实际质量持续低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一组合称为“完美平衡”。 +- [ ] 本地预览性能候选：在相同动态场景中 A/B 测量 Host 动态预览开/关的 CPU、GPU、encode time 和实际 FPS。只有稳定收益且捕获、编码和远端画面不受影响时，才允许直播开始后默认显示静止缩略图或隐藏动态预览并提供按需展开；未测前不宣称收益。 - [ ] Peer-assisted 媒体实验：分享端最多两个 child、每名 viewer 最多一个 child；信令服务只按深度和服务端加入序号确定性地形成两条 sticky 平衡链，不加入综合评分或主动重平衡。每个 viewer 将收到的标准 remote track 通过新的 `RTCPeerConnection` 转发，因此会逐跳解码和重编码；必须测量而不能称为 shared encode。服务端只在受房间数限制的内存状态中保存当前严格校验的质量设置，不写入 SQLite；host 鉴权后先重报设置，在线 viewers 接收更新，后加入者从鉴权快照取得当前值。该功能在 ADR-0004 的实验门槛通过并由后续 ADR 正式接受前不得成为产品路径。 - [ ] 原生 shared encode 分享端：后续独立的 Electron/原生分享端阶段必须保证一份兼容编码输出扇出到最多两套独立 WebRTC packetizer，并由单独 ADR 和测量约束。它降低分享端重复编码负载，但不消除每条出边各自的上传带宽。浏览器不保证跨 `RTCPeerConnection` 复用编码器，因此当前 Web spike 不实现、也不能宣称满足 shared encode 保证。 - [ ] 原生 shared encode relay：具备原生 relay 能力的 audience 同样最多维护两条下游媒体边，并应让一份兼容编码输出供两条边复用。普通浏览器 audience 继续作为 leaf 或使用当前会逐跳重编码的实验路径，不伪称 shared encode。 -- [ ] 条件式 SVC：仅当后续需求要求任何时刻严格只有一个编码输出时才评估可伸缩编码，让各连接独立选择基础层或增强层。只有精确 codec/`scalabilityMode` 协商、硬件或 power-efficient 路径和游戏性能都实测通过时才允许启用；不得静默回退为高负载软件 SVC，常驻 SVC 不是默认方案。 +- [ ] 条件式 SVC：仅当后续需求要求任何时刻严格只有一个编码输出时才评估可伸缩编码，让各连接独立选择基础层或增强层。该 spike 只能排在完整 A+B/C 证据与原生 fixed-`HIGH` 之后，必须读回 requested/applied codec/`scalabilityMode`，由 Media Capabilities 提供 power-efficient 候选证据并通过真实游戏性能验证；不得静默回退为高负载软件 SVC，常驻 SVC 不是默认方案。 - [ ] 分包/分层多树实验：把编码包、对象或可独立解码层拆到多个 peer 路径以分散上传，只能在独立 spike 中验证。必须保持亚秒级，处理多父重组、排序、丢包恢复、关键帧、同步和节点退出；未通过门槛前不进入产品路由控制器。 -- [ ] 可选 SFU：部署者可选择是否提供单节点 SFU 资源；配置后由 `main` 中 PR #17 的路由控制器把它作为 P2P 与 peer-assisted 之后的自动兜底，而不要求用户选模式。它保持默认关闭、可逆，并承担每名观看者的服务器出口成本。已关闭 PR #12 的显式整房 SFU 模式已被取代，不是当前选项；自动 fallback 仍须通过 ADR-0005 门槛才可部署。 +- [ ] 可选 SFU：部署者可选择是否提供单节点 SFU 资源；配置后由 `main` 中 PR #17 的路由控制器把它作为 P2P 与 peer-assisted 故障恢复穷尽后的自动兜底，而不要求用户选模式。它保持默认关闭、可逆，只向总计一至两个必要 root 出口；root 优先承担受限 peer descendants，没有可靠 relay root 时必要 viewer 只作为同一总上限内的零 descendants root。已关闭 PR #12 的显式整房 SFU 模式已被取代，不是当前选项；两条 host seed 都稳定选择 TURN 时只进入脱敏 shadow 评估，不改变 ADR-0005 的 failure-only 触发语义。 - [ ] 分享端桌面封装：用 Electron 固定 Chromium 版本、改善选源/热键/系统音频体验，同时保留 Web 观看端。 - [ ] Windows 应用音频：仅捕获所选游戏及其子进程的音频，避免把语音聊天或通知声一并分享。 - [ ] 连接自检：在开始分享前验证信令、STUN、TURN/UDP、TURN/TCP 及部署实际配置的可选 TURN/TLS 可达性，并给出不暴露凭据的诊断结果。 @@ -58,8 +59,8 @@ - 性能：在 RTT 不高于 40 ms、丢包不高于 1% 的受控 P2P 路径中，玻璃到玻璃延迟目标为 p50 不高于 150 ms、p95 不高于 250 ms；区域 TURN/UDP 路径的 p95 目标不高于 350 ms。 - 启播：观看者请求观看后，除用户授权耗时外，3 秒内出现首帧。 - 流畅性：选择 60 fps 档位时，不能在没有降档或提示的情况下连续 5 秒低于 50 个解码帧每秒。 -- 成本：除按边选择的 TURN 中继外，直连和 peer-assisted 媒体不经过应用服务器；STUN 与信令仅承担小流量；TURN 的实际流量必须可计量、限额和告警。Peer-assisted 只把约 `N * B` 的总观看流量分散到参与节点，不能宣称消除流量；SFU 则把该出口成本转移到服务器。 -- 安全：应用控制面全程 HTTPS/WSS，媒体端到端链路使用 WebRTC DTLS-SRTP；普通 `turn:` over TCP 不额外套 TLS，但不移除媒体层加密。TURN 使用短期凭据、配额和速率限制；仓库不得包含真实密钥。 +- 成本：除按边选择的 TURN 中继外，直连和 peer-assisted 媒体不经过应用服务器；STUN 与信令仅承担小流量。候选筛选使用[低服务器成本媒体路由](./research/low-server-media-routes.md)的 `B`/`B_pub`/`B_i`、`N`、`R` 模型；验收必须保留 host/root NIC、每条 TURN ingress/egress、SFU ingress/egress 和供应商计费的真实 hop bytes，不得用有效载荷近似折叠物理流量。 +- 安全：应用控制面全程 HTTPS/WSS；P2P/TURN 边在 WebRTC 端点间使用 DTLS-SRTP，普通 `turn:` over TCP 不额外套 TLS，但不移除该媒体层加密。普通 SFU 是逐跳 DTLS-SRTP，只有另行实现并验收应用层 E2EE 才能对 SFU operator 保持媒体端到端机密。TURN 使用短期凭据、配额和速率限制；仓库不得包含真实密钥。 - 隐私：P2P 会向房间内好友暴露网络地址，首版威胁模型仅覆盖可信熟人；若要求隐藏 IP，必须切换为服务器转发。 - 可维护性：代码、需求、调研、项目记忆、`AGENTS.md` 和 `.codex/` 均在 Git 中跟踪；开发配置不得依赖单一操作系统的绝对路径。 - 兼容性：分享端首个目标矩阵为 Windows 10/11 的当前 Chrome/Edge；观看端还必须覆盖当前 Android Chrome 和 iOS Safari，并实测自动播放限制、横竖屏、锁屏恢复与移动网络切换。Peer-assisted 实验只允许能力完整的前台桌面 Chrome/Edge 作为 relay，手机必须保持 leaf。 @@ -107,6 +108,8 @@ - Given 标准浏览器把收到的 remote track 直接加入下游 `RTCPeerConnection`，When 检查 relay 统计和 libwebrtc 路径，Then 明确记录新的 outbound encoder、编码耗时和 CPU/GPU 限制，不把它当作 encoded forwarding 或 shared encode 成果。 - Given ADR-0004 的任一硬门槛只有在引入 Encoded Transform/DataChannel/WebCodecs 媒体、自定义拥塞控制、FEC/RTX、多树、综合评分、转码、codec ladder、浏览器 RTP 注入 hack 或放宽 host fanout 后才能满足，When 结束 bounded spike，Then 放弃并删除浏览器 relay 实验运行时代码，保留标准 P2P 和可选 SFU；后续 native shared encode sender 仍按独立 ADR 推进，但不能据此把失败的 relay 拓扑判为通过。 - Given 新观看者会使预计总上行超过安全预算，When 其请求加入，Then 系统先遵守两边 fanout 预算选择 peer-assisted 或已配置的 SFU；没有合格路线时明确等待或拒绝，而不是创建第三条 host 边或持续堆积发送队列。质量不足只对经证据确认的观看路径使用共享 `LOW`，不能降低健康路径。 +- Given TURN 或 SFU 被某条路线使用，When 媒体已抵达一至两个可靠 root，Then 其余可行 viewers 仍由 sticky、受深度与故障半径限制的 peer descendants 服务；host/root fanout 不超过二，当前 browser relay 不超过一个 child，中央服务和任一 endpoint 都不得无界集中 fanout。 +- Given 没有任何满足能力、深度、路径和恢复门槛的可靠 relay root，When 控制器穷尽 peer recovery，Then 已配置 SFU 才可把必要 viewers 作为零 descendants roots，且总数仍不超过同一个一至二 root 上限，否则明确等待或失败；不得把该异常边界解释为默认整房 SFU。 ### 质量和恢复