docs(quality): reopen native SFU adaptation gate
This commit is contained in:
1 parent
7ee212d765
commit
3e3a18f54a
7 files changed
+103
-271
No files matched your search
@@ -1,288 +1,116 @@
|
||||
# ADR-0007: Path-Isolated Representation Quality
|
||||
# ADR-0007: LiveKit-Owned SFU Representation Adaptation
|
||||
|
||||
- Status: Accepted quality/representation decision; topology capacity follows ADR-0005
|
||||
- Status: Accepted design target; implementation and deployment held for the pinned physical gate
|
||||
- Date: 2026-08-19
|
||||
- Last reviewed: 2026-08-25
|
||||
|
||||
## Context
|
||||
|
||||
Screener needs to protect a healthy viewer from another viewer's weak network or
|
||||
decoder without growing host encode work with viewer count. Draft PR #28 proved
|
||||
that one `VideoEncoder` object can be reconfigured from feedback shared by two
|
||||
Pion/WebRTC legs. Its minimum-of-two target was deliberately conservative for a
|
||||
bounded experiment. It is not an acceptable product policy: a weak leg would
|
||||
lower the shared stream for every healthy leg, and the experiment did not test
|
||||
heterogeneous estimates.
|
||||
Direct and peer paths already have independent `RTCPeerConnection` congestion
|
||||
controllers. The SFU path is different: one Host publication serves several
|
||||
subscribers, so a weak subscriber needs a lower encoded representation without
|
||||
lowering every healthy subscriber or creating one encoder per Viewer.
|
||||
|
||||
A single ordinary, non-scalable encoded representation has one resolution,
|
||||
frame cadence, and rate-control result. Forwarding those packets cannot create
|
||||
a second quality. Per-viewer quality therefore requires another encoded
|
||||
representation, a negotiated scalable representation, or transcoding at a
|
||||
relay/SFU. Per-viewer encoders would violate the game's performance budget.
|
||||
A single non-scalable `HIGH` representation cannot provide that spatial
|
||||
downshift because an SFU forwards encoded packets and does not transcode them.
|
||||
The earlier production A/B compared an always-active `LOW+HIGH` publication
|
||||
against `LOW` disabled. It showed that the always-active lower representation
|
||||
competed with `HIGH` on that constrained Host-to-SFU path; it did not test
|
||||
LiveKit-owned demand-driven layer control and does not justify a permanent
|
||||
single-`HIGH` contract.
|
||||
|
||||
## Decision
|
||||
|
||||
Use path-isolated representations:
|
||||
1. Direct and peer paths retain independent stock WebRTC congestion control.
|
||||
Viewer feedback is never aggregated into a room-wide target.
|
||||
2. The Browser Host SFU publication provides bounded VP8 `HIGH` and `LOW`
|
||||
simulcast representations. The representation count is fixed and does not
|
||||
grow with Viewer count.
|
||||
3. LiveKit Dynacast and its per-subscriber stream allocator/BWE own publication
|
||||
layer activation and the actual layer forwarded to each SFU subscriber.
|
||||
Screener does not implement a quality score, bandwidth estimator, periodic
|
||||
layer controller, or per-Viewer encoder.
|
||||
4. `HIGH` is a subscriber ceiling, not a forced delivery layer. A weak downlink
|
||||
may receive `LOW` while another subscriber remains on `HIGH` when the pinned
|
||||
LiveKit stack can sustain that contract.
|
||||
5. AdaptiveStream is a display-demand input, not a network detector. It may be
|
||||
used only when the LiveKit `RemoteVideoTrack` is attached to the actual leaf
|
||||
Viewer element. It must not derive an SFU-fed relay's subscription from that
|
||||
relay page's element size or visibility because LiveKit cannot see the
|
||||
relay's peer children.
|
||||
6. ADR-0005 remains the only route owner. Layer choice does not change topology,
|
||||
endpoint capacity, SFU admission, or decoded-stall recovery authority.
|
||||
|
||||
1. Direct/peer paths keep their requested `HIGH` target and independent stock
|
||||
WebRTC congestion control. No viewer feedback changes another path's target.
|
||||
2. The Browser Host's single SFU publication sends one `HIGH` VP8
|
||||
representation. LiveKit/WebRTC independently controls each SFU downlink,
|
||||
but the application does not publish `LOW`, enable simulcast or Dynacast, or
|
||||
add a media-layer selector.
|
||||
3. Sender/viewer evidence diagnoses capture, encode, transport, receive, and
|
||||
decode behavior. It does not change topology or command ordinary built-in
|
||||
layer changes. ADR-0005 reacts only to an exact child edge's hard connection
|
||||
failure or non-paused decoded-frame stall.
|
||||
4. A future dual-representation Browser path requires a new accepted decision
|
||||
and physical evidence that the additional representation does not reduce
|
||||
`HIGH`, game performance, or the Host-to-SFU upload budget.
|
||||
5. The accepted Browser representation count is exactly one and never grows
|
||||
with Viewer count.
|
||||
Current source and production remain the deployed single-`HIGH` implementation
|
||||
until the gate below passes and a separate implementation is integrated. That
|
||||
is deployment state, not the accepted quality target.
|
||||
|
||||
The pinned two-layer LiveKit candidate failed this gate. On exact production,
|
||||
an always-active `LOW` consumed the same Host-to-SFU congestion budget and
|
||||
reduced the progressing 1904x928 `HIGH` stream from about 23 fps to about 9 fps
|
||||
under the measured public path. `LOW` therefore cannot remain active, and the
|
||||
current Browser contract preserves only `HIGH`. A constrained SFU ingress may
|
||||
still emit fewer frames because sender ceilings are not guarantees; that is a
|
||||
stock WebRTC result, not authority to restore `LOW`, change routes, or add a
|
||||
custom controller.
|
||||
## Pinned Physical Gate
|
||||
|
||||
There is no weighted score, device ranking, machine-learning controller, custom
|
||||
media selector, or continuous room-wide optimizer. Representation evidence does
|
||||
not create route states.
|
||||
Use LiveKit server `1.13.5`, client `2.22.0`, Chrome 151, one continuously
|
||||
changing VP8 source, and one `HIGH+LOW` screen-share publication. Record adjacent
|
||||
publisher outbound-RTP deltas by RID plus subscriber inbound resolution, frame,
|
||||
and byte deltas for these states:
|
||||
|
||||
The topology and representation budgets are independent invariants. Each
|
||||
non-server endpoint follows ADR-0005 steady outbound media-copy capacity `C`
|
||||
(`1`, `2`, or `3`, default `2`); the two-edge host result below is a historical
|
||||
experiment configuration, not a fixed policy. A native relay forwards selected
|
||||
encoded packets without decoding or re-encoding. Future dual-tree or striped
|
||||
delivery may reduce host upload from about two full copies toward one copy plus
|
||||
necessary redundancy, but it does not block this decision.
|
||||
1. no subscriber, then one `HIGH` subscriber;
|
||||
2. the same subscriber requesting `LOW`, then recovering to `HIGH`;
|
||||
3. a leaf subscriber using AdaptiveStream while its attached element changes
|
||||
between large, small, hidden, and visible;
|
||||
4. an SFU-fed relay without a LiveKit-attached display element.
|
||||
|
||||
With one encoded representation, LiveKit may adapt an SFU downlink but cannot
|
||||
create a second spatial representation. That does not alter relay eligibility.
|
||||
If the exact ingress later hard-fails or stops decoding while unpaused,
|
||||
ADR-0005 reparents that Viewer as a child and retains its subtree. Viewer
|
||||
quality evidence cannot trigger route mutation or publication changes.
|
||||
The gate must establish which layers actually encode and send, not merely which
|
||||
options were requested. It passes the product target only if LiveKit owns the
|
||||
transitions, a low demand can receive a lower spatial representation, recovery
|
||||
returns to `HIGH`, and an unrelated weak or hidden leaf does not lower a healthy
|
||||
subscriber. CPU, bitrate, and FPS evidence must distinguish an inactive layer
|
||||
from one that remains encoded but is not forwarded.
|
||||
|
||||
## Evidence Contract
|
||||
If the fixed stack cannot provide demand-driven activation, stop at the
|
||||
dependency decision. Do not compensate with a Screener layer controller.
|
||||
|
||||
For diagnosis and representation acceptance, correlate the same time interval
|
||||
and stream generation across three stages:
|
||||
## Evidence Boundary
|
||||
|
||||
| Stage | Required evidence |
|
||||
| --- | --- |
|
||||
| A. Capture | `MediaStreamTrack.getSettings()` width, height, and frame rate |
|
||||
| B. Host outbound | actual width/FPS/bitrate, target or available bitrate when present, interval encode time, `qualityLimitationReason`, RTT, loss/retransmission, selected direct/SFU route, and derived negotiated video codec/profile/parameters plus `scalabilityMode` when applicable |
|
||||
| C. Viewer inbound | actual width/FPS/bitrate, loss, jitter, interval decode/drop/freeze evidence, the corresponding derived codec/profile/parameters and applicable `scalabilityMode`, and actual decode behavior |
|
||||
The retained exact-production A/B used Chrome 151 with an always-active ordered
|
||||
`q,h` publication. The 1904x928 `h` stream was about 9.2 fps / 1.37 Mbps with
|
||||
both encodings active and about 23.3 fps / 3.32 Mbps with `q` inactive. Capture
|
||||
remained near 60 fps, encode cost was about 4.46 ms per frame, loss and
|
||||
retransmission were zero, and available outgoing bitrate was about 4.81 Mbps.
|
||||
This rejects always-on `LOW` for that path. It does not establish Dynacast,
|
||||
AdaptiveStream, heterogeneous subscriber behavior, or real-game performance.
|
||||
|
||||
Interpretation is deliberately ordered:
|
||||
|
||||
| Observation | Classification |
|
||||
| --- | --- |
|
||||
| Capture is already low | capture or constraint problem |
|
||||
| Capture high, outbound low, reason `cpu` | sender encode/resource pressure |
|
||||
| Capture high, outbound low, reason `bandwidth` | congestion, uplink, GCC, or current-path pressure |
|
||||
| Outbound healthy, inbound degraded | transport or receiver-path problem |
|
||||
| Inbound metrics healthy, image still blurry | insufficient bitrate/quantization, negotiated codec/profile/layer, or display scaling problem |
|
||||
|
||||
WebRTC stats are sampled dictionaries and many useful counters are cumulative.
|
||||
The evidence sampler must use adjacent, non-overlapping deltas and rebase on a missing
|
||||
field, stream/stat ID change, or counter reset. Missing values remain unknown;
|
||||
they are not zeros.
|
||||
|
||||
Implementation is staged. First correlate A and B locally in one sampling tick,
|
||||
with an explicit interval, media/stat identity, and valid deltas. A minimal
|
||||
authenticated C report may add receive/decode diagnosis after that probe is
|
||||
trustworthy. The B/C correlation uses normalized fields derived from negotiated
|
||||
parameters/stats and actual decode behavior; it must never upload raw SDP, raw
|
||||
stats, candidate addresses, or raw device/network identifiers. Opaque
|
||||
server-issued path and connection IDs authorize diagnostic correlation only.
|
||||
Do not build a general telemetry schema.
|
||||
|
||||
Authenticated Viewer quality evidence may diagnose only its current path. It
|
||||
cannot lower Host quality, start another representation, or change topology.
|
||||
|
||||
UA, platform, and device-model detection does not participate in quality or
|
||||
relay-capacity selection. Browser capability queries can guide a bounded probe,
|
||||
but runtime encode/decode behavior is authoritative.
|
||||
|
||||
## Standard Capability Short-Circuits
|
||||
|
||||
Inspect standard capabilities in order and stop at the first accepted path
|
||||
before custom dual-representation media. Static API or pinned-protocol
|
||||
incompatibilities can close a candidate without a browser run; runtime media
|
||||
gates start only after the A+B/C evidence contract is trustworthy. These
|
||||
results do not create an application media-layer selector or route authority:
|
||||
|
||||
1. **Web P2P simulcast: rejected
|
||||
(`no-go-web-p2p-simulcast-layer-selection`).** WebRTC exposes
|
||||
`getParameters()` and `setParameters()` on `RTCRtpSender`, including the
|
||||
sender-side `active` flag, but `RTCRtpReceiver` exposes only
|
||||
`getParameters()` and no standard per-RID layer-selection setter. A sender
|
||||
toggle is scoped to that sender and PeerConnection; separate viewer
|
||||
PeerConnections have no portable shared-encoder contract. The standard API
|
||||
therefore cannot provide one shared `HIGH`/`LOW` encode whose direct P2P
|
||||
receivers independently select a layer. Chrome was not run because the
|
||||
required product semantic is absent. In particular, the normative rule that
|
||||
`active=false` stops sending an encoding is not evidence that its physical
|
||||
encoder, CPU work, or GPU allocation is released.
|
||||
2. **Pinned LiveKit two-layer simulcast: rejected for the current Browser
|
||||
contract (`no-go-livekit-always-on-low`).** Client 2.22.0 can publish a
|
||||
screen-share original plus one lower simulcast encoding, but server 1.13.5
|
||||
Dynacast enables every quality at or below the highest requested quality.
|
||||
Any `HIGH` subscriber therefore keeps `LOW` active; turning Dynacast on does
|
||||
not isolate the Host-to-SFU congestion budget.
|
||||
|
||||
The exact production gate used Chrome 151, runtime `bf32859`, a continuously
|
||||
changing 1904x928 capture, and the same public SFU path. With ordered `q,h`
|
||||
encodings active, the `h` stream stabilized near 9.2 fps and 1.37 Mbps. With
|
||||
`q` inactive from the first sender-parameter application, `h` stabilized near
|
||||
23.3 fps and 3.32 Mbps while encoding took about 4.46 ms per frame, packet
|
||||
loss and retransmission remained zero, and the selected pair reported about
|
||||
4.81 Mbps available outgoing bitrate. Source and capture remained near 60
|
||||
fps. The lower representation therefore consumed the shared ingress budget
|
||||
and materially reduced `HIGH`; the candidate fails the existing stop line.
|
||||
|
||||
The current Browser decision is one `HIGH` representation without simulcast,
|
||||
Dynacast, or a manual layer controller. The measured public `q`-inactive A/B
|
||||
remained bandwidth-constrained, so this verdict does not claim SFU 60 fps.
|
||||
Chrome 151 later closed the exact-production single-representation gate on
|
||||
release `1d87615` with one no-RID encoding and progressing publisher and
|
||||
subscriber frames over LiveKit UDP.
|
||||
Reopening a dual/native representation is a new decision with new physical
|
||||
game, encoder, and upload evidence.
|
||||
3. **Current Web/LiveKit SVC: rejected
|
||||
(`no-go-web-svc-cross-path-hardware-contract`).** WebRTC-SVC adds
|
||||
`scalabilityMode` to sender encoding parameters, but it adds no receiver
|
||||
setter for choosing a base or enhancement layer. The setting is scoped to
|
||||
one `RTCRtpSender`; separate viewer PeerConnections still have no portable
|
||||
shared-encoder or cross-connection layer-selection contract. An applied
|
||||
`scalabilityMode` proves only the browser's configured mode. The SVC
|
||||
specification explicitly says successful mode probing cannot distinguish a
|
||||
hardware encoder from a software encoder, while Media Capabilities
|
||||
`powerEfficient` and RTCStats `powerEfficientEncoder` are user-agent-defined
|
||||
signals rather than hardware guarantees.
|
||||
|
||||
Pinned LiveKit narrows the result further. Client 2.22.0 overwrites SVC
|
||||
screen-share publication to `L1T3`: one spatial resolution and three temporal
|
||||
layers, so it provides no low-resolution base and exceeds this product's
|
||||
current one-representation contract. Its `RemoteTrackPublication.setVideoQuality()`
|
||||
controls per-subscriber spatial quality, while server 1.13.5 maps
|
||||
quality/dimensions/FPS to requested spatial/temporal maxima. Actual selection
|
||||
remains codec-dependent: VP8 has a temporal selector, whereas H.264/H.265
|
||||
simulcast is spatial-only. That selection exists only on an SFU downtrack;
|
||||
it does not extend to Screener's direct/peer receivers. The current
|
||||
subscriber only requests the publication, and LiveKit documents that
|
||||
Dynacast can pause an entire SVC
|
||||
stream but not individual SVC layers. Chrome ships the outgoing-track API;
|
||||
current Firefox and WebKit WebIDL do not expose the standard
|
||||
`scalabilityMode` member, and Firefox's implementation remains behind a
|
||||
preference. A pinned-SDK Safari compatibility branch is not a portable
|
||||
standard contract. These static blockers cannot be changed by a Chrome
|
||||
media run, so no harness was written and no browser was run.
|
||||
|
||||
None may reuse PR #28's minimum-of-two target, bypass a per-edge stock WebRTC
|
||||
congestion controller, or expand the current one-representation contract.
|
||||
|
||||
## SVC Static Verdict
|
||||
|
||||
This static verdict rejects current Web/LiveKit SVC; the two-layer LiveKit
|
||||
simulcast candidate is independently rejected by its physical gate. Requested
|
||||
and post-negotiation applied
|
||||
`scalabilityMode` remain useful diagnostics, because the browser may apply a
|
||||
different mode or omit the field when none was requested. They are not proof
|
||||
of the actual hardware implementation, and neither a support query nor a
|
||||
browser resource run can add SVC's missing direct/peer receiver-selection
|
||||
contract.
|
||||
|
||||
This rejection is scoped to the current standard Web/LiveKit shortcut, not to
|
||||
scalable codecs forever. A future native sender may reopen SVC only if its media
|
||||
API positively selects a hardware encoder without silent software fallback,
|
||||
publishes at most two decodable layers, reuses one encoded output across direct,
|
||||
peer, and SFU transports, preserves independent path selection, and passes the
|
||||
same game-performance and latency gates. That would be a new native capability
|
||||
decision, not completion of this browser spike.
|
||||
Quality evidence remains diagnostic. Track availability, bitrate, resolution,
|
||||
FPS, RTT, jitter, loss, codec and current LiveKit layer cannot authorize route
|
||||
mutation or predict another parent.
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
- A weak SFU Viewer can receive a real lower spatial representation when the
|
||||
framework selects it; the SFU does not transcode.
|
||||
- The Host publishes at most two SFU representations regardless of Viewer
|
||||
count, and Dynacast is responsible for avoiding unused work where supported.
|
||||
- An SFU-fed relay keeps a subscription ceiling sufficient for its subtree;
|
||||
local DOM visibility alone cannot pause that ingress.
|
||||
- A configured 60 fps or bitrate remains a ceiling, not a delivery guarantee.
|
||||
|
||||
- Healthy viewers are not reduced to the worst path.
|
||||
- Browser SFU uses one outbound `HIGH` representation, so publication and
|
||||
ingress cost stay bounded independently of Viewer count.
|
||||
- The diagnostic evidence remains inspectable.
|
||||
## Stop Lines
|
||||
|
||||
Negative:
|
||||
|
||||
- A weak SFU downlink cannot select a separate low-resolution representation;
|
||||
stock congestion control may reduce bitrate, resolution, or encoded cadence.
|
||||
- A configured 60 fps ceiling is not a delivery guarantee on a constrained
|
||||
Host-to-SFU path.
|
||||
- Browser capability reporting cannot by itself prove hardware or sustained
|
||||
decode performance. The current SVC path is rejected before that matrix;
|
||||
custom/native candidates still require it.
|
||||
|
||||
## Product Stop Lines
|
||||
|
||||
- Do not productize PR #28's minimum-of-two bitrate or frame-rate aggregation.
|
||||
- Do not use simulcast, Dynacast, SVC, or an SFU as a semantic bypass around
|
||||
PR #28's stock-GCC/RTX stop line or per-edge congestion control.
|
||||
- Do not lower `HIGH` because one viewer reports trouble.
|
||||
- Do not create one representation or encoder per viewer.
|
||||
- Do not use UA/device identity as a quality signal.
|
||||
- Do not enable software SVC as an invisible fallback.
|
||||
- Do not publish `LOW` in the current Browser contract.
|
||||
- Do not add a composite health score or media selector when built-in adaptation
|
||||
and ADR-0005 edge recovery suffice.
|
||||
|
||||
## Verification Gates
|
||||
|
||||
- Direct/peer paths retain independent stock WebRTC congestion control.
|
||||
- An active Browser SFU publication exposes exactly one outbound video RTP
|
||||
encoding using VP8, with no `q,h` RID simulcast or second representation.
|
||||
- Host and relay endpoint media copies follow ADR-0005 steady capacity `C`;
|
||||
SFU publication accounting is unchanged by the representation decision.
|
||||
- Built-in BWE downshift and recovery remain local to each SFU subscription and
|
||||
do not change topology. Exact ingress failure is handled only by ADR-0005's
|
||||
generic child reparent operation.
|
||||
- Controlled exact-production evidence records source, capture, encode, send,
|
||||
receive, decode, render, selected transport, bitrate, loss, and encoder cost;
|
||||
it does not require 60 fps where the measured path cannot carry it.
|
||||
- No second representation sends frames or bytes, and a Viewer quality report
|
||||
cannot start one.
|
||||
- Native relays show packet forwarding without an added decode/encode stage.
|
||||
- Spoofed, stale, duplicated, or rate-excessive viewer requests have no effect.
|
||||
- Controlled capture/CPU/bandwidth/direct/SFU/receiver/display cases produce the
|
||||
classifications in the evidence table without a room-wide downgrade.
|
||||
- Do not create one encoder or representation per Viewer.
|
||||
- Do not add application quality scoring, periodic polling, manual layer
|
||||
switching, or quality-driven reparenting.
|
||||
- Do not use AdaptiveStream without the LiveKit track attachment contract.
|
||||
- Do not infer Dynacast success from configuration or source inspection alone.
|
||||
- Do not apply SFU layer policy to ordinary direct or peer paths.
|
||||
|
||||
## References
|
||||
|
||||
Primary sources checked through 2026-08-25:
|
||||
|
||||
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [W3C Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/)
|
||||
- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/)
|
||||
- [W3C WebCodecs](https://www.w3.org/TR/webcodecs/)
|
||||
- [W3C WebRTC simulcast](https://www.w3.org/TR/webrtc/#simulcast-functionality)
|
||||
- [Chrome 111 WebRTC SVC extension](https://developer.chrome.com/blog/chrome-111-beta)
|
||||
- [Chromium `RTCRtpEncodingParameters` WebIDL](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/modules/peerconnection/rtc_rtp_encoding_parameters.idl)
|
||||
- [Firefox `RTCRtpEncodingParameters` WebIDL](https://searchfox.org/firefox-main/source/dom/webidl/RTCRtpParameters.webidl)
|
||||
- [Firefox WebRTC-SVC implementation status](https://bugzilla.mozilla.org/show_bug.cgi?id=1571470)
|
||||
- [WebKit `RTCRtpEncodingParameters` WebIDL](https://github.com/WebKit/WebKit/blob/main/Source/WebCore/Modules/mediastream/RTCRtpEncodingParameters.idl)
|
||||
- [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/)
|
||||
- [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/)
|
||||
- [LiveKit client 2.22.0 room defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/defaults.ts)
|
||||
- [LiveKit client 2.22.0 room and Dynacast options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/options.ts)
|
||||
- [LiveKit client 2.22.0 room option merge](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts)
|
||||
- [LiveKit client 2.22.0 SVC defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts)
|
||||
- [LiveKit client 2.22.0 screen-share SVC and publish-option handling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
|
||||
- [LiveKit client 2.22.0 SVC encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts)
|
||||
- [LiveKit client 2.22.0 subscriber quality control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts)
|
||||
- [LiveKit client 2.22.0 room options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/options.ts)
|
||||
- [LiveKit client 2.22.0 AdaptiveStream track attachment](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteVideoTrack.ts)
|
||||
- [LiveKit client 2.22.0 subscriber quality ceiling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts)
|
||||
- [LiveKit client 2.22.0 Dynacast handling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
|
||||
- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go)
|
||||
- [LiveKit client 2.22.0 Dynacast layer control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalVideoTrack.ts)
|
||||
- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go)
|
||||
- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go)
|
||||
@@ -17,7 +17,7 @@ Last updated: 2026-08-25
|
||||
- One Host publication may serve all SFU subscriptions. Its exact generation owns one ingress handle and exact Viewer subscription handles own egress; later SFU children reuse the publication, while reserved, committed, and draining handles remain charged until the managed room is deleted and proven absent. Actual SFU usage release wakes only rooms that previously lost a candidate to global admission, and SFU-fed and peer-fed endpoints use the same provisional-child transaction. Parent choice hard-filters authority, reachability, cycles, effective capacity, and producer transition, then orders by shallowest result, remaining steady sender capacity, stable join order, and peer identity; server admission is checked at the current cursor. Acyclicity and room admission bound maximum depth; depth is observed but has no separate hard cap. Healthy committed edges remain sticky: the controller repairs waiting, invalid, departed-parent, and overflow children but does not periodically rebalance the graph. Raw IP, claimed NAT type, UA, geography, all-pairs probes, weighted scores, and periodic room rebalancing are not route inputs.
|
||||
- Ordinary peer ICE is STUN-only. Browser LiveKit publisher and subscriber PCs use no external ICE server, retain LiveKit-signaled UDP candidates, and let standard ICE nominate a non-relay pair; deployment STUN remains available to ordinary peers and LiveKit server-side public-IP discovery. This Browser SFU ICE-server isolation avoids the reproduced external-STUN/SFU association conflict without constraining candidate type or count and without adding TURN, another route, a port, or a timeout. Credentials, room secrets, candidate details, and diagnostic data remain private and narrowly scoped. A future strict-firewall transport requires separate evidence and must remain inside LiveKit rather than becoming another Screener route candidate.
|
||||
- Browser peers use existing periodic WebRTC/LiveKit sampling to maintain one exact-route decoded-progress timestamp. New frames or route/connection changes reset it; a non-paused deadline reports once. An active SFU cadence treats an absent track/report or stats failure as no progress without fabricating metrics and stops only on deactivation/teardown. Other current-path stats remain diagnostic and cannot prove another parent better.
|
||||
- ADR-0007 accepts one Browser SFU VP8 `HIGH` representation without `LOW`, simulcast, Dynacast, or an application layer selector. Weak Viewers cannot change Host publication, healthy paths, or topology; dual representation requires a new decision and physical proof.
|
||||
- ADR-0007 targets fixed VP8 `HIGH+LOW` SFU simulcast with LiveKit-owned Dynacast/BWE. AdaptiveStream is leaf-display only and cannot govern relay ingress. Source and production stay single-`HIGH` pending the pinned gate.
|
||||
- Browser capture requests available share audio by default on initial capture and source switch, with an explicit video-only warning when no audio track is returned. Its screen-audio control offers live-switchable 64/128/256 kbps sender ceilings and defaults to 128. The last-wins quality wire owns desired state; each endpoint serially applies and reads back current Host/relay/SFU audio senders, exposes local failures, and gives future senders the latest desired value. Opus remains fixed, and without an applied-ack wire the Host does not claim room-wide atomic convergence. These are configuration ceilings, not fidelity claims.
|
||||
- Browser display video leaves `contentHint` unset so Chromium retains its screen-capture classification; returned audio tracks keep `contentHint = "music"`. Browser video is fixed to VP8 across direct, browser-relay, and SFU paths; codec selection, fallback to another media codec, quality-state/wire fields, and share-lifetime codec branches are not product features. Actual codec/profile/encoder stats remain diagnostic only.
|
||||
- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`, with the middle `1080p30` as the accepted default. `480p` means only an advanced `854x480` resolution choice whose FPS and bitrate remain independently selected; it is not a fourth recommended profile.
|
||||
@@ -40,7 +40,7 @@ Last updated: 2026-08-25
|
||||
|
||||
## Current Priority
|
||||
|
||||
1. Fix and deploy the active-SFU decoded-progress cadence and repeat controlled recovery; then finish real-network route, screen-audio, and real-game A/V acceptance. VP8 acceleration, mobile lifecycle, Native, distribution, and repository simplification remain later.
|
||||
1. Complete the pinned LiveKit native-representation gate and settle its implementation decision; then deploy the active-SFU decoded-progress cadence, repeat controlled recovery, and finish real-network route, screen-audio, and real-game A/V acceptance. VP8 acceleration, mobile lifecycle, Native, distribution, and repository simplification remain later.
|
||||
|
||||
The desktop Host background/minimized report is deferred until it reproduces on current production with a real game and synchronized media plus CPU/GPU evidence. The bounded current-Chrome screening found no immediate Host-page lifecycle drop, and removal of the old harmful `contentHint = "motion"` remains a plausible confound rather than a proven background fix. Mobile Viewer playback and assigned-relay survival remain a separate lifecycle gate.
|
||||
|
||||
|
||||
@@ -483,22 +483,25 @@ default. Chromium currently maps its configured per-stream mode into the
|
||||
outbound stats field; other browser implementations may omit any optional
|
||||
member, which remains null rather than a capability conclusion.
|
||||
|
||||
## Superseded Two-Layer Candidate Analysis (2026-08-22)
|
||||
## LiveKit Native Representation Boundary
|
||||
|
||||
This retained analysis explains the candidate that the 2026-08-25 physical gate
|
||||
rejected. It does not define current product truth, implementation scope, or a
|
||||
next gate; ADR-0007 owns the accepted single-`HIGH` decision.
|
||||
ADR-0007 owns the accepted target: fixed `HIGH+LOW` VP8 simulcast with layer
|
||||
activation and per-subscriber selection delegated to LiveKit. Current source and
|
||||
production remain single-`HIGH` until the pinned physical gate measures actual
|
||||
RID activity and subscriber output.
|
||||
|
||||
The candidate assumed one shared `HIGH+LOW` SFU publication could preserve
|
||||
healthy `HIGH` while LiveKit selected a lower representation per constrained
|
||||
subscriber. Static review established two limits before the runtime gate:
|
||||
ordinary WebRTC receivers have no portable per-RID selection API across
|
||||
separate PeerConnections, and LiveKit 1.13.5 Dynacast cumulatively enables every
|
||||
quality below the highest requested quality. A `HIGH` subscriber therefore
|
||||
keeps `LOW` active. The 2026-08-25 exact-production A/B then proved that this
|
||||
always-active lower representation materially reduced `HIGH`, so ADR-0007
|
||||
rejected the candidate. Sender/viewer evidence remains diagnostic and does not
|
||||
change topology or command a representation.
|
||||
separate PeerConnections. Source inspection also indicates that LiveKit 1.13.5
|
||||
Dynacast cumulatively enables qualities below the highest requested quality, so
|
||||
a `HIGH` subscriber may keep `LOW` active; this is a gate hypothesis, not a
|
||||
runtime verdict. The 2026-08-25 exact-production A/B proved only that the prior
|
||||
always-active lower representation materially reduced `HIGH`. It did not enable
|
||||
Dynacast or AdaptiveStream and therefore cannot reject native demand-driven
|
||||
adaptation. Sender/viewer evidence remains diagnostic and does not change
|
||||
topology or command a representation.
|
||||
|
||||
### Historical SFU Source Gate
|
||||
|
||||
|
||||
+1
-1
@@ -29,7 +29,7 @@ This is the current execution index. Git history owns completed timelines; [veri
|
||||
|
||||
## Current Milestone
|
||||
|
||||
1. Correct and deploy the active-SFU decoded-progress cadence, repeat the controlled recovery transaction, then finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model.
|
||||
1. Run ADR-0007's pinned LiveKit native `HIGH+LOW`/Dynacast/AdaptiveStream gate and settle the framework-owned representation decision. Then correct and deploy the active-SFU decoded-progress cadence, repeat the controlled recovery transaction, and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Application layer control, Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model.
|
||||
|
||||
## Active Boundaries
|
||||
|
||||
|
||||
+2
-1
@@ -6,7 +6,8 @@ Only items in **Now** are executable after their stated decision gate. A branch
|
||||
|
||||
## Now
|
||||
|
||||
1. **Finish real-network route acceptance.** First restore the existing active-SFU decoded-progress cadence when its track or stats report disappears, deploy it, and repeat the controlled recovery transaction. Then measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, screen-audio continuity and real-game A/V sync across representative IPv4/IPv6 NATs and mobile networks. Include an all-UDP-blocked case to establish bounded explicit failure. Do not add another watchdog, port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting.
|
||||
1. **Settle native SFU representation adaptation.** Run ADR-0007's pinned LiveKit `1.13.5` / client `2.22.0` Chrome gate for actual `HIGH+LOW` RID activity, subscriber layer selection/recovery, leaf AdaptiveStream sizing/visibility, and the no-attached-element relay boundary. Accept only framework-owned Dynacast/stream-allocation behavior; do not add a Screener quality controller.
|
||||
2. **Finish real-network route acceptance.** First restore the existing active-SFU decoded-progress cadence when its track or stats report disappears, deploy it, and repeat the controlled recovery transaction. Then measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, screen-audio continuity and real-game A/V sync across representative IPv4/IPv6 NATs and mobile networks. Include an all-UDP-blocked case to establish bounded explicit failure. Do not add another watchdog, port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting.
|
||||
|
||||
## Accepted Later Roadmap
|
||||
|
||||
|
||||
+2
-2
@@ -188,7 +188,7 @@ Viewer 本地连接详情只从当前 route 的唯一 inbound screen video/audio
|
||||
|
||||
`frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限;浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据,但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机,先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。
|
||||
|
||||
质量自适应采用 ADR-0007 的路径隔离单表示,而不是全房间按最差链路降档。Direct/peer 保留每条 PeerConnection 的 stock WebRTC GCC。Browser Host 的唯一 SFU publication 发布一层 `HIGH` VP8,不启用 `LOW`、simulcast、Dynacast、SVC 或应用媒体层 selector。Host 到 SFU 的 stock WebRTC 拥塞控制决定该共享 ingress 的实际码率、分辨率和帧率;LiveKit 为每个 Viewer 独立发送下行,但不能从单一编码表示生成另一档画质。应用证据只用于诊断,弱下行不命令 Host 降档或改变拓扑。任何未来 Browser 双表示候选都必须以新决策证明额外表示不挤占 `HIGH` 的上传、编码和游戏预算。
|
||||
质量自适应采用 ADR-0007 的路径隔离框架能力,而不是全房间按最差链路降档。Direct/peer 保留每条 PeerConnection 的 stock WebRTC GCC。Browser Host 的唯一 SFU publication 以固定 `HIGH+LOW` VP8 simulcast 提供两种空间表示,LiveKit Dynacast 负责停用未消费层,subscriber 的 stream allocator/BWE 独立选择实际下行层;`HIGH` 只是上限。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。AdaptiveStream 只适用于 remote track 直接 attach 到实际播放元素且不承担 peer children 的 leaf Viewer;relay ingress 不能跟随本地 DOM 可见性。当前单层 `HIGH` 源码与生产保持不动,直到 pinned LiveKit 物理门禁证明层启停、弱下行选择和恢复行为符合该模型。
|
||||
|
||||
质量证据只用于诊断,不形成全房分数或 parent authority。路由只消费 exact child edge 的 `failed/closed` 和非暂停 decoded-frame stall。prepare/commit/abort 绑定 authenticated sessions、share generation、base/pending revision 与 candidate connection;candidate child 的第一帧新 decoded video 是唯一应用层 media-ready,ICE connected 不能单独提交。
|
||||
|
||||
@@ -212,7 +212,7 @@ SVC 的标准能力门禁已经静态关闭为 `no-go-web-svc-cross-path-hardwar
|
||||
|
||||
路由状态只有 committed graph 和一个 pending child operation;该对象聚合 candidate list/cursor、一个 current candidate/预留和一个总 deadline。relay 自身 ingress 失效时,它作为 child 进入同一 reparent 并保留 subtree;disconnect 或 effective capacity 降低时让 direct/overflow children 逐个回到 reconcile。多个下游 edge 若各自失效,会独立迁移并自然排空 relay。权威 pause abort operation、保持 active graph、抑制 stall;resume 重新 reconcile。
|
||||
|
||||
标准表示能力排在任何自定义双表示媒体实现之前。当前 Web P2P simulcast 缺少 receiver 选 RID 与跨 PeerConnection shared-encode 契约;当前 Web/LiveKit SVC 也缺少 direct/peer 独立选层和可移植硬件保证。Browser SFU 因此只发布单层 `HIGH`;具体表示门槛由 ADR-0007 和[实时质量调研](./research/realtime-quality-adaptation.md)持有,不得绕过 stock WebRTC congestion control。
|
||||
标准表示能力排在任何自定义媒体实现之前。Web P2P simulcast 缺少 receiver 选 RID 与跨 PeerConnection shared-encode 契约;LiveKit SFU 则原生拥有 per-subscriber 层选择与 Dynacast。Browser SFU 只在 ADR-0007 的 pinned-stack 门禁通过后发布固定 `HIGH+LOW`,并完全委托框架启停和选择;不得把该能力扩散到 direct/peer,也不得绕过 stock WebRTC congestion control。
|
||||
|
||||
## 移动分享端能力边界与排期
|
||||
|
||||
|
||||
+1
-1
@@ -28,7 +28,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- P2P 优先:每条普通逻辑媒体边先尝试 direct/peer UDP,服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布;每名 Viewer 同时至多一个 active upstream,失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate;Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction,并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时,同一 reconcile 将 affected Host direct child(若无则稳定顺序中最新的 connected Host direct child)作为本次 child operation,创建唯一 Host publication 并把该 child 变为首个 SFU subscriber,subtree 不动;publication 取代释放的 Host peer slot,下一次扫描再处理原受限 Viewer,不建立 bootstrap 状态机。
|
||||
- NAT 与传输边界:STUN 必须可用,所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`,保留 LiveKit signaling 交付的授权 SFU UDP candidates,并由标准 ICE 选择 nominated non-relay pair;不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC,也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威;Screener 不合成 remote candidate、不预测端口、不分类 NAT,也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity;成功即停止,失败、超时或绑定失效即撤销,direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP,全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers,再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media;当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输,必须把它作为 LiveKit 内部能力重新决策,不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target;HTTPS/WSS 继续独立使用 TLS/TCP。
|
||||
- 质量控制:推荐档位始终只有 1080p60(8 Mbps)、1080p30(5 Mbps)、720p30(3 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用 `balanced`(平衡),`maintain-resolution`(清晰优先)、`maintain-framerate`(流畅优先)保留为显式选择。Browser 视频固定使用 VP8;界面、质量状态和 wire 不提供 codec 选择或切换。分享期间可直接换视频档位和 64/128/256 kbps 音频 sender ceiling,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由浏览器决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。不做自定义动态 FPS、综合评分或周期性全房间调档。
|
||||
- 路径隔离表示:direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC,不把 viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 只发送一层 `HIGH` VP8;不启用 `LOW`、simulcast、Dynacast、SVC 或应用媒体层 selector。SFU 每个 Viewer 的独立下行仍由 LiveKit/WebRTC 拥塞控制,弱下行不能命令 Host 降档或改变拓扑;单一编码表示也不能凭转发生成第二档画质。任何 Browser 双表示候选必须先以新决策证明不会挤占 `HIGH` 的上传、编码和游戏预算。
|
||||
- 路径隔离表示:direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC,不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 提供固定两层 VP8 `HIGH+LOW` simulcast;LiveKit Dynacast 与每个 subscriber 的 stream allocator/BWE 负责启停发布层和选择实际下行层,Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。`adaptiveStream` 只可用于将 LiveKit remote track 直接 attach 到实际播放元素的 leaf Viewer;SFU-fed relay 的订阅不得由本地元素尺寸或可见性决定,因为其 peer children 的需求不在 LiveKit 内。表示选择不改变拓扑、端点容量或路由失效权威。当前源码和生产仍是单层 `HIGH`,必须先通过 ADR-0007 的 pinned-stack 物理门禁才可实现和部署上述目标。
|
||||
- 路由失效输入:应用不推断 parent-wide 质量,也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`,或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时,使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`,route/connection变化或新帧重置、暂停抑制,同一 edge只报告一次,不增加轮询器。Active SFU route 暂时没有 video track、stats report 为空或 stats 读取失败时,现有采样 cadence 仍把该次作为无 decoded progress 推进 deadline;它不得停表、伪造连接指标或立即判定 edge 失效,只有 route 停用或 teardown 才停止该 cadence。该 operation 把失败的 exact direct parent 在当前 fact version 直接视为已尝试,先选择其他 direct parent,再进入有界 SFU suffix;新的外部事实才允许原 direct tuple 重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时,它作为child换父并保留subtree;disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。
|
||||
- 断线恢复:临时网络变化先交给 WebRTC/LiveKit 的 ICE、consent、重连和流状态。Direct edge 可在 exact current parent/child sessions 内采用 framework rebuild 的新 connection ID而不改变topology。应用只在上述exact edge失效后运行一次有界reconcile;任一步成功即停止。候选耗尽时hard-invalid、confirmed-departed或overflow edge被退休并进入明确waiting,healthy bootstrap edge保留。所有异步结果受authenticated session、share generation、base/pending route revision与candidate connection identity约束,未受影响分支保持sticky。权威暂停abort未提交candidate、保留active graph、抑制stall,并在resume后重新reconcile。
|
||||
- 同一 operation 的一个总 deadline 只按实际存在的 `direct | sfu` 语义阶段等分,不配置每个 candidate 的 magic timeout。阶段内的硬失败继续推进 cursor;silent ICE 到达 direct 阶段边界时跳过剩余 direct tuple,必须把派生份额留给 SFU。20 名 Viewer 同时加入时,每名 child 的 operation deadline 不会被人数除分,但单一 room-serial operation 可能形成线性排队尾延迟;在接受另一种并发模型前,只测量 queue wait、candidate start、first decoded frame 与 final route 的 p50/p95/max,不用 NAT 猜测、TCP probe 或 magic timeout 绕过排队。
|
||||
|
||||
Reference in new issue
Block a user