fix(media): reapply negotiated P2P quality settings [skip ci]

This commit is contained in:
AAA authored and GitHub committed 2026-08-21 15:55:37 +08:00
1 parent ef36543d6e
commit 4a216c8b92
6 files changed
+114 -19

No files matched your search

@@ -145,7 +145,7 @@ evidence windows remain limited to topology eligibility and diagnostics:
required product semantic is absent. In particular, the normative rule that
`active=false` stops sending an encoding is not evidence that its physical
encoder, CPU work, or GPU allocation is released.
2. **Pinned LiveKit two-layer simulcast: implemented default-off runtime candidate.**
2. **Pinned LiveKit two-layer simulcast: implemented; acceptance remains open.**
Client 2.22.0 can publish a screen-share original plus one lower simulcast
encoding. `RemoteTrackPublication.setVideoQuality(HIGH)` sets a per-subscriber
spatial-quality ceiling. Server 1.13.5 maps quality, dimensions, and FPS to
@@ -172,13 +172,15 @@ evidence windows remain limited to topology eligibility and diagnostics:
before sender activation/deactivation. The gate must prove that a healthy
P2P/`HIGH` path is unchanged, no third layer appears, the expected layer is
actually received, and hardware encoder, game FPS/p1 low, CPU/GPU, interval
encode cost, host upload, and `HIGH+LOW` bytes fit budget. The default-off
exact-room candidate now configures exactly two encodings, leaves Dynacast at
its default `false`, and sets each subscriber's ceiling to `HIGH`. It has not
been browser-tested or enabled by default, so no runtime performance claim
follows. If the always-on cost fails, test manual standard
sender activation/deactivation next; custom/native dual encoding follows
only if built-in and manual standard primitives fail.
encode cost, host upload, and `HIGH+LOW` bytes fit budget. Every active SFU
publication now configures exactly two ordered `q,h` encodings, leaves
Dynacast at its default `false`, and sets each subscriber's ceiling to
`HIGH`; the two-layer publication is not behind a separate quality flag.
Per-subscriber BWE, hardware cost, and root-with-children behavior remain
unverified, so no runtime performance claim follows. If the always-on cost
fails, test manual standard sender activation/deactivation next;
custom/native dual encoding follows only if built-in and manual standard
primitives fail.
3. **Current Web/LiveKit SVC: rejected
(`no-go-web-svc-cross-path-hardware-contract`).** WebRTC-SVC adds
`scalabilityMode` to sender encoding parameters, but it adds no receiver
+2 -2
View File
@@ -10,7 +10,7 @@ Last updated: 2026-08-21
- Minimize server bandwidth. Ordinary peer ICE is STUN-only. Failure tries restart, rebuild, alternate peer and bounded SFU/UDP roots; only then may the controller authorize short-lived TURN for one selected exceptional edge before clear failure. TURN is transport, not topology.
- The current release budget is two downstream edges per non-server node, including ordinary Web relays; upstream receive is free and UA/visibility do not change it. SFU normally feeds one or two roots that retain peer descendants; separately capped server edges may serve exceptional viewers that cannot attach behind a healthy root. A future evidence-driven tier may change the ceiling only through the owning requirement and design.
- Two-tree packet/layer striping may reduce endpoint upload toward one stream bitrate, but needs a bounded multi-parent, loss, sync, churn, and latency experiment; it is not in the current full-stream chains.
- Direct/peer keeps per-PC stock GCC. SFU paths start at a `HIGH` ceiling and share at most one `LOW`; next test LiveKit exactly-two built-in BWE on zero-descendant leaves. If it passes, no app media selector is built. Explicit quality, manual activation, then custom/native are later fallbacks; always-on `LOW` needs resource gates.
- Direct/peer keeps per-PC stock GCC. Active SFU publishes exactly `q,h` at a `HIGH` ceiling; built-in BWE may forward `LOW`. The zero-descendant quality/resource gate is next; if it passes, no app media selector is built. Explicit quality, manual activation, and custom/native remain fallbacks.
- A later explicit quality fallback evacuates children under a generation guard first. Autonomous BWE enters `suspect`; confirmation evacuates children, and no confirmed `FALLBACK` parent remains. Root-with-children impact is a default-on gate; capacity returns after longer recovery plus cooldown. Self-report alone never triggers it.
- Site access protects room creation, Host publication, and every code-only Viewer entry. A valid room-scoped fragment grant bypasses that site gate; otherwise site access is checked before public-watch or a private room password. Persist neither raw credential; keep accounts, ACLs, users, and session tables out.
- Keep decisions, snapshots, research, code, `AGENTS.md`, and `.codex/` in Git; rewrite memory/status in place. Research current primary sources before material work and reject speculative machinery.
@@ -52,7 +52,7 @@ Last updated: 2026-08-21
- Production uses nginx, Node.js 24.19.0, coturn 4.17.2, and LiveKit 1.13.5 on UDP 7882 (TCP fallback off). Ordinary peer ICE receives no TURN; the selected-edge tuple is configured only for the current controller edge, with no credential pre-advertised to ordinary peers. Coturn retains the old authenticated-relay ports. Services are healthy with zero restarts.
- ADR-0005 rejects `PEER_ASSISTED_ROOM_IDS`; `PEER_ASSISTED_MEDIA=true` enables every room and ordinary peers stay STUN-only. Initial or active Host ingress may consume one bound relay-only grant; ready/abort clears it and failure restores peer baseline. Stale room-ID config fails startup; `screener-v2` is the only deployed wire.
- Chrome 151/LiveKit localhost A/B cut failure-to-active/render from 1.481/2.257 seconds to 0.200/0.320; 31 new frames and 25 ms sampling kept host edges at two. It is headless synthetic 720p30 and includes SDK/network prewarm, not public-network evidence.
- Deployed Web uses LiveKit 2.22.0 `q,h`. Chrome 151 proves local SFU/UDP and, separately, active selected TURN/UDP Host ingress against production media services with Viewer frame progress and clean stop. Initial/peer-selected relay, BWE, and C+B remain open.
- Web SFU uses LiveKit 2.22.0 `q,h`; local SFU/UDP and selected-TURN ingress work, while BWE remains open. P2P now reapplies its selected profile after each answer on the same sender. A local A/B supports the lifecycle candidate, but natural ramp, real capture, production, and SFU validation remain open.
- Chrome 151 five-Viewer 720p30 cap2 evidence: Host/relay fanout two; both relay children +80 decoded frames/~1.1 MB; every Viewer decoded; clean stop. Functional only; resource and heterogeneous-network behavior remain open.
## Provisional Quality Targets
+59 -6
View File
@@ -1,6 +1,6 @@
# Realtime Screen-Share Quality Adaptation
- Research date: 2026-08-19
- Research date: 2026-08-21
- Scope: realtime game screen sharing in the browser
- Status: implementation input; real-device quality remains unverified
@@ -88,6 +88,51 @@ are ceilings rather than targets. Raising them cannot repair CPU or bandwidth
limitation and should only follow evidence that the encoder is already pinned
to the ceiling while spare transport capacity remains.
## Initial Balanced Lifecycle Evidence
On 2026-08-21 one bounded local Chrome 151 run used exact source `b447ab6`, a
synthetic 1920x1080 at 60 fps capture, and one direct UDP Viewer. After the
initial connection had existed for about 22 seconds, the Host outbound and
Viewer inbound were both 1280x720 at 57 fps and about 7.03 Mbps. The sender
reported `qualityLimitationReason=bandwidth`, despite about 12.59 Mbps of
available outgoing bitrate. Reapplying the identical 1080p60 `balanced` setting
through the real Host control produced 1920x1080 at 55 fps and about 7.11 Mbps
after 5.2 seconds, with the limitation cleared. Both ends reported zero packet
loss and no Viewer freeze.
The route revision, PeerConnection identity, RTP SSRC, RTP stats object, and
track identifier all remained unchanged. This rules out a new route, sender,
or capture generation as the source of that observed transition. It does not
prove visual quality, production behavior, or SFU behavior. Earlier samples in
the same run remained at 640x360 through roughly 20 seconds and reached 720p in
the two-second window immediately before the control action, so continued
stock BWE ramp-up remains a causal confound. No second browser run was made.
The Host control applies same-track capture constraints before updating sender
parameters, so this run also cannot isolate those two standard calls.
Static review does not support the simpler claim that Chromium discards every
pre-negotiation `setParameters()` call. Current libwebrtc stores sender init
parameters while no SSRC exists and transfers them into the negotiated sender.
The observed transition followed the same-setting action, but this single run
cannot attribute it to sender lifecycle or the reapply itself. The bounded
implementation change refreshes the same video-sender profile after each
accepted answer, serialized with existing sender mutations. It
changes no track, PeerConnection, ceiling, degradation preference, or room
setting and adds no periodic controller. A post-change real-capture P2P check
must still confirm that the negotiated sender reapply, without a new capture
generation, clears the reported startup condition.
This evidence does not justify an SFU change. The current SFU publisher always
publishes the ordered `q,h` pair when an SFU route is active and configures its
sender after SDK publication. The pinned client publication defaults to
`HIGH`, and Screener also requests a `HIGH` subscriber ceiling immediately
after subscribing. A low SFU receive layer can still be a valid server BWE
choice. Diagnose that case by correlating publisher high-layer stats with the
Viewer's actual inbound layer; do not infer a missing `HIGH` request or change
the application policy without those measurements. LiveKit's pinned client
also contains an SDP start-bitrate mitigation for initial video blur, but
Screener's accepted boundary forbids adding application SDP bitrate hacks.
The same production release is also reported to reduce game-stream frame rate
and consume noticeable Host resources. That report applies only to deployed
`769de201f7cc`, not automatically to the newer diagnostics on `main`. The Web
@@ -338,11 +383,13 @@ hardware encoder, game FPS/p1 low, CPU/GPU, interval encode cost, upload, and
per-layer byte budgets. Before default enablement, a separate root-with-children
gate must inject an autonomous downshift, observe `suspect`, evacuate after
bounded confirmation, and limit temporary descendant impact; no confirmed
`FALLBACK` root may retain children. At `main` commit `13b8dad`, Screener's
default-off publisher configures exactly two ordered `q`/`f` encodings, leaves
Dynacast at `false`, disables backup-codec publication, and its subscriber sets
a `HIGH` ceiling after subscribing. This is implemented configuration, not
browser or performance evidence. The subscriber also does not attach a
`FALLBACK` root may retain children. Screener's current publisher always
configures exactly two ordered `q`/`h` encodings whenever an SFU publication is
active, leaves Dynacast at `false`, disables backup-codec publication, and its
subscriber sets a `HIGH` ceiling after subscribing. The publication is not
behind a separate quality flag; its per-subscriber BWE and resource acceptance
gates remain unverified. This is implemented configuration, not browser or
performance evidence. The subscriber also does not attach a
`RemoteTrack`, so SDK `adaptiveStream` is not directly usable without changing
that ownership; built-in SFU bandwidth adaptation does not depend on enabling
that feature. If built-in selection fails the product gates, test explicit
@@ -504,6 +551,10 @@ resolution, frame rate, or bitrate.
- Every sender update derives from `getParameters()`, calls `setParameters()`,
then reads requested/applied bitrate, frame rate, scale, and preference.
Rejection or browser rewriting is visible rather than console-only.
- A P2P sender receives that update before its first offer and once more after
each accepted answer. The negotiated reapply uses the current selected
profile and existing sender; it neither rebuilds the connection nor changes
the browser-owned degradation decision.
- One strict room setting is last-wins for current/future peer relays and the
configured SFU publisher. Ordinary P2P keeps that state local and does not add
it to the authenticated wire.
@@ -596,6 +647,7 @@ is a separate optimization.
- [MDN `RTCRtpSender.setParameters()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters)
- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc)
- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc)
- [libwebrtc pre-negotiation sender parameter storage](https://webrtc.googlesource.com/src/+/refs/heads/main/pc/rtp_sender.cc)
- [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc)
- [libwebrtc conditional screen-share degradation mapping](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_stream_encoder.cc)
- [libwebrtc adaptation overview](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/adaptation.md)
@@ -606,6 +658,7 @@ is a separate optimization.
- [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/)
- [LiveKit client 2.22.0 SVC defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts)
- [LiveKit client 2.22.0 screen-share SVC override](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
- [LiveKit client 2.22.0 start-bitrate negotiation](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/PCTransport.ts)
- [LiveKit client 2.22.0 SVC encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts)
- [LiveKit client 2.22.0 subscriber quality control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts)
- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go)
+3 -3
View File
@@ -15,7 +15,7 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
## Current Snapshot
- Capture precedes room creation; source/quality changes and AV pause preserve peers. Recommended profiles and advanced defaults use `balanced`; clarity/fluid remain explicit. The browser chooses degradation; readback/stats report actual behavior.
- Capture precedes room creation; source/quality changes and AV pause preserve peers. Recommended profiles and advanced defaults use `balanced`; clarity/fluid remain explicit. P2P reapplies the selected profile after each answer on the same sender; the browser still owns degradation and stats/readback expose it.
- Production access is `screener-v2`: a valid private fragment grant enters directly; every code-only Viewer must first hold site access, after which public-watch accepts the code and private rooms still require their room password. `SITE_ACCESS_PASSWORD`, `/api/site-access`, and the site-access cookie are deployed atomically; raw credentials are never stored.
- Source UI uses adjacent nickname editing, distinct site/room prompts, and a shared room-code form. Status and details stay neutral before media proof; current evidence labels green P2P/yellow `SFU fallback`, while relay evidence adds `TURN` plus only a locally observed protocol.
- Native v2 is source-only (memory rooms, 300s reclaim); it follows authenticated direct-child assignments up to two and fails unsupported SFU/selected ingress boundedly.
@@ -42,10 +42,10 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
- The `691863e` cutover ran no browser/media/SFU/TURN/performance canary. Earlier `16f6eab` proof covers Host ingress only; audible quality, last mile, external cohorts, admission rescue, resources/performance and game blur remain open.
- Mobile is Viewer-only. Its persistent audible media element keeps media/signaling active while the page lives; iOS lock-screen, reclamation and background reconnection remain device gates. AirPlay/system mirroring is local output; live WebRTC `srcObject` has no portable in-app TV-output contract.
- Silent partitions can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; this remains unverified.
- The former release reportedly sustained bandwidth-limited blur on a capable LAN; Host refresh recovered while Viewer refresh did not. The new deployment has not yet reproduced or cleared it.
- Production startup blur remains open. One local Chrome synthetic same-`balanced` A/B kept route/PC/SSRC/track and coincided with 720p -> 1080p without loss/freeze, but natural ramp prevents causal, production, or SFU proof.
- ADR-0006 has one Viewer proof; two-edge/FIFO, hardware, endurance, public/downloaded-package, and browser-diversity proof remain open.
- Production fanout is Host2/Viewer2 and rejects child3. Relay resource behavior remains open.
- PR #49/#50 BWE is unverified/default-off: test zero-child leaves, then root impact. No local per-leaf UDP shaper; resume with Linux `tc` or public canary, never CDP. Web P2P/SVC shortcuts remain no-go.
- Active SFU already publishes `q,h`; only BWE/resource/zero-child/root gates are unverified. No local per-leaf UDP shaper; resume with Linux `tc` or public canary, never CDP. Web P2P/SVC shortcuts remain no-go.
- Headful fragment-to-sessionStorage consumption, invitation rotation after migration, and request/log leak inspection remain production UX gates. Accounts stay out.
- Structure debt: split `HybridMediaRouter`, `HostPage`, `SignalingServer`, and `ViewerPage` only at proven consumer boundaries, never by file length.
+9
View File
@@ -224,6 +224,15 @@ export class HostPeer {
}
await this.connection.setRemoteDescription(payload.description);
await this.flushCandidates();
await this.enqueueSenderMutation(async () => {
const videoSender = this.videoSender;
if (this.disposed || !videoSender) {
return false;
}
// Refresh requested bounds after negotiation; browser adaptation
// remains authoritative.
return this.configureSender(videoSender);
});
} else if (this.connection.remoteDescription) {
await this.connection.addIceCandidate(payload.candidate);
} else if (this.pendingCandidates.length < MAX_PENDING_CANDIDATES) {
+31
View File
@@ -480,6 +480,37 @@ describe("HostPeer source replacement", () => {
});
});
it("reapplies the selected video profile after the answer", async () => {
const video = createTrack("video", "video");
const audio = createTrack("audio", "audio");
const peer = createPeer(createStream(video, audio));
await expect(peer.start()).resolves.toBe(true);
const connection = FakePeerConnection.latest!;
expect(connection.senders[0]?.setParameters).toHaveBeenCalledOnce();
await peer.acceptSignal({
kind: "description",
connectionId: peer.connectionId,
description: { type: "answer", sdp: "test-answer" },
});
expect(connection.remoteDescription).toMatchObject({
type: "answer",
sdp: "test-answer",
});
expect(connection.senders[0]?.track).toBe(video);
expect(connection.senders[0]?.replaceTrack).not.toHaveBeenCalled();
expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2);
expect(
connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0],
).toMatchObject({
degradationPreference: "balanced",
encodings: [{ maxBitrate: 3_000_000, maxFramerate: 30 }],
});
expect(connection.senders[1]?.setParameters).toHaveBeenCalledOnce();
});
it("serializes initial sender configuration with a live profile update", async () => {
const peer = createPeer(
createStream(