fix(media): reapply negotiated P2P quality settings [skip ci]
This commit is contained in:
1 parent
ef36543d6e
commit
4a216c8b92
6 files changed
+114
-19
No files matched your search
@@ -145,7 +145,7 @@ evidence windows remain limited to topology eligibility and diagnostics:
|
||||
required product semantic is absent. In particular, the normative rule that
|
||||
`active=false` stops sending an encoding is not evidence that its physical
|
||||
encoder, CPU work, or GPU allocation is released.
|
||||
2. **Pinned LiveKit two-layer simulcast: implemented default-off runtime candidate.**
|
||||
2. **Pinned LiveKit two-layer simulcast: implemented; acceptance remains open.**
|
||||
Client 2.22.0 can publish a screen-share original plus one lower simulcast
|
||||
encoding. `RemoteTrackPublication.setVideoQuality(HIGH)` sets a per-subscriber
|
||||
spatial-quality ceiling. Server 1.13.5 maps quality, dimensions, and FPS to
|
||||
@@ -172,13 +172,15 @@ evidence windows remain limited to topology eligibility and diagnostics:
|
||||
before sender activation/deactivation. The gate must prove that a healthy
|
||||
P2P/`HIGH` path is unchanged, no third layer appears, the expected layer is
|
||||
actually received, and hardware encoder, game FPS/p1 low, CPU/GPU, interval
|
||||
encode cost, host upload, and `HIGH+LOW` bytes fit budget. The default-off
|
||||
exact-room candidate now configures exactly two encodings, leaves Dynacast at
|
||||
its default `false`, and sets each subscriber's ceiling to `HIGH`. It has not
|
||||
been browser-tested or enabled by default, so no runtime performance claim
|
||||
follows. If the always-on cost fails, test manual standard
|
||||
sender activation/deactivation next; custom/native dual encoding follows
|
||||
only if built-in and manual standard primitives fail.
|
||||
encode cost, host upload, and `HIGH+LOW` bytes fit budget. Every active SFU
|
||||
publication now configures exactly two ordered `q,h` encodings, leaves
|
||||
Dynacast at its default `false`, and sets each subscriber's ceiling to
|
||||
`HIGH`; the two-layer publication is not behind a separate quality flag.
|
||||
Per-subscriber BWE, hardware cost, and root-with-children behavior remain
|
||||
unverified, so no runtime performance claim follows. If the always-on cost
|
||||
fails, test manual standard sender activation/deactivation next;
|
||||
custom/native dual encoding follows only if built-in and manual standard
|
||||
primitives fail.
|
||||
3. **Current Web/LiveKit SVC: rejected
|
||||
(`no-go-web-svc-cross-path-hardware-contract`).** WebRTC-SVC adds
|
||||
`scalabilityMode` to sender encoding parameters, but it adds no receiver
|
||||
|
||||
@@ -10,7 +10,7 @@ Last updated: 2026-08-21
|
||||
- Minimize server bandwidth. Ordinary peer ICE is STUN-only. Failure tries restart, rebuild, alternate peer and bounded SFU/UDP roots; only then may the controller authorize short-lived TURN for one selected exceptional edge before clear failure. TURN is transport, not topology.
|
||||
- The current release budget is two downstream edges per non-server node, including ordinary Web relays; upstream receive is free and UA/visibility do not change it. SFU normally feeds one or two roots that retain peer descendants; separately capped server edges may serve exceptional viewers that cannot attach behind a healthy root. A future evidence-driven tier may change the ceiling only through the owning requirement and design.
|
||||
- Two-tree packet/layer striping may reduce endpoint upload toward one stream bitrate, but needs a bounded multi-parent, loss, sync, churn, and latency experiment; it is not in the current full-stream chains.
|
||||
- Direct/peer keeps per-PC stock GCC. SFU paths start at a `HIGH` ceiling and share at most one `LOW`; next test LiveKit exactly-two built-in BWE on zero-descendant leaves. If it passes, no app media selector is built. Explicit quality, manual activation, then custom/native are later fallbacks; always-on `LOW` needs resource gates.
|
||||
- Direct/peer keeps per-PC stock GCC. Active SFU publishes exactly `q,h` at a `HIGH` ceiling; built-in BWE may forward `LOW`. The zero-descendant quality/resource gate is next; if it passes, no app media selector is built. Explicit quality, manual activation, and custom/native remain fallbacks.
|
||||
- A later explicit quality fallback evacuates children under a generation guard first. Autonomous BWE enters `suspect`; confirmation evacuates children, and no confirmed `FALLBACK` parent remains. Root-with-children impact is a default-on gate; capacity returns after longer recovery plus cooldown. Self-report alone never triggers it.
|
||||
- Site access protects room creation, Host publication, and every code-only Viewer entry. A valid room-scoped fragment grant bypasses that site gate; otherwise site access is checked before public-watch or a private room password. Persist neither raw credential; keep accounts, ACLs, users, and session tables out.
|
||||
- Keep decisions, snapshots, research, code, `AGENTS.md`, and `.codex/` in Git; rewrite memory/status in place. Research current primary sources before material work and reject speculative machinery.
|
||||
@@ -52,7 +52,7 @@ Last updated: 2026-08-21
|
||||
- Production uses nginx, Node.js 24.19.0, coturn 4.17.2, and LiveKit 1.13.5 on UDP 7882 (TCP fallback off). Ordinary peer ICE receives no TURN; the selected-edge tuple is configured only for the current controller edge, with no credential pre-advertised to ordinary peers. Coturn retains the old authenticated-relay ports. Services are healthy with zero restarts.
|
||||
- ADR-0005 rejects `PEER_ASSISTED_ROOM_IDS`; `PEER_ASSISTED_MEDIA=true` enables every room and ordinary peers stay STUN-only. Initial or active Host ingress may consume one bound relay-only grant; ready/abort clears it and failure restores peer baseline. Stale room-ID config fails startup; `screener-v2` is the only deployed wire.
|
||||
- Chrome 151/LiveKit localhost A/B cut failure-to-active/render from 1.481/2.257 seconds to 0.200/0.320; 31 new frames and 25 ms sampling kept host edges at two. It is headless synthetic 720p30 and includes SDK/network prewarm, not public-network evidence.
|
||||
- Deployed Web uses LiveKit 2.22.0 `q,h`. Chrome 151 proves local SFU/UDP and, separately, active selected TURN/UDP Host ingress against production media services with Viewer frame progress and clean stop. Initial/peer-selected relay, BWE, and C+B remain open.
|
||||
- Web SFU uses LiveKit 2.22.0 `q,h`; local SFU/UDP and selected-TURN ingress work, while BWE remains open. P2P now reapplies its selected profile after each answer on the same sender. A local A/B supports the lifecycle candidate, but natural ramp, real capture, production, and SFU validation remain open.
|
||||
- Chrome 151 five-Viewer 720p30 cap2 evidence: Host/relay fanout two; both relay children +80 decoded frames/~1.1 MB; every Viewer decoded; clean stop. Functional only; resource and heterogeneous-network behavior remain open.
|
||||
|
||||
## Provisional Quality Targets
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Realtime Screen-Share Quality Adaptation
|
||||
|
||||
- Research date: 2026-08-19
|
||||
- Research date: 2026-08-21
|
||||
- Scope: realtime game screen sharing in the browser
|
||||
- Status: implementation input; real-device quality remains unverified
|
||||
|
||||
@@ -88,6 +88,51 @@ are ceilings rather than targets. Raising them cannot repair CPU or bandwidth
|
||||
limitation and should only follow evidence that the encoder is already pinned
|
||||
to the ceiling while spare transport capacity remains.
|
||||
|
||||
## Initial Balanced Lifecycle Evidence
|
||||
|
||||
On 2026-08-21 one bounded local Chrome 151 run used exact source `b447ab6`, a
|
||||
synthetic 1920x1080 at 60 fps capture, and one direct UDP Viewer. After the
|
||||
initial connection had existed for about 22 seconds, the Host outbound and
|
||||
Viewer inbound were both 1280x720 at 57 fps and about 7.03 Mbps. The sender
|
||||
reported `qualityLimitationReason=bandwidth`, despite about 12.59 Mbps of
|
||||
available outgoing bitrate. Reapplying the identical 1080p60 `balanced` setting
|
||||
through the real Host control produced 1920x1080 at 55 fps and about 7.11 Mbps
|
||||
after 5.2 seconds, with the limitation cleared. Both ends reported zero packet
|
||||
loss and no Viewer freeze.
|
||||
|
||||
The route revision, PeerConnection identity, RTP SSRC, RTP stats object, and
|
||||
track identifier all remained unchanged. This rules out a new route, sender,
|
||||
or capture generation as the source of that observed transition. It does not
|
||||
prove visual quality, production behavior, or SFU behavior. Earlier samples in
|
||||
the same run remained at 640x360 through roughly 20 seconds and reached 720p in
|
||||
the two-second window immediately before the control action, so continued
|
||||
stock BWE ramp-up remains a causal confound. No second browser run was made.
|
||||
The Host control applies same-track capture constraints before updating sender
|
||||
parameters, so this run also cannot isolate those two standard calls.
|
||||
|
||||
Static review does not support the simpler claim that Chromium discards every
|
||||
pre-negotiation `setParameters()` call. Current libwebrtc stores sender init
|
||||
parameters while no SSRC exists and transfers them into the negotiated sender.
|
||||
The observed transition followed the same-setting action, but this single run
|
||||
cannot attribute it to sender lifecycle or the reapply itself. The bounded
|
||||
implementation change refreshes the same video-sender profile after each
|
||||
accepted answer, serialized with existing sender mutations. It
|
||||
changes no track, PeerConnection, ceiling, degradation preference, or room
|
||||
setting and adds no periodic controller. A post-change real-capture P2P check
|
||||
must still confirm that the negotiated sender reapply, without a new capture
|
||||
generation, clears the reported startup condition.
|
||||
|
||||
This evidence does not justify an SFU change. The current SFU publisher always
|
||||
publishes the ordered `q,h` pair when an SFU route is active and configures its
|
||||
sender after SDK publication. The pinned client publication defaults to
|
||||
`HIGH`, and Screener also requests a `HIGH` subscriber ceiling immediately
|
||||
after subscribing. A low SFU receive layer can still be a valid server BWE
|
||||
choice. Diagnose that case by correlating publisher high-layer stats with the
|
||||
Viewer's actual inbound layer; do not infer a missing `HIGH` request or change
|
||||
the application policy without those measurements. LiveKit's pinned client
|
||||
also contains an SDP start-bitrate mitigation for initial video blur, but
|
||||
Screener's accepted boundary forbids adding application SDP bitrate hacks.
|
||||
|
||||
The same production release is also reported to reduce game-stream frame rate
|
||||
and consume noticeable Host resources. That report applies only to deployed
|
||||
`769de201f7cc`, not automatically to the newer diagnostics on `main`. The Web
|
||||
@@ -338,11 +383,13 @@ hardware encoder, game FPS/p1 low, CPU/GPU, interval encode cost, upload, and
|
||||
per-layer byte budgets. Before default enablement, a separate root-with-children
|
||||
gate must inject an autonomous downshift, observe `suspect`, evacuate after
|
||||
bounded confirmation, and limit temporary descendant impact; no confirmed
|
||||
`FALLBACK` root may retain children. At `main` commit `13b8dad`, Screener's
|
||||
default-off publisher configures exactly two ordered `q`/`f` encodings, leaves
|
||||
Dynacast at `false`, disables backup-codec publication, and its subscriber sets
|
||||
a `HIGH` ceiling after subscribing. This is implemented configuration, not
|
||||
browser or performance evidence. The subscriber also does not attach a
|
||||
`FALLBACK` root may retain children. Screener's current publisher always
|
||||
configures exactly two ordered `q`/`h` encodings whenever an SFU publication is
|
||||
active, leaves Dynacast at `false`, disables backup-codec publication, and its
|
||||
subscriber sets a `HIGH` ceiling after subscribing. The publication is not
|
||||
behind a separate quality flag; its per-subscriber BWE and resource acceptance
|
||||
gates remain unverified. This is implemented configuration, not browser or
|
||||
performance evidence. The subscriber also does not attach a
|
||||
`RemoteTrack`, so SDK `adaptiveStream` is not directly usable without changing
|
||||
that ownership; built-in SFU bandwidth adaptation does not depend on enabling
|
||||
that feature. If built-in selection fails the product gates, test explicit
|
||||
@@ -504,6 +551,10 @@ resolution, frame rate, or bitrate.
|
||||
- Every sender update derives from `getParameters()`, calls `setParameters()`,
|
||||
then reads requested/applied bitrate, frame rate, scale, and preference.
|
||||
Rejection or browser rewriting is visible rather than console-only.
|
||||
- A P2P sender receives that update before its first offer and once more after
|
||||
each accepted answer. The negotiated reapply uses the current selected
|
||||
profile and existing sender; it neither rebuilds the connection nor changes
|
||||
the browser-owned degradation decision.
|
||||
- One strict room setting is last-wins for current/future peer relays and the
|
||||
configured SFU publisher. Ordinary P2P keeps that state local and does not add
|
||||
it to the authenticated wire.
|
||||
@@ -596,6 +647,7 @@ is a separate optimization.
|
||||
- [MDN `RTCRtpSender.setParameters()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters)
|
||||
- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc)
|
||||
- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc)
|
||||
- [libwebrtc pre-negotiation sender parameter storage](https://webrtc.googlesource.com/src/+/refs/heads/main/pc/rtp_sender.cc)
|
||||
- [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc)
|
||||
- [libwebrtc conditional screen-share degradation mapping](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_stream_encoder.cc)
|
||||
- [libwebrtc adaptation overview](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/adaptation.md)
|
||||
@@ -606,6 +658,7 @@ is a separate optimization.
|
||||
- [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/)
|
||||
- [LiveKit client 2.22.0 SVC defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts)
|
||||
- [LiveKit client 2.22.0 screen-share SVC override](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
|
||||
- [LiveKit client 2.22.0 start-bitrate negotiation](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/PCTransport.ts)
|
||||
- [LiveKit client 2.22.0 SVC encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts)
|
||||
- [LiveKit client 2.22.0 subscriber quality control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts)
|
||||
- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go)
|
||||
|
||||
+3
-3
@@ -15,7 +15,7 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
|
||||
|
||||
## Current Snapshot
|
||||
|
||||
- Capture precedes room creation; source/quality changes and AV pause preserve peers. Recommended profiles and advanced defaults use `balanced`; clarity/fluid remain explicit. The browser chooses degradation; readback/stats report actual behavior.
|
||||
- Capture precedes room creation; source/quality changes and AV pause preserve peers. Recommended profiles and advanced defaults use `balanced`; clarity/fluid remain explicit. P2P reapplies the selected profile after each answer on the same sender; the browser still owns degradation and stats/readback expose it.
|
||||
- Production access is `screener-v2`: a valid private fragment grant enters directly; every code-only Viewer must first hold site access, after which public-watch accepts the code and private rooms still require their room password. `SITE_ACCESS_PASSWORD`, `/api/site-access`, and the site-access cookie are deployed atomically; raw credentials are never stored.
|
||||
- Source UI uses adjacent nickname editing, distinct site/room prompts, and a shared room-code form. Status and details stay neutral before media proof; current evidence labels green P2P/yellow `SFU fallback`, while relay evidence adds `TURN` plus only a locally observed protocol.
|
||||
- Native v2 is source-only (memory rooms, 300s reclaim); it follows authenticated direct-child assignments up to two and fails unsupported SFU/selected ingress boundedly.
|
||||
@@ -42,10 +42,10 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
|
||||
- The `691863e` cutover ran no browser/media/SFU/TURN/performance canary. Earlier `16f6eab` proof covers Host ingress only; audible quality, last mile, external cohorts, admission rescue, resources/performance and game blur remain open.
|
||||
- Mobile is Viewer-only. Its persistent audible media element keeps media/signaling active while the page lives; iOS lock-screen, reclamation and background reconnection remain device gates. AirPlay/system mirroring is local output; live WebRTC `srcObject` has no portable in-app TV-output contract.
|
||||
- Silent partitions can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; this remains unverified.
|
||||
- The former release reportedly sustained bandwidth-limited blur on a capable LAN; Host refresh recovered while Viewer refresh did not. The new deployment has not yet reproduced or cleared it.
|
||||
- Production startup blur remains open. One local Chrome synthetic same-`balanced` A/B kept route/PC/SSRC/track and coincided with 720p -> 1080p without loss/freeze, but natural ramp prevents causal, production, or SFU proof.
|
||||
- ADR-0006 has one Viewer proof; two-edge/FIFO, hardware, endurance, public/downloaded-package, and browser-diversity proof remain open.
|
||||
- Production fanout is Host2/Viewer2 and rejects child3. Relay resource behavior remains open.
|
||||
- PR #49/#50 BWE is unverified/default-off: test zero-child leaves, then root impact. No local per-leaf UDP shaper; resume with Linux `tc` or public canary, never CDP. Web P2P/SVC shortcuts remain no-go.
|
||||
- Active SFU already publishes `q,h`; only BWE/resource/zero-child/root gates are unverified. No local per-leaf UDP shaper; resume with Linux `tc` or public canary, never CDP. Web P2P/SVC shortcuts remain no-go.
|
||||
- Headful fragment-to-sessionStorage consumption, invitation rotation after migration, and request/log leak inspection remain production UX gates. Accounts stay out.
|
||||
- Structure debt: split `HybridMediaRouter`, `HostPage`, `SignalingServer`, and `ViewerPage` only at proven consumer boundaries, never by file length.
|
||||
|
||||
|
||||
@@ -224,6 +224,15 @@ export class HostPeer {
|
||||
}
|
||||
await this.connection.setRemoteDescription(payload.description);
|
||||
await this.flushCandidates();
|
||||
await this.enqueueSenderMutation(async () => {
|
||||
const videoSender = this.videoSender;
|
||||
if (this.disposed || !videoSender) {
|
||||
return false;
|
||||
}
|
||||
// Refresh requested bounds after negotiation; browser adaptation
|
||||
// remains authoritative.
|
||||
return this.configureSender(videoSender);
|
||||
});
|
||||
} else if (this.connection.remoteDescription) {
|
||||
await this.connection.addIceCandidate(payload.candidate);
|
||||
} else if (this.pendingCandidates.length < MAX_PENDING_CANDIDATES) {
|
||||
|
||||
@@ -480,6 +480,37 @@ describe("HostPeer source replacement", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("reapplies the selected video profile after the answer", async () => {
|
||||
const video = createTrack("video", "video");
|
||||
const audio = createTrack("audio", "audio");
|
||||
const peer = createPeer(createStream(video, audio));
|
||||
|
||||
await expect(peer.start()).resolves.toBe(true);
|
||||
const connection = FakePeerConnection.latest!;
|
||||
expect(connection.senders[0]?.setParameters).toHaveBeenCalledOnce();
|
||||
|
||||
await peer.acceptSignal({
|
||||
kind: "description",
|
||||
connectionId: peer.connectionId,
|
||||
description: { type: "answer", sdp: "test-answer" },
|
||||
});
|
||||
|
||||
expect(connection.remoteDescription).toMatchObject({
|
||||
type: "answer",
|
||||
sdp: "test-answer",
|
||||
});
|
||||
expect(connection.senders[0]?.track).toBe(video);
|
||||
expect(connection.senders[0]?.replaceTrack).not.toHaveBeenCalled();
|
||||
expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2);
|
||||
expect(
|
||||
connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0],
|
||||
).toMatchObject({
|
||||
degradationPreference: "balanced",
|
||||
encodings: [{ maxBitrate: 3_000_000, maxFramerate: 30 }],
|
||||
});
|
||||
expect(connection.senders[1]?.setParameters).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("serializes initial sender configuration with a live profile update", async () => {
|
||||
const peer = createPeer(
|
||||
createStream(
|
||||
|
||||
Reference in new issue
Block a user