diff --git a/docs/adr/0007-demand-driven-dual-representation-quality.md b/docs/adr/0007-demand-driven-dual-representation-quality.md index 746510d..509b6e6 100644 --- a/docs/adr/0007-demand-driven-dual-representation-quality.md +++ b/docs/adr/0007-demand-driven-dual-representation-quality.md @@ -146,7 +146,7 @@ evidence windows remain limited to topology eligibility and diagnostics: required product semantic is absent. In particular, the normative rule that `active=false` stops sending an encoding is not evidence that its physical encoder, CPU work, or GPU allocation is released. -2. **Pinned LiveKit two-layer simulcast: next bounded runtime candidate.** +2. **Pinned LiveKit two-layer simulcast: implemented default-off runtime candidate.** Client 2.22.0 can publish a screen-share original plus one lower simulcast encoding. `RemoteTrackPublication.setVideoQuality(HIGH)` sets a per-subscriber spatial-quality ceiling. Server 1.13.5 maps quality, dimensions, and FPS to @@ -157,7 +157,7 @@ evidence windows remain limited to topology eligibility and diagnostics: dynamic `LOW` stop while `HIGH` is subscribed, but dynamic stop is now an optimization rather than a hard requirement. - The next gate therefore publishes exactly `HIGH+LOW` with standard + The candidate therefore publishes exactly `HIGH+LOW` with standard simulcast/send encodings, leaves each LiveKit root's ceiling at `HIGH`, and first tests built-in per-subscriber SFU bandwidth adaptation on zero-descendant leaf viewers. Start with deterministic two-layer publication @@ -173,10 +173,11 @@ evidence windows remain limited to topology eligibility and diagnostics: before sender activation/deactivation. The gate must prove that a healthy P2P/`HIGH` path is unchanged, no third layer appears, the expected layer is actually received, and hardware encoder, game FPS/p1 low, CPU/GPU, interval - encode cost, host upload, and `HIGH+LOW` bytes fit budget. Screener currently - sets `simulcast: false`, leaves Dynacast at its default `false`, and does not - call `setVideoQuality()`, so no runtime claim follows and no browser is run - in this static SVC review. If the always-on cost fails, test manual standard + encode cost, host upload, and `HIGH+LOW` bytes fit budget. The default-off + exact-room candidate now configures exactly two encodings, leaves Dynacast at + its default `false`, and sets each subscriber's ceiling to `HIGH`. It has not + been browser-tested or enabled by default, so no runtime performance claim + follows. If the always-on cost fails, test manual standard sender activation/deactivation next; custom/native dual encoding follows only if built-in and manual standard primitives fail. 3. **Current Web/LiveKit SVC: rejected diff --git a/docs/project-memory.md b/docs/project-memory.md index c4e0803..f9d2dde 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -48,8 +48,8 @@ Last updated: 2026-08-19 - Chrome 151/LiveKit localhost A/B cut failure-to-active/render from 1.481/2.257 seconds to 0.200/0.320; 31 new frames and 25 ms sampling kept host edges at two. It is headless synthetic 720p30 and includes SDK/network prewarm, not public-network evidence. - Native drafts #16/#18/#22/#23/#25/#28 remain experiments: one WebCodecs object is not hardware proof; stock GCC+RTX is no-go, and no-RTX remains outside product code. - Chrome 151 synthetic topology/quality-control runs kept fanout 2/1 and all viewers decoding; one relay close recovered in 5.32 seconds. This is control evidence only. -- ADR-0007's dual publication/topology classifier is absent. Next measure LiveKit `HIGH+LOW` BWE on SFU leaves, then root suspect/evacuation; app evidence never duplicates normal layer selection. Current publishing is non-simulcast/Dynacast-off. -- Local Host A+B aligns one capture/outbound/transport generation; authenticated Viewer C adds a sanitized, read-only P2P window and fails closed for stale, ambiguous, or SFU-fed evidence. It retains no raw fmtp/SDP/stats and performs no media action. Dual publication and topology classification remain absent. +- The default-off exact-room SFU candidate now publishes exactly `HIGH+LOW` with Dynacast off and a per-subscriber `HIGH` ceiling. It is unverified and not enabled by default; next measure built-in BWE on zero-descendant leaves, then root suspect/evacuation. The topology classifier and explicit fallback remain absent, and root-with-children behavior is a default-on gate. +- Local Host A+B aligns one capture/outbound/transport generation; authenticated Viewer C adds a sanitized, read-only P2P window and fails closed for stale, ambiguous, or SFU-fed evidence. It retains no raw fmtp/SDP/stats and performs no media action. Topology classification remains absent. ## Provisional Quality Targets diff --git a/docs/status.md b/docs/status.md index 8ac746b..da01ac5 100644 --- a/docs/status.md +++ b/docs/status.md @@ -15,7 +15,7 @@ unconfigured, so production remains one host connection per viewer and P2P/TURN. - Current SQLite still stores only room ID and Host-token digest. Amended ADR-0002 accepts a later atomic `HOST_ADMISSION_PASSWORD`/`screener-v2` migration with default room-scoped private Viewer grants, explicit public-watch, rotate/revoke, and one nullable digest column; no runtime is implemented. - PR #44 merged the default-off exact-room candidate code: ordinary ICE is process-wide STUN-only, LiveKit SFU/UDP feeds at most two roots, browser relays stay at one child, the host stays at two, and exhaustion fails boundedly. Old production/coturn remains rollback; HTTPS/WSS stays TLS/TCP. - Fallback prewarm is token-free and dormant without configuration. Mobile/iPad viewers are leaves and healthy edges stay sticky. -- Manual readback and read-only A+B/C exist; dual publication/topology classification do not. Always-on `LOW` needs exactly-two, hardware, game, and upload gates. Browser audio is request/presence-only. +- Manual readback and read-only A+B/C exist. The default-off exact-room SFU candidate now configures exactly `HIGH+LOW`, keeps Dynacast off, and sets a per-subscriber `HIGH` ceiling; topology classification remains absent. Browser audio is request/presence-only. - A later explicit fallback evacuates first. Autonomous BWE enters `suspect`; confirmation evacuates children, and no confirmed `FALLBACK` parent remains. Root impact is a default-on gate; self-report alone cannot trigger it. ## Verified Evidence @@ -37,7 +37,7 @@ unconfigured, so production remains one host connection per viewer and P2P/TURN. - Production `769de201f7cc` reportedly sustains bandwidth-limited blur on a capable LAN; Host refresh recovered while Viewer refresh did not, and persistence across churn was also reported. This prioritizes, but does not prove, host sender/PC/GCC/capture generation. - Browser relays re-encode. ADR-0006 is no-go-unclassified: downstream checkpoints and two-edge/FIFO/TURN gates are absent. Spikes prove one WebCodecs object, not hardware; GCC+RTX is no-go and no-RTX weakens stats. - Browser fanout is host two/viewer one; any accepted endpoint relay stays capped at two downstream edges. -- ADR-0007/SFU C are incomplete. Next is LiveKit exactly-two BWE on SFU leaves, then root suspect/evacuation. Web P2P/current SVC shortcuts are no-go; publishing is non-simulcast/Dynacast-off. +- ADR-0007/SFU C remain unverified: the exactly-two candidate has no browser evidence or default enablement. Next is built-in BWE on zero-descendant SFU leaves, then root suspect/evacuation; root-with-children behavior is a default-on gate. Web P2P/current SVC shortcuts remain no-go. - The corrected standby smoke is localhost/headless/video-only; public DNS/TLS reuse, transport, audio, shaping, load, mobile, endurance, and sub-25 ms overlap remain open. - Scoped Viewer access is designed but unimplemented. Tests owe role bounds, fragment clearing/no-leak, expiry/cross-room/rotate/revoke, v1 tab termination, and SQLite locked migration/restore. Human passwords/accounts stay out. diff --git a/src/client/media/quality.ts b/src/client/media/quality.ts index 2a366ba..09a7c33 100644 --- a/src/client/media/quality.ts +++ b/src/client/media/quality.ts @@ -66,6 +66,17 @@ export interface VideoSenderParameterReadback { mismatches: Array; } +export interface TwoLayerVideoSenderParameterReadback { + low: VideoSenderParameterReadback; + high: VideoSenderParameterReadback; +} + +export const SCREEN_SHARE_LOW_SCALE = 2; + +export function screenShareLowBitrate(profile: QualityProfile): number { + return Math.max(150_000, Math.floor(profile.maxBitrate / 4)); +} + export function qualitySettingsEqual( left: QualitySettings, right: QualitySettings, @@ -172,8 +183,9 @@ function requestedScaleResolutionDownBy( function readVideoSenderParameters( parameters: RTCRtpSendParameters, includeAppliedScalabilityMode: boolean, + encodingIndex = 0, ): VideoSenderParameterValues { - const encoding = parameters.encodings[0]; + const encoding = parameters.encodings[encodingIndex]; const scalabilityMode = ( encoding as | (RTCRtpEncodingParameters & { scalabilityMode?: unknown }) @@ -194,6 +206,31 @@ function readVideoSenderParameters( }; } +function senderParameterReadback( + requested: VideoSenderParameterValues, + applied: VideoSenderParameterValues, +): VideoSenderParameterReadback { + const mismatches = ( + Object.keys(requested) as Array + ).filter((key) => !sameParameter(key, requested[key], applied[key])); + return { requested, applied, mismatches }; +} + +function requiredTwoLayerEncodingIndexes( + parameters: RTCRtpSendParameters, +): { low: number; high: number } { + if (parameters.encodings.length !== 2) { + throw new Error("SFU simulcast requires exactly two video encodings"); + } + if ( + parameters.encodings[0]?.rid !== "q" || + parameters.encodings[1]?.rid !== "f" + ) { + throw new Error("SFU simulcast requires ordered q and f video encodings"); + } + return { low: 0, high: 1 }; +} + function sameParameter( key: keyof VideoSenderParameterValues, requested: number | string | null, @@ -227,11 +264,48 @@ export async function configureVideoSender( const requested = readVideoSenderParameters(parameters, false); await sender.setParameters(parameters); const applied = readVideoSenderParameters(sender.getParameters(), true); - const mismatches = ( - Object.keys(requested) as Array - ).filter((key) => !sameParameter(key, requested[key], applied[key])); + return senderParameterReadback(requested, applied); +} - return { requested, applied, mismatches }; +export async function configureTwoLayerVideoSender( + sender: RTCRtpSender, + profile: QualityProfile, +): Promise { + const parameters = sender.getParameters(); + const indexes = requiredTwoLayerEncodingIndexes(parameters); + const highScale = requestedScaleResolutionDownBy(sender, profile); + const lowEncoding = parameters.encodings[indexes.low]!; + const highEncoding = parameters.encodings[indexes.high]!; + + lowEncoding.maxBitrate = screenShareLowBitrate(profile); + lowEncoding.maxFramerate = profile.maxFramerate; + lowEncoding.scaleResolutionDownBy = highScale * SCREEN_SHARE_LOW_SCALE; + highEncoding.maxBitrate = profile.maxBitrate; + highEncoding.maxFramerate = profile.maxFramerate; + highEncoding.scaleResolutionDownBy = highScale; + parameters.degradationPreference = profile.degradationPreference; + + const requestedLow = readVideoSenderParameters(parameters, false, indexes.low); + const requestedHigh = readVideoSenderParameters(parameters, false, indexes.high); + await sender.setParameters(parameters); + + const appliedParameters = sender.getParameters(); + const appliedIndexes = requiredTwoLayerEncodingIndexes(appliedParameters); + const appliedLow = readVideoSenderParameters( + appliedParameters, + false, + appliedIndexes.low, + ); + const appliedHigh = readVideoSenderParameters( + appliedParameters, + false, + appliedIndexes.high, + ); + + return { + low: senderParameterReadback(requestedLow, appliedLow), + high: senderParameterReadback(requestedHigh, appliedHigh), + }; } const PARAMETER_LABELS = { diff --git a/src/client/sfu/publisher.ts b/src/client/sfu/publisher.ts index c780033..7f79b1c 100644 --- a/src/client/sfu/publisher.ts +++ b/src/client/sfu/publisher.ts @@ -6,7 +6,10 @@ import type { } from "livekit-client"; import { - configureVideoSender, + configureTwoLayerVideoSender, + QUALITY_RESOLUTIONS, + SCREEN_SHARE_LOW_SCALE, + screenShareLowBitrate, senderParameterWarning, type QualityProfile, type VideoSenderParameterReadback, @@ -26,6 +29,11 @@ interface PublishedTrack { rawTrack: MediaStreamTrack; } +interface PublishedVideoConfiguration { + readback: VideoSenderParameterReadback; + warning: string | null; +} + type LiveKit = typeof import("livekit-client"); type PublisherState = | "idle" @@ -118,12 +126,16 @@ export class SfuPublisher { room, videoTrack, sdk.Track.Source.ScreenShare, - videoPublishOptions(profile), + videoPublishOptions(sdk, profile), ); if (!this.owns(room, generation)) { return false; } - const senderParameters = await configurePublishedVideo(video, profile); + const videoConfiguration = await configurePublishedVideo( + video, + profile, + sdk, + ); if (!this.owns(room, generation)) { return false; } @@ -144,7 +156,7 @@ export class SfuPublisher { this.video = video; this.audio = audio; this.profile = profile; - this.retainSenderParameters(senderParameters); + this.retainSenderParameters(videoConfiguration); this.state = "active"; return true; } catch (error) { @@ -251,9 +263,10 @@ export class SfuPublisher { this.audio = nextAudio; } - const senderParameters = await configurePublishedVideo( + const videoConfiguration = await configurePublishedVideo( previousVideo, profile, + sdk, ); if (!this.owns(room, generation)) { return false; @@ -262,7 +275,7 @@ export class SfuPublisher { if (previousAudio && nextAudioTrack) { previousAudio.rawTrack = nextAudioTrack; } - this.retainSenderParameters(senderParameters); + this.retainSenderParameters(videoConfiguration); return true; } catch (error) { if (!this.owns(room, generation)) { @@ -295,15 +308,19 @@ export class SfuPublisher { if (!this.owns(room, generation)) { return false; } - const senderParameters = await configurePublishedVideo( + const videoConfiguration = await configurePublishedVideo( previousVideo, profile, + sdk, ); if (!this.owns(room, generation)) { return false; } - this.senderParameters = senderParameters; - this.qualityWarning = failureWarning; + this.senderParameters = videoConfiguration.readback; + this.qualityWarning = mergeQualityWarnings( + failureWarning, + videoConfiguration.warning, + ); } return false; } catch (rollbackError) { @@ -325,17 +342,22 @@ export class SfuPublisher { } const video = this.video; const previousProfile = this.profile; - if (!video || !previousProfile) { + const sdk = this.sdk; + if (!video || !previousProfile || !sdk) { throw new Error("SFU publisher has no active video publication"); } try { - const readback = await configurePublishedVideo(video, profile); + const videoConfiguration = await configurePublishedVideo( + video, + profile, + sdk, + ); if (!this.owns(room, generation)) { return false; } this.profile = profile; - this.retainSenderParameters(readback); + this.retainSenderParameters(videoConfiguration); return true; } catch (error) { if (!this.owns(room, generation)) { @@ -346,12 +368,19 @@ export class SfuPublisher { ? `应用 SFU 发送参数失败：${error.message}` : "应用 SFU 发送参数失败"; try { - const readback = await configurePublishedVideo(video, previousProfile); + const videoConfiguration = await configurePublishedVideo( + video, + previousProfile, + sdk, + ); if (!this.owns(room, generation)) { return false; } - this.senderParameters = readback; - this.qualityWarning = failureWarning; + this.senderParameters = videoConfiguration.readback; + this.qualityWarning = mergeQualityWarnings( + failureWarning, + videoConfiguration.warning, + ); return false; } catch (rollbackError) { if (this.owns(room, generation)) { @@ -449,10 +478,10 @@ export class SfuPublisher { } private retainSenderParameters( - readback: VideoSenderParameterReadback, + configuration: PublishedVideoConfiguration, ): void { - this.senderParameters = readback; - this.qualityWarning = senderParameterWarning(readback); + this.senderParameters = configuration.readback; + this.qualityWarning = configuration.warning; } } @@ -494,27 +523,56 @@ async function unpublishTrack(room: Room, published: PublishedTrack): Promise { + sdk: LiveKit, +): Promise { const videoTrack = published.publication.videoTrack; const sender = videoTrack?.sender; if (!sender) { throw new Error("SFU video publication has no RTP sender"); } - const readback = await configureVideoSender(sender, profile); + const readbacks = await configureTwoLayerVideoSender(sender, profile); videoTrack.publishOptions = { ...videoTrack.publishOptions, - ...videoPublishOptions(profile), + ...videoPublishOptions(sdk, profile), + }; + const highWarning = senderParameterWarning(readbacks.high); + const lowWarning = senderParameterWarning(readbacks.low); + return { + readback: readbacks.high, + warning: mergeQualityWarnings( + highWarning, + lowWarning ? `低档表示：${lowWarning}` : null, + ), }; - return readback; } -function videoPublishOptions(profile: QualityProfile): TrackPublishOptions { +function mergeQualityWarnings(...warnings: Array): string | null { + const present = warnings.filter( + (warning): warning is string => warning !== null, + ); + return present.length > 0 ? present.join("；") : null; +} + +function videoPublishOptions( + sdk: LiveKit, + profile: QualityProfile, +): TrackPublishOptions { + const highResolution = QUALITY_RESOLUTIONS[profile.resolution]; return { - simulcast: false, + backupCodec: false, + simulcast: true, screenShareEncoding: { maxBitrate: profile.maxBitrate, maxFramerate: profile.maxFramerate, }, + screenShareSimulcastLayers: [ + new sdk.VideoPreset( + Math.floor(highResolution.width / SCREEN_SHARE_LOW_SCALE), + Math.floor(highResolution.height / SCREEN_SHARE_LOW_SCALE), + screenShareLowBitrate(profile), + profile.maxFramerate, + ), + ], degradationPreference: profile.degradationPreference, }; } diff --git a/src/client/sfu/subscriber.ts b/src/client/sfu/subscriber.ts index 06066da..88e7db5 100644 --- a/src/client/sfu/subscriber.ts +++ b/src/client/sfu/subscriber.ts @@ -95,7 +95,7 @@ export class SfuSubscriber { const host = room.remoteParticipants.get(HOST_IDENTITY); if (host) { for (const publication of host.trackPublications.values()) { - this.subscribeIfAllowed(publication, host, this.sdk.Track); + this.subscribeIfAllowed(publication, host, this.sdk); } } return true; @@ -155,7 +155,7 @@ export class SfuSubscriber { return; } try { - this.subscribeIfAllowed(publication, participant, sdk.Track); + this.subscribeIfAllowed(publication, participant, sdk); } catch { void this.failClosed(room, generation); } @@ -228,16 +228,21 @@ export class SfuSubscriber { private subscribeIfAllowed( publication: RemoteTrackPublication, participant: RemoteParticipant, - trackType: typeof Track, + sdk: LiveKit, ): void { if ( participant.identity !== HOST_IDENTITY || - !isScreenSource(publication.source, trackType) + !isScreenSource(publication.source, sdk.Track) ) { return; } this.desiredTrackSids.add(publication.trackSid); publication.setSubscribed(true); + if (publication.source === sdk.Track.Source.ScreenShare) { + // HIGH is a ceiling. LiveKit's per-subscriber BWE may still forward LOW + // while this path is constrained and return to HIGH after recovery. + publication.setVideoQuality(sdk.VideoQuality.HIGH); + } } private rollbackSubscriptions(room: Room): void { diff --git a/tests/quality.test.ts b/tests/quality.test.ts index 58a1395..3c52b16 100644 --- a/tests/quality.test.ts +++ b/tests/quality.test.ts @@ -3,8 +3,10 @@ import { afterEach, describe, expect, it, vi } from "vitest"; import { applyCaptureProfile, captureDisplay, + configureTwoLayerVideoSender, configureVideoSender, QUALITY_PROFILES, + screenShareLowBitrate, senderParameterWarning, setVideoPaused, } from "../src/client/media/quality.ts"; @@ -213,6 +215,93 @@ describe("realtime quality controls", () => { expect(readback.mismatches).not.toContain("scalabilityMode"); }); + it("configures ordered LOW and HIGH simulcast encodings without flattening LOW", async () => { + let applied = { + encodings: [{ rid: "q" }, { rid: "f" }], + } as RTCRtpSendParameters; + const sender = { + track: { + getSettings: () => ({ width: 2560, height: 1440 }), + }, + getParameters: () => applied, + setParameters: vi.fn(async (parameters: RTCRtpSendParameters) => { + applied = parameters; + }), + } as unknown as RTCRtpSender; + + const readbacks = await configureTwoLayerVideoSender( + sender, + QUALITY_PROFILES["1080p60"], + ); + + expect(applied).toMatchObject({ + degradationPreference: "maintain-resolution", + encodings: [ + { + rid: "q", + maxBitrate: 2_000_000, + maxFramerate: 60, + scaleResolutionDownBy: 8 / 3, + }, + { + rid: "f", + maxBitrate: 8_000_000, + maxFramerate: 60, + scaleResolutionDownBy: 4 / 3, + }, + ], + }); + expect(readbacks.low.requested.maxBitrate).toBe(2_000_000); + expect(readbacks.high.requested.maxBitrate).toBe(8_000_000); + expect(readbacks.low.mismatches).toEqual([]); + expect(readbacks.high.mismatches).toEqual([]); + }); + + it("keeps the standard LOW bitrate floor for small custom HIGH ceilings", () => { + expect( + screenShareLowBitrate({ + resolution: "720p", + maxFramerate: 30, + maxBitrate: 400_000, + degradationPreference: "balanced", + }), + ).toBe(150_000); + }); + + it.each([ + { + label: "drops HIGH", + after: { encodings: [{ rid: "q" }] }, + message: "exactly two video encodings", + }, + { + label: "reorders the RIDs", + after: { encodings: [{ rid: "f" }, { rid: "q" }] }, + message: "ordered q and f video encodings", + }, + { + label: "adds a third RID", + after: { encodings: [{ rid: "q" }, { rid: "h" }, { rid: "f" }] }, + message: "exactly two video encodings", + }, + ])("fails closed when sender readback $label", async ({ after, message }) => { + const before = { + encodings: [{ rid: "q" }, { rid: "f" }], + } as RTCRtpSendParameters; + const sender = { + track: { getSettings: () => ({ width: 1920, height: 1080 }) }, + getParameters: vi + .fn<() => RTCRtpSendParameters>() + .mockReturnValueOnce(before) + .mockReturnValue(after as RTCRtpSendParameters), + setParameters: vi.fn(async () => undefined), + } as unknown as RTCRtpSender; + + await expect( + configureTwoLayerVideoSender(sender, QUALITY_PROFILES["1080p60"]), + ).rejects.toThrow(message); + }); + it("pauses only the video track and can resume it", () => { const { stream, videoTrack } = createVideoStream(); diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts index cd2450f..02344d1 100644 --- a/tests/sfu-client.test.ts +++ b/tests/sfu-client.test.ts @@ -6,6 +6,30 @@ import { SfuSubscriber } from "../src/client/sfu/subscriber.ts"; type EventHandler = (...args: unknown[]) => void; const livekit = vi.hoisted(() => { + class FakeVideoPreset { + readonly encoding: { + maxBitrate: number; + maxFramerate?: number; + }; + + constructor( + readonly width: number, + readonly height: number, + maxBitrate: number, + maxFramerate?: number, + ) { + this.encoding = { maxBitrate, maxFramerate }; + } + + get resolution(): { width: number; height: number; frameRate?: number } { + return { + width: this.width, + height: this.height, + frameRate: this.encoding.maxFramerate, + }; + } + } + class FakeSender { parameters: RTCRtpSendParameters = { codecs: [], @@ -57,6 +81,36 @@ const livekit = vi.hoisted(() => { options: Record, ) => { const localTrack = new FakeLocalTrack(rawTrack); + const simulcastLayers = options.screenShareSimulcastLayers; + const highEncoding = options.screenShareEncoding; + if ( + rawTrack.kind === "video" && + options.simulcast === true && + Array.isArray(simulcastLayers) && + simulcastLayers.length === 1 && + simulcastLayers[0] instanceof FakeVideoPreset && + typeof highEncoding === "object" && + highEncoding !== null + ) { + const high = highEncoding as { + maxBitrate?: number; + maxFramerate?: number; + }; + localTrack.sender.parameters.encodings = [ + { + rid: "q", + maxBitrate: simulcastLayers[0].encoding.maxBitrate, + maxFramerate: simulcastLayers[0].encoding.maxFramerate, + scaleResolutionDownBy: 2, + }, + { + rid: "f", + maxBitrate: high.maxBitrate, + maxFramerate: high.maxFramerate, + scaleResolutionDownBy: 1, + }, + ]; + } const publication = { track: localTrack, videoTrack: rawTrack.kind === "video" ? localTrack : undefined, @@ -74,7 +128,16 @@ const livekit = vi.hoisted(() => { } class FakeRemotePublication { - readonly setSubscribed = vi.fn((_subscribed: boolean) => undefined); + subscribed = false; + requestedVideoQuality: string | null = null; + readonly setSubscribed = vi.fn((subscribed: boolean) => { + this.subscribed = subscribed; + }); + readonly setVideoQuality = vi.fn((quality: string) => { + if (this.subscribed) { + this.requestedVideoQuality = quality; + } + }); constructor( readonly trackSid: string, @@ -108,7 +171,7 @@ const livekit = vi.hoisted(() => { ); readonly disconnect = vi.fn(async (_stopTracks?: boolean) => undefined); - constructor() { + constructor(readonly options: Record = {}) { state.rooms.push(this); } @@ -140,6 +203,7 @@ const livekit = vi.hoisted(() => { FakeRemoteParticipant, FakeRemotePublication, FakeRoom, + FakeVideoPreset, state, }; }); @@ -161,10 +225,16 @@ const Track = { }, } as const; +const VideoQuality = { + HIGH: "high", +} as const; + vi.mock("livekit-client", () => ({ Room: livekit.FakeRoom, RoomEvent, Track, + VideoPreset: livekit.FakeVideoPreset, + VideoQuality, })); class FakeMediaStream { @@ -232,6 +302,7 @@ describe("SfuPublisher", () => { await expect(publisher.connect(connection)).resolves.toBe(true); const room = livekit.state.rooms[0]; + expect(room.options).toEqual({}); expect(room.connect).toHaveBeenCalledWith(connection.url, connection.token, { autoSubscribe: false, }); @@ -257,11 +328,22 @@ describe("SfuPublisher", () => { const sender = room.localParticipant.publications[0].track.sender; expect(room.localParticipant.publishTrack).toHaveBeenNthCalledWith(1, video, { source: Track.Source.ScreenShare, - simulcast: false, + backupCodec: false, + simulcast: true, screenShareEncoding: { maxBitrate: 8_000_000, maxFramerate: 60, }, + screenShareSimulcastLayers: [ + expect.objectContaining({ + width: 960, + height: 540, + encoding: { + maxBitrate: 2_000_000, + maxFramerate: 60, + }, + }), + ], degradationPreference: "maintain-resolution", }); expect(room.localParticipant.publishTrack).toHaveBeenNthCalledWith(2, audio, { @@ -269,6 +351,20 @@ describe("SfuPublisher", () => { dtx: false, }); expect(sender.setParameters).toHaveBeenCalledOnce(); + expect(sender.parameters.encodings).toEqual([ + expect.objectContaining({ + rid: "q", + maxBitrate: 2_000_000, + maxFramerate: 60, + scaleResolutionDownBy: 2, + }), + expect.objectContaining({ + rid: "f", + maxBitrate: 8_000_000, + maxFramerate: 60, + scaleResolutionDownBy: 1, + }), + ]); expect(publisher.getSenderParameters()).toEqual({ requested: { maxBitrate: 8_000_000, @@ -328,8 +424,16 @@ describe("SfuPublisher", () => { degradationPreference: "balanced", encodings: [ expect.objectContaining({ + rid: "q", + maxBitrate: 750_000, + maxFramerate: 30, + scaleResolutionDownBy: 2, + }), + expect.objectContaining({ + rid: "f", maxBitrate: 3_000_000, maxFramerate: 30, + scaleResolutionDownBy: 1, }), ], }), @@ -366,6 +470,41 @@ describe("SfuPublisher", () => { expect(publisher.getQualityWarning()).toContain("码率上限"); }); + it("reports a LOW rewrite while retaining HIGH sender readback", async () => { + const publisher = new SfuPublisher(); + await publisher.connect(connection); + await publisher.activate(stream(track("video", "video-1")), qualityProfile); + const sender = livekit.state.rooms[0].localParticipant.publications[0].track + .sender; + sender.setParameters.mockImplementationOnce(async (parameters) => { + sender.parameters = { + ...parameters, + encodings: parameters.encodings.map((encoding, index) => + index === 0 + ? { ...encoding, maxBitrate: 200_000 } + : { ...encoding }, + ), + }; + }); + + await expect( + publisher.updateProfile({ + resolution: "1080p", + maxFramerate: 30, + maxBitrate: 5_000_000, + degradationPreference: "balanced", + }), + ).resolves.toBe(true); + + expect(publisher.getSenderParameters()).toMatchObject({ + requested: { maxBitrate: 5_000_000 }, + applied: { maxBitrate: 5_000_000 }, + mismatches: [], + }); + expect(publisher.getQualityWarning()).toContain("低档表示"); + expect(publisher.getQualityWarning()).toContain("码率上限"); + }); + it("retains a visible warning when SFU sender parameters are rejected", async () => { const publisher = new SfuPublisher(); await publisher.connect(connection); @@ -388,6 +527,46 @@ describe("SfuPublisher", () => { ); }); + it("retains a LOW rewrite warning after rolling back sender parameters", async () => { + const publisher = new SfuPublisher(); + await publisher.connect(connection); + await publisher.activate(stream(track("video", "video-1")), qualityProfile); + const sender = livekit.state.rooms[0].localParticipant.publications[0].track + .sender; + sender.setParameters + .mockRejectedValueOnce(new Error("unsupported")) + .mockImplementationOnce(async (parameters) => { + sender.parameters = { + ...parameters, + encodings: parameters.encodings.map((encoding, index) => + index === 0 + ? { ...encoding, maxBitrate: 1_500_000 } + : { ...encoding }, + ), + }; + }); + + await expect( + publisher.updateProfile({ + resolution: "720p", + maxFramerate: 30, + maxBitrate: 3_000_000, + degradationPreference: "balanced", + }), + ).resolves.toBe(false); + + expect(publisher.getSenderParameters()).toMatchObject({ + requested: { maxBitrate: 8_000_000 }, + applied: { maxBitrate: 8_000_000 }, + mismatches: [], + }); + expect(publisher.getQualityWarning()).toContain( + "应用 SFU 发送参数失败：unsupported", + ); + expect(publisher.getQualityWarning()).toContain("低档表示"); + expect(publisher.getQualityWarning()).toContain("码率上限"); + }); + it("restores the previous video after a partially applied replacement fails", async () => { const previousVideo = track("video", "video-1"); const nextVideo = track("video", "video-2"); @@ -526,9 +705,19 @@ describe("SfuSubscriber", () => { expect(hostVideo.setSubscribed).not.toHaveBeenCalled(); expect(streams).toEqual([]); + hostVideo.setVideoQuality(VideoQuality.HIGH); + expect(hostVideo.requestedVideoQuality).toBeNull(); + hostVideo.setVideoQuality.mockClear(); + expect(subscriber.activate()).toBe(true); expect(hostVideo.setSubscribed).toHaveBeenCalledWith(true); expect(hostAudio.setSubscribed).toHaveBeenCalledWith(true); + expect(hostVideo.setVideoQuality).toHaveBeenCalledWith(VideoQuality.HIGH); + expect(hostVideo.requestedVideoQuality).toBe(VideoQuality.HIGH); + expect(hostVideo.setSubscribed.mock.invocationCallOrder[0]).toBeLessThan( + hostVideo.setVideoQuality.mock.invocationCallOrder[0]!, + ); + expect(hostAudio.setVideoQuality).not.toHaveBeenCalled(); expect(hostCamera.setSubscribed).not.toHaveBeenCalled(); expect(viewerScreen.setSubscribed).not.toHaveBeenCalled();