diff --git a/docs/adr/0007-path-isolated-representation-quality.md b/docs/adr/0007-path-isolated-representation-quality.md index 7f71563..f84c67c 100644 --- a/docs/adr/0007-path-isolated-representation-quality.md +++ b/docs/adr/0007-path-isolated-representation-quality.md @@ -1,6 +1,6 @@ # ADR-0007: LiveKit-Owned SFU Representation Adaptation -- Status: Framework adaptation deployed; lightweight lower representation accepted and pending +- Status: Lightweight lower representation implemented in current source; not deployed - Date: 2026-08-19 - Last reviewed: 2026-08-25 @@ -64,6 +64,12 @@ while a subscriber's default `HIGH` ceiling still selects the highest available encoding. Dynacast, server send-side BWE, and disabled AdaptiveStream remain unchanged. +Current source derives the lower preset from the actual capture dimensions and +passes it through LiveKit's public publication options. Retained publication +options give LiveKit the same preset on its own source-replacement or republish +boundary; Screener does not mutate the active lower encoding behind LiveKit's +Dynacast sender lock. + ## Pinned Physical Result The 2026-08-25 gate used LiveKit server `1.13.5`, client `2.22.0`, Chrome 151, diff --git a/docs/project-memory.md b/docs/project-memory.md index aba9225..3976d06 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -33,7 +33,7 @@ Last updated: 2026-08-25 - Current source and production admit up to `20` Viewers and implement the uniform `1/2/3` endpoint cap, exact prepare candidate, first-decoded-frame commit, strict rollback revision, decoded-progress stall, one direct/SFU operation deadline, and one event-driven controller. Active-SFU sampling remains live through an absent track/report or stats failure. Ordinary peer ICE is STUN-only and the sole application fallback is SFU/UDP. Exact SFU admission and empty Browser SFU ICE-server options remain unchanged. - Source and production use random four-digit memory rooms, dormant leases, exact-Host resume, restart loss, local preference replay, orthogonal grants/code entry, and no SQLite runtime. V12 uses a 22-character room-incarnation grant, `ROOM_NOT_FOUND`, no diagnostic-file export, and a compact UI showing the complete shorter URL. - Exact production artifacts, services, ports, and any change-scoped infrastructure recovery boundary are owned by [deployment](./deployment.md) and indexed by [status](./status.md). -- Source and production require v12, fixed VP8, and no video hint, and currently use pinned-default SFU half-resolution plus original-resolution simulcast, Dynacast, and server send-side BWE. The lightweight lower-representation correction is accepted but not yet implemented or deployed. Other current media and UI behavior matches the truths above. +- Source and production require v12, fixed VP8, and no video hint. Current source derives a half-actual-resolution SFU lower preset capped at 30 fps with proportional bitrate; production still uses pinned-default same-FPS lower plus original-resolution simulcast. Both retain Dynacast, server send-side BWE, and disabled AdaptiveStream. Other current media and UI behavior matches the truths above. - Browser SFU subscribers reconcile eligible Host screen publications across connect/activate event ordering, participant/track arrival, and reconnect. Production LiveKit logs only warn/error, and the application service has a bounded same-host LiveKit readiness preflight. - The deployed application tree passed TypeScript, 611 Web tests, both builds, hygiene, immutable artifact verification, and production postflight. Earlier exact-production gates closed direct/browser-relay VP8 cadence, active-audio ceilings, a single-representation SFU baseline, and one controlled exact-candidate SFU commit; the pinned-stack gate proved native representation selection. Controlled recovery on this release, representative networks, all-UDP-blocked failure, public SFU 60 fps, and real-game synchronization remain unproven. Exact metrics live in [verification status](./verification-status.md). - Chrome loopback, cap2/cap3, short resource, recovery, MBB, and A/V fixtures are bounded evidence only. They do not establish quality, resource limits, or product policy. diff --git a/docs/status.md b/docs/status.md index 246e5a5..fc86f85 100644 --- a/docs/status.md +++ b/docs/status.md @@ -22,14 +22,14 @@ This is the current execution index. Git history owns completed timelines; [veri - One event-driven controller owns the committed graph and at most one room-serial child operation. It uses one deterministic candidate list/cursor and one total direct-then-SFU deadline; exact admission and physical resources remain charged through drain, and the pending candidate commits only after an exact-generation decoded-frame proof. A short-lived 100 ms stats observer proves only that candidate inside the existing deadline. The active-path 2-second sampler now keeps its existing 15-second decoded-progress cadence when an SFU subscriber has no video track, no merged report, or a failed stats read, without adding a timer or fabricating metrics. Healthy edges remain sticky and current-path quality does not authorize reparenting. - Ordinary peer ICE is STUN-only and the only application fallback is the dedicated LiveKit SFU over UDP. SFU generations use explicit no-default ingress/egress capacities, exact `reserved | committed | draining` accounting, delete-plus-absence release, one Host publication, and exact Viewer subscription handles. No Screener TURN, ICE/TCP, media TCP, or TLS-relayed media route exists. - Current Browser SFU publisher and subscriber connects supply fresh `autoSubscribe: false` plus `rtcConfig: { iceServers: [] }` options. Ordinary peers retain deployment STUN; LiveKit signaling candidates, server-side public-IP discovery, route deadlines, ports, and retry behavior are unchanged. -- Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec. SFU publication fixes VP8 with no backup codec, leaves screen-share simulcast layers unset so pinned client `2.22.0` supplies its half-resolution plus original-resolution pair, enables Dynacast, retains LiveKit's default maximum subscriber ceiling, and keeps AdaptiveStream disabled. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation remains unchanged; ordinary Host and SFU publisher metrics select the exact current sender, relays reconcile audio-track additions and removals on the persistent stream, and an SFU reconnect reacquires the exact current publication/sender, avoids unchanged parameter/stat resets, or fails the route closed. +- Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec. SFU publication fixes VP8 with no backup codec and supplies LiveKit one half-actual-resolution lower preset capped at 30 fps with proportional bitrate beside the original representation; Dynacast, the default maximum subscriber ceiling, server send-side BWE, and disabled AdaptiveStream remain unchanged. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation remains unchanged; ordinary Host and SFU publisher metrics select the exact current sender, relays reconcile audio-track additions and removals on the persistent stream, and an SFU reconnect reacquires the exact current publication/sender, avoids unchanged parameter/stat resets, or fails the route closed. - Current source implements typed first-frame Viewer presentation and recovery, Chinese user-facing route/access failures, local connection details without a diagnostic-file export, authenticated Host-only on-demand route snapshots for automatic acceptance, explicit missing-room presentation, and responsive entry controls. Host invitation controls remain independent from two-state room-code entry. - Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`; current source defaults to `1080p30`. Advanced resolution adds `480p` as `854x480` without adding a fourth preset, and advanced FPS and bitrate remain independent. - The deployed application tree passed repository hygiene, TypeScript, all 611 Web tests in 44 files, both production builds, and immutable artifact/postflight checks. Earlier exact-production gates exercised direct, browser-relay, SFU audio continuity and all three audio ceilings, and sustained about 59.5 fps on direct and browser-relay paths; the pinned-stack gate proved native two-representation selection before deployment. Exact applicability remains in [verification status](./verification-status.md). ## Current Milestone -1. Apply and verify ADR-0007's lightweight half-resolution representation, then repeat the controlled active-SFU recovery transaction on exact production and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model. +1. Deploy and verify ADR-0007's implemented lightweight half-resolution representation, then repeat the controlled active-SFU recovery transaction on exact production and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model. ## Active Boundaries diff --git a/docs/todo.md b/docs/todo.md index 632cfa1..5407572 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -6,7 +6,7 @@ Only items in **Now** are executable after their stated decision gate. A branch ## Now -1. **Correct constrained SFU publication quality.** Keep the two LiveKit-owned VP8 spatial representations, Dynacast, send-side BWE, and no AdaptiveStream. Cap the half-resolution representation at 30 fps and derive its bitrate from the pinned SDK's proportional formula, including live profile updates. In the same mixed P2P/SFU dynamic-source gate, verify that HIGH recovers materially, P2P does not regress, and a constrained SFU downlink still selects and decodes LOW before deployment. Do not restore global `motion`, add more simulcast encodings, return to single HIGH, or add a Screener layer selector. +1. **Deploy and verify constrained SFU publication quality.** Current source keeps the two LiveKit-owned VP8 spatial representations, Dynacast, send-side BWE, and no AdaptiveStream. It derives the half-actual-resolution lower publication preset at no more than 30 fps with proportional bitrate and retains that option for LiveKit-owned source replacement or republish; it does not mutate LOW behind the SDK's Dynacast lock. In the same mixed P2P/SFU dynamic-source gate, verify that HIGH recovers materially, P2P does not regress, and a constrained SFU downlink still selects and decodes LOW before deployment. Do not restore global `motion`, add more simulcast encodings, return to single HIGH, or add a Screener layer selector. 2. **Finish real-network route acceptance.** Repeat the controlled active-SFU recovery transaction on exact production. Then measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, screen-audio continuity and real-game A/V sync across representative IPv4/IPv6 NATs and mobile networks. Include Host and Viewer TUN/VPN cases and an all-UDP-blocked case to distinguish an application ICE-option defect from operating-system interception and establish bounded explicit failure. Do not add another watchdog, port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting. ## Accepted Later Roadmap diff --git a/src/client/media/quality.ts b/src/client/media/quality.ts index b2fba3c..c5796d5 100644 --- a/src/client/media/quality.ts +++ b/src/client/media/quality.ts @@ -78,6 +78,9 @@ export const SCREEN_AUDIO_BITRATES = { } as const satisfies Record; export const SCREEN_AUDIO_RECEIVE_MAX_BITRATE = SCREEN_AUDIO_BITRATES["very-high"]; +export const SCREEN_SHARE_LOW_SCALE = 2; +const SCREEN_SHARE_LOW_MAX_FRAMERATE = 30; +const SCREEN_SHARE_LOW_MIN_BITRATE = 150_000; export interface VideoSenderParameterValues { maxBitrate: number | null; @@ -111,6 +114,21 @@ export function screenAudioBitrate( return SCREEN_AUDIO_BITRATES[resolveScreenAudioQuality(quality)]; } +export function screenShareLowFramerate(profile: QualityProfile): number { + return Math.min(profile.maxFramerate, SCREEN_SHARE_LOW_MAX_FRAMERATE); +} + +export function screenShareLowBitrate(profile: QualityProfile): number { + const lowFramerate = screenShareLowFramerate(profile); + return Math.max( + SCREEN_SHARE_LOW_MIN_BITRATE, + Math.floor( + (profile.maxBitrate * lowFramerate) / + (SCREEN_SHARE_LOW_SCALE ** 2 * profile.maxFramerate), + ), + ); +} + export function qualitySettingsEqual( left: QualitySettings, right: QualitySettings, @@ -304,6 +322,7 @@ export async function configureVideoSender( parameters.encodings = [{}]; } // Pinned LiveKit orders simulcast encodings by increasing spatial resolution. + // Its lower encoding remains SDK-owned so Dynacast cannot race this write. const encodingIndex = parameters.encodings.length - 1; parameters.encodings[encodingIndex]!.maxBitrate = profile.maxBitrate; parameters.encodings[encodingIndex]!.maxFramerate = profile.maxFramerate; diff --git a/src/client/sfu/publisher.ts b/src/client/sfu/publisher.ts index 2e94acd..93a8539 100644 --- a/src/client/sfu/publisher.ts +++ b/src/client/sfu/publisher.ts @@ -9,9 +9,13 @@ import { audioSenderParameterWarning, configureScreenAudioSender, configureVideoSender, + QUALITY_RESOLUTIONS, resolveScreenAudioQuality, screenAudioQualityEqual, screenAudioBitrate, + screenShareLowBitrate, + screenShareLowFramerate, + SCREEN_SHARE_LOW_SCALE, senderParameterWarning, videoQualitySettingsEqual, type AudioSenderParameterReadback, @@ -52,6 +56,11 @@ interface PublishedTrack { rawTrack: MediaStreamTrack; } +interface PublishedVideoOptionsState { + publication: TrackPublishOptions | undefined; + track: TrackPublishOptions | undefined; +} + interface PublishedVideoConfiguration { readback: VideoSenderParameterReadback; warning: string | null; @@ -188,7 +197,7 @@ export class SfuPublisher { room, videoTrack, sdk.Track.Source.ScreenShare, - videoPublishOptions(profile), + videoPublishOptions(sdk, profile, videoTrack), ); if (!this.owns(room, generation)) { return false; @@ -197,6 +206,7 @@ export class SfuPublisher { const videoConfiguration = await configurePublishedVideo( video, profile, + sdk, () => this.owns(room, generation), ); if (!videoConfiguration || !this.owns(room, generation)) { @@ -330,11 +340,18 @@ export class SfuPublisher { const previousAudio = this.audio; let videoReplaceAttempted = false; let audioReplaceAttempted = false; + let previousVideoOptions: PublishedVideoOptionsState | null = null; let audioConfiguration: PublishedAudioConfiguration | null = null; let audioWarning: string | null = null; this.stopStats(); try { + previousVideoOptions = retainPublishedVideoOptions( + previousVideo, + profile, + sdk, + nextVideoTrack, + ); videoReplaceAttempted = true; await replacePublishedTrack(previousVideo, nextVideoTrack); if (!this.owns(room, generation)) { @@ -392,6 +409,7 @@ export class SfuPublisher { const videoConfiguration = await configurePublishedVideo( previousVideo, profile, + sdk, () => this.owns(room, generation), ); if (!videoConfiguration || !this.owns(room, generation)) { @@ -418,6 +436,9 @@ export class SfuPublisher { if (!this.owns(room, generation)) { return false; } + if (previousVideoOptions) { + restorePublishedVideoOptions(previousVideo, previousVideoOptions); + } const failureWarning = "切换 SFU 分享来源失败"; // A rejected publish/unpublish may have changed server state without @@ -445,6 +466,7 @@ export class SfuPublisher { const videoConfiguration = await configurePublishedVideo( previousVideo, previousVideoProfile, + sdk, () => this.owns(room, generation), ); if (!videoConfiguration || !this.owns(room, generation)) { @@ -531,6 +553,7 @@ export class SfuPublisher { const configured = await configurePublishedVideo( video, profile, + sdk, () => this.owns(room, generation) && requestedRevision === this.profileRevision && @@ -562,6 +585,7 @@ export class SfuPublisher { const rolledBack = await configurePublishedVideo( video, previousVideoProfile, + sdk, () => this.owns(room, generation) && requestedRevision === this.profileRevision && @@ -1106,6 +1130,7 @@ async function unpublishTrack(room: Room, published: PublishedTrack): Promise boolean, ): Promise { const videoTrack = published.publication.videoTrack; @@ -1128,19 +1153,54 @@ async function configurePublishedVideo( } return null; } - const retainedPublishOptions = { - ...published.publication.options, - ...videoTrack.publishOptions, - ...videoPublishOptions(profile), - }; - published.publication.options = retainedPublishOptions; - videoTrack.publishOptions = retainedPublishOptions; + retainPublishedVideoOptions( + published, + profile, + sdk, + videoTrack.mediaStreamTrack, + ); return { readback, warning: senderParameterWarning(readback), }; } +function retainPublishedVideoOptions( + published: PublishedTrack, + profile: QualityProfile, + sdk: LiveKit, + track: MediaStreamTrack, +): PublishedVideoOptionsState { + const videoTrack = published.publication.videoTrack; + if (!videoTrack) { + throw new Error("SFU publication has no local video track"); + } + const previous = { + publication: published.publication.options, + track: videoTrack.publishOptions, + }; + const retained = { + ...published.publication.options, + ...videoTrack.publishOptions, + ...videoPublishOptions(sdk, profile, track), + }; + published.publication.options = retained; + videoTrack.publishOptions = retained; + return previous; +} + +function restorePublishedVideoOptions( + published: PublishedTrack, + previous: PublishedVideoOptionsState, +): void { + const videoTrack = published.publication.videoTrack; + if (!videoTrack) { + throw new Error("SFU publication has no local video track"); + } + published.publication.options = previous.publication; + videoTrack.publishOptions = previous.track; +} + async function configurePublishedAudio( published: PublishedTrack, sender: RTCRtpSender, @@ -1214,7 +1274,15 @@ function mergeQualityWarnings(...warnings: Array): string | null return present.length > 0 ? present.join("；") : null; } -function videoPublishOptions(profile: QualityProfile): TrackPublishOptions { +function videoPublishOptions( + sdk: LiveKit, + profile: QualityProfile, + track: MediaStreamTrack, +): TrackPublishOptions { + const requestedResolution = QUALITY_RESOLUTIONS[profile.resolution]; + const settings = track.getSettings(); + const sourceWidth = settings.width ?? requestedResolution.width; + const sourceHeight = settings.height ?? requestedResolution.height; return { backupCodec: false, videoCodec: "vp8", @@ -1222,6 +1290,14 @@ function videoPublishOptions(profile: QualityProfile): TrackPublishOptions { maxBitrate: profile.maxBitrate, maxFramerate: profile.maxFramerate, }, + screenShareSimulcastLayers: [ + new sdk.VideoPreset( + Math.floor(sourceWidth / SCREEN_SHARE_LOW_SCALE), + Math.floor(sourceHeight / SCREEN_SHARE_LOW_SCALE), + screenShareLowBitrate(profile), + screenShareLowFramerate(profile), + ), + ], degradationPreference: profile.degradationPreference, }; } diff --git a/tests/quality.test.ts b/tests/quality.test.ts index b5d962a..866cd60 100644 --- a/tests/quality.test.ts +++ b/tests/quality.test.ts @@ -11,6 +11,8 @@ import { QUALITY_RESOLUTIONS, qualitySettingsEqual, SCREEN_AUDIO_BITRATES, + screenShareLowBitrate, + screenShareLowFramerate, senderParameterWarning, setMediaPaused, } from "../src/client/media/quality.ts"; @@ -118,6 +120,14 @@ describe("realtime quality controls", () => { ).toBe(true); }); + it("derives a half-resolution lower representation without exceeding 30 fps", () => { + expect(screenShareLowFramerate(QUALITY_PROFILES["1080p60"])).toBe(30); + expect(screenShareLowBitrate(QUALITY_PROFILES["1080p60"])).toBe(1_000_000); + expect(screenShareLowFramerate(QUALITY_PROFILES["1080p30"])).toBe(30); + expect(screenShareLowBitrate(QUALITY_PROFILES["1080p30"])).toBe(1_250_000); + expect(screenShareLowBitrate(QUALITY_PROFILES["720p30"])).toBe(750_000); + }); + it("keeps audio selection orthogonal while defaulting old settings to music", () => { const saver = { ...QUALITY_PROFILES["1080p60"], diff --git a/tests/sfu-client.test.ts b/tests/sfu-client.test.ts index 894af8d..2375593 100644 --- a/tests/sfu-client.test.ts +++ b/tests/sfu-client.test.ts @@ -61,6 +61,7 @@ const livekit = vi.hoisted(() => { currentTrack: MediaStreamTrack; sender: FakeSender; publishOptions?: Record; + readonly replaceTrackOptions: Array | undefined> = []; savedDegradationPreference: RTCDegradationPreference | null = null; constructor(track: MediaStreamTrack) { @@ -74,6 +75,7 @@ const livekit = vi.hoisted(() => { readonly replaceTrack = vi.fn( async (nextTrack: MediaStreamTrack): Promise => { + this.replaceTrackOptions.push(this.publishOptions); this.currentTrack = nextTrack; this.sender.track = nextTrack; }, @@ -91,6 +93,19 @@ const livekit = vi.hoisted(() => { } } + class FakeVideoPreset { + readonly encoding: { maxBitrate: number; maxFramerate: number }; + + constructor( + readonly width: number, + readonly height: number, + maxBitrate: number, + maxFramerate: number, + ) { + this.encoding = { maxBitrate, maxFramerate }; + } + } + class FakeLocalParticipant { readonly publications: Array<{ track: FakeLocalTrack; @@ -120,20 +135,30 @@ const livekit = vi.hoisted(() => { ]; } const screenShareEncoding = options.screenShareEncoding; + const layers = options.screenShareSimulcastLayers; if ( rawTrack.kind === "video" && - options.simulcast === false && typeof screenShareEncoding === "object" && - screenShareEncoding !== null + screenShareEncoding !== null && + Array.isArray(layers) && + layers.length === 1 ) { - const encoding = screenShareEncoding as { + const high = screenShareEncoding as { maxBitrate?: number; maxFramerate?: number; }; + const lower = layers[0] as FakeVideoPreset; localTrack.sender.parameters.encodings = [ { - maxBitrate: encoding.maxBitrate, - maxFramerate: encoding.maxFramerate, + rid: "q", + maxBitrate: lower.encoding.maxBitrate, + maxFramerate: lower.encoding.maxFramerate, + scaleResolutionDownBy: 2, + }, + { + rid: "h", + maxBitrate: high.maxBitrate, + maxFramerate: high.maxFramerate, scaleResolutionDownBy: 1, }, ]; @@ -256,6 +281,7 @@ const livekit = vi.hoisted(() => { FakeRemoteParticipant, FakeRemotePublication, FakeRoom, + FakeVideoPreset, state, }; }); @@ -285,6 +311,7 @@ vi.mock("livekit-client", () => ({ Room: livekit.FakeRoom, RoomEvent, Track, + VideoPreset: livekit.FakeVideoPreset, })); class FakeMediaStream { @@ -324,11 +351,15 @@ const qualityProfile = { degradationPreference: "maintain-resolution", } as const; -function track(kind: "video" | "audio", id: string): MediaStreamTrack { +function track( + kind: "video" | "audio", + id: string, + settings: MediaTrackSettings = {}, +): MediaStreamTrack { return Object.assign(new EventTarget(), { id, kind, - getSettings: () => ({}), + getSettings: () => settings, }) as MediaStreamTrack; } @@ -541,6 +572,21 @@ describe("SfuPublisher", () => { }); await expect(publisher.replaceStream(stream(nextVideo))).resolves.toBe(true); expect(updates.at(-1)).toBeNull(); + const currentPublication = livekit.state.rooms[0].localParticipant.publications[0]; + expect(currentPublication.options).toMatchObject({ + screenShareSimulcastLayers: [ + { + width: 640, + height: 360, + encoding: { maxBitrate: 1_000_000, maxFramerate: 30 }, + }, + ], + }); + expect(currentPublication.track.replaceTrackOptions[0]).toMatchObject({ + screenShareSimulcastLayers: [ + expect.objectContaining({ width: 640, height: 360 }), + ], + }); resolveOld(senderReport(previousVideo.id, 5_000, 500_000, 300)); await Promise.resolve(); expect(updates.at(-1)).toBeNull(); @@ -647,7 +693,7 @@ describe("SfuPublisher", () => { expect(livekit.state.rooms[0]?.disconnect).toHaveBeenCalledWith(false); }); - it("leaves VP8 screen-share simulcast layers to LiveKit defaults", async () => { + it("publishes a lightweight half-resolution VP8 representation", async () => { const publisher = new SfuPublisher(); const video = track("video", "video-1"); const audio = track("audio", "audio-1"); @@ -667,6 +713,13 @@ describe("SfuPublisher", () => { maxBitrate: 8_000_000, maxFramerate: 60, }, + screenShareSimulcastLayers: [ + { + width: 960, + height: 540, + encoding: { maxBitrate: 1_000_000, maxFramerate: 30 }, + }, + ], degradationPreference: "maintain-resolution", }); expect( @@ -681,12 +734,18 @@ describe("SfuPublisher", () => { expect(sender.setParameters).toHaveBeenCalledOnce(); expect(sender.parameters.encodings).toEqual([ { + rid: "q", + maxBitrate: 1_000_000, + maxFramerate: 30, + scaleResolutionDownBy: 2, + }, + { + rid: "h", maxBitrate: 8_000_000, maxFramerate: 60, scaleResolutionDownBy: 1, }, ]); - expect(sender.parameters.encodings[0]).not.toHaveProperty("rid"); expect(publisher.getSenderParameters()).toEqual({ requested: { maxBitrate: 8_000_000, @@ -710,6 +769,37 @@ describe("SfuPublisher", () => { expect(room.disconnect).not.toHaveBeenCalled(); }); + it("derives the lower representation from the actual capture dimensions", async () => { + const publisher = new SfuPublisher(); + const video = track("video", "video-1", { width: 1904, height: 928 }); + await publisher.connect(connection); + + await expect(publisher.activate(stream(video), qualityProfile)).resolves.toBe( + true, + ); + + const publication = livekit.state.rooms[0].localParticipant.publications[0]; + expect(publication.options).toMatchObject({ + screenShareSimulcastLayers: [ + { + width: 952, + height: 464, + encoding: { maxBitrate: 1_000_000, maxFramerate: 30 }, + }, + ], + }); + expect(publication.track.sender.parameters.encodings).toEqual([ + expect.objectContaining({ + rid: "q", + scaleResolutionDownBy: 2, + }), + expect.objectContaining({ + rid: "h", + scaleResolutionDownBy: 1, + }), + ]); + }); + it.each([ ["saver", 64_000], ["music", 128_000], @@ -786,6 +876,13 @@ describe("SfuPublisher", () => { degradationPreference: "balanced", encodings: [ expect.objectContaining({ + rid: "q", + maxBitrate: 1_000_000, + maxFramerate: 30, + scaleResolutionDownBy: 2, + }), + expect.objectContaining({ + rid: "h", maxBitrate: 3_000_000, maxFramerate: 30, scaleResolutionDownBy: 1, @@ -1138,6 +1235,13 @@ describe("SfuPublisher", () => { degradationPreference: "maintain-framerate", encodings: [ { + rid: "q", + maxBitrate: 1_000_000, + maxFramerate: 30, + scaleResolutionDownBy: 2, + }, + { + rid: "h", maxBitrate: 3_000_000, maxFramerate: 30, scaleResolutionDownBy: 1, @@ -1152,6 +1256,13 @@ describe("SfuPublisher", () => { maxBitrate: 3_000_000, maxFramerate: 30, }, + screenShareSimulcastLayers: [ + { + width: 640, + height: 360, + encoding: { maxBitrate: 750_000, maxFramerate: 30 }, + }, + ], degradationPreference: "maintain-framerate", }); @@ -1308,6 +1419,12 @@ describe("SfuPublisher", () => { degradationPreference: "maintain-resolution", encodings: [ expect.objectContaining({ + rid: "q", + maxBitrate: 1_000_000, + maxFramerate: 30, + }), + expect.objectContaining({ + rid: "h", maxBitrate: 8_000_000, maxFramerate: 60, }), @@ -1355,13 +1472,20 @@ describe("SfuPublisher", () => { }); it("restores the previous video after a partially applied replacement fails", async () => { - const previousVideo = track("video", "video-1"); - const nextVideo = track("video", "video-2"); + const previousVideo = track("video", "video-1", { + width: 1904, + height: 928, + }); + const nextVideo = track("video", "video-2", { + width: 1280, + height: 720, + }); const publisher = new SfuPublisher(); await publisher.connect(connection); await publisher.activate(stream(previousVideo), qualityProfile); const localTrack = livekit.state.rooms[0].localParticipant.publications[0].track; localTrack.replaceTrack.mockImplementationOnce(async (replacement) => { + localTrack.replaceTrackOptions.push(localTrack.publishOptions); localTrack.currentTrack = replacement; throw new Error("sender swap failed after applying"); }); @@ -1371,6 +1495,25 @@ describe("SfuPublisher", () => { expect(localTrack.replaceTrack).toHaveBeenNthCalledWith(1, nextVideo); expect(localTrack.replaceTrack).toHaveBeenNthCalledWith(2, previousVideo); expect(localTrack.currentTrack).toBe(previousVideo); + expect(localTrack.publishOptions).toMatchObject({ + screenShareSimulcastLayers: [ + expect.objectContaining({ width: 952, height: 464 }), + ], + }); + expect(localTrack.sender.parameters.encodings).toEqual([ + expect.objectContaining({ rid: "q", scaleResolutionDownBy: 2 }), + expect.objectContaining({ rid: "h", scaleResolutionDownBy: 1 }), + ]); + expect(localTrack.replaceTrackOptions[0]).toMatchObject({ + screenShareSimulcastLayers: [ + expect.objectContaining({ width: 640, height: 360 }), + ], + }); + expect(localTrack.replaceTrackOptions[1]).toMatchObject({ + screenShareSimulcastLayers: [ + expect.objectContaining({ width: 952, height: 464 }), + ], + }); }); it("uses the screen audio preset when a replacement adds audio", async () => {