fix(sfu): lighten the lower representation (#255)
This commit is contained in:
1 parent
3f36af04f1
commit
d2fcf8d711
8 files changed
+279
-25
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# ADR-0007: LiveKit-Owned SFU Representation Adaptation
|
||||
|
||||
- Status: Framework adaptation deployed; lightweight lower representation accepted and pending
|
||||
- Status: Lightweight lower representation implemented in current source; not deployed
|
||||
- Date: 2026-08-19
|
||||
- Last reviewed: 2026-08-25
|
||||
|
||||
@@ -64,6 +64,12 @@ while a subscriber's default `HIGH` ceiling still selects the highest available
|
||||
encoding. Dynacast, server send-side BWE, and disabled AdaptiveStream remain
|
||||
unchanged.
|
||||
|
||||
Current source derives the lower preset from the actual capture dimensions and
|
||||
passes it through LiveKit's public publication options. Retained publication
|
||||
options give LiveKit the same preset on its own source-replacement or republish
|
||||
boundary; Screener does not mutate the active lower encoding behind LiveKit's
|
||||
Dynacast sender lock.
|
||||
|
||||
## Pinned Physical Result
|
||||
|
||||
The 2026-08-25 gate used LiveKit server `1.13.5`, client `2.22.0`, Chrome 151,
|
||||
|
||||
@@ -33,7 +33,7 @@ Last updated: 2026-08-25
|
||||
- Current source and production admit up to `20` Viewers and implement the uniform `1/2/3` endpoint cap, exact prepare candidate, first-decoded-frame commit, strict rollback revision, decoded-progress stall, one direct/SFU operation deadline, and one event-driven controller. Active-SFU sampling remains live through an absent track/report or stats failure. Ordinary peer ICE is STUN-only and the sole application fallback is SFU/UDP. Exact SFU admission and empty Browser SFU ICE-server options remain unchanged.
|
||||
- Source and production use random four-digit memory rooms, dormant leases, exact-Host resume, restart loss, local preference replay, orthogonal grants/code entry, and no SQLite runtime. V12 uses a 22-character room-incarnation grant, `ROOM_NOT_FOUND`, no diagnostic-file export, and a compact UI showing the complete shorter URL.
|
||||
- Exact production artifacts, services, ports, and any change-scoped infrastructure recovery boundary are owned by [deployment](./deployment.md) and indexed by [status](./status.md).
|
||||
- Source and production require v12, fixed VP8, and no video hint, and currently use pinned-default SFU half-resolution plus original-resolution simulcast, Dynacast, and server send-side BWE. The lightweight lower-representation correction is accepted but not yet implemented or deployed. Other current media and UI behavior matches the truths above.
|
||||
- Source and production require v12, fixed VP8, and no video hint. Current source derives a half-actual-resolution SFU lower preset capped at 30 fps with proportional bitrate; production still uses pinned-default same-FPS lower plus original-resolution simulcast. Both retain Dynacast, server send-side BWE, and disabled AdaptiveStream. Other current media and UI behavior matches the truths above.
|
||||
- Browser SFU subscribers reconcile eligible Host screen publications across connect/activate event ordering, participant/track arrival, and reconnect. Production LiveKit logs only warn/error, and the application service has a bounded same-host LiveKit readiness preflight.
|
||||
- The deployed application tree passed TypeScript, 611 Web tests, both builds, hygiene, immutable artifact verification, and production postflight. Earlier exact-production gates closed direct/browser-relay VP8 cadence, active-audio ceilings, a single-representation SFU baseline, and one controlled exact-candidate SFU commit; the pinned-stack gate proved native representation selection. Controlled recovery on this release, representative networks, all-UDP-blocked failure, public SFU 60 fps, and real-game synchronization remain unproven. Exact metrics live in [verification status](./verification-status.md).
|
||||
- Chrome loopback, cap2/cap3, short resource, recovery, MBB, and A/V fixtures are bounded evidence only. They do not establish quality, resource limits, or product policy.
|
||||
|
||||
+2
-2
@@ -22,14 +22,14 @@ This is the current execution index. Git history owns completed timelines; [veri
|
||||
- One event-driven controller owns the committed graph and at most one room-serial child operation. It uses one deterministic candidate list/cursor and one total direct-then-SFU deadline; exact admission and physical resources remain charged through drain, and the pending candidate commits only after an exact-generation decoded-frame proof. A short-lived 100 ms stats observer proves only that candidate inside the existing deadline. The active-path 2-second sampler now keeps its existing 15-second decoded-progress cadence when an SFU subscriber has no video track, no merged report, or a failed stats read, without adding a timer or fabricating metrics. Healthy edges remain sticky and current-path quality does not authorize reparenting.
|
||||
- Ordinary peer ICE is STUN-only and the only application fallback is the dedicated LiveKit SFU over UDP. SFU generations use explicit no-default ingress/egress capacities, exact `reserved | committed | draining` accounting, delete-plus-absence release, one Host publication, and exact Viewer subscription handles. No Screener TURN, ICE/TCP, media TCP, or TLS-relayed media route exists.
|
||||
- Current Browser SFU publisher and subscriber connects supply fresh `autoSubscribe: false` plus `rtcConfig: { iceServers: [] }` options. Ordinary peers retain deployment STUN; LiveKit signaling candidates, server-side public-IP discovery, route deadlines, ports, and retry behavior are unchanged.
|
||||
- Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec. SFU publication fixes VP8 with no backup codec, leaves screen-share simulcast layers unset so pinned client `2.22.0` supplies its half-resolution plus original-resolution pair, enables Dynacast, retains LiveKit's default maximum subscriber ceiling, and keeps AdaptiveStream disabled. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation remains unchanged; ordinary Host and SFU publisher metrics select the exact current sender, relays reconcile audio-track additions and removals on the persistent stream, and an SFU reconnect reacquires the exact current publication/sender, avoids unchanged parameter/stat resets, or fails the route closed.
|
||||
- Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec. SFU publication fixes VP8 with no backup codec and supplies LiveKit one half-actual-resolution lower preset capped at 30 fps with proportional bitrate beside the original representation; Dynacast, the default maximum subscriber ceiling, server send-side BWE, and disabled AdaptiveStream remain unchanged. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation remains unchanged; ordinary Host and SFU publisher metrics select the exact current sender, relays reconcile audio-track additions and removals on the persistent stream, and an SFU reconnect reacquires the exact current publication/sender, avoids unchanged parameter/stat resets, or fails the route closed.
|
||||
- Current source implements typed first-frame Viewer presentation and recovery, Chinese user-facing route/access failures, local connection details without a diagnostic-file export, authenticated Host-only on-demand route snapshots for automatic acceptance, explicit missing-room presentation, and responsive entry controls. Host invitation controls remain independent from two-state room-code entry.
|
||||
- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`; current source defaults to `1080p30`. Advanced resolution adds `480p` as `854x480` without adding a fourth preset, and advanced FPS and bitrate remain independent.
|
||||
- The deployed application tree passed repository hygiene, TypeScript, all 611 Web tests in 44 files, both production builds, and immutable artifact/postflight checks. Earlier exact-production gates exercised direct, browser-relay, SFU audio continuity and all three audio ceilings, and sustained about 59.5 fps on direct and browser-relay paths; the pinned-stack gate proved native two-representation selection before deployment. Exact applicability remains in [verification status](./verification-status.md).
|
||||
|
||||
## Current Milestone
|
||||
|
||||
1. Apply and verify ADR-0007's lightweight half-resolution representation, then repeat the controlled active-SFU recovery transaction on exact production and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model.
|
||||
1. Deploy and verify ADR-0007's implemented lightweight half-resolution representation, then repeat the controlled active-SFU recovery transaction on exact production and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model.
|
||||
|
||||
## Active Boundaries
|
||||
|
||||
|
||||
+1
-1
@@ -6,7 +6,7 @@ Only items in **Now** are executable after their stated decision gate. A branch
|
||||
|
||||
## Now
|
||||
|
||||
1. **Correct constrained SFU publication quality.** Keep the two LiveKit-owned VP8 spatial representations, Dynacast, send-side BWE, and no AdaptiveStream. Cap the half-resolution representation at 30 fps and derive its bitrate from the pinned SDK's proportional formula, including live profile updates. In the same mixed P2P/SFU dynamic-source gate, verify that HIGH recovers materially, P2P does not regress, and a constrained SFU downlink still selects and decodes LOW before deployment. Do not restore global `motion`, add more simulcast encodings, return to single HIGH, or add a Screener layer selector.
|
||||
1. **Deploy and verify constrained SFU publication quality.** Current source keeps the two LiveKit-owned VP8 spatial representations, Dynacast, send-side BWE, and no AdaptiveStream. It derives the half-actual-resolution lower publication preset at no more than 30 fps with proportional bitrate and retains that option for LiveKit-owned source replacement or republish; it does not mutate LOW behind the SDK's Dynacast lock. In the same mixed P2P/SFU dynamic-source gate, verify that HIGH recovers materially, P2P does not regress, and a constrained SFU downlink still selects and decodes LOW before deployment. Do not restore global `motion`, add more simulcast encodings, return to single HIGH, or add a Screener layer selector.
|
||||
2. **Finish real-network route acceptance.** Repeat the controlled active-SFU recovery transaction on exact production. Then measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, screen-audio continuity and real-game A/V sync across representative IPv4/IPv6 NATs and mobile networks. Include Host and Viewer TUN/VPN cases and an all-UDP-blocked case to distinguish an application ICE-option defect from operating-system interception and establish bounded explicit failure. Do not add another watchdog, port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting.
|
||||
|
||||
## Accepted Later Roadmap
|
||||
|
||||
@@ -78,6 +78,9 @@ export const SCREEN_AUDIO_BITRATES = {
|
||||
} as const satisfies Record<ScreenAudioQuality, number>;
|
||||
export const SCREEN_AUDIO_RECEIVE_MAX_BITRATE =
|
||||
SCREEN_AUDIO_BITRATES["very-high"];
|
||||
export const SCREEN_SHARE_LOW_SCALE = 2;
|
||||
const SCREEN_SHARE_LOW_MAX_FRAMERATE = 30;
|
||||
const SCREEN_SHARE_LOW_MIN_BITRATE = 150_000;
|
||||
|
||||
export interface VideoSenderParameterValues {
|
||||
maxBitrate: number | null;
|
||||
@@ -111,6 +114,21 @@ export function screenAudioBitrate(
|
||||
return SCREEN_AUDIO_BITRATES[resolveScreenAudioQuality(quality)];
|
||||
}
|
||||
|
||||
export function screenShareLowFramerate(profile: QualityProfile): number {
|
||||
return Math.min(profile.maxFramerate, SCREEN_SHARE_LOW_MAX_FRAMERATE);
|
||||
}
|
||||
|
||||
export function screenShareLowBitrate(profile: QualityProfile): number {
|
||||
const lowFramerate = screenShareLowFramerate(profile);
|
||||
return Math.max(
|
||||
SCREEN_SHARE_LOW_MIN_BITRATE,
|
||||
Math.floor(
|
||||
(profile.maxBitrate * lowFramerate) /
|
||||
(SCREEN_SHARE_LOW_SCALE ** 2 * profile.maxFramerate),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export function qualitySettingsEqual(
|
||||
left: QualitySettings,
|
||||
right: QualitySettings,
|
||||
@@ -304,6 +322,7 @@ export async function configureVideoSender(
|
||||
parameters.encodings = [{}];
|
||||
}
|
||||
// Pinned LiveKit orders simulcast encodings by increasing spatial resolution.
|
||||
// Its lower encoding remains SDK-owned so Dynacast cannot race this write.
|
||||
const encodingIndex = parameters.encodings.length - 1;
|
||||
parameters.encodings[encodingIndex]!.maxBitrate = profile.maxBitrate;
|
||||
parameters.encodings[encodingIndex]!.maxFramerate = profile.maxFramerate;
|
||||
|
||||
@@ -9,9 +9,13 @@ import {
|
||||
audioSenderParameterWarning,
|
||||
configureScreenAudioSender,
|
||||
configureVideoSender,
|
||||
QUALITY_RESOLUTIONS,
|
||||
resolveScreenAudioQuality,
|
||||
screenAudioQualityEqual,
|
||||
screenAudioBitrate,
|
||||
screenShareLowBitrate,
|
||||
screenShareLowFramerate,
|
||||
SCREEN_SHARE_LOW_SCALE,
|
||||
senderParameterWarning,
|
||||
videoQualitySettingsEqual,
|
||||
type AudioSenderParameterReadback,
|
||||
@@ -52,6 +56,11 @@ interface PublishedTrack {
|
||||
rawTrack: MediaStreamTrack;
|
||||
}
|
||||
|
||||
interface PublishedVideoOptionsState {
|
||||
publication: TrackPublishOptions | undefined;
|
||||
track: TrackPublishOptions | undefined;
|
||||
}
|
||||
|
||||
interface PublishedVideoConfiguration {
|
||||
readback: VideoSenderParameterReadback;
|
||||
warning: string | null;
|
||||
@@ -188,7 +197,7 @@ export class SfuPublisher {
|
||||
room,
|
||||
videoTrack,
|
||||
sdk.Track.Source.ScreenShare,
|
||||
videoPublishOptions(profile),
|
||||
videoPublishOptions(sdk, profile, videoTrack),
|
||||
);
|
||||
if (!this.owns(room, generation)) {
|
||||
return false;
|
||||
@@ -197,6 +206,7 @@ export class SfuPublisher {
|
||||
const videoConfiguration = await configurePublishedVideo(
|
||||
video,
|
||||
profile,
|
||||
sdk,
|
||||
() => this.owns(room, generation),
|
||||
);
|
||||
if (!videoConfiguration || !this.owns(room, generation)) {
|
||||
@@ -330,11 +340,18 @@ export class SfuPublisher {
|
||||
const previousAudio = this.audio;
|
||||
let videoReplaceAttempted = false;
|
||||
let audioReplaceAttempted = false;
|
||||
let previousVideoOptions: PublishedVideoOptionsState | null = null;
|
||||
let audioConfiguration: PublishedAudioConfiguration | null = null;
|
||||
let audioWarning: string | null = null;
|
||||
this.stopStats();
|
||||
|
||||
try {
|
||||
previousVideoOptions = retainPublishedVideoOptions(
|
||||
previousVideo,
|
||||
profile,
|
||||
sdk,
|
||||
nextVideoTrack,
|
||||
);
|
||||
videoReplaceAttempted = true;
|
||||
await replacePublishedTrack(previousVideo, nextVideoTrack);
|
||||
if (!this.owns(room, generation)) {
|
||||
@@ -392,6 +409,7 @@ export class SfuPublisher {
|
||||
const videoConfiguration = await configurePublishedVideo(
|
||||
previousVideo,
|
||||
profile,
|
||||
sdk,
|
||||
() => this.owns(room, generation),
|
||||
);
|
||||
if (!videoConfiguration || !this.owns(room, generation)) {
|
||||
@@ -418,6 +436,9 @@ export class SfuPublisher {
|
||||
if (!this.owns(room, generation)) {
|
||||
return false;
|
||||
}
|
||||
if (previousVideoOptions) {
|
||||
restorePublishedVideoOptions(previousVideo, previousVideoOptions);
|
||||
}
|
||||
const failureWarning = "切换 SFU 分享来源失败";
|
||||
|
||||
// A rejected publish/unpublish may have changed server state without
|
||||
@@ -445,6 +466,7 @@ export class SfuPublisher {
|
||||
const videoConfiguration = await configurePublishedVideo(
|
||||
previousVideo,
|
||||
previousVideoProfile,
|
||||
sdk,
|
||||
() => this.owns(room, generation),
|
||||
);
|
||||
if (!videoConfiguration || !this.owns(room, generation)) {
|
||||
@@ -531,6 +553,7 @@ export class SfuPublisher {
|
||||
const configured = await configurePublishedVideo(
|
||||
video,
|
||||
profile,
|
||||
sdk,
|
||||
() =>
|
||||
this.owns(room, generation) &&
|
||||
requestedRevision === this.profileRevision &&
|
||||
@@ -562,6 +585,7 @@ export class SfuPublisher {
|
||||
const rolledBack = await configurePublishedVideo(
|
||||
video,
|
||||
previousVideoProfile,
|
||||
sdk,
|
||||
() =>
|
||||
this.owns(room, generation) &&
|
||||
requestedRevision === this.profileRevision &&
|
||||
@@ -1106,6 +1130,7 @@ async function unpublishTrack(room: Room, published: PublishedTrack): Promise<vo
|
||||
async function configurePublishedVideo(
|
||||
published: PublishedTrack,
|
||||
profile: QualityProfile,
|
||||
sdk: LiveKit,
|
||||
ownsPublication: () => boolean,
|
||||
): Promise<PublishedVideoConfiguration | null> {
|
||||
const videoTrack = published.publication.videoTrack;
|
||||
@@ -1128,19 +1153,54 @@ async function configurePublishedVideo(
|
||||
}
|
||||
return null;
|
||||
}
|
||||
const retainedPublishOptions = {
|
||||
...published.publication.options,
|
||||
...videoTrack.publishOptions,
|
||||
...videoPublishOptions(profile),
|
||||
};
|
||||
published.publication.options = retainedPublishOptions;
|
||||
videoTrack.publishOptions = retainedPublishOptions;
|
||||
retainPublishedVideoOptions(
|
||||
published,
|
||||
profile,
|
||||
sdk,
|
||||
videoTrack.mediaStreamTrack,
|
||||
);
|
||||
return {
|
||||
readback,
|
||||
warning: senderParameterWarning(readback),
|
||||
};
|
||||
}
|
||||
|
||||
function retainPublishedVideoOptions(
|
||||
published: PublishedTrack,
|
||||
profile: QualityProfile,
|
||||
sdk: LiveKit,
|
||||
track: MediaStreamTrack,
|
||||
): PublishedVideoOptionsState {
|
||||
const videoTrack = published.publication.videoTrack;
|
||||
if (!videoTrack) {
|
||||
throw new Error("SFU publication has no local video track");
|
||||
}
|
||||
const previous = {
|
||||
publication: published.publication.options,
|
||||
track: videoTrack.publishOptions,
|
||||
};
|
||||
const retained = {
|
||||
...published.publication.options,
|
||||
...videoTrack.publishOptions,
|
||||
...videoPublishOptions(sdk, profile, track),
|
||||
};
|
||||
published.publication.options = retained;
|
||||
videoTrack.publishOptions = retained;
|
||||
return previous;
|
||||
}
|
||||
|
||||
function restorePublishedVideoOptions(
|
||||
published: PublishedTrack,
|
||||
previous: PublishedVideoOptionsState,
|
||||
): void {
|
||||
const videoTrack = published.publication.videoTrack;
|
||||
if (!videoTrack) {
|
||||
throw new Error("SFU publication has no local video track");
|
||||
}
|
||||
published.publication.options = previous.publication;
|
||||
videoTrack.publishOptions = previous.track;
|
||||
}
|
||||
|
||||
async function configurePublishedAudio(
|
||||
published: PublishedTrack,
|
||||
sender: RTCRtpSender,
|
||||
@@ -1214,7 +1274,15 @@ function mergeQualityWarnings(...warnings: Array<string | null>): string | null
|
||||
return present.length > 0 ? present.join(";") : null;
|
||||
}
|
||||
|
||||
function videoPublishOptions(profile: QualityProfile): TrackPublishOptions {
|
||||
function videoPublishOptions(
|
||||
sdk: LiveKit,
|
||||
profile: QualityProfile,
|
||||
track: MediaStreamTrack,
|
||||
): TrackPublishOptions {
|
||||
const requestedResolution = QUALITY_RESOLUTIONS[profile.resolution];
|
||||
const settings = track.getSettings();
|
||||
const sourceWidth = settings.width ?? requestedResolution.width;
|
||||
const sourceHeight = settings.height ?? requestedResolution.height;
|
||||
return {
|
||||
backupCodec: false,
|
||||
videoCodec: "vp8",
|
||||
@@ -1222,6 +1290,14 @@ function videoPublishOptions(profile: QualityProfile): TrackPublishOptions {
|
||||
maxBitrate: profile.maxBitrate,
|
||||
maxFramerate: profile.maxFramerate,
|
||||
},
|
||||
screenShareSimulcastLayers: [
|
||||
new sdk.VideoPreset(
|
||||
Math.floor(sourceWidth / SCREEN_SHARE_LOW_SCALE),
|
||||
Math.floor(sourceHeight / SCREEN_SHARE_LOW_SCALE),
|
||||
screenShareLowBitrate(profile),
|
||||
screenShareLowFramerate(profile),
|
||||
),
|
||||
],
|
||||
degradationPreference: profile.degradationPreference,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -11,6 +11,8 @@ import {
|
||||
QUALITY_RESOLUTIONS,
|
||||
qualitySettingsEqual,
|
||||
SCREEN_AUDIO_BITRATES,
|
||||
screenShareLowBitrate,
|
||||
screenShareLowFramerate,
|
||||
senderParameterWarning,
|
||||
setMediaPaused,
|
||||
} from "../src/client/media/quality.ts";
|
||||
@@ -118,6 +120,14 @@ describe("realtime quality controls", () => {
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("derives a half-resolution lower representation without exceeding 30 fps", () => {
|
||||
expect(screenShareLowFramerate(QUALITY_PROFILES["1080p60"])).toBe(30);
|
||||
expect(screenShareLowBitrate(QUALITY_PROFILES["1080p60"])).toBe(1_000_000);
|
||||
expect(screenShareLowFramerate(QUALITY_PROFILES["1080p30"])).toBe(30);
|
||||
expect(screenShareLowBitrate(QUALITY_PROFILES["1080p30"])).toBe(1_250_000);
|
||||
expect(screenShareLowBitrate(QUALITY_PROFILES["720p30"])).toBe(750_000);
|
||||
});
|
||||
|
||||
it("keeps audio selection orthogonal while defaulting old settings to music", () => {
|
||||
const saver = {
|
||||
...QUALITY_PROFILES["1080p60"],
|
||||
|
||||
+154
-11
@@ -61,6 +61,7 @@ const livekit = vi.hoisted(() => {
|
||||
currentTrack: MediaStreamTrack;
|
||||
sender: FakeSender;
|
||||
publishOptions?: Record<string, unknown>;
|
||||
readonly replaceTrackOptions: Array<Record<string, unknown> | undefined> = [];
|
||||
savedDegradationPreference: RTCDegradationPreference | null = null;
|
||||
|
||||
constructor(track: MediaStreamTrack) {
|
||||
@@ -74,6 +75,7 @@ const livekit = vi.hoisted(() => {
|
||||
|
||||
readonly replaceTrack = vi.fn(
|
||||
async (nextTrack: MediaStreamTrack): Promise<void> => {
|
||||
this.replaceTrackOptions.push(this.publishOptions);
|
||||
this.currentTrack = nextTrack;
|
||||
this.sender.track = nextTrack;
|
||||
},
|
||||
@@ -91,6 +93,19 @@ const livekit = vi.hoisted(() => {
|
||||
}
|
||||
}
|
||||
|
||||
class FakeVideoPreset {
|
||||
readonly encoding: { maxBitrate: number; maxFramerate: number };
|
||||
|
||||
constructor(
|
||||
readonly width: number,
|
||||
readonly height: number,
|
||||
maxBitrate: number,
|
||||
maxFramerate: number,
|
||||
) {
|
||||
this.encoding = { maxBitrate, maxFramerate };
|
||||
}
|
||||
}
|
||||
|
||||
class FakeLocalParticipant {
|
||||
readonly publications: Array<{
|
||||
track: FakeLocalTrack;
|
||||
@@ -120,20 +135,30 @@ const livekit = vi.hoisted(() => {
|
||||
];
|
||||
}
|
||||
const screenShareEncoding = options.screenShareEncoding;
|
||||
const layers = options.screenShareSimulcastLayers;
|
||||
if (
|
||||
rawTrack.kind === "video" &&
|
||||
options.simulcast === false &&
|
||||
typeof screenShareEncoding === "object" &&
|
||||
screenShareEncoding !== null
|
||||
screenShareEncoding !== null &&
|
||||
Array.isArray(layers) &&
|
||||
layers.length === 1
|
||||
) {
|
||||
const encoding = screenShareEncoding as {
|
||||
const high = screenShareEncoding as {
|
||||
maxBitrate?: number;
|
||||
maxFramerate?: number;
|
||||
};
|
||||
const lower = layers[0] as FakeVideoPreset;
|
||||
localTrack.sender.parameters.encodings = [
|
||||
{
|
||||
maxBitrate: encoding.maxBitrate,
|
||||
maxFramerate: encoding.maxFramerate,
|
||||
rid: "q",
|
||||
maxBitrate: lower.encoding.maxBitrate,
|
||||
maxFramerate: lower.encoding.maxFramerate,
|
||||
scaleResolutionDownBy: 2,
|
||||
},
|
||||
{
|
||||
rid: "h",
|
||||
maxBitrate: high.maxBitrate,
|
||||
maxFramerate: high.maxFramerate,
|
||||
scaleResolutionDownBy: 1,
|
||||
},
|
||||
];
|
||||
@@ -256,6 +281,7 @@ const livekit = vi.hoisted(() => {
|
||||
FakeRemoteParticipant,
|
||||
FakeRemotePublication,
|
||||
FakeRoom,
|
||||
FakeVideoPreset,
|
||||
state,
|
||||
};
|
||||
});
|
||||
@@ -285,6 +311,7 @@ vi.mock("livekit-client", () => ({
|
||||
Room: livekit.FakeRoom,
|
||||
RoomEvent,
|
||||
Track,
|
||||
VideoPreset: livekit.FakeVideoPreset,
|
||||
}));
|
||||
|
||||
class FakeMediaStream {
|
||||
@@ -324,11 +351,15 @@ const qualityProfile = {
|
||||
degradationPreference: "maintain-resolution",
|
||||
} as const;
|
||||
|
||||
function track(kind: "video" | "audio", id: string): MediaStreamTrack {
|
||||
function track(
|
||||
kind: "video" | "audio",
|
||||
id: string,
|
||||
settings: MediaTrackSettings = {},
|
||||
): MediaStreamTrack {
|
||||
return Object.assign(new EventTarget(), {
|
||||
id,
|
||||
kind,
|
||||
getSettings: () => ({}),
|
||||
getSettings: () => settings,
|
||||
}) as MediaStreamTrack;
|
||||
}
|
||||
|
||||
@@ -541,6 +572,21 @@ describe("SfuPublisher", () => {
|
||||
});
|
||||
await expect(publisher.replaceStream(stream(nextVideo))).resolves.toBe(true);
|
||||
expect(updates.at(-1)).toBeNull();
|
||||
const currentPublication = livekit.state.rooms[0].localParticipant.publications[0];
|
||||
expect(currentPublication.options).toMatchObject({
|
||||
screenShareSimulcastLayers: [
|
||||
{
|
||||
width: 640,
|
||||
height: 360,
|
||||
encoding: { maxBitrate: 1_000_000, maxFramerate: 30 },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(currentPublication.track.replaceTrackOptions[0]).toMatchObject({
|
||||
screenShareSimulcastLayers: [
|
||||
expect.objectContaining({ width: 640, height: 360 }),
|
||||
],
|
||||
});
|
||||
resolveOld(senderReport(previousVideo.id, 5_000, 500_000, 300));
|
||||
await Promise.resolve();
|
||||
expect(updates.at(-1)).toBeNull();
|
||||
@@ -647,7 +693,7 @@ describe("SfuPublisher", () => {
|
||||
expect(livekit.state.rooms[0]?.disconnect).toHaveBeenCalledWith(false);
|
||||
});
|
||||
|
||||
it("leaves VP8 screen-share simulcast layers to LiveKit defaults", async () => {
|
||||
it("publishes a lightweight half-resolution VP8 representation", async () => {
|
||||
const publisher = new SfuPublisher();
|
||||
const video = track("video", "video-1");
|
||||
const audio = track("audio", "audio-1");
|
||||
@@ -667,6 +713,13 @@ describe("SfuPublisher", () => {
|
||||
maxBitrate: 8_000_000,
|
||||
maxFramerate: 60,
|
||||
},
|
||||
screenShareSimulcastLayers: [
|
||||
{
|
||||
width: 960,
|
||||
height: 540,
|
||||
encoding: { maxBitrate: 1_000_000, maxFramerate: 30 },
|
||||
},
|
||||
],
|
||||
degradationPreference: "maintain-resolution",
|
||||
});
|
||||
expect(
|
||||
@@ -681,12 +734,18 @@ describe("SfuPublisher", () => {
|
||||
expect(sender.setParameters).toHaveBeenCalledOnce();
|
||||
expect(sender.parameters.encodings).toEqual([
|
||||
{
|
||||
rid: "q",
|
||||
maxBitrate: 1_000_000,
|
||||
maxFramerate: 30,
|
||||
scaleResolutionDownBy: 2,
|
||||
},
|
||||
{
|
||||
rid: "h",
|
||||
maxBitrate: 8_000_000,
|
||||
maxFramerate: 60,
|
||||
scaleResolutionDownBy: 1,
|
||||
},
|
||||
]);
|
||||
expect(sender.parameters.encodings[0]).not.toHaveProperty("rid");
|
||||
expect(publisher.getSenderParameters()).toEqual({
|
||||
requested: {
|
||||
maxBitrate: 8_000_000,
|
||||
@@ -710,6 +769,37 @@ describe("SfuPublisher", () => {
|
||||
expect(room.disconnect).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("derives the lower representation from the actual capture dimensions", async () => {
|
||||
const publisher = new SfuPublisher();
|
||||
const video = track("video", "video-1", { width: 1904, height: 928 });
|
||||
await publisher.connect(connection);
|
||||
|
||||
await expect(publisher.activate(stream(video), qualityProfile)).resolves.toBe(
|
||||
true,
|
||||
);
|
||||
|
||||
const publication = livekit.state.rooms[0].localParticipant.publications[0];
|
||||
expect(publication.options).toMatchObject({
|
||||
screenShareSimulcastLayers: [
|
||||
{
|
||||
width: 952,
|
||||
height: 464,
|
||||
encoding: { maxBitrate: 1_000_000, maxFramerate: 30 },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(publication.track.sender.parameters.encodings).toEqual([
|
||||
expect.objectContaining({
|
||||
rid: "q",
|
||||
scaleResolutionDownBy: 2,
|
||||
}),
|
||||
expect.objectContaining({
|
||||
rid: "h",
|
||||
scaleResolutionDownBy: 1,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["saver", 64_000],
|
||||
["music", 128_000],
|
||||
@@ -786,6 +876,13 @@ describe("SfuPublisher", () => {
|
||||
degradationPreference: "balanced",
|
||||
encodings: [
|
||||
expect.objectContaining({
|
||||
rid: "q",
|
||||
maxBitrate: 1_000_000,
|
||||
maxFramerate: 30,
|
||||
scaleResolutionDownBy: 2,
|
||||
}),
|
||||
expect.objectContaining({
|
||||
rid: "h",
|
||||
maxBitrate: 3_000_000,
|
||||
maxFramerate: 30,
|
||||
scaleResolutionDownBy: 1,
|
||||
@@ -1138,6 +1235,13 @@ describe("SfuPublisher", () => {
|
||||
degradationPreference: "maintain-framerate",
|
||||
encodings: [
|
||||
{
|
||||
rid: "q",
|
||||
maxBitrate: 1_000_000,
|
||||
maxFramerate: 30,
|
||||
scaleResolutionDownBy: 2,
|
||||
},
|
||||
{
|
||||
rid: "h",
|
||||
maxBitrate: 3_000_000,
|
||||
maxFramerate: 30,
|
||||
scaleResolutionDownBy: 1,
|
||||
@@ -1152,6 +1256,13 @@ describe("SfuPublisher", () => {
|
||||
maxBitrate: 3_000_000,
|
||||
maxFramerate: 30,
|
||||
},
|
||||
screenShareSimulcastLayers: [
|
||||
{
|
||||
width: 640,
|
||||
height: 360,
|
||||
encoding: { maxBitrate: 750_000, maxFramerate: 30 },
|
||||
},
|
||||
],
|
||||
degradationPreference: "maintain-framerate",
|
||||
});
|
||||
|
||||
@@ -1308,6 +1419,12 @@ describe("SfuPublisher", () => {
|
||||
degradationPreference: "maintain-resolution",
|
||||
encodings: [
|
||||
expect.objectContaining({
|
||||
rid: "q",
|
||||
maxBitrate: 1_000_000,
|
||||
maxFramerate: 30,
|
||||
}),
|
||||
expect.objectContaining({
|
||||
rid: "h",
|
||||
maxBitrate: 8_000_000,
|
||||
maxFramerate: 60,
|
||||
}),
|
||||
@@ -1355,13 +1472,20 @@ describe("SfuPublisher", () => {
|
||||
});
|
||||
|
||||
it("restores the previous video after a partially applied replacement fails", async () => {
|
||||
const previousVideo = track("video", "video-1");
|
||||
const nextVideo = track("video", "video-2");
|
||||
const previousVideo = track("video", "video-1", {
|
||||
width: 1904,
|
||||
height: 928,
|
||||
});
|
||||
const nextVideo = track("video", "video-2", {
|
||||
width: 1280,
|
||||
height: 720,
|
||||
});
|
||||
const publisher = new SfuPublisher();
|
||||
await publisher.connect(connection);
|
||||
await publisher.activate(stream(previousVideo), qualityProfile);
|
||||
const localTrack = livekit.state.rooms[0].localParticipant.publications[0].track;
|
||||
localTrack.replaceTrack.mockImplementationOnce(async (replacement) => {
|
||||
localTrack.replaceTrackOptions.push(localTrack.publishOptions);
|
||||
localTrack.currentTrack = replacement;
|
||||
throw new Error("sender swap failed after applying");
|
||||
});
|
||||
@@ -1371,6 +1495,25 @@ describe("SfuPublisher", () => {
|
||||
expect(localTrack.replaceTrack).toHaveBeenNthCalledWith(1, nextVideo);
|
||||
expect(localTrack.replaceTrack).toHaveBeenNthCalledWith(2, previousVideo);
|
||||
expect(localTrack.currentTrack).toBe(previousVideo);
|
||||
expect(localTrack.publishOptions).toMatchObject({
|
||||
screenShareSimulcastLayers: [
|
||||
expect.objectContaining({ width: 952, height: 464 }),
|
||||
],
|
||||
});
|
||||
expect(localTrack.sender.parameters.encodings).toEqual([
|
||||
expect.objectContaining({ rid: "q", scaleResolutionDownBy: 2 }),
|
||||
expect.objectContaining({ rid: "h", scaleResolutionDownBy: 1 }),
|
||||
]);
|
||||
expect(localTrack.replaceTrackOptions[0]).toMatchObject({
|
||||
screenShareSimulcastLayers: [
|
||||
expect.objectContaining({ width: 640, height: 360 }),
|
||||
],
|
||||
});
|
||||
expect(localTrack.replaceTrackOptions[1]).toMatchObject({
|
||||
screenShareSimulcastLayers: [
|
||||
expect.objectContaining({ width: 952, height: 464 }),
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("uses the screen audio preset when a replacement adds audio", async () => {
|
||||
|
||||
Reference in new issue
Block a user