fix(route): fix degraded edge recovery with same-parent regeneration
Merged after local npm run check, 20-Viewer Chromium loopback gate, and documentation validation. GitHub Actions validation was not started because the repository account billing/spending limit prevented runner start.
This commit is contained in:
1 parent
9b6ddb182f
commit
7833158ab9
13 files changed
+601
-29
No files matched your search
@@ -2,7 +2,7 @@
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-20
|
||||
- Last updated: 2026-08-28
|
||||
- Last updated: 2026-08-30
|
||||
|
||||
## Context
|
||||
|
||||
@@ -270,6 +270,13 @@ are degraded, one child moves and all remaining native states are observed
|
||||
again; relief can cancel the remaining work without a parent score or explicit
|
||||
capacity penalty.
|
||||
|
||||
If a persistently limited edge has no eligible different Peer parent, the same
|
||||
quality operation appends one same-parent regeneration candidate at the end of
|
||||
its existing cursor. It creates a new connection and sender generation while
|
||||
retaining the old edge until first-frame and comparative proof; the candidate
|
||||
uses the existing steady or one-slot overlap reservation. This is connection
|
||||
regeneration, not an in-place sender reset and not a new operation reason.
|
||||
|
||||
The Host capture/source is a separate fact. A stalled source cannot be repaired
|
||||
by topology. One limited Host-origin edge may try the same measured Peer move. SFU
|
||||
becomes a quality suffix only when every current Host-origin Peer edge, with a
|
||||
@@ -292,9 +299,13 @@ Host-to-SFU ingress proof and each exact SFU-to-Viewer subscription proof remain
|
||||
separate. A successful Viewer canary cannot authorize any other Viewer, and a
|
||||
new publication generation invalidates all prior ingress authority.
|
||||
|
||||
After a quality- or root-convergence commit, the new active identity starts from
|
||||
unknown and must establish a fresh healthy delta before another degradation can
|
||||
trigger a move. There is no periodic wake, weighted prediction, global
|
||||
After a quality- or root-convergence reparent commit, the new active identity
|
||||
starts from unknown and must establish a fresh healthy delta before another
|
||||
degradation can trigger a move. A same-parent regeneration commit instead
|
||||
records the exact new sender identity as a same-edge regeneration latch. A
|
||||
healthy sample or an identity/session change clears that latch; while it is set,
|
||||
only another same-edge regeneration is suppressed, while different-parent
|
||||
candidates remain eligible. There is no periodic wake, weighted prediction, global
|
||||
optimizer, persistent parent blacklist, custom congestion controller, or manual
|
||||
SFU layer selection. The result converges gradually to a local stable topology:
|
||||
no current degraded edge has a proved healthy candidate under the current graph
|
||||
|
||||
@@ -85,6 +85,10 @@ representations; Dynacast aggregates demand and server send-side BWE selects
|
||||
subscriber forwarding. AdaptiveStream stays disabled because any Viewer may
|
||||
relay its received track.
|
||||
|
||||
When a live native sender remains persistently limited after capacity recovers,
|
||||
the route controller may rebuild that exact edge with a fresh connection and
|
||||
clone; media adaptation still stays entirely framework-owned.
|
||||
|
||||
Screener does not maintain an application bitrate/resolution ladder, scene
|
||||
detector, periodic quality controller, manual SFU layer selector, or
|
||||
application-defined whole-room lowest-common-denominator target.
|
||||
|
||||
@@ -122,8 +122,11 @@ retained sender closes. A complete
|
||||
zero-frame current window counts as zero delivery only when an overlapping
|
||||
candidate actually decodes clean video; two zero-frame paths remain unknown. A
|
||||
missing or limited exact candidate sender cannot commit; an expired one-shot
|
||||
relative proof rejects that candidate. Ordinary quality moves remain
|
||||
Peer-to-Peer. SFU may
|
||||
relative proof rejects that candidate. When no different Peer parent is
|
||||
eligible, the same operation may append one same-parent connection regeneration
|
||||
candidate; it uses the same first-frame, overlap, and proof rules and is damped
|
||||
by its exact post-commit sender identity. Ordinary quality moves remain
|
||||
Peer-to-Peer. SFU may
|
||||
join quality work only as Host fanout relief when every current Host-origin Peer
|
||||
edge, with at least two such edges, is persistently degraded; one Viewer moves
|
||||
before the controller observes the new topology again. During that overlap, the
|
||||
|
||||
@@ -41,8 +41,11 @@ implementation and routine UI detail.
|
||||
layers. Current source defaults persistent native-edge local P2P convergence
|
||||
on, with a pre-share Host opt-out. SFU quality work is limited to multi-root
|
||||
Host fanout relief and each Viewer must prove its own candidate non-regression
|
||||
before commit. No weighted score, all-pairs probe, general parent-wide
|
||||
prediction, or periodic rebalance is accepted.
|
||||
before commit. A persistently limited edge with no different Peer parent may
|
||||
use one same-parent connection regeneration through that same operation; its
|
||||
exact post-commit sender identity damps only another same-edge regeneration.
|
||||
No weighted score, all-pairs probe, general parent-wide prediction, or
|
||||
periodic rebalance is accepted.
|
||||
- Browser video uses `motion`; each share chooses H.264 with VP8 fallback through
|
||||
an actual sender probe unless the Host explicitly selects VP8 or H264. Screen
|
||||
audio uses 64/128/192 kbps ceilings with 128 default.
|
||||
|
||||
@@ -110,6 +110,27 @@ with the controlled experiments, this supports partial shared-source coupling
|
||||
and track-generation retention; it does not establish a universal all-senders
|
||||
minimum or quantify its share relative to uplink contention.
|
||||
|
||||
### Same-edge generation recovery
|
||||
|
||||
On 2026-08-30, a production single-Viewer room provided a controlled comparison.
|
||||
The Host capture stayed at about 30 fps while one live P2P sender remained at
|
||||
480x270 and 14-16 fps after its available-outgoing estimate returned above
|
||||
5 Mbps. The route controller observed three persistent native limited windows,
|
||||
but had no different Peer candidate and therefore could not start a quality
|
||||
operation. A Viewer page refresh created a new connection and sender-owned clone
|
||||
on the same physical path; it climbed from the same source to 1920x1080 and
|
||||
30 fps within the following windows. This confirms that a live sender generation
|
||||
can remain adapted after network capacity recovers and that full connection
|
||||
replacement is the observed escape. The degradation onset occurred while
|
||||
application evidence was unavailable, so this record does not assign a trigger
|
||||
to a specific preceding event.
|
||||
|
||||
The accepted route change reuses that existing connection lifecycle: a persistent
|
||||
limited edge may try one same-parent candidate after ordinary Peer candidates,
|
||||
retaining the old edge until first-frame and comparative proof. It adds no media
|
||||
setting, score, periodic rebalance, or new transport. Physical public-network
|
||||
success rates and long-run retry behavior remain open evidence.
|
||||
|
||||
## H.264 Root Cause And Gate
|
||||
|
||||
Chromium's `has_trusted_rate_controller` is encoder coordination, not a Windows
|
||||
|
||||
+7
-4
@@ -76,7 +76,10 @@ This is the current execution index. Git history owns completed timelines;
|
||||
clean receive windows that do not regress resolution or rounded FPS. A zero-
|
||||
frame incumbent is zero delivery only against an overlapping candidate that
|
||||
decodes video. Peer candidates stay first; bounded SFU remains only the whole-
|
||||
Host-root suffix. The pre-share gate defaults on and locks while sharing.
|
||||
Host-root suffix; when no different Peer is eligible, the same operation may
|
||||
append one same-parent connection-regeneration candidate. Its old edge remains
|
||||
until proof, and its exact post-commit sender identity damps only another
|
||||
same-edge regeneration. The pre-share gate defaults on and locks while sharing.
|
||||
- Every direct, Browser-relay, and Host SFU video sender owns one clone of its
|
||||
source track; the original remains presentation and source authority only.
|
||||
Replacement, rollback, unpublish, physical LiveKit sender recreation, and
|
||||
@@ -112,9 +115,9 @@ This is the current execution index. Git history owns completed timelines;
|
||||
screening did not reproduce a Host-page drop; accepted Viewer lifecycle work
|
||||
prevents frozen JavaScript wall time from becoming an immediate route failure
|
||||
but is not capture keepalive.
|
||||
- Current-path quality can drive local convergence in source and production.
|
||||
Both default it on with a pre-share Host opt-out. Weighted/global optimization
|
||||
remains parked.
|
||||
- Current-path quality can drive local convergence in source and production,
|
||||
including bounded same-edge connection regeneration. Both default it on with a
|
||||
pre-share Host opt-out. Weighted/global optimization remains parked.
|
||||
- Repository simplification does not change or deploy product behavior beyond
|
||||
deleting proven dead private surfaces.
|
||||
- Native/executable senders, shared encode, and distribution packages remain
|
||||
|
||||
+2
-1
@@ -12,7 +12,8 @@ until their stated boundary is met.
|
||||
|
||||
## Parked Product Work
|
||||
|
||||
1. **Broader quality optimization.** Weighted/global optimization,
|
||||
1. **Broader quality optimization.** Same-edge connection regeneration is now
|
||||
part of the bounded local convergence path. Weighted/global optimization,
|
||||
parent-wide prediction, startup-limited inference, complete cross-clone
|
||||
source isolation, and alternative SFU policies remain unaccepted. Reopen only
|
||||
from exact path evidence; do not add a score, all-pairs probe, periodic
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Verification Status
|
||||
|
||||
Last updated: 2026-08-27
|
||||
Last updated: 2026-08-30
|
||||
|
||||
This file owns only cross-module physical evidence that still changes how the
|
||||
current product may be described. [Status](./status.md) owns exact source and
|
||||
@@ -42,9 +42,11 @@ Still required:
|
||||
admission at the 20-Viewer bound.
|
||||
|
||||
Current controlled Browser evidence supports the H.264/VP8 gate, `motion`
|
||||
content intent, startup-quality workaround, LiveKit-owned SFU adaptation, and
|
||||
the native-edge convergence invariants. It does not establish their quality or
|
||||
resource benefit on heterogeneous public networks. See [media quality](./product/media-quality.md)
|
||||
content intent, startup-quality workaround, LiveKit-owned SFU adaptation, native
|
||||
edge convergence, and same-edge connection regeneration invariants. The
|
||||
2026-08-30 production comparison confirms recovery on a fresh same-route sender,
|
||||
but does not establish its quality or resource benefit on heterogeneous public
|
||||
networks. See [media quality](./product/media-quality.md)
|
||||
and [realtime quality research](./research/realtime-quality-adaptation.md).
|
||||
|
||||
## Browser Lifecycle
|
||||
|
||||
@@ -2159,7 +2159,9 @@ export class HybridMediaRouter {
|
||||
|
||||
private debugTuple(roomId: string, tuple: CandidateTuple): string {
|
||||
return tuple.kind === "peer"
|
||||
? `p2p:${this.debugPeer(roomId, tuple.parentPeerId)}`
|
||||
? `p2p:${this.debugPeer(roomId, tuple.parentPeerId)}${
|
||||
tuple.regenerate ? ":regenerate" : ""
|
||||
}`
|
||||
: `sfu:${tuple.publication}`;
|
||||
}
|
||||
|
||||
|
||||
@@ -14,7 +14,13 @@ import {
|
||||
import { assertEndpointMediaCopyCapacity } from "../shared/media-copy-accounting.js";
|
||||
|
||||
export type CandidateTuple =
|
||||
| { kind: "peer"; parentPeerId: string; transport: "direct" }
|
||||
| {
|
||||
kind: "peer";
|
||||
parentPeerId: string;
|
||||
transport: "direct";
|
||||
/** Rebuild this exact edge with a fresh connection and media generation. */
|
||||
regenerate?: true;
|
||||
}
|
||||
| { kind: "sfu"; publication: "reuse" | "create" | "replace" };
|
||||
|
||||
export type CandidateReservation<Resource> =
|
||||
@@ -264,6 +270,14 @@ interface SenderQualityObservation {
|
||||
consumedAtFactVersion?: number;
|
||||
}
|
||||
|
||||
interface SenderQualityRegenerationBlock {
|
||||
childSessionId: string;
|
||||
parentPeerId: string;
|
||||
parentSessionId: string;
|
||||
connectionId: string;
|
||||
senderIdentity: string;
|
||||
}
|
||||
|
||||
export interface SenderQualityEvidenceInput {
|
||||
parentPeerId: string;
|
||||
parentSessionId: string;
|
||||
@@ -353,6 +367,10 @@ export class RoomRouteController<Resource = unknown> {
|
||||
SenderQualityObservation
|
||||
>();
|
||||
private readonly senderQualityBaselines = new Map<string, boolean>();
|
||||
private readonly senderQualityRegenerationBlocks = new Map<
|
||||
string,
|
||||
SenderQualityRegenerationBlock
|
||||
>();
|
||||
private sfuPublisherQualityObservation?: SfuPublisherQualityObservation;
|
||||
private readonly qualityBaselinesPending = new Set<string>();
|
||||
private readonly directContinuations = new Map<string, DirectContinuation>();
|
||||
@@ -587,6 +605,7 @@ export class RoomRouteController<Resource = unknown> {
|
||||
this.qualityObservations.clear();
|
||||
this.senderQualityObservations.clear();
|
||||
this.senderQualityBaselines.clear();
|
||||
this.senderQualityRegenerationBlocks.clear();
|
||||
this.qualityBaselinesPending.clear();
|
||||
this.sfuPublisherQualityObservation = undefined;
|
||||
this.directContinuations.clear();
|
||||
@@ -902,6 +921,7 @@ export class RoomRouteController<Resource = unknown> {
|
||||
this.qualityBaselinesPending.clear();
|
||||
this.senderQualityObservations.clear();
|
||||
this.senderQualityBaselines.clear();
|
||||
this.senderQualityRegenerationBlocks.clear();
|
||||
this.sfuPublisherQualityObservation = undefined;
|
||||
}
|
||||
this.touchFacts();
|
||||
@@ -1149,6 +1169,24 @@ export class RoomRouteController<Resource = unknown> {
|
||||
released: [],
|
||||
};
|
||||
}
|
||||
const regenerationBlock = this.senderQualityRegenerationBlocks.get(
|
||||
input.childPeerId,
|
||||
);
|
||||
if (
|
||||
regenerationBlock &&
|
||||
!senderQualityRegenerationBlockMatches(
|
||||
regenerationBlock,
|
||||
child.sessionId,
|
||||
input,
|
||||
)
|
||||
) {
|
||||
// A new connection or sender identity is a new generation. Do not carry
|
||||
// a same-edge regeneration latch across that identity fence.
|
||||
this.senderQualityRegenerationBlocks.delete(input.childPeerId);
|
||||
}
|
||||
if (input.state === "healthy") {
|
||||
this.senderQualityRegenerationBlocks.delete(input.childPeerId);
|
||||
}
|
||||
const sameFreshSenderIdentity =
|
||||
sameSenderIdentity &&
|
||||
previous !== undefined &&
|
||||
@@ -1553,6 +1591,7 @@ export class RoomRouteController<Resource = unknown> {
|
||||
// below; a visibility or source reset must not deadlock a persistently
|
||||
// limited edge waiting for a healthy sample.
|
||||
this.senderQualityBaselines.delete(childPeerId);
|
||||
this.senderQualityRegenerationBlocks.delete(childPeerId);
|
||||
}
|
||||
if (parentPeerId === this.options.hostPeerId) {
|
||||
this.sfuPublisherQualityObservation = undefined;
|
||||
@@ -2292,6 +2331,7 @@ export class RoomRouteController<Resource = unknown> {
|
||||
this.qualityBaselinesPending.clear();
|
||||
this.senderQualityObservations.clear();
|
||||
this.senderQualityBaselines.clear();
|
||||
this.senderQualityRegenerationBlocks.clear();
|
||||
this.sfuPublisherQualityObservation = undefined;
|
||||
this.hostPublication = null;
|
||||
this.participants.clear();
|
||||
@@ -2574,6 +2614,12 @@ export class RoomRouteController<Resource = unknown> {
|
||||
private commitAttempt(operation: ChildOperation<Resource>, attempt: Attempt<Resource>): Resource[] {
|
||||
const beforeResources = new Set(this.committedResources());
|
||||
const old = this.upstreamByViewer.get(operation.childPeerId);
|
||||
const sameParentRegeneration =
|
||||
operation.reason === "quality-convergence" &&
|
||||
attempt.tuple.kind === "peer" &&
|
||||
attempt.tuple.regenerate === true &&
|
||||
old?.kind === "peer" &&
|
||||
old.parentPeerId === attempt.tuple.parentPeerId;
|
||||
const createsHostRoot =
|
||||
attempt.tuple.kind === "peer" &&
|
||||
attempt.tuple.parentPeerId === this.options.hostPeerId &&
|
||||
@@ -2666,9 +2712,27 @@ export class RoomRouteController<Resource = unknown> {
|
||||
this.requireSenderQualityBaseline(operation.childPeerId, true);
|
||||
this.senderQualityBaselines.set(
|
||||
operation.childPeerId,
|
||||
operation.reason === "quality-convergence" ||
|
||||
operation.reason === "root-convergence",
|
||||
sameParentRegeneration
|
||||
? false
|
||||
: operation.reason === "quality-convergence" ||
|
||||
operation.reason === "root-convergence",
|
||||
);
|
||||
if (
|
||||
sameParentRegeneration &&
|
||||
attempt.tuple.kind === "peer" &&
|
||||
attempt.parentSessionId &&
|
||||
attempt.senderQualityIdentity
|
||||
) {
|
||||
this.senderQualityRegenerationBlocks.set(operation.childPeerId, {
|
||||
childSessionId: operation.childSessionId,
|
||||
parentPeerId: attempt.tuple.parentPeerId,
|
||||
parentSessionId: attempt.parentSessionId,
|
||||
connectionId: attempt.connectionId,
|
||||
senderIdentity: attempt.senderQualityIdentity,
|
||||
});
|
||||
} else {
|
||||
this.senderQualityRegenerationBlocks.delete(operation.childPeerId);
|
||||
}
|
||||
this.revision = attempt.revision;
|
||||
this.operation = undefined;
|
||||
this.pruneRetiringSfuAnchors();
|
||||
@@ -2867,6 +2931,31 @@ export class RoomRouteController<Resource = unknown> {
|
||||
: "unknown";
|
||||
}
|
||||
|
||||
private senderQualityRegenerationBlocked(
|
||||
childPeerId: string,
|
||||
edge: Extract<CommittedEdge<Resource>, { kind: "peer" }>,
|
||||
): boolean {
|
||||
const block = this.senderQualityRegenerationBlocks.get(childPeerId);
|
||||
if (!block) {
|
||||
return false;
|
||||
}
|
||||
const observation = this.senderQualityObservations.get(childPeerId);
|
||||
if (observation && observation.senderIdentity !== block.senderIdentity) {
|
||||
this.senderQualityRegenerationBlocks.delete(childPeerId);
|
||||
return false;
|
||||
}
|
||||
if (
|
||||
block.childSessionId !== edge.childSessionId ||
|
||||
block.parentPeerId !== edge.parentPeerId ||
|
||||
block.parentSessionId !== edge.parentSessionId ||
|
||||
block.connectionId !== edge.connectionId
|
||||
) {
|
||||
this.senderQualityRegenerationBlocks.delete(childPeerId);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private candidateSenderPersistentlyDegraded(
|
||||
attempt: Attempt<Resource>,
|
||||
nowMs: number,
|
||||
@@ -2991,7 +3080,23 @@ export class RoomRouteController<Resource = unknown> {
|
||||
this.hostFanoutNeedsSfuRelief(nowMs)
|
||||
? candidates.filter((candidate) => candidate.tuple.kind === "sfu")
|
||||
: [];
|
||||
return [...clearPeers, ...remainingPeers, ...sfu];
|
||||
const regenerationTuple: CandidateTuple = {
|
||||
kind: "peer",
|
||||
parentPeerId: current.parentPeerId,
|
||||
transport: "direct",
|
||||
regenerate: true,
|
||||
};
|
||||
const regenerationPlan =
|
||||
this.senderQualityRegenerationBlocked(childPeerId, current)
|
||||
? null
|
||||
: this.planCandidate(childPeerId, regenerationTuple);
|
||||
const regeneration =
|
||||
regenerationPlan &&
|
||||
regenerationPlan.endpointTransition.kind !== "bounded-gap" &&
|
||||
this.candidateValid(childPeerId, regenerationPlan)
|
||||
? [regenerationPlan]
|
||||
: [];
|
||||
return [...clearPeers, ...remainingPeers, ...sfu, ...regeneration];
|
||||
}
|
||||
|
||||
private selectQualityChild(nowMs: number): string | undefined {
|
||||
@@ -3803,7 +3908,16 @@ export class RoomRouteController<Resource = unknown> {
|
||||
operation.current.tuple.kind === "peer"
|
||||
? operation.current.tuple.parentPeerId
|
||||
: this.options.hostPeerId;
|
||||
const regeneratingChild =
|
||||
operation.current.tuple.kind === "peer" &&
|
||||
operation.current.tuple.regenerate === true;
|
||||
for (const childPeerId of this.childrenOf(producerPeerId)) {
|
||||
if (regeneratingChild && childPeerId === operation.childPeerId) {
|
||||
// A failed same-edge trial must not turn the retained sender into a
|
||||
// global baseline block. The operation's consumed fact version
|
||||
// already supplies one-shot retry damping.
|
||||
continue;
|
||||
}
|
||||
this.senderQualityObservations.delete(childPeerId);
|
||||
this.senderQualityBaselines.set(childPeerId, true);
|
||||
}
|
||||
@@ -4241,6 +4355,7 @@ export class RoomRouteController<Resource = unknown> {
|
||||
this.qualityObservations.delete(childPeerId);
|
||||
this.senderQualityObservations.delete(childPeerId);
|
||||
this.senderQualityBaselines.delete(childPeerId);
|
||||
this.senderQualityRegenerationBlocks.delete(childPeerId);
|
||||
if (this.upstreamByViewer.has(childPeerId)) {
|
||||
this.qualityBaselinesPending.add(childPeerId);
|
||||
this.senderQualityBaselines.set(childPeerId, true);
|
||||
@@ -4588,7 +4703,9 @@ export class RoomRouteController<Resource = unknown> {
|
||||
|
||||
private debugTuple(tuple: CandidateTuple): string {
|
||||
return tuple.kind === "peer"
|
||||
? `p2p:${this.debugPeer(tuple.parentPeerId)}`
|
||||
? `p2p:${this.debugPeer(tuple.parentPeerId)}${
|
||||
tuple.regenerate ? ":regenerate" : ""
|
||||
}`
|
||||
: `sfu:${tuple.publication}`;
|
||||
}
|
||||
|
||||
@@ -4661,7 +4778,23 @@ function safeAdd(total: number, delta: number): number {
|
||||
}
|
||||
|
||||
function tupleKey(tuple: CandidateTuple): string {
|
||||
return tuple.kind === "peer" ? `peer:${tuple.parentPeerId}` : `sfu:${tuple.publication}`;
|
||||
return tuple.kind === "peer"
|
||||
? `peer:${tuple.parentPeerId}${tuple.regenerate ? ":regenerate" : ""}`
|
||||
: `sfu:${tuple.publication}`;
|
||||
}
|
||||
|
||||
function senderQualityRegenerationBlockMatches(
|
||||
block: SenderQualityRegenerationBlock,
|
||||
childSessionId: string,
|
||||
input: SenderQualityEvidenceInput,
|
||||
): boolean {
|
||||
return (
|
||||
block.childSessionId === childSessionId &&
|
||||
block.parentPeerId === input.parentPeerId &&
|
||||
block.parentSessionId === input.parentSessionId &&
|
||||
block.connectionId === input.connectionId &&
|
||||
block.senderIdentity === input.senderIdentity
|
||||
);
|
||||
}
|
||||
|
||||
function edgeTupleKey<Resource>(edge: CommittedEdge<Resource>): string {
|
||||
|
||||
@@ -1843,6 +1843,53 @@ describe("ViewerRelay downstream ownership", () => {
|
||||
relay.dispose();
|
||||
});
|
||||
|
||||
it("replaces one active child with a same-parent prepared connection", async () => {
|
||||
const signals: Array<{ peerId: string; connectionId: string }> = [];
|
||||
const relay = new ViewerRelay(
|
||||
{ iceServers: [] },
|
||||
QUALITY_PROFILES["720p30"],
|
||||
{
|
||||
sendSignal: (peerId, payload) => {
|
||||
signals.push({ peerId, connectionId: payload.connectionId });
|
||||
return true;
|
||||
},
|
||||
},
|
||||
);
|
||||
relay.setStream(createStream(createTrack("video", "same-parent-video"), null));
|
||||
expect(relay.prepareChild(1, routeCandidate(1, "same-child"), ["same-child"])).toBe(true);
|
||||
await vi.waitFor(() => expect(signals).toHaveLength(1));
|
||||
const firstConnection = FakePeerConnection.latest!;
|
||||
await relay.acceptSignal("same-child", {
|
||||
kind: "description",
|
||||
connectionId: signals[0]!.connectionId,
|
||||
description: { type: "answer", sdp: "first-answer" },
|
||||
}, 1);
|
||||
firstConnection.connectionState = "connected";
|
||||
relay.activateChildren(1, ["same-child"]);
|
||||
expect(relay.getSnapshot("same-child")?.connectionId).toBe(
|
||||
signals[0]!.connectionId,
|
||||
);
|
||||
|
||||
expect(relay.prepareChild(2, routeCandidate(2, "same-child"), ["same-child"])).toBe(true);
|
||||
await vi.waitFor(() => expect(signals).toHaveLength(2));
|
||||
const secondConnection = FakePeerConnection.latest!;
|
||||
expect(firstConnection.connectionState).toBe("connected");
|
||||
await relay.acceptSignal("same-child", {
|
||||
kind: "description",
|
||||
connectionId: signals[1]!.connectionId,
|
||||
description: { type: "answer", sdp: "second-answer" },
|
||||
}, 2);
|
||||
secondConnection.connectionState = "connected";
|
||||
relay.activateChildren(2, ["same-child"]);
|
||||
|
||||
expect(firstConnection.connectionState).toBe("closed");
|
||||
expect(secondConnection.connectionState).toBe("connected");
|
||||
expect(relay.getSnapshot("same-child")?.connectionId).toBe(
|
||||
signals[1]!.connectionId,
|
||||
);
|
||||
relay.dispose();
|
||||
});
|
||||
|
||||
it("cleans prepared children on failure, replacement, rollback, session reset, and share stop", async () => {
|
||||
const relay = new ViewerRelay(
|
||||
{ iceServers: [] },
|
||||
|
||||
@@ -899,6 +899,87 @@ describe("HybridMediaRouter v9 runtime", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("prepares a same-parent connection regeneration after persistent degradation", async () => {
|
||||
const { store, sent, router } = harness(2);
|
||||
try {
|
||||
const room = await store.createRoom();
|
||||
const host = connectHost(store, room, {
|
||||
peerOnly: false,
|
||||
topologyOptimization: true,
|
||||
});
|
||||
complete(router, host);
|
||||
const viewer = connectViewer(store, room, "quality-regeneration");
|
||||
complete(router, viewer);
|
||||
await vi.waitFor(() =>
|
||||
expect(preparedFor(sent, viewer.sessionId)).toBeDefined(),
|
||||
);
|
||||
const initial = preparedFor(sent, viewer.sessionId)!;
|
||||
expect(initial.assignment.upstream).toEqual({
|
||||
kind: "peer",
|
||||
peerId: host.peerId,
|
||||
});
|
||||
router.handleRouteReady(viewer, {
|
||||
type: "route-ready",
|
||||
revision: initial.revision,
|
||||
phase: "prepare",
|
||||
});
|
||||
const edge = router.resolveActiveViewerMediaEdge(
|
||||
room.roomId,
|
||||
viewer.peerId,
|
||||
)!;
|
||||
|
||||
router.observeSenderQualityEvidence(host, {
|
||||
type: "sender-quality-evidence",
|
||||
childPeerId: viewer.peerId,
|
||||
connectionId: edge.connectionId,
|
||||
rtpStatsId: "old-sender-rtp",
|
||||
trackIdentifier: "old-track",
|
||||
sampleTimestampMs: 100,
|
||||
routeRevision: edge.revision,
|
||||
state: "healthy",
|
||||
diagnostics: senderDiagnostics("healthy"),
|
||||
});
|
||||
for (const sampleTimestampMs of [101, 102, 103]) {
|
||||
router.observeSenderQualityEvidence(host, {
|
||||
type: "sender-quality-evidence",
|
||||
childPeerId: viewer.peerId,
|
||||
connectionId: edge.connectionId,
|
||||
rtpStatsId: "old-sender-rtp",
|
||||
trackIdentifier: "old-track",
|
||||
sampleTimestampMs,
|
||||
routeRevision: edge.revision,
|
||||
state: "degraded",
|
||||
diagnostics: senderDiagnostics("degraded"),
|
||||
});
|
||||
}
|
||||
|
||||
await vi.waitFor(() => {
|
||||
const prepared = preparedFor(sent, viewer.sessionId);
|
||||
expect(prepared?.revision).toBeGreaterThan(edge.revision);
|
||||
});
|
||||
const regeneration = preparedFor(sent, viewer.sessionId)!;
|
||||
expect(regeneration.candidate).toMatchObject({
|
||||
transport: "direct",
|
||||
qualityProbe: true,
|
||||
});
|
||||
expect(regeneration.assignment.upstream).toEqual({
|
||||
kind: "peer",
|
||||
peerId: host.peerId,
|
||||
});
|
||||
expect(preparedFor(sent, host.sessionId)?.candidate.connectionId).toBe(
|
||||
regeneration.candidate.connectionId,
|
||||
);
|
||||
expect(
|
||||
router.resolveActiveViewerMediaEdge(room.roomId, viewer.peerId),
|
||||
).toMatchObject({
|
||||
connectionId: edge.connectionId,
|
||||
upstream: { kind: "peer", peerId: host.peerId },
|
||||
});
|
||||
} finally {
|
||||
await router.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps peer-only shares out of configured SFU fallback", async () => {
|
||||
const { store, sent, router } = harness(1, true);
|
||||
try {
|
||||
|
||||
@@ -563,6 +563,201 @@ describe("RoomRouteController", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("regenerates a persistently degraded edge on the same parent", () => {
|
||||
const routes = controller(2, { qualityConvergenceEnabled: true });
|
||||
addViewer(routes, A, 2);
|
||||
routes.hydrateEdge(A, peerEdge(HOST, "a_from_host"));
|
||||
observePersistentDegraded(routes, A, "a_from_host", 10);
|
||||
|
||||
const operation = routes.reconcile(14).operation!;
|
||||
expect(operation.candidates).toEqual([
|
||||
{
|
||||
tuple: {
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
transport: "direct",
|
||||
regenerate: true,
|
||||
},
|
||||
endpointTransition: { kind: "none", producerPeerId: HOST },
|
||||
},
|
||||
]);
|
||||
|
||||
const begin = beginCandidate(routes, {
|
||||
nowMs: 15,
|
||||
connectionId: "a_regenerated",
|
||||
reservation: { kind: "direct" },
|
||||
}).operation!;
|
||||
expect(routes.snapshot().upstreamByViewer.get(A)).toMatchObject({
|
||||
parentPeerId: HOST,
|
||||
connectionId: "a_from_host",
|
||||
usable: true,
|
||||
physicalActive: true,
|
||||
});
|
||||
const guard: CandidateGuard = {
|
||||
childPeerId: A,
|
||||
childSessionId: `${A}_session`,
|
||||
revision: begin.current!.revision,
|
||||
connectionId: "a_regenerated",
|
||||
};
|
||||
expect(
|
||||
routes.candidateReady(guard, 16, undefined, {
|
||||
relativeQualityApproved: true,
|
||||
}),
|
||||
).toMatchObject({ accepted: true, committed: false });
|
||||
|
||||
expect(
|
||||
routes.observeSenderQualityEvidence({
|
||||
parentPeerId: HOST,
|
||||
parentSessionId: "host_session",
|
||||
childPeerId: A,
|
||||
routeRevision: begin.current!.revision,
|
||||
connectionId: "a_regenerated",
|
||||
senderIdentity: "a-regenerated-rtp\u0000track",
|
||||
sampleTimestampMs: 17,
|
||||
state: "healthy",
|
||||
acceptedAtMs: 17,
|
||||
}).committed,
|
||||
).toBe(true);
|
||||
expect(routes.snapshot().upstreamByViewer.get(A)).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
connectionId: "a_regenerated",
|
||||
});
|
||||
expect(
|
||||
observeSenderState(routes, {
|
||||
childPeerId: A,
|
||||
connectionId: "a_regenerated",
|
||||
senderIdentity: "a-regenerated-rtp\u0000track",
|
||||
state: "healthy",
|
||||
acceptedAtMs: 18,
|
||||
}).accepted,
|
||||
).toBe(true);
|
||||
for (const acceptedAtMs of [19, 20, 21]) {
|
||||
expect(
|
||||
observeSenderState(routes, {
|
||||
childPeerId: A,
|
||||
connectionId: "a_regenerated",
|
||||
senderIdentity: "a-regenerated-rtp\u0000track",
|
||||
state: "degraded",
|
||||
acceptedAtMs,
|
||||
}).accepted,
|
||||
).toBe(true);
|
||||
}
|
||||
expect(routes.reconcile(22).operation?.candidates[0]?.tuple).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
regenerate: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps same-edge regeneration damped while allowing a different parent", () => {
|
||||
const routes = controller(2, { qualityConvergenceEnabled: true });
|
||||
addViewer(routes, A, 2);
|
||||
routes.hydrateEdge(A, peerEdge(HOST, "a_from_host"));
|
||||
observePersistentDegraded(routes, A, "a_from_host", 10);
|
||||
|
||||
routes.reconcile(14);
|
||||
const begin = beginCandidate(routes, {
|
||||
nowMs: 15,
|
||||
connectionId: "a_regenerated",
|
||||
reservation: { kind: "direct" },
|
||||
}).operation!;
|
||||
const guard: CandidateGuard = {
|
||||
childPeerId: A,
|
||||
childSessionId: `${A}_session`,
|
||||
revision: begin.current!.revision,
|
||||
connectionId: "a_regenerated",
|
||||
};
|
||||
routes.candidateReady(guard, 16, undefined, {
|
||||
relativeQualityApproved: true,
|
||||
});
|
||||
expect(
|
||||
routes.observeSenderQualityEvidence({
|
||||
parentPeerId: HOST,
|
||||
parentSessionId: "host_session",
|
||||
childPeerId: A,
|
||||
routeRevision: begin.current!.revision,
|
||||
connectionId: "a_regenerated",
|
||||
senderIdentity: "a-regenerated-rtp\u0000track",
|
||||
sampleTimestampMs: 17,
|
||||
state: "healthy",
|
||||
acceptedAtMs: 17,
|
||||
}).committed,
|
||||
).toBe(true);
|
||||
|
||||
for (const acceptedAtMs of [18, 19, 20]) {
|
||||
expect(
|
||||
observeSenderState(routes, {
|
||||
childPeerId: A,
|
||||
connectionId: "a_regenerated",
|
||||
state: "degraded",
|
||||
senderIdentity: "a-regenerated-rtp\u0000track",
|
||||
acceptedAtMs,
|
||||
}).accepted,
|
||||
).toBe(true);
|
||||
}
|
||||
expect(routes.reconcile(21).operation).toBeUndefined();
|
||||
|
||||
addViewer(routes, B, 2);
|
||||
routes.hydrateEdge(B, peerEdge(HOST, "b_from_host"));
|
||||
expect(
|
||||
observeSenderState(routes, {
|
||||
childPeerId: B,
|
||||
connectionId: "b_from_host",
|
||||
state: "healthy",
|
||||
acceptedAtMs: 22,
|
||||
}).accepted,
|
||||
).toBe(true);
|
||||
const withAlternate = routes.reconcile(23).operation;
|
||||
expect(withAlternate).toMatchObject({
|
||||
childPeerId: A,
|
||||
reason: "quality-convergence",
|
||||
});
|
||||
expect(withAlternate?.candidates.map((candidate) => candidate.tuple)).toEqual([
|
||||
{ kind: "peer", parentPeerId: B, transport: "direct" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("uses the existing one-slot overlap for same-parent regeneration", () => {
|
||||
const routes = controller(1, { qualityConvergenceEnabled: true });
|
||||
addViewer(routes, A, 1);
|
||||
routes.hydrateEdge(A, peerEdge(HOST, "a_from_host"));
|
||||
observePersistentDegraded(routes, A, "a_from_host", 10);
|
||||
|
||||
const operation = routes.reconcile(14).operation!;
|
||||
expect(operation.candidates[0]?.endpointTransition).toEqual({
|
||||
kind: "overlap",
|
||||
producerPeerId: HOST,
|
||||
});
|
||||
const begin = beginCandidate(routes, {
|
||||
nowMs: 15,
|
||||
connectionId: "a_regenerated",
|
||||
reservation: { kind: "direct", overlap: "a_overlap" },
|
||||
}).operation!;
|
||||
const guard: CandidateGuard = {
|
||||
childPeerId: A,
|
||||
childSessionId: `${A}_session`,
|
||||
revision: begin.current!.revision,
|
||||
connectionId: "a_regenerated",
|
||||
};
|
||||
routes.candidateReady(guard, 16, undefined, {
|
||||
relativeQualityApproved: true,
|
||||
});
|
||||
const committed = routes.observeSenderQualityEvidence({
|
||||
parentPeerId: HOST,
|
||||
parentSessionId: "host_session",
|
||||
childPeerId: A,
|
||||
routeRevision: begin.current!.revision,
|
||||
connectionId: "a_regenerated",
|
||||
senderIdentity: "a-regenerated-rtp\u0000track",
|
||||
sampleTimestampMs: 17,
|
||||
state: "healthy",
|
||||
acceptedAtMs: 17,
|
||||
});
|
||||
expect(committed.committed).toBe(true);
|
||||
expect(committed.released).toContain("a_overlap");
|
||||
});
|
||||
|
||||
it("keeps isolated limitation windows diagnostic", () => {
|
||||
const routes = controller(2, { qualityConvergenceEnabled: true });
|
||||
addViewer(routes, A, 2);
|
||||
@@ -650,7 +845,14 @@ describe("RoomRouteController", () => {
|
||||
routes.snapshot().revision,
|
||||
103,
|
||||
);
|
||||
expect(routes.reconcile(104).operation).toBeUndefined();
|
||||
const operation = routes.reconcile(104).operation!;
|
||||
expect(operation.reason).toBe("quality-convergence");
|
||||
expect(operation.candidates).toHaveLength(1);
|
||||
expect(operation.candidates[0]?.tuple).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
regenerate: true,
|
||||
});
|
||||
expect(routes.snapshot().upstreamByViewer.get(A)).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
@@ -1093,7 +1295,10 @@ describe("RoomRouteController", () => {
|
||||
acceptedAtMs: 5_111,
|
||||
});
|
||||
expect(settled).toMatchObject({ accepted: true, committed: false });
|
||||
expect(routes.snapshot().operation).toBeUndefined();
|
||||
expect(routes.snapshot().operation).toMatchObject({
|
||||
reason: "quality-convergence",
|
||||
cursor: 1,
|
||||
});
|
||||
expect(routes.snapshot().upstreamByViewer.get(A)).toMatchObject({
|
||||
parentPeerId: HOST,
|
||||
connectionId: "a_from_host",
|
||||
@@ -1517,6 +1722,14 @@ describe("RoomRouteController", () => {
|
||||
{
|
||||
tuple: { kind: "peer", parentPeerId: B, transport: "direct" },
|
||||
},
|
||||
{
|
||||
tuple: {
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
transport: "direct",
|
||||
regenerate: true,
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
@@ -1660,13 +1873,40 @@ describe("RoomRouteController", () => {
|
||||
routes.hydrateEdge(A, peerEdge(HOST, "a_from_host"));
|
||||
routes.hydrateEdge(B, peerEdge(HOST, "b_from_host"));
|
||||
observePersistentDegraded(routes, A, "a_from_host", 99);
|
||||
expect(routes.reconcile(103).operation).toBeUndefined();
|
||||
const firstRegeneration = routes.reconcile(103).operation!;
|
||||
expect(firstRegeneration.candidates[0]?.tuple).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
regenerate: true,
|
||||
});
|
||||
const firstAttempt = beginCandidate(routes, {
|
||||
nowMs: 104,
|
||||
connectionId: "a_regeneration_failed",
|
||||
reservation: { kind: "direct", overlap: "a_regeneration_overlap" },
|
||||
}).operation!;
|
||||
expect(
|
||||
routes.candidateFailed(
|
||||
{
|
||||
childPeerId: A,
|
||||
childSessionId: `${A}_session`,
|
||||
revision: firstAttempt.current!.revision,
|
||||
connectionId: "a_regeneration_failed",
|
||||
},
|
||||
105,
|
||||
).accepted,
|
||||
).toBe(true);
|
||||
observePersistentDegraded(routes, B, "b_from_host", 104);
|
||||
const operation = routes.reconcile(108).operation!;
|
||||
const childPeerId = operation.childPeerId;
|
||||
expect(operation.candidates.map((candidate) => candidate.tuple)).toEqual([
|
||||
{ kind: "peer", parentPeerId: A, transport: "direct" },
|
||||
{ kind: "sfu", publication: "create" },
|
||||
{
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
transport: "direct",
|
||||
regenerate: true,
|
||||
},
|
||||
]);
|
||||
const direct = beginCandidate(routes, {
|
||||
nowMs: 109,
|
||||
@@ -1752,7 +1992,28 @@ describe("RoomRouteController", () => {
|
||||
routes.hydrateEdge(A, peerEdge(HOST, "a_from_host"));
|
||||
routes.hydrateEdge(B, peerEdge(HOST, "b_from_host"));
|
||||
observePersistentDegraded(routes, A, "a_from_host", 99);
|
||||
expect(routes.reconcile(103).operation).toBeUndefined();
|
||||
const firstRegeneration = routes.reconcile(103).operation!;
|
||||
expect(firstRegeneration.candidates[0]?.tuple).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
regenerate: true,
|
||||
});
|
||||
const firstAttempt = beginCandidate(routes, {
|
||||
nowMs: 104,
|
||||
connectionId: "a_regeneration_failed",
|
||||
reservation: { kind: "direct", overlap: "a_regeneration_overlap" },
|
||||
}).operation!;
|
||||
expect(
|
||||
routes.candidateFailed(
|
||||
{
|
||||
childPeerId: A,
|
||||
childSessionId: `${A}_session`,
|
||||
revision: firstAttempt.current!.revision,
|
||||
connectionId: "a_regeneration_failed",
|
||||
},
|
||||
105,
|
||||
).accepted,
|
||||
).toBe(true);
|
||||
observePersistentDegraded(routes, B, "b_from_host", 104);
|
||||
const operation = routes.reconcile(108).operation!;
|
||||
const childPeerId = operation.childPeerId;
|
||||
@@ -1808,7 +2069,7 @@ describe("RoomRouteController", () => {
|
||||
acceptedAtMs: 113,
|
||||
}),
|
||||
).toMatchObject({ accepted: true, committed: false });
|
||||
expect(routes.snapshot().operation).toBeUndefined();
|
||||
expect(routes.reconcile(114).operation).toBeUndefined();
|
||||
expect(routes.snapshot().upstreamByViewer.get(childPeerId)).toMatchObject({
|
||||
kind: "peer",
|
||||
parentPeerId: HOST,
|
||||
|
||||
Reference in new issue
Block a user