feat(routing): rescue blocked relay admission (#68)
This commit is contained in:
1 parent
671ad6f22e
commit
48f3b94fed
14 files changed
+372
-39
No files matched your search
@@ -55,9 +55,11 @@ no UI, percentage rollout, or routing score.
|
||||
The signaling server assigns two sticky, balanced chains with a deterministic
|
||||
breadth-first walk. The host has capacity for at most two children and each
|
||||
viewer for at most one. Candidate parents are ordered by depth and server-issued
|
||||
join sequence. Joining a viewer does not move existing assignments. If a parent
|
||||
leaves, only its orphaned subtree root is assigned to the first available slot;
|
||||
the root's descendants stay attached. No RTT, bandwidth, CPU, geography,
|
||||
join sequence. Joining a viewer does not move existing assignments except for
|
||||
ADR-0005's bounded admission rescue: an unassigned one-slot relay may replace
|
||||
the oldest childless zero-capacity Host leaf when both Host slots are full. If a
|
||||
parent leaves, only its orphaned subtree root is assigned to the first available
|
||||
slot; the root's descendants stay attached. No RTT, bandwidth, CPU, geography,
|
||||
capability, or quality scoring is added.
|
||||
|
||||
For this ADR, one media edge is one downstream `RTCPeerConnection` carrying the
|
||||
|
||||
@@ -125,6 +125,18 @@ leaves and desktop-class browsers as one-child relays. The controller uses this
|
||||
binary capability only for future admission and recovery. Withdrawing capacity
|
||||
does not proactively migrate an otherwise healthy existing edge.
|
||||
|
||||
The source now implements one narrow admission exception to sticky assignment.
|
||||
On an active peer-only route with no SFU publication or pending prepare, a
|
||||
connected, childless Viewer that has no upstream or failed-parent history and
|
||||
advertises one relay slot may replace the oldest connected, childless,
|
||||
zero-capacity Host child when both Host slots are full. That leaf becomes the
|
||||
new relay's only child. The synchronous change preserves Host fanout two,
|
||||
browser fanout one and the depth bound, advances one route revision, and clears
|
||||
both changed upstream connection generations. Host-side reconciliation closes
|
||||
the stale child edge before starting the replacement. No healthy routed
|
||||
candidate, quality score, timer, global parent penalty, or periodic rebalance is
|
||||
part of this admission rescue.
|
||||
|
||||
The repository quality-reparent candidate adds one bounded parent-to-server
|
||||
evidence message and no second route owner. Signaling authenticates and
|
||||
forwards Viewer C under its existing session, connection, revision, parent,
|
||||
@@ -152,7 +164,8 @@ The router may create the existing Viewer route intent;
|
||||
the subtree change, revision, generation, prepare, commit and rollback.
|
||||
|
||||
Targeted tests cover the revision controller, protocol authorization, relay
|
||||
capacity, ordered client transitions, stale asynchronous work, server-restart
|
||||
capacity and deterministic admission rescue, ordered break-before-make client
|
||||
transitions, stale asynchronous work, server-restart
|
||||
resynchronization, one-shot credential recovery, peer failback, and optional
|
||||
standby warming. The corrected cold Chrome 151/LiveKit 1.13.5 localhost run took
|
||||
1.481 seconds from failure report to active and 2.257 seconds to a new rendered
|
||||
|
||||
@@ -27,7 +27,7 @@ Last updated: 2026-08-20
|
||||
- Mobile Web Host is unsupported; feature-detect and fail clearly. Mobile Viewer stays leaf-only. After Windows native, gate Android 14+; iOS waits for stable iOS 27 ScreenCaptureKit.
|
||||
- ADR-0005 accepts direct-first Peer ICE and bounded SFU roots. Its default-off exact-room slice is undeployed; production/coturn stay STUN-only. PR #12/selected-edge superseded.
|
||||
- Keep the controller exact-room only: room `1` is the STUN/SFU smoke. Preserve sticky progressing P2P, mobile leaves and break-before-make. Recovery spends one attempt per layer (ICE restart, same-parent rebuild, alternate peer, then SFU), never three identical retries; active SFU gets one fresh grant before Peer failback.
|
||||
- Quality reparenting needs three correlated C+B hard-bad windows and reuses the local-subtree peer/SFU/failure intent. Single-sided reports are diagnostic; one room cooldown blocks churn. No score, timer, or global parent penalty.
|
||||
- C+B quality reparenting is edge-local and cooldown-bound; no score, timer, or global parent penalty.
|
||||
- Flagship media is UDP; HTTPS/WSS stays TLS/TCP; the old release remains rollback-only.
|
||||
- Treat settings as ceilings and degradation as unclassified. Use correlated Host A+B/Viewer C and one-variable evidence; never force AV1, infer by UA, or create a composite score.
|
||||
- Production reports poor film audio and self-echo when system capture includes voice software. Diagnose audio A/B/C and sync; Web cannot isolate arbitrary processes and `maxBitrate` is not quality-up. A Windows 11 native candidate defaults to game-process-tree audio and never widens silently; Windows 10 remains unresolved/unsupported. See `docs/research/browser-screen-audio-quality.md`.
|
||||
@@ -50,7 +50,7 @@ Last updated: 2026-08-20
|
||||
- Chrome 151/LiveKit localhost A/B cut failure-to-active/render from 1.481/2.257 seconds to 0.200/0.320; 31 new frames and 25 ms sampling kept host edges at two. It is headless synthetic 720p30 and includes SDK/network prewarm, not public-network evidence.
|
||||
- Chrome 151 synthetic topology/quality-control runs kept fanout 2/1 and all viewers decoding; one relay close recovered in 5.32 seconds. This is control evidence only.
|
||||
- Room `1` publishes `HIGH+LOW` with Dynacast off and subscriber `HIGH` ceilings, and now has bounded local quality reparenting. Both remain unverified on real media; test zero-child leaves and calibrated C+B loss before broad rollout.
|
||||
- Generation-bound C+B needs three hard-bad windows; parent samples are one-use per connection. Pending SFU binds intent/guard. Stale/successful peer work releases quality exclusion; real failure may take it over. It uploads no raw endpoint metadata and computes no score.
|
||||
- C+B uses three hard-bad pairs, one-use samples and guarded intent/cooldown. Source-only admission rescue moves the oldest childless zero-capacity Host leaf below an unassigned one-slot relay in one revision; focused/typecheck pass, real media unverified.
|
||||
- Undeployed Web name/presence leaves Native wire/media unchanged and separates Viewer roster from Host diagnostics; focused checks pass.
|
||||
|
||||
## Provisional Quality Targets
|
||||
|
||||
@@ -3,9 +3,9 @@
|
||||
- Research date: 2026-08-19
|
||||
- Scope: at most eight trusted viewers, sub-second interactive media, endpoint
|
||||
downstream fanout at most two, and minimal central-server media egress
|
||||
- Status: candidate routes and bounded spike gates; only the Pion RTP
|
||||
transport-fanout sub-oracle is implemented, with no advanced media route
|
||||
integrated into Screener
|
||||
- Status: bounded C+B local reparenting is deployed for one exact room and the
|
||||
deterministic admission-rescue slice is implemented source-only; advanced
|
||||
encoded-media routes remain isolated candidates
|
||||
|
||||
## Terms
|
||||
|
||||
@@ -235,6 +235,16 @@ freeze share, 100-packet floor, three windows, five-second gap and 30-second
|
||||
cooldown are falsifiable candidate constants for production calibration, not
|
||||
standards-derived or claimed optimum values.
|
||||
|
||||
The correlated pair can therefore attribute a problem only to the current
|
||||
parent-to-child edge generation. It cannot prove that the parent is globally
|
||||
bad: the remote-loss value is that child's RTCP report and
|
||||
`qualityLimitationReason` belongs to one outbound video stream. LiveKit's pinned
|
||||
server implementation deliberately reduces published-track and subscribed-
|
||||
downtrack quality to a participant-wide minimum; that aggregate is useful for
|
||||
conferencing UI but is not a directional P2P parent score. Screener consequently
|
||||
excludes only the failed parent for the affected Viewer-rooted subtree and
|
||||
releases the quality-owned exclusion after a successful move.
|
||||
|
||||
With host degree two and the current browser-relay degree one, two balanced
|
||||
chains already minimize maximum depth at `ceil(N/2)`. A tree for `N` viewers
|
||||
still has `N` media edges, approximately `N*B` useful upload in aggregate, and
|
||||
@@ -243,10 +253,10 @@ not reduce either bandwidth quantity. Reordering an already balanced healthy
|
||||
tree also cannot reduce depth; a move needs a discrete admission, path, TURN,
|
||||
relay-resource, recovery, or future native-capacity benefit.
|
||||
|
||||
The admission-rescue slice is limited to its discrete capacity case. If a
|
||||
relay-capable viewer is unassigned because two zero-capacity roots, especially
|
||||
mobile leaves, occupy both host slots, insert that viewer above one
|
||||
deterministic childless root:
|
||||
The source admission-rescue slice is limited to its discrete capacity case. If
|
||||
a relay-capable viewer is unassigned because two zero-capacity roots,
|
||||
especially mobile leaves, occupy both host slots, it inserts that viewer above
|
||||
one deterministic childless root:
|
||||
|
||||
```text
|
||||
host -> new relay -> existing leaf
|
||||
@@ -254,15 +264,25 @@ host -> other root
|
||||
```
|
||||
|
||||
This admits the waiting viewer while keeping host fanout two and browser relay
|
||||
fanout one. It must pass the existing depth, codec, authorization, and recovery
|
||||
gates; otherwise the controller proceeds to the ADR-0005 SFU-root fallback or
|
||||
visible wait.
|
||||
fanout one. The router permits it only on an active peer-only route with no SFU
|
||||
publication or pending prepare, when the connected candidate currently has no
|
||||
upstream or children, newly offers one relay slot, and has no failed-parent
|
||||
history. The host must have exactly two children and the chosen child must be a
|
||||
connected, childless, zero-capacity leaf. Existing server join order chooses the
|
||||
oldest eligible leaf. One synchronous topology snapshot changes the host,
|
||||
candidate, and leaf; one route revision replaces both upstream generations and
|
||||
clears their old connection IDs. The Host reconciler closes every stale child
|
||||
edge before starting replacement children, so its physical fanout does not
|
||||
temporarily exceed two.
|
||||
|
||||
Eligibility is discrete: an active authenticated session, explicit relay
|
||||
Eligibility remains discrete: an active authenticated session, explicit relay
|
||||
capacity, compatible representation, a free downstream slot, acyclicity, and
|
||||
depth/edge budgets; a candidate already in the tree must also have healthy
|
||||
inbound media. For admission rescue, choose the eligible childless root by
|
||||
server join order; do not probe or rank alternate paths. Broader experiments
|
||||
depth/edge budgets. The implemented rescue never moves a candidate that already
|
||||
has an upstream, never handles a failed-path intent, and never selects a root by
|
||||
quality. An authenticated viewer can still lie about its relay capacity, so the
|
||||
exact-room canary and ordinary route-failure recovery remain required; the lie
|
||||
can cause at most this one local move before the candidate is no longer
|
||||
unassigned. Broader experiments
|
||||
may react to a hard media failure, a reviewed threshold-crossing event from
|
||||
correlated path evidence, or a proven native capacity change, but never a
|
||||
continuous optimizer. Do not use a weighted score, UA/device model, IP
|
||||
@@ -379,6 +399,9 @@ Sources checked on 2026-08-20:
|
||||
[room defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/defaults.ts),
|
||||
and [state reconciliation](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts)
|
||||
- pinned implementation behavior, not a universal timeout prescription.
|
||||
- [LiveKit server 1.13.5 participant quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/participant.go)
|
||||
and [connection scorer](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/connectionquality/scorer.go)
|
||||
- Apache-2.0; studied only, with no source copied.
|
||||
- [Jitsi Videobridge endpoint status defaults](https://github.com/jitsi/jitsi-videobridge/blob/master/jvb/src/main/resources/reference.conf)
|
||||
- first-transfer and inactivity reference values.
|
||||
- [SplitStream](https://www.microsoft.com/en-us/research/publication/splitstream-high-bandwidth-multicast-in-a-cooperative-environment/)
|
||||
|
||||
+1
-1
@@ -24,7 +24,7 @@ also contains a no-go Native v2 candidate, but production does not run it.
|
||||
- Fallback prewarm is token-free and limited to room `1`; mobile/iPad viewers are leaves and healthy edges stay sticky.
|
||||
- After an answer, a generation-bound 15s initial-connect deadline enters ICE restart; success/replacement/disposal cancels it. It is deployed but not mobile-verified.
|
||||
- Room `1` can publish exactly `HIGH+LOW` with Dynacast/backup codec off and subscriber `HIGH` ceilings; it has no retained real media frame.
|
||||
- Room `1` quality reparenting needs three C+B hard-bad pairs, reuses peer/SFU/failure and has one room cooldown; single-side reports do nothing. Thresholds need calibration.
|
||||
- Room `1` C+B reparenting remains uncalibrated; source-only admission rescue promotes an unassigned relay over one zero-capacity Host leaf without scores or periodic optimization.
|
||||
- Source-only roster; Native wire/media unchanged.
|
||||
|
||||
## Verified Evidence
|
||||
|
||||
+1
-1
@@ -394,7 +394,7 @@ Native v2 保持 STUN-only。该边界不增加 UI、用户选择或百分比框
|
||||
|
||||
恢复目标按证据和成本逐级升级:短暂 `disconnected` 先观察同代 bytes/stats,`failed` 才执行一次 ICE restart;仍失败时重建一次相同 parent,再尝试不同的合格 peer parent,peer 候选耗尽后才申请 SFU。每个动作最多一次且成功即停,不实现“三次相同重试”。初始 `connecting`/`checking` 的软提示和硬窗口尚未实现;硬窗口必须先用移动网络实测确定,不能直接复用现有 3 秒恢复常量。已实现的 SFU active 失败只刷新一次短期 grant,第二次失败回到新 peer baseline。Host 只保留当前 route revision 的 `connect | source | video-publish | sender-config | audio-publish | transport` 枚举阶段作为本地事件提示,不上传原始错误、URL、token、candidate 或地址。
|
||||
|
||||
实验只实现一个由服务端确定的广度优先拓扑:host 最多两个 child,每名 viewer 最多一个 child,因此形成两条按加入顺序平衡的 sticky 链。父节点只按深度和 `serverJoinSequence` 选择第一个空 slot。新 viewer 加入时不搬动已有节点;parent 离开时只把失去 parent 的 subtree root 重新分配到第一个空 slot,其 descendants 保持连接。该规则不采集或组合 RTT、带宽、CPU、地理位置与质量评分,也不主动重平衡。没有 parent 时明确等待或拒绝,不能创建分享端第三条边。
|
||||
实验只实现一个由服务端确定的广度优先拓扑:host 最多两个 child,每名 viewer 最多一个 child,因此形成两条按加入顺序平衡的 sticky 链。父节点只按深度和 `serverJoinSequence` 选择第一个空 slot。新 viewer 加入通常不搬动已有节点;唯一例外是纯 peer active route 没有 pending/SFU 时,无上游、无 child、无失败 parent 历史且刚声明一槽容量的 viewer,可在 Host 两槽已满时按加入序替换一个 childless、零容量 Host leaf,并把该 leaf 作为自己的唯一 child。Host 客户端先关闭旧 leaf edge 再启动新 relay edge,同一同步拓扑变更只推进一个 route revision。Parent 离开时仍只把失去 parent 的 subtree root 重新分配到第一个空 slot,其 descendants 保持连接。该规则不采集或组合 RTT、带宽、CPU、地理位置与质量评分,也不主动重平衡。没有 parent 时明确等待或拒绝,不能创建分享端第三条边。
|
||||
|
||||
该 one-child 浏览器实验在深度四时只能覆盖八名观看者,不能证明 20 人能力。目标 20 人需要资源门禁通过的 relay `downstreamCapacity=2`(两棵深度四二叉树理论可容纳 30 人,但这只是可达性)或受独立中央 egress cap 约束的 SFU 例外;必须实测逐跳重编码、上行、延迟和故障半径。未通过时保持当前默认八人,不增加深度或 Host fanout 来凑容量。
|
||||
|
||||
|
||||
+1
-1
@@ -123,7 +123,7 @@
|
||||
### 一对多与带宽保护
|
||||
|
||||
- Given 一名分享者和一至两名观看者,When 运行直接 P2P,Then 分享端为每人建立独立媒体边且总 fanout 不超过二。
|
||||
- Given peer-assisted spike 分别运行 1、3、5、8 名观看者,When 用户加入、离开或 relay 失效,Then 服务端使用同一个确定性广度优先规则形成两条 sticky 链,只为新 viewer 或失去 parent 的 subtree root 分配 parent,不主动搬动其他节点;分享端在任何瞬间都不超过两个 child 且每名 viewer 不超过一个,没有可用 parent 时明确等待或拒绝,而不是增加分享端 fanout。
|
||||
- Given peer-assisted spike 分别运行 1、3、5、8 名观看者,When 用户加入、离开或 relay 失效,Then 服务端使用同一个确定性广度优先规则形成两条 sticky 链,只为新 viewer 或失去 parent 的 subtree root 分配 parent;唯一的健康边搬动例外是:无上游的一槽 relay 在纯 peer active route、无 pending/SFU、Host 两槽已满时,按加入序把一个 childless、零容量 Host leaf 放到自己下游。分享端在任何瞬间都不超过两个 child 且每名 viewer 不超过一个;除此之外没有可用 parent 时明确等待或拒绝,不主动重平衡或增加分享端 fanout。
|
||||
- Given 一名分享者和八名观看者,When 在受控网络中运行已记录的质量档位,Then 记录每条边的拓扑深度、候选类型、码率、编码/解码负载、分享端与 relay 上行、首帧和玻璃到玻璃延迟;该条在真实矩阵完成前保持未验收。
|
||||
- Given candidate 将房间默认上限改为 20,When 在 exact-room 中连续运行 20 名观看者 30 分钟并注入 relay departure、重连和一个弱路径,Then 每名观看者持续解码、Host 下游边不超过二、每个 relay 不超过已验收能力、深度与故障半径有界、未受影响 subtree 不迁移,并记录端点/SFU 的 CPU、GPU、内存、上行、出口、帧率、丢包率、冻结和恢复;任一门槛失败时不得切换默认值。
|
||||
- Given peer-assisted host 鉴权或直播中连续修改质量设置,When 服务端校验其分辨率、FPS、码率和偏好,Then 只有 host 的完整有界设置被接受,last-wins 地应用到当前和以后建立的 relay child sender 以及启用的 SFU publisher;越界、缺字段和多余字段被拒绝,普通 P2P 的鉴权 wire 不新增该字段且不接受同步消息。
|
||||
|
||||
@@ -75,8 +75,8 @@ import type {
|
||||
} from "../types";
|
||||
import { HostPeer } from "../webrtc/host-peer";
|
||||
import {
|
||||
limitMediaAssignment,
|
||||
MAX_HOST_MEDIA_CHILDREN,
|
||||
reconcileBoundedMediaChildren,
|
||||
} from "../webrtc/media-assignment";
|
||||
import { sourceSwitchNotice } from "./host-page-notices";
|
||||
|
||||
@@ -757,23 +757,19 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
|
||||
childPeerIds: string[],
|
||||
generation: number,
|
||||
): void {
|
||||
const assignment = limitMediaAssignment(
|
||||
{ parentPeerId: null, childPeerIds },
|
||||
reconcileBoundedMediaChildren(
|
||||
peersRef.current.keys(),
|
||||
childPeerIds,
|
||||
MAX_HOST_MEDIA_CHILDREN,
|
||||
removePeer,
|
||||
(peerId) => {
|
||||
void startPeer(peerId, generation).catch((error: unknown) => {
|
||||
if (isCurrentGeneration(generation)) {
|
||||
setNotice(readableError(error));
|
||||
}
|
||||
});
|
||||
},
|
||||
);
|
||||
const assignedChildren = new Set(assignment.childPeerIds);
|
||||
for (const peerId of peersRef.current.keys()) {
|
||||
if (!assignedChildren.has(peerId)) {
|
||||
removePeer(peerId);
|
||||
}
|
||||
}
|
||||
for (const peerId of assignedChildren) {
|
||||
void startPeer(peerId, generation).catch((error: unknown) => {
|
||||
if (isCurrentGeneration(generation)) {
|
||||
setNotice(readableError(error));
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function handleSignalMessage(
|
||||
|
||||
@@ -17,6 +17,29 @@ export function limitMediaAssignment(
|
||||
};
|
||||
}
|
||||
|
||||
export function reconcileBoundedMediaChildren(
|
||||
currentPeerIds: Iterable<string>,
|
||||
nextChildPeerIds: readonly string[],
|
||||
maxChildren: number,
|
||||
removeChild: (peerId: string) => void,
|
||||
startChild: (peerId: string) => void,
|
||||
): void {
|
||||
const nextPeerIds = new Set(
|
||||
limitMediaAssignment(
|
||||
{ parentPeerId: null, childPeerIds: [...nextChildPeerIds] },
|
||||
maxChildren,
|
||||
).childPeerIds,
|
||||
);
|
||||
for (const peerId of currentPeerIds) {
|
||||
if (!nextPeerIds.has(peerId)) {
|
||||
removeChild(peerId);
|
||||
}
|
||||
}
|
||||
for (const peerId of nextPeerIds) {
|
||||
startChild(peerId);
|
||||
}
|
||||
}
|
||||
|
||||
export function viewerSignalMessage(
|
||||
peerAssisted: boolean,
|
||||
targetPeerId: string,
|
||||
|
||||
@@ -512,11 +512,27 @@ export class HybridMediaRouter {
|
||||
this.relayCapacitySessionsByRoom.set(participant.roomId, advertised);
|
||||
}
|
||||
advertised.set(participant.peerId, participant.sessionId);
|
||||
const controller = this.mediaRouteControllers.get(participant.roomId);
|
||||
const active = controller?.getActiveRoute();
|
||||
const routeIntent = this.viewerRouteIntentsByRoom
|
||||
.get(participant.roomId)
|
||||
?.get(participant.peerId);
|
||||
const allowAdmissionRescue =
|
||||
downstreamEdges === 1 &&
|
||||
active?.sfu.publicationGeneration === null &&
|
||||
active.assignments.get(participant.peerId)?.upstream.kind === "none" &&
|
||||
controller?.getPendingRoute() === undefined &&
|
||||
!this.pendingRoutePreparations.has(participant.roomId) &&
|
||||
(routeIntent === undefined || routeIntent.failedParentPeerId === null) &&
|
||||
!this.failedParentPeerIdsByViewer.get(
|
||||
viewerConnectionKey(participant.roomId, participant.peerId),
|
||||
)?.size;
|
||||
const changes = this.peerRelayTopology.setViewerRelayCapacity(
|
||||
participant.roomId,
|
||||
participant.peerId,
|
||||
downstreamEdges,
|
||||
this.connectedPeerIds(participant.roomId),
|
||||
{ rescueUnassignedRelay: allowAdmissionRescue },
|
||||
);
|
||||
if (changes.length > 0) {
|
||||
this.clearChangedConnectionIds(participant.roomId, changes);
|
||||
|
||||
@@ -9,6 +9,10 @@ export interface MediaAssignmentChange {
|
||||
mediaAssignment: MediaAssignment;
|
||||
}
|
||||
|
||||
export interface RelayCapacityUpdateOptions {
|
||||
rescueUnassignedRelay?: boolean;
|
||||
}
|
||||
|
||||
export const MAX_PEER_RELAY_DEPTH = 4;
|
||||
|
||||
interface RelayViewer {
|
||||
@@ -87,6 +91,7 @@ export class PeerRelayTopology {
|
||||
peerId: string,
|
||||
downstreamEdges: RelayDownstreamEdges,
|
||||
connectedPeerIds: ReadonlySet<string>,
|
||||
options: RelayCapacityUpdateOptions = {},
|
||||
): MediaAssignmentChange[] {
|
||||
const room = this.rooms.get(roomId);
|
||||
const viewer = room?.viewers.get(peerId);
|
||||
@@ -95,12 +100,20 @@ export class PeerRelayTopology {
|
||||
}
|
||||
|
||||
const before = snapshot(room);
|
||||
const previousDownstreamEdges = viewer.downstreamEdges;
|
||||
viewer.downstreamEdges = downstreamEdges;
|
||||
for (const [candidatePeerId, candidate] of orderedViewers(room)) {
|
||||
if (candidate.parentPeerId === null) {
|
||||
this.assignViewer(room, candidatePeerId, connectedPeerIds);
|
||||
}
|
||||
}
|
||||
if (
|
||||
options.rescueUnassignedRelay &&
|
||||
previousDownstreamEdges === 0 &&
|
||||
downstreamEdges === 1
|
||||
) {
|
||||
this.rescueUnassignedRelay(room, peerId, connectedPeerIds);
|
||||
}
|
||||
return changedAssignments(before, snapshot(room));
|
||||
}
|
||||
|
||||
@@ -284,6 +297,51 @@ export class PeerRelayTopology {
|
||||
children.splice(childIndex, 1);
|
||||
}
|
||||
}
|
||||
|
||||
private rescueUnassignedRelay(
|
||||
room: RelayRoom,
|
||||
peerId: string,
|
||||
connectedPeerIds: ReadonlySet<string>,
|
||||
): void {
|
||||
const hostPeerId = room.hostPeerId;
|
||||
const candidate = room.viewers.get(peerId);
|
||||
if (
|
||||
!hostPeerId ||
|
||||
!candidate ||
|
||||
!connectedPeerIds.has(hostPeerId) ||
|
||||
!connectedPeerIds.has(peerId) ||
|
||||
candidate.parentPeerId !== null ||
|
||||
candidate.childPeerIds.length !== 0 ||
|
||||
candidate.downstreamEdges !== 1 ||
|
||||
room.hostChildPeerIds.length !== downstreamCapacity(room, hostPeerId)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
const leafPeerId = room.hostChildPeerIds.find((childPeerId) => {
|
||||
const child = room.viewers.get(childPeerId);
|
||||
return (
|
||||
connectedPeerIds.has(childPeerId) &&
|
||||
child?.parentPeerId === hostPeerId &&
|
||||
child.childPeerIds.length === 0 &&
|
||||
child.downstreamEdges === 0
|
||||
);
|
||||
});
|
||||
if (!leafPeerId) {
|
||||
return;
|
||||
}
|
||||
|
||||
const leaf = room.viewers.get(leafPeerId)!;
|
||||
const leafIndex = room.hostChildPeerIds.indexOf(leafPeerId);
|
||||
room.hostChildPeerIds.splice(leafIndex, 1, peerId);
|
||||
room.hostChildPeerIds.sort(
|
||||
(left, right) =>
|
||||
room.viewers.get(left)!.order - room.viewers.get(right)!.order,
|
||||
);
|
||||
candidate.parentPeerId = hostPeerId;
|
||||
candidate.childPeerIds.push(leafPeerId);
|
||||
leaf.parentPeerId = peerId;
|
||||
}
|
||||
}
|
||||
|
||||
function findShallowestAvailableParent(
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
limitMediaAssignment,
|
||||
MAX_HOST_MEDIA_CHILDREN,
|
||||
MAX_VIEWER_MEDIA_CHILDREN,
|
||||
reconcileBoundedMediaChildren,
|
||||
viewerRestartMessage,
|
||||
viewerSignalMessage,
|
||||
} from "../src/client/webrtc/media-assignment.ts";
|
||||
@@ -40,6 +41,24 @@ describe("peer-assisted client assignment", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("removes stale host edges before starting replacement children", () => {
|
||||
const operations: string[] = [];
|
||||
|
||||
reconcileBoundedMediaChildren(
|
||||
new Set(["old-child", "kept-child"]),
|
||||
["kept-child", "new-child", "overflow-child"],
|
||||
MAX_HOST_MEDIA_CHILDREN,
|
||||
(peerId) => operations.push(`remove:${peerId}`),
|
||||
(peerId) => operations.push(`start:${peerId}`),
|
||||
);
|
||||
|
||||
expect(operations).toEqual([
|
||||
"remove:old-child",
|
||||
"start:kept-child",
|
||||
"start:new-child",
|
||||
]);
|
||||
});
|
||||
|
||||
it("leaves ordinary viewer signaling untargeted", () => {
|
||||
expect(viewerSignalMessage(false, "host_12345678", offer)).toEqual({
|
||||
type: "signal",
|
||||
|
||||
@@ -92,6 +92,87 @@ describe("PeerRelayTopology", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("deterministically promotes an unassigned relay over the oldest zero-capacity host leaf", () => {
|
||||
const topology = new PeerRelayTopology();
|
||||
const peers = connected("host", "viewer-a", "viewer-b", "viewer-c");
|
||||
topology.setHost("room", "host", peers);
|
||||
for (const peerId of ["viewer-a", "viewer-b", "viewer-c"]) {
|
||||
topology.addViewer("room", peerId, peers);
|
||||
}
|
||||
|
||||
const changes = topology.setViewerRelayCapacity(
|
||||
"room",
|
||||
"viewer-c",
|
||||
1,
|
||||
peers,
|
||||
{ rescueUnassignedRelay: true },
|
||||
);
|
||||
|
||||
expect(changes.map(({ peerId }) => peerId)).toEqual([
|
||||
"host",
|
||||
"viewer-a",
|
||||
"viewer-c",
|
||||
]);
|
||||
expect(topology.getAssignment("room", "host")).toEqual({
|
||||
parentPeerId: null,
|
||||
childPeerIds: ["viewer-b", "viewer-c"],
|
||||
});
|
||||
expect(topology.getAssignment("room", "viewer-c")).toEqual({
|
||||
parentPeerId: "host",
|
||||
childPeerIds: ["viewer-a"],
|
||||
});
|
||||
expect(topology.getAssignment("room", "viewer-a")).toEqual({
|
||||
parentPeerId: "viewer-c",
|
||||
childPeerIds: [],
|
||||
});
|
||||
expect(
|
||||
expectValidTree(
|
||||
topology,
|
||||
"room",
|
||||
"host",
|
||||
["viewer-a", "viewer-b", "viewer-c"],
|
||||
),
|
||||
).toBe(2);
|
||||
|
||||
const afterRescue = topology.getAssignments("room");
|
||||
topology.setViewerRelayCapacity("room", "viewer-c", 0, peers);
|
||||
expect(
|
||||
topology.setViewerRelayCapacity(
|
||||
"room",
|
||||
"viewer-c",
|
||||
1,
|
||||
peers,
|
||||
{ rescueUnassignedRelay: true },
|
||||
),
|
||||
).toEqual([]);
|
||||
expect(topology.getAssignments("room")).toEqual(afterRescue);
|
||||
});
|
||||
|
||||
it("does not rescue admission through a host child that already owns a subtree", () => {
|
||||
const topology = new PeerRelayTopology();
|
||||
const peers = connected("host", "a", "b", "c", "d", "candidate");
|
||||
topology.setHost("room", "host", peers);
|
||||
topology.addViewer("room", "a", peers);
|
||||
topology.addViewer("room", "b", peers);
|
||||
topology.setViewerRelayCapacity("room", "a", 1, peers);
|
||||
topology.setViewerRelayCapacity("room", "b", 1, peers);
|
||||
topology.addViewer("room", "c", peers);
|
||||
topology.addViewer("room", "d", peers);
|
||||
topology.addViewer("room", "candidate", peers);
|
||||
const before = topology.getAssignments("room");
|
||||
|
||||
expect(
|
||||
topology.setViewerRelayCapacity(
|
||||
"room",
|
||||
"candidate",
|
||||
1,
|
||||
peers,
|
||||
{ rescueUnassignedRelay: true },
|
||||
),
|
||||
).toEqual([]);
|
||||
expect(topology.getAssignments("room")).toEqual(before);
|
||||
});
|
||||
|
||||
it("does not migrate healthy edges when a relay withdraws capacity", () => {
|
||||
const topology = new PeerRelayTopology();
|
||||
const peers = connected("host", "viewer-a", "viewer-b", "viewer-c", "viewer-d");
|
||||
|
||||
@@ -1435,6 +1435,108 @@ describe("WebSocket signaling", () => {
|
||||
expect((await host.inbox.next("error")).code).toBe("FORBIDDEN");
|
||||
});
|
||||
|
||||
it("rescues an unassigned relay in one bounded route generation", async () => {
|
||||
const harness = await startHarness({ peerAssistedMedia: true });
|
||||
const host = await openClient(harness.webSocketUrl);
|
||||
const hostAuth = peerAssisted(
|
||||
await authenticate(host, harness.room, "host", "rescue-host"),
|
||||
);
|
||||
const first = await openClient(harness.webSocketUrl);
|
||||
const firstAuth = peerAssisted(
|
||||
await authenticate(
|
||||
first,
|
||||
harness.room,
|
||||
"viewer",
|
||||
"rescue-first",
|
||||
null,
|
||||
),
|
||||
);
|
||||
const second = await openClient(harness.webSocketUrl);
|
||||
const secondAuth = peerAssisted(
|
||||
await authenticate(
|
||||
second,
|
||||
harness.room,
|
||||
"viewer",
|
||||
"rescue-second",
|
||||
null,
|
||||
),
|
||||
);
|
||||
const relay = await openClient(harness.webSocketUrl);
|
||||
const relayAuth = peerAssisted(
|
||||
await authenticate(
|
||||
relay,
|
||||
harness.room,
|
||||
"viewer",
|
||||
"rescue-relay",
|
||||
null,
|
||||
),
|
||||
);
|
||||
expect(relayAuth.routeAssignment.upstream).toEqual({ kind: "none" });
|
||||
|
||||
host.socket.send(
|
||||
JSON.stringify({
|
||||
type: "signal",
|
||||
targetPeerId: firstAuth.peerId,
|
||||
payload: {
|
||||
kind: "description",
|
||||
connectionId: "admission-rescue-old-generation",
|
||||
description: { type: "offer", sdp: "v=0\r\n" },
|
||||
},
|
||||
}),
|
||||
);
|
||||
await first.inbox.next("signal");
|
||||
|
||||
relay.socket.send(
|
||||
JSON.stringify({ type: "relay-capacity", downstreamEdges: 1 }),
|
||||
);
|
||||
const [hostRoute, firstRoute, secondRoute, relayRoute] = await Promise.all([
|
||||
nextActiveRouteAfter(host, relayAuth.routeRevision),
|
||||
nextActiveRouteAfter(first, relayAuth.routeRevision),
|
||||
nextActiveRouteAfter(second, relayAuth.routeRevision),
|
||||
nextActiveRouteAfter(relay, relayAuth.routeRevision),
|
||||
]);
|
||||
|
||||
expect(
|
||||
new Set([
|
||||
hostRoute.revision,
|
||||
firstRoute.revision,
|
||||
secondRoute.revision,
|
||||
relayRoute.revision,
|
||||
]),
|
||||
).toEqual(new Set([relayAuth.routeRevision + 1]));
|
||||
expect(hostRoute.assignment).toMatchObject({
|
||||
upstream: { kind: "none" },
|
||||
childPeerIds: [secondAuth.peerId, relayAuth.peerId],
|
||||
sfuPublicationGeneration: null,
|
||||
});
|
||||
expect(firstRoute.assignment).toMatchObject({
|
||||
upstream: { kind: "peer", peerId: relayAuth.peerId },
|
||||
childPeerIds: [],
|
||||
});
|
||||
expect(secondRoute.assignment.upstream).toEqual({
|
||||
kind: "peer",
|
||||
peerId: hostAuth.peerId,
|
||||
});
|
||||
expect(relayRoute.assignment).toMatchObject({
|
||||
upstream: { kind: "peer", peerId: hostAuth.peerId },
|
||||
childPeerIds: [firstAuth.peerId],
|
||||
});
|
||||
expect(hostRoute.assignment.childPeerIds).toHaveLength(2);
|
||||
expect(relayRoute.assignment.childPeerIds).toHaveLength(1);
|
||||
|
||||
first.inbox.ignore("route-update");
|
||||
first.inbox.ignore("media-assignment");
|
||||
first.socket.send(
|
||||
JSON.stringify({
|
||||
type: "route-failed",
|
||||
revision: firstRoute.revision,
|
||||
phase: "active",
|
||||
connectionId: "admission-rescue-old-generation",
|
||||
}),
|
||||
);
|
||||
await first.inbox.expectNone(30);
|
||||
});
|
||||
|
||||
it("keeps the P2P authenticated wire unchanged and forbids profile updates", async () => {
|
||||
const harness = await startHarness();
|
||||
const host = await openClient(harness.webSocketUrl);
|
||||
|
||||
Reference in new issue
Block a user