diff --git a/AGENTS.md b/AGENTS.md index 93c5616..334af76 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -21,7 +21,7 @@ - Build private, low-latency game screen sharing for one broadcaster and up to `20` authenticated Viewers, not a public or large-scale streaming service. - A normal desktop or mobile browser is the initial Viewer target. A packaged sender or native capture helper is a later optimization. - Keep media distributed and automatic. Central services own rooms, authentication, signaling, STUN, observability, and bounded fallback resources; do not silently make the product always-SFU. -- STUN and direct/peer UDP remain first; the only application fallback is the dedicated LiveKit SFU over UDP. Screener configures no TURN, ICE/TCP, media TCP, or TLS-relayed media path. Every accepted path ends in bounded success or clear failure, and HTTPS/WSS transport is independent of media transport. +- Ordinary peer ICE remains STUN-only and direct/peer UDP remains first; the only application fallback is the dedicated LiveKit SFU over UDP. Browser SFU publisher and subscriber PCs retain LiveKit-signaled UDP candidates but do not inherit external STUN/TURN ICE servers; deployment STUN remains available to ordinary peers and LiveKit server-side public-IP discovery. Screener configures no TURN, ICE/TCP, media TCP, or TLS-relayed media path. Every accepted path ends in bounded success or clear failure, and HTTPS/WSS transport is independent of media transport. - Every non-server endpoint uses one server-authoritative steady outbound media-copy cap: default `2`, configurable as `1`, `2`, or `3`. A peer child or the Host's single SFU publication consumes one slot; upstream receive is free, and SFU subscriber egress is accounted at the server. Browser role, UA, visibility, and client advertisement do not create another tier. - The accepted route model is one committed graph, one event-driven reconciliation loop, and at most one room-serial child operation. That operation owns one deterministic candidate list/cursor, one current candidate with its reservations, and one total deadline; its exact child commits on the first newly decoded frame. Join/waiting, child reparent, relay ingress repair with subtree retention, confirmed departure, and effective-capacity reduction all use this operation. A failed exact candidate is already tried for that operation, so other direct parents come first and SFU is the bounded suffix; no persistent parent blacklist is created. - WebRTC and LiveKit own ICE/DTLS/consent, congestion control, transient reconnect, and SFU stream state. Screener does not add all-pairs endpoint probing, parent-wide quality inference, a weighted route score, periodic rebalancing, or an independent depth cap. SFU is one Host publication with per-Viewer subscriptions, and endpoint overlap plus SFU resources remain independently admitted and bounded. Any future strict-firewall transport must be accepted as a LiveKit-internal capability from real evidence, not added as another application route candidate. diff --git a/deploy/livekit/livekit.yaml.example b/deploy/livekit/livekit.yaml.example index 293fc1a..2d01eab 100644 --- a/deploy/livekit/livekit.yaml.example +++ b/deploy/livekit/livekit.yaml.example @@ -12,8 +12,10 @@ rtc: udp_port: 7882 tcp_port: 0 allow_tcp_fallback: false - # Do not inherit LiveKit's public Google STUN defaults. This is the same - # deployment-owned STUN-only listener advertised to ordinary peer edges. + # Use the deployment-owned STUN listener for server public-IP discovery and + # avoid LiveKit substituting a public default in its join response. The + # accepted Screener Browser SFU connect contract uses an empty ICE-server list; + # ordinary peer edges receive this listener through application signaling. stun_servers: - stun.example.com:3478 use_external_ip: true diff --git a/docs/README.md b/docs/README.md index df7a1e3..7f38e77 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,7 +25,7 @@ - [ADR-0001](./adr/0001-p2p-first-media-topology.md): historical P2P-first baseline; later access and automatic-routing decisions supersede its stale current-state details. - [ADR-0002](./adr/0002-memory-resident-protected-rooms.md): four-digit leased in-memory rooms, local Host defaults, scoped Viewer grants, and orthogonal code entry. - [ADR-0004](./adr/0004-peer-assisted-media-experiment.md): historical standard-WebRTC peer-assisted experiment and reusable evidence. -- [ADR-0005](./adr/0005-automatic-hybrid-media-routing.md): accepted and deployed P2P-first single-reconcile route model; remaining real-network validation is tracked in verification status. +- [ADR-0005](./adr/0005-automatic-hybrid-media-routing.md): accepted P2P-first single-reconcile route model; its core is deployed, while the accepted Browser SFU ICE-server-isolation amendment remains pending and real-network validation is tracked in verification status. - [ADR-0006](./adr/0006-fixed-high-native-sender-canary.md): proposed fixed-`HIGH` native canary, current no-go result, and staged revalidation boundary. - [ADR-0007](./adr/0007-demand-driven-dual-representation-quality.md): accepted path-isolated `HIGH + at most one LOW` policy, optional idle-layer stop, evidence contract, and SVC boundary. - [ADR-0008](./adr/0008-window-scoped-audio-capture.md): browser window-audio hint and the Windows WASAPI process-loopback boundary. diff --git a/docs/adr/0005-automatic-hybrid-media-routing.md b/docs/adr/0005-automatic-hybrid-media-routing.md index 3e4a634..be5276c 100644 --- a/docs/adr/0005-automatic-hybrid-media-routing.md +++ b/docs/adr/0005-automatic-hybrid-media-routing.md @@ -1,6 +1,8 @@ # ADR-0005: Automatic Hybrid Media Routing -- Status: Accepted; deployed in exact release `679fe3e` +- Status: Accepted. The core route model is deployed in exact release `679fe3e`; + the Browser SFU ICE-server-isolation amendment is accepted but not + yet implemented or deployed. - Date: 2026-08-20 - Last updated: 2026-08-24 @@ -220,6 +222,13 @@ second mutable graph. transport. An all-UDP-blocked network reaches a clear bounded failure; any future strict-firewall transport requires its own evidence and belongs inside LiveKit rather than becoming another application candidate. +- Browser SFU publisher and subscriber PCs explicitly use an empty ICE-server + list, retain LiveKit-signaled UDP candidates, and let standard ICE nominate a + non-relay pair. Candidate type, address family, count, and Browser socket + allocation are not application invariants. Ordinary peers still use + deployment STUN, and LiveKit server-side public-IP discovery may use that same + STUN service. This isolates SFU PCs from unnecessary external STUN/TURN + destinations without adding a route, transport, port, retry stage, or timeout. - Browser ICE connectivity checks are the authority for direct UDP reachability. Screener does not synthesize remote candidates, predict ports, classify NAT behavior, or use TCP reachability as a media-path probe. Direct exhaustion @@ -358,8 +367,10 @@ Production runs exact deployed application/runtime revision strict `screener-v11` Browser wire and use the direct/STUN peer and LiveKit SFU/UDP route model, including the route, first-frame, typed-status, and Host-only diagnostic contract described above. The operation owner is only -`route`. Exact operational evidence is owned by the deployment document; real -heterogeneous-network and SFU media validation remains open. +`route`. They still inherit LiveKit-provided STUN on Browser SFU PCs and do not +yet implement the accepted Browser SFU ICE-server isolation. Exact operational +evidence is owned by the deployment document; real heterogeneous-network and +SFU media validation remains open. ## Acceptance Boundary diff --git a/docs/deployment.md b/docs/deployment.md index 5f22ddf..3ab41ab 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -7,11 +7,14 @@ This page records production running exact deployed application/runtime revision Browser `screener-v11` contract. Product direction and pending work are owned by [project memory](./project-memory.md) and [the TODO ledger](./todo.md). -This section documents the repository's UDP-only deployment candidate: one -Node.js process provides the built Web client, room API, and WebSocket signaling -behind Caddy or nginx; application ICE advertises only STUN by default; LiveKit -supplies bounded SFU fallback capacity. Normal media remains distributed through -direct or peer edges whenever those paths work. +This section documents the accepted UDP-only deployment contract: one Node.js +process provides the built Web client, room API, and WebSocket signaling behind +Caddy or nginx; ordinary peer ICE advertises only STUN, while Browser LiveKit +PCs configure no external ICE server and retain LiveKit-signaled UDP candidates. +LiveKit supplies bounded SFU fallback capacity. Normal media remains distributed +through direct or peer edges whenever those paths work. Exact production release +`679fe3e` still inherits the LiveKit join response's STUN endpoint and has not +implemented this Browser SFU ICE-server isolation. A deployment may additionally provide one dedicated single-node LiveKit process as the current controller's automatic final media fallback. This capacity is @@ -61,7 +64,7 @@ self-hosted STUN listener owns UDP 3478. ```text browser -- HTTPS/WSS --> Caddy or nginx :443 --> Node.js :8787 browser <------------ DTLS-SRTP P2P ------------> browser -browser -- STUN binding/UDP --> coturn :3478 (application advertises STUN only) +browser -- ordinary-peer STUN/UDP --> coturn :3478 browser <---------- DTLS-SRTP/UDP ----------> LiveKit :7882 ``` @@ -161,7 +164,13 @@ The repository requires STUN and ordinary peer connections receive STUN-only ICE. An existing logical edge prefers direct/peer UDP; the only application suffix is the Host publication/SFU path. LiveKit participants receive only revision-bound `sfu-config` URL/token messages and negotiate within LiveKit's -separate UDP-only ICE domain. +separate UDP-only ICE domain. The accepted Browser publisher and subscriber +connect contract provides an explicit empty ICE-server list, retains the UDP +candidates delivered through LiveKit signaling, and lets standard ICE select a +nominated non-relay pair. Candidate type, count, address family, and Browser +socket allocation remain implementation observations. This does not remove +ordinary-peer STUN or the deployment STUN used by LiveKit to discover its own +public address. `PEER_ASSISTED_MEDIA=true` is the process-wide topology/SFU switch. Every normal room gets its own bounded controller state and ordinary peer connections remain @@ -451,8 +460,11 @@ with mode `0600`; put the same values in Screener's untracked process secrets. The example deliberately omits Redis and every recording, ingress, egress, webhook, external-TURN, and embedded-TURN service. It explicitly sets `tcp_port: 0` and `allow_tcp_fallback: false`, and points `stun_servers` at the -deployment's self-hosted STUN listener so pinned LiveKit cannot inherit its -default public Google STUN servers. Do not add Redis for this one-node workload. +deployment's self-hosted STUN listener for server-side public-IP discovery and +to prevent pinned LiveKit from substituting a public default in its join +response. The accepted Screener Browser publisher/subscriber connect override is +an explicit empty list, so those SFU PCs do not use either response after that +pending application change is deployed. Do not add Redis for this one-node workload. The service journal is the diagnostic log; keep its retention finite and access restricted. Pinned LiveKit 1.13.5 includes raw PublisherOffer SDP in info-level join records, so the tracked production baseline uses `logging.level: warn` and @@ -523,7 +535,10 @@ Run these checks from real external networks before closing route acceptance: 3. On a normal room, exhaust a peer route and verify exactly one Host publication serves only the admitted SFU subscriptions over LiveKit UDP 7882. Peer descendants stay on ordinary direct UDP and every non-server endpoint - obeys the configured `ENDPOINT_MEDIA_COPY_CAPACITY`. + obeys the configured `ENDPOINT_MEDIA_COPY_CAPACITY`. On each Browser SFU PC, + verify no local external STUN/TURN server is configured, a nominated + non-relay UDP pair reaches an authorized LiveKit-signaled SFU candidate, and + the exact Viewer presents a decoded frame. Do not gate on candidate type or count. 4. Block all UDP and verify bounded recovery ends clearly without ICE/TCP, TURN, or a long pseudo-connected path. 5. Exercise root departure, reconnect, SFU unavailable, route prepare rollback, diff --git a/docs/project-memory.md b/docs/project-memory.md index 627840d..e567080 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -15,7 +15,7 @@ Last updated: 2026-08-24 - The stable routing invariants are one authoritative upstream per Viewer, an acyclic source-reachable active graph, exact authenticated route attempts, endpoint and server admission, one room-serial child operation, and bounded success, wait, or failure. - One room controller reconciles allocation, child reparenting, and relay drain through the same operation. It finds one waiting Viewer or invalid edge and builds one deterministic tuple list. The operation owns its cursor, current candidate and reservations, fact version, plus one total deadline; only its current candidate may produce media. A prepare names that exact child, selected transport, and server-issued candidate connection ID. A usable old edge remains until the exact candidate child decodes its first new video frame. `settleReady` synchronously commits admission before graph promotion; success commits prepare revision `P`, while failure releases resources, broadcasts rollback `R > P` from one monotonic allocator, and advances the cursor without resetting the deadline. Exhaustion retires hard-invalid, departed-parent, or overflow edges into explicit waiting but retains a healthy bootstrap edge. Bounded-gap preflights server resources, replans the untried suffix after retirement, and appends one ordinary restore candidate when it cut healthy media. A relay with a bad ingress reparents itself as a child and retains its subtree. A disconnected endpoint, or one whose effective downstream capacity falls below its current child count, accepts no excess children; its direct or overflow children enter the same operation and a disconnected endpoint is removed when childless. Authoritative pause aborts the operation, suppresses decoded-progress stall decisions, and resumes from a fresh reconciliation. - One Host publication may serve all SFU subscriptions. Its exact generation owns one ingress handle and exact Viewer subscription handles own egress; later SFU children reuse the publication, while reserved, committed, and draining handles remain charged until the managed room is deleted and proven absent. Actual SFU usage release wakes only rooms that previously lost a candidate to global admission, and SFU-fed and peer-fed endpoints use the same provisional-child transaction. Parent choice hard-filters authority, reachability, cycles, effective capacity, and producer transition, then orders by shallowest result, remaining steady sender capacity, stable join order, and peer identity; server admission is checked at the current cursor. Acyclicity and room admission bound maximum depth; depth is observed but has no separate hard cap. Healthy committed edges remain sticky: the controller repairs waiting, invalid, departed-parent, and overflow children but does not periodically rebalance the graph. Raw IP, claimed NAT type, UA, geography, all-pairs probes, weighted scores, and periodic room rebalancing are not route inputs. -- Ordinary peer ICE is STUN-only, and current LiveKit media is UDP-only. Credentials, room secrets, candidate details, and diagnostic data remain private and narrowly scoped. A future strict-firewall transport requires separate evidence and must remain inside LiveKit rather than becoming another Screener route candidate. +- Ordinary peer ICE is STUN-only. Browser LiveKit publisher and subscriber PCs use no external ICE server, retain LiveKit-signaled UDP candidates, and let standard ICE nominate a non-relay pair; deployment STUN remains available to ordinary peers and LiveKit server-side public-IP discovery. This Browser SFU ICE-server isolation avoids the reproduced external-STUN/SFU association conflict without constraining candidate type or count and without adding TURN, another route, a port, or a timeout. Credentials, room secrets, candidate details, and diagnostic data remain private and narrowly scoped. A future strict-firewall transport requires separate evidence and must remain inside LiveKit rather than becoming another Screener route candidate. - Browser peers use existing periodic WebRTC/LiveKit stats to maintain one exact-route decoded-progress timestamp. New frames or route/connection changes reset it; a non-paused deadline reports the edge once. Track availability, bitrate, FPS, resolution, freeze, RTT, jitter, loss, sender stats, and SFU layer describe only the current path: they remain framework or diagnostic inputs and cannot prove another parent would be better. - Browser capture requests available share audio by default on initial capture and source switch, with an explicit video-only warning when no audio track is returned. Its screen-audio control offers live-switchable 64/128/256 kbps sender ceilings and defaults to 128. The last-wins quality wire owns desired state; each endpoint serially applies and reads back current Host/relay/SFU audio senders, exposes local failures, and gives future senders the latest desired value. Opus remains fixed, and without an applied-ack wire the Host does not claim room-wide atomic convergence. These are configuration ceilings, not fidelity claims. - Browser display video leaves `contentHint` unset so Chromium retains its screen-capture classification; returned audio tracks keep `contentHint = "music"`. Browser video is fixed to VP8 across direct, browser-relay, and SFU paths; codec selection, fallback to another media codec, quality-state/wire fields, and share-lifetime codec branches are not product features. Actual codec/profile/encoder stats remain diagnostic only. @@ -29,7 +29,7 @@ Last updated: 2026-08-24 ## Current Source And Production - Production runs exact deployed application/runtime revision `679fe3e7af634309322bea83b316641f51ad3d09`, release `679fe3e`, on the single strict `screener-v11` Browser wire; canonical `main` contains the same v11 runtime code, and auxiliary branches or older worktrees do not supersede it. -- Current source and production admit up to `20` Viewers and implement the uniform `1/2/3` endpoint cap, exact prepare candidate, first-decoded-frame commit, strict rollback revision, decoded-progress stall, a two-stage direct/SFU operation deadline, and one event-driven room controller. A short-lived exact-generation decoded-frame observer proves only the pending candidate inside the existing total deadline; the periodic stats sampler remains responsible for active-path diagnostics and stalls. Ordinary peer ICE is STUN-only and the sole application fallback is SFU/UDP; stale TURN environment keys fail startup. SFU admission uses exact publication and Viewer-subscription handles with explicit no-default ingress/egress capacities and a listener-fenced dedicated LiveKit owner. +- Current source and production admit up to `20` Viewers and implement the uniform `1/2/3` endpoint cap, exact prepare candidate, first-decoded-frame commit, strict rollback revision, decoded-progress stall, a two-stage direct/SFU operation deadline, and one event-driven room controller. A short-lived exact-generation decoded-frame observer proves only the pending candidate inside the existing total deadline; the periodic stats sampler remains responsible for active-path diagnostics and stalls. Ordinary peer ICE is STUN-only and the sole application fallback is SFU/UDP; stale TURN environment keys fail startup. SFU admission uses exact publication and Viewer-subscription handles with explicit no-default ingress/egress capacities and a listener-fenced dedicated LiveKit owner. The deployed Browser still inherits LiveKit-provided STUN on its SFU PCs and therefore does not yet implement the accepted Browser SFU ICE-server isolation. - Current source and production use the memory-resident room model: random free four-digit codes, the configurable dormant lease, exact-Host resume, restart loss, local creation-preference replay, orthogonal Viewer grants and `open | password | disabled` code entry, and direct invitation/policy controls. SQLite room persistence, `ROOM_DATABASE_PATH`, `ROOM_TTL_SECONDS`, and a writable room StateDirectory are absent from the current runtime. - Exact production artifacts, services, ports, and any change-scoped infrastructure recovery boundary are owned by [deployment](./deployment.md) and indexed by [status](./status.md). - Current source and production require v11, fix Browser direct, browser-relay, and SFU media to VP8, leave video `contentHint` unset, and contain no codec UI, quality state, wire field, fallback media codec, or share-lifetime codec branch. Both use `shareGeneration`-fenced Pause/Resume and implement live 64/128/256 kbps audio-ceiling mutation, typed first-frame Viewer presentation, privacy-safe on-demand route diagnostics, neutral room-code denial, responsive entry controls, exactly three recommended profiles with `1080p30` as default and advanced-only `854x480`. @@ -39,8 +39,9 @@ Last updated: 2026-08-24 ## Current Priority -1. Diagnose the screen-audio `1 kbps` readout and physically validate live audio ceilings. -2. Then physically validate exact direct, browser-relay, and SFU codec behavior on deployed Browser v11, refine the invitation UI, and finish real-network route acceptance. Browser VP8 hardware acceleration and mobile Viewer lifecycle remain Accepted Later; Native/executable, distribution, and repository-wide simplification remain later phases. +1. Implement and deploy Browser SFU ICE-server isolation, then prove Host publication and Viewer first-frame media under the reproducing TUN path and a mobile network. +2. Diagnose the screen-audio `1 kbps` readout and physically validate live audio ceilings. +3. Then physically validate exact direct, browser-relay, and SFU codec behavior on deployed Browser v11, refine the invitation UI, and finish real-network route acceptance. Browser VP8 hardware acceleration and mobile Viewer lifecycle remain Accepted Later; Native/executable, distribution, and repository-wide simplification remain later phases. The desktop Host background/minimized report is deferred until it reproduces on current production with a real game and synchronized media plus CPU/GPU evidence. The bounded current-Chrome screening found no immediate Host-page lifecycle drop, and removal of the old harmful `contentHint = "motion"` remains a plausible confound rather than a proven background fix. Mobile Viewer playback and assigned-relay survival remain a separate lifecycle gate. diff --git a/docs/research/low-server-media-routes.md b/docs/research/low-server-media-routes.md index 09cd77c..67aec45 100644 --- a/docs/research/low-server-media-routes.md +++ b/docs/research/low-server-media-routes.md @@ -21,6 +21,14 @@ always-SFU topology. When all outbound UDP is blocked, both accepted paths end in a clear bounded failure. Screener configures no TURN, ICE/TCP, media TCP, or TLS-relayed media path. +Browser SFU publisher and subscriber PCs use no external ICE server, retain +LiveKit-signaled UDP candidates, and let standard ICE nominate a non-relay pair. +Candidate type, address family, count, and Browser socket allocation remain +observations rather than product invariants. Ordinary peer PCs remain STUN-only, +and the LiveKit server may still use deployment STUN to discover its own public +address. This is one application SFU route, not a new route class or a TURN +substitute. + WebRTC and LiveKit expose broader framework transport capabilities, but those capabilities do not authorize another Screener route. Any future strict-firewall coverage requires real target-network evidence and a new accepted decision, and @@ -39,6 +47,60 @@ deployment](https://docs.livekit.io/transport/self-hosting/deployment/), and the [pinned LiveKit 1.13.5 configuration sample](https://github.com/livekit/livekit/blob/v1.13.5/config-sample.yaml). +## Browser SFU ICE-Server Isolation + +On 2026-08-24, exact repository dependency set `c7ec061`, Chrome 151 on Windows +11, pinned LiveKit Server 1.13.5, and pinned JS client 2.22.0 reproduced +intermittent initial LiveKit ICE failure while a Mihomo system-stack TUN owned +the public default route. The default arm and empty-ICE-server arm each used a +fresh Browser profile, then ran eight sequential attempts with a fresh page and +exact room per attempt under the same 12-second connection timeout and the same +640x360 at 30 fps canvas, VP8, single-layer publication. The default arm ran +first and the override was injected only through `Room.connect` as +`rtcConfig: { iceServers: [] }`. The temporary diagnostic harness was deleted; +this transcribed summary is bounded root-cause evidence, not a reproducible +regression artifact. No raw candidate, address, local port, token, or persistent +probe report was retained. + +LiveKit supplied the deployment STUN endpoint and its public UDP candidate set +to the client. In this bounded environment Chrome used one local UDP socket for +the external STUN and SFU destinations, while Mihomo's documented +endpoint-independent NAT option was not enabled. Browser socket allocation is +an observation from this reproduction, not an application contract. + +Eight isolated Host connections using that default configuration produced two +successful publications and six initial-ICE failures. A failed sampled attempt +showed 182 Browser connectivity requests with zero received responses; a +server-side pre-conntrack input counter observed 188 UDP packets and its output +counter observed 384 packets sourced by the SFU listener. The counters prove +server input and generated output-path traffic, not delivery beyond the host's +output hook. Together with the TUN association correlation and the controlled +A/B below, the best-supported cause for this reproduction is an +external-STUN/SFU association conflict after server output and before Browser +ICE receipt. The evidence does not generalize that cause to every TUN or VPN. + +Pinned LiveKit Server adds configured STUN to each join response and falls back +to public default STUN when none is configured. Pinned JS client only adopts +those servers when its caller did not provide `rtcConfig.iceServers`. An +explicit empty array therefore keeps the remotely signaled SFU candidate while +preventing additional local STUN gathering; it does not disable ICE. +With `rtcConfig: { iceServers: [] }`, eight of eight isolated Host attempts +formed a nominated non-relay UDP pair; this environment observed +peer-reflexive-to-host candidate types. The first connected sample appeared +within 0.5 to 1.7 seconds and publication completed 75 to 89 ms later. The same +configuration belongs on Browser publisher and subscriber connects; ordinary +peer STUN remains unchanged. Candidate type and count are diagnostic only. +Product acceptance still requires an exact deployed Host-publication and +Viewer-first-frame canary under the reproducing TUN path plus a separate mobile +network. + +Primary sources checked 2026-08-24: [Mihomo TUN configuration and +endpoint-independent NAT](https://wiki.metacubex.one/config/inbound/tun/), +[LiveKit Server 1.13.5 ICE-server response construction](https://github.com/livekit/livekit/blob/v1.13.5/pkg/service/roommanager.go), +[LiveKit JS client 2.22.0 connect-option assignment](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts), +[LiveKit JS client 2.22.0 RTC configuration merging](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/RTCEngine.ts), +and [pinned LiveKit media-transport external-IP discovery](https://github.com/livekit/mediatransportutil/blob/a3417d38cda0/pkg/rtcconfig/ip.go). + ## Historical Candidate Route Ladder At this dated checkpoint, the candidate plan used direct Host P2P first, then a diff --git a/docs/session-handoff.md b/docs/session-handoff.md index 668d0da..838a046 100644 --- a/docs/session-handoff.md +++ b/docs/session-handoff.md @@ -32,10 +32,14 @@ Current high-signal boundary: - docs/status.md owns the exact source/production/hold state. - docs/todo.md is the only executable work ledger and owns ordering. - ADR-0005 owns the accepted automatic route model: direct/STUN peer followed - only by the dedicated LiveKit SFU/UDP fallback. Production exact deployed + only by the dedicated LiveKit SFU/UDP fallback. Browser SFU publisher and + subscriber PCs explicitly use no external ICE server and check the dedicated + public SFU candidate directly; ordinary peers and LiveKit server-side public + IP discovery retain deployment STUN. Production exact deployed application/runtime revision `679fe3e7af634309322bea83b316641f51ad3d09`, release `679fe3e`, uses strict `screener-v11`; canonical `main` contains the - same v11 runtime code. The route uses one short-lived + same v11 runtime code but has not implemented or deployed that accepted + Browser SFU ICE-server isolation. The route uses one short-lived exact-generation decoded-frame observer only for the pending candidate inside the existing total deadline; the periodic stats sampler still owns active-path diagnostics and stalls. Real external-network and SFU media evidence remains diff --git a/docs/status.md b/docs/status.md index a37c3fb..e6125e8 100644 --- a/docs/status.md +++ b/docs/status.md @@ -12,6 +12,7 @@ This is the current execution index. Git history owns completed timelines; [veri - Production admits one Host plus 20 Viewers, applies endpoint capacity `2`, rejects stale Browser and executable-sender wires before room authority, and runs the one-controller exact-candidate route model with one staged total deadline. Rooms use random free four-digit codes, a 24-hour dormant lease, restart loss, local Host creation preferences, independent Viewer grants, and `open | password | disabled` code entry. LiveKit remains dedicated with `room.auto_create: false`, `max_participants: 21`, and global admission `1` publication ingress / `20` subscription egress. - Production media ports remain STUN-only UDP 3478 and LiveKit UDP 7882; coturn TCP/TLS, port 5349, media TCP, and relay ranges are disabled. The public Web ingress remains TCP 80/443; Node 8787 and LiveKit control/signaling 7880 are internal-only. The nft ruleset SHA-256 is `a895f0bb7b66ffe21071ee18acd84d4ea43ca16d4bca9517e6272e14d0ecced4`. - SFU subscribers reconcile Host screen publications across connect, activation, participant arrival, track publication, and reconnect. LiveKit runs at `warn`/Pion `error`, and the Screener unit waits boundedly for the same-host LiveKit control listener before startup. Postflight found zero warning-level Screener journal entries and an empty LiveKit namespace, so physical direct, browser-relay, and SFU codec behavior remains an open current-release gate. +- Production Browser SFU PCs currently inherit the LiveKit join response's external STUN endpoint. Under the reproducing Mihomo TUN path, isolated Host publication succeeded only `2/8`; server input/output hooks counted return-path work while Browser ICE received no response. The accepted Browser SFU ICE-server isolation is not deployed. - Production leaves video `contentHint` unset, keeps audio `contentHint = "music"`, and fixes Browser direct, browser-relay, and SFU video to VP8 without codec UI, quality state, wire fields, or fallback media codecs. ## Current Source @@ -20,6 +21,7 @@ This is the current execution index. Git history owns completed timelines; [veri - Current source implements one Host plus up to `20` Viewers, one steady outbound media-copy cap for every non-server endpoint (default `2`, static `1/2/3`), and the process-memory room model with random four-digit codes, a default 24-hour dormant lease, exact-Host resume, restart loss, local Host preference replay, independent token invitations, `open | password | disabled` code entry, and no SQLite runtime. - One event-driven controller owns the committed graph and at most one room-serial child operation. It uses one deterministic candidate list/cursor and one total direct-then-SFU deadline; exact admission and physical resources remain charged through drain, and the pending candidate commits only after an exact-generation decoded-frame proof. A short-lived 100 ms stats observer proves only that candidate inside the existing deadline, while the 2-second sampler remains responsible for active-path diagnostics and decoded-frame stalls. Healthy edges remain sticky and current-path quality does not authorize reparenting. - Ordinary peer ICE is STUN-only and the only application fallback is the dedicated LiveKit SFU over UDP. SFU generations use explicit no-default ingress/egress capacities, exact `reserved | committed | draining` accounting, delete-plus-absence release, one Host publication, and exact Viewer subscription handles. No Screener TURN, ICE/TCP, media TCP, or TLS-relayed media route exists. +- Current Browser SFU publisher and subscriber connects do not yet supply the accepted empty ICE-server override. Isolated Host A/B evidence passed `8/8` when external client ICE servers were absent and standard ICE used LiveKit-signaled SFU candidates, without TURN, another port, or a longer deadline. - Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec, and SFU publication explicitly uses VP8 with no backup codec. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation and ordinary `shareGeneration`-fenced Pause/Resume remain unchanged. - Current source implements typed first-frame Viewer presentation and recovery, Chinese user-facing route/access failures, privacy-safe failed-page export, authenticated Host-only on-demand route snapshots, neutral `ROOM_ACCESS_DENIED` room-code admission, and responsive entry controls. - Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`; current source defaults to `1080p30`. Advanced resolution adds `480p` as `854x480` without adding a fourth preset, and advanced FPS and bitrate remain independent. @@ -27,10 +29,11 @@ This is the current execution index. Git history owns completed timelines; [veri ## Current Milestone -1. Diagnose the reported screen-audio `1 kbps` readout and physically validate live audio ceilings. -2. Physically verify the deployed `screener-v11` fixed-VP8/no-video-hint direct, browser-relay, and SFU paths from actual codec, capture, outbound, and decoded stats. -3. Refine the Host invitation controls without changing grant semantics implicitly. -4. Finish representative ICE/STUN/SFU acceptance including mobile networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model. +1. Implement and deploy the accepted Browser SFU ICE-server isolation, then prove exact Host publication and Viewer first-frame media under the reproducing TUN path and a mobile network. +2. Diagnose the reported screen-audio `1 kbps` readout and physically validate live audio ceilings. +3. Physically verify the deployed `screener-v11` fixed-VP8/no-video-hint direct, browser-relay, and SFU paths from actual codec, capture, outbound, and decoded stats. +4. Refine the Host invitation controls without changing grant semantics implicitly. +5. Finish representative ICE/STUN/SFU acceptance including mobile networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model. ## Active Boundaries @@ -43,6 +46,9 @@ This is the current execution index. Git history owns completed timelines; [veri ## Current Hold -There is no active P0/P1 source or deployment hold. The fixed-VP8/no-video-hint Browser v11 contract is source-validated, deployed, and postflight-clean. Physical direct, browser-relay, and SFU codec evidence, plus audio, heterogeneous-network, SFU, -background, and mobile evidence remains open. Native/executable work and broad repository cleanup +Production has an active P1 core-route hold: Browser SFU initial ICE is +intermittent under the reproduced Mihomo TUN path; the best-supported cause is +the PC's external-STUN/SFU association conflict. The accepted Browser SFU +ICE-server isolation must be source-validated, deployed, and physically proven before +the route hold closes. Native/executable work and broad repository cleanup remain outside the current evidence boundary. diff --git a/docs/todo.md b/docs/todo.md index 66d5ccc..5019b17 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -6,10 +6,11 @@ Only items in **Now** are executable after their stated decision gate. A branch ## Now -1. **Diagnose and physically validate live screen audio.** First reproduce the Host diagnostic remaining at `1 kbps` after switching the shared source, and distinguish stale/reset statistics from an actual RTP bitrate problem. The app already requests audio and supplies system/window audio hints; record the selected surface, returned audio track, sender/receiver counters, and audible result because the browser owns the picker and may still return video only. Then switch the 64/128/256 kbps ceilings across direct, browser-relay, and SFU paths, create a new edge, and switch source while checking route continuity and A/V synchronization. Do not infer fidelity or room-wide atomic convergence from configuration alone. -2. **Physically validate fixed Browser VP8 with no video hint.** Production exact deployed application/runtime revision `679fe3e7af634309322bea83b316641f51ad3d09`, release `679fe3e`, already runs the strict v11 wire, VP8-only direct/browser-relay offers, VP8 SFU publication with no backup codec, unset video `contentHint`, and no codec UI/state/wire. Confirm actual codec, capture, outbound, and decoded stats on direct, browser-relay, and SFU paths; deployment postflight alone does not close this gate. -3. **Redesign the Host invitation controls.** Replace the current raw-link-heavy block with a compact copy-first layout while preserving the accepted copy-link, copy-room-code, code-entry policy, rotate, and revoke actions. Keep the credential out of logs and request URLs; changing the copied credential length is blocked on the explicit product decision below. -4. **Finish real-network route acceptance.** Measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, exact first-frame commit and rollback, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, and bounded explicit failure across representative IPv4/IPv6 NATs and mobile networks. Include an all-UDP-blocked case to establish the current failure boundary. Do not add port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting. +1. **Restore reliable Browser SFU initial ICE.** Apply `rtcConfig: { iceServers: [] }` to both pinned LiveKit publisher and subscriber connects so each SFU PC retains only LiveKit-signaled remote candidates as standard ICE inputs. Keep ordinary peer STUN and LiveKit server-side public-IP discovery unchanged; do not restore TURN, add a port, constrain candidate type/count, extend the route deadline, or add another recovery stage. Add focused connect-option tests, deploy one immutable application artifact, and close the P1 hold only after exact Host publication plus Viewer first-frame media passes under the reproducing TUN path and a separate mobile network. +2. **Diagnose and physically validate live screen audio.** First reproduce the Host diagnostic remaining at `1 kbps` after switching the shared source, and distinguish stale/reset statistics from an actual RTP bitrate problem. The app already requests audio and supplies system/window audio hints; record the selected surface, returned audio track, sender/receiver counters, and audible result because the browser owns the picker and may still return video only. Then switch the 64/128/256 kbps ceilings across direct, browser-relay, and SFU paths, create a new edge, and switch source while checking route continuity and A/V synchronization. Do not infer fidelity or room-wide atomic convergence from configuration alone. +3. **Physically validate fixed Browser VP8 with no video hint.** Production exact deployed application/runtime revision `679fe3e7af634309322bea83b316641f51ad3d09`, release `679fe3e`, already runs the strict v11 wire, VP8-only direct/browser-relay offers, VP8 SFU publication with no backup codec, unset video `contentHint`, and no codec UI/state/wire. Confirm actual codec, capture, outbound, and decoded stats on direct, browser-relay, and SFU paths; deployment postflight alone does not close this gate. +4. **Redesign the Host invitation controls.** Replace the current raw-link-heavy block with a compact copy-first layout while preserving the accepted copy-link, copy-room-code, code-entry policy, rotate, and revoke actions. Keep the credential out of logs and request URLs; changing the copied credential length is blocked on the explicit product decision below. +5. **Finish real-network route acceptance.** Measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, exact first-frame commit and rollback, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, and bounded explicit failure across representative IPv4/IPv6 NATs and mobile networks. Include an all-UDP-blocked case to establish the current failure boundary. Do not add port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting. ## Accepted Later Roadmap diff --git a/docs/verification-status.md b/docs/verification-status.md index 52f94d9..24d6a4a 100644 --- a/docs/verification-status.md +++ b/docs/verification-status.md @@ -22,6 +22,7 @@ Capacity and routing entries below describe the exact tested source or release o - Exact v10 runtime source `fdd5a4a529ff297f41c05ea3388bf484d76afe8f` passed repository hygiene, TypeScript typecheck, client and server production builds, all 610 Web tests in 45 files, the 257-test SFU admission gate, the access/privacy Browser gate, and the production-dependency audit with zero vulnerabilities. Its one-Host/20-Viewer Chrome 151 direct-loopback report passed all seven assertions: all Viewers decoded, the selected VP8 settings reached every participant, Host and relay media edges stayed within cap `2`, no orphan SFU publication appeared, and all four current-child timing distributions contained 20 values. The report SHA-256 is `802CBB7632E9AEA3C39FE831A7F7880D24140B2E83942CE0DFDA05DE4B408EED`. This validates the v10 source boundary, not H.264 quality, hardware attribution, physical capture, public-network behavior, or SFU packet flow. - Chrome 151 synthetic `1/3/5/8` and 720p30 runs preserved bounded fanout and decoding. A `9461e20`-era three-Viewer rerun kept Host2/Browser Viewer1, decoded every Viewer, and recovered a hard first-level relay departure in 5,380 ms; it did not exercise quality-triggered MBB. Sixteen-Viewer cap2/cap3 loopbacks decoded all Viewers and a ten-second follow-up recorded guarded sender and aggregate Chromium CPU evidence, but resolution/FPS remained low and the ordinary-PC data cannot rank caps, select product policy, or prove sustainable heterogeneous-network quality. See [peer-assisted media](./research/peer-assisted-media.md). - Chrome 151 with local LiveKit 1.13.5/client 2.22.0 proved one SFU/UDP root after `q,f` -> `q,h`: decoded frames advanced 3 -> 23, rendered frames reached 26, Host used one edge, and leaves were clean. This is functional evidence only. See [low-server media routes](./research/low-server-media-routes.md). +- Chrome 151 on the current Windows TUN path reproduced intermittent production LiveKit initial ICE before publication: server counters observed both ingress and egress while Browser ICE received no responses. Eight isolated Host attempts using the LiveKit-provided STUN endpoint succeeded `2/8`; explicitly empty Browser SFU `iceServers` succeeded `8/8`, observed a peer-reflexive-to-host UDP pair within 0.5--1.7 seconds, and published 75--89 ms later. This selects Browser SFU ICE-server isolation; the observed candidate type is diagnostic, not a gate, and the run does not close exact deployed Host/Viewer first-frame, mobile, quality, or endurance evidence. See [low-server media routes](./research/low-server-media-routes.md). - Host A+B and authenticated P2P Viewer C evidence is sanitized, generation-bound, diagnostic-only, and closed for stale or ambiguous reports. Production can expose a Host-local current SFU `h` snapshot and clears it on publisher identity changes; raw media metadata is not retained. See [realtime quality adaptation](./research/realtime-quality-adaptation.md). - Fresh Chrome 151 tab-capture probes identified two separate Browser boundaries. Removing video `contentHint = "motion"` changed VP8 from 14.93 fps at 428x208 to 29.73 fps at 1904x928 under the same 5 Mbps balanced setup. No-hint AMD H.264 remained about 14--16 fps with hundreds of outer rate-limiter drops, while the same MFT reached 29.93 fps under diagnostic Desktop SW BRC; OpenH264 was slower. The page cannot enable that Chromium field trial or select an adapter/MFT. No-hint VP8 used libvpx software and held about 30 fps in one- and two-sender loopback on the test machine, but the process CPU totals include capture and local decode and are not real-game or weaker-device evidence. This evidence selects fixed VP8 for the Browser contract; current source and production implement that decision, while physical production codec behavior remains unverified. See [realtime quality adaptation](./research/realtime-quality-adaptation.md), [Native H.264](./research/native-h264-hardware-decision.md), and [Native shared encode](./research/native-shared-encode-sender.md). - One bounded Chrome 151/Windows 25H2 current-browser screening used real `getDisplayMedia()` window capture, VP8/libvpx with no video hint, a continuously changing WPF source, and one same-machine loopback receiver. Focused, hidden-tab, and minimized 15-second windows measured about 29.4--29.6 capture/encoded/decoded fps and 1.35--1.50 ms encode time per frame; each boundary sample read an enabled, unmuted, live 1186x712 track and each end sample reported `qualityLimitationReason=none`. All rates used monotonic counter deltas over the outbound-RTP RTCStats timestamp interval; source/inbound timestamps and raw stats identities were not retained independently. The sample did not observe an immediate Host-document background drop; it does not cover exact production code, a real game, physical Viewer, CPU/GPU contention, audio, repetition, a hold beyond five minutes, or public routing. See [Browser background capture](./research/browser-background-capture.md). @@ -34,7 +35,7 @@ Capacity and routing entries below describe the exact tested source or release o - Host background: deferred until the report reproduces on exact production with a continuously changing real game or replay, a separate direct wired Viewer, synchronized capture/outbound/inbound plus CPU/GPU evidence, and repeated focused/unfocused/hidden/minimized windows. The current-Chrome Browser `getDisplayMedia()` WPF-window screening did not observe an immediate Host-page media drop, and the old harmful `contentHint = "motion"` release remains a plausible confound rather than a proven background cause. Current evidence authorizes no keepalive, Wake Lock, timer, codec switch, route change, or Native fallback. - Relay resources: current v11 source and production expose the authenticated `1/2/3` endpoint-cap boundary, and retained v10 local loopback evidence covers 20 Viewers at cap `2` only for that source. CPU/GPU/upload and heterogeneous-network measurements remain diagnostics, not authority to change the contract. - Room scale: current source and production admit 20 Viewers plus one Host, and the retained v10 local Browser convergence gate covers that bound on direct loopback. A real public-network 1:20 session, quality, and sustained resource matrix remain open. -- SFU: production uses one Host publication with exact Viewer subscriptions and no fixed root-count or room-wide lease rule. Real LiveKit lifecycle/JWT replay, shaped packet-flow, BWE downshift/recovery, resource use, and SFU-fed first-child evidence remain open. Use Linux `tc` or an authorized public canary for per-leaf shaping, never CDP throttling. +- SFU: production uses one Host publication with exact Viewer subscriptions and no fixed root-count or room-wide lease rule, but deployed Browser SFU PCs still mix the LiveKit-provided STUN endpoint with the public SFU candidate and are held on the reproduced TUN failure. After the accepted empty-ICE-server client rule is deployed, exact Host publication plus Viewer first-frame media must pass on that TUN path and a separate mobile network. Real LiveKit lifecycle/JWT replay, shaped packet-flow, BWE downshift/recovery, resource use, and SFU-fed first-child evidence remain open. Use Linux `tc` or an authorized public canary for per-leaf shaping, never CDP throttling. - Audio: verify the 64/128/256 kbps Share settings, negotiated/observed bitrate, audible music quality, route changes, and game A/V synchronization. Bitrate settings are ceilings, not quality proof. - Observability: the current v11 Viewer stage, local stats/export plumbing, identity-bound relayed-detail retention, export freshness guards, and on-demand Host route snapshot are deployed. Target-browser availability and actual values remain unverified for Host SFU sender evidence, codec/fmtp and encoder implementation/power efficiency, media-source and encoded FPS, A/V playout/jitter/concealment, selected candidate endpoints, and bounded same-pair STUN-response observation. All fields diagnose and have no recovery authority; they do not by themselves prove hardware encode, synchronization quality, or route quality. - Mobile: Android/iOS remain Viewer-only. Autoplay, rotation, iOS lock-screen/page reclamation, background reconnection, and network changes need device evidence. AirPlay/system mirroring is Viewer-local output, not a portable in-app WebRTC output contract. diff --git a/docs/方案设计.md b/docs/方案设计.md index aad9303..2f9b3cc 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -13,7 +13,7 @@ - 分享者主动选择屏幕、窗口或标签页,并请求可用音频; - 创建四位随机数字房间;production site access 密码严格控制建房/发布和 code-only 观看,默认 room-scoped capability link 让受邀 Viewer 直接观看,房间控制状态只驻留当前进程内存; - 每名观看者拥有独立的 `RTCPeerConnection`; -- 当前 production 要求 STUN,普通 peer ICE 只发 STUN servers;SFU 只开放 LiveKit ICE/UDP; +- 当前 production 要求 STUN,普通 peer ICE 只发 STUN servers;已接受但尚未实现/部署的 Browser SFU target 使用空 ICE-server 列表、保留 LiveKit signaling 的 UDP candidates 并由标准 ICE 选择; - 展示连接状态、direct/relay、RTT、实际码率、帧率和丢包等数据; - 超过部署配置上限的观看者被明确拒绝,不静默超载; - 在代码、测试和真实浏览器验证完成前,不宣称达到延迟或画质目标。 @@ -340,6 +340,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio - 质量协议的 host 授权、严格字段与数值范围、last-wins、当前/未来 relay、已配置 SFU 和普通 P2P wire 兼容;sender 的四字段 requested/applied 读回、拒绝与持续受限提示;screen audio SDP helper 覆盖 CRLF、existing/missing fmtp、unknown key preservation、幂等、payload number、rejected/non-audio,以及 empty/no-Opus/ambiguous/parser 异常原对象 fallback,`ViewerPeer` 覆盖 initial/restart/rebuild 时同一结果的 local/signaling 一致性,SFU publisher 精确断言 high-quality stereo/forceStereo/DTX 且不覆盖 RED; - HTTP/WS 集成:nginx per-source site-access `limit_req` 与统一错误不泄露 secret,应用不建立 IP limiter 状态;Viewer 可在 Origin/容量门下 upgrade;Host 首条鉴权同时要求 upgrade-captured site access 与 Host token,Viewer 要么持 exact-room grant,要么先有 site access 再按 `open/password/disabled`。覆盖原子创建 profile、同浏览器重建偏好、fragment/sessionStorage/replaceState、无 site access 的 code-only 统一拒绝、unknown/expired/disabled/wrong-or-absent-password/full code-only 使用完全相同的 `ROOM_ACCESS_DENIED` code/文案/close、grant 与 Host 的独立错误、raw grant/密码不进入 URL/日志/服务端持久状态,以及 rotate/revoke 后 grace/route 清理、旧 Viewer/上游媒体边关闭和 reconnect 拒绝; - 客户端策略与 stats 解析:会话替换不重连,ordinary ICE 与 LiveKit protocol 分离; +- SFU client isolation:`SfuPublisher` 与 `SfuSubscriber` 的 `Room.connect` mock 分别精确断言 `autoSubscribe: false` 与 `rtcConfig: { iceServers: [] }`,ordinary `HostPeer`/`ViewerPeer` 回归继续断言部署 STUN,防止空列表遗漏 Viewer 或扩散到 peer edge; - route planner property tests:候选输入乱序仍产生相同 list,健康边 sticky,硬过滤拒绝 stale/cycle/effective-capacity/unadmitted parent,优先结果深度和剩余容量并稳定打破平局;失败候选清理后 cursor 前进且总 deadline 不重置; - Viewer stage reducer:按上表覆盖 access/Host/queue、peer/SFU prepare、remote track 无首帧、首个 composited frame、autoplay rejection、健康媒体上的 signaling recovery、Host stop/offline 和 bounded failure;stale revision不能覆盖 current frame,raw server message不能进入 UI,零连接失败仍可导出 allowlisted stage/revision/code bucket; - route parser 覆盖 `/r/`、三位/五位、`0000`、ASCII 字母、Unicode 数字、额外 path segment、合法 query/hash 与 exact `1000..9999`,且 malformed 输入不发起鉴权;Host-only diagnostic request 拒绝 Viewer,schema 拒绝额外/raw/private 字段,departure/room delete清除 latest timing,zero-connection Viewer export仍成立; @@ -391,6 +392,8 @@ SFU admission 在应用单进程中只有一个注入实例。LiveKit credential SFU 的 publisher、subscriber、central admission、resource accounting、temporary overlap 和 fallback sequence 由 ADR-0005 统一定义。普通端点 child count 不能成为 server cap,兜底也不能形成无界 whole-room fanout。一个 Host publication 服务所有 SFU subscriptions;SFU-fed 与 peer-fed endpoint 使用相同的 provisional-child first-frame transaction。 +Browser SFU publisher 与 subscriber 在 `Room.connect` 时都传入同一个显式 `rtcConfig: { iceServers: [] }`。LiveKit signaling SDP 继续交付专用 SFU 的授权 UDP candidates,并由标准 ICE 选择 nominated non-relay pair;candidate type、地址族、数量和浏览器 socket 分配都不是应用不变量。该配置只阻止 pinned SDK 合并 join response 或默认公共 STUN,不删除 remote candidate、不改变 ordinary peer 的 STUN 配置,也不改变 LiveKit server 使用部署 STUN 发现自身公网地址。它移除 SFU PC 不需要的外部 STUN/TURN 目的地,避免本次 TUN reproduction 中的 external-STUN/SFU association 冲突。连接失败仍由现有 operation deadline 和 typed failure处理,不增加 TURN、端口、重试层或专用超时。 + relay 自身 ingress 失效时,reconcile 把它作为 child 换父并保留 subtree。endpoint disconnect 或 effective capacity 降低时,只保留稳定顺序内可容纳的 children,并把 overflow children 逐个送回同一 reconcile;offline childless participant 自然删除。Browser child复用现有 WebRTC/LiveKit stats sampling保存 exact active route的 `lastDecodedAt`;采样出现新 decoded frame就更新,route/connection变化就重置,超过统一 deadline只报告一次 exact edge。权威 pause abort pending operation、保留 active graph、让新参与者 waiting,并抑制/重置 decoded stall;resume重新从当前采样建立基线。Track availability、A/B/C、码率、FPS、模糊、sender stats和SFU layer只用于框架恢复或诊断。 实现迁移以当前 ADR-0005 invariants 为准;代码迁移只保留仍满足当前 identity、first-frame commit 和 bounded-recovery 约束的部分。 diff --git a/docs/需求理解.md b/docs/需求理解.md index 62fa1d8..833a312 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -26,7 +26,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - 一对多观看:每个房间允许一名 Host 和最多 `20` 名 authenticated Viewer;房间 admission 与端点下游容量、SFU admission 分别配置,超额加入必须明确拒绝。观看人数不从 endpoint cap 推导,20-Viewer smoke 只证明控制面与资源上界,不是任意网络质量声明。 - 端点发送容量:所有非服务器端点使用同一个服务端权威稳态 outbound media-copy capacity,由唯一部署项 `ENDPOINT_MEDIA_COPY_CAPACITY` 配置,默认 `2` 且只接受 `1`、`2` 或 `3`;不读取旧键或保留第二套 tier。peer child 或 Host 的唯一 SFU publication 各消耗一份,上游接收不消耗该预算。客户端只能声明不高于部署值的可用容量,角色、浏览器、UA、设备类型和页面可见性不产生不同上限。服务端在每次 `authenticated` 权威快照中返回最终 `endpointMediaCopyCapacity`;普通非 peer-assisted 模式也只授权稳定加入顺序中的至多 `C` 个 Host child,超额 Viewer 保持入房等待并在槽位释放后提升,信令不能绕过该集合。Browser Host 与 Viewer 严格校验该值并用它约束稳态 assignment;并行产生媒体的 candidate 必须持有唯一 `TransitionOverlapSlot`,active+pending 物理 copy 只可达到 `min(C + 1, 3)` 并在 commit/abort 后释放。`C=3` 满载时必须先收到 active bounded-gap retirement 再 prepare,绝不产生第四份 copy。SFU subscriber egress 由独立 server admission 记账。当前不兼容房间合同以单一 release 原子升级;缺失/非法 cap、旧 wire 与 executable sender都在取得room/route authority前fail closed。 - P2P 优先:每条普通逻辑媒体边先尝试 direct/peer UDP,服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布;每名 Viewer 同时至多一个 active upstream,失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate;Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction,并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时,同一 reconcile 将 affected Host direct child(若无则稳定顺序中最新的 connected Host direct child)作为本次 child operation,创建唯一 Host publication 并把该 child 变为首个 SFU subscriber,subtree 不动;publication 取代释放的 Host peer slot,下一次扫描再处理原受限 Viewer,不建立 bootstrap 状态机。 -- NAT 与传输边界:STUN 必须可用,所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威;Screener 不合成 remote candidate、不预测端口、不分类 NAT,也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity;成功即停止,失败、超时或绑定失效即撤销,direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP,全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers,再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media;当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输,必须把它作为 LiveKit 内部能力重新决策,不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target;HTTPS/WSS 继续独立使用 TLS/TCP。 +- NAT 与传输边界:STUN 必须可用,所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`,保留 LiveKit signaling 交付的授权 SFU UDP candidates,并由标准 ICE 选择 nominated non-relay pair;不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC,也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威;Screener 不合成 remote candidate、不预测端口、不分类 NAT,也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity;成功即停止,失败、超时或绑定失效即撤销,direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP,全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers,再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media;当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输,必须把它作为 LiveKit 内部能力重新决策,不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target;HTTPS/WSS 继续独立使用 TLS/TCP。 - 质量控制:推荐档位始终只有 1080p60(8 Mbps)、1080p30(5 Mbps)、720p30(3 Mbps)三档,默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`(854x480),其 FPS 与码率仍由高级控件独立选择;不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用 `balanced`(平衡),`maintain-resolution`(清晰优先)、`maintain-framerate`(流畅优先)保留为显式选择。Browser 视频固定使用 VP8;界面、质量状态和 wire 不提供 codec 选择或切换。分享期间可直接换视频档位和 64/128/256 kbps 音频 sender ceiling,不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,具体降级由浏览器决定;必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。不做自定义动态 FPS、综合评分或周期性全房间调档。 - 路径隔离双表示:direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC,不把 viewer 反馈合成全房目标。每条 SFU path 从 `HIGH` ceiling 开始;优先使用 pinned LiveKit 的标准两层 screen-share simulcast,让内建 per-subscriber BWE 独立降层和恢复,Screener 不实现媒体层 selector。应用只诊断实际接收层、硬件编码、游戏性能、资源和上传;SFU 自动降层不改变拓扑。`LOW` 停编只是资源优化,active representation/layer 硬上限为二且不随 viewer 增长。当前 Web P2P simulcast 与 Web/LiveKit SVC 不能证明跨 direct/peer 共享编码或可移植硬件路径;全部标准路径失败后才考虑 custom/native dual encode。 - 路由失效输入:应用不推断 parent-wide 质量,也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`,或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时,使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`,route/connection变化或新帧重置、暂停抑制,同一 edge只报告一次,不增加轮询器。该 operation 把失败的 exact direct parent 在当前 fact version 直接视为已尝试,先选择其他 direct parent,再进入有界 SFU suffix;新的外部事实才允许原 direct tuple 重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时,它作为child换父并保留subtree;disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。 @@ -114,6 +114,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap - Given 两端网络允许 UDP 打洞,When 观看者加入,Then `getStats()` 显示选中的候选对不是 `relay`,且服务器没有承载媒体流量。 - Given ordinary peer edge 被创建或重建,When 检查其 ICE 配置,Then 只包含部署 STUN。 +- Given Browser Host publisher 或 Viewer subscriber 进入 LiveKit SFU candidate,When 检查 pinned `Room.connect` 参数与实际 ICE,Then 两端都使用 `autoSubscribe: false` 和精确 `rtcConfig: { iceServers: [] }`,继续接收 LiveKit signaling 的 remote UDP candidates,并可由任意合法 candidate type 形成 nominated non-relay pair;该空列表不得扩散到 ordinary Host/Viewer peer PC。 - Given route candidate、signal、recovery 或 commit 的 authenticated room/endpoint session、share generation、base/pending revision 或 candidate connection identity 任一不匹配,When 旧消息或异步结果到达,Then 该操作 fail closed 且不能替换当前媒体;SFU publication generation 继续由 resource owner 单独验证。 - Given 新路线正在准备,When exact candidate child 尚未解码出第一帧新画面,Then 可用旧路线保持权威;首帧后只原子提交该 candidate,失败或统一 deadline 到期销毁候选并保留或恢复旧路线。 - Given 所有 direct 与 SFU 媒体路径耗尽,When 有界连接窗口结束,Then 界面显示通用等待或连接失败,不静默黑屏、不长时间假连接,也不尝试未授权路径。SFU 的精确 path order 和 resource accounting 按 ADR-0005 验证。